how to filter out / ignore specific lines when comparing text files with diff

Question

To further clarify what I am trying to do, I wrote the script below. I am attempting to audit some files between my QA and PRD environments and would like the final Diff output to Ignore hard coded values such as sql connections. I have about 6 different values to filer. I have tried several ways thus far I am not able to get any of them to work as needed. I am open to doing this another way if anyone has any ideas. I am pretty new to script development so Im open to any ideas or information. Thanks :)

#!/bin/bash
#*********************************************************************
# 
# Name: compareMD5.sh
# Date: 02/12/2018
# Script Location: 
# Author: Maggie o
# 
# Description: This script will pull absolute paths from a text file
#              and compare the files via ssh between QA & PRD on md5sum
#              output match or no match 
#              Then the file the non matching files will be imported to a 
#              tmp directory via scp
#              Files will be compared locally and exclude whitespace, 
#              spaces, comments, and hard coded values  
# NOTE: Script may take a several minutes to run
#       
# Usage: Auditing QA to PRD Pass 3
# nohup ./compareMD52.sh > /output/compareMD52.out 2> /error/compareMD52.err
# checking run ps -ef | grep compareMD52*
#**********************************************************************

rm /output/no_matchMD5.txt
rm /output/filesDiffer.txt
echo "Filename | Path" > /output/matchingMD5.txt
#Remove everything below tmp directory recursivly as it was created by previous script run 
rm -rf /tmp/*

for i in $(cat /input/comp_list.txt) #list of files with absolute paths output by compare script
do 
    export filename=$(basename "$i")  #Grab just the filename
    export path=$(dirname "$i") #Just the Directory
    qa_md5sum=$(md5sum "$i") #Get the md5sum
    qa_md5="${qa_md5sum%% *}" #remove the appended path
    export tmpdir=(/tmp"$path")
# if the stat is not null then run the if, if file is exisiting 
    if ssh oracle@Someconnection stat $path'$filename' \> /dev/null 2\>\&1 
    then 
    prd_md5sum=$(ssh oracle@Somelocation "cd $path; find -name '$filename' -
exec md5sum {} \;")
    prd_md5="${prd_md5sum%% *}" #remove the appended path

    if [[ $qa_md5 == $prd_md5 ]] #Match hash as integer
    then

        echo $filename $path  "  QA Matches PRD">> /output/matchingMD5.txt

    else 
        echo $i
        echo $tmpdir
        echo "Copying "$i" to "$tmpdir >> /output/no_matchMD5.txt
        #Copy the file from PRD to a tmp Dir in QA, keep dir structure to avoid issues of same filename exisiting in diffrent directorys 
        mkdir -p $tmpdir # -p creates only if not exisiting, does not produce errors if exisiting 
        scp oracle@Somelocation:$i $tmpdir # get the file from Prd, Insert into tmp Directory 
    fi

fi 

   done

for x in $(cat /output/no_matchMD5.txt) #do a local comapare using diff
do 
    comp_filename=$(basename "$x")
    #Ignore Comments, no white space, no blank lines, and only report if different but not How different  
    qa=(/tmp"$x")
    #IN TEST
    if diff  -bBq -I '^#' $x $qa >/dev/null 
    # Fails to catch files if the Comment then the start of a line
        then
            echo $comp_filename " differs more then just white space, or 
 comment"
            echo $x >> /output/filesDiffer.txt

    fi
done

Answer 1

您可以将输出通过管道grep -v到grep -v如下所示：

diff -bBq TEST.sh TEST2.sh | grep -v "^#"

Answer 2

我可以使用这种方法弄清楚

if diff -bBqZ -I '^#' <(grep -vE '(thing1|thing2|thing3)' $x) <(grep -vE '(thing1|thing2|thing3)' $prdfile)