Shell script to compare result with a text file?

Question

Currently when I execute getent database, it will give me a result in the following form:

    string1,string2,string3,string4,string5

I have a text file that has everything in this format:

    string1,number
    string2,number
    string3,number

Everytime I run my shell script, I want to get the content of my database and compare with the text file. If there are new strings in the database that do not exit in my text file, I want to write the new string at the end of my text with a random number so the text file will become:

    string1,number
    string2,number
    string3,number
    string4,number
    string5,number

Pretty new to shell..what's the best way and syntax I should look into?

Answer 1

The Basics

Taking this output from getent as an example:

$ getent passwd daemon
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin

We can process it as you request via:

$ getent passwd daemon | while IFS=: read s1 s2 s3 rest; do printf "%s,%s\n" $s1 $RANDOM $s2 $RANDOM $s3 $RANDOM; done
daemon,7721
x,9604
1,2061

How it works

• getent passwd daemon

  Replace this with whatever getent command that generates the data that want.

• |

  This tells the shell to feed the output of getent to the input of the next command.

• while IFS=: read s1 s2 s3 rest; do

  This starts a while loop by reading four variables from the next line of standard input. Since the output of getent is colon-separated, we set IFS to be : .

  In the question, you show the output of getent being comma separated. If that were the case, then replace IFS=: with IFS=, .

• printf "%s,%s\\n" $s1 $RANDOM $s2 $RANDOM $s3 $RANDOM

  This prints the first three string variables output with random numbers added.

• done

  This signals the end of the while loop.

Multi-line from

For those who prefer their shell commands spread over multiple lines:

getent passwd daemon | while IFS=: read s1 s2 s3 rest
do
    printf "%s,%s\n" $s1 $RANDOM $s2 $RANDOM $s3 $RANDOM
done

Comparing and updating a reference file

This will check for the existence of any of the first three strings in testfile and append to testfile those which were not already there:

getent passwd daemon | while IFS=: read s1 s2 s3 rest; do for s in "$s1" "$s2" "$s3"; do grep -qE "^$s," testfile || printf '%s,%s\n' "$s" "$RANDOM">>testfile; done; done

Or, in multi-line form:

getent passwd daemon | while IFS=: read s1 s2 s3 rest
do
    for s in "$s1" "$s2" "$s3"
    do
        grep -qE "^$s," testfile || printf '%s,%s\n' "$s" "$RANDOM">>testfile
    done
done

Answer 2

Get to know bash associative arrays set by declare -A , and in particular the difference between the following two statements:

echo 'A: ${strings[@]}='"${strings[@]}";
echo 'B: ${!strings[@]}='"${!strings[@]}";

A quick search for bash associative arrays turned up this tutorial .

As you are new to bash also look at:

1. IFS environment variable and how it impacts the read function.
2. lines like: for num in ${numbers//,/ }; do echo $num; done; for num in ${numbers//,/ }; do echo $num; done;

Answer 3

I am not completely sure I understood your problem statement. But from what I could understand, this might work:

$ cat file1
string1,number1
string2,number2
string3,number3

$ cat file2
string1,string2,string3,string4,string5


$ awk -F, 'NR==FNR{a[$1]++; print; next}{for (i=1;i<=NF;i++)if (!($i in a)) print $i"," (int(10000*rand()));}' file1 file2

string1,number1
string2,number2
string3,number3
string4,2377
string5,2910

Notes:
Number 10000 used here is just an arbitrary number. Change as required.
Also, due to an inherent issue, these random numbers can get repeated if the number of records is large.

Code explanation:
Record existing "string"s in an array named a for the first file. During file2 processing, if the entry is not found in array a, print a new entry with random number.