Compare a list of strings in Bash

Question

I have a list of rpm files in a file called rpmlist.txt which I have to compare with another list, newlist.txt, and see if they are same in Bash. For example, this is my requirement:

files inside rpmlist.txt

bash-4.4-9.10.1_x86_64
binutils-2.32-7.8.1_x86_64
bison-3.0.4-1.268_x86_64

files inside newlist.txt

bash-5.4-9.10.1_x86_64
binutils-2.32-7.8.1_x86_64
bison-6.0.4-1.268_x86_64

And print if they are matching or not. Any help would be appreciated

Answer 1

Try with:

#!/bin/bash

# Load files into arrays

readarray source_list < rpmlist.txt
readarray target_list < newlist.txt

# Check files size

source_size=${#source_list[@]}
target_size=${#target_list[@]}

if [ ${source_size} -ne ${target_size} ]; then
    echo "File lines count not matching!" >&2
    exit 1
fi

# Enum files

for (( i=0; i < ${source_size}; i++ )); do

  # Get file name

  source_file=${source_list[$i]}
  target_file=${target_list[$i]}

  # Remove CR/LF

  source_file=$(echo "${source_file}" | sed 's:\r$::')
  target_file=$(echo "${target_file}" | sed 's:\r$::')

  # Check if files exist

  if [ ! -f ${source_file} ] || [ ! -f ${target_file} ]; then
    echo "Source and/or Target does not exist." >&2
    exit 2
  fi

  # Compare files

  diff -q "${source_file}" "${target_file}"

done 

PS: I tested it and it works.

Edit (1)

Based on comments, I think you should replace my script with the following simple command:

cat rpmlist.txt | xargs -I "{}" grep "{}" newlist.txt

Edit (2) - Unmatched list

cat rpmlist.txt | xargs -I "{}" grep -v "{}" newlist.txt

Answer 2

This will cross-compare an arbitrary number of such files.

#!/bin/bash
set -e

declare -ar list_names=("$@")
declare -Ai "${list_names[@]}"

for list in "${list_names[@]}"; do
  declare -n set="$list"
  while IFS= read -r line; do
    ((++set["$line"]))
  done < "${list}.txt"
done

compare_lists() {
  local -rn set1="$1"
  local -rn set2="$2"
  local name
  echo "Lines in ${1}, but not in ${2}:"
  for name in "${!set1[@]}"; do
    ((set2["${name}"])) || printf '  %s\n' "$name"
  done
}

declare -i idx jdx
for ((idx = 0; idx < ${#list_names[@]}; ++idx)); do
  for ((jdx = idx + 1; jdx < ${#list_names[@]}; ++jdx)); do
    compare_lists "${list_names[idx]}" "${list_names[jdx]}"
    compare_lists "${list_names[jdx]}" "${list_names[idx]}"
  done
done

Example (when the above script is called listdiff.sh ):

$ ./listdiff.sh rpmlist newlist
Lines in rpmlist, but not in newlist:
  bison-3.0.4-1.268_x86_64
  bash-4.4-9.10.1_x86_64
Lines in newlist, but not in rpmlist:
  bison-6.0.4-1.268_x86_64
  bash-5.4-9.10.1_x86_64

And it can get more arguments than 2.