How to remote ls on list of files from bash shell script?

Question

I have a below shell script from which I am trying to copy 5 files in parallel. I am running my below shell script on machineA which tries to copy the file from machineB and machineC .

If the file is not there in machineB , then it should be there in machineC for sure.

I am using GNU Parallel here to download five files in parallel. And everything works fine if all the files are there -

#!/bin/bash

export PRIMARY=/data01/primary
export FILERS_LOCATION_1=machineB
export FILERS_LOCATION_2=machineC
export MEMORY_MAPPED_LOCATION=/bexbat/data/be_t1_snapshot
PRIMARY_PARTITION=(550 274 2 546 278 6 558 282 10 554 286 14) # this will have more file numbers

export dir1=/bexbat/data/re_t1_snapshot/20140501

# just iterating the file and doing ls and exit if any of the file is missing
for el in "${PRIMARY_PARTITION[@]}"
do
    ssh david@$FILERS_LOCATION_1 ls $dir3/t1_weekly_1680_"$el"_200003_5.data || ssh david@$FILERS_LOCATION_2 ls $dir3/t1_weekly_1680_"$el"_200003_5.data || echo "File number $el missing on both the filers for primary partition." >&2; exit 1
done

echo "All files present. Starting to copy now."

# copy the files now

Problem Statement:-

Before copying any files, I want to see whether all the files are already present in either of the machines (machineB or machineC) or not. If any of the file is missing, then I need to print out which file is missing and exit out of the shell script with non zero status.

Above script is not working as the way I have described. If it sees any of the file is present, then it exits automatically, it's not moving in the for loop to look for other files. And I am not sure why?

Is there anything wrong I am doing?

Answer 1

That ssh line in the loop doesn't do what you expect. The semi-colon has a lower precedence than the other operators, so when you suffix the line with ; exit 1 ; exit 1 that will be executed always. You could just use an if statement:

if ! ssh david@$FILERS_LOCATION_1 ls $dir3/t1_weekly_1680_"$el"_200003_5.data && \
   ! ssh david@$FILERS_LOCATION_2 ls $dir3/t1_weekly_1680_"$el"_200003_5.data;
then
    echo "File number $el missing on both the filers for primary partition." >&2
    exit 1
fi

Answer 2

ssh doesn't preserve quoting, so you need to escape commands locally to be unescaped by the remote shell.

for el in "${PRIMARY_PARTITION[@]}"
do
    printf -v cmd '%q ' test -e "$dir3/t1_weekly_1680_${el}_200003_5.data"
    ssh "david@$FILERS_LOCATION_1" "$cmd" \
    || ssh "david@$FILERS_LOCATION_2" "$cmd" \
    || { echo "File number $el missing on both the filers for primary partition." >&2;
         exit 1; }
done