Collect stdout logs for a shell script that runs inside another shell script

Question

I have a shell script called load_data.sh that runs inside a shell script called shell.sh

The contents of shell.sh

xargs --max-procs 10 -n 1 sh load_data.sh < tables.txt

This shell script runs on 10 tables at the same time in the tables.txt

Now I want to collect the Full logs of the load_data.sh So I did

xargs --max-procs 10 -n 1 sh load_data.sh < tables.txt |& tee-a logs.txt

But I am getting a mix of all the logs. What I want is the logs should be 1st table log then 2nd table log and then 3rd table logs and so on...

Is it possible to achieve that. If so how can I achieve that?

Answer 1

You can solve your problem by creating a separate logfile for each time your script is run. To get the logfiles to be created in sequence you can use the 'nl' utility to number each line of input.

nl -n rz tables.txt | xargs -n 2 --max-procs 10 sh -c './load_data.sh "$1" > log-$0'

Will produced logfiles in sequence

log-001
log-002
log-003 
..

To turn that back into one file you can just use 'cat'

cat log* > result