NodeJS child_process: run two process in parallel over ssh and stream outputs

Question

First off, this could have been easily accomplished with ssh2 , but I could not use it because it does not support certificate authentication .

Hence the problem at hand:

I am using child_process to run the ssh binary and connect to a remote server. I can run commands and get their outputs via stdin and stdout of the spawned process. But I have a use case where I need to run two commands in parallel . One is a long running command which I need to stream line by line and the other is a lightweight command whose input depends on the contents of the streamed lines. Is it possible to do this without spawning two ssh processes?

Workarounds using ssh2 or something similar are also welcome.

(Edit)Adding a descriptive example:

Process 1 is long running and keeps writing lines to stdout/file, Process 2 needs to run on each line as whenever a line is produced by Process 1 . At each point, when Process 2 for a line completes, I need both input line printed by Process 1 and the corresponding output of Process 2

Answer 1

there is a very simple way: use gnu parallel. https://www.gnu.org/software/parallel/ It is installed on Ubuntu fe with sudo apt -y install parallel

ssh host \
  'echo -e "date > 1.out && sleep 5\n date > 2.out && sleep 5" | parallel -j0'
ssh host cat 1.out
    > Th 14. Jun 10:54:15 CEST 2018
ssh host cat 2.out
    > Th 14. Jun 10:54:15 CEST 2018

Now you could ssh to cat 1.out and 2.out.

But let's consider your later edit:

prog1.sh:

for i in {1..5}; do
  echo $i >> 1.out
  sleep 1;
done
echo "done" >> 1.out
sleep 1
echo -e "\n" >> 1.out

prog2.sh (monitors 1.out and does some command, here tail):

inotifywait -e close_write,moved_to,create -m . |
  while read -r directory events filename; do
    if [ "$filename" = "1.out" ]; then
      # 1.out changed we need to do something
      res=$(tail -1 1.out)
        if [ $res = "done" ]; then
            break;
        fi
      fi
  done

ssh host 'echo -e "./prog2.sh \n ./prog1.sh" | parallel -j 2'