NodeJS - Looping through Array Sequentially with Timeout between each Element in Array

Question

I have a list of commands in an array that I need to run in order:

const commands = [
  `git clone https://github.com/EliLillyCo/${repo}.git`,
  `cd ${repo}`, `git checkout -b ${branch}`,
  'cp ../codeql-analysis.yml .github/workflows/',
  'git add .github/workflows/codeql-analysis.yml',
  `git push --set-upstream origin ${branch}`,
  'cd ../',
  `rm -r  ${repo}`,
];

They need to be ran in order as the commands rely on the previous command being ran.

Also, each command needs to have a 3 second wait before running the next command, because sometimes commands take time, especially command 1 and command 5 .

I am using a standard for loop which is then using setTimeout() that calls a function to run the commands, as such:

const a = require('debug')('worker:sucess');
const b = require('debug')('worker:error');

const { exec } = require('child_process');

function execCommand(command) {
  exec(command, (error, stdout, stderr) => {
    if (error) {
      b(`exec error: ${error}`);
      return;
    }
    a(`stdout: ${stdout}`);
    b(`stderr: ${stderr}`);
  });
}

const commands = [
   `git clone https://github.com/EliLillyCo/${repo}.git`,
   `cd ${repo}`, `git checkout -b ${branch}`,
   'cp ../codeql-analysis.yml .github/workflows/',
   'git add .github/workflows/codeql-analysis.yml',
   `git push --set-upstream origin ${branch}`,
   'cd ../',
   `rm -r  ${repo}`,
 ];

for (let i = 0; i < commands.length; i++) {
  setTimeout(execCommand(commands[i]), 3000);
}

But there is something wrong with the setTimeout() as it's returning this:

  worker:error TypeError [ERR_INVALID_CALLBACK]: Callback must be a function. Received undefined

What is the best way to approach the problem of looping through an array sequentially, whilst using a timeout?

Answer 1

I'd make execCommand return a promise so you know when it's done; you can't rely on timeouts (what if the task takes more than three seconds?) and since most of those commands will complete much faster than that, the timeouts hold things up unnecessarily.

Here's execCommand returning a promise:

function execCommand(command) {
    return new Promise((resolve, reject) => {
        exec(command, (error, stdout, stderr) => {
            if (error) {
                b(`exec error: ${error}`);
                reject(error);
                return;
            }
            a(`stdout: ${stdout}`);
            b(`stderr: ${stderr}`);
            resolve();
        });
    });
}

Then if you have top-level await available (modern Node.js and ESM modules):

// If you have top-level `await` available
try {
    for (const commmand of commands) {
        await execCommand(command);
    }
} catch (error) {
    // ...report/handle error...
}

If you don't, wrap it in an async IIFE:

(async () => {
    for (const commmand of commands) {
        await execCommand(command);
    }
})().catch(error => {
    // ...report/handle error...
});

---

Alternatively, you could use util.promisify on exec directly if you wanted to separately the execution from the handling of stdout / stderr , but doing them together was the minimal change to what you had, so that's what I stuck with.

Answer 2

Currenty you can't guarantee that the previous command will be completed when calling the next one. You call the next one automatically after 3000ms, but the previous one can take longer than expected and not be over yet.

You should add a mechanism to await each command, then launch the next one. Here's how using async/await :

const util = require('util');
const exec = util.promisify(require('child_process').exec);

const commands = [ ... ];

const execCommand = async (command) => {
    try {
        await exec(command)
    } catch (error) {
        b(`exec error: ${error}`);
        return;
    }
    a(`stdout: ${stdout}`);
    b(`stderr: ${stderr}`);
}

(async () => {
    for (let command of commands) {
        await execCommand(command);
    }
})();