JavaScript async in a loop with callbacks

Question

I have a tricky situation that needs to collect keys that belongs to certain types (types in a given array), then filter the collected keys and pass to a deletion function.

The collection process calls shell codes and process the results in a callback within a loop. I will need to wait until the whole loop of callback finishes then pass to the deletion function.

I am using shelljs in the node codes, basically look like the below:

var del_arr = [];

for (var i in types) {
  shell.exec(somecode with
    var [i], {
      silent: true
    },
    function(code, stdout, stderr) {
      somecode-processing/filtering stdout and pushes the results to del_arr;
    });
  //loop through array types[] and need to wait for all shell codes' callbacks to finish;
}

//then pass del_arr to deletion function

I wasn't able to build a async function in this format b/s of the shelljs callback. I also don't know how to use promise in this situation.

Can you tell me how to achieve this non-blocking process? Thanks

Answer 1

Turn child_process.exec into a promise:

function execWrapper(command, options) {
  return new Promise((resolve, reject) => {
     shell.exec(command, options, (error, out, err) => {
       if (error) return reject(error);
       resolve({out: out, err: err});
     })
  })
}

Then you can iterate over types and map each one to a promise:

const promises = types.map(type => execWrapper(type, {slient: true}));

Now wait for each promise to resolve, or for one to reject:

Promise.all(promises).then((del_arr) => {
  // del_arr is now a array of objects with the stdout and stderr of each type.
  // 
})

Answer 2

A good implementation of this case :

async function myAsyncFunction() {
  const promises = types.map((type) => myAsyncRequest(type));
  let del_arr = Promise.all(promises);
}

A good article that explains this :

https://medium.freecodecamp.org/avoiding-the-async-await-hell-c77a0fb71c4c

Answer 3

Try to convert shell.exec to Promise like

function shellPromise(command,option) {
  return Promise((resolv,reject)=>{
      shell.exec(command,option,(code,stdout,stderr)=>
            resolv({code:code,stdout:stdout,stderr:stderr})
      );
   };
};

Then you can use something like

for (var i in types){
    var result=await shellPromise(somecode with var[i], {silent:true});
    // somecode-processing/filtering stdout and pushes the results to del_arr;
}

Answer 4

You can also use async package in npm. It provides a function eachSeries that might come handy in your situation, without useing promises and dealing with callbacks only.

async.eachSeries(hugeArray, function iteratee(item, callback) {
    if (inCache(item)) {
        callback(null, cache[item]); // if many items are cached, you'll overflow
    } else {
        doSomeIO(item, callback);
    }
}, function done() {
    //...
});

For more details on how to use this function: https://caolan.github.io/async/