How does array.forEach() handle async functions?

Question

I am trying to iterate over a list of keywords, and call mysql.query() for each one of them.

Problem is, I need each query call to end before the next one begins. How do I make that happen?

I have tried making it work with Promise requests but I do not wholly understand how asynchronous calls work.

keywords.forEach(keyword => {
    sql.query("Select Id from Keyword Where Word= ?", [keyword], 
                function (err, ids) {...}
});

Answer 1

You can do it recursivly

function call(keywords, index){
    if(keyworkds[index]){
       sql.query("Select Id from Keyword Where Word= ?", [keyworkds[index].keyword], 
          function (err, ids) {
               call(keywords, index + 1)
         }
    }
}

call(keywords, 0)

Answer 2

forEach will not wait for the asynchronous function to be done. The reason you need to pass a callback is because that's the only real way you have to do something after completion.

For that reason forEach simply will not work. It's possible to rewrite this to have the second iteration happen asynchronously, but it's ugly as hell.

I would recommend (if you can) to switch to async / await and use a mysql client that supports promises. Example:

for (const keyword of keywords) {
  const result = await sql.query(
     "SELECT Id from Keyword Where Word = ?",
     [keyword]
  );
}

You can use mysql2/promise from the mysql2 package to use promisified mysql function.

Answer 3

You won't be able to do this with just the forEach. Each call of sql.query will fire as quickly as the loop iterates and will not wait until it finishes. Also, there is no guarantee those queries will resolve in the order you call them in.

At the bare minimum, you can use callback functions, but that would be some really ugly and difficult code to deal with. https://en.wikipedia.org/wiki/Pyramid_of_doom_(programming)

That leaves Promises and async/await. I strongly recommend spending some time on this topic as you will be running into it a lot

https://javascript.info/async

Answer 4

Instead of looking for an async / loop solution, you can solve your original problem with one SQL query:

const args = keywords.map(_ => "?").join();
sql.query("Select Id, Word from Keyword Where Word in (" + args + ")", keywords, 
    function (err, records) {...}
);

Answer 5

I think async js library ( https://caolan.github.io/async ) is a good choice for something similar to your problem and with this library you have a cleaner code without nested async calls that produce Pyramid of doom. Whenever you face a problem that has many async calls that will run either in parallel or synchronously you can use it. as the documentation said you can run only a single async operation at a time with series method like eachSeries.

async.eachSeries(keywords, function(keyword, callback) {

 sql.query("Select Id from Keyword Where Word= ?", [keyword], 
    function (err, ids) {
        if (err){
           //if you want to stop reminding queries call callback with err
           callback(err)
        } else {
           callback();
        }
    }
 }, function(err) {
    // if any of the queris produced an error, err would equal that error
    if( err ) {
        // One of the iterations produced an error.
        // All processing will now stop.
    console.log('A query failed to process');
    }else {
        console.log('All queries have been processed successfully');
 }
});

Answer 6

forEach doesn't do anything special with async functions. It will call an async function, but it won't wait for it to complete before moving on. Therefore, if you need to call async promises sequentially, then you need another approach.

It looks like though you aren't using a library that returns a promise, so you will need to wrap the call in a promise ( util.promisefy can help with that, or you can do it manually), or you can use a version of the library that does return promises. It is possible to do this with callbacks, but much harder.

The most straight forward is to use a for … of loop in side of an async function. In order to do this, you need a new enough version of javascript, and async functions are rather new. The basic idea look like this:

async function processCollection(collection) {
  for (let item of collection) {
    await processItem(item)
  }
}

Where processItem would be another async function or a function that returns a promise (which are basically the same thing, just different syntaxes).

Another way, that doesn't require async functions, is to chain together promises. This can be done in a functional way as follows:

collection.reduce((previousPromise, item) => {
  return previousPromose.then(() => processItem(item))
}, Promise.resolve(null))

(again, processItem is a function that returns a promise)

Whats goin on here is that by using reduce, you are basically calling promise.then(…).then(…).then(…)… once for each item in the collection. if the callback to then returns a promise, then it will wait for that promise to finish before calling the next then callback. So you get sequential execution. Also, any promise rejections are propagated as well, so errors will stop the next call from happening.