JS: Async/await doesn't seem to wait for result in a loop

Question

I've got an async map function. Within this I'm creating a set of documents, if they are not existing. The ID of the first created document should be set as parent for the next document.

await Promise.all(data.map(async (o, index) => {
  const key = o.name
  const value = o.value
  const query = { [key]: value }
  const parentID = refID

  console.log(parentID);

  if (parentID) query.parent = parentID

  // Check if document is existing
  let ref = await References.findOne(query)
  refID = ref ? ref._id : undefined

  // If not existing, create it
  if (!refID) {
    const refInserted = await References.insertOne(query)
    refID = refInserted ? refInserted.insertedId : undefined
  }

  console.log(refID)
}))

So I would expect this log output with alternating refID and parentID:

undefined (parentID first run)
yBZWYJcWBoSzx9qwy (refID first run)
yBZWYJcWBoSzx9qwy (parentID second run)
23poGAbg85LCqkcZc (refID second run)
23poGAbg85LCqkcZc (parentID third run)
yBJYzedxftLe2Xm4r (refID third run)
yBJYzedxftLe2Xm4r (parentID fourth run)
PpzjuZ4vfrktNH4ez (refID fourth run)

But I do get

undefined
undefined
undefined
undefined
undefined
undefined
yBZWYJcWBoSzx9qwy
23poGAbg85LCqkcZc
yBJYzedxftLe2Xm4r
PpzjuZ4vfrktNH4ez

which shows me that it is not alternating the log output, but running in two blocks although I've set await for the insertOne (using mongo native driver). So obviously I'm misunderstanding something:

What I think I'm doing is, that map is - in this case - the same like an forEach loop: After the first run the ID of the existing document or the ID of the new created document (if not exiting) is extracted. In the second run, this ID is set as parentID so for the second created document the parent key is set.

Answer 1

The problem you have here, is that you do not want to use the Promise.all if you want the evaluation to be done in sync. This is because Promise.all will accept an array of promises, and then allow those promises to be resolved whenever they do. Instead wrap the operation in an async function and then use a conventional loop:

async function processData(data){
    for (var index = data.length - 1; i >= 0; i--) {
        const o = data[index];
        const key = o.name
        const value = o.value
        const query = { [key]: value }
        const parentID = refID

        console.log(parentID);

        if (parentID) query.parent = parentID

        // Check if document is existing
        let ref = await References.findOne(query)
        refID = ref ? ref._id : undefined

        // If not existing, create it
        if (!refID) {
          const refInserted = await References.insertOne(query)
          refID = refInserted ? refInserted.insertedId : undefined
        }

        console.log(refID)
    }
}

Answer 2

Im guessing from you snippet that things are actually working as intended. The reason you are seeing that log output is because it will run the first 'phase' (a phase being the code up to the next await call) of your map function for each item of the array before moving on.

See this snippet:

 var arr = [1,2,3,4,5] function timer(dur) { return new Promise(res => { setTimeout(res, dur); }); } async function doStuff() { console.log("Beginning block...") await Promise.all(arr.map(async (x) => { console.log("Starting phase 1 for item:", x); await timer(500); console.log("Phase 1 complete, starting phase 2 for item:", x); await timer(500); console.log("Phase 2 complete for item:", x); })); console.log("Block ended...") } doStuff() 

So while you'll note that the phases for each item individually run sequentially, the same is not true for the entire collection. Eg, phase 1 will begin for each of them before it completes for any of them. That's just a consequence of the fact that map is synchronous but the function you're passing it is not.

If you absolutely want to enforce that all phases complete for an item before moving on, you'll have to await each one individually, not using Promise.all :

 var arr = [1,2,3,4,5] function timer(dur) { return new Promise(res => { setTimeout(res, dur); }); } async function doStuff() { console.log("Beginning block...") for (const x of arr) { console.log("Starting phase 1 for item:", x); await timer(500); console.log("Phase 1 complete, starting phase 2 for item:", x); await timer(500); console.log("Phase 2 complete for item:", x); }; console.log("Block ended...") } doStuff() 

But you'll note that the time to complete this is significantly longer, since each item can no longer be run in parallel.