What's the best way to resolve async tasks?

Question

I have an express server that accepts many uploaded files and converts/translates some of them.

What's the best (simple/efficient) way to resolve many asynchronous operations? I can make an array and use it like a checklist for which files are done, but this feels hack-y and I'm worried if any file errors the process will never complete, and the user will never be notified their files are finished/ready.

I'm sure I'm not the first person to encounter this, so any blog articles/online books on this problem are very welcome.

if (req.files)
    req.files.forEach(saveFile);
else
    console.log("\tDoesn't has multifiles", req.file);

function saveFile(file){
    //file is renamed/moved (async)
    if(isCertainFileType(file))
          //convert and save converted file. Do not send a 200 or update the index until this is complete       

}

updateDirIndex(); //index must be updated before sending 200 (reads all files in dir and writes to a file)

return res.status(200).send(index); //tell the browser their files are ready

Answer 1

As long as all your async tasks return a Promise you can wait for them to all resolve using Promise.all() .

let filesToProcess = [...]

// first process any files that need processing, returning an array of promises
let processPromises = filesToProcess.map(file => {
  // if it needs processing, do it and return the promise
  if (isCertainFileType(file)) {
    return doProcessing(file)
  }
  // otherwise just return the file, any truthy value is considered a resolved promise
  else {
    return file
  }
})

// then save all the files
let savePromises = processPromises.map(file => {
  return saveFile(file)
})

// wait for all saves to complete (resolve), handle results or handle errors
Promise.all(savePromises)
  .then((result) => {
    // update the directory index here because we know everything was successful
    updateDirIndex()
  })
  .catch((err) => console.error(err))

NOTE: The preceding made the assumption you wanted all processing to happen first, then save everything once the processing was complete. It's also possible to paralyze the processing AND saving into separate promise chains and collect them all with a Promise.all() at the end. This example is just easier to understand.