How to make node.js program wait for a PNG to parse?

Question

I am writing a project which requires me to load a lot of low resolution PNGs (~10k) to train my neural network. This is my code

NOTE: This project uses 'pngjs' module

for(var i = 0;i < 100;i++){
var input_dir = dirs[i];
console.log('file number: ', i);
fs.createReadStream(input_dir).pipe(new PNG({
          filterType: 4,}))
        .on("parsed", function(){
          console.log(input_dir);
          for(var y = 0; y < this.height; y++) {
            for (var x = 0; x < this.width; x++) {
              var idx = (this.width * y + x) << 2;
              // invert color
              this.data[idx] = 255 - this.data[idx];
              this.data[idx + 1] = 255 - this.data[idx + 1];
              this.data[idx + 2] = 255 - this.data[idx + 2];
              var sum = this.data[idx]+this.data[idx + 1]+this.data[idx + 2];
              sum /= 3;
              arr.push(sum);
            }
          }
          console.log(arr);
        });
}

After execution, I a ton of file number: in the terminal, after which all the images show up. Any way to make the program wait for the image to parse?

Answer 1

createReadStream will run asynchronously each time that you call it, you could wrap it in a function that returns a promise, then await each one of those promises with Promise.all.

function readStreamPromise(input_dir){
    return new Promise((resolve) => {
        fs.createReadStream(input_dir)
             .pipe(new PNG({filterType: 4,}))
             .on("parsed", () => {
                 // do parsing logic
                  resolve()
             });
        }
    }

then do something like

await Promise.all(dirs.map((inputDir) => {
    return readStreamPromise(inputDir)
})

of course this would need to be inside of an async function if you use the await

Answer 2

I suppose you would like to await its loading phase before proceeding. Luckily there is a keyword for that; namely await . It must be paired with async . Check it out.

Edit: The idea in general is that you would make an array of so-called Promises (related to async and await); one Promise for each image. You can then await the completion of all these asynchronous Promises, ie when all images are loaded. And after that you can do what you want while still having all images load asynchronously in beforehand. (Hint: Promise.all)