JavaScript/TypeScript - loop of promises doesn't wait

Question

I have a loop of 100 promises. I want to process 10 of those simultaneously, then print a statement, and continue with the next 10 in parallel. However, it does not wait for every 10 promises to finish their execution.

const promises = [];
for (let i = 1; i <= 100; i ++) {

   const blockPromise = this.web3.eth.getBlock(i).then((block: any) => {
        winston.info("Processing block " + i);
   }).catch((err: Error) => {
        winston.error(err);
   });

   promises.push(blockPromise);

   if (i % 10 === 0) {
       Promise.all(promises).then(() => {
            winston.info("+++ Processed " + 10 + " blocks");
       }).catch((err: Error) => {
            winston.error(err);
       });
   }

My expectation is something like:

Processing block 1
Processing block 3
Processing block 7
Processing block 2
Processing block 4
Processing block 5
Processing block 6
Processing block 9
Processing block 10
Processing block 8
+++ Processed 10 blocks
Processing block 12
...

However, it is chaotically mixed up. What am I doing wrong? Any help appreciated.

Answer 1

Another solution is to use array reduce to chain your promises:

 var groups = Array.from(new Array(10),(val,index) => 10*index); // => [0, 10, 20, 30, 40, 50, 60, 70, 80, 90] groups.reduce((m, o) => { return m.then(() => { var group = Array.from(new Array(10), (val, index) => o + index + 1); // [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], [11, 12, 13, 14, 15, 16, 17, 18, 19, 20], ... var promises = group.map(i => this.web3.eth.getBlock(i).then(() => winston.info("Processing block " + i)) ); return Promise.all(promises).then(() => winston.info("+++ Processed " + 10 + " blocks") ); }); }, Promise.resolve(true)); 

It just gives you the expected result:

Answer 2

Since you are trying to process 10 promises each time, you could await after each batch of 10 or would that defeat the purpose?

var promises = []; // You are mutating it so better not be a const
var blockPromise = null;
for (let i = 1; i <= 100; i ++) {

 blockPromise = this.web3.eth.getBlock(i).then((block: any) => {
      winston.info("Processing block " + i);
 }).catch((err: Error) => {
      winston.error(err);
 });

 promises.push(blockPromise);

 if (i % 10 === 0) {
     await Promise.all(promises);
     promises = [] // clear the processed promises as well
     winston.info("+++ Processed " + 10 + " blocks");
 }
}

Answer 3

Promises inside for loops mess up everything. Either you use async and await, or you need to reduce the whole thing to one promise chain:

let promise = Promise.resolve(), promises = [];
for( let i = 0; i <= 100; i++){
  promises.push(this.web3.eth.getBlock(i).then((block: any) => {
         winston.info("Processing block " + i);
     }).catch((err: Error) => {
         winston.error(err);
  }));

  if (i % 10 === 0){
   promise = Promise.all(promises.concat(promise))
      .then(_=>winston.info("+++ Processed " + 10 + " blocks");  
   promises = [];           
  }
}