How to chain promises in series in a while loop

Question

I have to make a query to multiple databases in series. Basically a set of promises that have to be completed in series. I am implementing it in a while loop. The databases are named based on dates. I also have to set a request delay of 400ms times the amount of loops passed (400*count) between each promise. If it helps I am using cloudant as my database, similar to mongodb/couchdb but an online instance.

So I have implemented a query function that will take a start and end date and will retrieve all the values in the range. If the dates are the same then it will query one database, otherwise it will query multiple databases for each day between the range of dates. I am mainly having trouble with the else block in the if statement as you can see below.

db.predQuery = (start, end, locationCode) => {
        return new Promise((resolve, reject) => {
            let data = [];
            const startDate = moment(start, "MM/DD/YYYY");
            const endDate = moment(end, "MM/DD/YYYY");
            if (startDate.diff(endDate) === 0) {
                // THIS IF BLOCK WORKS GOOD
                let dbName = `prediction_${String(locationCode)}_`;
                dbName += startDate.format("YYYY-MM-DD");
                const db = this.cloudant.use(dbName);
                return db.find({selector: {_id: {'$gt': 0}}}).then((body) => {
                    data = body.docs;
                    return resolve(data);
                });
            } else {
                // THIS BLOCK IS WHERE THE PROBLEM IS

                // This is to start off the promise chain in the while loop
                let chainProm = Promise.resolve(1);
                let iterator = moment(startDate);
                let count = 0;

                // While loop for the series of promises
                while (iterator.isBefore(endDate) || iterator.isSame(endDate)) {
                    //dbName Format: prediction_0_2019-05-28
                    let dbName = `prediction_${String(locationCode)}_`;
                    dbName += iterator.format("YYYY-MM-DD");
                    const db = this.cloudant.use(dbName);
                    count += 1;

                    // Set off chain of promises
                    chainProm = chainProm.then(() => {
                        setTimeout(()=>{
                            db.find({selector: {_id: {'$gt': 0}}}).then((body) => {
                                // Keep adding on to the old array
                                data = data.concat(body.docs);
                            });
                        },count*400);
                    });

                    // Move to the next day
                    iterator.add(1,'days');
                }
                // Once all done resolve with the array of all the documents
                return resolve (data);
            }
        })
    };

Basically the output is suppose to be an array of all the documents between the range of dates. Now the single date works, but when I do a range of dates it either the request doesn't not go through, or I hit the limit or it says the promise is never resolved. I don't think this may not be the best way to approach this, and am open to any solution. Any help is greatly appreciated.

Answer 1

Your chainProm is not being connected with the outer call of predQuery - the resolve is being called immediately.

You'll probably find it a lot easier to use async / await instead, the delay-inside-loop logic will be much easier to understand:

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
db.predQuery = async (start, end, locationCode) => {
  const startDate = moment(start, "MM/DD/YYYY");
  const endDate = moment(end, "MM/DD/YYYY");
  if (startDate.diff(endDate) === 0) {
    // THIS IF BLOCK WORKS GOOD
    let dbName = `prediction_${String(locationCode)}_`;
    dbName += startDate.format("YYYY-MM-DD");
    const db = this.cloudant.use(dbName);
    const body = await db.find({selector: {_id: {'$gt': 0}}});
    return body.docs;
  }
  const iterator = moment(startDate);
  const data = [];
  const testShouldIterate = () => iterator.isBefore(endDate) || iterator.isSame(endDate);
  let shouldIterate = testShouldIterate();
  while (shouldIterate) {
    //dbName Format: prediction_0_2019-05-28
    let dbName = `prediction_${String(locationCode)}_`;
    dbName += iterator.format("YYYY-MM-DD");
    const db = this.cloudant.use(dbName);
    const body = await db.find({selector: {_id: {'$gt': 0}}});
    data.push(body.docs);
    // Move to the next day
    iterator.add(1,'days');
    shouldIterate = testShouldIterate();
    // Don't delay on the final iteration:
    if (!shouldIterate) {
      break;
    }
    await delay(400);
  }
  // Once all done resolve with the array of all the documents
  return data;
};

With this implementation, any errors will be sent to the caller of predQuery (an error will result in the Promise it returns being rejected), so when calling predQuery , make sure to put a catch afterwards.

Answer 2

I haven't fully understood your code. However, if the queries are independent of one another it seems like Promise.all() would solve your problem:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

If you in fact need the promises to run in synchronous order, try simply awaiting for the promise resolve in the loop.