Promise 链接竞态条件

Question

I'm currently working on a rather simple logic for processing queued ZPL print jobs, which are stored in an array that's then being iterated sending n amount of copies per job to a printer.

I'm reducing the array into a promise chain mixing-in a sub-chain for each job which sends the copies to the printer. The calls to the printer are synchronous (ye I know...) so I wrapped each one of them into a Promise that only resolves when the printer received the copy, thus ensuring sequential processing.

In case of a failed transmission the current promise rejects with a hand-crafted error which is being caught in the main-chain.

So far the theory, alas there seems to be a kind of race-condition between the sub-chains.

I tried my best, but I simply don't see it...

Here some simplified code + fiddle, notice how the sub-chains are not running subsequential:

['job1', 'job2'].reduce((pMain, item, curIndex) => {

    var pSub = Promise.resolve();

    for (let i = 0; i < 2; i++) pSub = pSub.then(() => new Promise((resolve, reject) => setTimeout(reject, 2000)));

    return pMain.then(() => pSub);

}, Promise.resolve())
.then(() => /* print all done */)
.catch( handleError );

jsfiddle with console.logs here

Any advice is highly appreciated. Being stuck at something so trivial is a bummer.

Answer 1

Your pSub chains are all created and run synchronously during the reduce call. To become sequential, they need to go inside the then callback:

['job1', 'job2'].reduce((pMain, item, curIndex) => {
    return pMain.then(() => {
        var pSub = Promise.resolve();
        for (let i = 0; i < 2; i++)
            pSub = pSub.then(() => new Promise((resolve, reject) => setTimeout(reject, 2000)));
        return pSub;
    });
}, Promise.resolve())

Alternatively build only a single chain across the two loops:

['job1', 'job2'].reduce((promise, item, outerIndex) => {
    return Array.from({length: 2}).reduce((promise, _, innerIndex) => {
        return promise.then(() => new Promise((resolve, reject) => setTimeout(reject, 2000)));
    }, promise);
}, Promise.resolve())

---

Of course @jfriend is right, for sequential tasks you should just write async / await code:

for (const item of ['job1', 'job2']) {
    for (let i = 0; i < 2; i++) {
        await new Promise((resolve, reject) => setTimeout(reject, 2000));
    }
}

You can also easily put a try block on the right level with that solution.

Answer 2

So, by now, you already understand what you were doing wrong in your use of .reduce() to serialize promises. In comments, I made several suggestions that you either:

1. Use modern async/await (with transpiling if necessary)
2. Use pre-built libraries that are already offering asynchronous iteration
3. Write/use some tested utility functions that you can use instead of hand coding a .reduce() loop each time.

If #1 or #2 are not practical I suggested making your own tested utility functions because the .reduce() method of serialization is easy to get wrong and not always trivial for someone who hasn't already seen the code to know what it's doing whereas an appropriately named utility function that was written and tested once is simpler to use and understand (once the function is written) and obviously it makes reuse practical too.

For pre-built libraries, both Bluebird and Async have these capabilities (personally, I prefer Bluebird) and have used Bluebird myself on embedded projects (Raspberry Pi) running older versions of JS.

---

As for the tested utility functions, here are several you can quickly use.

iterateAsync() is like an asynchronous .forEach()

mapAsync() is like an asynchronous .map()

reduceAsync() is like an asynchronous .reduce()

All take an array as the first argument and a function that returns a promise as the second argument. These are ES5 compatible, but do assume Promise is available. Here are the three functions:

// iterate an array sequentially, calling a function (that returns a promise)
// on each element of the array
// The final resolved value is whatever the last call to fn(item) resolves to
// like an asynchronous .forEach()
function iterateAsync(array, fn) {
    return array.reduce(function(p, item) {
        return p.then(function() {
            return fn(item);
        });
    }, Promise.resolve());
}

// iterate an array sequentially, calling a function (that returns a promise)
// on each element of the array
// The final resolved value is an array of what all the fn(item) calls resolved to
// like an asynchronous .map()
function mapAsync(array, fn) {
    var results = [];
    return array.reduce(function(p, item) {
        return p.then(function() {
            return fn(item).then(function(val) {
                results.push(val);
                return val;
            });
        });
    }, Promise.resolve()).then(function() {
        return results;
    });
}

// iterate an array sequentially, calling a function fn(item, val)
// (that returns a promise) on each element of the array.  Like array.reduce(),
// the next fn(accumulator, item) is passed the previous resolved value and the promise
// that fn() returns should resolve to the value you want passed to the next
// link in the chain
// The final resolved value is whatever the last call to fn(item, val) resolves to
// like an asynchronous .reduce()
function reduceAsync(array, fn, initialVal) {
    return array.reduce(function(p, item) {
        return p.then(function(accumulator) {
            return fn(accumulator, item);
        });
    }, Promise.resolve(initialVal));
}

Please note that all of these are generally simpler with modern Javascript capabilities (particularly async/await ) so these are mostly for when those modern capabilities are not available or transpiling is not practical.

---

For completeness, I'll add that using .reduce() in this way is probably not idea for iterating very large arrays. That's because what this does is synchronously pre-build a promise chain p.then().then().then().then() with the number of .then() equal to the length of your array. If your array was very large (tens or hundreds of thousands of elements long), this could take a significant amount of memory to pre-build all those promises and chain them all together.

For very large arrays in a "limited environment" like you refer to, you may want to iterate more manually like this which doesn't pre-build any large structure and just uses promises one at a time:

function iterateAsync(list, fn) {
    return new Promise(function(resolve, reject) {
        var index = 0;

        function next(val) {
            if (index < list.length) {
                try {
                    fn(list[index++]).then(next, reject);
                } catch(e) {
                    reject(e);
                }
            } else {
                resolve(val);
            }
        }
        next();
    });
}

Answer 3

I guess there is a lot of ways to achieve this, but personally what I always do is to create an array of functions that return a Promise (a PromiseFactory, you might say).

const promiseFactoryA = () => {
  return new Promise(resolve => {
    console.log('PromiseA started...');
    setTimeout(() => {
      console.log('PromiseA resolved after 300ms');
      resolve();
    })
  }, 300);
}

const promiseFactories = [promiseFactoryA, promiseFactoryA];

Then, we can pass the array to this function that will run them sequentially:

const runPromiseSequentially = promiseFactories => {
  let result = Promise.resolve();

  promiseFactories.forEach(
    (promiseFactory) => {
      result = result.then(() => promiseFactory());
    },
  );

  return result;
}

runPromiseSequentially(promiseFactories);

Basically what it does is to ask the PromiseFactory to create the Promise when we want the operation to be started.

Example REPL

Though, if you could use async and await this would be unnecessary.