How to wait for the iteration to finish when pushing result of a callback function into an array

Question

What is the correct way to implement array.push so that it "array_of_results" is returned after the forEach iteration if finished?

const postgres = require("./postgres");

function get_array(value) {
    var array_of_results = []
    value.forEach( item => {
        postgres.query(item["id"],function(res){ 
            console.log(res) //gives proper res after empty array
            array_of_results.push(res);
        })
    });
    console.log(array_of_results)// prints empty array
    return array_of_results;
}

Edit: and postgres.js looks like :

const { Pool } = require("pg");
const pool = new Pool();

var query_string = "select...."

function query(id, call) {
    pool.query(query_string, [id], (err, res) => {
        if (err) {
            console.log(err.stack)
        } else {
            call(res.rows[0])
        }
    })
}

module.exports = {
    query
}

Answer 1

There are a few ways to do this, but first you need to understand what is actually happening.

In postgres.query(item["id"],function(res){ you are calling postgres.query with (1) an item ID and (2) a callback function. That call happens and then immediately continues in your calling code. So now you've just sent a bunch of requests to your database, and then immediately return an empty array. Those callbacks (2) have not been called yet.

To get the data back to your calling function, you'll need to either pass a callback instead of using return , or change to async/await.

Using async/await in every iteration of your loop is not as efficient, as you're waiting for each call to return sequentially. For the most efficient method, you will need to fire the requests and wait for them all to complete. You can do this by using promises.

You can modify your code to push a promise into an array for each iteration of the loop, then call (and await) Promise.all on the array of promises.

Here's a basic rewrite for you:

postgres.js:

function query(id) {
    return new Promise((resolve, reject) => {
        pool.query(query_string, [id], (err, res) => {
            if (err) {
                console.log(err.stack)
                reject(err)
            } else {
                resolve(res.rows[0])
            }
        })
    })
}

module.exports = {
    query
}

get_array implementation :

async function get_array(value) {
    var array_of_promises = [], array_of_results = []
    value.forEach( item => {
        array_of_promises.push(postgres.query(item["id"]));
    });
    array_of_results = await Promise.all(array_of_promises);
    console.log(array_of_results)// prints populated array
    return array_of_results;
}

Note that when you call get_array you'll have to use await before the call, eg change var array = get_array(items) to var array = await get_array(items) and using await in a function requires it to be declared as an async function.

If you can't declare it as an async function, you may change the calling code to consume the promise:

var arrayPromise = get_array(items);
arrayPromise.then((results) => {
    // do something with results
    // but remember you cannot _return_ from within a callback, as discussed above
});