Async/await with forEach and axios

Question

I'm stuck on a problem and I need your help. I need to use and API for autoregister servers in a software named rudder. For reach my goal, I must use 2 http request:

1. (GET) get all the server with the status pending
2. (POST) register my server

My first request is perfectly handle but I can't figure how to make the second.

Here my code (/src/controllers/registration.js):

async function index(req, res) {
  // Check parameter
  if (!req.body.hostname) {
    return res.status(422).json({ result: 'error', message: 'Missing hostname parameter' });
  }

  try {
    const pendingNodes = await axios.get(`${config.rudderURL}/rudder/api/latest/nodes/pending?include=minimal`, { headers });
    Object.keys(pendingNodes.data.data.nodes).forEach((key) => {
      // Search in all pending machine if our node is present
      const node = pendingNodes.data.data.nodes[key];
      if (req.body.hostname === node.hostname) {
        // Registration
        const response = async axios.get(`${config.rudderURL}/rudder/api/nodes/pending/${node.id}`, { headers });
        // console.log(response);
        return res.status(200).json({ result: 'success', message: 'Host added to rudder' });
      }
    });
  } catch (err) {
    return res.status(500).json({ result: 'error', message: `${err}` });
  }

  return res.status(500).json({ result: 'error', message: 'Host not found' });
}

As you can see, I'm using Object.keys(...).forEach(...) for iterate through the result of my first request.

For your information, the response of the first request is something like that:

{
    "action": "listPendingNodes",
    "result": "success",
    "data": {
        "nodes": [
            {
                "id": "f05f2bde-5416-189c-919c-954acc63dce7",
                "hostname": "foo",
                "status": "pending"
            },
            {
                "id": "b7f597ef-2b46-4283-bf70-d5e8cd84ba86",
                "hostname": "bar",
                "status": "pending"
            }
        ]
    }
}

I need to iterate through this output and compare the hostname string of each nodes with my input (simple post request from a script or postman).

If the 2 string are equals, my second request is launch and must register my host in server (the request is simplified here but you got it).

My problem is that I can't find a way to made the second request work. I try different solutions and read on some forum and website that it's complicated to work with loop and async/await at the same time but do you have an idea for me ?

When I try my code, I get an error message:

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client

From what I understood, the forEach(...) do not wait for my second request result. So return res.status(200).json({ result: 'success', message: 'Host added to rudder' }); is called then another one, probably return res.status(500).json({ result: 'error', message: 'Host not found' }); . Am I right ?

I mean, the real difficulty (or not?) is the Object.keys(...).forEach(...) . Maybe there is an easy way to not using this function but another one ? Or a refactoring of my code ?

Regards.

Answer 1

Async await doesn't work in the forEach loop. Convert your forEach to a for loop. Something like so:

let nodes = Object.keys(pendingNodes.data.data.nodes;
for (let i = 0; i < nodes.length; i++) {
    // Perform asynchronous actions and await them, it will work
    ...
}

Read more here: Using async/await with a forEach loop

Also, a function is async, in which you can await multiple statements. You've written async in-front of axios.get which basically returns a promise. You should be awaiting that.

Change the following from

const response = async axios.get(`${config.rudderURL}/rudder/api/nodes/pending/${node.id}`, { headers });

to

const response = await axios.get(`${config.rudderURL}/rudder/api/nodes/pending/${node.id}`, { headers });

Answer 2

The error you got

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client

It is because you call res.status(200) multiple times (you put it inside forEach loop). This response must be executed once only.

I've been digging into the solution and come up to use reduce , map and Promise.all

async function index(req, res) {
  // Check parameter
  ...

  try {
    const pendingNodes = await axios.get(`${config.rudderURL}/rudder/api/latest/nodes/pending?include=minimal`, { headers });

    // we get node ids in array with hostname is matched with req.body.hostname
    const nodeIdsToRegister = pendingNodes.data.data.nodes.reduce((result, node) => {
      return req.body.hostname === node.hostname ? [...result, node.id] : result;
    }, [])

    // use `Promise.all` to register all node Ids we got previously
    const registers = await Promise.all(nodeIdsToRegister.map(nodeId => axios.get(`${config.rudderURL}/rudder/api/nodes/pending/${node.id}`, { headers })));

    res.status(200).json({ result: 'success', message: 'Host added to rudder' });

  } catch (err) {
    return res.status(500).json({ result: 'error', message: `${err}` });
  }

  return res.status(500).json({ result: 'error', message: 'Host not found' });
}