返回多个异步响应

Question

I'm currently looking to set up an endpoint that accepts a request, and returns the response data in increments as they load.

The application of this is that given one upload of data, I would like to calculate a number of different metrics for that data. As each metric gets calculated asynchronously, I want to return this metric's value to the front-end to render.

For testing, my controller looks as follows, trying to use res.write

uploadData = (req, res) => {
   res.write("test");
   setTimeout(() => {
       res.write("test 2");
       res.end();
   }, 3000);
}

However, I think the issue stems from my client-side which I'm writing in React-Redux, and calling that route through an Axios call. From my understanding, it's because the axios request closes once receiving the first response, and the connection doesn't stay open. Here is what my axios call looks like:

axios.post('/api', data)
  .then((response) => {
      console.log(response);
  })
  .catch((error) => {
       console.log(error);
  });

Is there an easy way to do this? I've also thought about streaming, however my concern with streaming is that I would like each connection to be direct and unique between clients that are open for short amount of time (ie only open when the metrics are being calculated).

I should also mention that the resource being uploaded is a db, and I would like to avoid parsing and opening a connection multiple times as a result of multiple endpoints.

Thanks in advance, and please let me know if I can provide any more context

Answer 1

One way to handle this while still using a traditional API would be to store the metrics in an object somewhere, either a database or redis for example, then just long poll the resource.

For a real world example, say you want to calculate the following metrics of foo, time completed, length of request, bar, foobar.

You could create an object in storage that looks like this:

{
  id: 1,
  lengthOfRequest: 123,
 .....
}

then you would create an endpoint in your API that like so metrics/{id} and would return the object. Just keep calling the route until everything completes.

There are some obvious drawbacks to this of course, but once you get enough information to know how long the metrics will take to complete on average you can tweak the time in between the calls to your API.