Node.js: Handling outgoing HTTP request from node

Question

I've been wrestling with this problem for a while but could not find a good solution for this, so came here for help.

I have a node.js server; on receiving a request from a client, the server will contact 3rd party backend to grab some data, and return it back to the client.

The server to 3rd party backend communication involved multiple calls back and forth, and it typically takes ~3 seconds to finish this process.

If I fire off, say, 50 concurrent requests from test tools like JMeter, the performance degradation becomes severe very quickly, even causing timeouts for some of the later served calls.

Initially I started looking into asyncblock, but since it was running on fiber I wasn't seeing a big improvements in performance, so I started looking into threads.

The only mature module I could find was thread-a-gogo, but I also recently found out that you cannot use the required external modules(like crypto, for example) within threads spawned by TAGG.

Given that there are proxy products built with node.js I believe there is an efficient way to do this but I can't really think of other approaches if threads cannot use external modules.

Any advice would be appreciated.

I can't reveal the full detail due to NDA but here's the basic concept of what I'm doing. I'd like to send below logic to a separate thread.

asyncblock(flow){
    var result1 = flow.sync(externalRequest1(flow.callback());
    if result1 contains success message
         var result2 = flow.sync(externalRequest2(flow.callback());
         if result2 contains success message
             process result2 and return to client
}

Answer 1

First thing to check and be certain: are you using the node.js core HTTP Agent? If so, you are subject to maxSockets limit of 5 connections to the same server. Read the hyperquest README rant for details.

Secondly, be aware the remote side may impose abuse limitations as well, so check to see if there are issues there or if, for example, overall performance would be better if you used a single pool of 10 connections instead of opening unlimited number of simultaneous connections to the upstream server.