End to end profiling of nodejs REST Api

Question

I have written a Nodejs api that will be accessed using 5000 concurrent requests. I have test the api using Jmeter for 5000 concurrent users. The api is taking very little time initially but after 1000 request it takes 2 seconds to complete the request. The api code execution time is 400ms calculated by time difference of start and end of request.

Is there a way I can check how much time API request is spending in the event loop waiting for other requests to complete? Any tool that can help end to end profiling of the API?

Answer 1

Node.js supports profiling with the --prof flag, see eg https://nodejs.org/en/docs/guides/simple-profiling/ (as well as a bunch of other documentation and tutorials all around the internet). That doesn't answer "what did this particular request do or wait for?", but it does answer "what did the node process spend its time doing?" -- hopefully both questions have the same answer.

In addition to that, I would suggest to simplify your setup in order to narrow down what's going on. If you run only on one core (with fewer requests), does the issue still happen? If you reduce the number of concurrent requests, does the issue still happen? In the extreme, if you send requests serially, does the issue still happen? The more you can simplify things, the easier you'll be able to tell what's going on.

It could be that your implementation is leaking some resource (memory, file handlers, whatever), making things slower over time. It could be that 5000 concurrent requests are simply more than your quadcore can handle.