How node.js single thread model process concurrent request?

Question

Two concurrent requests R1 and R2 come to node.js server. Node.js server. Node.js is running with single thread T1. T1 takes 5 seconds to process the request R1. Assume this time spent in some actual processing of big function and there is no waiting/IO call.

My question is - will request R2 will be taken up by after 5 seconds(once R1 completes) or both(R1 and R2) will be run in round robin fashion?

If answer is sequential(ie R2 will be taken up after 5seconds), My followup question is say i have got 5k http concurrent requests and each request takes 2 ms, then last request will be served after 5k*2ms= 10 sec. Is it not bad ? Do i need to go for clustering here ?

Answer 1

will request R2 will be taken up by after 5 seconds(once R1 completes) or both( R1 and R2 ) will be run in round robin fashion?

Yes, R2 will be taken up only after R1 completes if R1 is synchronous.

In short, you can google nodejs event loop . There are a lot of great articles explaining how Node.js uses an event loop to handle requests.

The event loop is what allows Node.js to perform non-blocking I/O operations — despite the fact that JavaScript is single-threaded — by offloading operations to the system kernel whenever possible. (Source: https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick/ )

You are correct that Node.js is single-threaded. It cannot process anything else if it is blocked by a long running task. In your situation, you should either breakdown R1 into smaller pieces to be processed asynchronously, or you could use a child_process to offload the operation to another thread.

If answer is sequential(ie R2 will be taken up after 5seconds), My followup question is say i have got 5k http concurrent requests and each request takes 2 ms, then last request will be served after 5k*2ms= 10 sec. Is it not bad ? Do i need to go for clustering here?

It depends. 2ms is actually a long time for the computer to process a lot of things. Before you go for clustering, you should refactor your code to minimize the blocking code in the request handler as mentioned above. And before you buy more servers for clustering, you could fully utilize your CPU cores by cloning your application to other threads using the cluster library. A well-designed Node.js application should be capable to handle thousands of requests without issues. Otherwise, you might reconsider if Node.js is the best fit for your application.

Bonus: Let's listen to the inventor of Node.js - why he created Node.js

Answer 2

If your function is CPU-bound, you'll tie up the event loop. You have two options.

1. Shift to a different model of communication with other processes, perhaps running their own event loop: pre-fork or worker MPM. You can spawn those processes from within Node if you'd like using child_process
2. Limit CPU processing, and issue a .nextTick() to allow other things in the event loop to complete and resume later.