Node JS Processor Architecture

Question

I was trying to fetch a solution to my problem regarding node js architecture.

I know node js is single threaded and works on event looping and non-blocking mechanism.

But my problem is how the processor behind it works does it works synchronously and on a priority basis? or it works on multiple requests simultaneously on a single core machine.

Tried to verify this by creating two API's In 1st API I created a while loop whereas in the 2nd API I created a response of hello world.

Then I tried to hit both API's 1st and then 2nd but the second API was waiting for the first to end so here the processor was not working on the second API till the first end.

So can I say that the processor of node js will work in synchronous order but not in parallel?

So all the requests will have to wait in a queue for previous requests to complete?

Answer 1

From my perspective, the thing is, answer to this question is not that simple.

1) If we are talking about ExpressJS then yes it works synchronously

2) But let say we are talking about nodeJs built-in executions, like http module it can make calls asynchronously (but it depends upon OS on which NodeJs is running).

Different modules in NodeJs behave differently from synchronous to asynchronous. NodeJS V8 based VUlibs which are actually C++ based libraries and most of these libraries use threading mechanism. Like crypto module can run execution asynchronously up to some extent.

Answer 2

Your Javascript is single threaded - ALWAYS and it's not pre-emptive either. So an event occurs, some Javascript starts running in response to that event and NO other Javascript can run until that event handler returns control back to the interpreter by returning from the callback that was called to start the event handler.

At that point, another event can fire and run another event handler and again no other Javascript can run until that event handler returns back to the system.

This is true for ALL Javascript that node.js runs.

Ahhh, but not every thing you do in node.js is actually Javascript. Let's suppose you have two simple timers:

console.log("A");
setTimeout(function() {
   console.log("B");
   fs.readFile("smallfile.txt", function(err, data) {
       console.log("C");
   });
   console.log("H");
}, 1000);

console.log("D");

setTimeout(function() {
   console.log("E");
   fs.readFile("bigfile.txt", function(err, data) {
       console.log("F");
   });
   console.log("I");
}, 1000);

console.log("G");

This will log in the console

A
D
G
B
H
E
I
C
F

Let's discuss why it will do this.

The A is obvious, it's the first statement.

Then, the code calls setTimeout() which is non-blocking and asynchronous. So, all it does is schedule a timer for some time in the future and then immediately returns. The setTimeout() callback won't be ready to be called for 1000ms so it doesn't get called right now.

So, next after the setTimeout() call returns, we execute the console.log("D");

Then, it executes the next setTimeout() which again just schedules a timer for 1000ms for now and immediately returns.

Then, it executes console.log("G"); .

At this point, it returns control back to the JS interpreter and it has nothing else to do for a little while.

Then, approx 1000ms later, the first setTimeout() timer sticks a timer event in the event queue and since the JS interpreter is not running any other Javascript, it grabs that event and runs the JS callback associated with that event. That immediately logs console.log("B"); .

Then, it calls fs.readFile("smallfile.txt", ...) . This is again non-blocking and asynchronous so all it does is tell the underlying native code fs module implementation to go read all the data from this file and then returns immediately. It then runs console.log("H"); . At this point, it's done with this event handler and returns control back to the interpreter.

Meanwhile, the second setTimeout() event is ready so the JS interpreter pulls that event from the event queue and calls the callback associated with it.

That runs console.log("E"); and then calls fs.readFile("bigfile.txt", ...) . This is again non-blocking and asynchronous so all it does is tell the underlying native code fs module implementation to go read all the data from this file and then returns immediately. It then runs . This is again non-blocking and asynchronous so all it does is tell the underlying native code fs module implementation to go read all the data from this file and then returns immediately. It then runs . This is again non-blocking and asynchronous so all it does is tell the underlying native code fs module implementation to go read all the data from this file and then returns immediately. It then runs console.log("I");`. At this point, it's done with this event handler and returns control back to the interpreter.

Some time later, the fs.readFile("smallfile.txt", ...) finishes and inserts an event into the event queue. Since the JS interpreter has nothing else to do, it fetches that event and runs the callback associated with it and we see the results of console.log("C") . That is then finished and returns control back to the interpreter.

Some time later, the fs.readFile("bigfile.txt", ...) finishes and inserts an event into the event queue. Since the JS interpreter has nothing else to do, it fetches that event and runs the callback associated with it and we see the results of console.log("F") . That is then finished and returns control back to the interpreter.

At this point everything is done.

Hopefully you can see how asynchronous operations (implemented in native code can run in parallel with Javascript execution using other CPU cores) and they are synchronized with Javascript execution by placing events in the event queue. When the JS interpreter is done running an event handler, it can then look in the event queue for the next event to run, or if nothing there now, go to sleep until the next event is inserted in the event queue.

So, while native code implementations can run in parallel with Javascript, there's still only one thread of Javascript execution that runs at a time and another one can't run until the prior one returns control back to the interpreter by returning from whatever event handler callback start its execution.

Internally in the event loop, there are actually a bunch of different types of queues that are used for different types of events and they have somewhat different priorities relative to one another so it's all a little more complex than I've described it here, but the core concept is an event driven system that runs the Javascript associated with only one event at a time (single threaded) and native code implementations of some operations that can run in a parallel with Javascript execution that are synchronized with Javascript's single thread of execution via the event queue.