Node.Js Threadpool in Windows

Question

So my understanding is, any blocking file system operation (such as fs.readFileSync) is eventually delegated to one of the thread in thread pool to keep event loop free. Now, I am running my application from windows and the command i am using, set UV_THREADPOOL_SIZE=4 & node index.js

My sample code below,

const start = new Date().getTime();
readFile();
readFile();
readFile();
readFile();
readFile();

function readFile() {
  fs.readFileSync('./content/1.txt');
  const end = new Date().getTime();
  console.log('Time took: ', (end - start) / 1000);
}

Now no matter whether I set the thread pool size to one or four , the execution time remains almost same. FYI, there are two CPU cores in my PC. Therefore my expectation was, if I set thread pool size to four (or let the default settings work) out of my 5 function call to read the file, say first four takes x secs (I understand it won't be the exact time for both call but going to be very close), then the last one (x+n), where x & n are random numbers and represents time difference in seconds.

But that's not happening. Irrespective of the number of thread pool calls are taking same time to complete and getting completed one by one.

So, looks like my understanding about how node.js thread pool works is not right. Any help would be appreciated. Thanks.

Answer 1

The first issue is that you're using fs.readFileSync() . That means your file operations are only going to be requested one at a time. The 2nd one won't start until the first one is done. This has nothing to do with the thread pool. This is because you're using the blocking, synchronous version of readFile() . The JS interpreter will be blocked until the first fs.readFileSync() is done and the second one only gets to start after the first one is done and so on. So, because of that, in this case it won't really matter how many threads there are to serve the file system.

If you want to engage more than one thread in file operations, you need to use asynchronous file operations like fs.readFile() so you can have more than one file operation in flight at the same time and thus have more of an opportunity to use more than one thread.

Also, file operations on the same disk are not as scalable with multiple threads/CPUs as some other types of operations because the read/write heads can only be in one place at a time so even if you do change the code to successfully engage multiple threads or CPUs you can't get full parallel file access on the same drive due to the serialization of the read/write head position.

Here's an example of a test using the asynchronous fs.readFile() :

const start = new Date().getTime();
let cntr = 0;

readFile(0);
readFile(1);
readFile(2);
readFile(3);
readFile(4);

function readFile(i) {
  fs.readFile('./content/1.txt', function(err, data) {
      if (err) {
          console.log(err);
          return;
      }
      const end = new Date().getTime();
      console.log(`Time took: ${i} ${(end - start) / 1000}`)
      if (++cntr === 5) {
          console.log(`All Done. Total time: ${(end - start) / 1000)}`;
      }
  });
}

This test would likely be more meaningful if you read a different file (that wasn't already in the OS file cache) for each call to readFile() . As it is the 2-5 requests are likely just fetching data from memory in the OS file cache, not actually accessing the disk.