Is it necessary to place TensorFlow.js computations in a Node.js worker thread?

Question

My question is about whether it is a good idea to move TensorFlow.js computations (using @tensorflow/tfjs-node ) to a worker thread in Node.js. This question boils down to the question of to what extent the main thread is blocked when running TensorFlow computations using the native backend.

As I understand it, if I use the fully JavaScript version of TensorFlow ( @tensorflow/tfjs ) and call model.fit(x, y, options) , the computation that fits the model is actually happening on the main JavaScript thread. To be more precise, model.fit returns a Promise , so the computation is being executed asynchronously in microtasks, but those are still running on the main thread. These microtasks are blocking in the sense that execution of, say, an event handler, needs to wait for the microtask queue to empty. So in that case, it makes sense to move computation to a Web Worker to avoid blocking the main thread while computation is ongoing.

However, if I use the native version of TensorFlow ( @tensorflow/tfjs-node ), the "real work" is happening in native code. This is where there are two things that I don't know about, which are preventing me from answering the question in the title:

1. Is the computation behind model.fit running on the same thread as the main JS thread?
2. Regardless of the answer to 1, we still have microtasks related to the promise handling of model.fit . Does the fact that this happens in the microtask queue prevent other events from being handled until the microtask queue is empty?

Here's an example of what I'm asking about: let's suppose I have a Node.js application that sends and receives socket.io messages. One of these messages is a request to fit a TensorFlow.js model; when the application receives this message, it calls model.fit(x, y, options) . Supposing that I have not placed the TensorFlow computation in a worker thread, can the application still reply to socket.io ping messages from the server while the computation is ongoing?

Answer 1

The javascript runtime will run all the code in the main thread unless told to use workers. To answer the question, what happens with @tensorflow/tfjs-node is not different from what happens within the browser. The only difference is the backend for computation. Though, the promise runs in a microtask and are not meant to block the event loop, if the promise callback itself is a cpu intensive operation, it will block the main thread. It does happen when sometimes it can be observed that the browser is unresponsive. The same thing does happen on nodejs too.

To prevent those little freezes, another worker can be used that will be dedicated for training the model ( model.fit ) or predicting ( model.predict ). That way the main thread will always be available for handling other processing.

To continue further with the question in the case of a single main thread that is used, the socket.io messages will be queued whenever they arrive. But they will be processed only when your promise has completed. When the main thread had already started executing model.fit it won't be able to reply to the socket messages until model.fit resolves. This is what makes workers handy. Because while the worker is busy processing model.fit your main thread is freed to take care of any other incoming messages in the event loop