Will Node.js get blocked when processing large file uploads?

Question

Will Node.js get blocked when processing large file uploads?

Since Node.js only has one thread, is that true that when doing large file uploads all other requests will get blocked?

If so, how should I handle file uploads in nodejs?

Answer 1

All the I/O operations is handled by Node.js is using multiple threads internally; it's the programming interface to that I/O functionality that's single threaded, event-based, and asynchronous.

So the big upload of your example is performed by a separate thread that's managed by Node.js, and when that thread completes its work, your callback is put onto the event loop queue.

When you do CPU intensive task it blocks. Let's say we have a task compute() which needs to run almost continuously, and does some CPU intensive calculations.

---

Answer to the main question " How should I handle file uploads in nodejs? "
Check in your code (or the library) where you save file on the server, is it dependent on writefile() or writeFileSync() ?
If it is using writefile() then its asynchronous; But if it is writeFileSync() its is synchronous version.

---

Updates: In response to a comment:

"the answer "No, it won't block" is correct but explanation is completely wrong. JS is in one thread AND I/O is in one (same) thread. Event loop / asynchronous processing / callbacks make this possible. No multiple threads required. " - by andrey-sidorov

There is no async API for file operations so Node.js uses a thread pool for that. You can see it in the code of libuv . You can look at the source for fs.readFile in lib/fs.js , you'll see binding.read. Whenever you see binding in Node's core modules you're looking at a portal into the land of C++. This binding is made available using NODE_SET_METHOD(target, "read", Read). If you know any C, you might think this is a macro – it was originally, but it's now a function.

Going back to ASYNC_CALL in Read , one of the arguments is read : the syscall read . But wait, doesn't this function block?

Yes , but that's not the end of the story. An Introduction to libuv denotes the following:

" The libuv filesystem operations are different from socket operations. Socket operations use the non-blocking operations provided by the operating system. Filesystem operations use blocking functions internally, but invoke these functions in a thread pool and notify watchers registered with the event loop when application interaction is required. "

Summary: Node API method writeFile() is asynchronous, but that doesn't necessarily mean it's non-blocking underneath. As the libuv book points out, socket (network) code is non-blocking, but filesystems are more complicated. Some things are event-based (kqueue), others use thread pool (as in this case).

Consider going through C code on which Node.js is developed , for more information:

• Unix fs.c
• Windows fs.c

Answer 2

That depends on the functions used for accomplishing that task. If you are using asynchronous functions, then Node.js will not block. But there are also synchronous functions, for example fs.readFileSync ( FileSystem Doc ), that will block execution.

Just take care and choose asynchronous functions. This way Node.js will keep running while slow tasks/waits are completed by external libraries. Once those tasks are completed, the Event Loop will take care of the result and execute your callbacks.

You can read more about the Event Loop here: Understanding the Node.js event loop

Answer 3

That is the exact reason node.js being asynchronous.

Most (all?) functions in node.js involving Input/Output operations (where bottleneck is some other device than CPU or RAM) the operation happens on a sepparate thread, letting your node.js server do some other code while waiting.