Why is mongoose package very slow to load from network drive?

Question

When starting a NodeJS app stored on my network drive, which requires mongoose (also in node_modules on the same network drive), it takes about 15-20 seconds for the app to start. (Test app on local drive loads mongoose much faster.) My network drive is on a 100Mbit wired Ethernet network, and the actual transfer speeds are around 80Mbit per my tests. The Mongoose package itself is only about 1.75MB. Does it seriously load 150MB of other modules? or is something else going on that I can adjust or improve?

Thanks for any tips.

Answer 1

It's not the bandwidth but latency that is often the bottleneck when working across a network.

NodeJS, due to it's modular nature, is inherently filesystem-heavy; a typical node_modules folder can contain thousands of tiny .js files, and accessing them requires a lot of round-trips to the filesystem, and that can be very slow.

To load just the mongoose dependencies, 452 .js files need to be accessed.

In addition, NodeJS uses heuristics when searching for .js files, as there are different ways in which a Node project can be structured. That requires listing a directory first before reading a specific file (so an extra round-trip or two, per file).

To get a feel of it, try for example copying 1,000 small files over a network. The average throughput would be much less than of a single big file.