How we send in response large amount of data over 2 millions records from mongo expres to node js

Question

i have a post api in express that response a mongo collection data that has over 2 milions records how we have to send back in response. frontend: jquery ajax call backend: express mongo node

Answer 1

The problem arises because you are using more memory than you have. That's happens because you seem to be loading the 2 million records in memory then trying to JSON them before sending.

You will need to stream the mongo records to the response, that way you will have only a subset of the collection at a time.

cursor.stream().pipe(JSONStream.stringify()).pipe(res);

JSONStream is a package that can help you do that very easily. Cursor would be your mongo cursor.

Read more about stream on the nodeJS docs to understand how it works. Basically you're receiving one document, stringify it, and send it out, rince and repeat.