How to call a large 10000 row data with maintain speed in nodejs

Question

 pool.getConnection(function (err, connection) {
   connection.query("SELECT * FROM ALLURELIBRARY", function (err, rows) {
       connection.release();
       if (err) throw err;

       console.log(rows);

       res.render('index', { title: 'AllureCostCenter',data:rows });
   });
});

This request give me 10,000 data from my sql cloud. It takes about 5 to 10 second to process. Would you please tell me a better way to print this large amount of data in nodejs without time delaying

Answer 1

The more data you have the more longer it will take to retrieve. It's a regular behavior. If you are fixing it for 10.000 today, you i'll get the same problem tomorrow with 15.000.

Instead of performing one request and wait for all data to get load. You can use of cursors. Cursors allow you to retrieve some data, treat them and do it again until you treated all data.

Here is the cursors documentation of google-cloud.