使用node.js从mysql查询大量数据时，网页渲染失败

Question

I am trying to query a mysql database from node.js and display its results on a web page. My code works when the size of the mysql table is small. But when the size of the mysql table is big, around 4GB , then it takes quite some time to execute the query. Although the data is extracted from the mysql table, (I can see that in the node console as I am printing it out), the web page does not render the data. I am assuming that some timeout occurs and so, it does not render anything, just says 'web page not available'.

here is my code for it:

    var connection = mysql.createConnection({
         host: 'localhost',
         user: 'xxxx',
         password: 'xxxxxx',
         debug: true
       });
       connection.connect(function (err) {
          if (err) {
          console.log('error in connection is :' + err);
          res.send(500);
          } else {

          connection.query(query, function (err1, rows) {
          // connected! (unless `err` is set)
          if (err1) {
            console.log('error' + err1);
            var model = [{
                "error": "No data matching the query"
            }];
            res.json(model);

           } else {
            console.log('result of query' + rows);
            res.json(rows);
        }

    });
}

});

I am using dust.js for rendering the data.

Any pointers on what I am missing would be helpful.

Answer 1

You are probably not missing much, and it probably is the timeout you mention. It really depends on how you want your client to behave:

1) You could increase the client's timeout (maybe it's an internal application, and it's OK to increase? Maybe it's an XHR object and you can control the timeout, let it wait "in the background" while the user is busy doing other things on the page?).

2) You could change the client-side code to do this in the background, updating the DOM once it gets the results.

3) You could have a server-side queue with the results, immediately returning to the client a "job id" and let the client poll on it. Or maybe use "comet" and send a notification to the client once the job is complete.

As I said, you really need to define how you want your client to behave.