In Node.js, how do you store callback data from a module's function in the main app?

Question

I made a node web app and it works. However, I built it monolithic, and I'm attempting to break it out into modules for practice.

Long question short, in a module, how do I define a variable from callback results and store it in a way that it is available to the main app? is it as simple as storing the results in the module to a global variable and exporting that as well?

In my monolthic version:

app.js

var resultsArray = [];
function getDatafromHTTP(page){
    callback(data){
        resultsArray.push(data);//push json elements to array
        if(page < 50){page++;getDatafromHTTP(page);}
    }
}
getDatafromHTTP(0);
app.get(some function that displays the resultsArray)

The getDatafromHTTP function runs 50 times via the page variable.

Now that I tried to break it down:

app.js

var resultsArray =[];
var getDatafromHTTP = require(module.js).getDatafromHTTP
getDatafromHTTP(0);
app.get(some function that displays the resultsArray)

module.js

exports.getDatafromHTTP = function(page){
    callback(data){
        resultsArray.push(data);//push json elements to array
        if(page < 50){page++;getDatafromHTTP(page);}
    }
}
//error resultsArray not Defined.

I get why resultsArray is not defined in the module, and understand I can make resultsArray a variable in the module itself. If it was a return, i would simply define a variable in the main app based on the return of the function. But since the function gets data via a callback and not a return, whats the "right" way to get that data back into the main app and available for the app.get function? either as it builds, or after the 50 function runs completes?

Answer 1

There are a number of different ways to do this, but the most common pattern I've seen is to just add the value as a new property of app (or similar). So for example:

module.exports = function(app) {
  return function getHttpData(callback) {
    request.get('http://example.com/', (err, results, body) => {
      if (err) return cb(err);
      app.httpData = body;
      return cb(null, body);
    });
  }
}

Then in your calling code you'd do something like:

var app = express();
var getHttpData = require('./get-http-data.js')(app); // <-- note I'm passing the app object in here

// sometime later
getHttpData((err, data) => {
    console.log(data);
});

Obviously, I'm leaving off a few steps like some of the require statements and such, but hopefully you get the idea.

All that said, often you want to avoid global variables like that. In particular, storing things that potentially change like that in memory will lead to bugs later if your app has to scale to more than one process (each instance of the app would fetch separately, and possibly wind up with different states).

Answer 2

module.js

function getDataFromHTTP() {
  return callback(data) {
    return data;
  }
}

module.exports = getDataFromHTTP;

app.js

var getDataFromHTTP = require('module.js');

var resultsArray = [];
for (var page=0; page<50; page++) {
  resultsArray.push(getDataFromHTTP);
}