Callback Confusion and Collecting Data in Node

Question

I'm new to node and having trouble following what's going on here. my comment is below, but basically i'm really struggling to follow why the callback in utils.collectData is there (second parameter) and how it arbitrarily adds to the messages array. how would I know to naturally do this in the future? I know that the collectData function is defined as taking a callback as its second argument, so let's start from there...

var utils = require('./utils');

"POST": function(request, response) {
        //collect the data




       utils.collectData(request, function(message) {
            message.objectID = ++objectIDcounter;
            messages.push(message);
            utils.sendResponse(response, {
                objectID: message.objectID
            }, 201)
        })
    }
   // this is in utils.js:
    //why is a callback parameter passed to this function?
    exports.collectData = function(request, callback) {
        var data = ""
        request.on('data', function(chunk) {
            data += chunk;
        })
        request.on('end', function() {
    //why is the callback applied here? 
//Also, we are parsing the data here. why? because it is a buffer or a string here?
                callback(JSON.parse(data));
            })
        }

Answer 1

Asynchronous operations in Node.js follow a simple and consistent pattern. Mastering this will make your experience substantially more enjoyable and your applications faster and more stable.

I think that the other answers here are missing the point a bit. The method you are calling follows an asynchronous pattern that is ubiquitous in the Node.js ecosystem. There are various patterns to simplify this, but your first task should be to thoroughly understand exactly what is occurring and what effect that has upon your code. It's pretty simple:

asyncFunction(functionArguments, callback);

When you call any asynchronous function, your script continues to execute while that function's result is pending. Since you have no guarantee as to when your results will be ready, you supply the function with your intentions for the result before moving on. Often, the callback you supply will be an anonymous function, such as:

getExternalData('http://example.com', function(err, data) {
  console.log('I got the data!');
  // presumably do something with the data
});

The callback will almost invariably take two arguments: err and data (or err and result , or err and whateverYouAreLookingFor ). If an error occurs in the function, it will be returned to your callback, and if any data results from the function's execution, it will also be returned to your callback.

The beauty of this pattern is that you are never forced to block the event loop: your script remains happily responsive regardless of the the complexity of its task. When your boss drops a poorly conceived project onto your desk just after lunch on a Friday and then leaves work early to golf, you are, effectively, role playing as an asynchronous function.

The challenge here is that by the time your callback is called, it is the only surviving link to the original request. The implications of this are twofold:

1. Any logic for processing the response must either be contained within the callback or accessible to the callback; and

2. try/catch blocks are completely useless for handling any errors thrown by the asynchronous function.

The first implication is the genesis of what is often described as "callback hell": deeply nested callbacks within callbacks within callbacks. You can avoid this pretty easily by planning ahead, keeping your functions small, and following the same conventions for arguments.

The second implication means that your callback must check for errors. You typically won't see errors thrown, since the only way to safely handle thrown errors under this pattern would be to place nearly all code within try/catch blocks. Instead, errors will be passed to the callback as a first argument. If no error was generated, the first argument will be null.

const myCallback = function(err, data) {
  if (!!err) {
    console.log('Well, that was a failure');
    return err;
  }

  console.log('Everything's great!');
}

This ended up being a bit verbose, but it's critically important that you become comfortable with this pattern. Some of the core modules have normal and synchronous versions of functions, such as fs.readFile and fs.readFileSync . Don't succumb to using them; asynchronous is pretty simple once you're accustomed to it, and, just starting out, you need all the practice you can get.

Best of luck.

Answer 2

One of the main features of NodeJs is the concurrence (In a easy way). The best way to start programing in NodeJs is think that all the code that interact with an external resource must follow one of the next pattern:

• Callbacks
• Promise (ES2015)
• Async/Await (ES2016)

Why is this?

When you call an external resource in an async way (HTTP Service, Files, etc.). The main process create an internal thread and execute the code, this is doing to not block the main process.

Callbacks:

Was the first way to manage the concurrence in javascript, as functions are first-class, you only pass the function that you want to execute, and the main function choose when call it.

In your example, you are using this approach. You are reading the data from the request and store in the local variable, when the request launch the event 'end' you choose call the callback with the data as arguments and the "POST" use that data to manipulate the message and send the response.

In the future exist two more options, both could be used today but Async/Await is in stage 2 of tc39 http://tc39.github.io/ecmascript-asyncawait/ .

Promise:

Is the most popular way today, you can used native with the version 4 of NodeJs or you can use libraries like Q or BlueBird . This work a litle different, I recommend to read the pouchd post about promise, here i'm going to show your code written using promises:

var utils = require('./utils');

"POST": function(request, response) {
        //collect the data
       utils.collectData(request)
            .then(function(message) {
                message.objectID = ++objectIDcounter;
                messages.push(message);
                return utils.sendResponse(response, {
                    objectID: message.objectID
                }, 201);
            }))
            .catch(function(err) {
                //Some error happend with the reques
            });
    }

   // this is in utils.js:
    //why is a callback parameter passed to this function?
    exports.collectData = function(request) {
        return new Promise(function (resolve, reject) {
            var data = ""
            request.on('data', function(chunk) {
                data += chunk;
            });

            request.on('end', function() {
                resolve(JSON.parse(data));
            });

            request.on('error', function(err) {
                reject(err);
            });
        }    
    }

Answer 3

Even When I started programming in NodeJS from Java as it's a major shift in the designing our programs from Blocking IO to Non blocking IO I too faced to understand the above code.

Here in Non blocking IO everything works based on events & callbacks. In the above code on calling upon collectData, nodejs(eventloop) will be waiting for on(with callback to collect data) & on(with callback which you have provided to process the completed request) events.

As simple in NodeJS any blocking code(network call or access to any IO device) will be tagged with an event which will get triggered once the I/O operation is completed. Hope this helps.