how to think about asynchronous callback operations in Node.js

Question

Learnyounode is a commandline based learn-by-doing-and-getting-your-results-tested tutorial for javascript's node

In the seventh tutorial, the idea is to make something of a http client using node.js's http's get function.

The get function has two arguments

1. the url
2. a callback function for the asynchronous job

So I scourgd the surface of the internet looking for a way to continously accept data till the operation ended. After a while seeing that every answer was pretty much along the lines of

      function callback(res){

           res.on("data",function (data) { console.log(data.toString());})
   }


  http.get(url,callback)

I thought maybe failing the inbuilt tests would give me a clue to see how to make multiple calls but weirdly enough it passed the multiple calls test.So I thought the test called the file and hence the function, multiple times .. but after some tries..I realised that wasn't the case.

So my question : what exactly goes on behind a async call ? how is it possible for the mechanism to call it more than once? What other surprises should I expect ? to me this is taking black box thinking to a whole new level and I place it on par with thinking about list monads atm.

Answer 1

I think the key point is to understand that res is an EventEmitter (IncomingMessage to be more accurate), so what does the function named callback (only once on the response event of the httpClientRequest object you create when calling http.get()) is to attach an event listener to the data event of res. Without going into details, to optimise data flow, when you receive some data bytes from the network a buffer is filled and when is full the event 'data' is triggered so you can process the incoming chunk. Hence the callback you have set to the data event is executed on every chunk of data coming from the network