Why is my fs.readFile returning a buffer instead of XML?

Question

I have a function called fetchXML that is suppose to write an XML file to my root directory called feed.xml, and then I want to console.log the data inside feed.xml. I use fs.readFile AND I specify the encoding with 'utf-8' as shown in this question: Why does Node.js' fs.readFile() return a buffer instead of string?

But still the result of my console.log is a buffer. I checked inside feed.xml and it does indeed contain xml.

var out = fs.createWriteStream('./feed.xml');

var fetchXML = function() {
  var feedURL = 'http://www2.jobs2careers.com/feed.php?id=1237-2595&c=1&pass=HeahE0W1ecAkkF0l';
  var stream = request(feedURL).pipe(zlib.createGunzip()).pipe(out);

  stream.on('finish', function() {
    fs.readFile('./feed.xml', 'utf-8', function(err, data) {
      console.log(data);
    });
  });
}

fetchXML();

Answer 1

The main issue here is that err is set in this case and it will tell you that toString() failed (due to the size of the file). It then leaves the data it read as a Buffer and passes that as the second argument to the callback.

This could be perceived as a partial bug since most people probably would not expect to see a second argument passed in, but at the same time err is set (and you should always handle errors) and it does give you an opportunity to do something else with the (raw binary) data that was already read into memory.

As far as solutions go, you will probably want a streaming parser for large amounts of data like this (hundreds of megabytes). For XML, one such module that provides a streaming interface is node-expat .