如何修改node.js流

Question

I am streaming an xml file from S3. I need to build a new xml file with a different structure for sphinx search engine. I am already streaming the file from S3, piping it into my SAX parser but now I need to figure out how I can make modifications to the stream (after the SAX parser) and upload to S3.

parser.on('startElement', function(name, attrs) {
    // Do something
});

I found what seems to be a great S3 library that streams called knox , so I am currently using that library. I'm not stuck on this library, just what I found that seems to be decent. The code that they have to stream data to S3 in the example, is only from an HTTP request. I am relatively new to streams, since I have a PHP background.

Knox Example Stream:

http.get('http://google.com/doodle.png', function(res){
  var headers = {
      'Content-Length': res.headers['content-length']
    , 'Content-Type': res.headers['content-type']
  };
  client.putStream(res, '/doodle.png', headers, function(err, res){
    // Logic
  });
});

I am thinking I would need to do something on the lines of this.

parser.on('startElement', function(name, attrs) {

  var headers = {
      'Content-Length': res.headers['content-length']
    , 'Content-Type': res.headers['content-type']
  };
  client.putStream(res, '/doodle.png', headers, function(err, res){
    // Logic
  });

});

Any help is greatly appreciated. Thanks.

Answer 1

This discussion of Node's new streams speaks explicitly about transforming streams from S3.

NB: This is in reference to streams as implemented in Node 0.10.x

Answer 2

https://www.npmjs.org/package/through

This module will do what you need.