在Node.js中合并流

Question

I'm working with MongoDB's cursor streaming functionality. In my code multiple documents are joined together, so I want to denormalize the documents and then stream them to the client. I am confused on where to even start. Here is some pseudo-code of what I have tried:

var stream = new Readable({ objectMode: true });

var cursor = collection.find();
cursor.forEach(fetch);

function fetch(document) {
  stream.push(document);
  // Get all joined documents and run fetch() on them
}

return stream;

I get some errors because it does not implement _read . This method also makes it trickier to find when to call stream.push(null) .

What is the solution to this problem?

Answer 1

The _read method is necessary to implement a readable stream. If you prefer a simpler interface, you'll probably prefer to use a PassThrough stream:

var stream = new PassThrough({ objectMode: true });

var cursor = collection.find();
cursor.forEach(fetch);

function fetch(document) {
  stream.write(document);
  // Get all joined documents and run fetch() on them
}

return stream;

Using a readable stream can be useful if you intend to take care of back pressure but i'm not sure the mongodb API provides such mechanism anyway.

Also, look at the mongodb API to see how to properly check for the collection entries stream end and call the stream.end() method accordingly.