Nodejs infinitive stream from file read stream

Question

I'm facing troubles trying to accomplish pretty banal task. I need to create nodejs Readable stream from input txt file. I need to perform some transforming on this stream (create JSON object for each line).

Problem is that I want this stream to be infinitive: after last line is read, stream should just start from beginning. My solution works bit I'm getting warning message:

(node) warning: possible EventEmitter memory leak detected. 11 drain listeners added. Use emitter.setMaxListeners() to increase limit.

I hoped to find simple solutions without reading and buffering file directly.

//Transform stream object
var TransformStream = function () {
    Transform.call(this, {objectMode: true});
};
util.inherits(TransformStream, Transform);

TransformStream.prototype._transform = onTransform;
TransformStream.prototype._flush = onEnd;

var ts = new TransformStream();

var infinStream = function () {
    var r = fs.createReadStream(filePath);
    r.pipe(split(), {end: false})
        .pipe(ts, {end: false});
    r.once('end', function () {
        //console.log('\n\n\n\nRead file stream finished. Lines counted:\n\n\n\n' + detectionCounter);
        r.removeAllListeners();
        r.destroy();
        infinStream();
    });
    return r;
};
infinStream();

return ts;

Answer 1

From the comments:

I need server that will be live for 24h/day, and will simulate device output all the time.

To do that a recursive function is a good idea. The approach you make is ok. When you don't need different transforming tasks on your data, a stream is not really needed. Simple Events can do exactly what you want and they are easier to understand.

The error in your code is the point, where you put your listener. The listenster r.once is inside your recursive function. You are defining r inside your function, so with every function call a new r is created. Because of that r.once does not work like you are expecting it.

What you can do:

1. Make a recursive function which emitts an event
2. use the data from your event outside

This is just a simple concept by using simple events, which are fireing the whole time the data from your file:

// Your recursive function
var simulateDeviceEvents = function(){
  fs.readFile('device.txt', function (err, data) {
    if (err) throw err;
    // Just emit the event here
    Emitter.emit('deviceEvent', data);
  });
  //If this happens to fast you could also call it with
  //a timeout.
  simulateDeviceEvents();
};

// Start the function
simulateDeviceEvents();
//IMPORTANT: The listener must be defined outside your function!
Emitter.on('deviceEvent', function(data){
   // Do something with your data here
});