Run script from node.js, capture stdout, send via web socket

Question

I am calling a Ruby script through node.js which returns data via stdout, I have managed to get node to send some of the data through to its web socket but I am missing lines and don't really understand much about what I'm doing. From research it looks like I'm working with a stream but I can't get my head round much of the documentation as this is all very new and over my head :(

My Ruby script outputs as follows to stdout when run via a bash shell;

{"element":"SENS02","data":{"type":"SEN","descr":"T100"}}
{"element":"SENS01","data":{"type":"SEN","descr":"    "}}
{"element":"LED1","data":{"type":"LED","state":"1"}}
{"element":"LED2","data":{"type":"LED","state":"0"}}

When run at the console everything is good - when I call it via my node.js script I am only getting every 2nd line so for example node would output;

{"element":"SENS02","data":{"type":"SEN","descr":"T100"}}
{"element":"LED1","data":{"type":"LED","state":"1"}}

My node.js script section is below;

var cp = require('child_process');
var tail = cp.spawn('test.rb');

// Get updates from ruby script //
tail.stdout.on('data', function(chunk) {
   var pumper = chunk.toString().slice(0,58);
   var closer = JSON.parse(pumper);
   socket.emit('MAP.room1', closer);
});

I also get a crash once in a while with the following error as well if anyone can help;

undefined:0

^ SyntaxError: Unexpected end of input at Object.parse (native)

I need to be able to output every line as received from my ruby script. Anyone who answers please use my own code as above not random examples as it only confuses me hugely when trying to understand what relates to what and 'what it means for me'. I've tried looking at .pipe but all the examples confuse me and I don't understand how to adapt them for my own scripts. What is output from ruby is JS Objects 1 line at a time, I need my web socket to emit exactly the same thing 1 object at a time.

Having done console.log(chunk.toString()) to get a readable output it has relieved something odd - an extra line feed or something.

Output from console is;

{"element":"SENS02","data":{"type":"SEN","descr":"T100"}}
{"element":"SENS01","data":{"type":"SEN","descr":"    "}}

{"element":"LED1","data":{"type":"LED","state":"1"}}
{"element":"LED2","data":{"type":"LED","state":"0"}}

How can I get round this? And could this be causing my other 'Unexpected end of input' error?

If I do var pumper = chunk.slice(0,-1); console.log(pumper.toString()) var pumper = chunk.slice(0,-1); console.log(pumper.toString()) then I get the output as shown above with out the odd/extra/rouge line breaks. I'm still no further forward to getting this to output exactly that to the web socket though.

I can get this out of the web socket but it shows I have \\n still somehow?!?!

["{\\"element\\":\\"SENS02\\",\\"data\\":{\\"type\\":\\"SEN\\",\\"descr\\":\\"T100\\"}}\\n{\\"element\\":\\"SENS01\\",\\"data\\":{\\"type\\":\\"SEN\\",\\"descr\\":\\" \\"}}"]} I need to get that back into single flat line like I have at my console...

So having done as suggested below I now get the desired output by using this code. I still get ): Unexpected end of input logged in the console, that appears to be from additional line feeds I get from my Ruby script. Having run the script for a couple of days without crashes it appears all the data is getting through intact and parsed out to the web socket. I still need to read up and fully understand what the for is doing, but otherwise it appears to have fixed my issue.

tail.stdout.on('data', function(chunk) {
var closer = chunk.toString()

var sampArray = closer.split('\n');

 for (var i = 0; i < sampArray.length; i++) {
  try {
   var newObj = JSON.parse(sampArray[i]);
socket.emit('MAP.room1', newObj);
   } catch (err) {
console.log('): ' + err.message);
  }
 }
});

Answer 1

Try spawn ing you ruby script, exec is buffering the output and that just might be the reason for what you're seeing.

See

PS

Wrap JSON.parse with a try-catch block and print the data for which it failed when it fails

Answer 2

If you intend to use JSON.parse() for each data chunk, then you need to pre-parse each chunk on the newline \\n .

JSON.parse cannot parse multiple objects separated by newline as 2 distinct objects in the same string.

These are individually valid:

"{\"element\":\"SENS02\",\"data\":{\"type\":\"SEN\",\"descr\":\"T100\"}}"
"{\"element\":\"SENS02\",\"data\":{\"type\":\"SEN\",\"descr\":\"T100\"}}\n"
"{\"element\":\"SENS02\",\"data\":{\"type\":\"SEN\",\"descr\":\"T100\"}}\r\n"

This is not:

"{\"element\":\"SENS02\",\"data\":{\"type\":\"SEN\",\"descr\":\"T100\"}}\n{\"element\":\"SENS01\",\"data\":{\"type\":\"SEN\",\"descr\":\"    \"}}"

If the intent is to parse the entire data structure then you need to reorganize the output of your ruby script to something similar to below.

{
    "elements": [
        {
            "name": "SENS02",
            "data": {
                "type": "SEN",
                "descr": "T100"
            }
        },
        {
            "name": "SENS01",
            "data": {
                "type": "SEN",
                "descr": "    "
            }
        },
        {
            "name": "LED1",
            "data": {
                "type": "LED",
                "state": "1"
            }
        },
        {
            "name": "LED2",
            "data": {
                "type": "LED",
                "state": "0"
            }
        }
    ]
}

How You Might Parse the Input (Rough Example)

var sample = "{\"element\":\"SENS02\",\"data\":{\"type\":\"SEN\",\"descr\":\"T100\"}}\n{\"element\":\"SENS01\",\"data\":{\"type\":\"SEN\",\"descr\":\"    \"}}\n{\"element\":\"LED1\",\"data\":{\"type\":\"LED\",\"state\":\"1\"}}\n{\"element\":\"LED2\",\"data\":{\"type\":\"LED\",\"state\":\"0\"}}";

var sampArray = sample.split('\n');

for (var i = 0; i < sampArray.length; i++) {
    try {
        var newObj = JSON.parse(sampArray[i]);
        // socket.emit(newObj);
    } catch (err) {
        console.log('): ' + err.message);
    }
}