Loading Data from MongoDB and using cursorstream for Visualization

Question

I want to access real time data in MongoDB which will be updated online for map and graph visualization using javascript. I try to follow the example in MongoDB website using cursorstream. The code is as shown below. According to the example, this should work, but node does not recognize the .stream() part...

    var databaseUrl = "127.0.0.1/sensor_db2"; 

var collections = ["sensor"]
var db = require("mongojs").connect(databaseUrl, collections);


var stream = db.sensor.find().stream();

stream.on("data", function(item) {
    console.log(item);
});


stream.on('error', function (err) {
  console.error(err);
});

Then I got the error message like this:

    /Users/user/Documents/nodetest/nodetest.js:6
var stream = db.sensor.find().stream();
                              ^
TypeError: Object [object Object] has no method 'stream'
    at Object.<anonymous> (/Users/user/Documents/nodetest/nodetest.js:6:35)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3
/Users/user/Documents/nodetest/nodetest.js:6
var stream = db.sensor.find().stream();
                              ^
TypeError: Object [object Object] has no method 'stream'
    at Object.<anonymous> (/Users/user/Documents/nodetest/nodetest.js:6:35)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:901:3
BlackBook:nodetest user$ node nodetest.js

I run it using node.js.

Anyone can help what's wrong in it? Is there any other approach to achieve the same thing? (load data from mongodb and visualizes the data in a real-timechart)?

Answer 1

I haven't used mongojs before but it looks like your trying to cross streams with the native mongo driver, which offers a .stream() function on collections, and mongojs, which seems to offer it in a slightly different way. From mongojs' git hub page it looks like you can accomplish what you're trying to do with this.

https://github.com/mafintosh/mongojs#streaming-cursors

first npm install JSONStream

var mongojs = require('mongojs');
var JSONStream = require('JSONStream');

var db = mongojs.connect(databaseUrl, collections);
// pipe all documents in mycollection to stdout
db.sensor.find({}).pipe(JSONStream.stringify()).pipe(process.stdout);

I ran this on a collection on my database and it worked. Good luck with it. Hope it helped.

Answer 2

First of all, the reason you are getting that error is because of the .stream() you are attempting to use. As the previous answer hinted to, you can use .pipe().

However, from what I read, you desire to start this little script up and be able to capture new inserts into the collection. You can do this relatively easily using a capped collection.

To convert a non-capped collection to a capped collection, the command you issue is this:

> db.runCommand({"convertToCapped": "sensor", size: 100000});

The size parameter is the size in bytes. The reason for using a capped collection is that they are tailable; thus when new documents are inserted into the DB, you are able to retrieve them.

So with a capped collection, you should be able to do something like so:

var databaseUrl = "127.0.0.1/sensor_db2";
var collections = ["sensor"]
var db = require("mongojs").connect(databaseUrl, collections);


var stream = db.sensor.find({},{},{tailable:true, timeout:false});

stream.on("data", function(item) {
    console.log(item);
});


stream.on('error', function (err) {
  console.error(err);
});

You can find more information on capped collections on the mongodb docs page here . Also the tailable cursor example from the mongojs github page is located here .

I understand that you aren't using a capped collection, so this answer may not be your desired answer; however, I'm putting this as an answer in case you can use a capped collection.