Get mongodb collection with node

Question

I'm very new to express/mongo stack, so the question is pretty simple, couldn't find anything on stackoverflow which would resolve my problem, so here it is:

I have a index.js file which looks more or less like this:

var mongoose = require('mongoose');

// Connections
var developmentDb = 'mongodb://localhost/admin';
var usedDb;

// If we're in development...
if (process.env.NODE_ENV === 'development') {
    // set our database to the development one
    usedDb = developmentDb;
    // connect to it via mongoose
    mongoose.connect(usedDb);
}

// get an instance of our connection to our database
var db = mongoose.connection;

db.once('open', function callback () {
    console.log('Databsae Connection Successfully Opened at ' + usedDb);
});

module.exports = db;

Then I'm requireing it in my express route like this:

var express = require('express');
var router = express.Router();
var db = require('../../database');

/* GET users listing. */
router.get('/', function(req, res) {
    var users = db.collection('users');
    var test = users.find();
    res.send(test);
});

module.exports = router;

I'm sending request and the result I'm getting is 'undefined' so nothing is returned from the back end.

Db connection is 100% correct and works.

I'm not entirely sure, do I have to have a schema definition on express side, or is it possible to query any data, without knowing the schema?

Answer 1

What you are missing here is a bit of the "mongoose magic" that is actually happening "behind the scenes " as it were. It's also a base concept of most operations in node.js that the operations ( particularly where IO is concerned ) are largely asynchronous in nature, so you are generally working with callbacks that fire when the operation is complete.

Take this part of your listing:

// get an instance of our connection to our database
var db = mongoose.connection;

db.once('open', function callback () {
    console.log('Databsae Connection Successfully Opened at ' + usedDb);
});

module.exports = db;

So while you may have coded that in sequence, the actual order of events is not as you might think. While you can call the db object from mongoose.connection ( and actually this is a connection object and not the Db` object as implemented by the underlying driver ) there is no guarantee that the database is actually connected at this time. In fact, in all likelihood it is not.

The sort point here is that your database connection actually happens after you export the variable from the module, and not before. It does not wait for the preceding line to complete, nor can you make it do so.

Mongoose itself rather has a concept of "models" to represent the collections in your database. So the general approach is to define these model objects and use them for accessing the data:

var Model = mongoose.model( 'Model', modelSchema, 'optionalCollectionName' );

Model.find({}, function(err,data) { 
   // do stuff in the callback
});

Part of the reason for this ( aside from the schema definitions that are attached ) is that there is actually something else going on here related to the connection. These objects in fact have internal logic that only processes the actions on the tied "collection" object only when the connection to the database is available. So there is an "internal callback" function that is happening here where an internal connection object is actually being used.

Here is some simple code that "overrides" the usage of the internal methods to just try and get the underlying collection object from the driver. It will fail:

var mongoose = require('mongoose'),
    Schema = mongoose.Schema

mongoose.connect('mongodb://localhost/test');

var modelSchema = new Schema({},{ strict: false });
var Model = mongoose.model( 'Model', modelSchema, 'optionalCollectionName' );

Model.collection.find({}, function(err,data) { 
   // do stuff in the callback
});

Since this asked for the collection object as implemented in the underlying driver to be returned and the native .find() method from that object to be used, the problem occurs that the database is not actually connected yet. So in order to make this work, you would need to wrap the call inside an event handler that fired only when the database was truly connected. Or otherwise make sure that a connection had been made before this was called:

mongoose.connection.on('open',function(err,conn) {

    // Now we know we are connected.
    Model.collection.find({}, function(err,data) { 
       // do stuff in the callback
    });
});

So the model objects are actually doing this for you, and are providing their own implementations of the standard methods such as "find", "update" etc.

If you don't want to do this kind of wrapping, and the definition of models seems like too much work, and even using the { strict: false } modifier here, which relaxes the constraints on the schema to effectively allow any data, then you are probably better off using the base driver rather than mongoose.

But of course you want something smarter than just wrapping all the code inside a callback to the connection. Here is one approach to define an object that you can use to "fetch" the database connection for you, and make sure that nothing is executed until that connection is made:

var async = require('async'),
    mongodb = require('mongodb'),
    MongoClient = mongodb.MongoClient;


var Model = (function() {

  var _db,
      conlock;
  return {
    getDb: function(callback) {
      var err = null;
      if ( _db == null && conlock == null ) {
        conlock = 1;
        MongoClient.connect('mongodb://localhost/test',function(err,db) {
          _db = db;
          conlock == null;
          if (!err) {
            console.log("Connected")
          }
          callback(err,_db);
        });
      } else if ( conlock != null ) {
        var count = 0;
        async.whilst(
          function() { return ( _db == null ) && (count < 5) },
          function(callback) {
            count++
            setTimeout(callback,500);
          },
          function(err) {
            if ( count == 5 )
              err = new Error("connect wait exceeded");
            callback(err,_db);
          }
        );
      } else {
        callback(err,_db);
      }
    }
  };

})();


async.parallel(
  [
    function(callback) {
      console.log("call model");
      Model.getDb(function(err,db) {
        if (err) throw err;
        if (db != undefined)
          console.log("db is defined");
        callback();
      });
    },
    function(callback) {
      console.log("call model again");
      Model.getDb(function(err,db) {
        if (err) throw err;
        if (db != undefined)
          console.log("db is defined here as well");
        callback();
      });
    }
  ],
  function(err) {
    Model.getDb(function(err,db) {
      db.close();
    });
  }
);

That basically wraps up an object which we called "Model" here, with a single method to .getDb() . That method simply accepts a callback, which is there to be your actual piece of code that you want to access the database with, which in turn exposes the Db object from the connection.

The Db object is stored internally in that object, so it is basically a singleton which only ever connects to the database once. But as your logic is passed in aa callback function, then it will either just pass in the existing stored object, or wait until the connection is made before passing in the code.

Output from the sample usage should be:

call model
call model again
Connected
db is defined
db is defined here as well

And that shows the order of events to how they actually happen.

So there are different ways to handle this. Mongoose models "abstract" a lot of that for you. You can of course either take a base approach with the base driver as given in the example, or take that further and implement your own connection system including overridden methods that does much of the same thing that mongoose is doing underneath. There are also other wrapper libraries that already do this without the schema concepts that is generally inherent to mongoose as well.

Basically though, every higher level library above the base driver is doing much the same as described, where there are wrappers around the methods to make sure the connection is there without needing to embed all of your code in an event listener that is checking for that.