Efficient Socket.io distribution with Mongoose stream

Question

I'm trying to create an efficient streaming node.js app, where the server would connect to a stream (capped collection) in MongoDB with mongoose , and then emit the stream directly to the client browsers.

What I'm worried about is the scalability of my design. Let me know if I'm wrong, but it seems that right now, for every new web browser that is opened, a new connection to MongoDB will also be opened (it won't re-use the previously utilized connection), and therefore there will be a lot of inefficiencies if I have a lot of user connected at the same time. How can I improve that?

I'm thinking of a one server - multiple client type of design in socket.io but I don't know how to achieve that.

Code below:

server side (app.js) :

io.on('connection', function (socket) {
  console.log("connected!");
  var stream = Json.find().lean().tailable({ "awaitdata": true, numberOfRetries: Number.MAX_VALUE}).stream();
  stream.on('data', function(doc){
    socket.emit('rmc', doc);
  }).on('error', function (error){
    console.log(error);
  }).on('close', function () {
    console.log('closed');
  });
});

client side (index.html) :

socket.on('rmc', function(json) {
  doSomething(); // it just displays the data on the screen
});

Answer 1

Unfortunately this will not depend only on mongo performance . unless you have a high level of concurrency (+1000 streams) you shouldn't worry about mongo (for the moment).

because with that kind of app you have bigger problems example: the data type and compression , buffer overflows , bandwith limit , socket.io limits , os limits . These are the kind of problems you will most likely face first.

now to answer your question. As far as i know no you are not opening a connection to mongo per user. the users are connected to the app not the database . the app is connected with the database.

lastly , these links will help you understand and tweak your queries for this kind of job (streaming)

https://github.com/Automattic/mongoose/issues/1248

https://codeandcodes.com/tag/mongoose-vs-mongodb-native/

http://drewww.github.io/socket.io-benchmarking/

hope it helps !