MongoDB Optimal Performance - How Many Persistent Connections

Question

I have a mongodb server in production serving on an EC2 instance. According to the mongodb official documentation, persistent DB connections should ALWAYS be used in production. I've been experimenting with about 50 persistent connections and was getting frequent connection errors (approx 33% of the time) while testing. I'm using this code:

$pid = 'db_'.rand(1,50);
$mongo = new Mongo("mongodb://{$user}:{$pass}@{$host}", array('persist' => $pid) );

Some background on the application, it's a link tracking application that is still ramping up - and is in the range of 500 - 1k writes per hour, nothing too crazy... yet.

I'm wondering if I simply need to allow more persistent connections? How does one determine the right balance of persistent connections versus server resources available?

Thanks in advance everyone.

Answer 1

The persist value is no longer supported as of the most recent driver (1.2.0).

Truth is, it was never really clear what it did in typical Apache+PHP setups. There are several comments on the Google Groups and elsewhere asking for detail, but I did not any evidence that persist or persistent was ever tested with any depth.

Instead, it's all been replaced by connection pooling "out of the box". The connection pooling has obviously been through some changes within the 1.2 line with the addition of the MongoPool class.

There is still no detailed explanation of how the pooling works with Apache, but at least you don't have to worry about persist .

Now despite all of this mess, I have handled 1000 times that traffic on a single MongoDB server via the PHP driver without lots of connection problems.

Are you catching the exceptions?

Can you provide more details about the exact exception?

There may be a code solution.

Answer 2

Are you opening a new connection for each PHP page request, or using a connection pool with 50 persistent connections? If you're opening a new connection each time then you might be quickly running out of resources.

Each connection uses an additional thread on the server, so you could be hitting a limit on the number of threads of network connections, check your server logs in /var/lib/mongodb for errors.

If you're using the official MongoDB PHP driver , then as far as I know it should handle connection pooling for you automatically. If you're connecting to Mongo from 50 separate clients, then consider putting a queue in front of Mongo to buffer the writes.

Answer 3

http://php.net/manual/en/mongo.connecting.php

without Persistent Connections x1000

It takes approximately 18 seconds to execute

Persistent

...it takes less than .02 seconds