Persistent connection or connection pooling in PHP54+ Nginx + PHPFPM + MongoDB

Question

I am using pecl mongo 1.4.x driver( http://pecl.php.net/package/mongo/1.4.1 ), with the setup mentioned in the title on a moderate traffic services (5K - 10K request per min).

And I've found that, in mongoDB the Auth command is taking a large chunck of traffic, and connection request rate is like 30-50 per second.

This impacts the performance seriously(lock ratio up, memory management doesn't cops nicely)

And if I do netstat in a box(which I have 5-8 boxes in total), I see 2-3K mongo connections in total (some in WAIT some in ESTABLISHED) per box.

My question is how can I reduce the # of connection to mongoDB, how to setup persistent connection properly?

It seems the way of persistent connection working in PECL mongoDB Driver has been changing since 1.2 then 1.3 and it performs slightly differently in 1.4.

Here is the way I invoke the client with the driver:
$mongo = new MongoClient(
"host1:11004,host2:11004", array(
'replicaSet' => MG_REPLICASET,
'password'=>"superpasswd",
'username'=>"myuser",
'db'=>"mydb",
'journal' => true,
"readPreference"=> MongoClient::RP_SECONDARY_PREFERRED
) );

Answer 1

In the 1.4 version all connections are persistent, unless you close them yourself - which you should never do. You will see a connection per IP/username/password/database combination from each PHP processing unit. In your case, per each PHPFPM process. In order to reduce the number of connections, you need to have less username/password/database combinations. However, with 8 boxes, and 50 FPM processes and 3 nodes in your replicaset you're already at 1200 connections - without even taken into account database/username/password differences. There is not much that you can do about that , but it shouldn't have a big influence in performance. You are much more likely to hit RAM/slow disk limitations.

Answer 2

I guess I found a solution to avoid exceessive mongo connection request.

We need to set PHP_FCGI_MAX_REQUESTS (or pm. max_requests in php-fpm) to a larger number, so the process won't recycle too often.

Also I need to make sure pm.request_terminate_timeout is not too small, so workers won't be killed too often.