How should I handle a “too many connections” issue with mysql?

Question

The web startup I'm working at gets a spike in number of concurrent web users from 5000 on a normal day to 10,000 on weekends. This Saturday the traffic was so high that we started getting a "too many connections" error intermittently. Our CTO fixed this by simply increasing the max_connections value on the tatabase servers. I want to know if using one persistent connection is a better solution here?

ie instead of using:

$db = new mysqli('db_server_ip', 'db_user', 'db_user_pass', 'db_name');

We use:

$db = new mysqli('p:db_server_ip', 'db_user', 'db_user_pass', 'db_name');

We're already using multiple MySQL servers and as well as multiple web servers (Apache + mod_php).

Answer 1

You should share the database connection across multiple web requests. Every process that is running on the application server should get an own mysql connection, that is kept open as long as the process is running and reused for every web request that comes in.

Answer 2

From the PHP Docs :

Persistent connections are good if the overhead to create a link to your SQL server is high.

And

Note, however, that this can have some drawbacks if you are using a database with connection limits that are exceeded by persistent child connections. If your database has a limit of 16 simultaneous connections, and in the course of a busy server session, 17 child threads attempt to connect, one will not be able to.

Persistent connections aren't the solution to your problem. Your problem is that your burst usage is beyond the limits set in your database configuration, and potentially your infrastructure. What your CTO did, increasing the connection limit, is a good first step. Now you need to monitor the resource utilization on your database servers to make sure they can handle the increased load from additional connections. If they can, you're fine. If you start seeing the database server running out of resources, you'll need to set up additional servers to handle the burst in traffic.

Answer 3

Too Many Connections

Cause This is error is caused by

• a lot of simultaneous connections, or
• by old connections not being released soon enough

You already did SHOW VARIABLES LIKE "max_connections"; and increased the value.

Permanent Connections

If you use permanent or persistent database connections , you have to always take the MySQL directive wait_timeout into account. Closing won't work, but you could lower the timeout. So used resources will be faster available again. Utilize netstat to find out whats going on exactly as described here https://serverfault.com/questions/355750/mysql-lowering-wait-timeout-value-to-lower-number-of-open-connections .

Do not forget to free your result sets to reduce wasting of db server resources.

Be advised to use temporary, short lived connections instead of persistent connections. Introducing persistence is pretty much against the whole web request-response flow, because it's stateless. You know: 1 pconnect request, causes an 8 hour persistant connection dangling around at the db server, waiting for the next request, which never comes. Multiply by number of users and look at your resources.

Temporary connections

If you use mysql_connect() - do not forget to mysql_close(). Set new_link set to false and pass the CLIENT_INTERACTIVE flag. You might adjusting interactive_timeout , which helps in stopping old connections blocking up the work.

If the problem persists, scale

If the problem remains, then decide to scale. Either by adding another DB server and putting a proxy in front, (MySQL works well with HAProxy) or by switching to an automatically scaling cloud-service.

I really doubt, that your stuff is correctly configured. How can this be a problem, when you are already running multiple MySQL servers, as well as multiple web servers? Please describe your load balancing setup.

Sounds like Apache 2.2 + mod_php + MySQL + unknown balancer, right?

Maybe try

• Apache 2.4 + mod_proxy_fcgi + PHP 5.5/5.6 (php-fpm) + MySQL (InnoDb) + HAProxy or

• Nginx + PHP 5.5/5.6 (php-fpm) + MySQL (InnoDb) + HAProxy .