What's the risk in setting the Postgres connection pool size too high?

Question

I am running a tonne of jobs in parallel using Sidekiq and a lot of them are failing to connect to the database because I've only got a connection pool size of 5.

I'd like to just bump that up to 15 (at least on localhost) but was wondering what the possible negative consequences of that might be.

Setup is Ruby on Rails, default poolsize is 5.

Answer 1

It depends on many factors like:

1. how much memory you want to allocate to your database pool
2. how long your connections last
3. the timeout on the connections
4. the locality of your database server compared to your app/web server

There are other tweaks that some connection pools have also such as the minimum number of connections to have open (even if not used), and the maximum open connections which looks like what you are trying to set.

I have heard that you can potentially saturate your network card with as little as 10 open connections.

I think the only answer is to monitor your cpu/memorry/io usage based on what you have so you have some sort of a baseline , and then bump the connection count up and compare.

Personally I think you should be fine with 15 connections assuming you arent' already pushing your server to the limit or have a tiny VM with 256MB of ram :)

Setting the value too high could saturate the # of allowable open connections to postgres (check the default but it may be around 100). This could especially be a problem if you prematurely shut your services down without allowing it to gracefully close the connections. Then when you try and restart your app server it will error out saying that postgres is not allowing any more connections. This isn't a problem of setting it too high as this would happen in either case but it would def. accelerate the issue.