使用Python和PostgreSQL跟踪数据库连接问题

Question

I periodically get connection issues to PostgreSQL - either "FATAL: remaining connection slots are reserved for non-replication superuser connections" or "QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30" depending on whether psycopg or Pyramid is raising the exception. Having established that the transaction manager is properly installed, it's frustrating to not know why I am still running out of connections.

I know the connection data is in pg_stat_activity but it's a single snapshot. Is there any way of seeing connections over time so that I can see just what is actually running over a period of time (ideally from before it's an issue up until the time the issue requires an application restart)?

Answer 1

The first part is in properly identifying all of the queries running at a point in time. For that I used this query:

SELECT (SELECT COUNT(1) FROM pg_stat_activity) AS total_connections,
    (SELECT COUNT(1) FROM pg_stat_activity
     WHERE current_query in ('<IDLE>', '<IDLE> in transaction'))
       AS idle_connections,
    current_query
FROM pg_stat_activity
WHERE current_query NOT IN ('<IDLE>', '<IDLE> in transaction')
    AND NOT procpid=pg_backend_pid();

NOTE! "current_query" is simply called "query" in later versions of postgresql (from 9.2 on)

This strips out all idle database connections (seeing IDLE connections is not going to help you fix it) and the "NOT procpid=pg_backend_pid()" bit excludes this query itself from showing up in the results (which would bloat your output considerably). You can also filter by datname if you want to isolate a particular database.

I needed these results in a way that was really easy to query them and so I used a table on the database. This should work: CREATE TABLE connection_audit ( snapshot timestamp without time zone NOT NULL DEFAULT now(), total_connections integer, idle_connections integer, query text ) WITH ( OIDS=FALSE );

This will store the current timestamp in "snapshot", the total and idle connections and the query itself.

I wrote a script to insert the top query into the table and saved that into a file called "pg_connections.sql".

I ran a script to insert these results into the table every second:

while true ; do psql -U user -d database_name -f 'pg_connections.sql' >> connections.log ; sleep 1; done

What this is effectively doing is writing all CURRENTLY executing queries to the table.

Tailing the connections.log file showed me if the script was running as expected (but it isn't really required). Obviously, running a script like this every second can be taxing on a system but it's a short-term measure when you don't have any other way of finding this information out so it should be worth it. Run this script for as long as you need to accumulate sufficient data and hopefully it should pay dirt.