postgresql "idle in transaction" with all locks granted

Question

A very simple delete (by key) on a small table (700 rows) every now and then stays "idle in transaction" for minutes (takes milliseconds usually) even though all the locks are marked as "granted".

What can I do to pinpoint what causes it? I'm using this select:

  SELECT a.datname,
     c.relname,
     l.transactionid,
     l.mode,
     l.GRANTED,
     a.usename,
     a.waiting,
     a.query, 
     a.query_start,
     age(now(), a.query_start) AS "age", 
     a.pid 
FROM  pg_stat_activity a
 JOIN pg_locks         l ON l.pid = a.pid
 JOIN pg_class         c ON c.oid = l.relation
ORDER BY a.query_start;

which shows a lot of "RowExclusiveLock"s but all are granted... so I don't see what is causing this spikes of delays.

Answer 1

This is a problem of the application server.

Sessions are in state "idle in transaction" when the application does not end the transaction with COMMIT or ROLLBACK . This is to be considered a bug in the application.

The locks remain (and are of course granted, otherwise the session could not be idle) until the transaction ends.

From PostgreSQL 9.6 on, you can set the parameter idle_in_transaction_session_timeout to terminate such transactions automatically with a ROLLBACK , but that is a band-aid to avoid problems on the database rather than a solution.

Answer 2

It could also be due to a combination of 1) exhaustion of a connection pool 2) Transactions within transactions 3) Implicit postgres transactions within transactions. This article opened my eyes on this issue https://www.birdie.care/blog/birdie-engineering-update

This explains why there are no deadlocks in the database itself. It's only because app's connection pool ceiling is reached.