Occasionally retrieving “connection timed out” errors when querying Postgresql

Question

I get this error every so often when utilizing sqlx with pgx, and I believe it's a configuration error on my end and a db concept I'm not grasping:

error: 'write tcp [redacted-ip]:[redacted-port]->[redacted-ip]:[redacted-port]: write: connection timed out

This occurs when attempting to read from the database. I init sqlx in the startup phase:

package main

import (
    _ "github.com/jackc/pgx/stdlib"
    "github.com/jmoiron/sqlx"
)

//NewDB attempts to connect to the DB
func NewDB(connectionString string) (*sqlx.DB, error) {
    db, err := sqlx.Connect("pgx", connectionString)

    if err != nil {
        return nil, err
    }

    return db, nil
}

Any structs responsible for interacting with the database have access to this pointer. The majority of them utilize Select or Get , and I understand those return connections to the pool on their own. There are two functions that use Exec , and they only return the result and error at the end of the function.

Other Notes

• My Postgres db supports 100 max_connections
• I only showed a few active connections at the time of this error
• I am not using SetMaxIdleConnections or SetMaxOpenConnections
• Refreshing the page and triggering the request again always works

Any tips on what might be happening here?

EDIT: I did not mention this server is on compose.io, which in turn is hosted on AWS. Is it possible AWS turns these connections into zombies because they've been open for so long and the timeout occurs after unsuccessfully trying them one by one?

Answer 1

With the help of some rough calculations, I've set the maximum lifetime of these connections to 10 minutes. I inserted this code into the init function in the question above to limit the number of open connections, idle connections, and to limit to the life of the connection to 30s.

db.SetConnMaxLifetime(time.Duration(30) * time.Second)
db.SetMaxOpenConns(20)
db.SetMaxIdleConns(20)

Hopefully this helps someone else.

SELECT * FROM pg_stat_activity; is great for nailing down connections as well.