go doesn't seem to honor persistent connections setting

Question

I have a reverse proxy built using go. I have set it to use persistent connections. But when I monitor the open connections (using lsof -i) on my machine, looks like its closing connections within a few minutes despite me setting it for 60 minutes.

If I set other timeouts in the settings below, like ResponseHeaderTimeout, those are being honored.

    myProxy.Transport = &transport{&http.Transport{
        Proxy: http.ProxyFromEnvironment,
        DialContext: (&net.Dialer{
            Timeout: 200 * time.Millisecond,
        }).DialContext,
        MaxIdleConns:          0,                       //no limit
        MaxIdleConnsPerHost:   10,
        IdleConnTimeout:       60 * time.Minute, 
    }}

Answer 1

Idle connections in http.Transport relies upon HTTP keep-alive as mentioned in http.Transport documentation . These settings are used by the transport while exchanging http payloads with peers (by adding appropriate keep-alive headers).

Having said that, HTTP keep-alive has to be agreed upon by both the parties in exchange (client and the server). Most of the servers restrict maximum number of open connections AND the duration (timeout) of open connections with keep-alive requests. This is done to avoid potential DOS attack and give all it's consumers fair chance with limited system resources at server (each keep-alive connection adds overhead on server).

A 60 minute idle timeout is way too high and I suspect any of the server will honor that (unless you control the server configuration on other end as well). Typically this is kept in few seconds to give enough time to a single HTTP page and all it's resources (css, js, images) to load from server using a single connection. Once a page loads successfully, there's no reason for the connection to be kept open in general browsing world. Apache HTTPD, by default, keeps KeepAliveTimeout as 5 seconds.

It's also worth noting, as clearly stated in Mozilla Keep-Alive documentation , timeouts longer than the TCP timeout may be ignored if no keep-alive TCP message is set at the transport level.

Given all these facts, this is absolutely normal for you to see what you see (connections being automatically closed). The transport will automatically initiate new connections as and when needed.