understanding correct http keep-alive implementation

Question

I need to try to fix a bug in Mono. The bug has been reported and people tried to figure out a solution, for years. In my situation, it is easily reproducible as long as there is some latency. I might be able to fix the problem, but first I have to understand what would be the correct behavior.

We have a server serving http requests over keep-alive connections. Connections are configured to have a timeout and a limit of requests served before the connection is closed by the server.

A simple test periodically issues requests using .NET/Mono HttpClient::GetASync(uri) method.

    while (true)
    {
        try
        {
            var httpResponse = await httpClient.GetAsync("https://192.168.1.22/api/v1/system/status/", cancellationToken);
            if (httpResponse.IsSuccessStatusCode)
            {
                Console.Write(".");
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine("HttpClientGetStatus- Exception - " + ex.GetType() + " " + ex.Message);
        }
       Thread.Sleep(this.StatusFreq * 1000);
    }

On Microsoft .NET, it works fine. On Mono, whether Windows/Linux/OSX, with timing set to hit server limits, this test throws an exception. Mono implementation comes down to WebConnection::ReadDone, which calls Stream::ReadDone. Stream::ReadDone apparently understands that the FIN packet sent by the server means that the stream was closed and returns 0 as it should. WebConnection::ReadDone interprets this as an error and immediately throws the exception.

What would be the correct behavior? Why is there no exception with .NET?

Thank you

Answer 1

I did a bunch of digging, here and through the Mono code. There is clearly a bug in Mono. I fixed it, and will submit a patch in the next few days. RFC2616 section 8.1 deals with persistent connections. The section 8.1.4 "Practical Considerations" is relevant to our bug.

Citation:

A client, server, or proxy MAY close the transport connection at any time. ... clients, servers, and proxies MUST be able to recover from asynchronous close events. Client software SHOULD reopen the transport connection and retransmit the aborted sequence of requests without user interaction so long as the request sequence is idempotent (see section 9.1.2)...

My patch checks whether the connection is keep-alive and tries to recover, as per RFC