.net core httpClient calls fail randomly with many different error messages

Question

I have an app that behaves fine most of the time, but I randomly see errors where the request does not complete. The error messages I've seen are:

Authentication failed because the remote party has closed the transport stream.

The SSL connection could not be established

Response ended prematurely

A task was cancelled

The connection was closed by the remote host

These are sometimes nested as InnerException messages. When I run a curl script every 10 seconds all day long, I can't reproduce the problem. However, I made a do.net core app on the same server and I was able to see this error happen by firing many requests at the remote API. So the error is somehow related to the do.net core method of calling this remote API.

I am using the do.net core HttpClient like so:

var response = await _HttpClient.SendAsync(requestMessage).ConfigureAwait(false);

I've tried every TLS version, I'm using app.UseHttpsRedirection(); in Startup, but I'm not sure what else I can do to find the source of this problem. I'm adding retry logic to prevent total failure, but I'd really like to know what is happening in the code to make this fail randomly.

Answer 1

I found the problem. My HttpClient was experiencing "socket exhaustion" under heavy loads .

The solution for me was to make the HttpClient a singleton and reuse it

public class HttpClientSingleton
{
    private static HttpClient client;

    private HttpClientSingleton() { }

    public static HttpClient Client
    {
        get
        {
            if (client == null)
            {
                client = new HttpClient();
            }
            return client;
        }
    }
}