dotnet 核心 2.1 httpclient System.Net.Sockets.SocketException

Question

since 2 weeks by trying connect our backendsystem some times we get the following exception:

System.Net.Http.HttpRequestException: Unknown error -1 ---> System.Net.Sockets.SocketException: Unknown error -1 at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Threading.Tasks.ValueTask`1.get_Result()

But at this time, I can call the API Server via Postman. My Code calling the API Server is Like:

using (var client = this.ClientFactory(serviceUserName, serviceUserPassword))
{
    try
    {
        using (var response = await client.SendAsync(request, System.Net.Http.HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false))
        {
            return checkAction(response);
        }
    }
    catch (HttpRequestException ex)
    {
        this.logger.LogError(ex, "http error occured while checking availability for " + url);
        return false;
    }
    catch (TaskCanceledException ex)
    {
        this.logger.LogError(ex, url + " timed out");
        return false;
    }
    catch (Exception ex)
    {
        this.logger.LogError(ex, url + " general exception!");
        return false;
    }
}

Where ClientFactory() function looks like

private HttpClient ClientFactory(string serviceUserName, string serviceUserPassword)
{
    var client = new HttpClient();
    client.Timeout = TimeSpan.FromSeconds(this.clientTimeoutSeconds);
    if (!string.IsNullOrEmpty(serviceUserName) && !string.IsNullOrEmpty(serviceUserPassword))
    {
        var basicAuthToken = Convert.ToBase64String(Encoding.ASCII.GetBytes(serviceUserName + ":" + serviceUserPassword));
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", basicAuthToken);
    }

    return client;
}

I have no idea how I could debugg this problem. The application is on a Cloud Foundry server running on kestrel web server.

UPDATE

            var client = this.ClientFactory(serviceUserName, serviceUserPassword);
        try
        {
            var response = await client.SendAsync(request, System.Net.Http.HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);
            return checkAction(response);
        }
        catch (HttpRequestException ex)
        {
            this.logger.LogError(ex, "http error occured while checking availability for " + url);
            return false;
        }
        catch (TaskCanceledException ex)
        {
            this.logger.LogError(ex, "availability check for " + url + " timed out");
            return false;
        }
        catch (Exception ex)
        {
            this.logger.LogError(ex, "availability check for " + url + " general exception!");
            return false;
        }

Where ClientFactory() function now looks looks like

        private HttpClient ClientFactory(string serviceUserName, string serviceUserPassword)
    {
        var client = this.clientFactory.CreateClient();
        client.Timeout = TimeSpan.FromSeconds(this.clientTimeoutSeconds);
        if (!string.IsNullOrEmpty(serviceUserName) && !string.IsNullOrEmpty(serviceUserPassword))
        {
            var basicAuthToken = Convert.ToBase64String(Encoding.ASCII.GetBytes(serviceUserName + ":" + serviceUserPassword));
            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", basicAuthToken);
        }

        return client;
    }

Now I use the IHttpClientFactory injected by the constructor:

        public MyConstructor(IHttpClientFactory clientFactory)
    {
        this.clientFactory = clientFactory;
    }

But it dosen't help I still get the Exception above and other exception like:

System.Net.Http.HttpRequestException: Too many open files in system --- System.Net.Sockets.SocketException: Too many open files in system at System.Net.Sockets.Socket..ctor(AddressFamily addressFamily, SocketType socketType, ProtocolType protocolType) at System.Net.Sockets.DualSocketMultipleConnectAsync..ctor(SocketType socketType, ProtocolType protocolType) at System.Net.Sockets.Socket.ConnectAsync(SocketType socketType, ProtocolType protocolType, SocketAsyncEventArgs e) at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

So my question again, how can I debug the problem? It occurrences only on the prodcution and after few hours or days...

Answer 1

You should NOT create a new instance of HttpClient all the time. You should reuse it. If you dont reuse it you can run into socket exhaustion (and a bunch of other problems).

https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/

Since you are using dotnet core, have a look at HttpClientFactory https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests

So a quick fix for you would be to create a new static HttpClient and not dispose it. But then you can get DNS issues, so don't do that . https://github.com/dotnet/corefx/issues/11224

So: Use HttpClientFactory, Microsoft has solved all the problems mentioned above :)