为什么可以使用`HttpClient`发出并发HTTP请求的阈值？

Question

我注意到存在可以使用.NET core的HttpClient进行的并发HTTP请求的某种阈值，即，当我有<= 1,000个请求时，它似乎可以正常工作，但是接近10,000个是有问题的。 这是相关的失败代码：

using System;
using System.Net.Http;
using System.Threading.Tasks;

namespace RequestsGalore
{
    class Program
    {
        static HttpClient Client { get; set; } = new HttpClient();

        static void Main(string[] args)
        {
            var url = "http://example.com";

            int requests = 10_000;
            var tasks = new Task<HttpResponseMessage>[requests];

            for (int i = 0; i < requests; i++)
            {
                tasks[i] = Client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
            }

            Task.WaitAll(tasks);

            for (int i = 0; i < requests; i++)
            {
                Console.WriteLine(tasks[i].Result.StatusCode);
            }
        }
    }
}

，但例外：

Unhandled Exception: System.AggregateException: One or more errors occurred.
(An error occurred while sending the request.)
(A task was canceled.)
.
. [MANY OF THE ABOVE TWO MESSAGES]
.
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException: Connection reset by peer
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.WaitAllCore(Task[] tasks, Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at RequestsGalore.Program.Main(String[] args) in /home/[REDACTED]/Downloads/RequestsGalore/Program.cs:line 27

以及有关我的机器的一些信息：

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.401
 Commit:    729b316c13

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  19.04
 OS Platform: Linux
 RID:         ubuntu.19.04-x64
 Base Path:   /usr/share/dotnet/sdk/2.2.401/

Host (useful for support):
  Version: 2.2.6
  Commit:  7dac9b1b51

.NET Core SDKs installed:
  2.2.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Answer 1

“对等连接重置”指向另一端，而不是您的代码，从而删除了连接。

Answer 2

此代码打开了与目标Web服务器的太多并发连接，可能触发了反拒绝服务保护。 有一种简单的方法可以将并发请求限制到.NET Framework ServicePoint中的任何目标，并且存在默认限制。

在.NET Core中，未使用ServicePoint。 然后使用HttpClientHandler设置限制：

        var url = "http://example.com";
        HttpClientHandler handler = new HttpClientHandler();
        handler.MaxConnectionsPerServer = 10;
        Client = new HttpClient(handler);