為什么可以使用`HttpClient`發出並發HTTP請求的閾值？

Question

我注意到存在可以使用.NET core的HttpClient進行的並發HTTP請求的某種閾值，即，當我有<= 1,000個請求時，它似乎可以正常工作，但是接近10,000個是有問題的。 這是相關的失敗代碼：

using System;
using System.Net.Http;
using System.Threading.Tasks;

namespace RequestsGalore
{
    class Program
    {
        static HttpClient Client { get; set; } = new HttpClient();

        static void Main(string[] args)
        {
            var url = "http://example.com";

            int requests = 10_000;
            var tasks = new Task<HttpResponseMessage>[requests];

            for (int i = 0; i < requests; i++)
            {
                tasks[i] = Client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
            }

            Task.WaitAll(tasks);

            for (int i = 0; i < requests; i++)
            {
                Console.WriteLine(tasks[i].Result.StatusCode);
            }
        }
    }
}

，但例外：

Unhandled Exception: System.AggregateException: One or more errors occurred.
(An error occurred while sending the request.)
(A task was canceled.)
.
. [MANY OF THE ABOVE TWO MESSAGES]
.
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException: Connection reset by peer
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.WaitAllCore(Task[] tasks, Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at RequestsGalore.Program.Main(String[] args) in /home/[REDACTED]/Downloads/RequestsGalore/Program.cs:line 27

以及有關我的機器的一些信息：

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.401
 Commit:    729b316c13

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  19.04
 OS Platform: Linux
 RID:         ubuntu.19.04-x64
 Base Path:   /usr/share/dotnet/sdk/2.2.401/

Host (useful for support):
  Version: 2.2.6
  Commit:  7dac9b1b51

.NET Core SDKs installed:
  2.2.401 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

Answer 1

“對等連接重置”指向另一端，而不是您的代碼，從而刪除了連接。

Answer 2

此代碼打開了與目標Web服務器的太多並發連接，可能觸發了反拒絕服務保護。 有一種簡單的方法可以將並發請求限制到.NET Framework ServicePoint中的任何目標，並且存在默認限制。

在.NET Core中，未使用ServicePoint。 然后使用HttpClientHandler設置限制：

        var url = "http://example.com";
        HttpClientHandler handler = new HttpClientHandler();
        handler.MaxConnectionsPerServer = 10;
        Client = new HttpClient(handler);