[英]Why is there a threshold of concurrent HTTP requests I can make using `HttpClient`?
I noticed that there exists some kind of threshold for concurrent HTTP requests I can making using .NET core's HttpClient
ie it seems to work fine when I have <= 1,000 requests, but nearing 10,000 is problematic. 我注意到存在可以使用.NET core的HttpClient
进行的并发HTTP请求的某种阈值,即,当我有<= 1,000个请求时,它似乎可以正常工作,但是接近10,000个是有问题的。 Here is the relevant failing code: 这是相关的失败代码:
using System;
using System.Net.Http;
using System.Threading.Tasks;
namespace RequestsGalore
{
class Program
{
static HttpClient Client { get; set; } = new HttpClient();
static void Main(string[] args)
{
var url = "http://example.com";
int requests = 10_000;
var tasks = new Task<HttpResponseMessage>[requests];
for (int i = 0; i < requests; i++)
{
tasks[i] = Client.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
}
Task.WaitAll(tasks);
for (int i = 0; i < requests; i++)
{
Console.WriteLine(tasks[i].Result.StatusCode);
}
}
}
}
, and the exception: ,但例外:
Unhandled Exception: System.AggregateException: One or more errors occurred.
(An error occurred while sending the request.)
(A task was canceled.)
.
. [MANY OF THE ABOVE TWO MESSAGES]
.
---> System.Net.Http.HttpRequestException: An error occurred while sending the request.
---> System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
---> System.Net.Sockets.SocketException: Connection reset by peer
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.WaitAllCore(Task[] tasks, Int32 millisecondsTimeout, CancellationToken cancellationToken)
at RequestsGalore.Program.Main(String[] args) in /home/[REDACTED]/Downloads/RequestsGalore/Program.cs:line 27
And some information about my machine: 以及有关我的机器的一些信息:
$ dotnet --info
.NET Core SDK (reflecting any global.json):
Version: 2.2.401
Commit: 729b316c13
Runtime Environment:
OS Name: ubuntu
OS Version: 19.04
OS Platform: Linux
RID: ubuntu.19.04-x64
Base Path: /usr/share/dotnet/sdk/2.2.401/
Host (useful for support):
Version: 2.2.6
Commit: 7dac9b1b51
.NET Core SDKs installed:
2.2.401 [/usr/share/dotnet/sdk]
.NET Core runtimes installed:
Microsoft.AspNetCore.All 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.2.6 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
To install additional .NET Core runtimes or SDKs:
https://aka.ms/dotnet-download
“对等连接重置”指向另一端,而不是您的代码,从而删除了连接。
This code opens too many concurrent connections to the target web server, probably triggering anti-denial-of-service protections. 此代码打开了与目标Web服务器的太多并发连接,可能触发了反拒绝服务保护。 There's a simple way to limit the concurrent requests to any target in .NET Framework the ServicePoint , and there are default limits in place. 有一种简单的方法可以将并发请求限制到.NET Framework ServicePoint中的任何目标,并且存在默认限制。
In .NET Core ServicePoint is not used. 在.NET Core中,未使用ServicePoint。 And you set the limit using the HttpClientHandler : 然后使用HttpClientHandler设置限制:
var url = "http://example.com";
HttpClientHandler handler = new HttpClientHandler();
handler.MaxConnectionsPerServer = 10;
Client = new HttpClient(handler);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.