[英]“Connection reset” exception with Apache HttpClient 4.5 on Mac
I need to get up to 1000 threads to send HTTP request simultaneously. 我需要最多1000个线程来同时发送HTTP请求。 Yet it seems on Mac I am hitting some limit, which causes Apache HTTPClient to return exception:
但是在Mac上似乎我达到了一些限制,这导致Apache HTTPClient返回异常:
INFO: I/O exception (java.net.SocketException) caught when processing request to {}->http://my.url:80: Connection reset
The limit appears to be somewhere between 200 and 300 concurrent connections. 限制似乎在200到300个并发连接之间。
I created a simple app and able to reproduce the issue even with that app: 我创建了一个简单的应用程序,即使使用该应用程序也能够重现该问题:
import java.io.IOException;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
public class App implements Runnable
{
private static final int NUMBER_THREADS = 1000;
public static void main( String[] args )
{
Thread[] threads = new Thread[NUMBER_THREADS];
for(int i = 0; i < NUMBER_THREADS; i++)
{
threads[i] = new Thread(new App());
}
for(int i = 0; i < NUMBER_THREADS; i++)
{
threads[i].start();
}
}
public void run()
{
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet("http://my.url/test.html");
CloseableHttpResponse response = null;
try
{
System.out.println( System.nanoTime() + " " + Thread.currentThread().getName() + " started" );
response = httpclient.execute(httpget);
System.out.println( System.nanoTime() + " " + Thread.currentThread().getName() + " finished" );
}
catch (ClientProtocolException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
try
{
if(response != null)
{
response.close();
}
}
catch (IOException e)
{
e.printStackTrace();
}
}
}
}
I updated some Mac settings, but that didn't help anything: 我更新了一些Mac设置,但这无济于事:
> sysctl -a | grep files
kern.maxfiles = 12288
kern.maxfilesperproc = 10240
kern.maxfiles: 12288
kern.maxfilesperproc: 10240
kern.num_files: 3202
> ulimit -n 10240
> ulimit -n
10240
> sysctl -a | grep somax
kern.ipc.somaxconn: 128
> sudo sysctl -w kern.ipc.somaxconn=2048
Password:
kern.ipc.somaxconn: 128 -> 2048
> sysctl -a | grep somax
kern.ipc.somaxconn: 2048
So I want to understand where this limit is coming from? 所以我想了解这个限制来自何处? OsX / Apache HttpClient itself?
OsX / Apache HttpClient本身? And is it possible to control it?
并且可以控制它吗?
Thanks in advance. 提前致谢。
The problem was indeed with maxfiles, but I was not changing them right. 问题确实存在于maxfiles中,但是我没有正确更改它们。 This article helped me to solve it by showing the correct procedure for updating file limits on Mac OS X Yosemite / El Capitan.
此文章帮助我通过显示更新在Mac OS X优胜美地/埃尔卡皮坦文件限制的正确程序来解决它。
Create and update /Library/LaunchDaemons/limit.maxfiles.plist file: 创建和更新/Library/LaunchDaemons/limit.maxfiles.plist文件:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>limit.maxfiles</string> <key>ProgramArguments</key> <array> <string>launchctl</string> <string>limit</string> <string>maxfiles</string> <string>65536</string> <string>65536</string> </array> <key>RunAtLoad</key> <true/> <key>ServiceIPC</key> <false/> </dict> </plist>
Create /Library/LaunchDaemons/limit.maxproc.plist file: 创建/Library/LaunchDaemons/limit.maxproc.plist文件:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple/DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>limit.maxproc</string> <key>ProgramArguments</key> <array> <string>launchctl</string> <string>limit</string> <string>maxproc</string> <string>2048</string> <string>2048</string> </array> <key>RunAtLoad</key> <true /> <key>ServiceIPC</key> <false /> </dict>
Add .bashprofile with the following lines: 添加.bashprofile并包含以下几行:
ulimit -n 65536 ulimit -u 2048
Restart your system 重新启动系统
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.