Unable to span more than a specific number of threads in jmeter

Question

I am trying to run a jmeter load test remotely with a master windows box and four slave linux boxes. In this configuration, I am able to spawn only a maximum of 983 threads per slave box, after which I see exceptions thrown saying that it was unable create a new thread due to a out of memory exception. My hardware configurations are decently high with 64 cores and 64 GB RAM. So hardware should not be a bottleneck. The setting I am using for start heap size is 3072m. In most forums, it is advised to reduce this number which has a default of 512k. But if I keep in any lower, while starting jmeter-server in the slave box, I am prompted that my initial heap size is too low.

The other configuration that I used was one of the linux box as a standalone server in which case I was able to spawn around 10k threads without any issues.

I have already increased my ulimit to 32768.

Is there a way to get spawn more threads using the master slave configuration or should I totally drop it and run the script individually on all four boxes in parallel and compile the data. If so, can you give me some pointers for the second approach?

Answer 1

• 512K is for Xss not Xmx, you can keep Xmx to what you have set it.
• Use Java 64 Bit JVMs not 32 bit, and use most up to date java version
• You can also run many JMeter instances per box with the power you have if you find issues
• Ensure you use last jmeter version

And read this:

• http://www.dzone.com/links/see_how_to_make_jmeter_run_thousands_of_threads_w.html

Answer 2

Following are some key findings that I made. These numbers are proportional to how big my machine was. You can check that using:

cat /proc/meminfo

• Changed the ulimit setting using:

  ulimit -n 32768

• It makes a difference if you are root or not. I am not sure if you have to be root or just a sudo user. But logging in as root did somehow allow me to spawn more threads.

• Changed some sysctl settings.

  /sbin/sysctl net.core.wmem_max=8388608

  /sbin/sysctl net.core.rmem_max=8388608

  /sbin/sysctl net.ipv4.tcp_rmem="4096 87380 8388608"

  /sbin/sysctl net.ipv4.tcp_wmem="4096 87380 8388608"

  /sbin/sysctl net.ipv4.tcp_tw_reuse=0

  /sbin/sysctl net.ipv4.tcp_tw_recycle=0

  /sbin/sysctl net.ipv4.tcp_keepalive_time=1800

  /sbin/sysctl net.ipv4.tcp_max_syn_backlog=4096