JMeter: better performance with Java impl of HTTP Sampler than with HTTPClient impl

Question

I have been using JMeter for a few years now and have been using the Apache HTTPClient sampler implementation as it seems to be the recommended implementation, and the Apache HTTPClient is a mature component. Recently, however (and quite by accident), I have discovered that the Java implementation seems to perform and scale better.

I am testing against a java server application developed in-house. Switching JMeter from using the HTTPClient4 implementation of the HTTP Sampler to the Java implementation yields upwards of 400%-450% higher throughput. I have tested versions 2.7 and 2.9 (in 2.9 HTTPClient4 is now the default implementation for the HTTP Sampler). I am using a fairly beefy machine to host JMeter (Dual Intel(R) Xeon(R) E5-2640 0 @ 2.50GHz, 6 core, hyper-threaded) running Red Hat Enterprise Linux (2.6.32-358.el6.x86_64) and an identical machine to host the server application being tested.

I run a series of tests scaling virtual user load from 1 to 300 in the steps: 1, 10, 25, 50, 100, 150, 200, 250 and 300. Each test is discrete, not a continuous ramp. When using the HTTPClient sampler, watching top, I can see that JMeter is using between 700%-1100%, regardless of the virtual user load (except single user of course). With the Java implementation I see higher CPU utilization (upwards of 1600%). It is important to note the tests are identical in that I change ONLY the HTTP implementation in the jmx file from test to test, and nothing else. Server under test is restarted test to test as well to make all things fair.

Monitoring the server under test I can see that it's CPU utilization is also higher with JMeter using the Java HTTP Sampler over the HTTPClient4 implementation. Suggesting that the Java impl is sending more requests concurrently than the HTTPClient4. At higher user loads, and with the HTTPClient4 impl, the server under test runs at 100% CPU utilization at most. Suggesting that it is only using one cpu/core/hardware thread at a time. Perhaps it is important to note that the response times of the requests are very fast. 90th percentile response time is in the 2-25 millisecond range.

I've researched on google, here and other sites to determine why (I may take a shot at profiling JMeter if time permits). My initial theory is that Java impl is more efficient at reusing connections than the HTTPClient? Or that the feature richness HTTPClient comes at a performance penalty? I guess my question is two fold: 1) Anyone else experience this? 2) Anyone have any theories or explanation as to why Java impl appears to perform so much better?

Thanks!

Answer 1

You opened a bug for it:

• https://issues.apache.org/bugzilla/show_bug.cgi?id=55023

Fix is available in version >= 2.10.