Performance comparison of a Java program between Linux and Windows

Question

I ran a piece of Java code (measuring the time taken to parse and validate xml files wrt a schema) on both Windows 7 and Linux (Debian). Now, as it should the time taken on the Linux platform must be very less as compared to the Windows platform. But the user time recorded in both cases is the same.

I am in a fix because I also ran a simple benchmarking Java program (which used two nested loops running for some 900000 times) on both the platforms. And the user time taken on the Linux platform is around 60 times more than on Windows.

I must also mention that I am using the thaiopensource API in the Java program. (the jing/trang project). My system uses 32 bit Linux (Debian) (Java is also 32 bit) and has a 3 GB RAM. On Linux, I use openJDK 1.5, with IcedTea Virtual Machine. My IDE on Windows is Eclipse Helios.

Kindly suggest me if there is some way to get around this ie the time taken to execute the parsing and validating program on Linux must be less wrt Windows. But it is not. Kindly suggest a way. Thank you.

Answer 1

For a fair comparison, you need to minimise the variation between tests. ie you must use the same version of Java (ideally the latest or at least something current) If you are not making system calls, the OS used makes no difference, it is likely to be something else which is different.

I would suggest you use the latest version of Java 7 on both machine with the -server option. (Windows 32-bit defaults to the client JVM which is different)

I would also ensure your test runs for at least 10 seconds to ensure you have a meaningful comparison.

BTW: There is no particular reason Linux has to be faster is you are using the same hardware.

Answer 2

Writing such microbenchmark tests are harder you may think at first.

For a start, I would recommend reading the Robust Java benchmarking series at IBM developer works:

• See a description of some issues you want to avoid, and
• some statistical solutions you can use to extract some kind of information (it is not always clear and trivial to conclude the results, see the latter article I linked for details).

Also take a look on Brent Boyer's site for further information (and to download the library you can use for benchmarking).