在自动化Java性能测试期间监视和记录系统指标的工具

Question

We have an application using Spring Integration in its core, and have created performance tests to see what is the processing speed (msgs/sec) for different generated input types.

This process is automated, so whenever such test is run, a separate instance is created in cloud, and disposed after done & output artefacts copied.

What I want to do is to have those performance tests monitored during the run for basic system metrics -- CPU, memory, I/O, GC runs/time. Obviously, the result of this should be some CSV files with metrics readings (eg, once or twice a second).

So my question is: Are there any good and configurable tools for these purposes?

I'm in the middle of investigation, but profiling tools I reviewed so far mainly require human interaction and are UI oriented.

An option I'm considering is writing a separate tool to access MXBean & use it to log such data during the performance tests. Just wondering if anything good is around.

Please note that this application is running in Tomcat, however for the performance tests we are only using Spring Integration's File endpoints.

Please also note, that 'switchable' component within application is also possible solution. However, I'm currently looking for application-agnostic external tools-first solution.

Answer 1

Command-line tools come to help for this kind of a scenario:

On a Linux/Solaris based environment:

1. Before you run/trigger your JVM for Spring based application, you can run tools like vmstat , sar in a background mode with its output redirected to a flat file - which helps capture CPU, Memory and other such statistics. Use top with options or mpstat to get thread-level statistics if you seem to be hitting a performance problem, to do bottleneck analysis.
2. Now run the JVM with arguments like printgc , -Xloggc to write JVM verbose output to a flat file or print gc statictics. Look under Debugging options section for JVM arguments in Java HotSpot VM Options for more options you need.

Tip: create a shell script combining both commands above to run at the same time and achieve your requirement.

On a Windows environment:

1. For OS statistics gathering on commandline, you could use typeperf or tracerpt (CSV supported).
2. Now run the JVM with arguments like printgc , -Xloggc to write JVM verbose output to a flat file or print gc statictics. Look under Debugging options section for JVM arguments in Java HotSpot VM Options for more options you need.

Answer 2

Jmeter is a tool to develop performance and scalability tests (defining http requests and being able to load the server with them) , but also has a plugin to allow for monitoring a target system for system metrics such as CPU utilization, memory utlization etc and also JMX type statistics:

Available JMX metric types:

gc-time - time spent in garbage collection, milliseconds (used method) memory-usage - heap memory used by VM, bytes (method used) memory-committed - heap memory committed by VM, bytes (method used) memorypool-usage - heap memory pool usage, bytes (method used) memorypool-committed - heap memory pool committed size, bytes (method used) class-count - loaded class count in VM (used method) compile-time - time spent in compilation, milliseconds (used method)"

Check http://jmeter-plugins.org/wiki/PerfMonMetrics/ for more details of this plugin.