带图形输出的轻量级微基准库（Java）

Question

Is there a good Java library for taking the legwork out of writing good micro-benchmarks? I'm thinking something which can provide (with a minimum of hassle) provide text (CSV or HTML, take your pick) output of results and maybe graphs summarizing results. Ideally, it should be something that plays nicely with JUnit or equivalent, and should be simple to configure benchmarks with variable parameters.

I've looked at japex, but found it too heavyweight ( 25 MB of libraries to include?!) and frankly it was just a pain to work with. Virtually nonexistent documentation, mucking about with ant, XML, and paths... etc.

Answer 1

A few of us from the Google Collections team are in the early days of building something that satisfies your needs. Here's the code to measure how long foo() takes:

public class Benchmark1 extends SimpleBenchmark {
  public void timeFoo(int reps) {
    for (int i = 0; i < reps; i++) {
      foo();
    }
  }
}

Neither the API nor the tool itself is particularly stable. We aren't even ready to receive bug reports or feature requests! If I haven't scared you off yet, I invite you to take Caliper for a spin.

Answer 2

Oracle now has JMH. Not only is it written by members of the JIT team (who will take out much of the legwork of writing good micro-benchmarks), but it also has other neat features like pluggable profilers (including those that will print assembly of your hotspots with per-line cpu time).

It prints out tables. Not sure about graphs. The benchmarks can be configured with variable parameters. The documentation is fairly good.

It is easy to set up and get going. I've got it integrated with JUnit, but the developers provide a Maven archetype to get started.