Java VisualVM giving bizarre results for CPU profiling - Has anyone else run into this?

Question

I have written this small (and brutally inefficient) class and wanted to profile it using the Java VisualVM.

public class Test {

    public static void main(String[] args) throws IOException {
        BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
        br.readLine();
        int n = Integer.parseInt(args[0]);
        int fib = fib(n);
        System.out.println(fib);
    }

    private static int fib(int n) {
        if (n < 2) {
            return n;
        }
        return fib(n-1)+fib(n-2);
    }
}

The results are bizarre. The results are completely dominated by calls to ConnectionHandler.run().

(98.2%) sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run()
(1.7%) java.lang.Thread.join(long)
(0%) java.lang.String.equals(Object)
etc...

There are probably around one hundred methods profiled and not one of them is fib(int)!

It is inconceivable that my program is actually spending all of its time in these methods. They appear to be the profiler connecting to my jvm and doing its thing.

What am I doing wrong?

Edited for clarity: If you pass in 45 for the n this application runs for 20 well profiled seconds. The program I was profiling originally (not a fibonacci calculator) pegs all four cores on my cpu at 100% and I was doing profiling runs lasting up to 5 minutes. These had the same results and methods from my application did not appear high up on the hot spot method list.

It varies from run to run but ConnectionHandler.run() is always at the top and usually accounts for ~99% of the profile time.

Second Edit: I have tried using the sampler and I am now getting results that are consistent with what JProfiler is producing. The downside to this is that I don't get the stack trace information that comes with profiling. But for my immediate needs this is excellent.

Something I have discovered while playing around is that VisualVM counts wall-clock time for method calls while profiling them.

In my specific case my application has a main thread which launches worker threads and immediately blocks waiting for a message on a queue.

This means that the blocking method will appear to take up almost all of the time on the profiler despite the fact that it is not this method that is eating up my CPU.

I would expect that the same is true of the sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run() method which does its job nicely - but when it terminates it becomes one of the longest running methods in my application - repeatedly .

Answer 1

I don't think that's inconceivable at all. You have an application where the "payload" is fairly minuscule (though that of course depends on the value of n ), and you have to accept that the extra effort required (to connect the profiler and shift all the information across to it) will swamp that payload.

This is not the sort of application I would be profiling in the first place since it's pretty obvious that the vast amount of time would be spent in fib anyway (for non-trivial values of n ), marking that as an obvious target for optimisation.

I would be more inclined to use the profiler for more substantial applications where:

• it's not obvious where the optimisation effort should go; and
• there is some substantial amount of work to do in the payload.

If you really want to test that code, you probably need to bump up its effect by (for example) replacing:

int fib = fib(n);

with:

for (int i = 0; i < 100000; i++) {
    int fib = fib(n);
)

---

I'll tell you one thing to watch out for though. I don't know the internals of any particular JVM but using a recursive method where the reduction of the argument is slow is usually a bad idea, one that leads to stack space being exhausted pretty quickly.

By that, I mean a binary search is a good candidate since it removes half the remaining search space with each recursion level (so that a search space of a billion items is only 30 levels).

On the other hand, using recursion for a Fibonacci sequence on the number 1,000,000,000 would take about a billion levels and most stacks would have a hard time containing that.

Tail end recursion optimisation may avoid that problem but you need to be careful in case that optimisation isn't done.

Answer 2

Based on Ron's answer was able to improve the result by stopping the JVM right after the start, then activated the profiler, and finally after that continue the excetion (by pressing enter). It's crude.

class Foobar {
    /* First line in Class */
      static {
        try {
            System.in.read();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
    /* .. */
    public static void main(..) {
        doMagic()
    }
}        

Answer 3

jvisualvm profiling probably weaves bytecode into classes as they are loaded. Since your program has just one class and it is already initialized by the time jvisualvm arrives on the scene, I would suppose it cannot be instrumented.

Move your fib method into another class and try profiling again. You might add a jvm option "-verbose:class" to double check that the class isn't loaded before you enable cpu profiling in jvisualvm.

Edit: Thanks JB for the comment. Forget my classloading hogwash. My intuition is that the fib method is too tightly coupled to the main method, so it is effectively bytecode currently being executed.

Answer 4

My guess is that you are passing too small a value to fib, and the program just doesn't run long enough to register. To get meaningful data for nearly anything when profiling (or bench-marking) you generally need an elapsed time of at least a few seconds.