Heap dump != virtual memory?

Question

Not really being knowledgeable about Java and especially about debugging in Java, but taking a heap dump in Jenkins using Monitoring and then decoding it in Eclipse with MAT shows total memory used 169.4 MB, while in Jenkins monitoring the memory seems to be constantly used a lot and GCs are running frequently. -XmX is 4G.

How come I only get 169.4 MB with MAT? Could it be because before making a dump Jenkins executes a GC? If so, can I avoid it to see the full memory dump?

Answer 1

Understanding memory

Yes, a Java heap dump and a virtual memory dump (called "crash dump" or "memory dump" on Windows) are different things.

The Java heap dump only contains the memory relevant for Java, ie the place where Java objects reside. A Java heap dump is analyzed with tools like MAT (as mentioned by you) or the Java Heap Analysis Tool

A Windows crash dump of a (user mode) process contains all virtual memory, where virtual memory is the term of the operating system providing the memory. On Windows, that's all memory allocated via VirtualAlloc .

The OS virtual memory will include the Java heap, since Java can only request memory from the operating system.

So, when comparing memory sizes, it's important to understand whether the tool is Java specific or OS general.

In your case, Monitoring looks much like a generic tool, since it deals with a process list and CPU times, nothing which seems to be Java specific. On the other side, MAT is clearly a Java tool from its description.

Memory differences

So how much can the Java heap size differ from the virtual memory size?

Much:

1. Loaded EXEs/DLLs account to the virtual memory, but not to the Java heap
2. Memory used by native code (eg via JNI) accounts to virtual memory, but not to the Java heap
3. Memory requested by Java from the OS, but not used by Java yet, definitely accounts as virtual memory, but could be reported as "free" from Java perspective.

Answer 2

Apparently, the tools that collect heap dump execute GC to reduce the size of the dump. Since things that can be GCed shouldn't produce the OOM this is intended rather to find memory leaks than troubleshoot memory usage.

Answer 3

VM requests virtual memory to store variety of data. VM then allocates some memory to store variables (that is heap), native code (not heap), reserves some memory that is not used yet. As a result virtual memory is strictly larger than heap. There isn't any clear correlation between heap and virtual memory given that you can write a simple infinite recursion (heap will be almost as large as virtual memory) or load a large dll in a hello world program (heap is much smaller than virtual memory).