JMAP dump size is 4.5 GB. Eclipse MAT is showing total heap of 415 MB, how do i analyze the remaining dump?

Question

I am running a web application using Tomcat 6. We are using open JDK 6. Ours is AMD 64bit Ubuntu 11.04 server.

Memory dump is taken by this command

jmap -dump:format=b,file=/home/demon/Desktop/mymemorydump.hprof 2762

Memory dump is of 4.5 GB. When we were analyzing it using eclipse MAT, Strong Reachable objects were 80 MB, Unreachable objects were 335 MB.. How to analyze the rest of the dump..

Also memory taken on my server by this process is 4.5 GB. Now this memory never goes down to normal level of 1 GB. And this memory slowly increases to 8-9 GB, then our tomcat stops responding(Actully there is no out of memroy error).

My tomcat config is

/usr/lib/jvm/java-6-openjdk/bin/java -Djava.util.logging.config.file=/var/lib/tomcat6/conf/logging.properties -Djava.awt.headless=true -Xms1024m -Xmx6144m -XX:+UseConcMarkSweepGC -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/share/tomcat6/endorsed -classpath /usr/share/tomcat6/bin/bootstrap.jar -Dcatalina.base=/var/lib/tomcat6 -Dcatalina.home=/usr/share/tomcat6 -Djava.io.tmpdir=/tmp/tomcat6-tmp org.apache.catalina.startup.Bootstrap start

Answer 1

You may have a Thread leak, having created an excessive number of threads which never terminate. Each thread allocates stack space, which isn't shown above. Pressing Ctrl-\\ (unix) (or Ctrl-Break in windows) will dump a stack trace of every thread (you may need to redirect tomcat's STDERR to avoid it scrolling off your terminal). If you didn't start tomcat from the terminal, then use "kill -QUIT YourPID" rather than Ctrl-.

Also, jconsole (in the jdk bin directory) can attach to your executable, before it stops responding. It might be able to show you a leak of threads (or heap) before things get out of hand.