java server with jetty memory leak issue

Question

I´m running this application with a Jetty server embedded. The application is slowly consuming more memory dispite the fact that the heap size does not change at all.

this is the command to start the application :

java -server -Xms1G -Xmx1G -Dfile.encoding=UTF-8

This is the chart from new relic with the RAM usage increase it started using 1140MB and after 8 hours finished with 1290MB

This is the jvisualvm chart of the heap.

Any suggestions ?

[EDIT1] add dump https://www.dropbox.com/s/1gt1i9dhjtjauf0/gameserver-20160226-2107.zip?dl=0

[EDIT2] Here are some notes from my current investigation

= How Monitor memory outside the heap add parameters

-XX:NativeMemoryTracking=detail -XX:+UnlockDiagnosticVMOptions -XX:+PrintNMTStatistics

run command jcmd 3322 VM.native_memory summary @see https://devcenter.heroku.com/articles/java-memory-issues

= Checking the default thread stack size java -XX:+PrintFlagsFinal -version | grep ThreadStackSize

intx ThreadStackSize = 1024
How to reclaim the memory used by a Java thread stack?

= Understand the max memory Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss] @see https://plumbr.eu/blog/memory-leaks/why-does-my-java-process-consume-more-memory-than-xmx

= Actions Reduce the size of the thread stacks. (In theory, you can go as low as 64K ...)

-XX:ThreadStackSize=256

Reduce the Minimum percentage of heap free after GC to avoid expansion.

-XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=10

Now will continue monitoring ...

Answer 1

After lots of different types of investigations, I came back to the Eclipse Memory analyzer and tried once more ...

But this time I decided to trust the leak report that was saying :

One instance of "org.hibernate.internal.SessionFactoryImpl" loaded by "sun.misc.Launcher$AppClassLoader @ 0xc001d4f8" occupies 2.956.808 (21,05%) bytes. The memory is accumulated in one instance of "org.hibernate.internal.SessionFactoryImpl" loaded by "sun.misc.Launcher$AppClassLoader @ 0xc001d4f8".

Then I shift my investigation the my DAO implementation, expecting that I forgot to close one or other EntityManager call. That wasn´t the case, all of then were with a nice close() method call after use.

Then I realize that the problem could be on hibernate itself as the problematic object were a SessionFactory Impl, so I changed my DAO implementation to clear Hibernate first level cache every time I create an Entity Manager, as I wasn´t able to find a way to disable it.

End of the day result = IT WORKED !! :) the memory changes a litle and come back after some minutes, but don´t grow in a crazy way anymore (1000MB in 24hours).

Here is the code I changed, hope it helps someone.

public EntityManager getEntityManager(){
    if( emf == null ){
        if (parameters == null) {
            emf = Persistence.createEntityManagerFactory(persistenceUnitName);
        } else {
            emf = Persistence.createEntityManagerFactory(persistenceUnitName, parameters);
        }
    } else {
        emf.getCache().evictAll();
    }

    EntityManager em = emf.createEntityManager();
    return em;
}

The key is here : emf.getCache().evictAll();