Neo4j GC overhead limit exceeded

Question

I was running a test and I got a "GC overhead limit exceeded" error. I understood that it is because I loaded too many primitives in the cache. Am I wrong?

My question then, is how can we prevent ourselves from this? For example, can we evaluate the size of needed memory based on the number of primitives? Is there like a tip to approximatively know about it?

My boss want to know how many primitives can we manage at the same time. I assume it is related to JVM settings but can't manage to find which settings.

Sorry if it is dumb questions, I'm not used to JVM settings and peformance and I have a pretty huge lack of knowledge about it atm. Trying and willing to understand though!

Jimmy.

Answer 1

Understanding the details of Java garbage collections is by far not a trivial thing. Since you're question is rather unspecific, I can just provide a rather unspecific answer as well. There's a section in the Neo4j reference manual on JVM settings, http://docs.neo4j.org/chunked/stable/configuration-jvm.html .

Another idea depending on your graph and heap size is to change the implementation type for the object cache. There is a soft (default in community edition), weak and strong cache implementation. Additionally the enterprise edition comes with hpc (high performance cache) implementation which significantly reduces the number of full garbage collections by dynamically adjusting the cache size. For more read http://docs.neo4j.org/chunked/stable/configuration-caches.html#_object_cache .

Answer 2

For me it was helping to add/uncomment the following lines in neo4j-wrapper.conf :

wrapper.java.initmemory=4096
wrapper.java.maxmemory=4096

The problem is the RAM size.

You can find this file in "C:\\Program Files\\Neo4j Community\\neo4j-community-2.1.3\\conf" on a Windows system eg

PS: another good article on what else could help you ( http://jexp.de/blog/2014/06/load-csv-into-neo4j-quickly-and-successfully/ ).