带有小图的Neo4J在Ubuntu 16.04上占用了大量内存

Question

I have installed Neo4J v3.3.0 (community edition) in an Ubuntu 16.04 virtual machine (Hyper-V) with 8GB and 4 cores.

I have a very small graph (30 nodes) and it is used just for reading (about 1 hit every 3 seconds), it seldomly gets written to. We want to expand the graph a lot more but every three days (sometimes less) our server crashes because Java is taking more than 2GB and top showed 300% CPU usage.

To me this makes no sense at all, could you please let me know how to configure Java or Neo4J in order to prevent this?

Thanks

I have the following configuration in my /etc/neo4j/neo4j.conf file:

dbms.query_cache_size=5000
dbms.threads.worker_count=4
dbms.memory.heap.initial_size=2g
dbms.memory.heap.max_size=2g

dbms.memory.pagecache.size=2g

The log files show the following error when this happens:

ERROR [o.n.b.v.r.c.RunnableBoltWorker] Worker for session 'ecfe4a7f-1714-4ba3-9e98-a692bf153b45' crashed. Java heap space java.lang.OutOfMemoryError: Java heap space

There are also these suspicious messages (which there are a lot of):

WARN [o.n.k.i.c.MonitorGc] GC Monitor: Application threads blocked for 4680ms.

ERROR [o.n.b.v.t.BoltMessagingProtocolV1Handler] Failed to write response to driver Unable to write to the closed output channel org.neo4j.bolt.v1.packstream.PackOutputClosedException: Unable to write to the closed output channel

WARN [io.netty.channel.DefaultChannelPipeline] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. syscall:read(..) failed: Connection reset by peer

New Information

I did an:

netstat -an | grep ESTABLISHED

I had a lot of open connections. We are using the following javascript driver in nodeJS ( https://github.com/neo4j/neo4j-javascript-driver ). Will check if we are not properly checking connections.

It seems I am closing correctly all connections properly with:

session.close();
driver.close();

The connections still remain open until I exit the application.

Final Comments

There was a place in my code where I was not closing connections.

https://github.com/neo4j/neo4j-javascript-driver/issues/275

Answer 1

I can say, that Neo4j works fine for me on 8GB of RAM with 10mln nodes and 30mln relations without special tuning.

top shows 300% CPU usage probably when garbage collection is done. So I vote for increasing heap size

/etc/neo4j/neo4j.conf

parameter

dbms.memory.heap.max_size=3g

On larger databases high CPU consumption mean absence of indexes.

To list indexes:

CALL db.indexes();

to create one:

CREATE INDEX ON :Label(prop_name);

If you get OOM errors (see dmesg) and java gets killed by linux (not crashes), try to install server OS, like CentOS without GUI et c, to free some memory. 8GB is more than enough for Neo4j with 8-10GB size database.