Neo4J: Importing a large Cypher dump

Question

I have a large dump (millions of nodes and relationships) from a Neo4J 2.2.5 database in Cypher format (produced with neo4j-sh -c dump ), that I'm trying to import into a 3.0.3 instance.

However, the import process ( neo4j-sh < dump.cypher ) slows down drastically after a few minutes, down to a couple records per second.

Is there any way to speed up this process? I tried upgrading the database as described in the manual , but the new instance crashes with an exception about a version mismatch in the store format.

Answer 1

Neo4j 3.0 comes with a bin/neo4j-admin tool for exactly this purpose.

try bin/neo4j-admin import --mode database --from /path/to/db

see: http://neo4j.com/docs/operations-manual/current/deployment/upgrade/#upgrade-instructions

The cypher dump is not useful for large database, it's only for smaller setups (a few thousand nodes) for demos etc.

FYI: In Neo4j 3.0 the cypher export procedure from APOC is much more suited for large scale cypher dumps.

Update

You can also try to upgrade from 2.2 to 2.3 first. Eg by using neo4j-shell

add allow_store_upgrade=true to your neo4j.properties` in 2.3

and then do: bin/neo4j-shell -path /path/to/db -config conf/neo4j.properties -c quit

If it is finished that backup of your db is on Version 2.3

Then you should be able to use neo4j-admin -import ...

Answer 2

I recently had this same symptom with my CSV import slowing to death. My load-csv cypher script had too many rels.

So I divided my load in two. First create the nodes, then the relations and most connected nodes. HIH.

Back to your issue First, try to increase the memory for the JVM. In NEO/conf, there is a wrapper file. At the beginning are the memory settings.

Lastly, from an instance w/ your data, export to multiple CSV files and import them in your new server.