Importing data into Neo4J from CSV

Question

I am trying to import a CSV containing nodes (+200000) and edges between them, into Neo4J.

1) For some reason that I could not find out, when the file size is over 5000 lines (or so), relationships do not get created at all.

USING PERIODIC COMMIT 100
LOAD CSV WITH HEADERS FROM "file:///export_conceptos_50000.txt" AS csvLine
FIELDTERMINATOR '\t'
MERGE (c:Concepto {nom: csvLine.concepto1_sin, cat: csvLine.c1cat})
MERGE (d:Concepto {nom: csvLine.concepto2_sin, cat: csvLine.c2cat})
FOREACH(ignoreMe IN CASE WHEN csvLine.relacion='Is_a' THEN [1] ELSE [] END |     MERGE (c)-[:Is_a]->(d) )
FOREACH(ignoreMe IN CASE WHEN csvLine.relacion='Finding_site' THEN [1] ELSE [] END |     MERGE (c)-[:Finding_site]->(d) )

So this is the original issue: edges are not created.

2) As an alternative, I tried to split the file into smaller ones, and then import via neo4j-shell (Neo4J Linux shell utility).

This is the command line:

./neo4j-shell -file /usr/share/neo4j/scripts/query.cypher -path /usr/share/neo4j/neo4j-community-3.1.1/data/databases/graph.db

and this is the output:

ERROR (-v for expanded information):
        Error starting org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory, /usr/share/neo4j/neo4j-community-3.1.1/data/databases/graph.db

which I guess is due to the fact that there is already a Neo4J engine running.

Then, how should I specify the target database at the command line?

Thanks!

Answer 1

1. you might want to do multi-pass, as otherwise you might get problems with eager loading of your CSV data

2. perhaps there is something wrong in your conditionals, if you do multi-pass anyway you can also change them to a simple WHERE

like this:

USING PERIODIC COMMIT 100000
LOAD CSV WITH HEADERS FROM "file:///export_conceptos_50000.txt" AS csvLine
FIELDTERMINATOR '\t'
MERGE (c:Concepto {nom: csvLine.concepto1_sin, cat: csvLine.c1cat});

USING PERIODIC COMMIT 100000
LOAD CSV WITH HEADERS FROM "file:///export_conceptos_50000.txt" AS csvLine
FIELDTERMINATOR '\t'
MERGE (d:Concepto {nom: csvLine.concepto2_sin, cat: csvLine.c2cat});

USING PERIODIC COMMIT 10000
LOAD CSV WITH HEADERS FROM "file:///export_conceptos_50000.txt" AS csvLine
FIELDTERMINATOR '\t'
WITH csvLine WHERE csvLine.relacion='Is_a'
MATCH (c:Concepto {nom: csvLine.concepto1_sin, cat: csvLine.c1cat})
MATCH (d:Concepto {nom: csvLine.concepto2_sin, cat: csvLine.c2cat})
MERGE (c)-[:Is_a]->(d);


USING PERIODIC COMMIT 10000
LOAD CSV WITH HEADERS FROM "file:///export_conceptos_50000.txt" AS csvLine
FIELDTERMINATOR '\t'
WITH csvLine WHERE csvLine.relacion='Finding_site'
MATCH (c:Concepto {nom: csvLine.concepto1_sin, cat: csvLine.c1cat})
MATCH (d:Concepto {nom: csvLine.concepto2_sin, cat: csvLine.c2cat})
MERGE (c)-[:Finding_site]->(d);