运行大量密码的有效方法是什么？

Question

I want to import data into my Neo4j database. From my raw data, I generate a lot of cypher.

For example, I have a list of cypher like this (up to hundreds of thousand):

MERGE (product:PRODUCT{name:'X phone'}) MERGE (product)-[:RATE]-(review:REVIEW{content:'worst phone ever'})
MERGE (product:PRODUCT{name:'X phone'}) MERGE (product)-[:RATE]-(review:REVIEW{content:'cheapest phone ever'})
MERGE (product:PRODUCT{name:'Y phone'}) MERGE (product)-[:RATE]-(review:REVIEW{content:'even worse than phone X'})
MERGE (product:PRODUCT{name:'X phone'}) MERGE (product)-[:RATE]-(review:REVIEW{content:'better than newly release Y version'})

My current solution is run the cypher line-by-line from file using Neo4j driver in Python.

from neo4j.v1 import GraphDatabase
import sys

class CypherClient:
    """
    The client that execute cypher
    """
    def __init__(self, uri, auth):
        self.driver = GraphDatabase.driver(uri, auth=auth)

    def run_cypher(self, cypher):
        """
        execute single cypher
        :param cypher: the cypher in str
        :return: no return anything at all
        """
        with self.driver.session() as session:
            session.run(cypher).single()

if __name__=="__main__":

    """
    execute cypher from file
    each line is independent cypher
    python exec_cypher_file.py outcypher.txt 
    """

    # replace URI and authentication here
    uri = "bolt://localhost:7687"
    auth = ("neo4j", "IAmPusheenTheCat")

    counter = 0

    if len(sys.argv) < 2:
        test()
    else:
        client = CypherClient(uri, auth)
        infile = sys.argv[1]
        errfile = open(infile+".err.txt", 'w')
        for line in open(infile):
            # print(line)
            try:
                client.run_cypher(line)
            except:
                print(str(counter) + " " + line+"\n")
                errfile.write(str(counter) + " " + line+"\n")
            counter+=1
            if counter % 100 == 0 or counter < 100:
                print(counter)
        errfile.close()
    print('done')

What could I do to improve the efficiency of running bulk cypher ?

Answer 1

CSV loading tends to be very efficient, so if you have your data in CSV form you can use LOAD CSV .

Otherwise, you can check Michael Hunger's article on effective batch updates which uses UNWIND to process a list of inputs as a batch.