使用Py2neo在Neo4j中创建许多节点

Question

I have for some time tried to create a lot of notes in py2neo. The nodes is based on a tweet live-stream, where I want to plot the tweet, information about who tweeted it and Relation between tweets and other nodes (eg. re-tweets, user mentions and tags).

What I have tried to, is makint a large cypher query, using MERGE to create/get the ID of the user and tweet, and link them. My idea is to have the following nodes:

• User node
• Tweet
• Tags
• Location (For tweet)
• Location (For user)
• Language
• Gender
• TimeZone

And links where I need lookup:

• Mentions
• Re-Tweet

This is a lot of writing, and I do something like this:

statement = "MERGE (tUser:TwitterUser {id:{tuID}}) " \
            "ON CREATE SET " \
            "tUser.displayName = {tdNAME}, " \
            "tUser.summary = {tdSummary}, " \
            "tUser.link = {tdLink}, " \
            "tUser.preferredUsername = {tdPreferredUsername}, " \
            "tUser.account_created = {tdAccount_created}, " \
            "tUser.last_lookup = 'Newer' " \
            "" \
            "MERGE (user:Person {name:{userName}})-[:twitter_acct]->(tUser) " \
            "" \
            "MERGE (gender:Gender {gender: {GENDER}})" \
            "MERGE (user)-[:has_gender]->(gender) " \
            "" \
            "MERGE (user)-[:tweeted]->(tweet:Tweet {id:{tID}}) " \
            "ON CREATE SET " \
            "tweet.type = {tType}, " \
            "tweet.link = {tLink}, " \
            "tweet.body = {tBody}, " \
            "tweet.postedTime = {tPostedTime} " \
            "" \
            "MERGE (timezone:TimeZone {name:{timeZoneName}}) " \
            "MERGE (user)-[:has_time]->(timezone)" \
            "" \
            "MERGE (user)-[:use]->(generator:Generator {name: {generator}}) " \
            "ON CREATE SET " \
            "generator.link = {generatorLink} " \
            "" \
            "MERGE (tweet)-[:tweeted_in]->(tLocation:Location {name: {tLocationName}}) " \
            "MERGE (tLocation)-[:in]->(tCountry:Country {name: {tCountryName}}) " \
            "" \
            "MERGE (user)-[:lives_in]->(uLocation:Location {name: {uLocationName}}) " \
            "" \
            "RETURN user"

The problem is: When I try to insert the tweet in my Neo4J database, it can't follow, also, when I try to just do it with a set of data I have made, it still do it slow. I have tried to use batch'es, but still to slow.

Would the solusion be to make less noeds, get a better machine or..? use Schema (And how do I get the right ID of eg. a usernode if i restart the service).

Answer 1

It's hard to know which direction to point you in without some quantification of "a lot" and "slow". These are very subjective terms.

Generally speaking, you'll want to make sure that you combine a number of server interactions into a single request, either via a large Cypher transaction or via the (legacy) batch mechanism. It is certainly possible to get quite a lot of performance out of the REST interface, if used cleverly.

Outside of that, you can of course look at something server side: perhaps a Java extension or, for initial loads, one of the bulk import tools.