Neo4j GCR cache_type & multi-thread performance

Question

I've tried an experiment about SNS, and I have used both cypher and traversal API to query. For now, each query costs about 40 ms. But when I use multi-thread to query, the performance drops sharply, each query costs almost 200 ms. There are about 1 million nodes and 200 million relationships, and I use gcr cache_type. Please refer to my experiment for the details if you want.

I use visualVM to find out the hotspot of the multi-thread, and I find NodeManager.lockId costs a lot of time(lock in fact). The NodeManager.lockId source is below:

private ReentrantLock lockId( int id )
{
    int stripe = (id / 32768) % LOCK_STRIPE_COUNT;
    if ( stripe < 0 )
    {
        stripe *= -1;
    }
    ReentrantLock lock = loadLocks[stripe];
    lock.lock();
    return lock;
}

public Node getNodeById( int nodeId ) throws NotFoundException
{
    NodeImpl node = nodeCache.get( nodeId );
    if ( node != null )
    {
        return new NodeProxy( nodeId, this );
    }
    ReentrantLock loadLock = lockId( nodeId );
    try
    {
        if ( nodeCache.get( nodeId ) != null )
        {
            return new NodeProxy( nodeId, this );
        }
        if ( !persistenceManager.loadLightNode( nodeId ) )
        {
            throw new NotFoundException( "Node[" + nodeId + "]" );
        }
        node = new NodeImpl( nodeId );
        nodeCache.put( nodeId, node );
        return new NodeProxy( nodeId, this );
    }
    finally
    {
        loadLock.unlock();
    }
}

It seems that there will be a lock while getting a node if the node is not cached. So I want to know the mechanism of GCR cache-type, because if I set the node_cache_size and relationship_cache_size default, it works faster than I set:

config.put( "node_cache_size", "1G");
config.put( "relastionship_cache_size", "8G");

Please tell me the details about GCR cache-type, and how to config the cache_type and warmup. I want to improve the performance when I use multi-thread. Thanks

Answer 1

GCR cache is documented at bottom of http://docs.neo4j.org/chunked/stable/configuration-caches.html#_object_cache . For more on GCR implementation details look at the code at https://github.com/neo4j/neo4j/tree/master/enterprise/ha/src/main/java/org/neo4j/kernel/impl/cache and the tests at https://github.com/neo4j/neo4j/tree/master/enterprise/ha/src/test/java/org/neo4j/kernel/impl/cache

For warming up, iterate over all nodes and relationships using GlobalGraphOperations.getAllNodes() and GlobalGraphOperations.getAllRelationships() .