Quickly adding edge counts to a document in ArangoDB

Question

Not too complicated: I want to count the edges of each document and save the number in the document. I've come up with two queries that work; unfortunately since I have millions of edges both are quite slow. Is there a faster way to update documents with a property storing their number of edges? (just a count at a point in time)

AQL queries that are functional but slow:

FOR doc IN Documents
    LET inEdgesCount  = LENGTH(GRAPH_NEIGHBORS('edgeGraph', doc,{direction: 'inbound', maxDepth:1})
    LET outEdgesCount = LENGTH(GRAPH_NEIGHBORS('edgeGraph', doc,{direction: 'outbound', maxDepth:1})
    UPDATE doc WITH {inEdgesCount: inEdgesCount, outEdgesCount: outEdgesCount} In Documents

or:

FOR e IN Edges
    COLLECT docId = e._to WITH COUNT INTO counter
    UPDATE SPLIT(docId,'/')[1] WITH {inEdgeCount: counter}

(and then repeat for outbound edges)

---

As an aside, is there any way to view either query speed (eg FOR executions per second) or percentage completion? I've been trying to judge speed by using LIMITed queries to start with, but the time required doesn't seem to scale linearly.

Answer 1

With ArangoDB 2.8 you can use graph pattern matching traversals to execute this with better performance:

FOR doc IN documents
  LET inEdgesCount = LENGTH(FOR v IN 1..1 INBOUND doc GRAPH 'edgeGraph' RETURN 1)
  LET outEdgesCount = LENGTH(FOR v IN 1..1 OUTBOUND doc GRAPH 'edgeGraph' RETURN 1)
  UPDATE doc WITH
     {inEdgesCount: inEdgesCount, outEdgesCount: outEdgesCount} In Documents

Currently ArangoDB doesn't have a way to monitor the progress of long running tasks. With ArangoDB 3.0 we're going to introduce a new monitoring framkework that allows better inspection of whats actually going on in the server. However, with 3.0 it won't be able to gather live statistics; we may see this further down the 3.x road later this year. Judging percentage completion may become possible for easy tasks like creating indices, but on queries its rather going to be the number of documents read/written so far.

We did similar queries for validating whether a graph obeys a power law