OrientDB in distributed keeps getting ConcurrentModificationException

Question

I'm using orientdb community edition 2.2.9 with pyorient's binary serialiser (in development branch).

I have 3 nodes running on AWS.

I'm only using 1 node as a master for writing/reading and the other nodes for reading and replication.

I have configured the nodes with the following:

<properties>
    <entry value="2147483647" name="ridBag.embeddedToSbtreeBonsaiThreshold"/>
    <entry value="-1" name="index.embeddedToSbtreeBonsaiThreshold"/>
</properties>

I'm not using Java so the MVCC examples in the documentation is not really helping me.

I'm also not using transactions and I start the server with the following parameters:

java -Dcache.level1.enabled=false -Ddb.mvcc=false 

I read in the docs that you can't disable mvcc any more so I assume that mvcc setting is useless.

I use rabbitmq and celery to queue tasks. When running orientDB in normal mode, when I get a "ConcurrentModificationException" error, celery will just retry that task and it usually succeeds. When running in distributed mode, that task keeps failing because the vertices version never seem to match up.

It doesn't matter what I do, I keep getting " ConcurrentModificationException ".

I can see in the config that mvcc is still enabled.

I thought I tried everything the documentation suggested but all the nuances of running in distributed mode is scattered all over the documentation and not in one place. It's so easy to miss something :(

How can I avoid this issue?

Answer 1

Setting "writeQuorum" to 1 fixed this issue for me. Also, running "executionMode" as "asynchronous" also created problems.

Also, removing the following from my config also may have helped with this issue:

....
<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
    <parameters>
        ....
        <parameter value="com.orientechnologies.orient.server.distributed.conflict.ODefaultReplicationConflictResolver" name="conflict.resolver.impl"/>
        ....
    </parameters>
</handler>
...

Here is the default-distributed-db-config.json that got things working for me.

{
  "autoDeploy": true,
  "readQuorum": 1,
  "writeQuorum": 1,
  "executionMode": "synchronous",
  "readYourWrites": true,
  "servers": {
    "*": "master"
  },
  "clusters": {
    "internal": {
    },
    "*": {
     "servers": ["<NEW_NODE>"]
    }
  }
}

I hope it helps someone.