Concurrency issues when querying the Google App Engine datastore

Question

My application is currently facing a problem where a race condition sometimes lead multiple threads to write the same value to the datastore.

The pseudocode looks like this:

public void writeToDatastore(ValueObject obj){
   boolean objectExists = checkDatastoreForObject(obj);

   if(!objectExists){
      doSomeStuff();
      writeObjectToDatastore(obj);
   }
}

Sometimes, two or more threads read from the datastore before anything has been written, which leads to more than one instance of the object being persisted.

Adding the synchronized keyword (or other concurrency constructs) fixes the problem - provided that there is only one instance of the application running.

How can these types of concurrency issues be resolved in a distributed environment?

Answer 1

When calling Datastore from App Engine, use a transaction. First try to get() the entity to see if it's there. If not, put() the entity.

Here's some sample code that demonstrates transactions using the App Engine API: https://github.com/GoogleCloudPlatform/java-docs-samples/blob/master/appengine/datastore/src/test/java/com/example/appengine/TransactionsTest.java

Answer 2

Please refer to this: https://cloud.google.com/appengine/articles/scaling/contention

Avoiding datastore contention - Datastore contention occurs when a single entity or entity group is updated too rapidly. The datastore will queue concurrent requests to wait their turn. Requests waiting in the queue past the timeout period will throw a concurrency exception. If you're expecting to update a single entity or write to an entity group more than several times per second, it's best to re-work your design early-on to avoid possible contention once your application is deployed.