Testing MongoDB in Java: concurrency problems

Question

I'm testing the part of my Java application where I store data in a MongoDB database. My test setup looks like this:

public class MongoDataStoreTest {

    private MongoClient client;

    @Before
    public void before() throws UnknownHostException {
        this.client = new MongoClient();
    }

    @After
    public void after() throws InterruptedException {
        this.client.dropDatabase("testdb");
        this.client.close();
    }

}

In my tests I execute some code which does the following:

1. I create a DB instance with: DB database = client.getDB("testdb")
2. I add a collection in the database: database.getCollection("testcoll")
3. And then I insert a BasicDBObject: collection.insert(object, WriteConcern.SAFE)
4. Directly after this I query the database using the standard cursor method.

As can be seen in my test setup code, after each test I drop the database and close all client connections. I execute ten such tests. When running them locally everything happens as I expect. The objects are inserted and afterwards the database is dropped for each test (I can see this in the mongo log). However when executing this on a Jenkins server it sometimes happens that when querying the database, an object of the previous test is still in that database, although that database should have been dropped. This looks like a concurrency problem to me, but I can't see where the race condition is situated. I have no access to the database log on the Jenkins server. Does anyone know what I should change to make sure my tests always succeed?

Answer 1

Dont't drop the database. There might be some internal references to it in mongo. I don't beleive, that your test-case needs the DB to be dropped. Usually it's enough simply to remove the all documents in the collections under test

Answer 2

To clear MongoDB databases our code looks like this:

  public void clearData() {
    try {
      for (String collection : datastore.getDB().getCollectionNames()) {
        // We must not mess with system indexes and users as this will cause
        // errors
        if (!collection.startsWith("system.")) {
          // Do not drop the entire database or full collections as this
          // will lead to missing index errors (for no obvious reason).
          datastore.getDB().getCollection(collection).drop();
        }   
      }   
    } catch (MongoException e) {
      LOG.log(Level.INFO,
              "Could not fetch all collection names - this is a permission thing, but can be ignored");
    }   

    // The indexes are not automatically recreated (for no obvious
    // reason) - ensure they are still there after the drop().
    datastore.ensureIndexes();
    datastore.ensureCaps();
  }

Answer 3

The problem was caused by the dropDatabase operation. This operation seemed to take longer on the Jenkins server than on my local machine. Since MongoDB doesn't seem to wait until the database is completely dropped it added the new document in the old (dropping) database. To keep my tests as independent as possible I solved the problem by generating a different unique database name for each test.