如何编写数据库单元测试？

Question

I'm using MongoDB, Java, JDO and Maven.

I ran a simple main application that stores some junk data and retrieves it from mongodb and it looks ok, but I want to start writing proper unit tests. Mocking out the actual database interactions doesn't seem very helpful to me so technically I want to write functional tests that check the full interaction with a database.

My unit test is set up like this:

static final Logger LOG = LoggerFactory.getLogger(DaoTest.class);
static final int PORT = 100001;
static Process mongod;
static PersistenceManagerFactory pmf;

// Class under test.
Dao dao;

@BeforeClass
public static void beforeAll() throws Exception {
  File mongoDir = new File(
      System.getProperty("java.io.tmpdir"),
      "mongodb-" + System.currentTimeMillis());
  mongoDir.deleteOnExit();
  mongod = Runtime.getRuntime().exec(String.format(
      "/bin/sh -c mongod --dbpath=%s --port=%d",
      mongoDir.getAbsolutePath(),
      PORT));
  LOG.info("Mongodb using {} on port {}.", mongoDir.getAbsolutePath(), PORT);
  Thread.sleep(1000);
  pmf = JDOHelper.getPersistenceManagerFactory("mongodbtest");
  LOG.info("DB connection URL: {}.", pmf.getConnectionURL());
}

@AfterClass
public static void afterAll() throws Exception {
  mongod.destroy();
}

@Before
public void setUp() throws Exception {
  dao = new Dao(pmf);
}

src/test/resources/META-INF/persistence.xml is like this:

<?xml version="1.0" encoding="UTF-8" ?>
<persistence ...>
  <persistence-unit name="mongodbtest">
    <properties>
      <property name="javax.jdo.PersistenceManagerFactoryClass"
          value="org.datanucleus.api.jdo.JDOPersistenceManagerFactory" />
      <property name="javax.jdo.option.ConnectionURL" value="mongodb://localhost:100001/contacts"/>
      <property name="javax.jdo.option.RetainValues" value="true"/>
      <property name="datanucleus.autoCreateSchema" value="true" />
      <property name="datanucleus.validateTables" value="false" />
      <property name="datanucleus.validateConstraints" value="false" />
      <property name="datanucleus.storeManagerType" value="mongodb" />
    </properties>
  </persistence-unit>
</persistence>

And finally I execute my unit test with mvn clean test and that alone will work until I add an actual test that attempts save data. I get the following error:

Mongodb using /var/folders/00/0l550000h01000cxqpysvccm002cmm/T/mongodb-1373820849219 on port 100001.
DB connection URL: mongodb://localhost:100001/contacts.
Jul 14, 2013 12:54:10 PM com.mongodb.DBTCPConnector initDirectConnection
WARNING: Exception executing isMaster command on localhost/127.0.0.1:27017
java.io.IOException: couldn't connect to [localhost/127.0.0.1:27017] bc:java.net.ConnectException: Connection refused
    at com.mongodb.DBPort._open(DBPort.java:214)
    at com.mongodb.DBPort.go(DBPort.java:107)

...

Jul 14, 2013 12:54:10 PM com.mongodb.DBTCPConnector initDirectConnection
WARNING: Exception executing isMaster command on localhost/127.0.0.1:27017
java.io.IOException: couldn't connect to [localhost/127.0.0.1:27017] bc:java.net.ConnectException: Connection refused
    at com.mongodb.DBPort._open(DBPort.java:214)
    at com.mongodb.DBPort.go(DBPort.java:107)

...

ul 14, 2013 12:54:10 PM com.mongodb.DBPortPool gotError
WARNING: emptying DBPortPool to localhost/127.0.0.1:27017 b/c of error
java.io.IOException: couldn't connect to [localhost/127.0.0.1:27017] bc:java.net.ConnectException: Connection refused
    at com.mongodb.DBPort._open(DBPort.java:214)
    at com.mongodb.DBPort.go(DBPort.java:107)

...

Jul 14, 2013 12:54:10 PM org.datanucleus.store.valuegenerator.AbstractGenerator obtainGenerationBlock
INFO: Error encountered allocating block of IDs : can't call something : localhost/127.0.0.1:27017//localhost:100001/contacts

127.0.0.1:27017 is mongodb's default settings but I have no idea why it's using that. I can see it's picking up the configs from my persistence.xml file so I'm confused. The mongo docs say the connection string URL format is:

mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]

So far I've only been running this through maven because that's only as far as I got so far but another question I'll have eventually is how to run these unit tests in Eclipse. I assume the byte code enhancement process has to happen for these tests to be run in Eclipse and I don't know how to set that up.

I'm guessing I'm not the first person to want to do something like this though. Are there any standard practices for writing functional database tests in Java? Am I going about it all wrong?

Answer 1

To answer my own question. There were three issues:

1) The connection string should have been:

mongo:/127.0.0.1/contacts

(note one less "/")

2) I needed to add mongoDir.mkdir();

3) The command to start the server should be

String command = String.format(
    "/bin/sh -c mongod --dbpath=%s --port=%d", 
    mongoDir.getAbsolutePath(), 
    PORT);
Runtime.getRuntime().exec(new String[] { "/bin/sh", "-c", command });

I actually ended up creating a JUnit4 Runner for this so I could have the same database throughout all tests and left up to the individual tests to manage the state of that database.

Answer 2

If you want to run a database instance to test against (that runs only for the time that your Maven build is running) then you might be interested in the Maven plugin that I wrote based on the flapdoodle Embedded Mongo API:

embedmongo-maven-plugin

One benefit is that the plugin downloads the Mongo distribution automatically, so there's no need for people to install Mongo before they can run your Maven build. It's also easy to move to a different Mongo version; you just change the plugin configuration.

Answer 3

Think about your goal I ran a simple main application that stores some junk data and retrieves it from mongodb and it looks ok, but I want to start writing proper unit tests.

If you work in team, you will find a problem with current source code. How if your another team member use staging db ? It will make your data useless, inconsistent and painful to reproduce bug related with db.

I think the solution from @joelittlejohn is better. Use embedded-mongo on your project. Currently I use https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo for my project. You can do anything with your mongo without any hesitation. Because it's just happen when you run test .