Why my JUnit test cases exhibit different output when run as a Test Suite and individual Junit file

Question

I have a set of JUnit test files and also a test suite (Suite class) file which holds the reference to all individual Junit test files.

All are database oriented. The database used is Mysql and I am using Eclipse IDE to run the tests.

When running each file individually I get the correct value and the assertion is correct but when running from the test suite it shows a different value.

I have made each JUnit test file to access the database independently with a different database name (even though the table structure is same)

Whether we need to prevent the JUnit test cases from running parallel / the database related statements need to be verified ?

Answer 1

I would suggest you look into :

• any static data (a singleton, static methods, etc...)
• the order of tests (is it possible that some of the tests depends on some data that another test create, erase, or modify ?)
• if you're using a framework that allow suite-wide setUp or tearDown methods, could they somehow break individual tests ?
• since you're using a database, is it possible that somehow your code is transactionnal, but when running the tests in a Suite some transactions are not commited at the right time (for example, not before the whole suite is finished) ?

Answer 2

There are 2 reasons that this may happen.

One is that you have not constructed the suite properly and some tests share resources with others.

And the other is that when you finish a test, you don't rollback the database and so the next test finds database in an erroneous state.