How does Eclipse actually run Junit tests?

Question

I am running into a discrepancy when running Junit tests in Eclipse and Ant. Here is the scenario:

Everything ran as intended in Eclipse, however, I could not get an accurate Junit report when running through an Ant build script I whipped up. I made a couple of changes to our test runner and test cases (in a nutshell I added the Test suite() method to all of my test cases) which returns a new Junit4TestAdapter, and had our custom runner execute RunNotifier.fireTestAssumptionFailed(Failure) instead of fireTestAssumption. Now everything runs fine in Ant, but failures are marked as passed when run in Eclipse.

Is there any documentation for Eclipse that explains exactly how it runs Junit tests? I am basically looking to know exactly how Eclipse executes Junit tests, whether it's run directly through Ant, if it uses Java to interface with Junit, etc. If anyone knows an actual solution to the issue, I welcome that as well, but I would really like to try solving this one on my own, I just need a point in the right direction.

Answer 1

To answer your problem first, if you have a discrepancy between the Ant junit and Eclipse JUnit, it's probably a classpath or environmental problem. The easiest way is to find a test that performs differently between the two and print out the System properties, and work at it from that direction. Another thing to try would be to run the ant scripts from within Eclipse, to see if this makes any difference (because the environment will change)

Eclipse does not use Ant to run the tests.

---

As for how Eclipse runs JUnit tests, here is a quick overview. Be warned: there is some deep magic in the Eclipse JUnit plugin.

Eclipse has 4 JUnit plugins, which are all installed by default in most configurations:

1. org.eclipse.jdt.junit: git://dev.eclipse.org/org.eclipse.jdt/org.eclipse.jdt.junit.git
2. org.eclipse.jdt.junit.core: git://dev.eclipse.org/org.eclipse.jdt/org.eclipse.jdt.junit.core.git
3. org.eclipse.jdt.junit.runtime: git://dev.eclipse.org/org.eclipse.jdt/org.eclipse.jdt.junit.runtime.git
4. org.eclipse.jdt.junit4.runtime: git://dev.eclipse.org/org.eclipse.jdt/org.eclipse.jdt.junit4.runtime.git

These are git mirrors of the actual CVS repositories. Last time I tried to use them, they didn't compile, but they'll give you the code and you can at least import the projects into Eclipse and look at them.

If we ignore the configuration pages, how the plugin creates run configurations, the code for the JUnit view itself and how it finds the relevant tests to run, we can concentrate on how it runs the tests.

The core classes are org.eclipse.jdt.junit.launcher.JUnitLaunchConfigurationDelegate and org.eclipse.jdt.internal.junit.runner.RemoteTestRunner . JUnitLaunchConfigurationDelegate reads the launch configuration and forks a JVM in which the tests will be run. The main class for this new JVM is RemoteTestRunner . The tests to be run are passed as parameters to the forked JVM, either as a single test or as a list of tests in a temporary file if you're doing Run as JUnit on a project. If you're debugging, this new JVM can be kept alive by checking the Keep alive when debugging checkbox in the run configuration. In this case, the JVM will stick around and reruns of existing tests will be sent via sockets.

RemoteTestRunner runs the tests and passes back the results via sockets to Eclipse, which then updates the JUnit view. The heart of this is org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference , which runs the test (for JUnit 4), and org.eclipse.jdt.internal.junit4.runner.JUnit4TestListener , which is the RunListener for these tests. JUnit4TestListener extends RunListener , but doesn't override testAssumptionFailure , which is probably why your tests are passing in Eclipse. RunListener.testAssumptionFailure is an empty method, it does nothing, so your notifications will be ignored.

I would start by cloning the git repos, importing the projects into Eclipse and trying to work through the code.

Answer 2

Download the plugin development tools (PDT) for Eclipse.

Start a plugin project and add org.eclipse.jdt.internal.junit.runner and org.eclipse.jdt.junit.core to your plugin.xml's dependancies.

Explore those classes and you will get an idea of how JUnit is launched. It does not use Ant at all.

To see how failures are marked and manipulate the reporting you should look at org.eclipse.jdt.internal.junit4.runtime assumming you are using JUnit 4.

In sum, be prepared to spend a while learning and developing a plugin if you want to change how Eclipse's JUnit works.