How to be notified of errors in JUnit test methods annotated with @Parameters

Question

I am trying to record errors that occur when creating parameters for JUnit parameterized tests. Is there any way of being notified when a failure occurs in a method annotated with @Parameters, without modifying the method to catch exceptions?

The methods annotated with @Parameters do not appear to trigger the testFailure() method of a JUnit RunListener.

(The following is in response to the comment to show code)

This is the outline of the code I am running:

(1) The test: (the exception occurs in the 'testCases()' method, adding try/catch within this method works but I am trying to avoid that because there are hundreds of similar tests)

@RunWith(MyParameterizedRunner.class)
public class MyTest {

    @Parameters
    public static List<String[]> testCases() {
      ... [**exception occurs here**]
    }
    ...
}

(2) The Runner (the exception appears to be thrown in the constructor by the 'super(klass)' call, before the listener is even added; I cannot figure out how to catch the exception here since 'super(klass)' must be the first line of the constructor)

public class MyParameterizedRunner extends Parameterized {

    public MyParameterizedRunner(Class<?> klass) throws Throwable {
        super(klass);
    }

    @Override
    public void run(RunNotifier notifier) {
        notifier.addListener(new MyListener());
        super.run(notifier);
    }
}

(3) The Listener: (this has not been activated at the point the above exception occurs)

public class MyListener extends RunListener {
    public void testFailure( Failure fail) {
      ...
    }
}

Answer 1

The JUnit runner will declare an initialization error if there is an uncaught exception which fails all the tests in that class. In your JUnit results you will just see the single initializationError.

The RunListener javadoc suggests most of the test* methods are only called for "atomic tests" which would not catch this, since the @Parameterized method is what generates the "atomic tests".

It looks like you need to look at the testRunFinished method, iterate over its getFailures(), and look for Failure objects that reference initializationError.

UPDATE: Since the @Parameterized method runs before your test runner is initialized, you could actually create a new test method that calls your @Parameterized method to make sure that the @Parameterized method works. This new test would have to live outside of your @Parameterized class of course, but it is a simple refactoring.

This is a useful test if the @Parameterized method is expected to fail occasionally, which might happen if your parameters come from an external source or a source that changes frequently.