Confusing execution order of JUnit methods

Question

I've created a custom runner by extending Suite :

public class MyRunner extends Suite {

    public MyRunner(Class<?> klass, RunnerBuilder builder) throws InitializationError {
        super(klass, builder);
    }

    @Override
    public void run(RunNotifier notifier) {
        notifier.addListener(new MyRunListener());
        notifier.fireTestRunStarted(getDescription());
        super.run(notifier);
    }
}

As seen in run this registeres a MyRunListener which looks like this:

class MyRunListener extends RunListener {
    @Override
    public void testRunStarted(Description description) {
        System.err.println("1: run started");
    }

    @Override
    public void testStarted(Description description) {
        System.err.println("2: test started");
    }

    @Override
    public void testFinished(Description description) {
        System.err.println("3: test finished");
    }

    @Override
    public void testRunFinished(Result result) {
        System.err.println("4: run finished");
    }
}

I've added 1: , 2: , 3: and 4: according to the order in which I expect these methods to be called.

I've then created a test suite that looks as follows:

@RunWith(MyRunner.class)
@Suite.SuiteClasses({ MyTest.class })
public class MyTestSuite {
}

And the following test:

public class MyTest {
    @BeforeClass
    public static void beforeClass() {
        System.err.println("A: beforeClass");
    }

    @Before
    public void before() {
        System.err.println("B: before");
    }

    @Test
    public void test() {
        System.err.println("C: Running actual test...");
    }

    @After
    public void after() {
        System.err.println("D: after");
    }

    @AfterClass
    public static void afterClass() {
        System.err.println("E: afterClass");
    }
}

Again, output labeled according to the order I expect.

Here's the output I get when I run MyTestSuite through IntelliJ (as a JUnit run configuration):

2: test started

B: before
C: Running actual test...
D: after

3: test finished
E: afterClass

4: run finished
1: run started
A: beforeClass

Why on earth am I getting that output order? I must be doing something very wrong in my runner, but I've just implemented it according to tutorials on the web. (Using JUnit 4.12)

Answer 1

As @DmitryB suggests in the comments, it seems to be some form of output buffering. The various parts of the output is then stitched together in a weird way after the test has finished.

I added Thread.sleep(1000) after each printout and this is how it looks when I run it in IntelliJ:

The workaround for now is to use the gradle task run configuration rather than the JUnit one.

Filed this ticket at Jetbrains.