How to run surefire tests in parallel while forcing some to run sequentially?

Question

I'm working on maven project with thousands of tests that run very slowly (takes 2h to run all the tests). So I tried to run the tests in parallel by configuring surefire plugin as follow:

<configuration>
  <failIfNoTests>false</failIfNoTests>
  <reuseForks>false</reuseForks>
  <reuseForks>true</reuseForks>
  <forkCount>2C</forkCount>
  <systemPropertyVariables>
    <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
  </systemPropertyVariables>
  <parallel>suites</parallel>
  <threadCount>12</threadCount>
</configuration>

But then some of the tests fail in their @Before and @After methods where we initialize and clean up some resources (it seems to be related to ports conflicts).

I tried to add this annotation @net.jcip.annotations.NotThreadSafe to the failing tests as described in surefire documentation to run them sequentially and avoid conflicts. However this didn't work and these tests still fail!!!

Any pointer on how to force some surefire tests to run sequentially on same JVM process and same thread while the rest will run in parallel (potentially on different JVM processes)?

EDIT 1 Now, I tried to split the configuration of surefire into two: one for sequential tests and another one for parallel tests. However this approach does not seem to enhance the execution time as I still have tests that fail and it still takes 2h to run all tests.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${surefire.version}</version>
  <!-- Sequential tests -->
  <configuration>
    <includes>**/*Sequential.java</includes>
    <failIfNoTests>false</failIfNoTests>
    <reuseForks>false</reuseForks>
    <systemPropertyVariables>
      <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
    </systemPropertyVariables>
  </configuration>
  <executions>
    <!-- Parallel tests -->
    <execution>
      <id>parallel-tests</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>**/*.java</includes>
        <excludes>
          <exclude>**/*Sequential.java</exclude>
        </excludes>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>true</reuseForks>
        <forkCount>2C</forkCount>
        <parallel>suites</parallel>
        <threadCount>12</threadCount>
      </configuration>
    </execution>
  </executions>
</plugin>

Answer 1

I'd go with your "split the configuration of surefire into two" approach.

However your configuration seems to be faulty because you only have one execution defined.

Might want to try something like this (define 2 executions with their own configurations):

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>${surefire.version}</version>

  <executions>
     <!-- Sequential tests -->
    <execution>
      <id>sequential-tests</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>**/*Sequential.java</includes>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>false</reuseForks>
        <systemPropertyVariables>
            <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
        </systemPropertyVariables>
      </configuration>
    </execution>      
    <!-- Parallel tests -->
    <execution>
      <id>parallel-tests</id>
      <phase>test</phase>
      <goals>
        <goal>test</goal>
      </goals>
      <configuration>
        <includes>**/*.java</includes>
        <excludes>
          <exclude>**/*Sequential.java</exclude>
        </excludes>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>true</reuseForks>
        <forkCount>2C</forkCount>
        <parallel>suites</parallel>
        <threadCount>12</threadCount>
      </configuration>
    </execution>
  </executions>
</plugin>

Answer 2

Here is my final maven configuration for running the tests as wanted:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${surefire.version}</version>
      <!-- Sequential tests -->
      <configuration>
        <failIfNoTests>false</failIfNoTests>
        <reuseForks>false</reuseForks>
        <systemPropertyVariables>
          <jacoco-agent.destfile>target/jacoco.exec</jacoco-agent.destfile>
        </systemPropertyVariables>
      </configuration>
      <executions>
        <!-- Default tests -->
        <execution>
          <id>default-test</id>
          <configuration>
            <skip>true</skip>
          </configuration>
        </execution>
        <!-- Parallel tests -->
        <execution>
          <id>parallel-tests</id>
          <phase>test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <includes>**/*.java</includes>
            <excludes>
              <exclude>**/*Sequential.java</exclude>
            </excludes>
            <failIfNoTests>false</failIfNoTests>
            <reuseForks>true</reuseForks>
            <forkCount>2C</forkCount>
            <parallel>suites</parallel>
            <threadCount>12</threadCount>
          </configuration>
        </execution>
        <!-- Sequential tests -->
        <execution>
          <id>sequential-tests</id>
          <phase>test</phase>
          <goals>
            <goal>test</goal>
          </goals>
          <configuration>
            <skip>${skip.sequential.tests}</skip>
            <includes>**/*Sequential.java</includes>
            <reuseForks>false</reuseForks>
          </configuration>
        </execution>
      </executions>
    </plugin>