Use Gradle test-retry plugin in custom test task definition

Question

I have a custom task definition to run specific test files with special settings per test. My task definition looks like this:

task retryTest(type: Test) {
    description = 'Dummy Retry Test'
    group = 'verification'
    maxHeapSize = '2048m'
    include '**/*SpecificIntegrationTest.class'
}

Now, some tests in this setup are flaky and I try do rerun them a second time like this:

plugins {
    id "org.gradle.test-retry" version "1.1.1"
}

task retryTest(type: Test) {
    description = 'Dummy Retry Test'
    group = 'verification'
    maxHeapSize = '2048m'
    include '**/*SpecificIntegrationTest.class'
    test {
        retry {
            maxRetries = 2
        }
    }
}

I wrote a test class that always fails the first time, but succeeds the second time:

public class RetryTest {

    private int execCount = 0;

    @Test
    public void throwException() {
        if (execCount == 0) {
            execCount++;
            throw new NotImplementedException();
        }
    }
}

Unfortunately, the test is only executed once and the complete test suite fails. I can run the tests successfully by using a custom rule as described in https://stackoverflow.com/a/55178053/6059889

Is there some way to use the test-retry plugin with custom task definitions?

Answer 1

Your task config is wrong. It should be:

task retryTest(type: Test) {
    description = 'Dummy Retry Test'
    group = 'verification'
    maxHeapSize = '2048m'
    include '**/*SpecificIntegrationTest.class'
    retry {
        maxRetries = 2
    }
}

Answer 2

task retryTest(type: Test) {
  description = 'Dummy Retry Test'
  group = 'verification'
  maxHeapSize = '2048m'
  include '**/*SpecificIntegrationTest.class'
  reports {
    junitXml {
      mergeReruns = true
    }
  }

  retry {
    maxRetries = 3
  }
}