多线程环境下TestNG重试测试

Question

我一直在使用 TestNG 重试机制如下 -

public class TestRetryAnalyzer implements IRetryAnalyzer {

public static final int MAX_RETRY_COUNT = 3;
private static final AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);

public static void resetCount() {
    count.set(MAX_RETRY_COUNT);
}

public int getCount() {
    return count.get();
}

private boolean isRetryAvailable() {
    return (count.get() > 0);
}

@Override
public boolean retry(ITestResult result) {
    boolean retry = false;
    if (isRetryAvailable()) {
        System.out.println("Going to retry test case: " + result.getMethod() + ", " + (((MAX_RETRY_COUNT - count.get()) + 1)) + " out of " + MAX_RETRY_COUNT);
        retry = true;
        count.decrementAndGet();
    }
    return retry;
}

}

public class TestRetryListener implements IAnnotationTransformer {

@Override
public void transform(final ITestAnnotation annotation, final Class testClass, final Constructor testConstructor,
                      final Method testMethod) {
    IRetryAnalyzer retryAnalyzer = annotation.getRetryAnalyzer();
    if (retryAnalyzer == null) {
        annotation.setRetryAnalyzer(TestRetryAnalyzer.class);
    }
}

和测试 -

public class SimpleTest {

private int count = 0;

@Test
public void test() {
    count++;
    if (count % 3 != 0) {
        Assert.fail("Injected Failure");
    }
    count = 0;
}

和 testng.xml 文件 -

<suite name="Suite1" verbose="1">
<listeners>
    <listener class-name="main.java.TestRetryListener" />
    <listener class-name="org.uncommons.reportng.HTMLReporter" />
    <listener class-name="org.uncommons.reportng.JUnitXMLReporter" />
</listeners>
<test name="Sample" >
    <classes>
        <class name="main.java.SimpleTest" />
    </classes>
</test>
<test name="Sample2" >
    <classes>
        <class name="main.java.SimpleTest" />
    </classes>
</test>
<test name="Sample3" >
    <classes>
        <class name="main.java.SimpleTest" />
    </classes>
</test>
<test name="Sample4" >
    <classes>
        <class name="main.java.SimpleTest" />
    </classes>
</test>
<test name="Sample5" >
    <classes>
        <class name="main.java.SimpleTest" />
    </classes>
</test>

当我只运行一个测试时，重试机制运行良好（1 次通过，2 次跳过）。 但是当我运行上面 testng.xml 文件中提到的 5 个测试时，它们开始失败。 我是否会在没有并行运行测试的情况下遇到并发问题？ 我该如何摆脱它？

我正在使用 testNG 6.9.10

Answer 1

TestNG 框架仅调用一次 TestRetryListener，因此堆中将只存在一个 TestRetryAnalyzer 实例。 多个测试共享实例变量“count”（AtomicInteger 类的实例）。 对于每个测试，它不会重置为 3，这意味着 AtomicInteger 对象仅实例化一次，并且 TestRetryAnalyzer 类使用相同的引用。

为了克服上述问题，我们必须在每次测试后将计数重置为 3。 为此，您必须进行以下更改

public class SimpleTest {

private int count = 0;

@Test
public void test() {
    count++;
    System.out.println("Test Count ... " + count);
    if (count % 4 != 0) {
        Assert.fail("Injected Failure");
    }
    count = 0;
}

@AfterTest
public void afterTest() {
    TestRetryAnalyzer.resetCount();
}

}

公共类 TestRetryAnalyzer 实现了 IRetryAnalyzer {

private static int MAX_RETRY_COUNT = 3;

private static AtomicInteger count = new AtomicInteger(MAX_RETRY_COUNT);

public static void resetCount() {
    count.set(MAX_RETRY_COUNT);
}

private boolean isRetryAvailable() {
    return (count.intValue() > 0);
}

@Override
public boolean retry(ITestResult result) {
    boolean retry = false;
    if (isRetryAvailable()) {
        retry = true;
        count.decrementAndGet();
        System.out.println("Retry Analyzer count is : " + count);
    }
    return retry;
}

}

希望这有帮助:)

Answer 2

我正在使用ThreadLocal实例来处理这个问题。 该解决方案完美运行。

public class RetryAnalyzer implements IRetryAnalyzer {

    private ThreadLocal<Integer> count = ThreadLocal.withInitial(() -> 0);

    private static final int maxTry = TestRunner.getRunConfig().getRerun();
    private static final Logger logger = LogManager.getLogger(RetryAnalyzer.class);

    @Override
    public synchronized boolean retry(ITestResult result) {
        if (!result.isSuccess()) {                      //Check if test not succeed
            if (count.get() < maxTry) {                 //Check if maxtry count is reached
                count.set(count.get() + 1);             //Increase the maxTry count by 1
                result.setStatus(ITestResult.FAILURE);  //Mark test as failed
                logger.debug("Now going to retry " + result.getMethod().getMethodName());
                logger.debug("Retry count is " + count);
                return true;                            //Tells TestNG to re-run the test
            } else {
                result.setStatus(ITestResult.FAILURE);  //If maxCount reached,test marked as failed
                count.set(0);                           //Reset count to 0 for next set of tests
                logger.debug("Reset the retry count for " + result.getMethod().getMethodName());
            }
        } else {
            result.setStatus(ITestResult.SUCCESS);      //If test passes, TestNG marks it as passed
        }
        return false;
    }
}