如果 20% 或第 20 个测试用例测试方法失败，我如何停止 selenium 自动化？

Question

I have 100 test cases, if 20% or first 20 test cases all fail how can I stop the execution? Already have testng ITestResult where should I break the build?

@Test(retryAnalyzer = ReTryFail.class, dataProvider = "SanityTCTest", dataProviderClass = utility.Xlsdataprovider.class, groups = "Dashboard", alwaysRun = true)
public void Sanity_TC001(LinkedHashMap<String, String> data) throws InterruptedException, SQLException {

 Some code 
}

@Test(retryAnalyzer = ReTryFail.class, dataProvider = "SanityTCTest", dataProviderClass = utility.Xlsdataprovider.class, groups = "Dashboard", alwaysRun = true)
public void Sanity_TC002(LinkedHashMap<String, String> data) throws InterruptedException, SQLException {

 Some code 
}

@Test(retryAnalyzer = ReTryFail.class, dataProvider = "SanityTCTest", dataProviderClass = utility.Xlsdataprovider.class, groups = "Dashboard", alwaysRun = true)
public void Sanity_TC003(LinkedHashMap<String, String> data) throws InterruptedException, SQLException {

 Some code 
}
///////////////////////////////

Where can I break this suite if Result, "FAIL is over 20? DO i require to create new class or can I add in same below?

@AfterMethod(alwaysRun = true)
public void reporterDataResults(ITestResult Result) throws InterruptedException {
    boolean flag = false;
    Testfail = TestResultStatus.Testfail;

    /*System.out.println("test fail flag in AfterMethod: " + Testfail); */
    if (Result.getStatus() == ITestResult.SKIP) {
        Resmark.put(Result.getName(), "");
        captureScreenShot(Result, "SKIP", getDriver());
        Reporter.log(Result.getName() + " is SKIPPED");
        Add_Log.info(Result.getName() + " is SKIPPED");
        TestResultTL.put(Result.getName(), "SKIP");
        if (!(getDriver() == null)) {
            closeWebBrowser();
        }
    } else if (Result.getStatus() == ITestResult.FAILURE) {

        Collection<String> values = TestResultStatus.mpaskeysnew.get(Result.getName());
        String resultout = String.join(" | ", values);
        System.out.println(resultout);
        Resmark.put(Result.getName(), resultout);

        captureScreenShot(Result, "FAIL", getDriver());
        Reporter.log(Result.getName() + " is FAIL");
        Add_Log.info(Result.getName() + " is FAIL");
        if (!(getDriver() == null)) {
            closeWebBrowser();
        }
        TestResultTL.put(Result.getName(), "FAIL");


    } else {
        captureScreenShot(Result, "PASS", getDriver());
        Resmark.put(Result.getName(), "");
        Reporter.log(Result.getName() + " is PASS");
        Add_Log.info(Result.getName() + " is PASS");
        if (!(getDriver() == null)) {
            closeWebBrowser();
        }
        TestResultTL.put(Result.getName(), "PASS");

    }
    Testskip = false;
    TestResultStatus.Testfail = false;

}

Answer 1

You can implementISuiteListener and in onFinish method you'll have access to ISuite and ISuiteResult

Then you can do

public void onFinish(ISuite suite) {

        final Map<java.lang.String,ISuiteResult>  res = suite.getResults();

                for (ISuiteResult r : res.values()) {
                  context =  r.getTestContext()  ;
                  failedTestCases =context.getFailedTests().size();
                 }
         }

size() will give you number of failed test for that suite. Once you know that number you can implement to stop execution using strategies in this

If your test cases are in different suites then in each call to onFinish method you can count number of failed test cases per suite and based on that stop execution.

One other alternative is to implement ITestListener . In onTestFailure method you have access to ITestResult

You can count how many times onTestFailure method is called and based on that stop execution. I think implementing ITestListener is more suitable and easy in your case.

Here, I edited to explain how you'd implement listener

import org.testng.ISuiteListener;

public class listener implements Itestlistener {
            public int i = 0;

             public void onTestFailure(ITestResult result) {

                     result.getName();
                     i++;
                     //your break logic goes here
                       if (i ==20){
                              // do something or call some function to stop execution
                             }
         }

     }

You can read more about testng listenershere .

For your EDIT above (if you want to go that way). Though I still think you should implement listener, which is more cleaner. It will be called only when the test fails.

But do the same thing as I did in onTestFailure method, add a counter and increase it in else if .

   public int i = 0;  //do this in your class

then in your method

 else if (Result.getStatus() == ITestResult.FAILURE) {
    i++;  //increase counter here
    Collection<String> values = TestResultStatus.mpaskeysnew.get(Result.getName());
    String resultout = String.join(" | ", values);
    System.out.println(resultout);
    Resmark.put(Result.getName(), resultout);

    captureScreenShot(Result, "FAIL", getDriver());
    Reporter.log(Result.getName() + " is FAIL");
    Add_Log.info(Result.getName() + " is FAIL");
    if (!(getDriver() == null)) {
        closeWebBrowser();
    }
       TestResultTL.put(Result.getName(), "FAIL");
      if (i==20){
         // stop execution here
      }

  }