堆栈内存溢出
  繁体   English   中英

testng-跳过多个测试

[英]testng - skip multiple tests

 原文  2019-03-14 10:41:48  9   2    java/ selenium/ testng

问题描述

我有一个测试案例，如果失败，将有100多个测试失败。 这100个测试属于不同的测试类别。 在所有这些测试上都使用dependsOnMethod很麻烦。 如果第一个测试失败，是否有更可维护的应用范围来跳过这100个测试？

信息：

  • 所有测试都在同一个测试套件中。
  • 测试类在不同的<test>

2 个解决方案

解决方案1
 1  2019-03-16 06:33:17

这是你如何做到这一点：

  1. 创建一个IInvokedMethodListener的实现，并在其beforeInvocation检查是否已设置一个属性（用于检测失败）以指示失败。 如果设置了此属性，则将测试方法标记为已跳过，为其设置例外并从beforeInvocation引发异常
  2. 在afterInvocation()实现中，检查执行的测试方法是否失败。 如果是，则在套件对象级别设置failure属性。

只需确保不与此设置并行运行测试，因为当测试同时运行时，此逻辑将不起作用。

下面的示例显示了所有这些操作。

这是检测监听器的失败。 

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

public class EarlyFailureDetectingListener implements IInvokedMethodListener {
  private static final String SKIP_FURTHER_EXECUTION = "skipFurtherExecution";

  @Override
  public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
    if (testResult
        .getTestContext()
        .getSuite()
        .getAttributeNames()
        .contains(SKIP_FURTHER_EXECUTION)) {
      testResult.setStatus(ITestResult.SKIP);
      IllegalStateException exception = new IllegalStateException("Failure detected");
      testResult.setThrowable(exception);
      String msg =
          String.format(
              "Skipping execution of %s.%s()",
              method.getTestMethod().getTestClass().getRealClass().getName(),
              method.getTestMethod().getMethodName());
      System.err.println(msg);
      throw exception;
    }
  }

  @Override
  public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
    if (testResult.getStatus() == ITestResult.FAILURE) {
      testResult.getTestContext().getSuite().setAttribute(SKIP_FURTHER_EXECUTION, Boolean.TRUE);
    }
  }
}

测试类如下所示： 

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestClassOne {

  @Test
  public void testMethod() {
    Assert.fail("Simulating a failure");
  }

  @Test
  public void anotherTestMethod() {
    TestClassTwo.printer();
  }
}

import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class TestClassTwo {

  @Test
  public void testMethod() {
    printer();
  }

  @Test
  public void anotherTestMethod() {
    printer();
  }

  static void printer() {
    ITestResult result = Reporter.getCurrentTestResult();
    String msg =
        String.format(
            "Executing ====> %s.%s() on Thread [%d]",
            result.getTestClass().getRealClass().getName(),
            result.getMethod().getMethodName(),
            Thread.currentThread().getId());
    System.err.println(msg);
  }
}

import org.testng.ITestResult;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class TestClassThree {

  @Test
  public void testMethod() {
    printer();
  }

  @Test
  public void anotherTestMethod() {
    printer();
  }

  static void printer() {
    ITestResult result = Reporter.getCurrentTestResult();
    String msg =
        String.format(
            "Executing ====> %s.%s() on Thread [%d]",
            result.getTestClass().getRealClass().getName(),
            result.getMethod().getMethodName(),
            Thread.currentThread().getId());
    System.err.println(msg);
  }
}

这是套件xml 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="55160447_suite" parallel="false" verbose="2">
  <listeners>
    <listener
      class-name="com.rationaleemotions.stackoverflow.qn55160447.EarlyFailureDetectingListener"/>
  </listeners>
  <test name="55160447_test_one">
    <classes>
      <class name="com.rationaleemotions.stackoverflow.qn55160447.TestClassOne">
      </class>
    </classes>
  </test>
  <test name="55160447_test_two">
    <classes>
      <class name="com.rationaleemotions.stackoverflow.qn55160447.TestClassTwo">
      </class>
    </classes>
  </test>
  <test name="55160447_test_three">
    <classes>
      <class name="com.rationaleemotions.stackoverflow.qn55160447.TestClassThree">
      </class>
    </classes>
  </test>
</suite>

这是执行输出： 

...
... TestNG 7.0.0-beta3 by Cédric Beust (cedric@beust.com)
...
Executing ====> com.rationaleemotions.stackoverflow.qn55160447.TestClassOne.anotherTestMethod() on Thread [1]

java.lang.AssertionError: Simulating a failure

    at org.testng.Assert.fail(Assert.java:97)
    at com.rationaleemotions.stackoverflow.qn55160447.TestClassOne.testMethod(TestClassOne.java:10)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:131)
    at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:570)
    at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:170)
    at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:790)
    at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:143)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
    at org.testng.TestRunner.privateRun(TestRunner.java:763)
    at org.testng.TestRunner.run(TestRunner.java:594)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:398)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:392)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
    at org.testng.SuiteRunner.run(SuiteRunner.java:304)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1146)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.runSuites(TestNG.java:997)
    at org.testng.TestNG.run(TestNG.java:965)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

PASSED: anotherTestMethod
FAILED: testMethod
java.lang.AssertionError: Simulating a failure
    at org.testng.Assert.fail(Assert.java:97)
    at com.rationaleemotions.stackoverflow.qn55160447.TestClassOne.testMethod(TestClassOne.java:10)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:131)
    at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:570)
    at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:170)
    at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:790)
    at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:143)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
    at org.testng.TestRunner.privateRun(TestRunner.java:763)
    at org.testng.TestRunner.run(TestRunner.java:594)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:398)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:392)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
    at org.testng.SuiteRunner.run(SuiteRunner.java:304)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1146)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.runSuites(TestNG.java:997)
    at org.testng.TestNG.run(TestNG.java:965)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)


Skipping execution of com.rationaleemotions.stackoverflow.qn55160447.TestClassTwo.anotherTestMethod()


===============================================Skipping execution of com.rationaleemotions.stackoverflow.qn55160447.TestClassTwo.testMethod()
    55160447_test_one
    Tests run: 2, Failures: 1, Skips: 0
===============================================

Test ignored.

Test ignored.
SKIPPED: anotherTestMethod
java.lang.IllegalStateException: Failure detected
    at com.rationaleemotions.stackoverflow.qn55160447.EarlyFailureDetectingListener.beforeInvocation(EarlyFailureDetectingListener.java:18)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker$InvokeBeforeInvocationWithoutContextStrategy.callMethod(InvokedMethodListenerInvoker.java:88)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker.invokeListener(InvokedMethodListenerInvoker.java:61)
    at org.testng.internal.BaseInvoker.runInvokedMethodListeners(BaseInvoker.java:55)
    at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:547)
    at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:170)
    at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:790)
    at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:143)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
    at org.testng.TestRunner.privateRun(TestRunner.java:763)
    at org.testng.TestRunner.run(TestRunner.java:594)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:398)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:392)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
    at org.testng.SuiteRunner.run(SuiteRunner.java:304)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1146)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.runSuites(TestNG.java:997)
    at org.testng.TestNG.run(TestNG.java:965)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

SKIPPED: testMethod
java.lang.IllegalStateException: Failure detected
    at com.rationaleemotions.stackoverflow.qn55160447.EarlyFailureDetectingListener.beforeInvocation(EarlyFailureDetectingListener.java:18)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker$InvokeBeforeInvocationWithoutContextStrategy.callMethod(InvokedMethodListenerInvoker.java:88)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker.invokeListener(InvokedMethodListenerInvoker.java:61)
    at org.testng.internal.BaseInvoker.runInvokedMethodListeners(BaseInvoker.java:55)
    at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:547)
    at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:170)
    at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:790)
    at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:143)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
    at org.testng.TestRunner.privateRun(TestRunner.java:763)
    at org.testng.TestRunner.run(TestRunner.java:594)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:398)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:392)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
    at org.testng.SuiteRunner.run(SuiteRunner.java:304)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1146)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.runSuites(TestNG.java:997)
    at org.testng.TestNG.run(TestNG.java:965)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)


===============================================
    55160447_test_two
    Tests run: 2, Failures: 0, Skips: 2
===============================================

Test ignored.

Skipping execution of com.rationaleemotions.stackoverflow.qn55160447.TestClassThree.anotherTestMethod()
Skipping execution of com.rationaleemotions.stackoverflow.qn55160447.TestClassThree.testMethod()

Test ignored.
SKIPPED: anotherTestMethod
java.lang.IllegalStateException: Failure detected
    at com.rationaleemotions.stackoverflow.qn55160447.EarlyFailureDetectingListener.beforeInvocation(EarlyFailureDetectingListener.java:18)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker$InvokeBeforeInvocationWithoutContextStrategy.callMethod(InvokedMethodListenerInvoker.java:88)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker.invokeListener(InvokedMethodListenerInvoker.java:61)
    at org.testng.internal.BaseInvoker.runInvokedMethodListeners(BaseInvoker.java:55)
    at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:547)
    at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:170)
    at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:790)
    at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:143)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
    at org.testng.TestRunner.privateRun(TestRunner.java:763)
    at org.testng.TestRunner.run(TestRunner.java:594)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:398)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:392)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
    at org.testng.SuiteRunner.run(SuiteRunner.java:304)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1146)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.runSuites(TestNG.java:997)
    at org.testng.TestNG.run(TestNG.java:965)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

SKIPPED: testMethod
java.lang.IllegalStateException: Failure detected
    at com.rationaleemotions.stackoverflow.qn55160447.EarlyFailureDetectingListener.beforeInvocation(EarlyFailureDetectingListener.java:18)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker$InvokeBeforeInvocationWithoutContextStrategy.callMethod(InvokedMethodListenerInvoker.java:88)
    at org.testng.internal.invokers.InvokedMethodListenerInvoker.invokeListener(InvokedMethodListenerInvoker.java:61)
    at org.testng.internal.BaseInvoker.runInvokedMethodListeners(BaseInvoker.java:55)
    at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:547)
    at org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:170)
    at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
    at org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:790)
    at org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:143)
    at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
    at org.testng.TestRunner.privateRun(TestRunner.java:763)
    at org.testng.TestRunner.run(TestRunner.java:594)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:398)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:392)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:355)
    at org.testng.SuiteRunner.run(SuiteRunner.java:304)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1146)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1067)
    at org.testng.TestNG.runSuites(TestNG.java:997)
    at org.testng.TestNG.run(TestNG.java:965)
    at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
    at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)


===============================================
    55160447_test_three
    Tests run: 2, Failures: 0, Skips: 2
===============================================

Test ignored.

===============================================
55160447_suite
Total tests run: 6, Passes: 1, Failures: 1, Skips: 4
===============================================


Process finished with exit code 0

解决方案2
 -1  2019-03-14 14:30:11

虽然我仍然建议使用dependsOnMethods或以不同方式构建您的测试来解决您的问题，但您也可以以编程方式跳过这些测试。

为此，请设置MethodInterceptor 。 在TestContext中，您可以调用setAttribute来存储一个指示测试失败的值，然后在拦截器中，可以通过更改相应返回的列表来跳过任何方法。

但正如我在开始时所说的，与相当简单的dependsOnMethods方法相比，考虑增加的额外复杂性。 提醒自己，测试也可以用来记录事物。

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 跳过 IAnnotationTransformer TestNG 中的测试或课程 Testng：如果有异常，请跳过/失败的其余测试 如何在 TestNG 中跳过或忽略测试的执行 @AfterClass 中的失败导致其余测试跳过，TestNG 多次运行TestNG测试 TestNG在多个类别中测试优先级 使用Gradle和TestNG对多个节点进行分区测试 您可以使用testng工厂创建多个测试吗？ Selenium - TestNG - 如何在并行使用中根据 Maven 命令行传递的参数跳过/禁用一些测试 在运行时跳过TestNg测试
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM