[英]stop pytest tests if several tests have a specific exception
I want to stop the tests suite using pytest.exit()
if any of the tests fails with a specific exception.如果任何测试因特定异常而失败,我想使用
pytest.exit()
停止测试套件。
For example: 50 tests, any of them can fail at some point with that exception, I want to stop the execution if at least 2 of these tests fail over this exception.例如:50 个测试,其中任何一个测试都可能在某个时候因该异常而失败,如果这些测试中至少有 2 个因该异常而失败,我想停止执行。
I've tried to keep a global counter ( a fixture with scope='session'
) between tests and update it every time I catch this exception, but was unable to keep it's value between tests.我试图在测试之间保留一个全局计数器(具有
scope='session'
的夹具),并在每次捕获此异常时更新它,但无法在测试之间保持它的值。
any ideas?有任何想法吗?
This is possible through the use of Hooks .这可以通过使用Hooks来实现。
Specifically we will take advantage of two specific hooks, pytest_sessionstart
and pytest_exception_interact
.具体来说,我们将利用两个特定的钩子,
pytest_sessionstart
和pytest_exception_interact
。 We will use pytest_sessionstart
to keep track of the number of specific exceptions we are willing to tolerate, think of this as the place to store the "global counter" that you mention.我们将使用
pytest_sessionstart
来跟踪我们愿意容忍的特定异常的数量,将其视为存储您提到的“全局计数器”的地方。 The other hook, pytest_exception_interact
will be used to interact with failed tests to check the type of exception that was returned, if any.另一个钩子
pytest_exception_interact
将用于与失败的测试进行交互,以检查返回的异常类型(如果有)。
Create a conftest.py
file in the root directory of your test folder and place the following:在测试文件夹的根目录中创建一个
conftest.py
文件并放置以下内容:
import pytest
EXC_MAP = {
ZeroDivisionError: 1,
KeyError: 1
}
def pytest_sessionstart(session):
session.__exc_limits = EXC_MAP
def pytest_exception_interact(node, call, report):
session = node.session
type_ = call.excinfo.type
if type_ in session.__exc_limits:
if session.__exc_limits[type_] == 0:
pytest.exit(f"Reached max exception for type: {type_}")
else:
session.__exc_limits[type_] -= 1
The EXC_MAP
is the map of exceptions --> failures that we are willing to allow in our test invocation. EXC_MAP
是异常的 map --> 我们愿意在测试调用中允许的失败。 In the pytest_sessionstart
hook we set a private variable on the session
to keep track of these variables.在
pytest_sessionstart
钩子中,我们在session
上设置了一个私有变量来跟踪这些变量。 In the pytest_exception_interact
hook we get the type of exception that was raised by the test, we check it against the thresholds, and if the count for that exception has reached 0, we exit from pytest
, skipping the remained of the tests.在
pytest_exception_interact
钩子中,我们得到测试引发的异常类型,我们根据阈值检查它,如果该异常的计数达到 0,我们从pytest
退出,跳过剩余的测试。
Below is an example test script and the output in the terminal.下面是一个示例测试脚本和终端中的 output。
def foo(a, b):
return a / b
def test_foo():
result = foo(1, 0)
assert result == 1
def test_foo1():
result = foo(1, 1)
assert result == 1
def test_foo2():
result = foo(1, 0)
assert result == 1
def test_foo3():
result = foo(1, 1)
assert result == 1
And when running these the terminal shows:运行这些时,终端显示:
collected 4 items
test_foo.py F.F
========================================================================== FAILURES ==========================================================================
__________________________________________________________________________ test_foo __________________________________________________________________________
def test_foo():
> result = foo(1, 0)
test_foo.py:6:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = 1, b = 0
def foo(a, b):
> return a / b
E ZeroDivisionError: division by zero
test_foo.py:2: ZeroDivisionError
_________________________________________________________________________ test_foo2 __________________________________________________________________________
def test_foo2():
> result = foo(1, 0)
test_foo.py:16:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = 1, b = 0
def foo(a, b):
> return a / b
E ZeroDivisionError: division by zero
test_foo.py:2: ZeroDivisionError
================================================================== short test summary info ===================================================================
FAILED test_foo.py::test_foo - ZeroDivisionError: division by zero
FAILED test_foo.py::test_foo2 - ZeroDivisionError: division by zero
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: Reached max exception for type: <class 'ZeroDivisionError'> !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================ 2 failed, 1 passed in 0.20s =================================================================
We can see that it collected all 4 tests, but only ran 3 of them because the threshold for ZeroDivisionError
was reached prior to running the final test.我们可以看到它收集了所有 4 个测试,但只运行了其中的 3 个,因为在运行最终测试之前已经达到
ZeroDivisionError
的阈值。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.