[英]How to parametrize tests with json array test data using pytest in python?
[英]Using pytest's parametrize, how can I skip the remaining tests if one test case fails?
我正在使用pytest.mark.parametrize
将越来越长的输入输入到一个相当慢的测试函数中,如下所示:
@pytest.mark.parametrize('data', [
b'ab',
b'xyz'*1000,
b'12345'*1024**2,
... # etc
])
def test_compression(data):
... # compress the data
... # decompress the data
assert decompressed_data == data
因为压缩大量数据需要很长时间,所以我想在一次失败后跳过所有剩余的测试。 例如,如果测试因输入b'ab'
(第一个)而失败,则b'xyz'*1000
和b'12345'*1024**2
以及所有其他参数化都应跳过(或 xfail 而不执行)。
我知道可以将标记附加到单个参数化,如下所示:
@pytest.mark.parametrize("test_input,expected", [
("3+5", 8),
("2+4", 6),
pytest.param("6*9", 42, marks=pytest.mark.xfail),
])
但我不知道如何根据前一个测试用例的状态有条件地应用这些标记。 有没有办法做到这一点?
在执行测试之前评估标记,因此无法传递某种依赖于其他测试结果的声明性标记(如skipif
)。 不过,您可以在挂钩中应用自定义测试跳过逻辑。 修改增量测试 - pytest
文档中的测试步骤配方:
# conftest.py
import pytest
def pytest_sessionstart(session):
session.failednames = set()
def pytest_runtest_makereport(item, call):
if call.excinfo is not None:
item.session.failednames.add(item.originalname)
def pytest_runtest_setup(item):
if item.originalname in item.session.failednames:
pytest.skip("previous test failed (%s)" % item.name) # or use pytest.xfail like in the other answer
示例测试
@pytest.mark.parametrize('i', range(10))
def test_spam(i):
assert i != 3
产量:
=================================== test session starts ===================================
collected 10 items
test_spam.py::test_spam[0] PASSED
test_spam.py::test_spam[1] PASSED
test_spam.py::test_spam[2] PASSED
test_spam.py::test_spam[3] FAILED
test_spam.py::test_spam[4] SKIPPED
test_spam.py::test_spam[5] SKIPPED
test_spam.py::test_spam[6] SKIPPED
test_spam.py::test_spam[7] SKIPPED
test_spam.py::test_spam[8] SKIPPED
test_spam.py::test_spam[9] SKIPPED
========================================= FAILURES ========================================
_______________________________________ test_spam[3] ______________________________________
i = 3
@pytest.mark.parametrize('i', range(10))
def test_spam(i):
> assert i != 3
E assert 3 != 3
test_spam.py:5: AssertionError
====================== 1 failed, 3 passed, 6 skipped in 0.06 seconds ======================
def pytest_runtest_makereport(item, call):
markers = {marker.name for marker in item.iter_markers()}
if call.excinfo is not None and 'skiprest' in markers:
item.session.failednames.add(item.originalname)
def pytest_runtest_setup(item):
markers = {marker.name for marker in item.iter_markers()}
if item.originalname in item.session.failednames and 'skiprest' in markers:
pytest.skip(item.name)
用法:
@pytest.mark.skiprest
@pytest.mark.parametrize('somearg', ['a', 'b', 'c'])
def test_marked(somearg):
...
编辑:我将我的解决方案调整为通过 usint pytest.skip
在夹具中接受的解决方案。
以下黑客可以解决您的问题。 假测试函数test_compression
总是失败,但只执行一次。 但它要求您在测试函数中添加alrady_failed
夹具的检查:
import pytest
@pytest.fixture(scope="function")
def skip_if_already_failed(request, failed=set()):
key = request.node.name.split("[")[0]
failed_before = request.session.testsfailed
if key in failed:
pytest.skip("previous test {} failed".format(key))
yield
failed_after = request.session.testsfailed
if failed_before != failed_after:
failed.add(key)
@pytest.mark.parametrize("data", [1, 2, 3, 4, 5, 6])
def test_compression(data, skip_if_already_failed):
assert data < 3
这是输出:
$ py.test -v sopytest.py
================================== test session starts ==================================
platform darwin -- Python 3.6.6, pytest-3.8.0, py-1.6.0, pluggy-0.7.1 -- ...
cachedir: .pytest_cache
rootdir: ..., inifile:
collected 6 items
sopytest.py::test_compression[1] PASSED [ 16%]
sopytest.py::test_compression[2] PASSED [ 33%]
sopytest.py::test_compression[3] FAILED [ 50%]
sopytest.py::test_compression[4] SKIPPED [ 66%]
sopytest.py::test_compression[5] SKIPPED [ 83%]
sopytest.py::test_compression[6] SKIPPED [100%]
======================================= FAILURES ========================================
__________________________________ test_compression[3] __________________________________
data = 3, skip_if_already_failed = None
@pytest.mark.parametrize("data", [1, 2, 3, 4, 5, 6])
def test_compression(data, skip_if_already_failed):
> assert data < 3
E assert 3 < 3
sopytest.py:18: AssertionError
===================== 1 failed, 2 passed, 3 skipped in 0.08 seconds =====================
接受的答案不再有效, item.session
没有failednames
参数。
灵感来自https://docs.pytest.org/en/latest/example/simple.html#incremental-testing-test-steps
还有一种与公认的非常接近的解决方案:
conftest.py
MARKER_SKIPREST = "skiprest"
MARKER_PARAMETRIZE = "parametrize"
_failed_parametrized_tests = set() # History of failed tests
def pytest_runtest_makereport(item, call):
"""Memorizes names of failed Parametrized tests"""
marker_names = {marker.name for marker in item.iter_markers()}
if {MARKER_SKIPREST, MARKER_PARAMETRIZE}.issubset(marker_names):
if call.excinfo is not None:
_failed_parametrized_tests.add(item.originalname)
def pytest_runtest_setup(item):
"""Check if the test has already failed with other param.
If yes - xfail this test"""
marker_names = {marker.name for marker in item.iter_markers()}
if {MARKER_SKIPREST, MARKER_PARAMETRIZE}.issubset(marker_names):
if item.originalname in _failed_parametrized_tests:
pytest.xfail("Previous test failed")
setup.cfg
[tool:pytest]
markers =
skiprest: Skip rest of params in parametrized test if one test one of the tests in the sequence has failed.
test_?.py
pytest.mark.skiprest
pytest.mark.parametrize('data', [
b'ab',
b'xyz'*1000,
b'12345'*1024**2,
... # etc
])
def test_compression(data):
...
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.