Skip parametrized tests generated by pytest_generate_tests at module level

Question

I would like to be able to parametrize a couple of tests from a config file, but at the same time being able to skip those tests unless a specific command option is issued.

I can skip tests by adding the following code at the top of the test module:

from json import loads
import pytest
@pytest.mark.skipif(pytest.config.getvalue("-k") != "smoke",
                    reason="Smoke tests must be explicitly launched through -k smoke option")

The tests are not executed when issuing py.test or python -m pytest unless the option -k smoke is added.

I also can create parametrized tests from config files by:

def pytest_generate_tests(metafunc):
    with open('tests/test_smoke.json','r') as fp:
        confs = loads(fp.read().decode("utf-8-sig"))

        for arg in metafunc.funcargnames:
            if arg == "conf":
                metafunc.parametrize("conf",confs)

And example of test that is parametrized would be:

def test_that_require_conf(conf):
    assert not conf

The problem is that both things do not work nicely together. The test is not skipped when pytest_generate_tests is used. If I add an option in pytest_generate_tests in order to avoid the parametrization, then the call yo pytest fails because conf fixture required by test_that_require_conf can not be found.

Any idea about how to achieve this?

Answer 1

I see two options: (I supposed that your option is stored as smoke )

1) in the first option you need to change your pytest_generate_tests . The tests will be skipped as one

def pytest_generate_tests(metafunc):
    for arg in metafunc.funcargnames:
         if arg == "conf":
            if metafunc.config.option.keyword != 'smoke':
                confs = pytest.skip("Smoke tests must....")
            else:
                with open('tests/test_smoke.json', 'r') as fp:
                    confs = loads(fp.read().decode("utf-8-sig"))

            metafunc.parametrize("conf", confs)

the output will be:

collected 0 items / 1 skipped

==================== 1 skipped in 0.01 seconds ========================

2) The second options will skip any test individually

def test_that_require_conf(request, conf):
    if request.config.option.smoke != 'smoke':
        pytest.skip('Smoke tests must....")
    assert conf

the output will

collected 3 items

tests/test_2.py::test_that_require_conf[1] SKIPPED
tests/test_2.py::test_that_require_conf[2] SKIPPED
tests/test_2.py::test_that_require_conf[3] SKIPPED

====================== 3 skipped in 0.02 seconds ======================