Pytest - tmpdir_factory in pytest_generate_tests

Question

So I have two main portion of code:

1. Generates an extensive collection of config files in a directory.
2. Runs a single config file generated by the previous.

I want to run a test, where first I execute code1 and generate all files and than for each config file run code2 and verify that the results if good. My attempt so far was:

@pytest.fixture(scope='session')
def pytest_generate_tests(metafunc, tmpdir_factory):
    path = os.path.join(tmpdir_factory, "configs")
    gc.main(path,
            gc.VARIANTS, gc.MODELS,
            default_curvature_avg=0.0,
            curvature_avg_variation=0.9,
            default_gradient_avg=0.0,
            gradient_avg_variation=0.9,
            default_inversion="approximate",
            vary_inversion=False,
            vary_projections=True)
    params = []
    for model in os.listdir(path):
        model_path = os.path.join(path, model)
        for dataset in os.listdir(model_path):
            dataset_path = os.path.join(model_path, dataset)
            for file_name in os.listdir(dataset_path):
                config_file = os.path.join(dataset_path, file_name)
                folder = os.path.join(dataset_path, file_name[:-5])
                tmpdir_factory.mktemp(folder)
                params.append(dict(config_file=config_file, output_folder=folder))
                metafunc.addcall(funcargs=dict(config_file=config_file, output_folder=folder))

def test_compile_and_error(config_file, output_folder):
    final_error = main(config_file, output_folder)
    assert final_error < 0.9

However, the tmpdir_factory fixture does not work for the pytest_generate_tests method. My questions is how to achieve my goal by generating all of the tests?

Answer 1

First and most importantly, pytest_generate_tests is meant to be a hook in pytest and not a name for a fixture function. Get rid of the @pytest.fixture before it and have another look in its docs . Hooks should be written in the conftest.py file or a plugin file, and are collected automatically according to the pytest_ prefix.

Now for your matter: Just use a temporary directory manually using:

import tempfile
import shutil

dirpath = tempfile.mkdtemp()

inside pytest_generate_tests . Save dirpath in a global in the conftest, and delete in pytest_sessionfinish using

# ... do stuff with dirpath
shutil.rmtree(dirpath)

Source: https://stackoverflow.com/a/3223615/3858507

Remember that if you have more than one test case, pytest_generate_tests will be called for each one. So you better save all your tempdirs in some list and delete all of them in the end. In contrast, if you only need one tempdir than think about using the hook pytest_sesssionstart to create it there and use it later.