按自定义顺序运行 Pytest 类

Question

I am writing tests with pytest in pycharm. The tests are divided into various classes.
I would like to specify certain classes that have to run before other classes .
I have seen various questions on stackoverflow (such as specifying pytest tests to run from a file and how to run a method before all other tests ).
These and various others questions wanted to choose specific functions to run in order. This can be done, I understand, using fixtures or with pytest ordering .
I don't care which functions from each class run first. All I care about is that the classes run in the order I specify. Is this possible?

Answer 1

Approach

You can use the pytest_collection_modifyitems hook to modify the order of collected tests ( items ) in place. This has the additional benefit of not having to install any third party libraries.

With some custom logic, this allows to sort by class.

Full example

Say we have three test classes:

1. TestExtract
2. TestTransform
3. TestLoad

Say also that, by default, the test order of execution would be alphabetical, ie:

TestExtract -> TestLoad -> TestTransform

which does not work for us due to test class interdependencies.

We can add pytest_collection_modifyitems to conftest.py as follows to enforce our desired execution order:

#conftest.py
def pytest_collection_modifyitems(items):
    """Modifies in place test items to execute test classes in a given order."""
    CLASS_ORDER = ["TestExtract", "TestTransform", "TestLoad"]

    # Enforce module order by iteratively pushing tests in a class to
    # the end of the execution queue
    sorted_items = items.copy()
    for class_ in CLASS_ORDER:
        items_class = [item.cls.__name__ for item in sorted_items]
        first_index = items_class.index(class_)
        last_index = len(items_class) - items_class[::-1].index(class_)
        sorted_items = (
            sorted_items[:first_index]
            + sorted_items[last_index:]
            + sorted_items[first_index:last_index]
        )
    items[:] = sorted_items

Some comments on the implementation details:

• items_class maps the class of each item
• first_index and last_index refer to the first and last indices of items belonging to the test class under consideration
• It is assumed that test classes have unique names
• Test classes can live in different modules
• CLASS_ORDER does not have to be exhaustive. You can reorder just those classes on which to want to enforce an order (but note: if reordered, any non-reordered class will execute before any reordered class)
• The test order within the classes is kept unchanged
• The sorting algorithm might not be as efficient as it could be (but honestly I have yet to find a test suite in which this could possibly matter)
• items must be modified in place, hence the final items[:] assignment