Collecting all the tests implemented in pytest along with markers

Question

I am looking forward to list all the (pytest) test cases in the form of CSV.

Use case scenario: We already have test cases defined for multiple platforms and databases. These test cases are organized/grouped based on different markers (Ex: platform_linux, platform_centos, db_maria, db_mongo, db_redis etc).

I want to create a CSV file, that contains all the markers and test cases that belong the marker.

Sample of CSV file:

> Marker, Test Cases
> platform_linux, [test_linux1, test_linux2...]
> db_mariadb, [test_mariadb1, test_mariadb2...]

Goal here is to process the CSV file and derive all the tests running on Linux platform with mariadb.

I did get get some answers on collecting test cases (--collect-only) can it be used to create desired CSV?, Do we have existing plugins to achieve the same?

Answer 1

I'm not aware of a suitable plugin (though there are hundreds of them, so it may still exist), but you can implement this yourself by implementing pytest_collection_modifyitems . You get there in the collection phase after all items are collected. The collected test items are passed there as a list, which you may use to write out the csv.

You would probably make this dependent on some option with the file name as value. Here is an example:

top-level conftest.py

import csv

def pytest_addoption(parser):
    parser.addoption("--output-tests-csv", action="store", default=None)

def pytest_collection_modifyitems(config, items):
    csv_out = config.getoption("--output-tests-csv")
    if csv_out:
        tests = {}
        # collect all tests with matching markers
        for item in items:
            for mark in item.iter_markers():
                if mark.name.startswith(("platform_", "db_")):
                    tests.setdefault(mark.name, []).append(item.name)
        # write tests in csv (one line per marker)
        with open(csv_out, 'w', newline='') as csvfile:
            writer = csv.writer(csvfile, delimiter=',')
            for marker, tests in tests.items():
                writer.writerow([marker, tests])

This assumes that the markers to consider start with platform_ or db_ , you may have to adapt this, also the wanted csv format, but this should give you an idea.

Answer 2

There is pytest-csv plugin that provides option to generate required CSV with markers, markers_as_parameters etc.

https://github.com/nicoulaj/pytest-csv

Sample usage (customizing the columns in the CSV):

py.test --csv tests.csv --csv-columns host,function,status,duration,parameters_as_columns

Note: requirement pytest >= 6.0