Collecting pytest results in pandas dataframe

Question

I would like to parse and collect pytest results in pandas dataframe. Is there any way to parse it ?

I only found this reference, which am not sure how to use it. Collecting and Reporting pytest Results

Answer 1

You could implement a custom makereport hook in your conftest.py . A simple example:

import pytest
import pandas as pd


df = pd.DataFrame(columns=('failed', 'nodeid', ))


@pytest.hookimpl(tryfirst=True, hookwrapper=True)
def pytest_runtest_makereport(item, call):
    global df
    outcome = yield
    rep = outcome.get_result()

    if rep.when == 'call':
        df = df.append({'failed': rep.failed, 'nodeid': rep.nodeid}, ignore_index=True)

Answer 2

You can use pytest-harvest for this. Simply install it, and you can directly use the pre-defined fixtures:

import pytest
import time

@pytest.mark.parametrize('p', ['world', 'self'], ids=str)
def test_foo(p):
    """
    A dummy test, parametrized so that it is executed twice
    """
    print('\n   hello, ' + p + ' !')
    time.sleep(len(p) / 10)

def test_synthesis(module_results_df):
    """
    Shows that the `module_results_df` fixture already contains what you need
    """
    # drop the 'pytest_obj' column
    module_results_df.drop('pytest_obj', axis=1, inplace=True)

    print("\n   `module_results_df` dataframe:\n")
    print(module_results_df)

Yields

>>> pytest -s -v

============================= test session starts =============================
...
collecting ... collected 3 items
test_basic.py::test_foo[world] 
   hello, world !
PASSED
test_basic.py::test_foo[self] 
   hello, self !
PASSED
test_basic.py::test_synthesis 
   `module_results_df` dataframe:

                 status  duration_ms      p
test_id                                    
test_foo[world]  passed   500.028610  world
test_foo[self]   passed   400.022745   self
PASSED

========================== 3 passed in 0.05 seconds ===========================

You can also start from the 'dict' fixture, which contains more details about setup/teardown times, and convert it to a dataframe using the provided helper methods. See documentation for details.

Finally, if you wish to also use parameters, fixtures, steps... you might wish to look at this datascience benchmark example

I'm the author by the way ;)