如何通过检查传递给pytest_runtest_teardown的Item对象来确定测试是通过还是失败？

Question

Pytest allows you to hook into the teardown phase for each test by implementing a function called pytest_runtest_teardown in a plugin:

def pytest_runtest_teardown(item, nextitem):
    pass

Is there an attribute or method on item that I can use to determine whether the test that just finished running passed or failed? I couldn't find any documentation for pytest.Item and hunting through the source code and playing around in ipdb didn't reveal anything obvious.

Answer 1

You may also consider call.excinfo in pytest_runtest_makereport:

def pytest_runtest_makereport(item, call):
    if call.when == 'setup':
        print('Called after setup for test case is executed.')
    if call.when == 'call':
        print('Called after test case is executed.')
        print('-->{}<--'.format(call.excinfo)) 
    if call.when == 'teardown':
        print('Called after teardown for test case is executed.')

The call object contains a whole bunch of additional information (test start time, stop time, etc.).

Refer: http://doc.pytest.org/en/latest/_modules/_pytest/runner.html

def pytest_runtest_makereport(item, call):
    when = call.when
    duration = call.stop-call.start
    keywords = dict([(x,1) for x in item.keywords])
    excinfo = call.excinfo
    sections = []
    if not call.excinfo:
        outcome = "passed"
        longrepr = None
    else:
        if not isinstance(excinfo, ExceptionInfo):
            outcome = "failed"
            longrepr = excinfo
        elif excinfo.errisinstance(pytest.skip.Exception):
            outcome = "skipped"
            r = excinfo._getreprcrash()
            longrepr = (str(r.path), r.lineno, r.message)
        else:
            outcome = "failed"
            if call.when == "call":
                longrepr = item.repr_failure(excinfo)
            else: # exception in setup or teardown
                longrepr = item._repr_failure_py(excinfo,
                                            style=item.config.option.tbstyle)
    for rwhen, key, content in item._report_sections:
        sections.append(("Captured %s %s" %(key, rwhen), content))
    return TestReport(item.nodeid, item.location,
                      keywords, outcome, longrepr, when,
                      sections, duration)

Answer 2

The Node class don't have any information regarding the status of the last test, however we do have the status of the total number of failed tests (in item.session.testsfailed ), and we can use it:

1. We can add a new member to the item.session object (not so nice, but you gotta love python!). This member will save the status of the last testsfailed - item.session.last_testsfailed_status .
2. If testsfailed > last_testsfailed_status - the last test the run just failed.

---

import pytest
import logging

logging.basicConfig(
    level='INFO',
    handlers=(
        logging.StreamHandler(),
        logging.FileHandler('log.txt')
    )
)

@pytest.mark.hookwrapper
def pytest_runtest_teardown(item, nextitem):
    outcome = yield

    if not hasattr(item.session, 'last_testsfailed_status'):
        item.session.last_testsfailed_status = 0

    if item.session.testsfailed and item.session.testsfailed > item.session.last_testsfailed_status:
        logging.info('Last test failed')

    item.session.last_testsfailed_status = item.session.testsfailed

Answer 3

Initially, I was also struggling to get the Test Status, so that I can use it to make a custom report.
But, after further analysis of pytest_runtest_makereport hook function, I was able to see various attributes of 3 params (item, call, and report). Let me just list some of it out:

1. Call:
   • excinfo (this further drills down to carry traceback if any)
   • start (start time of the test in float value since epoch time)
   • stop (stop time of the test in float value since epoch time)
   • when (can take values - setup, call, teardown)
2. item:
   • _fixtureinfo (contains info abt any fixtures you have used)
   • nodeid (the test_name assumed by pytest)
   • cls (contains the class info of test, by info I mean the variables which were declared and accessed in the class of test)
   • funcargs (what parameters you have passed to your test along with its values)
3. report:
   • outcome (this carries the test status)
   • longrepr (contains the failure info including the traceback)
   • when (can take values - setup, call, teardown. please note that depending on its value the report will carry the values)

FYI: there are other attributes for all the above 3 params, I have mentioned in few. Below is the code snipped depicting, of how I have hooked the function and used.

def pytest_runtest_makereport(item, call, __multicall__):
report = __multicall__.execute()

if (call.when == "call") and hasattr(item, '_failed_expect'):
    report.outcome = "failed"
    summary = 'Failed Expectations:%s' % len(item._failed_expect)
    item._failed_expect.append(summary)
    report.longrepr = str(report.longrepr) + '\n' + ('\n'.join(item._failed_expect))

if call.when == "call":
    ExTest.name = item.nodeid
    func_args = item.funcargs
    ExTest.parameters_used = dict((k, v) for k, v in func_args.items() if v and not hasattr(v, '__dict__'))
    # [(k, v) for k, v in func_args.items() if v and not hasattr(v, '__dict__')]
    t = datetime.fromtimestamp(call.start)
    ExTest.start_timestamp = t.strftime('%Y-%m-%d::%I:%M:%S %p')
    ExTest.test_status = report.outcome
    # TODO Get traceback info (call.excinfo.traceback)  
    return report

Answer 4

Hook wrappers are the way to go - allow all the default hooks to run & then look at their results.

Below example shows 2 methods for detecting whether a test has failed (add it to your conftest.py )

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    # Because this is a hookwrapper, calling `yield` lets the actual hooks run & returns a `_Result`
    result = yield
    # Get the actual `TestReport` which the hook(s) returned, having done the hard work for you
    report = result.get_result()

    # Method 1: `report.longrepr` is either None or a failure representation
    if report.longrepr:
        logging.error('FAILED: %s', report.longrepr)
    else:
        logging.info('Did not fail...')

    # Method 2: `report.outcome` is always one of ['passed', 'failed', 'skipped'] 
    if report.outcome == 'failed':
        logging.error('FAILED: %s', report.longrepr)
    elif report.outcome == 'skipped':
        logging.info('Skipped')
    else:  # report.outcome == 'passed'
        logging.info('Passed')

See TestReport documentation for details of longrepr and outcome

(It doesn't use pytest_runtest_teardown as the OP requested but it does easily let you check for failure)