pytest run markers via configuration file

Question

I am running some functional tests using pytest for command line testing. I have my tests in a yml file which is used for test configuration. I want to provide some markers in my yml file which can be used by pytest instead of using the decorator approach inside the test file.

eg

---
test_name: xyz
test_type: smoke
command: hello --help
expected: hello world
assertions:
  - testResult.actual_command_returnCode == 0
  - len(testResult.expected_value_result) != 0
  - testResult.expected_value_result[0] == testResult.actual_command_output

Is there a way i can specify markers in my yml file and pytest can used the same for understanding which specific tests it needs to run?

Answer 1

py.test does not read yaml files, you will have to write your own code that does that and adds markers to your test functions. You can do that with a hook function placed in your conftest.py file. Here is an example that places a marker smoke on a test named test_a :

def pytest_collection_modifyitems(session, config, items):
    for i in items:
        if i.name == "test_a":
            i.add_marker("smoke")

This function (if present in conftest.py ) is automatically called by py.test once, right after it collects a list of tests from your directory and before any tests are actually run. You can modify the list items as you wish.

(NOTE my example looks only at the test name, so it will match a function named test_a in any test file; you may need a different way of selecting the actual test to which to apply the marker, eg, compare the value of i.nodeid , which is a string like path/file.py::function_name .)

This is only an example, you should replace the code with something that reads your yaml file and decides which markers to add to what tests, depending on what it finds in that file.

Answer 2

I created a hook in my conftest.py file as follows

def pytest_collection_modifyitems(config,items):
  with open("C:\Users\xyz\mno\cli_automation\test_configuration\project\test_command.yml", 'r') as stream:
    data_loaded = yaml.load(stream)
    for i in items:
        if i.name == data_loaded.get("test_name"):
            i.add_marker(data_loaded.get("test_type"))
            pytest_runtest_call(i)

I am getting an attribute error when i do the pytest_runtest_call(i) as AttributeError: 'Function' object has no attribute '_evalxfail' . Checked in the source code of [ https://docs.pytest.org/en/2.8.7/_modules/_pytest/skipping.html] , its failing at line 102 at evalxfail = item._evalxfail not sure why