How to parametrize tests with json array test data using pytest in python?

Question

I have some test cases and test data where test data is in JSON array form as below:

{
    "valid_data": [
       {
           "id": "1234",
           "name": "John"
       },
       {
           "id": "2234",
           "name": "Mary"
       },
       {
           "id": "3234",
           "name": "Kenny"
       },
    ],
    "invalid_data": [
       {
           "id": "1234",
           "name": "Mary"
       },
       {
           "id": "2234",
           "name": "Kenny"
       },
       {
           "id": "3234",
           "name": "John"
       },
    ]
}

I am now testing an API which will take in JSON as input and will respond with a status code. If the id and name match inside the database, it will respond with 200 OK. Else with 400 Bad Request.

So here are my current test cases:

def get_test_data(filename):
    folder_path = os.path.abspath(Path(os.path.dirname(__file__)))
    folder = os.path.join(folder_path, 'TestData')
    jsonfile = os.path.join(folder, filename)
    with open(jsonfile) as file:
        data = json.load(file)
    return data

def test_valid_ok(database_api):
    data = get_test_data('test_data.json')

    response = database_api.get_user_info(data)
    assert requests.codes['ok'] == response.status_code

In my conftest I just declared the method database_api and take in the URL as the parameter then it will send a post request to the api. For this part had no problem I have tested which is working fine.

However with current structure and code, I can only have 1 json data inside the json file. I would like to have a data-driven test which able to run multiple times based on my test data inside the json file.

I have checked the pytest official documents and various online sources which suggest to use pytest parametrized function but I couldn't get it right with json file.

Thanks if anyone could help!

Answer 1

You can use pytest_generate_tests hook for parametrizing with dynamic data.

First create a fixture that can be called by the test function to get the test data.

# in conftest.py
@pytest.fixture()
def test_data(request):
    return request.param

Now you can parametrize it so that it is called for each test data set:

#in conftest.py
def pytest_generate_tests(metafunc):
    testdata = get_test_data('test_data.json')
    metafunc.parametrize('test_data', testdata, indirect=True)

The testdata passed to the parametrize function has to be a list. So you need to modify the input data a bit before passing it. I modified the get_test_data function a bit.

#in conftest.py
def get_test_data(filename):
    folder_path = os.path.abspath(os.path.dirname(__file__))
    folder = os.path.join(folder_path, 'TestData')
    jsonfile = os.path.join(folder, filename)
    with open(jsonfile) as file:
        data = json.load(file)

    valid_data = [(item, 1) for item in data['valid_data']]
    invalid_data = [(item, 0) for item in data['invalid_data']]

    # data below is a list of tuples, with first element in the tuple being the 
    # arguments for the API call and second element specifies if this is a test 
    # from valid_data set or invalid_data. This can be used for putting in the
    # appropriate assert statements for the API response.
    data = valid_data + invalid_data

    return data

And now your test function could look like:

#in test_API.py
def test_(test_data):
    response = database_api.get_user_info(test_data[0])

    # Add appropriate asserts. 
    # test_data[1] == 1 means 200 response should have been received
    # test_data[1] == 0 means 400 response should have been received

Answer 2

I just wrote a package called parametrize_from_file to solve exactly this problem. Here's how it would work for this example:

import parametrize_from_file

# If the JSON file has the same base name as the test module (e.g. "test_api.py"
# and "test_api.json"), this parameter isn't needed.
path_to_json_file = ...

@parametrize_from_file(path_to_json_file, 'valid_data')
def test_valid_data(id, name):
    request = dict(id=id, name=name)
    response = database_api.get_user_info(request)
    assert response.status_code == 200

@parametrize_from_file(path_to_json_file, 'invalid_data')
def test_invalid_data(id, name):
    request = dict(id=id, name=name)
    response = database_api.get_user_info(request)
    assert response.status_code == 400

You could simplify this code a bit by reorganizing the JSON file slightly:

# test_api.json
{
    "test_id_name_requests": [
       {
           "request": {
               "id": "1234",
               "name": "John"
           },
           "status_code": 200
       },
       {
           "request": {
               "id": "2234",
               "name": "Mary"
           },
           "status_code": 200
       },
       {
           "request": {
               "id": "3234",
               "name": "Kenny"
           },
           "status_code": 200
       },
       {
           "request": {
               "id": "1234",
               "name": "Mary"
           },
           "status_code": 400
       },
       {
           "request": {
               "id": "2234",
               "name": "Kenny"
           },
           "status_code": 400
       },
       {
           "request": {
               "id": "3234",
               "name": "John"
           },
           "status_code": 400
       },
    ],
}

With this file, only one test function is needed and no arguments need to be given to the @parametrize_from_file decorator:

# test_api.py
import parametrize_from_file

@parametrize_from_file
def test_id_name_requests(request, status_code):
    response = database_api.get_user_info(request)
    assert response.status_code == status_code