如何用N个不同的路径测试功能？

Question

I have a function which validates a model class where it has to check every member of the class to be non-null or non-empty in case of a String.

The logic for this is_complete function is something like this:

def is_complete(profile):
    if profile.first_name in (None, ''):
        return False
    elif profile.last_name in (None, ''):
        return False
    elif profile.dob is None:
        return False
    .
    .
    .# and checks so on for all members of the profile instance
    .
    .

    return True

My question since the number of possible paths the execution can take is quite large and increases in proportion to the number of member variables of profile to be checked, how does one reliably write tests for all the possible paths?

Right now, I have two simple test cases:

1. where only some of the members are set and checks assertFalse(is_complete(foo))
2. where all the members are set and checks assertTrue(is_complete(foo))

But I have a feeling that this may not be enough.

Answer 1

I'm not sure what you mean by having MxN paths. In the posted code you have as many paths as fields + 1.

Create a helper method that creates a profile that is complete and passes is_complete .

Add a test method to verify is_complete(profile) is True for the complete profile .

Add one test method for each field, with the following steps:

• Call the helper method to create a complete profile that we know would pass is_complete
• Break the field under test
• Verify that is_complete returns False

You will have as many test methods as fields + 1.

Btw, instead of this:

 if profile.first_name in (None, ''): 

You can writer simpler:

if not profile.first_name:

Answer 2

You can test this with randomization:

• Write a function that creates a random Profile object with all the fields filled (could be random, but valid).
• Randomly change one or more fields to either None or empty
• Pass the object to the is_complete() function.

I can show this only in Java with Qala Datagen . I assume you're interested in the negative paths, so it can look like this:

@Test void profileIsNotComplete_ifAtLeastOneFieldIsBlank() {
    Profile p = Profile.random();
    callOneOrMore(
        () -> profile.setName(nullOrEmpty()),
        () -> profile.setLastName(nullOrEmpty()),
        ...);
    assertFalse(profile.isComplete();
}

Note, that this code actually tests more - it also checks the combination of fields set to null/empty. If in Python there are no such libs, you can write some utility methods for yourself.

NB : only one path is going to be tested per one execution. You can run it many times (a thousand?) to make sure that all the paths pass. Then in CI you can run it only once if this is not a mission-critical functionality and you're not afraid that it's going to break often.

Otherwise if you really want this to be covered 100% for every invocation , you can violate some of the good practices and just combine all these tests in one (otherwise there are going to be too many tests which complicate reading them):

@Test void profileIsNotComplete_ifOneFieldIsBlank() {
    assertFalse(Profile.random().setName(null).isComplete());
    assertFalse(Profile.random().setName("").isComplete());

    assertFalse(Profile.random().setLastName(null).isComplete());
    assertFalse(Profile.random().setLastName("").isComplete());
    ...
}

This doesn't test combinations though, but you can combine both approaches. The positive case is very easy in both approaches:

@Test void profileIsComplete_ifAllFieldsAreFilled() {
    assertTrue(Profile.random());
}

More info on randomized testing is here .