在这种情况下有多少个测试用例？

Question

I have a method which does many things, so that mean many side affects are created. For instance, say a single call to REST API returns a JSON object with many fields set. In this case if we want to check for each individual fields should we have one single test method which will contain many assertEquals or should we have one single test method per field validation containing one assertEquals.

Similarly a method can have many other side affects, for eg, saving to the database, sending emails etc. In this case should I have one unit test method per side affect?

In addition, if there are multiple test inputs per SUT methods then will this affect the decision of how many test methods are created?

Moreover, it might be related to story that means the story says this, this sub functionalities belong to the story in that case should not they belong to the same test method? Because if the requirement changes then all the test methods per side affects need to be changed as well. Would that be manageable?

Answer 1

In this case if we want to check for each individual fields should we have one single test method which will contain many assertEquals or should we have one single test method per field validation containing one assertEquals.

Worrying about the number of asserts in a test case is like worrying about how many lines are in a function. It's a first order approximation of complexity, but it's not really what you should be concerned with.

What you should be concerned with is how hard that test is to maintain, and how good its diagnostics are. When the test fails, will you be able to figure out why? A lot of that depends on how good your assertEquals is with large data structures.

json = call_rest();
want = { ...whatever you expect... };
assertEquals( json.from_json(), want );

If that just tells you they're not equal, that's not very useful. You then have to manually go in and look at json and want . If it dumps out both data structures, that's also not very useful, you have to look for the differences by eye.

But if it presents a useful diff of the two data structures, that is useful. For example, Perl's Test2 will produce diagnostics like this.

use Test2::Bundle::Extended;

is { foo => 23, bar => 42, baz => 99 },
   { foo => 22, bar => 42, zip => 99 };

done_testing;

# +-------+------------------+---------+------------------+
# | PATH  | GOT              | OP      | CHECK            |
# +-------+------------------+---------+------------------+
# | {foo} | 23               | eq      | 22               |
# | {zip} | <DOES NOT EXIST> |         | 99               |
# | {baz} | 99               | !exists | <DOES NOT EXIST> |
# +-------+------------------+---------+------------------+

---

Then there's the question of maintenance, and again this comes down to how good your assertEquals is. A single assertEquals is easy to read and maintain.

json = call_rest();
want = { ...whatever you expect... };
assertEquals( json.from_json(), want );

There's very little code, and want is very clear about what is expected.

While multiple gets very wordy.

json = call_rest();
have = json.from_json();
assertEquals( have['thiskey'], 23 );
assertEquals( have['thatkey'], 42 );
assertEquals( have['foo'], 99 );
...and so on...

It can be difficult to know which assert failed, you have to tell by line number or (if your test suite supports it) manually naming each assert which is more work and more maintenance and one more thing to get wrong.

OTOH individual assertEquals allow for more flexibility. For example, what if you only want to check certain fields? What if there are acceptable values in a range?

# bar just has to contain a number
assert( have['bar'].is_number );

But some test suites support this via a single assert.

want = {
    thiskey: 23,
    thatkey: 42,
    foo:     99,
    bar:     is_number
}

assertEquals( json.from_json, want );

is_number is a special object that tells assert it's not a normal quality check, but to just check that the value is a number. If your test suite supports this style, it's generally superior to writing out a bunch of asserts. The declarative approach means less code to write, read, and maintain.

The answer is: it depends on how good your testing tools are!