Should I add white/black box rendundant Unit Tests?

Question

I've written black-box unit tests for my project.

After a refactoring, I've adopted a strategy pattern in my code.

This code is covered by the black-box unit test, even after the refactoring .

However I was wondering: should I add white-box unit tests , for example, checking that each strategy is doing what is supposed to?

Or is this redundant because I already have the black box that are the checking the final outcome ?

Answer 1

One of the primary goals of testing in general and also for unit-testing is to find bugs (see Myers, Badgett, Sandler: The Art of Software Testing, or, Beizer: Software Testing Techniques, but also many others). In your project you may have a more relaxed position on this, but there are many software projects where it would have serious consequences if implementation level bugs escape to later development phases or even to the field. Some say, your goal should rather be to increase confidence in your code - and this is also true, but confidence can only be a consequence of doing testing right. If you don't test to find bugs, then I will simply not have confidence in your code after you have finished testing.

When finding bugs is a primary goal of unit-testing, then attempts to keep unit-test suites completely independent of implementation details is likely to result in inefficient test suites - that is, test suites that are not suited to find all bugs that could be found. Different implementations have different potential bugs. If you don't use unit-testing for finding these bugs, then any other test level (integration, subsystem, system) is definitely less suited for finding them systematically.

Thus, your statement that you have tested your code initially using black box tests already leaves me with a doubt that the test suite was fully effective in the first place. And, consequently, yes, I would add specific tests for each of the strategies.

However, keep in mind that the goal to have an effective test suite is in competition with another goal, namely to have a maintenance friendly test suite. But I see finding bugs as the primary goal and test suite maintainability as a secondary goal. Still, even when going into white box testing try to keep the maintenance effort low: Only use a white box test for finding bugs that a black box test would not also find. And, try hiding use of implementation details between test helper functions.