How to test assertions?

Question

I am using a unit testing framework to test my libraries. I have quite a few assertions in the library to make sure that programmer errors are caught in debug builds. Now I want to make sure I am testing for all possible programmer errors.

Eg in a Table class, I want to make sure that the rows and cols passes are not greater than the rows and cols the table has. Let's assume I forget to test for cols. I would like to have my unit tests perform a test where the assertion should fire, and if not, fail the test. Is that possible?

Answer 1

The question then becomes are you planning on refactoring your code so that if this condition occurs you'll throw an exception instead of relying on <cassert> functionality to clue you into a problem? If so, you can just check that the exception was thrown. If not, then it's going to be more difficult to test an assert statement from <cassert> . Unit test frameworks like CUTE have an ASSERT_THROWS macro just for exception testing. I'd check your framework.

Also, it's been the case with the shops where I've worked that they frown upon assert and prefer exceptions. Calling abort doesn't help automated testing. Actually, it prohibits it. Just my two cents.

Answer 2

There are three possible ways:

• converting assertions to exceptions, or

• running each test a separate program whose exit code you (or rather, the test framework) checks, or

• let an assertion that fires, write information about itself, which the test framework then can pick up.

As far as I know no commercial or widely used unit test framework supports the last two ways. I have used checking of process exit code for hobby programming, but only with a small personal unit test framework implemented in Python and C++ ("hobby programming": that means I have no good data on how well it scales to large scale programming). The main case for process exit code testing is where the code has static assertions, that you want to be sure are triggered when they are intended to be.

Summing up, with extant test frameworks, conversion to exception is AFAIK your only option.

Cheers & hth.

Answer 3

The answer is dependent on the particular test framework that you are using, and will probably be found with very little searching in google.

The first hit for "boost unittest test assert" points to this question in StackOverflow: Testing for assert in the Boost Test framework

The second hit for "cppunit test assert" points to this documentation page: Making Assertions

Try searching in the internet for your concrete framework.