Is there a type of logic/flow errors that cannot be discovered by unit tests?

Question

Assuming I have contracts defined and clear requirements (covering input ranges, boundary values etc.), and unit tests validating all those conditions, can there still be integration errors when I put those units together? I do not consider external services now. I have seen the following as an example of integration error but I believe this is just a missing test at unit level:

class Engine
{
   int RPM;

   void SetRPMtoZero()
   {
      RPM=0;
   }
}

class Display
{
  CalculateAverage(Engine e)
  {
    if (e.IsRunning)
    {
      int X=smth/e.RPM;  //could be division by 0
    }
  }
}

class IntegratingClass
{
  Engine e
  Display d..

  ...

  e.SetRPMtoZero();
  d.CalculateAverage(e);

  //this sequence would lead to the division by zero

}

I do not think this shows an integration error - the CalculateAverage simply lacks the check for RPM!=0.

Is there actually a type of logical errors (or control flow) that cannot by discovered by unit testing?

Answer 1

I agree with you that the example rather is about a missing unit test for the class Display.

However, maybe the author of that example wanted to point out that the coupling between units can be stronger than expected or than described in a contract. Furthermore this example makes already clear that on integration level things can be different than expected on unit test level (maybe it was not intended that the SetRPMtoZero method is called just before the CalculateAverage method). Usually you don't see that on unit test level, unless you have the best specification in the world.

So, what else you may encounter when integrating your units (at the same time this is also what unit-tests can't tell you):

• Even on the first stage of integration test, the integration itself (fe linking the units together, if we talk about a language like c++), can fail, if the interface changed in one unit, but not adapted accordingly in the other(s).
• If the units use common resources (like a file, memory), then reservation/release of a resource may cause blocked states or data corruption
• If you cover the source code of a unit by unit test with 100% than this does not mean that all of the code is really necessary. On integration level you may realize that there is lot of code, which can't even be reached, when putting the units together. This can help you to find out, which parts of your code are probably "dead code".
• Performance restrictions or resource limits (like limited memory)
• Misinterpretation of the contract or errors in its implementation

Answer 2

Interesting question. It's stupid on the one hand. OBVIOUSLY there must be some kind of logic that can't be discovered through unit testing, or else unit testing would be the Oracle of Delphi.

I think the important philosophical consideration here, is the concept of EMERGENT COMPLEXITY. Many thinkers have pointed this out in the past. Moore's Law is probably the best example [that transistors double in complexity roughly every 2.5 years]. But it's a general principal.

So to abstract the law of emergent complexity to software testing: Would 5 software units be more, equal, or less complex separately or more complex together? When you put it that way, it's obvious that 5 units working together MUST be more complex than the individual units seperatly. This is known as "More than the sum of it's parts" and is a general rule for systems.