How to unittest multiple similar classes

Question

I have a program with multiple very similar classes:

class BlackBox1():
    def calc(self, a, b):
        return a + b

class BlackBox2():
    def calc(self, a, b):
        return a * b
...

Now I want to write unittests for all those classes. Of course I could write separate tests for each Blackbox. Anyway, since each blackbox has exactly the same method calc(a, b) to be tested, I wonder, if there is something like a “best practise”, to automatically give classes and expected results to a abstract test framework, something like

import unittest
class TestAbstractBox(unittest.TestCase):
    def setUp(self):
        self.box = blackbox()
        self.param_a = a
        self.param_b = b
        self.expected_result = result

    def test_calc_method(self):
        real_result = self.box.calc(self.param_a, self.param_b)
        self.assertEqual(real_result, self.expected_result, 
                        "{0} gives wrong result".format(self.box.__class__))

TAbstractTest = unittest.defaultTestLoader.loadTestsFromTestCase(TestAbstractBox)

Is there a way to pass {"blackbox": Blackbox1, "a": 3, "b": 5, "result": 8} and {"blackbox": Blackbox2, "a": 4, "b": 7, "result": 28} to the TestAbstractBox class to not have multiple times the same code, but have a easy way to test new Blackboxes?

Answer 1

You can simply add these classes to setUp :

class TestAbstractBox(unittest.TestCase):
  def setUp(self):
    self.boxes = [Blackbox1(), Blackbox2()]
    self.param_a = a
    self.param_b = b
    self.expected_result = result

  def test_calc_method(self):
    for box in self.boxes:
      real_result = self.box.calc(self.param_a, self.param_b)
      self.assertEqual(real_result, self.expected_result, 
          "{0} gives wrong result".format(self.box.__class__))

EDIT Version 2:

class TestAbstractBox(unittest.TestCase):
  def setUp(self):
    self.boxes = [
      { "blackbox":Blackbox1(), "a":a1, "b":b1, "expected_result":result1 },
      { "blackbox":Blackbox2(), "a":a2, "b":b2, "expected_result":result2 },
    ]

  def test_calc_method(self):
    for box in self.boxes:
      real_result = box["blackbox"].calc(box["a"], box["b"])
      self.assertEqual(real_result, box["expected_result"], 
          "{0} gives wrong result".format(box["blackbox"].__class__))

Answer 2

I think I found a solution, basically following this answer :

import unittest

class BlackBox1():
    def calc(self, a, b):
        return a + b

class BlackBox2():
    def calc(self, a, b):
        return a * b

class TestBlackBox1(unittest.TestCase):
    test_params = {"blackbox": BlackBox1, "a": 3, "b": 5, "result": 8}
    def setUp(self):
        self.box = self.test_params["blackbox"]()
        self.param_a = self.test_params["a"]
        self.param_b = self.test_params["b"]
        self.expected_result = self.test_params["result"]

    def test_calc_method(self):
        real_result = self.box.calc(self.param_a, self.param_b)
        self.assertEqual(real_result, self.expected_result,
                        "{0} gives wrong result: {1} instead of {2}".format
                        (self.box.__class__, real_result, self.expected_result))

class TestBlackBox2(TestBlackBox1):
    test_params = {"blackbox": BlackBox2, "a": 4, "b": 7, "result": 28}

TBB1 = unittest.defaultTestLoader.loadTestsFromTestCase(TestBlackBox1)
TBB2 = unittest.defaultTestLoader.loadTestsFromTestCase(TestBlackBox2)

Anyway, thanks for the other answers and sorry for not finding this way in the first place.

Answer 3

I don't think this is such a good approach.

A test method should do exactly one test, not loop over several assert* calls. Otherwise in case of a failure you would not be able to to easily tell which test failed. After all that's the whole point of doing unittests.

Also, you should not risk breaking existing tests by adding new variants for new classes, or refactoring existing tests into some common methods.

In the case of several classes with identical methods (think of implementing a common interface in several derived classes) it may be useful to put all the tests for the common properties of the interface in to one base class and then deriving individual test classes for each class under test. Those derived classes would then implement suitable setUp and tearDown methods. They may also add more test cases specific to the class under test.