Python unittest : how to specify custom equality predicate?

Question

This could be an easy question; I'd like to use a custom equality operator in a Python unittest test case. So for instance, supposing I want to test a "number-to-string" function, and I want to perform a case-insensitive string comparison.

Here's what I'd like to write:

class MyTest(unittest.TestCase):
    def testFoo(self):
        self.assertCheck(ci_string_eq,i2s(24),"Twenty-Four")

The problem is that assertCheck isn't a thing.

Some obvious workarounds:

• use assertTrue; the problem is that then a test case failure becomes opaque and unhelpful; "expected True, got False". Bleah.
• dig into unittest and extend it myself; well, I'm hoping to avoid that :)

I hope I'm missing something obvious?

Many thanks in advance!

EDIT: some have suggested that I override __eq__ . This is not what I want. Specifically, the __eq__ method is used by clients of my code, to determine whether two objects should be considered "the same" (cf. "extensional equality"). For the purposes of testing, though, I want to test using a different predicate. So overriding __eq__ does not solve my problem.

Answer 1

The built in unittest module has a specific method for this called addTypeEqualityFunc . You can read about it here . You just have to write your equality function and pass it and simply use the assertEqual method as usual.

Answer 2

Here's the full list of supported assertions in Python 3.6's unittest module.

As you can see, there is no assertion that takes a custom predicate to evaluate, but you can get a more helpful error message by passing a custom error message to your assertion method through the msg argument.

For instance:

class MyTest(unittest.TestCase):
  def testFoo(self):
    self.assertEqual(i2s(24),"Twenty-Four",msg="i2s(24) should be 'Twenty-Four'")

If that is not enough for you, you don't really need to go around digging into unittest : You could define a class that extends unittest 's TestCase with the methods you need, ie:

class CustomTestCase(unittest.TestCase):
  def assertCheck(self):
    ...

And then you would define your tests as:

class MyTest(CustomTestCase):
  def testFoo(self):
    self.assertCheck(...)

Answer 3

The good news is that there isn't any complicated wiring to make a custom assertion with your own rules. Just do the comparison, gather any helpful information, then call fail(msg) if needed. That will take care of any reporting you need.

Of course, I'm so lazy that I don't even like to gather the helpful information. What I often find useful is to strip out the irrelevant stuff from both the expected and the actual data, then use the regular assertEquals(expected, actual) .

Here's an example of both techniques, plus a bonus one that uses longMessage to include context:

# file scratch.py

from unittest import TestCase
import sys

def convert(i):
    results = 'ONE TOO THREE'.split()
    return results[i-1]


class FooTest(TestCase):
    def assertResultEqual(self, expected, actual):
        expected_lower = expected.lower()
        actual_lower = actual.lower()
        if expected_lower != actual_lower:
            self.fail('Results did not match: {!r}, {!r}, comparing {!r}, {!r}'.format(
                expected,
                actual,
                expected_lower,
                actual_lower))

    def assertLazyResultEqual(self, expected, actual):
        self.assertEqual(expected.lower(), actual.lower())

    def assertLongLazyResultEqual(self, expected, actual):
        self.longMessage = True
        self.assertEqual(expected.lower(),
                         actual.lower(),
                         'originals: {!r}, {!r}'.format(expected, actual))

    def test_good_convert(self):
        expected = 'One'

        s = convert(1)

        self.assertResultEqual(expected, s)
        self.assertLazyResultEqual(expected, s)
        self.assertLongLazyResultEqual(expected, s)

    def test_bad_convert(self):
        expected = 'Two'

        s = convert(2)

        self.assertResultEqual(expected, s)

    def test_lazy_bad_convert(self):
        expected = 'Two'

        s = convert(2)

        self.assertLazyResultEqual(expected, s)

    def test_long_lazy_bad_convert(self):
        expected = 'Two'

        s = convert(2)

        self.assertLongLazyResultEqual(expected, s)

That generates the following output, including context and reports of pass and failure counts.

$ python -m unittest scratch
F.FF
======================================================================
FAIL: test_bad_convert (scratch.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/don/workspace/scratch/scratch.py", line 43, in test_bad_convert
    self.assertResultEqual(expected, s)
  File "/home/don/workspace/scratch/scratch.py", line 18, in assertResultEqual
    actual_lower))
AssertionError: Results did not match: 'Two', 'TOO', comparing 'two', 'too'

======================================================================
FAIL: test_lazy_bad_convert (scratch.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/don/workspace/scratch/scratch.py", line 50, in test_lazy_bad_convert
    self.assertLazyResultEqual(expected, s)
  File "/home/don/workspace/scratch/scratch.py", line 21, in assertLazyResultEqual
    self.assertEqual(expected.lower(), actual.lower())
AssertionError: 'two' != 'too'
- two
?  ^
+ too
?  ^


======================================================================
FAIL: test_long_lazy_bad_convert (scratch.FooTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/don/workspace/scratch/scratch.py", line 57, in test_long_lazy_bad_convert
    self.assertLongLazyResultEqual(expected, s)
  File "/home/don/workspace/scratch/scratch.py", line 27, in assertLongLazyResultEqual
    'originals: {!r}, {!r}'.format(expected, actual))
AssertionError: 'two' != 'too'
- two
?  ^
+ too
?  ^
 : originals: 'Two', 'TOO'

----------------------------------------------------------------------
Ran 4 tests in 0.002s

FAILED (failures=3)

If the custom comparison applies to a specific class, then you can add a custom equality operator for that class. If you do that in your setUp() method, then all the test methods can just call assertEquals() with that class, and your custom comparison will be called.

Answer 4

You can override __eq__ method of return value class without convoluting the original unit by subclassing it.

class Helper(FooReturnValueClass):

  def __init__(self, obj=None, **kwargs):
     self.obj = obj
     # any other attrs
     # probably good idea to skip calling super

  def __eq__(self, other):
    # logic

class MyTest(unittest.TestCase):

    def testFoo(self):
      expect = self.Helper(...)
      actual = ClassUnderTest.foo(...)
      self.assertEqual(expect, foo) # Order is important