Mock modules and subclasses (TypeError: Error when calling the metaclass bases)

Question

To compile documentation on readthedocs, the module h5py has to be mocked. I get an error which can be reproduced with this simple code:

from __future__ import print_function
import sys

try:
    from unittest.mock import MagicMock
except ImportError:
    # Python 2
    from mock import Mock as MagicMock


class Mock(MagicMock):
    @classmethod
    def __getattr__(cls, name):
        return Mock()

sys.modules.update({'h5py': Mock()})

import h5py
print(h5py.File, type(h5py.File))


class A(h5py.File):
    pass

print(A, type(A))


class B(A):
    pass

The output of this script is:

<Mock id='140342061004112'> <class 'mock.mock.Mock'>
<Mock spec='str' id='140342061005584'> <class 'mock.mock.Mock'>
Traceback (most recent call last):
  File "problem_mock.py", line 32, in <module>
class B(A):
TypeError: Error when calling the metaclass bases
    str() takes at most 1 argument (3 given)

What is the correct way to mock h5py and h5py.File ?

It seems to me that this issue is quite general for documentation with readthedocs where some modules have to be mocked. It would be useful for the community to have an answer.

Answer 1

You can't really use Mock instances to act as classes ; it fails hard on Python 2, and works by Python 3 only by accident (see below).

You'd have to return the Mock class itself instead if you wanted them to work in a class hierarchy:

>>> class A(Mock):  # note, not called!
...     pass
...
>>> class B(A):
...     pass
...
>>> B
<class '__main__.B'>
>>> B()
<B id='4394742480'>

If you can't import h5py at all , that means you'll need to keep a manually updated list of classes where you return the class rather than an instance:

_classnames = {
    'File',
    # ...
}

class Mock(MagicMock):
    @classmethod
    def __getattr__(cls, name):
        return Mock if name in _classnames else Mock()

This is not foolproof; there is no way to detect the parent instance in a classmethod, so h5py.File().File would result in yet another 'class' being returned even if in the actual implementation that was some other object instead. You could partially work around that by creating a new descriptor to use instead of the classmethod decorator that would bind to either the class or to an instance if one is available; that way you at least would have a context in the form of self._mock_name on instances of your Mock class.

---

In Python 3, using MagicMock directly without further customisation works when used as a base class:

>>> from unittest.mock import MagicMock
>>> h5py = MagicMock()
>>> class A(h5py.File): pass
...
>>> class B(A): pass
...

but this is not really intentional and supported behaviour; the classes and subclasses are 'specced' from the classname string:

>>> A
<MagicMock spec='str' id='4353980960'>
>>> B
<MagicMock spec='str' id='4354132344'>

and thus have all sorts of issues down the line as instanciation doesn't work:

>>> A()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/unittest/mock.py", line 917, in __call__
    return _mock_self._mock_call(*args, **kwargs)
  File "/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.5/unittest/mock.py", line 976, in _mock_call
    result = next(effect)
StopIteration