Python mocking exception to method calls in a context manager

Question

I'm working on unittesting some python3 code that calls sendmail on an SMTP class via a context manager and attempts to catch exceptions to log them. I can successfully mock the SMTP class and do some checking on it in other tests (eg verifying that send_message was actually called), but I can't seem to get the method call to send_message on the class to raise an exception to log the error.

Code to be tested (from siteidentity_proxy_monitoring.py):

def send_alert(message, email_address):
    with SMTP('localhost') as email:
        try:
            email.send_message(message)
        except SMTPException:
            # retry the send
            print('exception raised') # debugging statement
            try:
                email.send_message(message)
            except:
                logging.error(
                    'Could not send email alert to %s', email_address
                )

Unittest method:

@unittest.mock.patch('siteidentity_proxy_monitoring.SMTP')
@unittest.mock.patch('siteidentity_proxy_monitoring.logging')
def test_logging_when_email_fails(self, mock_logger, mock_smtp):
    """
    Test that when alert email fails to send, an error is logged
    """
    test_print('Testing logging when email send fails')
    email_instance = mock_smtp.return_value
    email_instance.send_message.side_effect = SMTPException
    siteidentity_proxy_monitoring.send_alert(
        'test message',
        'email@example.com'
    )
    mock_logger.error.assert_called_with(
        'Could not send email alert to %s', 'email@example.com'
    )

Output from the test result:

[TEST] ==> Testing logging when email send fails
F
======================================================================
FAIL: test_logging_when_email_fails (__main__.TestSiteidentityMonitors)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/lib/python3.6/unittest/mock.py",line 1179, in patched
    return func(*args, **keywargs)
  File "tests/siteidentity_monitor_tests.py", line 108, in 
test_logging_when_email_fails
    'Could not send email alert to %s', 'email@example.com' 
  File "/lib/python3.6/unittest/mock.py", line 805, in assert_called_with
     raise AssertionError('Expected call: %s\nNot called' % (expected,))
AssertionError: Expected call: error('Could not send email alert to %s', 'email@example.com')
Not called

----------------------------------------------------------------------
Ran 4 tests in 0.955s

FAILED (failures=1)

I feel like I'm missing something related to the calls to __enter__ and __exit__ , but I can't quite seem to tease out why my patching doesn't seem to trigger the side effect where I expect it to. Unfortunately, most of the examples and documentation I've come across don't quite go that in depth with mocking method calls within contexts (as far as I've understood them, anyway).

Answer 1

Got stuck on similar issue just now, this is how get around it:

1. Put a import pdb;pdb.set_trace() before line email.send_message(message)
2. Run your test.
3. When you get dropped into the PDB session, type email.send_message and you'll see something like <Mock name='mock_smtp().__enter__().send_message' id='...'> .
4. In your test case, replace all () s with return_value . In your case: mock_smtp.return_value.enter.return_value.send_message.side_effect = SMTPException .

Answer 2

Here is the same test, using pytest and mocker fixture :

def test_logging_when_email_fails(mocker):
    mock_logging = mocker.MagicMock(name='logging')
    mocker.patch('siteidentity_proxy_monitoring.logging', new=mock_logging)
    mock_SMTP = mocker.MagicMock(name='SMTP',
                                 spec=siteidentity_proxy_monitoring.SMTP)
    mocker.patch('siteidentity_proxy_monitoring.SMTP', new=mock_SMTP)
    mock_SMTP.return_value.__enter__.return_value.send_message.side_effect = SMTPException

    siteidentity_proxy_monitoring.send_alert(
        'test message',
        'email@example.com'
    )

    mock_logging.error.assert_called_once_with(
        'Could not send email alert to %s', 'email@example.com')

You may find the way I wrote the test more interesting than the test itself - I have created a python library to help me with the syntax.

Here is how I approached your problem in a systematic way:

We start with the test call you want and my helper library to generate the mocks to the referenced modules and classes, while also preparing the asserts:

from mock_autogen.pytest_mocker import PytestMocker

import siteidentity_proxy_monitoring

def test_logging_when_email_fails(mocker):
    print(PytestMocker(siteidentity_proxy_monitoring).mock_referenced_classes().mock_modules().prepare_asserts_calls().generate())
    siteidentity_proxy_monitoring.send_alert(
        'test message',
        'email@example.com'
    )

Now the test obviously fails due to the fact we haven't mocked SMTP yet, but the print output is useful:

# mocked modules
mock_logging = mocker.MagicMock(name='logging')
mocker.patch('siteidentity_proxy_monitoring.logging', new=mock_logging)
# mocked classes
mock_SMTP = mocker.MagicMock(name='SMTP', spec=siteidentity_proxy_monitoring.SMTP)
mocker.patch('siteidentity_proxy_monitoring.SMTP', new=mock_SMTP)
mock_SMTPException = mocker.MagicMock(name='SMTPException', spec=siteidentity_proxy_monitoring.SMTPException)
mocker.patch('siteidentity_proxy_monitoring.SMTPException', new=mock_SMTPException)
# calls to generate_asserts, put this after the 'act'
import mock_autogen
print(mock_autogen.generator.generate_asserts(mock_logging, name='mock_logging'))
print(mock_autogen.generator.generate_asserts(mock_SMTP, name='mock_SMTP'))
print(mock_autogen.generator.generate_asserts(mock_SMTPException, name='mock_SMTPException'))

I've placed the relevant mocks before the test call and added the generate asserts call afterwards:

def test_logging_when_email_fails(mocker):
    mock_logging = mocker.MagicMock(name='logging')
    mocker.patch('siteidentity_proxy_monitoring.logging', new=mock_logging)
    mock_SMTP = mocker.MagicMock(name='SMTP',
                                 spec=siteidentity_proxy_monitoring.SMTP)
    mocker.patch('siteidentity_proxy_monitoring.SMTP', new=mock_SMTP)

    siteidentity_proxy_monitoring.send_alert(
        'test message',
        'email@example.com'
    )

    import mock_autogen
    print(mock_autogen.generator.generate_asserts(mock_logging,
                                                  name='mock_logging'))
    print(mock_autogen.generator.generate_asserts(mock_SMTP, name='mock_SMTP'))

This time, the test execution gave me some useful asserts:

mock_logging.assert_not_called()
assert 1 == mock_SMTP.call_count
mock_SMTP.assert_called_once_with('localhost')
mock_SMTP.return_value.__enter__.assert_called_once_with()
mock_SMTP.return_value.__enter__.return_value.send_message.assert_called_once_with('test message')
mock_SMTP.return_value.__exit__.assert_called_once_with(None, None, None)

We never got to the logging part, since we threw no SMTPException! Luckily we can alter one of the asserts in a minor way to create the side affect we want (this is where a lot of guessing is done normally and the tool is really helpful):

def test_logging_when_email_fails(mocker):
    mock_logging = mocker.MagicMock(name='logging')
    mocker.patch('siteidentity_proxy_monitoring.logging', new=mock_logging)
    mock_SMTP = mocker.MagicMock(name='SMTP',
                                 spec=siteidentity_proxy_monitoring.SMTP)
    mocker.patch('siteidentity_proxy_monitoring.SMTP', new=mock_SMTP)
    mock_SMTP.return_value.__enter__.return_value.send_message.side_effect = SMTPException

    siteidentity_proxy_monitoring.send_alert(
        'test message',
        'email@example.com'
    )

    import mock_autogen
    print(mock_autogen.generator.generate_asserts(mock_logging,
                                                  name='mock_logging'))
    print(mock_autogen.generator.generate_asserts(mock_SMTP,
                                                  name='mock_SMTP'))

The generator created the right mocks for mock_logging:

mock_logging.error.assert_called_once_with('Could not send email alert to %s', 'email@example.com')

You also got a more exact assert, which can be used to ensure your tested code does retries:

from mock import call
mock_SMTP.return_value.__enter__.return_value.send_message.assert_has_calls(
    calls=[call('test message'), call('test message'), ])

And that's how I got the code I originally placed!