How to mock external servers in Python unit tests?

Question

I have several unit tests that take a long time (minutes) because of calls to external services (Twitter, Facebook, Klout, etc.)

I'd like to cache the results from these services and serve them transparently, with minimal changes to my current tests. The cache key depends on the URL, query arguments, headers, etc., so it's pretty complicated.

What's the best way to do this?

Answer 1

You would (should) usually use some kind of adapter to connect to these external services, modules. These are your interfaces to the outside world and can be mock ed and fake responses created depending on scenario.

I've experimented with a number of mocking libraries and finally found Mock to be the one most suitable for me.

Answer 2

Heres the code, We use the requests library to call external API's. So we create context processor with a mock requests object.

So if we were testing the get_data function this would be how we would mock the request to the external API:

import requests
import mock
import unittest


def get_data(url):
    resp = requests.get(url)
    return resp

class GetDataTest(unittest.TestCase):

    def test_get_data(self):
        with mock.patch.object(requests, 'get') as get_mock:
            get_mock.return_value = mock_response = mock.Mock()
            mock_response.status_code = 200
            mock_response.content = {'twitter_handle': '@twitter'}
            resp = get_data("http://this_address_does_not_exist.com")
            self.assertEqual(resp.status_code, 200)
            self.assertEqual(resp.content['twitter_handle'], '@twitter')

Answer 3

Technically if it uses external services it is not a unit test but an integration test. For unit test and expedite your test code use mock objects. You can find details on python mock objects here:
http://python-mock.sourceforge.net/