在另一个夹具中使用 requests_mock 时，模拟的 URL 不会转发给处理程序

Question

I am creating my own fixture to simulate a service endpoint needed in my unit tests. In order to intercept the HTTP requests, I use requests_mock as follows:

@pytest.fixture
def sparql_endpoint(requests_mock):
    yield lambda uri, initial_data: Endpoint(requests_mock, uri, initial_data)

and in Endpoint.__init__ I do the following:

    m.post(url=uri, raw=self.handle_post)
    m.get(url=uri, raw=self.handle_get)

In my actual testcase I inject the endpoint and initialize it:

def test_basic_select(my_endpoint):
    repo_uri = 'https://my.rdfdb.com/repo/sparql'
    rdf_files = ['tests/upper_ontology.ttl',
                 'tests/domain_ontology.ttl',
                 'tests/instance_data.ttl']
    endpoint = sparql_endpoint(repo_uri, rdf_files)

Which does, in fact, initialize the mocked endpoint and I see Mocker.start() get invoked if I set a breakpoint there. However, later in the testcase I get the following:

..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:222: in urlopen
    return opener.open(url, data, timeout)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:525: in open
    response = self._open(req, data)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:543: in _open
    '_open', req)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:503: in _call_chain
    result = func(*args)
..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:1360: in https_open
    context=self._context, check_hostname=self._check_hostname)

E               urllib.error.URLError: <urlopen error [Errno 11001] getaddrinfo failed>

..\..\AppData\Local\Programs\Python\Python37\lib\urllib\request.py:1319: URLError

Because it cannot resolve the fake URL I gave it. So, did I somehow mess up the handler registration so that the Matcher is not kicking the request there? Why is urlopen still trying to resolve the host?

Answer 1

It would appear you're not starting the mocker anywhere? I don't have a lot of experience with pytest, but the fact that the request is getting through to urllib implies it never hit the mock object.

Have you got a self contained example that can be run? I'd guess it's to do with the way you are yielding the requests_mock object. The mocker will works like a normal context manager and so will stop when you exit the scope, but it's hard to say without tracing it.

Answer 2

I have figured out the issue, which was due to the underlying code ( SPARQLWrapper ) not using requests , instead using urlopen directly, and thus bypassing my mock. In order to be able to intercept both types of access, I resorted to using HTTPretty , which did a more thorough mocking. The code ended up looking as follows:

First, in the fixture itself, I enabled the mocking:

    httpretty.set_default_thread_timeout(60)
    # enable HTTPretty so that it will monkey patch the socket module
    httpretty.enable(verbose=True, allow_net_connect=False)

    yield lambda uri, initial_data, **kwargs: Endpoint(uri, initial_data, **kwargs)

    # disable afterwards, so that you will have no problems in code that uses that socket module
    httpretty.disable()
    # reset HTTPretty state (clean up registered urls and request history)
    httpretty.reset()

Note that I yield a lambda that creates the instance implementing the fixture, so after it is cleaned up, httpretty can clean up as well.

In the fixture initialization, I create the bindings:

        httpretty.register_uri(httpretty.GET, uri,
                               body=self.handle_get)
        httpretty.register_uri(httpretty.POST, uri,
                               body=self.handle_post)

When I inject the fixture, I had to make an extra call to actually create it:

def test_request_get(sparql_endpoint):
    repo_uri = 'https://my.rdfdb.com/repo/sparql'
    rdf_files = ['tests/upper_ontology.ttl',
                 'tests/domain_ontology.ttl',
                 'tests/instance_data.ttl']
    # This calls the lambda defined in the fixture
    endpoint = sparql_endpoint(repo_uri, rdf_files)