[英]How can I mock requests and the response?
I am trying to use Pythons mock package to mock Pythons requests
module.我正在尝试使用Pythons mock package来模拟 Pythons requests
模块。 What are the basic calls to get me working in below scenario?让我在以下情况下工作的基本电话是什么?
In my views.py, I have a function that makes variety of requests.get() calls with different response each time在我的 views.py 中,我有一个 function 每次都会发出各种不同响应的 requests.get() 调用
def myview(request):
res1 = requests.get('aurl')
res2 = request.get('burl')
res3 = request.get('curl')
In my test class I want to do something like this but cannot figure out exact method calls在我的测试 class 我想做这样的事情,但无法弄清楚确切的方法调用
Step 1:步骤1:
# Mock the requests module
# when mockedRequests.get('aurl') is called then return 'a response'
# when mockedRequests.get('burl') is called then return 'b response'
# when mockedRequests.get('curl') is called then return 'c response'
Step 2:第2步:
Call my view呼唤我的观点
Step 3:第 3 步:
verify response contains 'a response', 'b response', 'c response'验证响应包含“a 响应”、“b 响应”、“c 响应”
How can I complete Step 1 (mocking the requests module)?如何完成第 1 步(模拟请求模块)?
This is how you can do it (you can run this file as-is):这是你可以做到的(你可以按原样运行这个文件):
import requests
import unittest
from unittest import mock
# This is the class we want to test
class MyGreatClass:
def fetch_json(self, url):
response = requests.get(url)
return response.json()
# This method will be used by the mock to replace requests.get
def mocked_requests_get(*args, **kwargs):
class MockResponse:
def __init__(self, json_data, status_code):
self.json_data = json_data
self.status_code = status_code
def json(self):
return self.json_data
if args[0] == 'http://someurl.com/test.json':
return MockResponse({"key1": "value1"}, 200)
elif args[0] == 'http://someotherurl.com/anothertest.json':
return MockResponse({"key2": "value2"}, 200)
return MockResponse(None, 404)
# Our test case class
class MyGreatClassTestCase(unittest.TestCase):
# We patch 'requests.get' with our own method. The mock object is passed in to our test case method.
@mock.patch('requests.get', side_effect=mocked_requests_get)
def test_fetch(self, mock_get):
# Assert requests.get calls
mgc = MyGreatClass()
json_data = mgc.fetch_json('http://someurl.com/test.json')
self.assertEqual(json_data, {"key1": "value1"})
json_data = mgc.fetch_json('http://someotherurl.com/anothertest.json')
self.assertEqual(json_data, {"key2": "value2"})
json_data = mgc.fetch_json('http://nonexistenturl.com/cantfindme.json')
self.assertIsNone(json_data)
# We can even assert that our mocked method was called with the right parameters
self.assertIn(mock.call('http://someurl.com/test.json'), mock_get.call_args_list)
self.assertIn(mock.call('http://someotherurl.com/anothertest.json'), mock_get.call_args_list)
self.assertEqual(len(mock_get.call_args_list), 3)
if __name__ == '__main__':
unittest.main()
Important Note: If your MyGreatClass
class lives in a different package, say my.great.package
, you have to mock my.great.package.requests.get
instead of just 'request.get'.重要提示:如果您的MyGreatClass
类位于不同的包中,例如my.great.package
,您必须模拟my.great.package.requests.get
而不仅仅是“request.get”。 In that case your test case would look like this:在这种情况下,您的测试用例将如下所示:
import unittest
from unittest import mock
from my.great.package import MyGreatClass
# This method will be used by the mock to replace requests.get
def mocked_requests_get(*args, **kwargs):
# Same as above
class MyGreatClassTestCase(unittest.TestCase):
# Now we must patch 'my.great.package.requests.get'
@mock.patch('my.great.package.requests.get', side_effect=mocked_requests_get)
def test_fetch(self, mock_get):
# Same as above
if __name__ == '__main__':
unittest.main()
Enjoy!享受!
Try using the responses library .尝试使用响应库。 Here is an example from their documentation :这是他们文档中的一个示例:
import responses
import requests
@responses.activate
def test_simple():
responses.add(responses.GET, 'http://twitter.com/api/1/foobar',
json={'error': 'not found'}, status=404)
resp = requests.get('http://twitter.com/api/1/foobar')
assert resp.json() == {"error": "not found"}
assert len(responses.calls) == 1
assert responses.calls[0].request.url == 'http://twitter.com/api/1/foobar'
assert responses.calls[0].response.text == '{"error": "not found"}'
It provides quite a nice convenience over setting up all the mocking yourself.它为自己设置所有模拟提供了相当不错的便利。
There's also HTTPretty :还有HTTPretty :
It's not specific to requests
library, more powerful in some ways though I found it doesn't lend itself so well to inspecting the requests that it intercepted, which responses
does quite easily它不是特定于requests
库的,在某些方面更强大,尽管我发现它不适合检查它拦截的请求,哪些responses
很容易做到
There's also httmock .还有httmock 。
A new library gaining popularity recently over the venerable requests
is httpx
, which adds first-class support for async.最近在古老的requests
中流行的一个新库是httpx
,它增加了对异步的一流支持。 A mocking library for httpx is: https://github.com/lundberg/respx httpx 的模拟库是: https ://github.com/lundberg/respx
Here is what worked for me:这对我有用:
import mock
@mock.patch('requests.get', mock.Mock(side_effect = lambda k:{'aurl': 'a response', 'burl' : 'b response'}.get(k, 'unhandled request %s'%k)))
I used requests-mock for writing tests for separate module:我使用requests-mock为单独的模块编写测试:
# module.py
import requests
class A():
def get_response(self, url):
response = requests.get(url)
return response.text
And the tests:和测试:
# tests.py
import requests_mock
import unittest
from module import A
class TestAPI(unittest.TestCase):
@requests_mock.mock()
def test_get_response(self, m):
a = A()
m.get('http://aurl.com', text='a response')
self.assertEqual(a.get_response('http://aurl.com'), 'a response')
m.get('http://burl.com', text='b response')
self.assertEqual(a.get_response('http://burl.com'), 'b response')
m.get('http://curl.com', text='c response')
self.assertEqual(a.get_response('http://curl.com'), 'c response')
if __name__ == '__main__':
unittest.main()
this is how you mock requests.post, change it to your http method这就是您模拟 requests.post 的方式,将其更改为您的 http 方法
@patch.object(requests, 'post')
def your_test_method(self, mockpost):
mockresponse = Mock()
mockpost.return_value = mockresponse
mockresponse.text = 'mock return'
#call your target method now
Here is a solution with requests Response class.这是一个带有请求响应类的解决方案。 It is cleaner IMHO.恕我直言,它更干净。
import json
from unittest.mock import patch
from requests.models import Response
def mocked_requests_get(*args, **kwargs):
response_content = None
request_url = kwargs.get('url', None)
if request_url == 'aurl':
response_content = json.dumps('a response')
elif request_url == 'burl':
response_content = json.dumps('b response')
elif request_url == 'curl':
response_content = json.dumps('c response')
response = Response()
response.status_code = 200
response._content = str.encode(response_content)
return response
@mock.patch('requests.get', side_effect=mocked_requests_get)
def test_fetch(self, mock_get):
response = requests.get(url='aurl')
assert ...
I started out with Johannes Farhenkrug 's answer here and it worked great for me.我从Johannes Farhenkrug的答案开始,它对我很有用。 I needed to mock the requests library because my goal is to isolate my application and not test any 3rd party resources.我需要模拟 requests 库,因为我的目标是隔离我的应用程序而不是测试任何第三方资源。
Then I read up some more about python's Mock library and I realized that I can replace the MockResponse class, which you might call a 'Test Double' or a 'Fake', with a python Mock class.然后我阅读了更多关于 python 的Mock库的信息,我意识到我可以用 python Mock 类替换 MockResponse 类,你可能称之为“Test Double”或“Fake”。
The advantage of doing so is access to things like assert_called_with
, call_args
and so on.这样做的好处是可以访问诸如assert_called_with
、 call_args
之类的东西。 No extra libraries are needed.不需要额外的库。 Additional benefits such as 'readability' or 'its more pythonic' are subjective, so they may or may not play a role for you.诸如“可读性”或“更 Pythonic”之类的其他好处是主观的,因此它们可能对您起作用,也可能不会起作用。
Here is my version, updated with using python's Mock instead of a test double:这是我的版本,使用 python 的 Mock 而不是测试替身进行了更新:
import json
import requests
from unittest import mock
# defube stubs
AUTH_TOKEN = '{"prop": "value"}'
LIST_OF_WIDGETS = '{"widgets": ["widget1", "widget2"]}'
PURCHASED_WIDGETS = '{"widgets": ["purchased_widget"]}'
# exception class when an unknown URL is mocked
class MockNotSupported(Exception):
pass
# factory method that cranks out the Mocks
def mock_requests_factory(response_stub: str, status_code: int = 200):
return mock.Mock(**{
'json.return_value': json.loads(response_stub),
'text.return_value': response_stub,
'status_code': status_code,
'ok': status_code == 200
})
# side effect mock function
def mock_requests_post(*args, **kwargs):
if args[0].endswith('/api/v1/get_auth_token'):
return mock_requests_factory(AUTH_TOKEN)
elif args[0].endswith('/api/v1/get_widgets'):
return mock_requests_factory(LIST_OF_WIDGETS)
elif args[0].endswith('/api/v1/purchased_widgets'):
return mock_requests_factory(PURCHASED_WIDGETS)
raise MockNotSupported
# patch requests.post and run tests
with mock.patch('requests.post') as requests_post_mock:
requests_post_mock.side_effect = mock_requests_post
response = requests.post('https://myserver/api/v1/get_widgets')
assert response.ok is True
assert response.status_code == 200
assert 'widgets' in response.json()
# now I can also do this
requests_post_mock.assert_called_with('https://myserver/api/v1/get_widgets')
Repl.it links: Repl.it 链接:
https://repl.it/@abkonsta/Using-unittestMock-for-requestspost#main.py https://repl.it/@abkonsta/Using-unittestMock-for-requestspost#main.py
https://repl.it/@abkonsta/Using-test-double-for-requestspost#main.py https://repl.it/@abkonsta/Using-test-double-for-requestspost#main.py
If you want to mock a fake response, another way to do it is to simply instantiate an instance of the base HttpResponse class, like so:如果你想模拟一个假响应,另一种方法是简单地实例化一个基础 HttpResponse 类的实例,如下所示:
from django.http.response import HttpResponseBase
self.fake_response = HttpResponseBase()
One possible way to work around requests is using the library betamax, it records all requests and after that if you make a request in the same url with the same parameters the betamax will use the recorded request, I have been using it to test web crawler and it save me a lot time.解决请求的一种可能方法是使用库 betamax,它记录所有请求,然后如果您在相同的 url 中使用相同的参数发出请求,betamax 将使用记录的请求,我一直在使用它来测试网络爬虫它为我节省了很多时间。
import os
import requests
from betamax import Betamax
from betamax_serializers import pretty_json
WORKERS_DIR = os.path.dirname(os.path.abspath(__file__))
CASSETTES_DIR = os.path.join(WORKERS_DIR, u'resources', u'cassettes')
MATCH_REQUESTS_ON = [u'method', u'uri', u'path', u'query']
Betamax.register_serializer(pretty_json.PrettyJSONSerializer)
with Betamax.configure() as config:
config.cassette_library_dir = CASSETTES_DIR
config.default_cassette_options[u'serialize_with'] = u'prettyjson'
config.default_cassette_options[u'match_requests_on'] = MATCH_REQUESTS_ON
config.default_cassette_options[u'preserve_exact_body_bytes'] = True
class WorkerCertidaoTRT2:
session = requests.session()
def make_request(self, input_json):
with Betamax(self.session) as vcr:
vcr.use_cassette(u'google')
response = session.get('http://www.google.com')
https://betamax.readthedocs.io/en/latest/ https://betamax.readthedocs.io/en/latest/
This worked for me, although I haven't done much complicated testing yet.这对我有用,虽然我还没有做太多复杂的测试。
import json
from requests import Response
class MockResponse(Response):
def __init__(self,
url='http://example.com',
headers={'Content-Type':'text/html; charset=UTF-8'},
status_code=200,
reason = 'Success',
_content = 'Some html goes here',
json_ = None,
encoding='UTF-8'
):
self.url = url
self.headers = headers
if json_ and headers['Content-Type'] == 'application/json':
self._content = json.dumps(json_).encode(encoding)
else:
self._content = _content.encode(encoding)
self.status_code = status_code
self.reason = reason
self.encoding = encoding
Then you can create responses :然后您可以创建响应:
mock_response = MockResponse(
headers={'Content-Type' :'application/json'},
status_code=401,
json_={'success': False},
reason='Unauthorized'
)
mock_response.raise_for_status()
gives给
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http://example.com
Can you use requests-mock instead?你可以使用requests-mock代替吗?
Suppose your myview function instead takes a requests.Session
object, makes requests with it, and does something to the output:假设您的 myview 函数取而代之的是一个requests.Session
对象,用它发出请求,并对输出做一些事情:
# mypackage.py
def myview(session):
res1 = session.get("http://aurl")
res2 = session.get("http://burl")
res3 = session.get("http://curl")
return f"{res1.text}, {res2.text}, {res3.text}"
# test_myview.py
from mypackage import myview
import requests
def test_myview(requests_mock):
# set up requests
a_req = requests_mock.get("http://aurl", text="a response")
b_req = requests_mock.get("http://burl", text="b response")
c_req = requests_mock.get("http://curl", text="c response")
# test myview behaviour
session = requests.Session()
assert myview(session) == "a response, b response, c response"
# check that requests weren't called repeatedly
assert a_req.called_once
assert b_req.called_once
assert c_req.called_once
assert requests_mock.call_count == 3
You can also use requests_mock
with frameworks other than Pytest - the documentation is great.您还可以将requests_mock
与 Pytest 以外的框架一起使用 - 文档很棒。
I will add this information since I had a hard time figuring how to mock an async api call.我将添加此信息,因为我很难弄清楚如何模拟异步 api 调用。
Here is what I did to mock an async call.这是我为模拟异步调用所做的。
Here is the function I wanted to test这是我想测试的功能
async def get_user_info(headers, payload):
return await httpx.AsyncClient().post(URI, json=payload, headers=headers)
You still need the MockResponse class你仍然需要 MockResponse 类
class MockResponse:
def __init__(self, json_data, status_code):
self.json_data = json_data
self.status_code = status_code
def json(self):
return self.json_data
You add the MockResponseAsync class您添加 MockResponseAsync 类
class MockResponseAsync:
def __init__(self, json_data, status_code):
self.response = MockResponse(json_data, status_code)
async def getResponse(self):
return self.response
Here is the test.这是测试。 The important thing here is I create the response before since init function can't be async and the call to getResponse is async so it all checked out.这里重要的是我之前创建了响应,因为init函数不能是异步的,并且对 getResponse 的调用是异步的,所以它全部被检查出来。
@pytest.mark.asyncio
@patch('httpx.AsyncClient')
async def test_get_user_info_valid(self, mock_post):
"""test_get_user_info_valid"""
# Given
token_bd = "abc"
username = "bob"
payload = {
'USERNAME': username,
'DBNAME': 'TEST'
}
headers = {
'Authorization': 'Bearer ' + token_bd,
'Content-Type': 'application/json'
}
async_response = MockResponseAsync("", 200)
mock_post.return_value.post.return_value = async_response.getResponse()
# When
await api_bd.get_user_info(headers, payload)
# Then
mock_post.return_value.post.assert_called_once_with(
URI, json=payload, headers=headers)
If you have a better way of doing that tell me but I think it's pretty clean like that.如果你有更好的方法告诉我,但我认为这样很干净。
The simplest way so far:迄今为止最简单的方法:
from unittest import TestCase
from unittest.mock import Mock, patch
from .utils import method_foo
class TestFoo(TestCase):
@patch.object(utils_requests, "post") # change to desired method here
def test_foo(self, mock_requests_post):
# EXPLANATION: mocked 'post' method above will return some built-in mock,
# and its method 'json' will return mock 'mock_data',
# which got argument 'return_value' with our data to be returned
mock_data = Mock(return_value=[{"id": 1}, {"id": 2}])
mock_requests_post.return_value.json = mock_data
method_foo()
# TODO: asserts here
"""
Example of method that you can test in utils.py
"""
def method_foo():
response = requests.post("http://example.com")
records = response.json()
for record in records:
print(record.get("id"))
# do other stuff here
For those, who don't want to install additional libs for pytest , there is an example.对于那些不想为pytest安装额外库的人, 有一个例子。 I will duplicate it here with some extension, based on examples above:根据上面的示例,我将在此处复制它并添加一些扩展名:
import datetime
import requests
class MockResponse:
def __init__(self, json_data, status_code):
self.json_data = json_data
self.status_code = status_code
self.elapsed = datetime.timedelta(seconds=1)
# mock json() method always returns a specific testing dictionary
def json(self):
return self.json_data
def test_get_json(monkeypatch):
# Any arguments may be passed and mock_get() will always return our
# mocked object, which only has the .json() method.
def mock_get(*args, **kwargs):
return MockResponse({'mock_key': 'mock_value'}, 418)
# apply the monkeypatch for requests.get to mock_get
monkeypatch.setattr(requests, 'get', mock_get)
# app.get_json, which contains requests.get, uses the monkeypatch
response = requests.get('https://fakeurl')
response_json = response.json()
assert response_json['mock_key'] == 'mock_value'
assert response.status_code == 418
assert response.elapsed.total_seconds() == 1
============================= test session starts ==============================
collecting ... collected 1 item
test_so.py::test_get_json PASSED [100%]
============================== 1 passed in 0.07s ===============================
Just a helpful hint to those that are still struggling, converting from urllib or urllib2/urllib3 to requests AND trying to mock a response- I was getting a slightly confusing error when implementing my mock:只是对那些仍在苦苦挣扎的人的有用提示,从 urllib 或 urllib2/urllib3 转换为请求并尝试模拟响应 - 我在实现模拟时遇到了一个稍微令人困惑的错误:
with requests.get(path, auth=HTTPBasicAuth('user', 'pass'), verify=False) as url:
AttributeError: __enter__属性错误:__enter__
Well, of course, if I knew anything about how with
works (I didn't), I'd know it was a vestigial, unnecessary context (from PEP 343 ).好吧,当然,如果我对工作原理with
任何了解(我不知道),我会知道这是一个退化的、不必要的上下文(来自PEP 343 )。 Unnecessary when using the requests library because it does basically the same thing for you under the hood .使用 requests 库时是不必要的,因为它在后台为您做的事情基本相同。 Just remove the with
and use bare requests.get(...)
and Bob's your uncle .只需删除with
并使用裸requests.get(...)
和Bob's your uncle 。
For pytest users there is a convinient fixture from https://pypi.org/project/pytest-responsemock/对于 pytest 用户,有一个来自https://pypi.org/project/pytest-responsemock/的便利装置
For example to mock GET to http://some.domain you can:例如模拟 GET 到http://some.domain你可以:
def test_me(response_mock):
with response_mock('GET http://some.domain -> 200 :Nice'):
response = send_request()
assert result.ok
assert result.content == b'Nice'
I will demonstrate how to detach your programming logic from the actual external library by swapping the real request with a fake one that returns the same data.我将演示如何通过将真实请求与返回相同数据的假请求交换来将您的编程逻辑与实际外部库分离。 In your view if external api call then this process is best在您看来,如果外部 api 调用,那么这个过程是最好的
import pytest
from unittest.mock import patch
from django.test import RequestFactory
@patch("path(projectname.appname.filename).requests.post")
def test_mock_response(self, mock_get, rf: RequestFactory):
mock_get.return_value.ok = Mock(ok=True)
mock_get.return_value.status_code = 400
mock_get.return_value.json.return_value = {you can define here dummy response}
request = rf.post("test/", data=self.payload)
response = view_name_view(request)
expected_response = {
"success": False,
"status": "unsuccessful",
}
assert response.data == expected_response
assert response.status_code == 400
Using requests_mock
is easy to patch any requests使用requests_mock
很容易修补任何请求
pip install requests-mock
from unittest import TestCase
import requests_mock
from <yourmodule> import <method> (auth)
class TestApi(TestCase):
@requests_mock.Mocker()
def test_01_authentication(self, m):
"""Successful authentication using username password"""
token = 'token'
m.post(f'http://localhost/auth', json= {'token': token})
act_token =auth("user", "pass")
self.assertEqual(act_token, token)
import responses
@responses.activate
def test_existence_is_false(self):
responses.add(
responses.GET,
url_path,
json={},
status=404,
)
is_exist = method_send_get_request_to_url_path()
self.assertFalse(is_exist)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.