[英]How can I see the entire HTTP request that's being sent by my Python application?
In my case, I'm using the requests
library to call PayPal's API over HTTPS.就我而言,我使用
requests
库通过 HTTPS 调用 PayPal 的 API。 Unfortunately, I'm getting an error from PayPal, and PayPal support cannot figure out what the error is or what's causing it.不幸的是,我收到了来自 PayPal 的错误,并且 PayPal 支持无法弄清楚错误是什么或导致它的原因。 They want me to "Please provide the entire request, headers included".
他们希望我“请提供整个请求,包括标题”。
How can I do that?我怎样才能做到这一点?
A simple method: enable logging in recent versions of Requests (1.x and higher.)一个简单的方法:在最新版本的请求(1.x 和更高版本)中启用日志记录。
Requests uses the http.client
and logging
module configuration to control logging verbosity, as described here . Requests 使用
http.client
和logging
模块配置来控制日志记录的详细程度,如here所述。
Code excerpted from the linked documentation:从链接文档中摘录的代码:
import requests
import logging
# These two lines enable debugging at httplib level (requests->urllib3->http.client)
# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# The only thing missing will be the response.body which is not logged.
try:
import http.client as http_client
except ImportError:
# Python 2
import httplib as http_client
http_client.HTTPConnection.debuglevel = 1
# You must initialize logging, otherwise you'll not see debug output.
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
requests.get('https://httpbin.org/headers')
$ python requests-logging.py
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): httpbin.org
send: 'GET /headers HTTP/1.1\r\nHost: httpbin.org\r\nAccept-Encoding: gzip, deflate, compress\r\nAccept: */*\r\nUser-Agent: python-requests/1.2.0 CPython/2.7.3 Linux/3.2.0-48-generic\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Type: application/json
header: Date: Sat, 29 Jun 2013 11:19:34 GMT
header: Server: gunicorn/0.17.4
header: Content-Length: 226
header: Connection: keep-alive
DEBUG:requests.packages.urllib3.connectionpool:"GET /headers HTTP/1.1" 200 226
r = requests.get('https://api.github.com', auth=('user', 'pass'))
r
is a response. r
是响应。 It has a request attribute which has the information you need.它有一个请求属性,其中包含您需要的信息。
r.request.allow_redirects r.request.headers r.request.register_hook
r.request.auth r.request.hooks r.request.response
r.request.cert r.request.method r.request.send
r.request.config r.request.params r.request.sent
r.request.cookies r.request.path_url r.request.session
r.request.data r.request.prefetch r.request.timeout
r.request.deregister_hook r.request.proxies r.request.url
r.request.files r.request.redirect r.request.verify
r.request.headers
gives the headers: r.request.headers
给出标题:
{'Accept': '*/*',
'Accept-Encoding': 'identity, deflate, compress, gzip',
'Authorization': u'Basic dXNlcjpwYXNz',
'User-Agent': 'python-requests/0.12.1'}
Then r.request.data
has the body as a mapping.然后
r.request.data
将主体作为映射。 You can convert this with urllib.urlencode
if they prefer:如果他们愿意,您可以使用
urllib.urlencode
进行转换:
import urllib
b = r.request.data
encoded_body = urllib.urlencode(b)
depending on the type of the response the .data
-attribute may be missing and a .body
-attribute be there instead.根据响应的类型,
.data
-attribute 可能会丢失,而.body
-attribute 会代替。
You can use HTTP Toolkit to do exactly this.您可以使用HTTP Toolkit来完成此操作。
It's especially useful if you need to do this quickly, with no code changes: you can open a terminal from HTTP Toolkit, run any Python code from there as normal, and you'll be able to see the full content of every HTTP/HTTPS request immediately.如果您需要快速执行此操作而无需更改代码,它尤其有用:您可以从 HTTP Toolkit 打开终端,照常从那里运行任何 Python 代码,您将能够看到每个 HTTP/HTTPS 的完整内容立即请求。
There's a free version that can do everything you need, and it's 100% open source.有一个免费版本可以做你需要的一切,而且它是 100% 开源的。
I'm the creator of HTTP Toolkit;我是 HTTP Toolkit 的创建者; I actually built it myself to solve the exact same problem for me a while back!
实际上,我自己构建了它来为我解决完全相同的问题! I too was trying to debug a payment integration, but their SDK didn't work, I couldn't tell why, and I needed to know what was actually going on to properly fix it.
我也试图调试支付集成,但他们的 SDK 不起作用,我不知道为什么,我需要知道实际发生了什么才能正确修复它。 It's very frustrating, but being able to see the raw traffic really helps.
这非常令人沮丧,但能够看到原始流量确实有帮助。
If you're using Python 2.x, try installing a urllib2 opener.如果您使用的是 Python 2.x,请尝试安装urllib2 opener。 That should print out your headers, although you may have to combine that with other openers you're using to hit the HTTPS.
这应该会打印出您的标头,尽管您可能必须将其与用于访问 HTTPS 的其他开启器结合起来。
import urllib2
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPHandler(debuglevel=1)))
urllib2.urlopen(url)
The verbose
configuration option might allow you to see what you want. verbose
配置选项可能允许您查看所需内容。 There is an example in the documentation . 文档中有一个例子。
NOTE: Read the comments below: The verbose config options doesn't seem to be available anymore.注意:阅读下面的评论:冗长的配置选项似乎不再可用。
A much simpler way to debug HTTP local requests is to use netcat.调试 HTTP 本地请求的一种更简单的方法是使用 netcat。 If you run
如果你跑
nc -l 1234
you'll start listening on port 1234
for HTTP connections.您将开始在端口
1234
上侦听 HTTP 连接。 You can access it via http://localhost:1234/foo/foo/...
.您可以通过
http://localhost:1234/foo/foo/...
访问它。
On the terminal, you'll see whatever raw data you sent to the endpoint.在终端上,您将看到发送到端点的任何原始数据。 For example:
例如:
POST /foo/foo HTTP/1.1
Accept: application/json
Connection: keep-alive
Host: example.com
Accept-Language: en-en
Authorization: Bearer ay...
Content-Length: 15
Content-Type: application/json
{"test": false}
No logging system completely works, (as of requests 2.26 anyway, very old versions had maybe another behaviour)没有日志系统完全有效,(无论如何,从请求 2.26 开始,非常旧的版本可能有另一种行为)
The good solution is to use 'hooks' and print details as they happen.好的解决方案是使用“钩子”并在它们发生时打印细节。
This is fairly well explained here : https://findwork.dev/blog/advanced-usage-python-requests-timeouts-retries-hooks/这在这里得到了很好的解释: https : //findwork.dev/blog/advanced-usage-python-requests-timeouts-retries-hooks/
under "printing everything",在“打印所有内容”下,
but in case the link dies here is the important parts但如果链接失效,这里是重要的部分
import requests
from requests_toolbelt.utils import dump
def logging_hook(response, *args, **kwargs):
data = dump.dump_all(response)
print(data.decode('utf-8'))
http = requests.Session()
http.hooks["response"] = [logging_hook]
http.get("https://api.openaq.org/v1/cities", params={"country": "BA"})
The result this time will be a complete trace of sent query and received response.这次的结果将是发送查询和接收响应的完整跟踪。
I've tried it successfully with POST and lots of headers : it works.我已经用 POST 和很多标题成功地尝试过它:它有效。 Don't forget to pip install requests_toolbelt.
不要忘记 pip install requests_toolbelt。
# Output
< GET /v1/cities?country=BA HTTP/1.1
< Host: api.openaq.org
> HTTP/1.1 200 OK
> Content-Type: application/json; charset=utf-8
> Transfer-Encoding: chunked
> Connection: keep-alive
>
{
"meta":{
"name":"openaq-api",
"license":"CC BY 4.0",
"website":"https://docs.openaq.org/",
"page":1,
"limit":100,
"found":1
},
"results":[
{
"country":"BA",
"name":"Goražde",
"city":"Goražde",
"count":70797,
"locations":1
}
]
}
A previous answer seems to have been downvoted because it started with "nothing completely works" and then provides this perfect solution:先前的答案似乎已被否决,因为它以“没有任何东西完全有效”开头,然后提供了这个完美的解决方案:
requests_toolbelt
collection of utilities with pip install requests-toolbelt
.requests_toolbelt
与公用事业收集pip install requests-toolbelt
。 import requests from requests_toolbelt.utils import dump response = requests.get("https://v2.jokeapi.dev/joke/Any?safe-mode") print(dump.dump_all(response).decode("utf-8"))
As others have noted there is a nice requests-toolbelt
module with convenient functions to dump request and response content using requests hooks.正如其他人所指出的,有一个不错的
requests-toolbelt
模块,它具有使用请求钩子转储请求和响应内容的便捷功能。 Unfortunately (as of now) there is only a hook to be invoked on a successful completion of a request.不幸的是(到目前为止)只有一个钩子可以在请求成功完成时被调用。 It's not always a case.
情况并非总是如此。 Ie request could end up with
ConnectionError
or Timeout
exceptions.即请求可能以
ConnectionError
或Timeout
异常结束。
The requests-toolbelt
module on it's own also provides public functions to dump completed requests only. requests-toolbelt
模块本身也提供公共功能来仅转储已完成的请求。 However using a bit of non-public APIs and Session sub-classing it's possible to implement logging of requests before send and logging of responses after receive.然而,使用一些非公共 API 和 Session 子类,可以在发送之前实现请求日志记录,在接收之后实现响应日志记录。
NOTE: code relies on implementation details/non-public APIs of requests-toolbelt
module and thus make unexpectedly break in the future:注意:代码依赖于
requests-toolbelt
模块的实现细节/非公共 API,因此将来会意外中断:
import requests
from requests_toolbelt.utils import dump
class MySession(requests.Session):
def send(self, req, *args, **kwargs):
prefixes = dump.PrefixSettings(b'< ', b'> ')
data = bytearray()
try:
dump._dump_request_data(req, prefixes, data)
resp = super().send(req, *args, **kwargs)
dump._dump_response_data(resp, prefixes, data)
finally:
print(data.decode('utf-8'))
return resp
and here is an example of usage:这是一个使用示例:
>>> MySession().get('https://httpbin.org/headers')
< GET /headers HTTP/1.1
< Host: httpbin.org
< User-Agent: python-requests/2.25.1
< Accept-Encoding: gzip, deflate
< Accept: */*
< Connection: keep-alive
<
> HTTP/1.1 200 OK
> Date: Fri, 19 Aug 2022 10:43:51 GMT
> Content-Type: application/json
> Content-Length: 225
> Connection: keep-alive
> Server: gunicorn/19.9.0
> Access-Control-Allow-Origin: *
> Access-Control-Allow-Credentials: true
>
{
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate",
"Host": "httpbin.org",
"User-Agent": "python-requests/2.25.1"
}
}
>>> MySession().get('https://non.existent')
< GET / HTTP/1.1
< Host: non.existent
< User-Agent: python-requests/2.25.1
< Accept-Encoding: gzip, deflate
< Accept: */*
< Connection: keep-alive
<
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 73, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/usr/lib/python3.10/socket.py", line 955, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
...
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.