[英]Selenium Python - Get Network response body
I use Selenium to react to the reception of data following a GET request from a website.我使用 Selenium 对来自网站的 GET 请求后的数据接收做出反应。 The API called by the website is not public, so if I use the URL of the request to retrieve the data, I get
{"message":"Unauthenticated."}
.网站调用的 API 是不公开的,所以如果我使用请求的 URL 来检索数据,我会得到
{"message":"Unauthenticated."}
。
All I've managed to do so far is to retrieve the header of the response.到目前为止,我所做的只是检索响应的 header。
I found here that using driver.execute_cdp_cmd('Network.getResponseBody', {...})
might be a solution to my problem.我在这里发现使用
driver.execute_cdp_cmd('Network.getResponseBody', {...})
可能是我的问题的解决方案。
Here is a sample of my code:这是我的代码示例:
import json
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
capabilities = DesiredCapabilities.CHROME
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}
driver = webdriver.Chrome(
r"./chromedriver",
desired_capabilities=capabilities,
)
def processLog(log):
log = json.loads(log["message"])["message"]
if ("Network.response" in log["method"] and "params" in log.keys()):
headers = log["params"]["response"]
body = driver.execute_cdp_cmd('Network.getResponseBody', {'requestId': log["params"]["requestId"]})
print(json.dumps(body, indent=4, sort_keys=True))
return log["params"]
logs = driver.get_log('performance')
responses = [processLog(log) for log in logs]
Unfortunately, the driver.execute_cdp_cmd('Network.getResponseBody', {...})
returns:不幸的是,
driver.execute_cdp_cmd('Network.getResponseBody', {...})
返回:
unknown error: unhandled inspector error: {"code":-32000,"message":"No resource with given identifier found"}
Do you know what I am missing?你知道我错过了什么吗?
Do you have any idea on how to retrieve the response body?您对如何检索响应正文有任何想法吗?
Thank you for your help!谢谢您的帮助!
Probably some responses don't have a body, thus selenium throws an error that "no resource" for given identifier was found.可能有些响应没有正文,因此 selenium 会抛出一个错误,即“找不到给定标识符的资源”。 Error message is a bit ambiguous here.
错误消息在这里有点模棱两可。
Try doing like this:尝试这样做:
from selenium.common import exceptions
try:
body = chromedriver.execute_cdp_cmd('Network.getResponseBody', {'requestId': log["params"]["requestId"]})
log['body'] = body
except exceptions.WebDriverException:
print('response.body is null')
This way responses without body will not crash your script.这样没有正文的响应不会使您的脚本崩溃。
In order to retrieve response body, you have to listen specifically to Network.responseReceived
:为了检索响应正文,您必须专门收听
Network.responseReceived
:
def processLog(log):
log = json.loads(log["message"])["message"]
if ("Network.responseReceived" in log["method"] and "params" in log.keys()):
body = driver.execute_cdp_cmd('Network.getResponseBody', {'requestId': log["params"]["requestId"]})
However, I ended using a different approach relying on requests
.但是,我最终使用了另一种依赖
requests
的方法。 I just retrieved the authorization token from the browser console (Network > Headers > Request Headers > Authorization) and used it to get the data I wanted:我刚刚从浏览器控制台(网络 > 标头 > 请求标头 > 授权)中检索了授权令牌,并使用它来获取我想要的数据:
import requests
def get_data():
url = "<your_url>"
headers = {
"Authorization": "Bearer <your_access_token>",
"Content-type": "application/json"
}
params = {
key: value,
...
}
r = requests.get(url, headers = headers, params = params)
if r.status_code == 200:
return r.json()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.