I am using Scrapy, and in the response_is_ban function, I have two objects, a {Request} object, and a {Response} object.
from scrapy.http import Request
For some reason, the Scrapy Request is returning different results than a standard Curl.
Is there a way I can rerun the {Request} object?
Something like:
response = request.get_again()
The goal is to figure out exactly what is different between my Curl function and the Scrapy request object, and why they would be returning different results. Any suggestions would be welcomed.
Edit: Another question I have, where in the code does Scrapy actually execture the request? Is there a request.get() function somewhere I could put a debug point on?
对于调试,您可以从scrapy.shell
会话中使用fetch
: https : scrapy.shell
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.