如何在Python中重新运行scrapy请求对象？

Question

I am using Scrapy, and in the response_is_ban function, I have two objects, a {Request} object, and a {Response} object.

from scrapy.http import Request

For some reason, the Scrapy Request is returning different results than a standard Curl.

Is there a way I can rerun the {Request} object?

Something like:

response = request.get_again()

The goal is to figure out exactly what is different between my Curl function and the Scrapy request object, and why they would be returning different results. Any suggestions would be welcomed.

Edit: Another question I have, where in the code does Scrapy actually execture the request? Is there a request.get() function somewhere I could put a debug point on?

Answer 1

对于调试，您可以从scrapy.shell会话中使用fetch ： https : scrapy.shell