[英]scrapy ,Why does the scrapy.Request class call the parse() method by default?
为什么scrapy.Request类默认情况下会调用parse()方法,而我对此不太了解
scrapy.Request源代码的一部分
class Request(object_ref):
def __init__(self, url, callback=None, method='GET', headers=None, body=None,
cookies=None, meta=None, encoding='utf-8', priority=0,
dont_filter=False, errback=None, flags=None):
self._encoding = encoding # this one has to be set first
self.method = str(method).upper()
self._set_url(url)
self._set_body(body)
assert isinstance(priority, int), "Request priority not an integer: %r" % priority
self.priority = priority
assert callback or not errback, "Cannot use errback without a callback"
self.callback = callback
self.errback = errback
....
但是这个默认的回调是None,所以我对此很困惑
if "msg" in text_json and text_json["msg"] == "login":
for url in self.start_urls:
yield scrapy.Request(url, dont_filter=True, headers=self.headers)
这是在Scrapy内核中确定的 ,请参阅以下request.callback or spider.parse
部分:
def call_spider(self, result, request, spider):
result.request = request
dfd = defer_result(result)
dfd.addCallbacks(request.callback or spider.parse, request.errback)
return dfd.addCallback(iterate_spider_output)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.