刮擦异常CloseSpider关闭蜘蛛功能

Question

引发“ CloseSpider”异常时会发生什么以及如何处理。

from scrapy.exceptions import CloseSpider
    #code
    def parse(self,response):
        #code
        if condition:
            raise CloseSpider(reason="bandwidth_exceed")


    @classmethod
    def from_crawler(cls, crawler, *args, **kwargs):
        spider = super(FlickrSearchSpider, cls).from_crawler(crawler, *args, **kwargs)
        crawler.signals.connect(spider.spider_closed, signal=scrapy.signals.spider_closed)
        return spider
    def spider_closed(self):
        print("completed")

我如何区分spider_closed与正常运行或满足的条件（ reason="bandwidth_exceed" ）。 我希望能够尽可能处理不同的问题

Answer 1

非常简单，只需定义closed ：

def closed(self, reason):
    if reason == "bandwidth_exceed":
        process_bandwidth_exceed()
    else:
        something_else()
    pass