如何从管道访问scrapy统计信息

Question

From the scrapy api I know that a crawler contains the stats attribute, but how can I access it from a custom pipeline?

class MyPipeline(object):

    def __init__(self): 
        self.stats = ???

Answer 1

Your pipeline is an extension and you want it to access the stats attribute. An extension receives the Crawler object through the from_crawler(cls, crawler) method.

All in all, you should do something like

def __init__(self, stats):
    self.stats = stats

@classmethod
def from_crawler(cls, crawler):
    return cls(crawler.stats)

http://scrapy.readthedocs.org/en/latest/topics/stats.html#topics-stats

Answer 2

also stats available from spider.crawler, for example (v1.1.0):

class ObjPipeline(object):
    def process_item(self, item, spider):
        spider.crawler.stats.inc_value('scraped_items')
        ...