如何在解析方法内部访问对象？

Question

I can't figure out how to access an object inside parse function. I want to create an object Check which needs Product object to be created. products attribute is a list of objects which are sources of urls .

class GenericScraper(scrapy.Spider):
    name = 'will_be_overriden'
    custom_settings = {'CONCURRENT_REQUESTS': 32,
                       'DOWNLOAD_DELAY':0.5}
    def __init__(self, occs):
        super(GenericScraper,self).__init__()
        self.name = products[0].site.name
        self.products = products
        self.xpath = self.product[0].site.xpaths.first().xpath

    def start_requests(self):
        for product in self.products:
            yield scrapy.Request(url=product.url, callback=self.parse)

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        text = hxs.select(self.xpath+'/text()').extract()
        Check.objects.create(text=text,product=product) # CAN'T ACCESS CURRENT PRODUCT

        responselog.debug(response)

Is it possible?

Answer 1

use the Request meta attribute to communicate between callbacks. I assume you want to associate a product object to each request you make, so something like:

def start_requests(self):
    for product in self.products:
        yield scrapy.Request(
            url=product.url, 
            callback=self.parse, 
            meta={'product': product},
        )

def parse(self, response):
    current_product = response.meta['product']
    ...