当带有extract()的Scrapy选择器返回None时如何设置默认值？

Question

I am trying to yield the value of a tag that isn't always present in the pages that I scrape with Scrapy. I am using the extract() function rather than extract_first() . Therefore I cannot seem to set a default value, like suggested in this SO post .

This doesn't work:

def parse(self, response):
        yield {
          'comments': response.css('[itemprop=commentCount]::attr(content)').extract(default=None)
          }

How can I set None as default when I want to use extract() rather than extract_first() ?

Thanks very much in advance!

Answer 1

Try this syntax:

{'comments': response.css('[itemprop=commentCount]::attr(content)').extract() or None}

If result of response.css(CSS) is empty list, then None will be assigned as value of comments key. Otherwise, actual value will be assigned

Answer 2

.extract() yields the output as a list and .extract_first() yields a string.

response.xpath('xpath_of_the_component').extract_first(default="default_value").split()

This line of code will again convert the string to a list and set the default value, if not available.