I have a question, I've started using Scrapy over the BeautifulSoup and I new to Scrapy so this error threw me away, I've tried to fix it bit with no success.
I wanted to use Item Loader to yield scrapped items, but I get error, I do not understand what is the problem.
Can someone explain to me what am I doing wrong.
This is my spider code, quotes.py :
from scrapy import Spider
from scrapy.loader import ItemLoader
from quotes_spider.items import QuotesSpiderItem
class QuotesSpider(Spider):
name = 'quotes'
allowed_domains = ['http://quotes.toscrape.com']
start_urls = ["http://quotes.toscrape.com"]
def parse(self, response):
l = ItemLoader(item=QuotesSpiderItem(), response=response)
quotes = response.xpath('//*[@class="quote"]')
for quote in quotes:
text = quote.xpath('.//*[@class="text"]/text()').extract_first()
author = quote.xpath('.//*[@itemprop="author"]/text()').extract_first()
tags = quote.xpath('.//*[@class="tag"]/text()').extract_first ()
l.add_value('Text', text)
l.add_value('Author', author)
l.add_value('Tags', tags)
return l.load_item()
This is my items.py file:
import scrapy
class QuotesSpiderItem(scrapy.Item):
text = scrapy.Field()
author = scrapy.Field()
tags = scrapy.Field()
This the error that I get when I try to run it:
Traceback (most recent call last):
File "/home/djurovic/.local/lib/python3.6/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/djurovic/Desktop/quotes_spider/quotes_spider/spiders/quotes.py", line 20, in parse
l.add_value('Text', text)
File "/home/djurovic/.local/lib/python3.6/site-packages/scrapy/loader/__init__.py", line 77, in add_value
self._add_value(field_name, value)
File "/home/djurovic/.local/lib/python3.6/site-packages/scrapy/loader/__init__.py", line 91, in _add_value
processed_value = self._process_input_value(field_name, value)
File "/home/djurovic/.local/lib/python3.6/site-packages/scrapy/loader/__init__.py", line 148, in _process_input_value
proc = self.get_input_processor(field_name)
File "/home/djurovic/.local/lib/python3.6/site-packages/scrapy/loader/__init__.py", line 137, in get_input_processor
self.default_input_processor)
File "/home/djurovic/.local/lib/python3.6/site-packages/scrapy/loader/__init__.py", line 154, in _get_item_field_attr
value = self.item.fields[field_name].get(key, default)
You have no fields Text
, Author
and Tags
in QuotesSpiderItem
, their names are text
, author
and tags
. Letter case matters.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.