[英]Scrapy works in shell but not in the code
I am facing an issue while developing my first spider in scrapy.我在用scrapy开发我的第一个蜘蛛时遇到了一个问题。 I am able to get the proper information in scrapy shell but it does not work when I implement it in the code.
我能够在scrapy shell中获得正确的信息,但是当我在代码中实现它时它不起作用。 I've read similar posts here but I still was not able to figure out what I'm doing wrong.
我在这里读过类似的帖子,但我仍然无法弄清楚我做错了什么。
import scrapy
from scrapy.loader import ItemLoader
from ..items import ScrapingamazonItem
class AmazonSpiderSpider(scrapy.Spider):
name = 'amazon_spider'
start_urls = ['https://www.amazon.com/s?k=Office+Chair&lo=grid&crid=1N60K12GUA798&qid=1601040579&sprefix=chair&ref=sr_pg_1']
def parse(self, response):
items = response.css('.s-asin .sg-col-inner')
for item in items:
loader = ItemLoader(item=ScrapingamazonItem(), selector=item)
loader.add_css('ProductName', '.a-color-base.a-text-normal::text')
yield loader.load_item()
I am running it using scrapy crawl amazon_spider -o file.csv.我正在使用scrapy crawl amazon_spider -o file.csv 运行它。 The file returns empty.
该文件返回空。
Any help is deeply appreciated!任何帮助深表感谢! :)
:)
Try尝试
for item in items:
loader = ItemLoader(item=ScrapingamazonItem(), selector=item)
loader.add_css('ProductName', '.a-color-base.a-text-normal::text')
yield loader.load_item()
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.