[英]Scrapy: How to get return values from Scrapy.Request?
I'm currently trying to figure out how to get a return value from a scrapy request.我目前正在尝试弄清楚如何从 scrapy 请求中获取返回值。 The first snippet shows the way I'm doing the request.
第一个片段显示了我执行请求的方式。 In the parse function (2nd snippet) I'm extracting some values which I would like to return with two lists.
在解析 function (第二个片段)中,我正在提取一些我想用两个列表返回的值。 My question is: How can I get these two values after the yield statement in the first snippet?
我的问题是:如何在第一个代码段中的 yield 语句之后获得这两个值?
while i < pages:
# default link is combined with the specific page number
url = link+str(i)
yield scrapy.Request(url=url, callback=self.parse)
i = i+1
2nd snippet:第二个片段:
def parse(self, response):
# here the code is filling x_array and y_array with values
return x_array, y_array
Thank you so much for your help!!非常感谢你的帮助!!
Instead of just returning values, Requests from Scrapy can fill up Items (a dictionary-like structure), which you can treat further in Item Pipelines.来自 Scrapy 的请求不仅可以返回值,还可以填充 Items(类似字典的结构),您可以在 Item Pipelines 中进一步处理。 In your case, it suffices to add this to your item.py file:
在您的情况下,将其添加到您的 item.py 文件中就足够了:
from scrapy.item import Item, Field
class TestItem(Item):
x_array = Field()
y_array = Field()
In your spider, you fill up the item and yield it like this:在你的蜘蛛中,你填写项目并像这样产生它:
from ..items import TestItem
def parse(self, response):
x_array = ['test1', 'test2']
y_array = ['test3']
item = TestItem()
item['x_array'] = x_array
item['y_array'] = y_array
yield item
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.