[英]Scrapy CrawlSpider Attribute Not Found
使用Scrapy 1.4.0和我在網上找到的修改后的模板,出現以下錯誤:
AttributeError:模塊“ scrapy”沒有屬性“ CrawlSpider”
該日志似乎沒有顯示其他任何令人感興趣的內容。
碼:
import scrapy
from scrapy.spiders import Rule
from scrapy.linkextractors import LinkExtractor
class TechcrunchSpider(scrapy.CrawlSpider):
#name of the spider
name = 'stltoday'
#list of allowed domains
allowed_domains = ['http://graphics.stltoday.com/']
#starting url for scraping
start_urls = ['http://graphics.stltoday.com/apps/payrolls/salaries/2_1/']
rules = [
Rule(LinkExtractor(
allow=['/apps/payrolls/salaries/.*/$']),
callback='parse',
follow=True),
]
#setting the location of the output csv file
custom_settings = {
'FEED_URI' : 'tmp/stltoday.csv'
}
def parse(self, response):
#Remove XML namespaces
response.selector.remove_namespaces()
#Extract article information
name = response.xpath("//th::text").extract()
allother = response.xpath('//table[@class="table--department"]//td').extract()
for item in zip(name,allother):
scraped_info = {
'name' : item[0],
'allother' : item[1]
}
yield scraped_info
該錯誤意味着scrapy
模塊不包含CrawlSpider
類。 如評論中所述,這是因為scrapy文檔發生了更改。 快速解決方案應該是改變
class TechcrunchSpider(scrapy.CrawlSpider):
至
class TechcrunchSpider(scrapy.spiders.CrawlSpider):
這應該解決問題!
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.