带有scrapy版本0.22.1的多页抓取-“无法导入名称CrawlSpider”错误是什么意思？

Question

I am trying to write a spider to crawl across multiple pages, via the following URL: http://bookshop.lawsociety.org.uk/ecom_lawsoc/public/saleproducts.jsf?catId=EBOOK I'm using Scrapy version 0.22.1 to do this. However, I am getting a "cannot import name CrawlSpider" message. I have pasted the code for the spider below. Can someone determine where I have gone wrong here?

from scrapy.spider import CrawlSpider, Rule
from scrapy.linkextractors.sgml import SgmlLinkExtractor
from scrapy.selector import Selector
from scrapy.item import BookpagesItem 

class BookpagesSpider(CrawlSpider):
name = "book_sample"
allowed_domains = ["bookshop.lawsociety.org.uk"]
start_urls = ["http://bookshop.lawsociety.org.uk/ecom_lawsoc/public/saleproducts.jsf?catId=EBOOK",
              ]
rules = (
    Rule(SgmlLinkExtractor(allow=('//*[@id="productList:scrollernext"]', )), callback='parse_item', follow= True),
    Rule(SgmlLinkExtractor(allow=('//p/a[contains(@id, "productList")]', )), callback='parse_item', follow= True),
)

def parse_item(self, response):
    sel = Selector(response)
    sites = sel.xpath('//div[@class="dataListDiv"]')
    items = []
    for site in sites:
        item = BooksItem()
        item['title'] = site.xpath('//div/a/h3[@class="saleProductsTitle"]/text()').extract()
        item['link'] = site.xpath('//p/a[contains(@id, "productList")]').extract()
        item['price'] = site.xpath('//*[@class="saleProductsPrice"]/text()').extract()
        item['category'] = site.xpath('//span[contains(@id, "category")]/text()').extract()
        item['authors'] = site.xpath('//span[contains(@id, "author")]/text()').extract()
        item['date'] = site.xpath('//span[contains(@id, "publicationDate")]/text()').extract()
        item['publisher'] = site.xpath('//span[contains(@id, "publisher")]/text()').extract()
        item['isbn'] = site.xpath('//span[contains(@id, "isbn")]/text()').extract()
        items.append(item)
    return items

The items.py code is:

from scrapy.item import Item, Field

class BookpagesItem(Item):
# define the fields for your item here like:
# name = Field()
title = Field()
link = Field()
price = Field()
category = Field()
authors = Field()
date = Field()
publisher = Field()
isbn = Field()

Answer 1

It means from scrapy.spider import CrawlSpider, Rule isn't correct.

Looking at the Scrapy documentation, it should probably be from scrapy.contrib.spiders import CrawlSpider

Any time you get the NameError - Cannot import name foo error, you're looking at an incorrect import, so you can narrow that down to just your import statements. You can look in the library's documentation for the correct location, or the source code itself if that's available.

I searched the scrapy documentation and found this: http://doc.scrapy.org/en/0.24/topics/spiders.html#crawlspider