AttributeError: 'module' object 没有属性 'Spider'

Question

刚开始学scrapy，所以跟着scrapy的文档。 我刚刚写了那个站点中提到的第一个蜘蛛。

import scrapy

class DmozSpider(scrapy.Spider):
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
    ]

    def parse(self, response):
        filename = response.url.split("/")[-2]
        with open(filename, 'wb') as f:
            f.write(response.body)

在项目的根目录上运行此scrapy crawl dmoz命令后，它显示以下错误。

2015-06-07 21:53:06+0530 [scrapy] INFO: Scrapy 0.14.4 started (bot: tutorial)
2015-06-07 21:53:06+0530 [scrapy] DEBUG: Enabled extensions: LogStats, TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
Traceback (most recent call last):
  File "/usr/bin/scrapy", line 4, in <module>
    execute()
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 132, in execute
    _run_print_help(parser, _run_command, cmd, args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 97, in _run_print_help
    func(*a, **kw)
  File "/usr/lib/python2.7/dist-packages/scrapy/cmdline.py", line 139, in _run_command
    cmd.run(args, opts)
  File "/usr/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 43, in run
    spider = self.crawler.spiders.create(spname, **opts.spargs)
  File "/usr/lib/python2.7/dist-packages/scrapy/command.py", line 34, in crawler
    self._crawler.configure()
  File "/usr/lib/python2.7/dist-packages/scrapy/crawler.py", line 36, in configure
    self.spiders = spman_cls.from_crawler(self)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 37, in from_crawler
    return cls.from_settings(crawler.settings)
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 33, in from_settings
    return cls(settings.getlist('SPIDER_MODULES'))
  File "/usr/lib/python2.7/dist-packages/scrapy/spidermanager.py", line 23, in __init__
    for module in walk_modules(name):
  File "/usr/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 65, in walk_modules
    submod = __import__(fullpath, {}, {}, [''])
  File "/home/avinash/tutorial/tutorial/spiders/dmoz_spider.py", line 3, in <module>
    class DmozSpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'

Answer 1

您正在使用旧 Scrapy（0.14.4）和最新的文档。

解决方案：升级到最新版本的Scrapy或读取适合当前安装版本的旧文档

Answer 2

使用sudo pip install scrapy 。 如果你得到Python.h missing错误，那么用sudo apt-get install python-dev安装python头文件（ 参考 ）

Answer 3

在我的例子中，错误信息有点不同

AttributeError: partially initialized module 'scrapy' has no attribute 'Spider' (很可能是由于循环导入)

这是由于我的脚本文件的名称 (scrapy.py)。 请确保您正在运行的目录中没有另一个名为scrapy.py的文件，因为它将在真正的 scrapy 模块之前导入。

Answer 4

也许试试：

from scrapy import Spider

如果你想使用它的类，仅仅导入模块是不够的