简体   繁体   中英

Scrapy dmoz tutorial: _init_() takes at most 2 arguments (3 given)

PS C:\\users\\steve\\tutorial> scrapy crawl dmoz

Traceback (most recent call last):

File "c:\python27\scripts\scrapy-script.py", line 9, in <module> 
  load_entry_point('scrapy==1.0.3', 'console_scripts', 'scrapy')()
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\cmdline.py",
  cmd.crawler_process = CrawlerProcess(settings)
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\crawler.py",
  super(CrawlerProcess, self).__init__(settings)
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\crawler.py",
  self.spider_loader = _get_spider_loader(settings)
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\crawler.py",
  return loader_cls.from_settings(settings.frozencopy())
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\spiderloader.
  return cls(settings)
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\spiderloader.
  for module in walk_modules(name):
File "C:\Python27\lib\site-packages\scrapy-1.0.3-py2.7.egg\scrapy\utils\misc.py
  submod = import_module(fullpath)
File "C:\Python27\lib\importlib\__init__.py", line 37, in import_module
  __import__(name)
File "C:\users\steve\tutorial\tutorial\spiders\dmoz.py", line 4, in <module>
  class dmozspider(spiders):
TypeError: Error when calling the metaclass bases module.__init__() takes at most 2 arguments (3 given)

My dmoz spider python script is here

from scrapy import spiders


class dmozspider(spiders):  
    name = "dmoz"
    allowed_domains = ["dmoz.org"]
    start_urls = [
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Books/",
        "http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/"
     ]

    def parse(self, response):
        filename = response.url.split("/")[-2] + '.html'
        with open(filename, 'wb') as f:
            f.write(response.body)

The problem is you're importing "spiders", and using it as your base class. "spiders" is the package that contains the spiders, namely the Spider class. To use it, use:

from scrapy.spiders import Spider


class dmozspider(Spider):
    ...  # Rest of your code

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM