无法在云上部署 scrapy 蜘蛛（zyte）：AttributeError：'ScrapyArgumentParser' object 没有属性'add_option'

Question

我正在尝试部署我制作的蜘蛛爬虫，代码在我的计算机上运行良好，当我尝试将其部署到https://app.zyte.com/时，为了在云上运行它我得到了这个错误：

warnings.warn("Setuptools is replacing distutils.")
Packing version 1.0
Deploying to Scrapy Cloud project "615653"
Deploy log last 30 lines:

Deploy log location: C:\Users\hp\AppData\Local\Temp\shub_deploy_4a5i8o6z.log

    Error: Deploy failed: b'{"status": "error", "message": "Internal error"}'
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 148, in _run_usercode
        _run(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 103, in _run
        _run_scrapy(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 111, in _run_scrapy
        execute(settings=settings)
      File "/app/python/lib/python3.8/site-packages/scrapy/cmdline.py", line 149, in execute
        cmd.add_options(parser)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/commands/shub_image_info.py", line 23, in add_options
        parser.add_option("--debug", action="store_true",
    AttributeError: 'ScrapyArgumentParser' object has no attribute 'add_option'
    Traceback (most recent call last):
      File "/usr/local/bin/shub-image-info", line 8, in <module>
        sys.exit(shub_image_info())
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 209, in shub_image_info
        _run_usercode(None, ['scrapy', 'shub_image_info'] + sys.argv[1:],
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 148, in _run_usercode
        _run(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 103, in _run
        _run_scrapy(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 111, in _run_scrapy
        execute(settings=settings)
      File "/app/python/lib/python3.8/site-packages/scrapy/cmdline.py", line 149, in execute
        cmd.add_options(parser)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/commands/shub_image_info.py", line 23, in add_options
        parser.add_option("--debug", action="store_true",
    AttributeError: 'ScrapyArgumentParser' object has no attribute 'add_option'
    {"message": "shub-image-info exit code: 1", "details": null, "error": "image_info_error"}
    {"status": "error", "message": "Internal error"}

Answer 1

乍一看，它看起来像..可能是在scrapinghub.yml文件中定义的堆栈参数与您的项目中使用的 scrapy 版本之间存在差异的结果。

预计定义的堆栈应与使用的 scrapy 版本匹配。

有关 scrapy 云堆栈的更多信息（ 文档）。