無法在雲上部署 scrapy 蜘蛛（zyte）：AttributeError：'ScrapyArgumentParser' object 沒有屬性'add_option'

Question

我正在嘗試部署我制作的蜘蛛爬蟲，代碼在我的計算機上運行良好，當我嘗試將其部署到https://app.zyte.com/時，為了在雲上運行它我得到了這個錯誤：

warnings.warn("Setuptools is replacing distutils.")
Packing version 1.0
Deploying to Scrapy Cloud project "615653"
Deploy log last 30 lines:

Deploy log location: C:\Users\hp\AppData\Local\Temp\shub_deploy_4a5i8o6z.log

    Error: Deploy failed: b'{"status": "error", "message": "Internal error"}'
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 148, in _run_usercode
        _run(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 103, in _run
        _run_scrapy(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 111, in _run_scrapy
        execute(settings=settings)
      File "/app/python/lib/python3.8/site-packages/scrapy/cmdline.py", line 149, in execute
        cmd.add_options(parser)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/commands/shub_image_info.py", line 23, in add_options
        parser.add_option("--debug", action="store_true",
    AttributeError: 'ScrapyArgumentParser' object has no attribute 'add_option'
    Traceback (most recent call last):
      File "/usr/local/bin/shub-image-info", line 8, in <module>
        sys.exit(shub_image_info())
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 209, in shub_image_info
        _run_usercode(None, ['scrapy', 'shub_image_info'] + sys.argv[1:],
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 148, in _run_usercode
        _run(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 103, in _run
        _run_scrapy(args, settings)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/crawl.py", line 111, in _run_scrapy
        execute(settings=settings)
      File "/app/python/lib/python3.8/site-packages/scrapy/cmdline.py", line 149, in execute
        cmd.add_options(parser)
      File "/usr/local/lib/python3.8/site-packages/sh_scrapy/commands/shub_image_info.py", line 23, in add_options
        parser.add_option("--debug", action="store_true",
    AttributeError: 'ScrapyArgumentParser' object has no attribute 'add_option'
    {"message": "shub-image-info exit code: 1", "details": null, "error": "image_info_error"}
    {"status": "error", "message": "Internal error"}

Answer 1

乍一看，它看起來像..可能是在scrapinghub.yml文件中定義的堆棧參數與您的項目中使用的 scrapy 版本之間存在差異的結果。

預計定義的堆棧應與使用的 scrapy 版本匹配。

有關 scrapy 雲堆棧的更多信息（ 文檔）。