Scrapy：如何从settings.py文件应用设置

Question

我已经在settings.py文件中指定了外部文件的名称，我想在启动时将其导入。 但是，当我运行它时，出现错误： AttributeError: 'MySpider' object has no attribute 'settings' 。 如何在启动Spider时导入？

到目前为止，我的代码是：

class MySpider(CrawlSpider):

    name = 'MySpider'

    def __init__(self, *args, **kwargs):

        super(MySpider, self).__init__(*args, **kwargs)

        self.crawl_behaviour = FileOps.get_crawl_behaviour(self.settings.attributes.get('CRAWL_BEHAVIOUR'))

        self.rules = (

            Rule(LinkExtractor(allow=self.crawl_behaviour), follow=True),

        )

Answer 1

这对我有用。

from scrapy.conf import settings


self.crawl_behaviour = settings['CRAWL_BEHAVIOUR']

Answer 2

您可以这样使用：

from scrapy import Spider
from scrapy.utils.project import get_project_settings

class MySpider(Spider):
    name = "MySpider"
    allowed_domains = ["example.com"]
    start_urls = ['https://www.example.com/']

    def __init__(self, query='', *args, **kwargs):
        super(MySpider, self).__init__(*args, **kwargs)
        p_setting = get_project_settings()
        self.start_urls = ['https://www.example.com']
        self.query = query
        self.crawl_behavior = p_setting.get("CRAWL_BEHAVIOUR")