Call a function in Settings from spider Scrapy

Question

I have two spiders A and B . I need to call a function which is defined in the spider settings.py file

Project Name |--Project Name | |-- spiders | | |-- __init__.py | | |-- A.py | | |-- B.py | |-- __init__.py | |-- items.py | |-- pipelines.py | |-- settings.py

There is a function in settings.py, which I need to access from A.py and B.py on close of spider

settings.py

def revoke_ip():
    logging.info('Revoking access')

This is what I have tried from A.py:

def closed(self, reason):
    logging.info('Spider terminating because of %s' % reason)
    current_project_settings = get_project_settings()
    revoke_ip_call = getattr(current_project_settings, "revoke_ip")
    revoke_ip_call()

But this thing doesn't work nor does what is mentioned here

Is there anything that I am doing wrong or any other way to do it?

Answer 1

When importing a file, Python only searches the current directory, the directory that the entry-point script is running from, and sys.path which includes locations such as the package installation directory. You can import the settings file to call the function. To do this, add this to your function:

import sys
sys.path.insert(0, '../')
import settings

Answer 2

What worked for me, thanks to the answer that @Jose posted, was that as settings.py was in a different directory than the spider which I was running and Python only searches the current directory.

So I tried checking the path of the file that it gives every-time I run the spider and apparently the path that I get was

/tmp/unpacked-eggs/__main__.egg/project name/spiders

So, what I had to do was:

import sys
import os
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)) + '/../')
import settings