[英]Scrapy: TypeError: __init__() missing 1 required positional argument: 'settings'
I have a Scrapy middleware: 我有一个Scrapy中间件:
class ProxyMiddleware(object):
def __init__(self, settings):
self.proxy_file = settings.get('PROXY_FILE')
fin = open(self.proxy_file)
self.proxy_list = list()
for line in fin.readlines():
parts = line.strip().split()
proxy = parts[2] + '://' + parts[0] + ':' + parts[1]
self.proxy_list.append(proxy)
def process_request(self, request, spider):
request.meta['proxy'] = random.choice(self.proxy_list)
But an error occurred when running, what's the settings? 但运行时出错,设置是什么?
If you need a settings
object to initialize your middleware, you need to define a from_crawler()
class method, otherwise scrapy initializes the middleware without arguments . 如果需要
settings
对象来初始化中间件,则需要定义from_crawler()
类方法, 否则scrapy会在不使用参数的情况下初始化中间件 。
Take a look at the built-in middleware for inspiration, for example HttpErrorMiddleware
看看内置的中间件获取灵感,例如
HttpErrorMiddleware
class HttpErrorMiddleware(object):
@classmethod
def from_crawler(cls, crawler):
return cls(crawler.settings)
def __init__(self, settings):
self.handle_httpstatus_all = settings.getbool('HTTPERROR_ALLOW_ALL')
self.handle_httpstatus_list = settings.getlist('HTTPERROR_ALLOWED_CODES')
In your case it would be something like: 在你的情况下,它将是这样的:
class ProxyMiddleware(object):
@classmethod
def from_crawler(cls, crawler):
return cls(crawler.settings)
def __init__(self, settings):
self.proxy_file = settings.get('PROXY_FILE')
fin = open(self.proxy_file)
self.proxy_list = list()
for line in fin.readlines():
parts = line.strip().split()
proxy = parts[2] + '://' + parts[0] + ':' + parts[1]
self.proxy_list.append(proxy)
def process_request(self, request, spider):
request.meta['proxy'] = random.choice(self.proxy_list)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.