尝试解析列表数据时，Scrapy抛出Traceback

Question

我在Windows Vista 64位上运行Scrapy.org版本2.7 64位。 我有一些Scrapy代码试图在以下代码中包含的URL处解析表中包含的数据：

from scrapy.spider import Spider
from scrapy.selector import Selector
from scrapy.utils.markup import remove_tags
from scrapy.cmdline import execute
import re


class MySpider(Spider):
    name = "wiki"
    allowed_domains = ["whoscored.com"]
    start_urls = ["http://www.whoscored.com/Players/3859/Fixtures/Wayne-Rooney"]

def parse(self, response):

    for row in response.selector.xpath('//table[@id="player-fixture"]//tr[td[@class="tournament"]]'):
    # Is this row contains goal symbols?
        list_of_goals = row.xpath('//span[@title="Goal"')
        if list_of_goals:
            print remove_tags(list_of_goals).encode('utf-8')     

execute(['scrapy','crawl','wiki'])

但是，它引发以下错误：

Traceback (most recent call last):
  File "c:\Python27\lib\site-packages\twisted\internet\base.py", line 1201, in mainLoop
    self.runUntilCurrent()
  File "c:\Python27\lib\site-packages\twisted\internet\base.py", line 824, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "c:\Python27\lib\site-packages\twisted\internet\defer.py", line 383, in callback
    self._startRunCallbacks(result)
  File "c:\Python27\lib\site-packages\twisted\internet\defer.py", line 491, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "c:\Python27\lib\site-packages\twisted\internet\defer.py", line 578, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "c:\Python27\lib\site-packages\scrapy\spider.py", line 56, in parse
    raise NotImplementedError
exceptions.NotImplementedError:

谁能告诉我这里的问题是什么？ 我正在尝试对表中的所有项目进行屏幕打印，包括目标和辅助列中的数据。

谢谢

Answer 1

您的缩进是错误的：

class MySpider(Spider):
    name = "wiki"
    allowed_domains = ["whoscored.com"]
    start_urls = ["http://www.whoscored.com/Players/3859/Fixtures/Wayne-Rooney"]

    def parse(self, response):

        for row in response.selector.xpath('//table[@id="player-fixture"]//tr[td[@class="tournament"]]'):
        # Is this row contains goal symbols?
            list_of_goals = row.xpath('//span[@title="Goal"')
            if list_of_goals:
                print remove_tags(list_of_goals).encode('utf-8')

使用Spider class时，必须实现parse方法，这就是该方法在源代码中的样子：

def parse(self, response):
        raise NotImplementedError

您的缩进是错误的，因此解析不是该类的一部分，因此您尚未实现所需的方法。

raise NotImplementedError可以确保在从Spider基类继承时编写所需的parse方法。

现在，您只需要找到正确的xpath ;）