Python / Scrapy：回调函数永不调用

Question

我正在使用scrapy来爬网Google Play项目配置文件，但未执行回调函数。 我在代码中找不到问题（没有错误）。 你能告诉我任何解决方案吗？

# -*- coding: utf-8 -*-
import scrapy

import time

from scrapy.spider import BaseSpider
from scrapy.selector import HtmlXPathSelector
from playcrawl.items import PlaycrawlItem
from scrapy.http import Request

from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.linkextractor import LinkExtractor

class GoogleplaySpider(CrawlSpider):
    name = 'googleplay'
    allowed_domains = ['play.google.com']
    start_urls = ['https://play.google.com/store/apps/category/GAME']

    rules = (
        Rule(LinkExtractor(allow=('/store/apps'))),
        Rule(LinkExtractor(allow=('/store/apps/details\?')),callback="parse_item")
        )

    def parse_item(self, response):
        hxs = HtmlXPathSelector(response)

        item = PlaycrawlItem()
        item["pub"] = hxs.select('//a[@class = "document-subtitle primary"]/span[1]').select("text()").extract()
        item["email"] = hxs.select('//a[contains(@class, "dev-link") and starts-with(@href, "mailto")]').select("@href").extract()[0][7:]

        f = open("D:\\_scrapy\\playcrawl\\data_emails.txt", "a")
        f.write(item["email"] + "\n")
        f.close()

        print("\n\n\n\n" + item["email"] + "\n\n\n\n")
        time.sleep(0)

        return item #yield item

Answer 1

我已经测试了您的代码，原因很简单。

蜘蛛只是没有符合第二Rule 。

尝试这个：

rules = (
        Rule(LinkExtractor(allow=('/store/apps')),callback="parse_item"),
        Rule(LinkExtractor(allow=('/store/apps/details\?')),callback="parse_item")
        )

然后它起作用了，所以没有关于代码的错误，而是关于逻辑的错误。