Python Scrapy - start_url 中有多个变量

Question

I want to make make my start_url a bit more dynamic than it currently is but my adjusted code doesn't seem to work.

In order to make it more dynamic, I've added 2 more variables (month and day), and changed over to using the start_requests method instead of start_urls , however, the scraper now returns zero items:

import scrapy

class SuhbaSpider(scrapy.Spider):
    name = "suhbaDate"
# old working line of code 
#    start_urls = ["http://saltanat.org/videos.php?date={yyyy}-06-15".format(yyyy=yyyy) for yyyy in range(2013,2020)]

# new block of code (replaced start_urls with start_requests), not working
    def start_requests(self):
        for yyyy in range(2013,2020):
            for mm in range(12,12):
                for dd in range(14,15):
                    url = "http://saltanat.org/videos.php?date={yyyy}-{mm}-{dd}".format(yyyy=yyyy,mm=mm,dd=dd) 
                    yield Request(url, meta={'start_url':url}, callback=self.parse)
                    print(yyyy,mm,dd,url)

    def parse(self, response):
        for video in response.xpath("//tr[@class='video-doclet-row']"):
            item = dict()
            item["video"] = video.xpath(".//span[@class='download make-cursor']/a/@href").extract_first()

            videoid = video.xpath(".//span[@class='media-info make-cursor']/@onclick").extract_first()
            url = "http://saltanat.org/ajax_transcription.php?vid=" + videoid[21:-2]
            request = scrapy.Request(url, callback=self.parse_transcript)
            request.meta['item'] = item
            yield request

    def parse_transcript(self, response):
        item = response.meta['item']
        item["transcript"] = response.xpath("//a[contains(@href,'english')]/@href").extract_first()
        yield item

Any assistance will be appreciated

Answer 1

So there were a couple of problems with the above code code

1. Looking at the Traceback output it was found that the actual problem was this: NameError: global name 'Request' is not defined which seems to be a bug

2. The url in question needed a leading zero for the mm and dd variables

The solution

1. Included this line at the top of the script from scrapy.http.request import Request

2. Rewrote the start_requests loop with itertools and zfill:

def start_requests(self):
    for yyyy,mm,dd in itertools.product(range(2013,2020),range(6,7),range(14,22)):
        mm = str(mm).zfill(2)
        dd = str(dd).zfill(2)
        url = "http://saltanat.org/videos.php?date={0}-{1}-{2}".format(yyyy,mm,dd) 
        yield Request(url, meta={'start_url':url}, callback=self.parse)

* ignore the actual dates, those are for testing