简体   繁体   English

Scrapy Start_url错误

[英]Scrapy Start_url error

I am new to scrapy i am trying to scrape date from the pages of range(1,70000000) the code I is used is 我是新手,我试图从range(1,70000000)的页面中抓取日期,我使用的代码是

import scrapy, json, re
from blackberry.items import BlackberryItem
class BlackSpider(scrapy.Spider):
    name = 'datas'
    start_urls = [
              'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' %page for page in xrange(1, 10000000),
              'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%y for y in xrange(10000000, 20000000),
              'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%a for a in xrange(20000000, 30000000),
              'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%b for b in xrange(40000000, 50000000),
              'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%c for c in xrange(50000000, 60000000),
              'https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482'%d for d in xrange(60000000, 70000000)
              ]

But i got this error : 但是我得到了这个错误:

"y is not defined"

One of the possible solutions is as follow. 可能的解决方案之一如下。

import scrapy
import json
import re
from blackberry.items import BlackberryItem
class BlackSpider(scrapy.Spider):
    name = 'datas'
    start_urls = ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(10000000, 20000000)]
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(20000000, 30000000)]
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(30000000, 40000000)]
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(40000000, 50000000)]
    start_urls += ['https://appworld.blackberry.com/cas/content/%s?countryid=100&lang=en&callback=_content_2360&_=1499177414482' % page for page in xrange(50000000, 60000000)]

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM