scrapy python中的ssl握手失败错误

Question

Presently i am working on scrapy and following is my spider.py code

class Example(BaseSpider):
    name = "example"
    allowed_domains = {"http://www.example.com"}


    start_urls = [
        "https://www.example.com/rwjhamilton/index.cfm?&ijobcatid=100&ijobrowset=1&cjobattr1=All&template=dsp_job_list.cfm"
    ]


    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        href_tags = hxs.select('//font[@class="bannertext"]/u/a/@href').extract()
        print href_tags,">>>>>>>>>>>>>>>>"

Result :

2012-07-19 17:32:20+0530 [example] ERROR: Error downloading <GET https://www.example.com/rwjhamilton/index.cfm?&ijobcatid=100&ijobrowset=1&cjobattr1=All&template=dsp_job_list.cfm>: [('SSL routines', 'SSL23_READ', 'ssl handshake failure')]

This error seems very new to scrapy and really i don't know how to solve this, can anyone let me know why this error came and how to solve it. Actually i am trying to collect the href tags in the above url and it seems there totally 40 href tags

Also sometimes the response is downloading and sometimes it is displaying

2012-07-19 17:39:15+0530 [example] DEBUG: Retrying <GET https://www.example.com/rwjhamilton/index.cfm?&ijobcatid=100&ijobrowset=1&cjobattr1=All&template=dsp_job_list.cfm> (failed 1 times): Connection to the other side was lost in a non-clean fashion

Please let me know how to solve this. Thanks in advance.....

Answer 1

I think this issue has been fixed here

https://github.com/scrapy/scrapy/pull/186

It has been fixed after 0.15 release of scrapy

Answer 2

I met this error too:

2013-03-07 06:58:01+0800 [CrawlSpider] ERROR: Error downloading <GET a_website_url>: Connection to the other side was lost in a non-clean fashion.

My scrapy version is 0.16.3. The reason is that it is a restricted website for my country.