Scrapy csv在多行输出

Question

Here's my spider:

from scrapy.spider import BaseSpider
from scrapy.selector import Selector
from ..items import TutorialItem

class Tutorial1(BaseSpider):
name = "Tut"
allowed_domains = ['nytimes.com']
start_urls = ["http://nytimes.com",] 

def parse(self, response):
    sel = Selector(response)
    sites = sel.xpath('//div[@class="span-ab-layout layout"]')
    items = []

    for site in sites:
        item = TutorialItem()
        item['title'] = map(unicode.strip, site.select('//h2[@class="story-heading"]/a/text()').extract())
        item['time'] = map(unicode.strip, site.select('//time[@class="timestamp"]/text()').extract())
        yield item

Here is my output:

author time By PETER BAKER,By JONATHAN M. KATZ and RICHARD PÉREZ-PEÃ'A,By NEIL MacFARQUHAR,By RON NIXON,By RICHARD GOLDSTEIN,By LOUISE STORY and ALEJANDRA XANIC von BERTRAB,By DAVID CARR,By AO SCOTT,By JERÉ LONGMAN,By THE EDITORIAL BOARD,By JON BECKMANN,By CJ HUGHES,By JOANNE KAUFMAN 10:26 AM ET,1:08 PM ET,11:57 AM ET,8:33 AM ET,10:01 AM ET,12:35 PM ET,1:47 PM ET,10:36 AM ET,10:26 AM ET,9:49 AM ET,12:05 PM ET,9:21 AM ET,12:22 PM ET,11:52 AM ET,8:59 AM ET

By PETER BAKER,By JONATHAN M. KATZ and RICHARD PÉREZ-PEÃ'A,By NEIL MacFARQUHAR,By RON NIXON,By RICHARD GOLDSTEIN,By LOUISE STORY and ALEJANDRA XANIC von BERTRAB,By DAVID CARR,By AO SCOTT,By JERÉ LONGMAN,By THE EDITORIAL BOARD,By JON BECKMANN,By CJ HUGHES,By JOANNE KAUFMAN 10:26 AM ET,1:08 PM ET,11:57 AM ET,8:33 AM ET,10:01 AM ET,12:35 PM ET,1:47 PM ET,10:36 AM ET,10:26 AM ET,9:49 AM ET,12:05 PM ET,9:21 AM ET,12:22 PM ET,11:52 AM ET,8:59 AM ET

I made the indention so it was clear where it was duplicating.

My problem occurs when I go to print out my work in CSV is always comes out in 1 giant row. It also makes a duplicate column for some reason. Can anyone help me with this dilemma?

Answer 1

I was able to find it by experimenting with:

hxs = HtmlXPathSelector(response)

Apparently, there is a huge difference between Selector and HtmlPatchSelector