scrapy python call spider from spider

Question

I have one spiner which scrape the page and gets all the urls.

i have another spiner which get a url and scrap on it.

i want to call the second spiner for each link i get from the first spiner.

the code for getting all links from the first spiner

for site in sites:
            Link = site.xpath('a/@href').extract()

but i don't know how to call the spiner for each Link

help please

Answer 1

I guess you better unite the two spiders and do something like:

def get_links(self, response):
    for site in sites:
        link = site.xpath('a/@href').extract()[0]
        yield Request(url=link, callback=self.scrape_them)

def scrape_them(self, response):
    # by now scrapy called the link and you get the response
    ...