Im trying to make my crawler crawl the next page so I wrote this function:
def requestURL(self, hrefPath):
print(hrefPath)
url = "http://www.presseportal.de/" + hrefPath
yield scrapy.Request(url, callback=self.parse)
Its not even printing the variable hrefPath
so I suspect the function isnt called, although im pretty sure I called it right like this: self.requestURL(hrefPath)
Im very new to scrapy so Im very sorry if this question seems stupid
I figured it out: Instead of writing
yield scrapy.Request(url, callback=self.parse)
write return
yield scrapy.Request(url, callback=self.parse)
in the function and when you call the function in the parse function write: yield requestURL(hrefPath)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.