I have been using scrapy for some time now and I wish to know what is the best way to parse JSON. There are specially designed spiders for XML and CSV in scrapy viz. XMLFeedSpider
and CSVFeedSpider
. Is there any such spider for JSON too ?
There is no default JSON parser for Scrapy, but you can use the json
module to parse your response and convert it to items:
jsonresponse = json.loads(response.body_as_unicode())
However if you plan to write a general solution like CSVFeedSpider
feel free to contribute it to the Scrapy project but I guess json.loads
does the trick of consuming JSON responses.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.