How to insert Data from Python Scrapy Extension to MySql Database table?

Question

I am building an Extension for Python Scrapy to get Crawler related details like its start time, end time, crawler status(opened, closed or active).. Now I need to store current timestamp in my MySql Database table. Can anybody help me with this? I need a code to connect MySql Database from Scrapy code and do database related Queries like Insert, Select , etc.. Thanks..

Answer 1

You need to add an Item Pipeline to your project, add method process_item as described here: http://doc.scrapy.org/en/0.18/topics/item-pipeline.html#process_item

In that method you have to insert data to MySQL as described here: http://dev.mysql.com/doc/connector-python/en/index.html