我应该如何优化/构建数据库以收集网上商店的价格发展信息？

Question

I have a small personal project, which is supposed to collect data from different webshops. What I basically do, is run a cron script every night. This script uses the Simple HTML DOM Parser for PHP to fetch prices for products in selected product groups.

As of now, my database consists of three tables:
- stores Name, URL etc for each webshop
- products URL, product names etc for each product
- prices Prices for every day each product

My question is the prices table. Every time the cron script runs, it saves new entries with price data for each product (300+), even if the products price is unchanged. I know I can prevent saving unnecessary data with a check to see wether the price is changed or not. But then again a product can be taken out of stock, leaving me no information of when it went out of stock (which it would if I saved the price each day). How would you guys do this more effective? The cron script would potentially take a long time to execute because of the DOM parsing, and I want to be sure everything is parsed and added to database as expected.

Answer 1

I guess you could keep track of each DOM you parsed, and store a checksum of it to see if it has changed when you load it again the next night. If the checksum is the same, you'll know you need no parsing and no updating because nothing will have changed.