在JavaScript中存储web-scraping的输出

Question

I've got a small web-scraping utility in Perl that I am thinking of reimplementing in JavaScript to share with a couple of other people. Ideally I would like to send them a page with the complete JavaScript program embedded to run standalone on their own machines. I need to be able to store results from run to run. I had expected to be able to find an interface to an RDBMS such as SQLite. I am surprised that there doesn't seem to be anything like this for Firefox. What am I looking for as far as a relational database storage solution from JavaScript?

Answer 1

You should consider sticking a RESTful API in front of a database server. Chose your stack. If you are not married to a RDBMS consider CouchDB, which has a wonderful RESTful API out of the box.

Answer 2

A slightly more "inside-the-scope-of-your-question" answer is to use a DB API like Google's Gears API: Link to Gears

This provides a local browser storage solution.

Answer 3

This is a slightly "outside-the-scope-of-your-question" answer, but it may be better to look at a free scraping utility that already handles XSS and RDBMS storage. Check out WSO2's mashup server.

Link to Mashup Server Project Page

It uses X-family (XPath, XSLT, XML) acquisition and formatting and Javascript for any business logic. You can save the results to a DB or expose them as a web-service. Pretty flexible Java-based solution. Just need a JVM to run.

Best of luck!