How to create client side full text search engine with pure JavaScript and HTML5?

Question

I need to create client side search engine for an offline html application in offline storage like CDROM. I already create it with Win32 and the catalog file is near 2GB (MDB file).

My offline web application use CEF (Chromium Embedded Framework) and PhoneGAP so I can disable the browser web security. I thought the IndexedDB may help me but , I couldn't find anyway to copy the storage file to other media.

1. My catalog file will be near 2GB, so I can't read all record to memory.
2. I need to make the output media cross-platfrom with PhoneGAP, so I can't use activeX objects for final search. Also I can disable web security restriction by that tools.
3. I can create the catalog file and media with any windows application. the key note that the output file that query the catalog should be be pure JavaScript.

Is there any HTML5 big storage system available (I can disable web security by PhoneGAP) to create offline HTML application search catalog that can be copied to other removable media?

Answer 1

Reyes is doing a fulltext search engine for this exact purpose : https://github.com/reyesr/fullproof

During last september Paris JS, he did a presentation of the concepts behind : http://kornr.net/prez/paris.js22/#/

Hope it answers your need.

Answer 2

I think meanwhile lunr.js must be considered. It has 3-times the stars on Github.

Answer 3

Saving whole database in .json file definitely wouldn't be efficient, but splitting your database to chunks and saving them as .json files loaded on demand may work just fine - it would require testing though.

And - MongoDB stores database in BSON . You can also think about storing your database in this file format and loading it from your application (I would suggest splitting the whole database to smaller parts anyway).

// edit - If you're going to give away the database as file, then I don't think that it's possible to set any kind of permissions. You could try to encrypt/obfuscate the file, but I have no idea how would that affect the performance or database' contents.

And yes, it is possible to have a standalone html+js application that would read the BSON database without any kind of server. Check this link if you want to know more. I've heard of this before, but I have completely no experience when it comes to handling BSON files in pure JS, so I can't help you out with this. All I could do was to let you know that such a thing exists :)