使用PouchDB / CouchDB进行有效的数据库设计

Question

So I was reading a lot about how to actually store and fetch data in an efficient way. Basically my application is about time management/capturing for projects. I am very happy for any opinion on which strategy I should use or even suggestions for other strategies. The main concern is about the limited resources for local storage on the different Browsers.

This is the main data I have to store:

db_projects: This is a database where the projects itself are stored.

db_timestamps: Here go the timestamps per project whenever a project is running.

---

I came up with the following strategies:

1: Storing the status of the project in the timestamps

When a project is started, there is addad a timestamp to db_timestamps like so:

db_timestamps.put({
   _id: String(Date.now()),
   title: projectID,
   status: status //could be: 1=active/2=inactive/3=paused
})...

This follows the strategy to only add data to the db and not modify any entries. The problem I see here is that if I want to get all active projects for example, I would need to query the whole db_timestamp which can contain thousands of entries. Since I can not use the ID to search all active projects, this could result in a quite heavy DB query.

2: Storing the status of the project in db_projects

Each time a project changes it's status, there is a update to the project itself. So the "get all active projects"-query would be much resource friendly, since there are a lot less projects than timestamps. But this would also mean that each time a status change happens, the project entry would be revisioned and therefor would produce "a lot" of overhead. I'm also not sure if the compaction feature would do a good job, since not all revision data is deleted (the documents are, but the leaf revisions not). This means for a state change we have at least the _rev information which is still a string of 34 chars for changing only the status (1 char). Or can I delete the leaf revisions after conflict resolution?

3: Storing the status in a separate DB like db_status

This leads to the same problem as in #2 since status changes lead to revisions on this DB. Or if the states would be added in "only add data"-mode (like in #1), it would just quickly fill with entries.

Answer 1

The general problem is that you have a limited amount of space that you could put into indexedDB. On the other hand the principle of ChouchDB is that storage space is cheap (which it is indeed true when you store on the server side only). Here an interesting discussion about that.

So this is the solution that I use for now. I am using a mix between solution 1 and solution 2 from above with the following additions:

1. Storing only the timesamps in a synced Database (db_timestamps) with the "only add data" principle.
2. Storing the projects and their states in a separate local (not synced) database (db_projects). Therefor I still use pouchDB since it has a lot simpler API than indexedDB.
3. Storing the new/changed project status in each timestamp aswell (so you could rebuild db_projects out of db_timestams if needed)
4. Deleting db_projects every so often and repopulate it, so the revision data (overhead for this db in my case) is eliminated and the size is acceptable.

I use the following code to rebuild my DB:

//--------------------------------------------------------------------
function rebuild_db_project(){   
    db_project.allDocs({
      include_docs: true,
      //attachments: true
    }).then(function (result) {
        // do stuff
        console.log('I have read the DB and delete it now...');
        deleteDB('db_project', '_pouch_DB_Projekte');
        return result;
    }).then(function (result) {
        console.log('Creating the new DB...'+result);
        db_project = new PouchDB('DB_Projekte');

        var dbContentArray = [];
        for (var row in result.rows) {
            delete result.rows[row].doc._rev; //delete the revision of the doc. else it would raise an error on the bulkDocs() operation
            dbContentArray.push(result.rows[row].doc);
        }
        return db_project.bulkDocs(dbContentArray);      
    }).then(function(response){
        console.log('I have successfully populated the DB with: '+JSON.stringify(response));
    }).catch(function (err) {
      console.log(err);
    });  
}

//--------------------------------------------------------------------
function deleteDB(PouchDB_Name, IndexedDB_Name){
    console.log('DELETE');

    new PouchDB(PouchDB_Name).destroy().then(function () {
      // database destroyed
      console.log("pouchDB destroyed.");
    }).catch(function (err) {
      // error occurred
    });

    var DBDeleteRequest = window.indexedDB.deleteDatabase(IndexedDB_Name);
    DBDeleteRequest.onerror = function(event) {
      console.log("Error deleting database.");
    };
    DBDeleteRequest.onsuccess = function(event) {
      console.log("IndexedDB deleted successfully"); 
      console.log(request.result); // should be null
    };
}

So I not only use the pouchDB.destroy() command but also the indexedDB.deleteDatabase() command to get the storage freed nearly completely (there is still some 4kB that are not freed, but this is insignificant to me.)

The timings are not really proper but it works for me. I'm happy if somone has an idea to make the timing work properly (The problem for me is that indexedDB does not support promises).