如何在主线程和 Web Worker 之间共享 pouchDB

Question

In a web app, using pouchDB , I have a slow running function that finishes by updating a document in the DB. I want to move it off the main UI thread and into a web worker. However, we have lots of other code using pouchDB still in the main thread (eg the change event listener, but also code that deals with other documents). (For reference the database size is on the order of 100MB; Vue2 is used so, in general, the UI can update when the data changes.)

This is where I seem to come unstuck immediately:

• Shared memory is basically out, as all the browsers disable it by default
• Even if it wasn't, pouchDB is a class, and cannot be transferred(?).
• Isolating all the db code, including the changes handler, into one web worker is a huge refactor; and then we still have the issue of having to pass huge chunks of data in and out of that web worker.
• Move all the code that uses the data into the web worker too, and just have the UI thread pass messages back and forth, is an even bigger refactor, and I've not thought through how it might interfere with Vue.

That seems to leave us with a choice of two extremes. Either rewrite the whole app from the ground up, possibly dropping Vue, or just do the slow, complex calculation in a web worker, then have it pass back the result, and continue to do the db.put() in the main UI thread.

Is it really an all or nothing situation? Are there any PouchDB "tricks" that allow working with web workers, and if so will we need to implement locking?

Answer 1

You're missing an option, that I would chose in your situation. Write a simple adapter that allows your worker code to query the DB in main thread via messages. Get your data, process it in the worker and send it back.

You only need to "wrap" the methods that you need in the worker. I recommend writing a class or set of functions that are async in your worker, to make the code readable.

You don't need to worry about the amount of data passed. The serialization and de-serialization is quite fast and the transfer is basically memcpy , so that does not take any reasonable time.

Answer 2

I found this adapter plugin, which I guess counts as the "PouchDB trick" I was after: https://github.com/pouchdb-community/worker-pouch

It was trivial to add (see below), and has been used in production for 6-7 weeks, and appears to have fixed the problems we saw. (I say appears , as it is quite hard to see it having any effect, and we didn't have a good way to reproduce the slowdown problems users were seeing.)

const PouchDB = require('pouchdb-browser').default
const pouchdbWorker = require('worker-pouch')
PouchDB.adapter('worker', pouchdbWorker)

The real code is like this, but usePouchDBWorker has always been kept as true :

const PouchDB = require('pouchdb-browser').default
// const pouchdbDebug = require('pouchdb-debug')
if (usePouchDBWorker) {
  const pouchdbWorker = require('worker-pouch')
  PouchDB.adapter('worker', pouchdbWorker)
}

This code is used in both web app and Electron builds. The web app is never used with older web browsers, so read the github site if that might be a concern for your own use case.