Blobstore备份策略Google App Engine Python

Question

I'm looking for a way to backup the blobstore on Google App Engine to recover from accidental deletion. I'd like to be able to make a backup of the datastore and blobstore at the same time, so that I could recover the entire system.

One thing I tried was using Google Cloud Storage for my blobs, and then making backup copies of buckets. However, if I delete a file in the blobstore viewer (which also deletes it in google cloud storage), and then restore the file from a bucket backup using gsutil, the blobinfo is still lost.

I believe I would have to store all the file paths (google cloud storage paths) in my datastore, and then during a restore, recreate blob keys for all those files using create_gs_key(). Is the info stored in the blobviewer (file name, content type, size, etc) stored in the datastore?

Is there anyway to backup / restore blobstore data without regenerating keys?

Answer 1

Take a look at Google Cloud Storage's object versioning feature. Enable versioning on a bucket and when you delete an object it gets archived. Later you can choose to restore or permanently delete the archived version:

Create an object:

>>> PUT /bucket/obj

<<< HTTP/1.1 200 OK
<<< x-goog-generation: 12345

Then archive it:

>>> DELETE /bucket/obj

<<< HTTP/1.1 200 OK

Now it is inaccessible:

>>> GET /bucket/obj

<<< HTTP/1.1 404 NOT FOUND

However you can still request the archived version explicitly:

>>> GET /bucket/obj?generation=12345

<<< HTTP/1.1 200 OK

Until you delete the archived version:

>>> DELETE /bucket/obj?generation=12345

<<< HTTP/1.1 200 OK

Of course... this doesn't protect you from accidentally deleting the archived version.

Answer 2

You could try using the datastoreadmin backup and restore .

It will back up the datastore entities with the blob keys intact. According to the following, you will still need to back up the blobs separately as you are already doing.

Note: Although we do back up datastore entity properties of type BlobKey and Blob, we do not back up the related blobs themselves that are stored in Blobstore or Google Cloud Storage. Nor do we back up the datastore BlobInfo entities associated with these blobs.

Note you can also schedule these backups on a regular basis.