在Java中在app引擎上制作blobstore实体副本的最佳方法是什么？

Question

Here is our simple usecase: user2 wants to copy user1's document into his or her own repository within our application. Should be simple, right? All we need to do is create a second identical blob in the blobstore with the key returned that we can associate with user2. We must be missing something here. It appears that app engine blob store's primary function is to handle blobs uploaded from and downloaded to a browser, and a simple copy operation initiated server-side is not so simple.

The obvious solution seemed to be using the the experimental file api in java, but no love. It works, until you get up in file size beyond a MB or so, then it fails, somewhat unpredictably. Reading it all into the server layer also seems silly, when we just need to make a copy in the storage layer. In addition, the odds of us getting an experimental feature through into our production environment is slim, although non-zero.

Some information about our environment: the app is written in Java and we're using the blobstore, not cloud storage and are committed to it for now. We're a small departmental team and would like to make the case that app engine is a great platform to use, but this one has us stumped. S3 makes this blindingly simple, are we missing something really stupid here?

Answer 1

We ended up scrapping the idea of making a programatic copy of the blob with the file api and going with a reference as Kalle suggested in his comment, and created a new xref entity that stores information about the copy and the original. When an image or file is deleted, we check the xfef entity for references and delete the ones that point to that image/file (ie created if the deleted image/file was copied from another one). If we don't find any xrefs at all, we delete the blob itself. We didn't like the privacy/compliance implications of leaving orphaned blobs laying around, and even though storage is cheap every $$$ helps. We also liked the idea of keeping a clean house so to speak.

Answer 2

Solution 1 : I will launch a Google Compute Engine instance and use the command gsutil to do the copy.

And then shutdown the instance when it's done. This is the fastest way to do the copy to my knowledge

gsutil documentation

Solution 2 : But I will personally choose to use a counter as said in the comments, because the point that you said is scary will be the same problem with the copy. So just use counters with strong unit testing on those for example that will be less scary.

An idea to make it less scary is when you reach 0 for your counter you don't delete the blob right away but do a job to do this later on. By using Scheduled task in Google App Engine. And delete the file and your actual record of it a month later for example.

Answer 3

As already mentioned in the comment, keep one blob and pass the key around. But you really never need to delete. It is good practice to keep the blob for archive purposes. So how would delete actually work? In your datastore model, have a boolean delete field. You don't remove the blob key from an entity upon deletion. But rather, you mark the boolean field as true . This way, your product has a record of every user who has ever owned a file. But the user does not need to ever know.