MongoDb GridFs update Java

Question

MongoDB gives a great driver for GridFS in java, in the form of a GridFSBucket , however, as of posting the api (v3.2) doesn't provide an update command, so we have to do both a delete and an insert. I'm worried about concurrency issues, and despite my best efforts at researching it, I still don't quite understand how Mongo handles locking. for instance, mongo's api has this example in python:

 def update_photo_content(input_file, section, slug):                   # 1
 fs = GridFS(db, 'cms.assets')                                          # 2
                                                                        # 3
 # Delete the old version if it's unlocked or was locked more than 5    # 4
 #    minutes ago                                                       # 5
 file_obj = db.cms.assets.find_one(                                     # 6
     { 'metadata.section': section,                                     # 7
       'metadata.slug': slug,                                           # 8
       'metadata.locked': None })                                       # 9
 if file_obj is None:                                                   #10
     threshold = datetime.utcnow() - timedelta(seconds=300)             #11
     file_obj = db.cms.assets.find_one(                                 #12
         { 'metadata.section': section,                                 #13
           'metadata.slug': slug,                                       #14
           'metadata.locked': { '$lt': threshold } })                   #15
 if file_obj is None: raise FileDoesNotExist()                          #16
 fs.delete(file_obj['_id'])                                             #17
                                                                        #18
 # update content, keep metadata unchanged                              #19
 file_obj['locked'] = datetime.utcnow()                                 #20
 with fs.new_file(**file_obj):                                          #21
     while True:                                                        #22
         chunk = input_file.read(upload_file.chunk_size)                #23
         if not chunk: break                                            #24
         upload_file.write(chunk)                                       #25
 # unlock the file                                                      #26
 db.assets.files.update(                                                #27
     {'_id': upload_file._id},                                          #28
     {'$set': { 'locked': None } } )                                    #29

but I don't know what methods in the python driver actually access the database, or understand the lock

for instance, on line 17, what is the mongo shell/ java equivelent, and is this accessing the db?

and for the lock, it doesn't seem to be any special field used by the mongo server, so do I need to account for it with a busy wait or something in my driver, or am I missing how mongo handles it?

I would prefer to keep the same id for the files after they are updated, so this answer probably wouldn't work for me

Answer 1

I believe you got the example code from the Metadata and Asset Management use case in the MongoDB documentation.

Please note that the use case's locking mechanism is not handled by MongoDB server at all. The locking was done in the application level. From the page in question:

Because uploading the photo spans multiple documents and is a non-atomic operation, you must “lock” the file during upload by writing datetime.utcnow() in the record.

GridFS is not a server-side feature, but rather a convention on how files larger than the BSON document size limit of 16 MB can be stored in MongoDB. There is no server-enforced features of GridFS, and you are free to interpret (or misinterpret) what is stored in the GridFS collections as you wish (see GridFS collections ).

The method presented in the answer https://stackoverflow.com/a/30074812/5619724 is probably the most straightforward method to update a GridFS file. Of course, you are free to perform a more complex locking mechanism in your application should you wish.

Regarding your wish to retain the same ID for updated files, please note that you don't need to depend on MongoDB-supplied ObjectId . The GridFS convention allows you to put any metadata in the metadata field . You can put your unique ID inside this field.