Mongo-go-driver GridFS metadata

Question

I have written a chat app for the company I work in and I worked with the mgo driver for some time. Now we refactor the mgo to the official mongo driver. I have implemented GridFS to work with the chat files as they are not big and it simplifies the job. The previous mgo driver when saving files had a list of data which one of the fields was contentType (Great right?)

So after refactoring most of the services that were included in this task I have noticed that the new official mongo driver does not do this??

So I have decided to try and add this field manually but then I got to the point that I don't understand how I can?

Tried with options.GridFSUpload().SetMetadata(metadata) but I don't understand the logic of it and the internet is really minimal with the result about the new mongo driver working in GO.

Anyone can give me a hint how to add custom fields to file docs? Like contentType!!

Really appreciate it.

This is an example of what I tried to do

// GridFSInsert -
func GridFSInsert(fileName string, data []byte, metadata ...bsonx.Elem) (primitive.ObjectID, error) {
    checkMongoConnection(false)
    var fileID primitive.ObjectID
    bucket, bucketErr := gridFs.NewBucket(
        Mongo.Client.Database(Mongo.DBName),
        options.GridFSBucket().SetName(gridFSColName),
    )
    if bucketErr != nil {
        return fileID, bucketErr
    }
    uploadStream, uploadStreamErr := bucket.OpenUploadStream(
        fileName,
        options.GridFSUpload().SetMetadata(metadata),
    )
    if uploadStreamErr != nil {
        return fileID, uploadStreamErr
    }
    defer uploadStream.Close()

    fileSize, writeErr := uploadStream.Write(data)
    if writeErr != nil {
        return fileID, writeErr
    }
    fileID = uploadStream.FileID
    log.Printf("Write file to DB was succesful, File size: %d", fileSize)

    return fileID, nil
}

Sorry if I missed something as I'm not that experienced with GO as I would like to.

Thanks for any help

Answer 1

Here is an example of SetMetadata() .

opts := options.GridFSUpload()
opts.SetMetadata(bsonx.Doc{{Key: "content-type", Value: bsonx.String("application/json")}})
if ustream, err = bucket.OpenUploadStream("test.txt", opts); err != nil {
    t.Fatal(err)
}

Here is the full example

Answer 2

There is no logic you are trying to understand. The reason why you can't find much about contentType in new official mongo driver is because contentType has been deprecated in gridfs spec long before the driver was written.

I must admit the gridfs documentation doesn't mention it. In fact official mongofiles cli still uses legacy format.

The spec puts it straight:

Note: some older versions of GridFS implementations allowed applications to add arbitrary fields to the files collection document at the root level. New implementations of GridFS will not allow this , but must be prepared to handle existing files collection documents that might have additional fields.

And if you like more detailed official reasoning :

Why is contentType deprecated?

Most fields in the files collection document are directly used by the driver, with the exception of: metadata, contentType and aliases. All information that is purely for use of the application should be embedded in the 'metadata' document. Users of GridFS who would like to store a contentType for use in their applications are encouraged to add a 'contentType' field to the 'metadata' document instead of using the deprecated top-level 'contentType' field.

Which kinda makes sense. The driver follows wording of the spec literally - there is no way to create contentType property anywhere but in metadata , yet Bucket.Find will still return contentType of files created by "older versions".

The one-off transition from legacy gridfs to new format can be as simple as:

db.getCollection("fs.files").aggregate([
    {$addFields: { 
        "length" : {$toLong: "$length"},
        "metadata.contentType": { $ifNull: [ "$contentType", "$metadata.contentType" ] } 
    }},
    { $out : "fs.files" }
])

Assuming your bucket is default "fs" and you are not going to upload files in legacy format. If you have a luxury of free space, it won't be a terrible idea to out to new temporary collection, validate it, then rename.

If you have to support legacy format for any reason, you still can access gridfs collections directly:

// in your code snippet after
fileID = uploadStream.FileID

// update the document that represent uploaded file
files := db.Collection("fs.files")
updateResult, err := files.UpdateOne(
    context.Background(),
    bson.D{{"_id", fileID}},
    bson.D{{"$set", bson.D{{"contentType", contentType}}}},
)

Where "fs" is your bucket name, and contentType is the string value you want to set as a contentType.

Bear in mind, that "some older versions" use int32 for file length tho. The new driver expects it to be int64. It should be okay for Find-like operations that work with *.fiiles collections alone but may cause problems downloading such files with new official driver.