GridFS: Can't find a codec for custom class

Question

I am working with a legacy application which has a data model in which a given MongoDB document could, occasionally, exceed 16MB in size. This of course would trigger an exception in Mongo, and so we have decided to shift a couple of collections over to GridFS, which does not have this size limitation. However, while we were able to successfully shift one of the collections over to GridFS, we get the following error when trying to insert POJOs which were being stored in the other collection:

org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.tjb.entities.SomePojo

I have researched this here and elsewhere, and have found a few possible fixes. However, these fixes seem to only be for normal Mongo, not GridFS.

Is there some way to configure a custom codec for GridFS? I am assuming that there probably is a way to get GridFS to store the POJO, since regular Mongo already has the ability to do so.

For reference, the POJO contains only String , Java primitive, List , and object fields. A few of the fields have been marked as @Transient , in case that would matter. I am using Spring data as the interface into Mongo.

Answer 1

After struggling with this problem for 1+ days, I tried to instead use Gson to write a simple JSON String to GridFS, which should have eliminated and issue with the serialization of the underlying Java object. To my surprise, the same codec error persisted. Upon closer inspection, I tracked down the error to being caused by these troubling lines:

DBObject metaData = new BasicDBObject();
metaData.put(Fields.DOCUMENTS_LIST, listOfSomePojo);

Here listOfSomePojo was an ArrayList of some POJO, each of whose objects had a field I wanted to use for searching. This is what was causing the codec error, because GridFS couldn't figure out what I was trying to do. My intention was to include the collection, each object of which had an _id field for searching. Here is the workaround I used:

List<String> docIds = deal.getDocumentsList().stream()
    .map(x -> x.getId())
    .collect(Collectors.toList());
metaData.put(Fields.DOCUMENTS_LIST, docIds);

That is, I just added a plain List<String> to one of the metadata fields instead, and then searched using it. This fixed the codec error, and now everything is working.

As a side note, if you are having a similar problem, please also make sure that any Object you wish to store in GridFS is serializable, as well as all children Objects are also serializable. If not, it will also not work.

Answer 2

I'm not sure if it's exactly related to the problem you describe but I was having similar issues - writing the GridFSFile got to a point where it was working but any reading of the file (say, in a Cursor<GridFSFile> ) led to a codec failure. (GridFSFile has not public constructor, configure a Codec etc etc) I eventually tracked it down in the debugger and saw that there was a decoder coming up but it was an AutomaticPojo codec, not the GridFSFileCodec.

This pointed me finally to where I was adding in my other codecs to the registry. That code looked like :

CodecProvider someCodec = PojoCodecProvider.builder().....
List<CodecProvider> lotsOfThese = new ArrayList<>(); 
CodecRegistry cr = new ProvidersCodecRegistry(lotsOfThese);
CodecRegistry whole = CodecRegistries.fromRegistries(cr, db.getCodecRegistry());
db = db.withCodecRegistry(whole);

But what I didn't understand is that codecs have matching rules and order therefore matters - the union of registries (cr, defaults) will not behave the same as (defaults, cr) - and even though I thought I had a package rule in my PojoCodecProvider it appears to have matched the GridFSFile before it got to the default in the codec registry.

Flipping the order in the union solved several issues for me.

CodecRegistry whole = CodecRegistries.fromRegistries(db.getCodecRegistry(), cr);