如何从 Mongo DB 读取 com.fasterxml.jackson.databind.node.TextNode 并转换为 Map<String, Object> ?

Question

We are using SpringDataMongoDB in a Spring-boot app to manage our data.

Our previous model was this:

public class Response implements Serializable {
    //...
    private JsonNode errorBody; //<-- Dynamic
    //...
}

JsonNode FQDN is com.fasterxml.jackson.databind.JsonNode

Which saved documents like so in the DB:

"response": {
  ...
        "errorBody": {
          "_children": {
            "code": {
              "_value": "Error-code-value",
              "_class": "com.fasterxml.jackson.databind.node.TextNode"
            },
            "message": {
              "_value": "Error message value",
              "_class": "com.fasterxml.jackson.databind.node.TextNode"
            },
            "description": {
              "_value": "Error description value",
              "_class": "com.fasterxml.jackson.databind.node.TextNode"
            }
          },
          "_nodeFactory": {
            "_cfgBigDecimalExact": false
          },
          "_class": "com.fasterxml.jackson.databind.node.ObjectNode"
     },
  ...
 }

We've saved hundreds of documents like this on the production database without ever the need to read them programmatically as they are just kind of logs.

As we noticed that this output could be difficult to read in the future, we've decided to change the model to this:

public class Response implements Serializable {
    //...
    private Map<String,Object> errorBody;
    //...
}

The data are now saved like so:

"response": {
  ...
        "errorBody": {
          "code": "Error code value",
          "message": "Error message value",
          "description": "Error description value",
          ...
        },
  ...
 }

Which, as you may have noticed is pretty much more simple.

When reading the data, ex: repository.findAll()

The new format is read without any issue.

But we face these issues with the old format:

org.springframework.data.mapping.MappingException: No property v found on entity class com.fasterxml.jackson.databind.node.TextNode to bind constructor parameter to!

Or

org.springframework.data.mapping.model.MappingInstantiationException: Failed to instantiate com.fasterxml.jackson.databind.node.ObjectNode using constructor NO_CONSTRUCTOR with arguments

Of course the TextNode class has a constructor with v as param but the property name is _value and ObjectNode has no default constructor: We simply can't change that.

We've created custom converters that we've added to our configurations.

public class ObjectNodeWriteConverter implements Converter<ObjectNode, DBObject> {    
    @Override
    public DBObject convert(ObjectNode source) {
        return BasicDBObject.parse(source.toString());
    }
}

public class ObjectNodeReadConverter implements Converter<DBObject, ObjectNode> {
    @Override
    public ObjectNode convert(DBObject source) {
        try {
            return new ObjectMapper().readValue(source.toString(), ObjectNode.class);
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
    }
}

We did the same for TextNode

But we still got the errors.

The converters are read as we have a ZonedDateTimeConverter that is doing his job.

We can not just wipe out or ignore the old data as we need to read them too in order to study them.

How can we set up a custom reader that will not fail reading the old format ?

Answer 1

Since old format is predefined and you know a structure of it you can implement custom deserialiser to handle old and new format at the same time. If errorBody JSON Object contains any of these keys: _children , _nodeFactory or _class you know it is an old format and you need to iterate over keys in _children JSON Object and get _value key to find a real value. Rest of keys and values you can ignore. Simple implementation could look like below:

import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.TreeNode;
import com.fasterxml.jackson.databind.DeserializationContext;
import com.fasterxml.jackson.databind.JsonDeserializer;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.json.JsonMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
import lombok.Data;
import lombok.ToString;

import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.Collections;
import java.util.Iterator;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

public class JsonMongo2FormatsApp {
    public static void main(String[] args) throws IOException {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();

        JsonMapper mapper = JsonMapper.builder().build();
        Response response = mapper.readValue(jsonFile, Response.class);
        System.out.println(response.getErrorBody());
    }
}

@Data
@ToString
class Response {

    @JsonDeserialize(using = ErrorMapJsonDeserializer.class)
    private Map<String, String> errorBody;
}

class ErrorMapJsonDeserializer extends JsonDeserializer<Map<String, String>> {

    @Override
    public Map<String, String> deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
        TreeNode root = p.readValueAsTree();
        if (!root.isObject()) {
            // ignore everything except JSON Object
            return Collections.emptyMap();
        }
        ObjectNode objectNode = (ObjectNode) root;
        if (isOldFormat(objectNode)) {
            return deserialize(objectNode);
        }

        return toMap(objectNode);
    }

    protected boolean isOldFormat(ObjectNode objectNode) {
        final List<String> oldFormatKeys = Arrays.asList("_children", "_nodeFactory", "_class");
        final Iterator<String> iterator = objectNode.fieldNames();

        while (iterator.hasNext()) {
            String field = iterator.next();
            return oldFormatKeys.contains(field);
        }

        return false;
    }

    protected Map<String, String> deserialize(ObjectNode root) {
        JsonNode children = root.get("_children");
        Map<String, String> result = new LinkedHashMap<>();
        children.fields().forEachRemaining(entry -> {
            result.put(entry.getKey(), entry.getValue().get("_value").toString());
        });

        return result;
    }

    private Map<String, String> toMap(ObjectNode objectNode) {
        Map<String, String> result = new LinkedHashMap<>();
        objectNode.fields().forEachRemaining(entry -> {
            result.put(entry.getKey(), entry.getValue().toString());
        });

        return result;
    }
}

Above deserialiser should handle both formats.

Answer 2

As I understood your issue, with the first model, you didn't really have a problem to save or to read in database but, once you wanted to fetch these datas, you noticed that the output is difficult to read. So your problem is to fetch a well readable output then you don't need to change the first model but to extends these classes and overide the toString method to change its behavior while fetching.

There are at least three classes to extends:

• TextNode : you can't overide the toString method do that the custom class just print the value

• ObjectNode : I can see that there are at least four field inside this class that you want to fecth the value: code , message , description . They are type of TextNode so you can replace them by thier extended classes. Then overide the toString method so that It print fieldName: field.toString() for each field

• JsonNode : You can then extend this class and use the custom classes created above, overide the toString method so that It print as you want and use It instead of the common JsonNode

To work like that will make you avoid the way you save or you read the datas but just to fecth on the view.

You can consider it as a little part of the SOLID principle especially the OCP (Open an close principle: avoid to change the class behavoir but extends it to create a custom behavior) and the LSP (Liskov Substitution Principle: Subtypes must be behaviorlly substituable for thier base types).

Answer 3

Michal Ziober answer was not completely solving the problem as we need to tell SpringData MongoDb that we want him to use the custom deserializer (Annotating the model does not work with Spring data mongodb):

1. Define the custom deserializer

public class ErrorMapJsonDeserializer extends JsonDeserializer<Map<String, Object>> {

    @Override
    public Map<String, Object> deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
        TreeNode root = p.readValueAsTree();
        if (!root.isObject()) {
            // ignore everything except JSON Object
            return Collections.emptyMap();
        }
        ObjectNode objectNode = (ObjectNode) root;
        if (isOldFormat(objectNode)) {
            return deserialize(objectNode);
        }
        return toMap(objectNode);
    }

    protected boolean isOldFormat(ObjectNode objectNode) {
        final List<String> oldFormatKeys = Arrays.asList("_children", "_nodeFactory", "_class");
        final Iterator<String> iterator = objectNode.fieldNames();
        while (iterator.hasNext()) {
            String field = iterator.next();
            return oldFormatKeys.contains(field);
        }
        return false;
    }

    protected Map<String, Object> deserialize(ObjectNode root) {
        JsonNode children = root.get("_children");
        if (children.isArray()) {
            children = children.get(0);
            children = children.get("_children");
        }
        return extractValues(children);
    }

    private Map<String, Object> extractValues(JsonNode children) {
        Map<String, Object> result = new LinkedHashMap<>();
        children.fields().forEachRemaining(entry -> {
            String key = entry.getKey();
            if (!key.equals("_class"))
                result.put(key, entry.getValue().get("_value").toString());
        });
        return result;
    }


    private Map<String, Object> toMap(ObjectNode objectNode) {
        Map<String, Object> result = new LinkedHashMap<>();
        objectNode.fields().forEachRemaining(entry -> {
            result.put(entry.getKey(), entry.getValue().toString());
        });
        return result;
    }
}

2. Create a Custom mongo converter and feed him with the custom deserializer.

Actually we do not feed him with the serializer directly but by the mean of an ObjectMapper configured with that Custom deserializer

public class CustomMappingMongoConverter extends MappingMongoConverter {

    //The configured objectMapper that will be passed during instatiation
    private ObjectMapper objectMapper; 

    public CustomMappingMongoConverter(DbRefResolver dbRefResolver, MappingContext<? extends MongoPersistentEntity<?>, MongoPersistentProperty> mappingContext, ObjectMapper objectMapper) {
        super(dbRefResolver, mappingContext);
        this.objectMapper = objectMapper;
    }

    @Override
    public <S> S read(Class<S> clazz, Bson dbObject) {
        try {
            return objectMapper.readValue(dbObject.toString(), clazz);
        } catch (IOException e) {
            throw new RuntimeException(dbObject.toString(), e);
        }
    }


    //in case you want to serialize with your custom objectMapper as well
    @Override
    public void write(Object obj, Bson dbo) {
        String string = null;
        try {
            string = objectMapper.writeValueAsString(obj);
        } catch (JsonProcessingException e) {
            throw new RuntimeException(string, e);
        }
        ((DBObject) dbo).putAll((DBObject) BasicDBObject.parse(string));
    }

}

3. Create and configure the object mapper then instantiate the custom MongoMappingConverter and add it to Mongo configurations

public class MongoConfiguration extends AbstractMongoClientConfiguration {

   
    //... other configuration method beans
   
    @Bean
    @Override
    public MappingMongoConverter mappingMongoConverter() throws Exception {
        DbRefResolver dbRefResolver = new DefaultDbRefResolver(mongoDbFactory());
        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.configure(FAIL_ON_UNKNOWN_PROPERTIES, false);
        objectMapper.registerModule(new SimpleModule() {
            {
                addDeserializer(Map.class, new ErrorMapJsonDeserializer());
            }
        });
        return new CustomMappingMongoConverter(dbRefResolver, mongoMappingContext(), objectMapper);
    }
}