Fetch data from mongo db in spring boot application where collection name & fields to fetch are known at runtime

Question

I need to create a spring boot batch job in which, I need to fetch data from mongoDB where I don't have info about collection name & fields to fetch at coding time. I get this info only when the batch starts. Eg When batch starts, I can read properties file where I get collection name by 1 property & another property gives list of fields to fetch, third filed provides criteria/condition for query So, due to this, I cant have a Java POJO defined which have mapping for collection or I cant create any MongoRepository\Template (collection name is known at runtime).

What I want to know is, just like plain old native SQL, if i get to know fields name & table name, on the fly, SQL can be build & can be fired to get the data:

String dynamicQuery = "SELECT " + commaSeperatedFieldsList + " FROM " + tableName + " WHERE " + criteria;

Is there any way by which same thing can be achieved in spring boot + mongo DB?

Answer 1

You can use MongoTemplate for this which can be autowired as spring provides and configures it for you automatically.

It has a

find(Query query, Class<T> entityClass, String collectionName)

method which lets you define a custom collection name and a custom entityClass.

For the dynamic query use BasicQuery as Query impl to pass a raw mongo json query and fields/projection as json if you want to limit the fields returned.

Use org.bson.Document as entityClass which is basically a Map implementation which lets you iterate over the fields in a dynamic manner.

mongoTemplate.find(new BasicQuery("{ name: \"mongodb\"}", "{ name: 1}"), Document.class, "your-collection-name").forEach(x -> {
    x.get("name"); // access a specific field

    x.forEach((key, value) -> {
        // iterate over all fields
    });
});

When you deal with a large result consider using MongoTemplate 's stream() method in the same way as this doesn't load all documents into memory at once and you can process it during execution one by one.