使用 spring boot 将大量数据从一个集合复制到 Mongodb 中的另一个集合

Question

We are writing a scheduler to take backup of data from one collection to another collection in Mongodb using spring boot. The data can be 500K to 1Million docs. Once the copy was completed we should delete the data from old collection. Currently we are using spring data pagination to get the chunks of data and saving to new collection and then deleting.

Is this approach fine or any optimistic approach is suggestible.

Answer 1

As you using spring data with pagination for this task, it means you utilizing container for process documents.

You can trigger set of system command(mongo export and after import delete data from source) from scheduler.

Example like..

SystemCommandTasklet tasklet = new SystemCommandTasklet();

tasklet.setCommand("");
tasklet.setWorkingDirectory("/home/merlin");