What is the best approach for saving statistical data on a file using spring framework?

Question

What is the best approach for saving statistical data on a file using spring framework? is there any available library that offers reading and updating the data on a file? or should I build my own IO code?

I already have a relational database, but don't like the approach of creating an additional table to save the calculated values in different multiple tables with joins, also don't want to add more complexity to the project by using an additional database for just one task like MongoDB.

To understand the complexity of this report, Imagine you are drawing a chart with a total number of daily transactions for a full year with billions of records at any time with a lot of extra information like( total and average with different currencies on different rates).

So, my approach was to generate those data in a file on a regular basis, so later I don't need to generate them again once requested, only accumulate the new dates if available to the file

Is this approach fine? and what is the best library to do that in an efficient way?

Update

I found this answer useful for why sometimes people prefer using flat files rather than the relational or non-relational one

Is it faster to access data from files or a database server?

Answer 1

I would preferet to use MongoDB for such purposes, but if you need simple approach, you can write your data to csv\\excel file. Just using I\\O

    List<String> data = new ArrayList<>();
    data.add("head1;head2;head3");
    data.add("a;b;c");
    data.add("e;f;g");
    data.add("9;h;i");

    Files.write(Paths.get("my.csv"), data);

That is all) How to convert your own object, to such string 'filed1;field2' I think you know. Also you can use apache-poi csv library, but I think this is way much faster.

    Files.write(Paths.get("my.csv"), data, StandardOpenOption.APPEND);

If you want to append data to existed file, there are many different options in StandardOpenOption.

For reading you should use Files.readAllLines(Paths.get("my.csv")); it will return you list of strings. Also you can read lines in range.

But if you need to retrieve one column, or update two columns where, and so on. You should read about MongoDB or other not relational databases. It is difficult write about MongoDB here, you should read documentation.

Enjoy)

Answer 2

I found a library that can be used to write/read CSV files easily and can be mapped to objects as well Jackson data formats

Find an example with spring