Efficient and scalable way to sort large amount of strings in Java

Question

I am looking for some ideas idea on sorting large amount of strings from an input file and print out the sorted results to a new file in Java. The requirement is that the input file could be extremely large. I need to consider the performance in the solution, so any ideas?

Answer 1

External Sorting technique is generally used to sort huge amounts of data. May be this is what you need.

externalsortinginjava is the java library for this.

Answer 2

Is an SQL database available? If you inserted all the data into a table, with the sortable column or section indexed, you may (or may not) be able to output the sorted result more efficiently. This solution may also be helpful if the amount of data, outweighs the amount of RAM available.

It would be interesting to know how large, and what the purpose is.

Answer 3

Break the file into amounts you can read in memory. Sort each amount and write to a file. (If you could fit everything into memory you are done) Merge sort the resulting files into a single sorted file.

You can also do a form of radix sort to improve CPU efficiency, but the main bottleneck is all the re-writing and re-reading you have to do.