Efficient way to sort large vectors in-memory

Question

I need to sort large vectors of user defined data-type of 32 GB size in memory on 64GB RAM High Performance Computing (HPC) machine again and again according to some user defined function. I am using std::sort for the same. However, it is turning out that std::sort is taking a large amount of time >1 hour. Is there some other function which I may use which sorts faster than std::sort.

I am using the following gcc version on my system: gcc (Ubuntu/Linaro 4.6.4-6ubuntu2) 4.6.4

Answer 1

You may want to use some mix of std::sort() (which uses quicksort algorithm) and mergesort and apply multithreading here.

For example, having some amount of cores in CPU, it might be reasoble to split the data, sort each part by std::sort on the cores simultaneously, and then merge them just as a mergesort merges sorted arrays.

Regards