Optimal method to mmap a file to RAM?

Question

I am using mmap to read a file and I only recently found out that it is not actually getting it into RAM, but is only creating a virtual address space for it. This will cause any accessing of the data to still use disk which I want to avoid, so I want to read it all into RAM.

I am reading the file via:

char* cs_virt;
cs_virt = (char*)mmap(0, nchars, PROT_READ, MAP_PRIVATE, finp, offset);

and when I loop after this, I see that the virtual memory for this process has, indeed, been blown up. I want to copy this into RAM, though, so I do the following:

char* cs_virt;
cs_virt = (char*)mmap(0, nchars, PROT_READ, MAP_PRIVATE, finp, offset);
cs = (char*)malloc(nchars*sizeof(char));
for(int ichar = 0; ichar < nchars; ichar++) {
    cs[ichar] = cs_virt[ichar]; 
}

Is this the best method? If not, what is a more efficient method to do this? I have this taking place in a function and cs is declared outside the function. Once I exit the function, I will retain cs , but will cs_virt need to be deleting or will it go away on it's own since it is declared locally in the function?

Answer 1

If you are using Linux, you may be able to use MAP_POPULATE :

MAP_POPULATE (since Linux 2.5.46)
Populate (prefault) page tables for a mapping. For a file mapping, this causes read-ahead on the file. Later accesses to the mapping will not be blocked by page faults. MAP_POPULATE is supported for private mappings only since Linux 2.6.23.

This may be useful if you have time to spare when you mmap() but your later accesses need to be responsive. Consider also MAP_LOCKED if you really need the file to be mapped in and never swapped back out.

Answer 2

MPI and I/O is a murky issue. HDF5 seems to be the most common library that can help you with that, but it often needs tuning for the particular cluster, which is often impossible for mere users of the cluster. A colleague of mine had better success with SIONlib , and was able to get his code working on nearly 1e6 cores on JUGENE with that, so I'd have look at that.

In both cases you will probably need to adapt your file format. In the case of my colleague it even paid of to write the data in parallel fashion using SIONlib, and to later do e sequential postprocessing to "defragment" the holes left be the parallel access pattern that SIONlib chose. It might be similar for input.