非阻塞mlock（）

Question

Is there such a thing as non-blocking mlock()? Under heavy traffic, I don't want my threads block waiting for I/O. I'd rather just tell the Linux kernel what region I need from a mmap()'d file using mlock() and then get notified when the pages has been fetched. (As far as I know, the standard mlock() call is blocking.)

Answer 1

The mlock interface does not seem to have any built-in functionality like what you want, so I think the only way to achieve it is to use a separate thread to perform the mlock and have that thread notify you (via a condition variable, semaphore, or other mechanism) when the mlock has returned. Obviously this is going to incur some overhead, but if your goal is to get realtime latency guarantees rather than improving overall runtime/average latency, it's still an obvious win.

Of course it's hard to make any realtime assumptions unless you use mlockall since your code could get swapped out. So it might make more sense to use mlockall and POSIX AIO (or a similar but cleaner-API system implemented yourself in terms of threads) to do reads rather than using mmap . Then you have a hard guarantee that, once your data is fetched, it can't be swapped out.

Answer 2

I believe that you want a combination of madvise() or posix_madvise() and mincore() .

You would use a madvise call to ask the kernel for MADV_WILLNEED . Then you would have to poll using mincore to check if the pages had been read into memory.

If the system is under heavy memory load it is possible for the pages to never be read in by the madvise call, so you would need a timeout and a fallback to a blocking read mode.

Answer 3

If you mmap() a file, then you treat the file as memory, the VM pages in/out file blocks depending on your usage, which may benefit from read ahead.

There are various ways to do non-blocking I/O (asyncio, poll, select, epoll) but mlock() is about holding a memory region in RAM, not allowing it to be paged out to the swap partition; though under severe memory pressure the kernel may not honour that.

Most likely, mmap(2) will suffice as if you are using memory pages, they will not be selected for page out, but kept in memory anyway, so consider if this question is premature optomisation, the kernel tries hard to provide (by default) good performance.