Linux上的虚拟内存大小

Question

I'm trying to understand in depth the virtual memory management on OS Linux.

I don't really understand how the OS determine the size of the VM for a process.

I know that an 32-bits x86 OS can give up to 3 GB of vm adress space... Is it always true ?

In my case, i have about 110 MB physical memory with and 32-bits Linux, and my main process have a vm adress space about 660 MB. However, only 50 MB are in physical memory (the RSS of my process), so my physical RAM isn't full. The rest is free and almost the whole is used by page cache. That's seems to be a normal behaviour.

If i check /proc/my_process_PID/smap, there are several 8 MB anonymous VMA.

My actual problem is that i need to make a additionnal 10 MB malloc in the code, but unfortunately OOM-Killer kills my process (out-of-memory) ... I think there are no more free available pages in the vm for the Heap, isn't it ? Is there an huge memory leak somewhere ?

Why the OS doesn't extend my process vm size so ?

For information the vm size is unlimited : ulimit -v : unlimited

Answer 1

You can have 3GB of virtual memory per process (approximately, on many 32-bit Linux), and keep on creating new processes taking up gigabytes upon gigabytes of virtual memory. There's a small overhead in the kernel, but virtual memory is very cheap. The amount of address space you're using is probably not important, and it probably won't trigger the OOM killer.

However, your system only has so much RAM. As you start using pages in your address space (writing to them), the kernel is forced to find physical RAM to map them to. If there's no physical RAM, the kernel can evict other pages from RAM -- either swap them out or discard them. But if it can't evict any pages, then it triggers the OOM killer.

Running out of address space will cause malloc to return NULL on my system, instead of triggering the OOM killer.

It sounds like your process is simply using too much RAM. RSS isn't the amount of memory your process uses, it's just the amount that's in physical RAM right now . If your process has a memory leak and keeps growing, RSS will eventually stop growing -- because for every new page you use, the kernel will evict one page from your process.

Try using a memory profiler, like Valgrind . This will help you sort out what memory you should be worried about (mallocs) and what memory you can ignore (shared libraries and other memory mapped files). The kernel (and /proc) won't give you enough detail.

Answer 2

The total amount of virtual memory space available on a linux system is (roughly) RAM + swap space - kernel overhead . The RAM is the hardware you have installed and the kernel overhead is roughly a constant (varies between kernel versions though), so the only easy way to control the total VM space available is by adding or removing swap space.

In addition to the total limit, there's also a per-process VM limit. This is configurable and (on 32bit linux at least) is at most 3GB, but may be quite a bit less. ulimit -v will tell you this limit or can be used to change it.

When a process requests more VM space (generally via malloc), the kernel will look at all of these limits, and if any of them would be exceeded, will return 0. The OOM killer, on the other hand, only kicks in when you're getting close to the total VM limit. However, when the OOM killer kills you, you just die -- there's no out-of-memory error or any opportunity to catch it.

So if you really are running into the total VM limit, and want to avoid that, you can either allocate more swap space, or get rid of (kill or not start in the first place) other processes that are using a lot of VM space so as to free up some for your program.