Debian “unable to allocate memory”

Question

I have Debian jessie installed.

kernel version:

Linux srv1 3.16-3-amd64 #1 SMP Debian 3.16.5-1 (2014-10-10) x86_64 GNU/Linux

It has 32Gb memory installed. It seems more then enough for my task. Heavy utilized asterisk process leak a lot of memory and produce a lot of troubles.

Asterisk itself and bash from time to time reports "unable to allocate memory".

At the same time based on attached top report server has 7 Gb unused memory.

It will be great if someone will help to figure out what is wrong: - what kind of resources were exhausted - what need to be tuned for 100% server resources utilization.

Top:

Tasks: 130 total, 1 running, 129 sleeping, 0 stopped, 0 zombie
%Cpu0 : 6,0 us, 1,3 sy, 0,0 ni, 21,5 id, 70,8 wa, 0,0 hi, 0,3 si, 0,0 st
%Cpu1 : 70,2 us, 0,3 sy, 0,0 ni, 24,8 id, 4,6 wa, 0,0 hi, 0,0 si, 0,0 st
%Cpu2 : 3,0 us, 0,7 sy, 0,0 ni, 84,6 id, 11,7 wa, 0,0 hi, 0,0 si, 0,0 st
%Cpu3 : 6,9 us, 0,7 sy, 0,0 ni, 78,2 id, 14,2 wa, 0,0 hi, 0,0 si, 0,0 st
%Cpu4 : 3,3 us, 0,7 sy, 0,0 ni, 84,3 id, 11,7 wa, 0,0 hi, 0,0 si, 0,0 st
%Cpu5 : 4,0 us, 0,7 sy, 0,0 ni, 90,1 id, 5,3 wa, 0,0 hi, 0,0 si, 0,0 st

KiB Mem: 32985292 total, 25834636 used, 7150656 free, 38312 buffers
KiB Swap: 58592252 total, 1767420 used, 56824832 free. 37988 cached Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

7615 asterisk 20 0 3147628 2,813g 1820 S 69,8 8,9 5:35.84 php
2389 asterisk 20 0 20,150g 1,207g 2176 S 28,5 3,8 247:42.19 asterisk
976 mysql 20 0 1411844 19392 2624 S 1,3 0,1 15:13.28 mysqld
21651 root 20 0 24876 2824 2316 R 0,7 0,0 0:02.08 top
...

Answer 1

Your server is using the memory solely for application data. In your top excerpt the buffers and cached are very low. Since asterisk is probably not very disk intensive, that seems fine. But your swap is also being used. This contradicts your assumption that 32GB memory seems enough.

It would be a good idea to install the sysstat package to monitor your system on what is really going on. Top shows only the current memory and process information. sysstat with the included sar command records system information every few minutes to retrieve it later for analysis.