堆栈内存溢出
  简体   繁体   English

mmap返回ENOMEM

[英]mmap returning ENOMEM

 原文  2017-11-15 10:16:09  3   1    c/ linux/ mmap

问题描述

I read every post linked to this topic and didn't find anything that was able to solve my problem. 我阅读了与该主题相关的每篇文章，但没有找到能够解决我的问题的内容。

I'm trying to map a 900Mb MAP_SHARED backed by an underlying file. 我正在尝试映射由基础文件支持的900Mb MAP_SHARED。 We have 2Gb of RAM and only 2Gb of VIRTUAL MEMORY available (1Gb reserved for Kernel + 1Gb reserved for Hypervisor). 我们有2Gb的RAM和仅2Gb的虚拟内存（为内核保留1Gb，为管理程序保留1Gb）。 

ptr = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0);

I was thinking about a lack of VMEM available so I tried to put "sysctl -w vm.overcommit_memory=1" but it failed with this too. 我当时正在考虑缺少可用的VMEM，因此我尝试放置“ sysctl -w vm.overcommit_memory = 1”，但是这样做也失败了。

I made a little loop (below) to know at which size mmap is really failing and I found it was next to 700Mb, so it works fine up to 700Mb and then failed. 我做了一个小循环（下图），以了解mmap实际发生故障的大小，我发现它的大小接近700Mb，因此它可以正常工作到700Mb，然后发生故障。 

ptr = mmap(NULL, size, PROT_READ, MAP_SHARED, fd, 0);
if ((MAP_FAILED == ptr) && (errno == ENOMEM)) {
    for (i = 0; i < 16; i++) {
        ptr = mmap(NULL, size / 16, PROT_READ, MAP_SHARED, fd, (size / 16) * i);
        if (MAP_FAILED == ptr) {
            printf("map num:%d failed errno:%d", i, errno);
            break;
        }
    }

Thanks for your help. 谢谢你的帮助。

PS: shm_open and ftruncate are not returning any error. PS：shm_open和ftruncate不返回任何错误。 ftruncate and mmap are working with the same size. ftruncate和mmap使用相同的大小。

free -m is returning Swap 0 0 0. free -m返回Swap 0 0 0。

EDIT 1: Obviously it cames up that I've no 900Mb contiguous space in VMEM, like I said with little mmap, stair by stair, I'm able to reach a mmap of ~720Mo but I didn't find a workaround about this contiguous limitation. 编辑1：很明显，我在VMEM中没有900Mb的连续空间，就像我说的那样，只有很小的mmap，一个台阶一个台阶，我能够达到〜720Mo的mmap，但是我没有找到解决方法连续限制。 Do I have to split it on several file descriptor or something like that ? 我是否必须将其拆分为几个文件描述符或类似的文件？

EDIT 2: 编辑2：

With some shmem size improvement I'm now able to mmap(870Mb), almost there :D But I don't understand why there is a HUGE GAP of ~880Mb between the heap and the first library, does someone have an hint for me? 通过一些shmem大小的改进，我现在几乎可以映射（870Mb）了：D但我不明白为什么堆和第一个库之间的间隙大约为880Mb ，有人对我有提示吗 ？

Below the result of proc/$PID/maps 低于proc / $ PID / maps的结果 

08048000-08081000 r-xp 00000000 08:02 2564       /usr/bin/myapp
08081000-0820a000 rw-p 00038000 08:02 2564       /usr/bin/myapp
0820a000-09151000 rwxp 00000000 00:00 0          [heap]
40000000-40022000 r-xp 00000000 08:02 14697      /lib/ld-2.22.so
40022000-40023000 r--p 00021000 08:02 14697      /lib/ld-2.22.so
40023000-40024000 rw-p 00022000 08:02 14697      /lib/ld-2.22.so
40024000-40025000 rw-p 00000000 00:00 0 
40027000-40040000 r-xp 00000000 08:02 14692      /lib/libpthread-2.22.so
40040000-40041000 r--p 00018000 08:02 14692      /lib/libpthread-2.22.so
40041000-40042000 rw-p 00019000 08:02 14692      /lib/libpthread-2.22.so
40042000-40044000 rw-p 00000000 00:00 0 
40044000-4004b000 r-xp 00000000 08:02 14686      /lib/librt-2.22.so
4004b000-4004c000 r--p 00006000 08:02 14686      /lib/librt-2.22.so
4004c000-4004d000 rw-p 00007000 08:02 14686      /lib/librt-2.22.so
4004d000-401b2000 r-xp 00000000 08:02 39034      /usr/lib/libxml2.so.2.9.2
401b2000-401b7000 rw-p 00164000 08:02 39034      /usr/lib/libxml2.so.2.9.2
401b7000-401b8000 rw-p 00000000 00:00 0 
401b8000-40368000 r-xp 00000000 08:02 14699      /lib/libc-2.22.so
40368000-40369000 ---p 001b0000 08:02 14699      /lib/libc-2.22.so
40369000-4036b000 r--p 001b0000 08:02 14699      /lib/libc-2.22.so
4036b000-4036c000 rw-p 001b2000 08:02 14699      /lib/libc-2.22.so
4036c000-4036f000 rw-p 00000000 00:00 0 
4036f000-40372000 r-xp 00000000 08:02 14881      /lib/libdl-2.22.so
40372000-40373000 r--p 00002000 08:02 14881      /lib/libdl-2.22.so
40373000-40374000 rw-p 00003000 08:02 14881      /lib/libdl-2.22.so
40374000-40375000 rw-p 00000000 00:00 0 
40375000-4038a000 r-xp 00000000 08:02 14698      /lib/libz.so.1.2.8
4038a000-4038b000 rw-p 00015000 08:02 14698      /lib/libz.so.1.2.8
4038b000-403d6000 r-xp 00000000 08:02 14664      /lib/libm-2.22.so
403d6000-403d7000 r--p 0004a000 08:02 14664      /lib/libm-2.22.so
403d7000-403d8000 rw-p 0004b000 08:02 14664      /lib/libm-2.22.so
403d8000-403da000 rw-p 00000000 00:00 0 
403da000-40b6a000 rw-s 17180000 00:05 2845       /dev/vmfileshm20
40b6a000-40b6b000 ---p 00000000 00:00 0 
40b6b000-4136b000 rw-p 00000000 00:00 0 
4136b000-4156c000 rw-s 17a00000 00:05 2758       /dev/vmfileshm12
4156c000-4176d000 rw-s 17c80000 00:05 2801       /dev/vmfileshm16
4176d000-4176e000 ---p 00000000 00:00 0 
4176e000-41f6e000 rw-p 00000000 00:00 0 
41f6e000-42e6f000 rw-s 17f00000 00:05 2670       /dev/vmfileshm4
42f00000-42f21000 rw-p 00000000 00:00 0 
42f21000-43000000 ---p 00000000 00:00 0 
43000000-43021000 rw-p 00000000 00:00 0 
43021000-43100000 ---p 00000000 00:00 0 
43100000-44001000 rw-s 1ad80000 00:05 2713       /dev/vmfileshm8
44001000-44002000 ---p 00000000 00:00 0 
44002000-44802000 rw-p 00000000 00:00 0 
44802000-449ff000 rw-s 1d300000 00:05 2890       /dev/vmfileshm24
449ff000-44a00000 ---p 00000000 00:00 0 
44a00000-45200000 rw-p 00000000 00:00 0 
45200000-45201000 ---p 00000000 00:00 0 
45201000-45a01000 rw-p 00000000 00:00 0 
45a01000-47570000 rw-s 1d780000 00:05 2933       /dev/vmfileshm28
47570000-47aef000 rw-s 28380000 00:05 2967       /dev/vmfileshm31
47aef000-47af0000 ---p 00000000 00:00 0 
47af0000-482f0000 rw-p 00000000 00:00 0 
482f0000-490e1000 r--s 2c480000 00:05 3025       /dev/vmfileshm37
490e1000-49ed2000 r--s 2c480000 00:05 3025       /dev/vmfileshm37
49ed2000-4acc3000 r--s 2c481000 00:05 3025       /dev/vmfileshm37
4acc3000-4bab4000 r--s 2c482000 00:05 3025       /dev/vmfileshm37
4bab4000-4c8a5000 r--s 2c483000 00:05 3025       /dev/vmfileshm37
4c8a5000-4d696000 r--s 2c484000 00:05 3025       /dev/vmfileshm37
4d696000-4e487000 r--s 2c485000 00:05 3025       /dev/vmfileshm37
4e487000-4f278000 r--s 2c486000 00:05 3025       /dev/vmfileshm37
4f278000-50069000 r--s 2c486000 00:05 3025       /dev/vmfileshm37
50069000-50e5a000 r--s 2c487000 00:05 3025       /dev/vmfileshm37
50e5a000-51c4b000 r--s 2c488000 00:05 3025       /dev/vmfileshm37
51c4b000-52a3c000 r--s 2c489000 00:05 3025       /dev/vmfileshm37
52a3c000-5382d000 r--s 2c48a000 00:05 3025       /dev/vmfileshm37
5382d000-5461e000 r--s 2c48b000 00:05 3025       /dev/vmfileshm37
5461e000-5540f000 r--s 2c48c000 00:05 3025       /dev/vmfileshm37
5540f000-56200000 r--s 2c48d000 00:05 3025       /dev/vmfileshm37
56200000-56ff1000 r--s 2c48d000 00:05 3025       /dev/vmfileshm37
56ff1000-57de2000 r--s 2c48e000 00:05 3025       /dev/vmfileshm37
57de2000-58bd3000 r--s 2c48f000 00:05 3025       /dev/vmfileshm37
58bd3000-599c4000 r--s 2c490000 00:05 3025       /dev/vmfileshm37
599c4000-5a7b5000 r--s 2c491000 00:05 3025       /dev/vmfileshm37
5a7b5000-5b5a6000 r--s 2c492000 00:05 3025       /dev/vmfileshm37
5b5a6000-5c397000 r--s 2c493000 00:05 3025       /dev/vmfileshm37
5c397000-5d188000 r--s 2c494000 00:05 3025       /dev/vmfileshm37
5d188000-5df79000 r--s 2c494000 00:05 3025       /dev/vmfileshm37
5df79000-5ed6a000 r--s 2c495000 00:05 3025       /dev/vmfileshm37
5ed6a000-5fb5b000 r--s 2c496000 00:05 3025       /dev/vmfileshm37
5fb5b000-6094c000 r--s 2c497000 00:05 3025       /dev/vmfileshm37
6094c000-6173d000 r--s 2c498000 00:05 3025       /dev/vmfileshm37
6173d000-6252e000 r--s 2c499000 00:05 3025       /dev/vmfileshm37
6252e000-6331f000 r--s 2c49a000 00:05 3025       /dev/vmfileshm37
6331f000-64110000 r--s 2c49b000 00:05 3025       /dev/vmfileshm37
64110000-64f01000 r--s 2c49b000 00:05 3025       /dev/vmfileshm37
64f01000-65cf2000 r--s 2c49c000 00:05 3025       /dev/vmfileshm37
65cf2000-66ae3000 r--s 2c49d000 00:05 3025       /dev/vmfileshm37
66ae3000-678d4000 r--s 2c49e000 00:05 3025       /dev/vmfileshm37
678d4000-686c5000 r--s 2c49f000 00:05 3025       /dev/vmfileshm37
686c5000-694b6000 r--s 2c4a0000 00:05 3025       /dev/vmfileshm37
694b6000-6a2a7000 r--s 2c4a1000 00:05 3025       /dev/vmfileshm37
6a2a7000-6b098000 r--s 2c4a1000 00:05 3025       /dev/vmfileshm37
6b098000-6be89000 r--s 2c4a2000 00:05 3025       /dev/vmfileshm37
6be89000-6cc7a000 r--s 2c4a3000 00:05 3025       /dev/vmfileshm37
6cc7a000-6da6b000 r--s 2c4a4000 00:05 3025       /dev/vmfileshm37
6da6b000-6e85c000 r--s 2c4a5000 00:05 3025       /dev/vmfileshm37
6e85c000-6f64d000 r--s 2c4a6000 00:05 3025       /dev/vmfileshm37
6f64d000-7043e000 r--s 2c4a7000 00:05 3025       /dev/vmfileshm37
7043e000-7122f000 r--s 2c4a8000 00:05 3025       /dev/vmfileshm37
7122f000-72020000 r--s 2c4a8000 00:05 3025       /dev/vmfileshm37
72020000-72e11000 r--s 2c4a9000 00:05 3025       /dev/vmfileshm37
72e11000-73c02000 r--s 2c4aa000 00:05 3025       /dev/vmfileshm37
73c02000-749f3000 r--s 2c4ab000 00:05 3025       /dev/vmfileshm37
749f3000-757e4000 r--s 2c4ac000 00:05 3025       /dev/vmfileshm37
757e4000-765d5000 r--s 2c4ad000 00:05 3025       /dev/vmfileshm37
765d5000-773c6000 r--s 2c4ae000 00:05 3025       /dev/vmfileshm37
773c6000-781b7000 r--s 2c4af000 00:05 3025       /dev/vmfileshm37
781b7000-78fa8000 r--s 2c4af000 00:05 3025       /dev/vmfileshm37
78fa8000-79d99000 r--s 2c4b0000 00:05 3025       /dev/vmfileshm37
79d99000-7ab8a000 r--s 2c4b1000 00:05 3025       /dev/vmfileshm37
7ab8a000-7b97b000 r--s 2c4b2000 00:05 3025       /dev/vmfileshm37
7b97b000-7c76c000 r--s 2c4b3000 00:05 3025       /dev/vmfileshm37
7c76c000-7d55d000 r--s 2c4b4000 00:05 3025       /dev/vmfileshm37
7d55d000-7e34e000 r--s 2c4b5000 00:05 3025       /dev/vmfileshm37
7e34e000-7f13f000 r--s 2c4b6000 00:05 3025       /dev/vmfileshm37
7f91e000-7f93f000 rw-p 00000000 00:00 0          [stack]
00000000-00000000 r-xp 00000000 00:00 0          [vdso]

EDIT 3: Now I've moved all little shared memories (see below) just after the heap and let the >900Mb one above the lib (in order to have the maximum available space for my big shared mem) 编辑3：现在，我已经在堆之后移动了所有小的共享内存（请参见下文），并在库的上方放置了一个大于900Mb的内存（以便为我的大型共享内存提供最大的可用空间） 

08048000-08081000 r-xp 00000000 08:02 2652       /usr/bin/myapp
08081000-0820a000 rw-p 00038000 08:02 2652       /usr/bin/myapp
0820a000-09151000 rwxp 00000000 00:00 0          [heap]
0b001000-0b791000 rw-s 17180000 00:05 2655       /dev/shm20
0b791000-0b992000 rw-s 17a00000 00:05 2567       /dev/shm12
0b992000-0bb93000 rw-s 17c80000 00:05 2611       /dev/shm16
0bb93000-0ca94000 rw-s 17f00000 00:05 2479       /dev/shm4
0ca94000-0d995000 rw-s 19e00000 00:05 2523       /dev/shm8
0d995000-0db92000 rw-s 1ad80000 00:05 2700       /dev/shm24
0db92000-0f701000 rw-s 1b680000 00:05 2743       /dev/shm28
0f701000-0fc80000 rw-s 1eb80000 00:05 2776       /dev/shm31
40000000-40022000 r-xp 00000000 08:02 14697      /lib/ld-2.22.so
40022000-40023000 r--p 00021000 08:02 14697      /lib/ld-2.22.so
40023000-40024000 rw-p 00022000 08:02 14697      /lib/ld-2.22.so
40024000-40025000 rw-p 00000000 00:00 0 
40027000-40040000 r-xp 00000000 08:02 14692      /lib/libpthread-2.22.so
40040000-40041000 r--p 00018000 08:02 14692      /lib/libpthread-2.22.so
40041000-40042000 rw-p 00019000 08:02 14692      /lib/libpthread-2.22.so
40042000-40044000 rw-p 00000000 00:00 0 
40044000-4004b000 r-xp 00000000 08:02 14686      /lib/librt-2.22.so
4004b000-4004c000 r--p 00006000 08:02 14686      /lib/librt-2.22.so
4004c000-4004d000 rw-p 00007000 08:02 14686      /lib/librt-2.22.so
4004d000-401b2000 r-xp 00000000 08:02 39031      /usr/lib/libxml2.so.2.9.2
401b2000-401b7000 rw-p 00164000 08:02 39031      /usr/lib/libxml2.so.2.9.2
401b7000-401b8000 rw-p 00000000 00:00 0 
401b8000-40368000 r-xp 00000000 08:02 14699      /lib/libc-2.22.so
40368000-40369000 ---p 001b0000 08:02 14699      /lib/libc-2.22.so
40369000-4036b000 r--p 001b0000 08:02 14699      /lib/libc-2.22.so
4036b000-4036c000 rw-p 001b2000 08:02 14699      /lib/libc-2.22.so
4036c000-4036f000 rw-p 00000000 00:00 0 
4036f000-40372000 r-xp 00000000 08:02 14881      /lib/libdl-2.22.so
40372000-40373000 r--p 00002000 08:02 14881      /lib/libdl-2.22.so
40373000-40374000 rw-p 00003000 08:02 14881      /lib/libdl-2.22.so
40374000-40375000 rw-p 00000000 00:00 0 
40375000-4038a000 r-xp 00000000 08:02 14698      /lib/libz.so.1.2.8
4038a000-4038b000 rw-p 00015000 08:02 14698      /lib/libz.so.1.2.8
4038b000-403d6000 r-xp 00000000 08:02 14664      /lib/libm-2.22.so
403d6000-403d7000 r--p 0004a000 08:02 14664      /lib/libm-2.22.so
403d7000-403d8000 rw-p 0004b000 08:02 14664      /lib/libm-2.22.so
403d8000-403da000 rw-p 00000000 00:00 0 
403da000-403db000 ---p 00000000 00:00 0 
403db000-40bdb000 rw-p 00000000 00:00 0 
40bdb000-40bdc000 ---p 00000000 00:00 0 
40bdc000-413dc000 rw-p 00000000 00:00 0 
41400000-41421000 rw-p 00000000 00:00 0 
41421000-41500000 ---p 00000000 00:00 0 
41500000-41521000 rw-p 00000000 00:00 0 
41521000-41600000 ---p 00000000 00:00 0 
41600000-41601000 ---p 00000000 00:00 0 
41601000-41e01000 rw-p 00000000 00:00 0 
41e01000-41e02000 ---p 00000000 00:00 0 
41e02000-42602000 rw-p 00000000 00:00 0 
42602000-42603000 ---p 00000000 00:00 0 
42603000-42e03000 rw-p 00000000 00:00 0 
42e03000-42e04000 ---p 00000000 00:00 0 
42e04000-43604000 rw-p 00000000 00:00 0 
43604000-47065000 r--s 22280000 00:05 2835       /dev/shm37
47065000-4aac6000 r--s 22283000 00:05 2835       /dev/shm37
4aac6000-4e527000 r--s 22287000 00:05 2835       /dev/shm37
4e527000-51f88000 r--s 2228a000 00:05 2835       /dev/shm37
51f88000-559e9000 r--s 2228e000 00:05 2835       /dev/shm37
559e9000-5944a000 r--s 22292000 00:05 2835       /dev/shm37
5944a000-5ceab000 r--s 22295000 00:05 2835       /dev/shm37
5ceab000-6090c000 r--s 22299000 00:05 2835       /dev/shm37
6090c000-6436d000 r--s 2229d000 00:05 2835       /dev/shm37
6436d000-67dce000 r--s 222a0000 00:05 2835       /dev/shm37
67dce000-6b82f000 r--s 222a4000 00:05 2835       /dev/shm37
6b82f000-6f290000 r--s 222a8000 00:05 2835       /dev/shm37
6f290000-72cf1000 r--s 222ab000 00:05 2835       /dev/shm37
72cf1000-76752000 r--s 222af000 00:05 2835       /dev/shm37
76752000-7a1b3000 r--s 222b3000 00:05 2835       /dev/shm37
7a1b3000-7dc14000 r--s 222b6000 00:05 2835       /dev/shm37
7ff25000-7ff46000 rw-p 00000000 00:00 0          [stack]
00000000-00000000 r-xp 00000000 00:00 0          [vdso]

As mmap continued to fail with ENOMEM error even if it got enough place to map it I decided to chunk the mmap. 由于mmap继续失败并出现ENOMEM错误，即使它有足够的位置进行映射，我还是决定对mmap进行分块。 

for (i = 0; ((i < NB_CHUNK) && (ptr != MAP_FAILED)); i++) {
              ptr = mmap(NULL,
                         ALIGN_ON_PAGE_SIZE((BIG_SIZE / NB_CHUNK)),
                         PROT_READ,
                         MAP_SHARED,
                         fd,
                         (off_t)ALIGN_ON_PAGE_SIZE((BIG_SIZE / NB_CHUNK) * i));
}

Now I've no more ENOMEM error and the mmap is doing well but when I try to read a value in it I'm only reading zeroed memory. 现在，我不再遇到ENOMEM错误，并且mmap运行良好，但是当我尝试读取其中的值时，我只读取了零内存。

The question is, Can I chunk a mmap this way ? 问题是，我可以通过这种方式对地图进行分块吗？

1 个解决方案

解决方案1
 0  2018-08-27 03:06:34

Maybe you should look at /proc/<pid>/limits while running your process, find "Max address space" row and see its "Soft Limit" column. 也许您应该在运行进程时查看/ proc / <pid> / limits，找到“最大地址空间”行，并查看其“软限制”列。 mmap will cause ENOMEM for limited address space. mmap将导致ENOMEM的地址空间有限。 For this situation, you can call setrlimit(2) in your program to modify Max address space of process. 对于这种情况，可以在程序中调用setrlimit（2）来修改进程的最大地址空间。

For example: 例如： 

// set maximum size of the virtual address space to 1GiB
struct rlimit rlim;
rlim_t max_mem = 1 << 30;
rlim.rlim_cur = max_mem;
setrlimit(RLIMIT_AS, &rlim);

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 Musl mmap ENOMEM - Musl mmap ENOMEM mmap（）在1TB ANONYMOUS文件中使用ENOMEM失败？ - mmap() fail with ENOMEM on a 1TB ANONYMOUS file? 为什么mmap（）在1TB稀疏文件上使用ENOMEM失败？ - Why does mmap() fail with ENOMEM on a 1TB sparse file? mmap返回“没有此类文件或目录” - mmap returning 'No such file or directory' mmap 函数总是返回 MAP_FAILED - mmap function always returning MAP_FAILED 使用未声明的标识符“ ENOMEM” - Use of undeclared identifier 'ENOMEM' 消息队列ENOMEM - message queue ENOMEM 何时将errno分配给ENOMEM？ - When should errno be assigned to ENOMEM? 尝试读取或写入旋风V FPGA上的PIO地址时，mmap（）不断返回MAP_FAILED - mmap() keeps returning MAP_FAILED whe trying to read or write to a PIO address on cyclone V FPGA Mmap和结构 - Mmap and structure
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM