Musl mmap ENOMEM

Question

I compiled some cross compilers against musl(x86_64, i686, arm). I need to compile code, that allocate like 2048 Mb +- 200Mb. However I noticed some errors with i686 musl compiler:

#include <stdio.h>
#include <sys/mman.h>
#include <sys/resource.h>
#include <errno.h>
int main(){
  void*ptr = mmap(0, 2147483647,PROT_READ|PROT_WRITE, 
            MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);       // malloc(2147483647)
  if(ptr == MAP_FAILED){
    printf("ERROR MMAP - "); 
    if(errno == ENOMEM){
      struct rlimit ra, rd; 
      if( getrlimit(RLIMIT_AS, &ra) < 0)
        printf("RLIMIT AS ERR\n");  
      if( getrlimit(RLIMIT_DATA, &rd) < 0)
        printf("RLIMIT AS ERR\n");   
      printf("ENOMEM, RLIMIT_AS: (%lu:%lu); RLIMIT_DATA: (%lu:%lu)\n", ra.rlim_cur, ra.rlim_max, rd.rlim_cur, rd.rlim_max); 
    }
  }
  else 
    printf("SUCCESS\n"); 
}

I know that there is no mistake with building i686-linux-musl. To prove it I, for example, downloaded i686-linux-musl from https://musl.cc/

I dont know why ldd tell me, that c is static, but file tell me it's dynamically linked.

However if I compile it with i686-linux-gnu (i586)

What's the problem with musl? EDITED. I used strace and got. i686-linux-musl:

execve("./c", ["./c"], 0x7ffe6a24f548 /* 61 vars */) = 0
strace: [ Process PID=1723318 runs in 32 bit mode. ]
set_thread_area({entry_number=-1, base_addr=0xf7ff75ac, limit=0x0fffff, seg_32bit=1, contents=0, read_exec_only=0, limit_in_pages=1, seg_not_present=0, useable=1}) = 0 (entry_number=12)
set_tid_address(0xf7ff7654)             = 1723318
prlimit64(0, RLIMIT_AS, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
prlimit64(0, RLIMIT_DATA, NULL, {rlim_cur=RLIM64_INFINITY, rlim_max=RLIM64_INFINITY}) = 0
ioctl(1, TIOCGWINSZ, {ws_row=40, ws_col=235, ws_xpixel=0, ws_ypixel=0}) = 0
writev(1, [{iov_base="ERROR MMAP - ENOMEM, RLIMIT_AS: "..., iov_len=92}, {iov_base=")\n", iov_len=2}], 2ERROR MMAP - ENOMEM, RLIMIT_AS: (4294967295:4294967295); RLIMIT_DATA: (4294967295:4294967295)
) = 94
exit_group(0)

Whereis mmap?? And with i586-linux-gnu:

execve("./c2", ["./c2"], 0x7fff3e161878 /* 61 vars */) = 0
strace: [ Process PID=1723806 runs in 32 bit mode. ]
ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
mmap(NULL, 2147483647, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x77f34000
write(1, "SUCCESS\n", 8SUCCESS
)                = 8
exit(8)                                 = ?
+++ exited with 8 +++

Answer 1

By looking at the source code of musl's mmap we see that it forces the size to be less than PTRDIFF_MAX , which is probably 2147483647, and returns the ENOMEM error if it's not.

Ask for 1 byte less, and it will actually try to mmap.

However, the memory fragmentation concern still applies. On a 32-bit architecture, allocating 2GB of memory may succeed, but it won't always succeed and you have to be prepared for the possibility that it may not. Many 32-bit operating systems limit you to a maximum of 2GB of address space - not 4GB - including the memory you allocate and your program itself. On those operating systems, you won't ever be able to allocate 2GB. I think 32-bit Windows and 32-bit Linux work this way.

Answer 2

What's happening there is that the address space is fragmented. While you may still have enough free memory and enough free address space for the process, there is simply not a 2GB long contiguous range of unused addresses. Therefore, the mmap call fails, as it should. The different libraries of course have different allocation patterns and will get loaded at different addresses (or even random addresses with ASLR), so the size of the largest free block of address space will vary (it can even be random on purpose with ASLR).

You have to allocate the memory in smaller non-contiguous chunks, whether you like it or not. Alternatively, if you compile for 64 bit, you get a much larger address space where it's no problem at all to fit in a 2GB allocation.