Mapped file doesn't take memory on read, but does on write

Question

I've observed a weird behavior, where a large memory mapped file(2GB), doesn't take actual physical RAM when i read from it, but does when i start writing to it.

int err = 0;

int fd = open("large_file", O_RDWR);
if (fd == -1) {
    return errno;
}

void *map = mmap(NULL, size, PROT_READ | PROT_WRITE, 
                 MAP_PRIVATE, fd, 0); 
if (map == MAP_FAILED) {
     err = errno;
     goto out;
}

for (unsigned int i = 0; i < 2 << 30; i += PAGE_SIZE) {
      volatile uint8_t val = 0xff;

      // val = ((volatile uint8_t *) map)[i]; /* Doesn't take actual memory */
      // ((volatile uint8_t *) map[i]) = val; /* Takes actual memory */
}

while (1)
    sleep(10);

out:
   if (fd != -1)
      close(fd);
   if (map != MAP_FAILED)
      munmap(map, size);
return err;

This is htop screenshot when i READ from the mapped file(look at "check")

This is htop screenshot when i WRITE to the mapped file(look at "check"

My laptop is pretty much IDLE, nothing else can take physical RAM other than my test process, so you can see when i write to the mapped file it actually takes 2GB(copy-on-write), and when reading, it doesn't. RES-SHR also fits in both of my cases.

I can't explain it.. the kernel maps a new page regardless if it's a read or a write to physical RAM. If it wasn't present, it should be mapped and take actual RAM after a page fault exception. The mapped file is not mapped to other processes, so it isn't shared. The test process is the only one and the first one that maps the file into memory.

EDIT: I've added the (volatile) keyword to make sure the compiler doesn't optimize the critical code. This doesn't make any difference. This particular piece of assembly code is the output of reading from the memory mapped file:

mov    BYTE PTR [rbp-0xbd],0xff /* volatile uint8_t val = 0xff; */
mov    edx,DWORD PTR [rbp-0xbc] 
mov    rax,QWORD PTR [rbp-0xb0]
add    rax,rdx
movzx  eax,BYTE PTR [rax]     
mov    BYTE PTR [rbp-0xbd],al   /* val = ((volatile uint8_t *)map)[i]; */
add    DWORD PTR [rbp-0xbc],0x1000 /* i += PAGE_SIZE; */

Any thoughts?

Answer 1

Probably your compiler just optimizes the read accesses. It can do so, since you never do anything with it. Easiest would be to change the declaration of val :

uint8_t volatile val = ... ;

but you could also declare map accordingly:

uint8_t volatile*map = mmap(NULL, size, PROT_READ | PROT_WRITE, 
             MAP_PRIVATE, fd, 0);

(Declaring the sought type for map also avoids to have the nasty cast later on.)