内存映射后，有缓存时，进程仍然消耗物理 memory

Question

I'm trying to understand mmap . As i know, mmap should map virtual address to page cache & thus there is no need to copy data from page cache to a process's virtual memory, and eventually there is a single copy of data in the whole machine.

However, when I try to mmap and read it, I can see the memory increases twice as file reading size, do I interpret it incorrectly or anything wrong about my code?

Memory consumption before testing:

$ free -m
               total        used        free      shared  buff/cache   available
Mem:            3924        1391        2280          13         251        2292
Swap:              0           0           0

I run below python code:

import mmap
import os
  
import time
# file2.db is a 2 GB file
with open("/var/tmp/file2.db", "r") as f:
  with mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ) as mm:
    x= mm.read(500000000)
      time.sleep(10000)

$ python3 mmap_read.py &

Memory consumption after testing:

$ free -m
               total        used        free      shared  buff/cache   available
Mem:            3924        1703        1575          13         644        1980
Swap:              0           0           0

I further check syscall used by process, looks like there is no data copy

$ sudo perf record python3 mmap_read.py & # record syscall
$ sudo perf report

Result

Samples: 128  of event 'cpu-clock:pppH', Event count (approx.): 1292929280
Overhead  Command  Shared Object      Symbol
  24.22%  python3  [kernel.kallsyms]  [k] do_user_addr_fault
   4.69%  python3  [kernel.kallsyms]  [k] rmqueue
   3.91%  python3  [kernel.kallsyms]  [k] __add_to_page_cache_locked
   3.91%  python3  [kernel.kallsyms]  [k] charge_memcg
   3.91%  python3  libc.so.6          [.] 0x00000000001a0e81
   3.12%  python3  [kernel.kallsyms]  [k] __lock_text_start
   3.12%  python3  [kernel.kallsyms]  [k] xas_load
   3.12%  python3  libc.so.6          [.] 0x00000000001a0ef0
   2.34%  python3  [kernel.kallsyms]  [k] __mod_lruvec_state
   2.34%  python3  [kernel.kallsyms]  [k] do_anonymous_page
   2.34%  python3  [kernel.kallsyms]  [k] free_unref_page_list
   2.34%  python3  [kernel.kallsyms]  [k] release_pages
   2.34%  python3  libc.so.6          [.] 0x00000000001a0e6f
   1.56%  python3  [kernel.kallsyms]  [k] __cgroup_throttle_swaprate
   1.56%  python3  [kernel.kallsyms]  [k] __mod_node_page_state
   1.56%  python3  [kernel.kallsyms]  [k] filemap_map_pages
   1.56%  python3  [kernel.kallsyms]  [k] pmd_page_vaddr
   1.56%  python3  [kernel.kallsyms]  [k] pmd_pfn
   1.56%  python3  [kernel.kallsyms]  [k] xa_get_order
   1.56%  python3  libc.so.6          [.] 0x00000000001a0e4c
   1.56%  python3  libc.so.6          [.] 0x00000000001a0e7d
   1.56%  python3  libc.so.6          [.] 0x00000000001a0e86
   1.56%  python3  libc.so.6          [.] 0x00000000001a0f47
   0.78%  python3  [kernel.kallsyms]  [k] __bio_add_page
   0.78%  python3  [kernel.kallsyms]  [k] __handle_mm_fault
   0.78%  python3  [kernel.kallsyms]  [k] __mem_cgroup_charge
   0.78%  python3  [kernel.kallsyms]  [k] __page_set_anon_rmap
   0.78%  python3  [kernel.kallsyms]  [k] arch_local_irq_enable
   0.78%  python3  [kernel.kallsyms]  [k] blk_mq_dispatch_rq_list
   0.78%  python3  [kernel.kallsyms]  [k] cgroup_rstat_updated
   0.78%  python3  [kernel.kallsyms]  [k] clear_page_erms
   0.78%  python3  [kernel.kallsyms]  [k] do_set_pte
   0.78%  python3  [kernel.kallsyms]  [k] elv_rqhash_add
   0.78%  python3  [kernel.kallsyms]  [k] finish_task_switch.isra.0
   0.78%  python3  [kernel.kallsyms]  [k] get_mem_cgroup_from_mm
   0.78%  python3  [kernel.kallsyms]  [k] handle_mm_fault
   0.78%  python3  [kernel.kallsyms]  [k] handle_pte_fault
   0.78%  python3  [kernel.kallsyms]  [k] kthread_blkcg
   0.78%  python3  [kernel.kallsyms]  [k] page_counter_try_charge
   0.78%  python3  [kernel.kallsyms]  [k] pmd_val
   0.78%  python3  [kernel.kallsyms]  [k] try_charge_memcg
   0.78%  python3  [kernel.kallsyms]  [k] xas_find
   0.78%  python3  [kernel.kallsyms]  [k] zap_pte_range
   0.78%  python3  libc.so.6          [.] 0x00000000001a0e5a
   0.78%  python3  libc.so.6          [.] 0x00000000001a0e76
   0.78%  python3  libc.so.6          [.] 0x00000000001a0f02
   0.78%  python3  libc.so.6          [.] 0x00000000001a0f07
   0.78%  python3  libc.so.6          [.] 0x00000000001a0f27
   0.78%  python3  libc.so.6          [.] 0x00000000001a0f57
   0.78%  python3  libc.so.6          [.] 0x00000000001a0f5f
   0.78%  python3  python3.10         [.] 0x000000000012161a
   0.78%  python3  python3.10         [.] 0x000000000012d084

I would expect the buff/cache grows & used is the same, as the process should reference the data in page cache, any idea on that? Any help is appreciated.

Answer 1

no need to copy data from page cache to a process's virtual memory,

However you explicitly do that here:

 x= mm.read(500000000)

Which uses 500000000 bytes of page cache and requires 500000000 of anonymous memory for bytes object referred to by the variable x .

To avoid copying the file you can use a memoryview object instead.

x = memoryview(mm)[:500000000]

Or alternatively use the mmap object directly.