Storing Per-Process Data in Kernel Module / Passing Data Between sys_enter and sys_exit Probe

Question

. 。 Answers to this question are eligible for a +100 reputation bounty. anroesti wants to to this question. 这个问题。

Familiarity with how Linux Kernel Tracepoints work is not necessarily required to help with this question, it is just what is motivating this problem. In essence, I am looking for a way to store per-process data for a kernel module, without modifying the Linux source (eg struct task_struct ), and ideally without using locks. Here is my specific question:

I have a kernel module that hooks into the sys_enter (defined here for x86_64 , aarch64 ) and sys_exit ( x86_64 , aarch64 ) tracepoints. For each system call issued, I need to pass some data between the enter probe and the exit probe.

Some things I have considered: I could...

• ...use one global variable -- but that will be shared between concurrently executing system calls on different CPUs, creating a race.
• ...use one global map from PID (of the process issuing the system call) to my data, together with locks -- but that will unnecessarily require synchronization between all CPUs on each system call . I would like to avoid this, since the data is "local" to each issued system call, so I feel like there should be a way to keep it local and not add costly synchronization.
• ...use a per-CPU global variable -- but (it is my understanding that) a process may move to another CPU during the system call execution, making this approach incorrect.
• ... kmalloc ing some memory for my custom data upon each system call entry, then pass the address to that memory by clobbering one of the registers in struct pt_regs (both the entry and exit probe receive a pointer to said struct) -- but then I will have a memory leak for system calls that do not trigger the exit probe (such as sys_exit , which never returns).

I am open to any suggestions how these ideas could be refined to address the problems I listed, or any completely different ideas that I am not thinking of.

Answer 1

I'd use an RCU enabled hashtable, for safety.

The first option isn't actually doable, as you stated.

The third one requires you to track which process is using which CPU, which seems unnecessary.

The leaking problem of the fourth option can probably be solved somehow, but allocating memory on each system call can introduce a serious delay. Of course that accessing the hashtable will also slow down the system, but It won't trigger a memory allocation for each system call, so I assume it'll be less harmful.

Also, I may be wrong here, but if you assume that only process creation/destruction will introduce changes to table itself (not to the data within each entry, but the location and hash value of each row) than maybe you won't even have to synchronize on each system call, but only on ones that will cause process creation/destruction.