How to monitor and log what module holds locks in linux kernel?

Question

Is it possible to monitor what module holds a lock in linux kernel?

I mean how can I know who locks a spin_lock , semaphore , mutex etc.

for user space:

Linux: How can I find the thread which holds a particular lock?

Answer 1

Normally, when a soft-lockup is detected, the kernel will print the current stack trace. For example:

INFO: task bdi-default:19 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
bdi-default   D 0000000000000001     0    19      2 0x00000000
ffff88013cd7fc60 0000000000000046 ffff88013cd7fc10 ffffffff810096f0
ffff88013806f4f8 0000000000000000 0000000000d7fc20 ffff880028313b40
ffff88013cd7b0b8 ffff88013cd7ffd8 000000000000f4e8 ffff88013cd7b0b8
Call Trace:
[<ffffffff810096f0>] ? __switch_to+0xd0/0x320
[<ffffffff814eca40>] ? thread_return+0x4e/0x77e
[<ffffffff814ed8c5>] schedule_timeout+0x215/0x2e0
[<ffffffff814ed543>] wait_for_common+0x123/0x180
[<ffffffff8105fa50>] ? default_wake_function+0x0/0x20
[<ffffffff814ed65d>] wait_for_completion+0x1d/0x20
[<ffffffff810909f9>] kthread_create+0x99/0x120
[<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100
[<ffffffff8100bc0e>] ? apic_timer_interrupt+0xe/0x20
[<ffffffff81134bb7>] bdi_forker_task+0x187/0x310
[<ffffffff81134a30>] ? bdi_forker_task+0x0/0x310
[<ffffffff81090886>] kthread+0x96/0xa0
[<ffffffff8100c14a>] child_rip+0xa/0x20
[<ffffffff810907f0>] ? kthread+0x0/0xa0
[<ffffffff8100c140>] ? child_rip+0x0/0x20

If the call trace doesn't provide enough information, try to enable lockup detection in the kernel (require kernel re-compilation)

 make menuconfig
         \--> Kernel Hacking
           \--> Detect Soft Lockups

Then the call trace will show more information starting with:

BUG: soft lockup detected on CPU#0!

To see the call-trace, check dmesg, or you better use serial console to catch all kernel log (in some cases, due to soft lockup the kernel may hang w/o spitting all the logs to the file, but with serial control - or other alternatives such as netconsole etc - you can catch the logs).