How they do debugging Linux Kernel Core

Question

Now a days debugging become so advanced that even 'core kernel source code' can be debugged using Virtual environment.

But after reading couple of blog related to Kernel Core development it was not clear whether they are debugging using Virtual environment.

They have mentioned that they rely on 'Printing message' rather than using debugging tool, at-least for core component.

So, I Request from 'Linux Kernel Experts' to let me know what is good practice followed while debugging Kernel?

Answer 1

I've tried multiple approaches when trying to debug the kernel.

1. Sometimes, the easiest way is to just add a few printk statements based on my own conditional values, monitor the serial log and see what's going on. Its especially useful when the function in question is invoked quite often, but you are interested only in a subset of those.
2. QEMU GDB debugging. I have a buildroot filesystem setup. This means the kernel is lean and it boots up real fast. I start qemu with the -s -S flags, and attach gdb as target remote :1234 . Additionally, there aren't very many userspace processes in this setup so its easier to debug the kernel.
3. VMWare stub. Assuming you are running an Ubuntu VM, it is possible that you can attach gdb to a VMware stub and debug the kernel. Personally, I never have had to pursue this route, but I look forward to trying it out someday.
4. If you have a kernel for a device that gets stuck in a bootloop and it does not print out any debug information out onto serial, it still might be helpful to try and boot it up using QEMU. Sure, the booting up will probably fail as the kernel tries to load up drivers, but you should be able to attach gdb, get a stack trace and see what the root cause is(perhaps a recursive call).