如何在看门狗执行之前将日志复制到闪存？

Question

How to execute some script (in my case it would script which copies logs to flash or copies logs remotely) before watchdog execution?

Should I modify linux kernel watchdog driver? If so in which method? Or maybe it is possible somehow to configure this by:

/etc/default/watchdog
/etc/watchdog.conf

However we have busybox installed where watchdog configuration is limited. I cannot find anything on google, what is suprised as this is basic problem which needs to be solved - everybody wants to have logs after watchdog reset in persistent memory, flash what is not /var/log/ path.

Of course solution to copy from time to time logs to flash in normal device lifecycle is not good idea as there should be some solution how to do this when watchdog timeout on feeding /dev/watchdog expires.

Answer 1

On a linux kernel newer than 4.9 you should have the availability of the pretimeout governor framework which would allow you to write a linux kernel driver which would react on the detection of a pre-timeout. A solution like this is well beyond the scope of a simple question and answer, so I'm leaving my original answer stand.

TL;DR:

If the problem is detectable while the OS is still running you can flush the logs. If the problem is caused by the OS locking up then you won't have an opportunity to fix the issue as hardware will reset the box.

There are two things here:

• Watchdog device
• Watchdog program

The watchdog device is typically a hardware timer that will do 'something specifically low level' when it's timer expires. The most common low level thing to do is reset the box. There is no OS involvement in this if it happens in hardware. You will have no opportunity to do anything high level once that timer runs out - eg writing log files somewhere.

The watchdog program is a tool that reassures the watchdog device periodically as long as it's check conditions are met.

The busybox watchdog timer's condition is a simple loop (pseudo code):

while (1) {
    # reassure watchdog
    # sleep some time
}

so if the program stops running - eg by an OS lockup or termination of the program then the underlying hardware will simply kick the box.

The 'bigger' watchdog binary provides a bunch of checks, and if they fail, then it will trigger the repair-binary option in the /etc/watchdog.conf to try to recover. This would be a potential point to flush the logs.