Lecture
A watchdog timer, less commonly called a supervisory timer (literally «watchdog» timer), is a hardware-implemented scheme for monitoring system hangs. It is a timer that is periodically reset by the system being monitored. If no reset occurs within a certain interval of time, the system is forcibly restarted. In some cases the watchdog timer can send the system a signal to reboot (a «soft» reboot), while in others the reboot is performed in hardware (by shorting the RST signal line or something similar). In most cases there are special facilities that make it possible to determine the cause of the reset. For example, whether it was the first reset at power-on, a hardware reset triggered by a button or a signal, or whether the watchdog timer fired. In some processors the watchdog timer raises an interrupt rather than a general reset.
Automated systems that operate without a human operator are also subject to errors, hangs and other failures (including hardware ones); using watchdog timers increases their operational stability, since no manual reset is required. That is why their most common application is embedded systems of every kind.
During normal operation the computer regularly restarts the watchdog timer to prevent it from elapsing, or «timing out». If, due to a hardware fault or a program error, the computer fails to restart the watchdog timer, the timer will elapse and generate a timeout signal. The timeout signal is used to initiate corrective action. Corrective actions typically include placing the computer and its associated hardware in a safe state and invoking a computer reboot.
Microcontrollers often include an integrated, built-in watchdog timer. In other computers the watchdog timer may reside in a neighbouring chip that connects directly to the CPU , or it may be located on an external expansion card inside the computer chassis.
Physically, a watchdog timer can be:
When a watchdog timer is implemented as a standalone chip, for example in the ADM690 — ADM695 series, it may also perform supply voltage monitoring functions.
A watchdog timer integrated circuit (Texas Instruments TPS3823). One pin receives the timer restart signal (the «kick» [ a ] ) from the computer; another pin outputs the timeout signal.
Watchdog timers are essential in remote automated systems, such as this Mars exploration rover.
Watchdog timers are commonly found in embedded systems and other computer-controlled equipment where people cannot easily gain access to the hardware or would be unable to respond to faults in a timely manner. In such systems the computer cannot depend on a human to invoke a reboot if it hangs; it must be self-sufficient. For example, remote embedded systems such as space probes are physically inaccessible to human operators; they may become permanently disabled if they cannot recover from faults autonomously. In robots and other automated machinery, a fault in the controlling computer can lead to equipment damage or injury before a human is able to react, even if the computer is easily accessible. A watchdog timer is normally used in such cases.
Watchdog timers are also used to monitor and limit the execution time of software on a normally functioning computer. For example, a watchdog timer may be used when running untrusted code in a sandbox in order to limit the CPU time available to that code and thus prevent certain types of denial-of-service attacks. In real-time operating systems a watchdog timer may be used to monitor a time-critical task in order to guarantee that it completes within the maximum allotted time and, if it fails to do so, to terminate the task and report an error.
Some watchdog timers allow kicks only within a time window. Kicks that occur outside the window have no effect on the timer and may be treated as errors.
The process of restarting a watchdog timer is usually referred to as «kicking» [ a ] the watchdog. Kicking is normally done by writing to a watchdog control port or by setting a particular bit in a register . Alternatively, some tightly coupled [ b ] watchdog timers are kicked by executing a special machine-language instruction. An example of this is the CLRWDT (clear watchdog timer) instruction found in the instruction set of some PIC microcontrollers.
On computers running operating systems , the watchdog restart is usually invoked through a device driver . For example, in the Linux operating system a user-space program kicks the watchdog by communicating with the watchdog device driver, typically by writing a null character to /dev/watchdog or by calling the KEEPALIVE ioctl . The device driver, which serves to abstract the watchdog hardware away from user-space programs, may also be used to configure the timeout period and to start and stop the timer.
Some watchdog timers allow a kick only within a specific time window. The window timing is usually measured relative to the previous kick or, if the watchdog has not yet been kicked, relative to the moment the watchdog was enabled. The window opens after a delay following the previous kick and closes after a further delay. If the computer attempts to kick the watchdog before or after the window, the watchdog will not be restarted, and in some implementations this will be treated as an error and will trigger corrective action.
A screenshot of the wdctl program, which shows watchdog timer status
A watchdog timer is said to be enabled while it is running and disabled while it is idle. At power-on, the watchdog timer may be enabled unconditionally, or it may be initially disabled and require an external signal to enable it. In the latter case, the enable signal may be generated automatically by the hardware or it may be generated under software control.
When generated automatically, the enable signal is usually derived from the computer's reset signal. In some systems the reset signal is used directly to enable the watchdog timer. In other systems the reset signal is delayed so that the watchdog is enabled some time after the reset. This delay gives the computer time to boot before the watchdog is enabled. Without this delay, the watchdog would time out and cause a subsequent reset before the computer could start its application software — the software that kicks the watchdog — and the system would become stuck in an endless loop of incomplete reboots.
Watchdog timers come in a variety of configurations, and many of them allow their configurations to be changed. For example, the watchdog timer and the CPU may share a common clock signal , as shown in the block diagram below, or they may have independent clock signals. A basic watchdog timer has a single timer stage that, on timeout, normally resets the CPU:
Sometimes two or more timers are cascaded to form a multistage watchdog timer , where each timer is called a timer stage or simply a stage . For example, the block diagram below shows a three-stage watchdog circuit. In a multistage watchdog, the processor kicks only the first stage. When the first stage times out, a corrective action is initiated and the next stage in the cascade is started. When each subsequent stage times out, it triggers a corrective action and starts the next stage. When the final stage times out, a corrective action is initiated but no further stage is started, since the end of the cascade has been reached. Typically, single-stage watchdog timers are used simply to restart the computer, whereas multistage watchdog timers trigger a series of corrective actions in sequence, with the final stage triggering a computer restart. [ 4 ]
Watchdog timers may have fixed or programmable time intervals. Some watchdog timers allow the time interval to be programmed by selecting from several selectable discrete values. In others, the interval can be programmed to arbitrary values. Watchdog time intervals are typically in the range from ten milliseconds to a minute or more. In a multistage watchdog timer, each timer may have its own unique time interval.
A watchdog timer may initiate any of several types of corrective action, including a maskable interrupt , a non-maskable interrupt , a hardware reset , activation of a fail-safe state, power cycling, or combinations of these. Depending on its architecture, the type of corrective action or actions that the watchdog can trigger may be fixed or programmable. Some computers (for example, PC-compatible ones) require a pulsed signal to invoke a hardware reset. In such cases the watchdog timer normally triggers the hardware reset by activating an internal or external pulse generator, which in turn produces the required reset pulses.
In embedded and control systems, watchdog timers are often used to activate fail-safe circuitry. When activated, the fail-safe circuitry drives all control outputs to a safe state (for example, switching off motors, heaters and high voltages ) in order to prevent injury and equipment damage while the fault persists. In a two-stage watchdog timer, the first timer is often used to activate the fail-safe outputs and to start the second timer stage; the second stage will reset the computer if the fault cannot be cleared before that timer elapses.
Sometimes watchdog timers are used to trigger the writing of system state information, which can be useful in recovering from a failure , or of debugging information (which can be useful in determining the cause of the failure) to a persistent medium. In such cases a second timer, which is started when the first timer times out, is normally used to reboot the computer later, after enough time has been allowed for the data write to complete. This gives time for the information to be saved, but guarantees that the computer will be rebooted even if the write process fails.
For example, the diagram above shows a plausible configuration of a two-stage watchdog timer. During normal operation the computer regularly kicks Stage1 to prevent a timeout. If the computer fails to kick Stage1 (for example, because of a hardware fault or a programming error), Stage1 will eventually time out. This event will start the Stage2 timer and simultaneously notify the computer (by means of a non-maskable interrupt) that a reset is imminent. Until Stage2 times out, the computer can attempt to write state information, debugging information, or both. In the final instance, the computer will be reset by the Stage2 timeout.
A watchdog timer provides automatic detection of catastrophic faults that prevent the computer from kicking it. However, computers often have other, less severe types of faults that do not prevent kicking but that nevertheless require watchdog supervision. To handle these faults, a computer system is normally designed so that its watchdog will be kicked only if the computer judges the system to be operating properly. The computer determines whether the system is operating properly by carrying out one or more fault-detection tests, and will kick the watchdog only if all the tests pass.
On computers running an operating system and multiple processes , a single simple test may be insufficient to guarantee correct operation, since it may fail to detect a latent failure condition and therefore allow the watchdog to be kicked even though a failure condition exists. For example, in the case of the Linux operating system, a user-space watchdog daemon may simply kick the watchdog periodically without performing any tests. As long as the daemon runs normally, the system will be protected against severe system failures such as a kernel panic . To detect less severe failures, the daemon can be configured to perform tests covering resource availability (for example, sufficient memory and file descriptors , reasonable CPU time), evidence of expected process activity (for example, running system daemons, the presence or updating of particular files), overheating and network activity, and it can also run system-specific test scripts or programs.
When a failed test is detected, the computer may attempt to perform a sequence of corrective actions under software control, culminating in a software-initiated reboot. If the software is unable to invoke a reboot, the watchdog timer will time out and cause a hardware reset. This is essentially a multistage watchdog in which the software provides the first and intermediate timer stages, and the hardware reset forms the final stage. For example, on a Linux system the watchdog daemon may attempt a soft restart, which may be preferable to a hardware reset because file systems will be safely unmounted and the error information will be logged. However, it is important to have the backstop provided by the hardware timer, since a soft restart may fail under a number of failure conditions.
Comments