What is the SysRq facility and how do I use it?

Issue

Environment

Red Hat Enterprise Linux 3, 4, 5, and 6

Resolution

What is the "Magic" SysRq key?

According to the Linux kernel documentation:

It is a 'magical' key combo you can hit which the kernel will respond to regardless of whatever else it is doing, unless it is completely locked up.

The sysrq key is one of the best (and sometimes the only) way to determine what a machine is really doing. It is useful when a system appears to be "hung" or for diagnosing elusive, transient, kernel-related problems.

How do I enable and disable the SysRq key?

For security reasons, Red Hat Enterprise Linux disables the SysRq key by default. To enable it, run:

# echo 1 > /proc/sys/kernel/sysrq

To disable it:

# echo 0 > /proc/sys/kernel/sysrq

To enable it permanently, set the kernel.sysrq value in /etc/sysctl.conf to 1. That will cause it to be enabled on reboot.

# grep sysrq /etc/sysctl.conf
kernel.sysrq = 1

Since enabling sysrq gives someone with physical console access extra abilities, it is recommended to disable it when not troubleshooting a problem or to ensure that physical console access is properly secured.

How do I trigger a sysrq event?

There are several ways to trigger a sysrq event. On a normal system, with an AT keyboard, sysrq events can be triggered from the console with the following key combination:

Alt+PrintScreen+[CommandKey]

For instance, to tell the kernel to dump memory info (command key "m"), you would hold down the Alt and Print Screen keys, and then hit the m key.

Note that this will not work from an X Window System screen. You should first change to a text virtual terminal. Hit Ctrl+Alt+F1 to switch to the first virtual console prior to hitting the sysrq key combination.

On a serial console, you can achieve the same effect by sending a Breaksignal to the console and then hitting the command key within 5 seconds. This also works for virtual serial console access through an out-of-band service processor or remote console like HP iLO, Sun ILOM and IBM RSA. Refer to service processor specific documentation for details on how to send a Breaksignal; for example, How to trigger SysRq over an HP iLo Virtual Serial Port (VSP).

If you have a root shell on the machine (and the system is responding enough for you to do so), you can also write the command key character to the/proc/sysrq-trigger file. This is useful for triggering this info when you are not on the system console or for triggering it from scripts.

# echo 'm' > /proc/sysrq-trigger

When I trigger a sysrq event that generates output, where does it go?

When a sysrq command is triggered, the kernel will print out the information to the kernel ring buffer and to the system console. This information is normally logged via syslog to /var/log/messages.

Unfortunately, when dealing with machines that are extremely unresponsive, syslogd is often unable to log these events. In these situations, provisioning a serial console is often recommended for collecting the data.

What sort of sysrq events can be triggered?

There are several sysrq events that can be triggered once the sysrq facility is enabled. These vary somewhat between kernel versions, but there are a few that are commonly used: