sysrq This file controls whether the "kernel magic sysrq key" is enabled or not This feature may have to be explicitly turned on during

compilation. If /proc/sys/kernel/sysrq exists, the feature is available; otherwise, you’ll need to recompile your kernel before using it. It is recommended to have this feature enabled because it can help to diagnose some of the tricky system hangs and crashes.

The basic idea is that the kernel can be interrupted to display certain information by bypassing the rest of the operating system via the ALT-SysRq hotkey combination. In many cases where the machine seems to be hung, the ALT-SysRq key can still be used to gather kernel information for examination and/or forwarding to a distribution cause’s support area or other experts.

To enable this feature, do the following as root:

To test the kernel magic, switch to your first virtual console. You need not log in because the key combination triggers the kernel directly. Hold down the right ALT key, then press and hold the PrtSc/SysRq key, then press the number 5. You should see something similar to the following:

SysRq : Changing Loglevel Loglevel set to 5

If you do not see this message, it could be that the kernel is set to send messages to virtual console 10 by default. Press CTRL-ALT-F10 to switch to virtual console 10 and check to see if the messages appear there. If they do, then you know that the kernel magic is working properly. If you'd like to switch where the messages get sent by default, say, virtual console 1 by default instead of 10, then run this command as root:

/usr/sbin/klogmessage -r 1

This change will only be in effect until the next reboot, so to make the change permanent, grep through your system's startup scripts for "klogmessage" to determine where it gets set to virtual console 10 and change it to whichever virtual console you wish. For my SuSE Pro 9.0 system, this setting occurs in / etc/init.d/boot.klog.

Where the messages get sent is important to note because in the event your system hangs and kernel magic may be of use, you'll need to have your system already be on the virtual console where messages appear. This is because it is very likely that the kernel won't respond to the CTRL-ALT-Function keys to switch virtual consoles.

So what can you do with the kernel magic stuff then?

Press ALT-SysRq-h to see a Help screen. You should see the following:

SysRq : HELP : loglevel0-8 reBoot Crash Dumpregisters tErm kIll saK showMem showPc unRaw Sync showTasks Unmount

If you're seeing these messages you can gather this information to determine

the cause of the problem. Some of the commands such as showTasks will dump

a large amount of data, so it is highly recommended that a serial console be set up to gather and save this information. See the "Setting up a Serial Console" section for more information. Note however, that depending on the state of the kernel, the information may be saved to the /var/log/messages file as well so you may be able to retrieve it after a reboot.

The most important pieces of information to gather would be showPc,

showMem, showTasks. Output samples of these commands are shown here. Note

a bit of data is dumped. Dumpregisters is also valuable to have, but it requires

special configuration and is not enabled by default. After capturing this

information, it is advisable to execute the Sync and reBoot commands to properly

restart the system if an Oops or other kernel error was encountered. Simply using kernel magic at any given time is usually harmless and does not require

a Sync and or reBoot command to be performed.

3.4.2.7.1 showPc Output:

SysRq : Show Regs

Pid: 0, comm: swapper

EIP: 0010:[default_idle+36/48] CPU: 0 EFLAGS: 00003246 Tainted:

➥PF

EIP: 0010:[<c0106f94>] CPU: 0 EFLAGS: 00003246 Tainted: PF EAX: 00000000 EBX: c0106f70 ECX: 00000000 EDX: 00000019 ESI: c0326000 EDI: c0326000 EBP: ffffe000 DS: 0018 ES: 0018 CR0: 8005003b CR2: 4001a000 CR3: 05349000 CR4: 000006d0 Call Trace: [cpu_idle+50/96] [rest_init+0/32]

Call Trace: [<c0106ff2>] [<c0105000>]

3.4.2.7.2 showMem Output:

SysRq : Show Memory Mem-info:

Free pages: 15316kB ( 2044kB HighMem) Zone:DMA freepages: 3780kB

Zone:Normal freepages: 9492kB Zone:HighMem freepages: 2044kB

( Active: 128981, inactive: 102294, free: 3829 )

1*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 3780kB) 65*4kB 54*8kB 72*16kB 157*32kB 15*64kB 1*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 9492kB) 7*4kB 2*8kB 3*16kB 15*32kB 9*64kB 1*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 2044kB)

Swap cache: add 295, delete 32, find 0/0, race 0+0 Free swap: 1026940kB

262000 pages of RAM 32624 pages of HIGHMEM 3973 reserved pages 457950 pages shared 263 pages swap cached

33 pages in page table cache Buffer memory: 74760kB Cache memory: 778952kB

3.4.2.7.3 showTasks Output:

SysRq : Show Memory SysRq : Show State

free sibling task PC stack pid father child younger older init S CDFED120 236 1 0 4746

(NOTLB)

Call Trace: [schedule_timeout+99/176] [process_timeout+0/16]

➥[do_select+481/560] [__pollwait+0/208] [sys_select+80 8/1232] Call Trace: [<c0125923>] [<c01258b0>] [<c0154471>] [<c01540d0>] ➥[<c0154818>] [system_call+51/64] [<c0108dd3>] keventd S C4925AC0 0 2 1 3 ➥(L-TLB)

Call Trace: [context_thread+259/448] [rest_init+0/32]

➥[rest_init+0/32] [arch_kernel_thread+35/48] [context_thread+

➥0/448]

Call Trace: [<c0129d53>] [<c0105000>] [<c0105000>] [<c0107333>]

➥[<c0129c50>]

ksoftirqd_CPU S CDFED080 0 3 1 4 2

➥(L-TLB)

Call Trace: [rest_init+0/32] [ksoftirqd+183/192] [arch_kernel_thread+35/48] [ksoftirqd+0/192]

Call Trace: [<c0105000>] [<c0121e57>] [<c0107333>] [<c0121da0>] kswapd S C764E680 1260 4 1 5 3

➥(L-TLB)

Call Trace: [kswapd+171/176] [rest_init+0/32] [rest_init+0/32] [arch_kernel_thread+35/48] [kswapd+0/176]

Call Trace: [<c013c4ab>] [<c0105000>] [<c0105000>] [<c0107333>]

➥[<c013c400>]

bdflush S C02DEEE0 60 5 1 6 4

➥ (L-TLB)

Call Trace: [interruptible_sleep_on+61/96] [bdflush+195/208] [rest_init+0/32] [arch_kernel_thread+35/48] [bdflush+

0/208]

Call Trace: [<c011a24d>] [<c01490a3>] [<c0105000>] [<c0107333>]

➥[<c0148fe0>]

kupdated S C4925AC0 820 6 1 7 5

➥(L-TLB)

Call Trace: [schedule_timeout+99/176] [process_timeout+0/16] [kupdate+205/384] [rest_init+0/32] [rest_init+0/32]

Call Trace: [<c0125923>] [<c01258b0>] [<c014917d>] [<c0105000>]

➥[<c0105000>]

[arch_kernel_thread+35/48] [kupdate+0/384] [<c0107333>] [<c01490b0>]

3.4.2.8 tainted This file gives an indication of whether or not the kernel