]>
Commit | Line | Data |
---|---|---|
7fb2e8a4 MCC |
1 | ================================================= |
2 | Using kgdb, kdb and the kernel debugger internals | |
3 | ================================================= | |
4 | ||
5 | :Author: Jason Wessel | |
6 | ||
7 | Introduction | |
8 | ============ | |
9 | ||
10 | The kernel has two different debugger front ends (kdb and kgdb) which | |
11 | interface to the debug core. It is possible to use either of the | |
12 | debugger front ends and dynamically transition between them if you | |
13 | configure the kernel properly at compile and runtime. | |
14 | ||
15 | Kdb is simplistic shell-style interface which you can use on a system | |
16 | console with a keyboard or serial console. You can use it to inspect | |
17 | memory, registers, process lists, dmesg, and even set breakpoints to | |
18 | stop in a certain location. Kdb is not a source level debugger, although | |
19 | you can set breakpoints and execute some basic kernel run control. Kdb | |
20 | is mainly aimed at doing some analysis to aid in development or | |
21 | diagnosing kernel problems. You can access some symbols by name in | |
22 | kernel built-ins or in kernel modules if the code was built with | |
23 | ``CONFIG_KALLSYMS``. | |
24 | ||
25 | Kgdb is intended to be used as a source level debugger for the Linux | |
26 | kernel. It is used along with gdb to debug a Linux kernel. The | |
27 | expectation is that gdb can be used to "break in" to the kernel to | |
28 | inspect memory, variables and look through call stack information | |
29 | similar to the way an application developer would use gdb to debug an | |
30 | application. It is possible to place breakpoints in kernel code and | |
31 | perform some limited execution stepping. | |
32 | ||
33 | Two machines are required for using kgdb. One of these machines is a | |
34 | development machine and the other is the target machine. The kernel to | |
35 | be debugged runs on the target machine. The development machine runs an | |
36 | instance of gdb against the vmlinux file which contains the symbols (not | |
37 | a boot image such as bzImage, zImage, uImage...). In gdb the developer | |
38 | specifies the connection parameters and connects to kgdb. The type of | |
39 | connection a developer makes with gdb depends on the availability of | |
40 | kgdb I/O modules compiled as built-ins or loadable kernel modules in the | |
41 | test machine's kernel. | |
42 | ||
43 | Compiling a kernel | |
44 | ================== | |
45 | ||
46 | - In order to enable compilation of kdb, you must first enable kgdb. | |
47 | ||
48 | - The kgdb test compile options are described in the kgdb test suite | |
49 | chapter. | |
50 | ||
51 | Kernel config options for kgdb | |
52 | ------------------------------ | |
53 | ||
821c6df8 MCC |
54 | To enable ``CONFIG_KGDB`` you should look under |
55 | :menuselection:`Kernel hacking --> Kernel debugging` and select | |
56 | :menuselection:`KGDB: kernel debugger`. | |
7fb2e8a4 MCC |
57 | |
58 | While it is not a hard requirement that you have symbols in your vmlinux | |
59 | file, gdb tends not to be very useful without the symbolic data, so you | |
821c6df8 MCC |
60 | will want to turn on ``CONFIG_DEBUG_INFO`` which is called |
61 | :menuselection:`Compile the kernel with debug info` in the config menu. | |
7fb2e8a4 MCC |
62 | |
63 | It is advised, but not required, that you turn on the | |
821c6df8 MCC |
64 | ``CONFIG_FRAME_POINTER`` kernel option which is called :menuselection:`Compile |
65 | the kernel with frame pointers` in the config menu. This option inserts code | |
7fb2e8a4 MCC |
66 | to into the compiled executable which saves the frame information in |
67 | registers or on the stack at different points which allows a debugger | |
68 | such as gdb to more accurately construct stack back traces while | |
69 | debugging the kernel. | |
70 | ||
71 | If the architecture that you are using supports the kernel option | |
821c6df8 | 72 | ``CONFIG_STRICT_KERNEL_RWX``, you should consider turning it off. This |
7fb2e8a4 MCC |
73 | option will prevent the use of software breakpoints because it marks |
74 | certain regions of the kernel's memory space as read-only. If kgdb | |
75 | supports it for the architecture you are using, you can use hardware | |
821c6df8 | 76 | breakpoints if you desire to run with the ``CONFIG_STRICT_KERNEL_RWX`` |
7fb2e8a4 MCC |
77 | option turned on, else you need to turn off this option. |
78 | ||
79 | Next you should choose one of more I/O drivers to interconnect debugging | |
80 | host and debugged target. Early boot debugging requires a KGDB I/O | |
81 | driver that supports early debugging and the driver must be built into | |
82 | the kernel directly. Kgdb I/O driver configuration takes place via | |
83 | kernel or module parameters which you can learn more about in the in the | |
821c6df8 | 84 | section that describes the parameter kgdboc. |
7fb2e8a4 | 85 | |
821c6df8 | 86 | Here is an example set of ``.config`` symbols to enable or disable for kgdb:: |
7fb2e8a4 | 87 | |
821c6df8 MCC |
88 | # CONFIG_STRICT_KERNEL_RWX is not set |
89 | CONFIG_FRAME_POINTER=y | |
90 | CONFIG_KGDB=y | |
91 | CONFIG_KGDB_SERIAL_CONSOLE=y | |
7fb2e8a4 MCC |
92 | |
93 | Kernel config options for kdb | |
94 | ----------------------------- | |
95 | ||
96 | Kdb is quite a bit more complex than the simple gdbstub sitting on top | |
97 | of the kernel's debug core. Kdb must implement a shell, and also adds | |
98 | some helper functions in other parts of the kernel, responsible for | |
99 | printing out interesting data such as what you would see if you ran | |
821c6df8 | 100 | ``lsmod``, or ``ps``. In order to build kdb into the kernel you follow the |
7fb2e8a4 MCC |
101 | same steps as you would for kgdb. |
102 | ||
103 | The main config option for kdb is ``CONFIG_KGDB_KDB`` which is called | |
821c6df8 MCC |
104 | :menuselection:`KGDB_KDB: include kdb frontend for kgdb` in the config menu. |
105 | In theory you would have already also selected an I/O driver such as the | |
106 | ``CONFIG_KGDB_SERIAL_CONSOLE`` interface if you plan on using kdb on a | |
7fb2e8a4 MCC |
107 | serial port, when you were configuring kgdb. |
108 | ||
109 | If you want to use a PS/2-style keyboard with kdb, you would select | |
821c6df8 MCC |
110 | ``CONFIG_KDB_KEYBOARD`` which is called :menuselection:`KGDB_KDB: keyboard as |
111 | input device` in the config menu. The ``CONFIG_KDB_KEYBOARD`` option is not | |
112 | used for anything in the gdb interface to kgdb. The ``CONFIG_KDB_KEYBOARD`` | |
7fb2e8a4 MCC |
113 | option only works with kdb. |
114 | ||
821c6df8 | 115 | Here is an example set of ``.config`` symbols to enable/disable kdb:: |
7fb2e8a4 | 116 | |
821c6df8 MCC |
117 | # CONFIG_STRICT_KERNEL_RWX is not set |
118 | CONFIG_FRAME_POINTER=y | |
119 | CONFIG_KGDB=y | |
120 | CONFIG_KGDB_SERIAL_CONSOLE=y | |
121 | CONFIG_KGDB_KDB=y | |
122 | CONFIG_KDB_KEYBOARD=y | |
7fb2e8a4 MCC |
123 | |
124 | Kernel Debugger Boot Arguments | |
125 | ============================== | |
126 | ||
127 | This section describes the various runtime kernel parameters that affect | |
128 | the configuration of the kernel debugger. The following chapter covers | |
129 | using kdb and kgdb as well as providing some examples of the | |
130 | configuration parameters. | |
131 | ||
132 | Kernel parameter: kgdboc | |
133 | ------------------------ | |
134 | ||
135 | The kgdboc driver was originally an abbreviation meant to stand for | |
136 | "kgdb over console". Today it is the primary mechanism to configure how | |
137 | to communicate from gdb to kgdb as well as the devices you want to use | |
138 | to interact with the kdb shell. | |
139 | ||
140 | For kgdb/gdb, kgdboc is designed to work with a single serial port. It | |
141 | is intended to cover the circumstance where you want to use a serial | |
142 | console as your primary console as well as using it to perform kernel | |
143 | debugging. It is also possible to use kgdb on a serial port which is not | |
144 | designated as a system console. Kgdboc may be configured as a kernel | |
145 | built-in or a kernel loadable module. You can only make use of | |
146 | ``kgdbwait`` and early debugging if you build kgdboc into the kernel as | |
147 | a built-in. | |
148 | ||
149 | Optionally you can elect to activate kms (Kernel Mode Setting) | |
150 | integration. When you use kms with kgdboc and you have a video driver | |
151 | that has atomic mode setting hooks, it is possible to enter the debugger | |
152 | on the graphics console. When the kernel execution is resumed, the | |
153 | previous graphics mode will be restored. This integration can serve as a | |
154 | useful tool to aid in diagnosing crashes or doing analysis of memory | |
155 | with kdb while allowing the full graphics console applications to run. | |
156 | ||
157 | kgdboc arguments | |
158 | ~~~~~~~~~~~~~~~~ | |
159 | ||
821c6df8 MCC |
160 | Usage:: |
161 | ||
162 | kgdboc=[kms][[,]kbd][[,]serial_device][,baud] | |
7fb2e8a4 MCC |
163 | |
164 | The order listed above must be observed if you use any of the optional | |
165 | configurations together. | |
166 | ||
167 | Abbreviations: | |
168 | ||
169 | - kms = Kernel Mode Setting | |
170 | ||
171 | - kbd = Keyboard | |
172 | ||
173 | You can configure kgdboc to use the keyboard, and/or a serial device | |
174 | depending on if you are using kdb and/or kgdb, in one of the following | |
175 | scenarios. The order listed above must be observed if you use any of the | |
176 | optional configurations together. Using kms + only gdb is generally not | |
177 | a useful combination. | |
178 | ||
179 | Using loadable module or built-in | |
180 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
181 | ||
182 | 1. As a kernel built-in: | |
183 | ||
821c6df8 MCC |
184 | Use the kernel boot argument:: |
185 | ||
186 | kgdboc=<tty-device>,[baud] | |
7fb2e8a4 MCC |
187 | |
188 | 2. As a kernel loadable module: | |
189 | ||
821c6df8 MCC |
190 | Use the command:: |
191 | ||
192 | modprobe kgdboc kgdboc=<tty-device>,[baud] | |
7fb2e8a4 MCC |
193 | |
194 | Here are two examples of how you might format the kgdboc string. The | |
195 | first is for an x86 target using the first serial port. The second | |
196 | example is for the ARM Versatile AB using the second serial port. | |
197 | ||
198 | 1. ``kgdboc=ttyS0,115200`` | |
199 | ||
200 | 2. ``kgdboc=ttyAMA1,115200`` | |
201 | ||
202 | Configure kgdboc at runtime with sysfs | |
203 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
204 | ||
205 | At run time you can enable or disable kgdboc by echoing a parameters | |
206 | into the sysfs. Here are two examples: | |
207 | ||
821c6df8 | 208 | 1. Enable kgdboc on ttyS0:: |
7fb2e8a4 | 209 | |
821c6df8 | 210 | echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc |
7fb2e8a4 | 211 | |
821c6df8 | 212 | 2. Disable kgdboc:: |
7fb2e8a4 | 213 | |
821c6df8 | 214 | echo "" > /sys/module/kgdboc/parameters/kgdboc |
7fb2e8a4 | 215 | |
821c6df8 MCC |
216 | .. note:: |
217 | ||
218 | You do not need to specify the baud if you are configuring the | |
219 | console on tty which is already configured or open. | |
7fb2e8a4 MCC |
220 | |
221 | More examples | |
222 | ^^^^^^^^^^^^^ | |
223 | ||
224 | You can configure kgdboc to use the keyboard, and/or a serial device | |
225 | depending on if you are using kdb and/or kgdb, in one of the following | |
226 | scenarios. | |
227 | ||
821c6df8 MCC |
228 | 1. kdb and kgdb over only a serial port:: |
229 | ||
230 | kgdboc=<serial_device>[,baud] | |
231 | ||
232 | Example:: | |
7fb2e8a4 | 233 | |
821c6df8 | 234 | kgdboc=ttyS0,115200 |
7fb2e8a4 | 235 | |
821c6df8 | 236 | 2. kdb and kgdb with keyboard and a serial port:: |
7fb2e8a4 | 237 | |
821c6df8 | 238 | kgdboc=kbd,<serial_device>[,baud] |
7fb2e8a4 | 239 | |
821c6df8 | 240 | Example:: |
7fb2e8a4 | 241 | |
821c6df8 | 242 | kgdboc=kbd,ttyS0,115200 |
7fb2e8a4 | 243 | |
821c6df8 | 244 | 3. kdb with a keyboard:: |
7fb2e8a4 | 245 | |
821c6df8 | 246 | kgdboc=kbd |
7fb2e8a4 | 247 | |
821c6df8 | 248 | 4. kdb with kernel mode setting:: |
7fb2e8a4 | 249 | |
821c6df8 | 250 | kgdboc=kms,kbd |
7fb2e8a4 | 251 | |
821c6df8 | 252 | 5. kdb with kernel mode setting and kgdb over a serial port:: |
7fb2e8a4 | 253 | |
821c6df8 | 254 | kgdboc=kms,kbd,ttyS0,115200 |
7fb2e8a4 | 255 | |
821c6df8 MCC |
256 | .. note:: |
257 | ||
258 | Kgdboc does not support interrupting the target via the gdb remote | |
259 | protocol. You must manually send a :kbd:`SysRq-G` unless you have a proxy | |
260 | that splits console output to a terminal program. A console proxy has a | |
261 | separate TCP port for the debugger and a separate TCP port for the | |
262 | "human" console. The proxy can take care of sending the :kbd:`SysRq-G` | |
263 | for you. | |
7fb2e8a4 MCC |
264 | |
265 | When using kgdboc with no debugger proxy, you can end up connecting the | |
266 | debugger at one of two entry points. If an exception occurs after you | |
267 | have loaded kgdboc, a message should print on the console stating it is | |
268 | waiting for the debugger. In this case you disconnect your terminal | |
269 | program and then connect the debugger in its place. If you want to | |
270 | interrupt the target system and forcibly enter a debug session you have | |
821c6df8 | 271 | to issue a :kbd:`Sysrq` sequence and then type the letter :kbd:`g`. Then you |
7fb2e8a4 | 272 | disconnect the terminal session and connect gdb. Your options if you |
821c6df8 | 273 | don't like this are to hack gdb to send the :kbd:`SysRq-G` for you as well as |
7fb2e8a4 MCC |
274 | on the initial connect, or to use a debugger proxy that allows an |
275 | unmodified gdb to do the debugging. | |
276 | ||
821c6df8 MCC |
277 | Kernel parameter: ``kgdbwait`` |
278 | ------------------------------ | |
7fb2e8a4 MCC |
279 | |
280 | The Kernel command line option ``kgdbwait`` makes kgdb wait for a | |
281 | debugger connection during booting of a kernel. You can only use this | |
282 | option if you compiled a kgdb I/O driver into the kernel and you | |
283 | specified the I/O driver configuration as a kernel command line option. | |
284 | The kgdbwait parameter should always follow the configuration parameter | |
285 | for the kgdb I/O driver in the kernel command line else the I/O driver | |
286 | will not be configured prior to asking the kernel to use it to wait. | |
287 | ||
288 | The kernel will stop and wait as early as the I/O driver and | |
289 | architecture allows when you use this option. If you build the kgdb I/O | |
290 | driver as a loadable kernel module kgdbwait will not do anything. | |
291 | ||
821c6df8 MCC |
292 | Kernel parameter: ``kgdbcon`` |
293 | ----------------------------- | |
7fb2e8a4 | 294 | |
821c6df8 MCC |
295 | The ``kgdbcon`` feature allows you to see :c:func:`printk` messages inside gdb |
296 | while gdb is connected to the kernel. Kdb does not make use of the kgdbcon | |
7fb2e8a4 MCC |
297 | feature. |
298 | ||
299 | Kgdb supports using the gdb serial protocol to send console messages to | |
300 | the debugger when the debugger is connected and running. There are two | |
301 | ways to activate this feature. | |
302 | ||
821c6df8 MCC |
303 | 1. Activate with the kernel command line option:: |
304 | ||
305 | kgdbcon | |
7fb2e8a4 | 306 | |
821c6df8 | 307 | 2. Use sysfs before configuring an I/O driver:: |
7fb2e8a4 | 308 | |
821c6df8 | 309 | echo 1 > /sys/module/kgdb/parameters/kgdb_use_con |
7fb2e8a4 | 310 | |
821c6df8 | 311 | .. note:: |
7fb2e8a4 | 312 | |
821c6df8 | 313 | If you do this after you configure the kgdb I/O driver, the |
7fb2e8a4 MCC |
314 | setting will not take effect until the next point the I/O is |
315 | reconfigured. | |
316 | ||
821c6df8 MCC |
317 | .. important:: |
318 | ||
319 | You cannot use kgdboc + kgdbcon on a tty that is an | |
320 | active system console. An example of incorrect usage is:: | |
321 | ||
322 | console=ttyS0,115200 kgdboc=ttyS0 kgdbcon | |
7fb2e8a4 MCC |
323 | |
324 | It is possible to use this option with kgdboc on a tty that is not a | |
325 | system console. | |
326 | ||
821c6df8 MCC |
327 | Run time parameter: ``kgdbreboot`` |
328 | ---------------------------------- | |
7fb2e8a4 MCC |
329 | |
330 | The kgdbreboot feature allows you to change how the debugger deals with | |
331 | the reboot notification. You have 3 choices for the behavior. The | |
332 | default behavior is always set to 0. | |
333 | ||
821c6df8 | 334 | .. tabularcolumns:: |p{0.4cm}|p{11.5cm}|p{5.6cm}| |
7fb2e8a4 | 335 | |
821c6df8 MCC |
336 | .. flat-table:: |
337 | :widths: 1 10 8 | |
7fb2e8a4 | 338 | |
821c6df8 MCC |
339 | * - 1 |
340 | - ``echo -1 > /sys/module/debug_core/parameters/kgdbreboot`` | |
341 | - Ignore the reboot notification entirely. | |
7fb2e8a4 | 342 | |
821c6df8 MCC |
343 | * - 2 |
344 | - ``echo 0 > /sys/module/debug_core/parameters/kgdbreboot`` | |
345 | - Send the detach message to any attached debugger client. | |
7fb2e8a4 | 346 | |
821c6df8 MCC |
347 | * - 3 |
348 | - ``echo 1 > /sys/module/debug_core/parameters/kgdbreboot`` | |
349 | - Enter the debugger on reboot notify. | |
7fb2e8a4 MCC |
350 | |
351 | Using kdb | |
352 | ========= | |
353 | ||
354 | Quick start for kdb on a serial port | |
355 | ------------------------------------ | |
356 | ||
357 | This is a quick example of how to use kdb. | |
358 | ||
821c6df8 | 359 | 1. Configure kgdboc at boot using kernel parameters:: |
7fb2e8a4 | 360 | |
821c6df8 | 361 | console=ttyS0,115200 kgdboc=ttyS0,115200 |
7fb2e8a4 MCC |
362 | |
363 | OR | |
364 | ||
365 | Configure kgdboc after the kernel has booted; assuming you are using | |
821c6df8 | 366 | a serial port console:: |
7fb2e8a4 | 367 | |
821c6df8 | 368 | echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc |
7fb2e8a4 MCC |
369 | |
370 | 2. Enter the kernel debugger manually or by waiting for an oops or | |
371 | fault. There are several ways you can enter the kernel debugger | |
821c6df8 MCC |
372 | manually; all involve using the :kbd:`SysRq-G`, which means you must have |
373 | enabled ``CONFIG_MAGIC_SysRq=y`` in your kernel config. | |
7fb2e8a4 | 374 | |
821c6df8 | 375 | - When logged in as root or with a super user session you can run:: |
7fb2e8a4 | 376 | |
821c6df8 | 377 | echo g > /proc/sysrq-trigger |
7fb2e8a4 MCC |
378 | |
379 | - Example using minicom 2.2 | |
380 | ||
821c6df8 | 381 | Press: :kbd:`CTRL-A` :kbd:`f` :kbd:`g` |
7fb2e8a4 MCC |
382 | |
383 | - When you have telneted to a terminal server that supports sending | |
384 | a remote break | |
385 | ||
821c6df8 | 386 | Press: :kbd:`CTRL-]` |
7fb2e8a4 | 387 | |
821c6df8 | 388 | Type in: ``send break`` |
7fb2e8a4 | 389 | |
821c6df8 | 390 | Press: :kbd:`Enter` :kbd:`g` |
7fb2e8a4 | 391 | |
821c6df8 | 392 | 3. From the kdb prompt you can run the ``help`` command to see a complete |
7fb2e8a4 MCC |
393 | list of the commands that are available. |
394 | ||
395 | Some useful commands in kdb include: | |
396 | ||
821c6df8 MCC |
397 | =========== ================================================================= |
398 | ``lsmod`` Shows where kernel modules are loaded | |
399 | ``ps`` Displays only the active processes | |
400 | ``ps A`` Shows all the processes | |
401 | ``summary`` Shows kernel version info and memory usage | |
402 | ``bt`` Get a backtrace of the current process using :c:func:`dump_stack` | |
403 | ``dmesg`` View the kernel syslog buffer | |
404 | ``go`` Continue the system | |
405 | =========== ================================================================= | |
7fb2e8a4 MCC |
406 | |
407 | 4. When you are done using kdb you need to consider rebooting the system | |
821c6df8 | 408 | or using the ``go`` command to resuming normal kernel execution. If you |
7fb2e8a4 MCC |
409 | have paused the kernel for a lengthy period of time, applications |
410 | that rely on timely networking or anything to do with real wall clock | |
411 | time could be adversely affected, so you should take this into | |
412 | consideration when using the kernel debugger. | |
413 | ||
414 | Quick start for kdb using a keyboard connected console | |
415 | ------------------------------------------------------ | |
416 | ||
417 | This is a quick example of how to use kdb with a keyboard. | |
418 | ||
821c6df8 | 419 | 1. Configure kgdboc at boot using kernel parameters:: |
7fb2e8a4 | 420 | |
821c6df8 | 421 | kgdboc=kbd |
7fb2e8a4 MCC |
422 | |
423 | OR | |
424 | ||
821c6df8 | 425 | Configure kgdboc after the kernel has booted:: |
7fb2e8a4 | 426 | |
821c6df8 | 427 | echo kbd > /sys/module/kgdboc/parameters/kgdboc |
7fb2e8a4 MCC |
428 | |
429 | 2. Enter the kernel debugger manually or by waiting for an oops or | |
430 | fault. There are several ways you can enter the kernel debugger | |
821c6df8 MCC |
431 | manually; all involve using the :kbd:`SysRq-G`, which means you must have |
432 | enabled ``CONFIG_MAGIC_SysRq=y`` in your kernel config. | |
7fb2e8a4 | 433 | |
821c6df8 | 434 | - When logged in as root or with a super user session you can run:: |
7fb2e8a4 | 435 | |
821c6df8 | 436 | echo g > /proc/sysrq-trigger |
7fb2e8a4 | 437 | |
821c6df8 | 438 | - Example using a laptop keyboard: |
7fb2e8a4 | 439 | |
821c6df8 | 440 | Press and hold down: :kbd:`Alt` |
7fb2e8a4 | 441 | |
821c6df8 | 442 | Press and hold down: :kbd:`Fn` |
7fb2e8a4 | 443 | |
821c6df8 | 444 | Press and release the key with the label: :kbd:`SysRq` |
7fb2e8a4 | 445 | |
821c6df8 | 446 | Release: :kbd:`Fn` |
7fb2e8a4 | 447 | |
821c6df8 | 448 | Press and release: :kbd:`g` |
7fb2e8a4 | 449 | |
821c6df8 | 450 | Release: :kbd:`Alt` |
7fb2e8a4 MCC |
451 | |
452 | - Example using a PS/2 101-key keyboard | |
453 | ||
821c6df8 | 454 | Press and hold down: :kbd:`Alt` |
7fb2e8a4 | 455 | |
821c6df8 | 456 | Press and release the key with the label: :kbd:`SysRq` |
7fb2e8a4 | 457 | |
821c6df8 | 458 | Press and release: :kbd:`g` |
7fb2e8a4 | 459 | |
821c6df8 | 460 | Release: :kbd:`Alt` |
7fb2e8a4 | 461 | |
821c6df8 | 462 | 3. Now type in a kdb command such as ``help``, ``dmesg``, ``bt`` or ``go`` to |
7fb2e8a4 MCC |
463 | continue kernel execution. |
464 | ||
465 | Using kgdb / gdb | |
466 | ================ | |
467 | ||
468 | In order to use kgdb you must activate it by passing configuration | |
469 | information to one of the kgdb I/O drivers. If you do not pass any | |
470 | configuration information kgdb will not do anything at all. Kgdb will | |
471 | only actively hook up to the kernel trap hooks if a kgdb I/O driver is | |
472 | loaded and configured. If you unconfigure a kgdb I/O driver, kgdb will | |
473 | unregister all the kernel hook points. | |
474 | ||
475 | All kgdb I/O drivers can be reconfigured at run time, if | |
476 | ``CONFIG_SYSFS`` and ``CONFIG_MODULES`` are enabled, by echo'ing a new | |
477 | config string to ``/sys/module/<driver>/parameter/<option>``. The driver | |
478 | can be unconfigured by passing an empty string. You cannot change the | |
479 | configuration while the debugger is attached. Make sure to detach the | |
480 | debugger with the ``detach`` command prior to trying to unconfigure a | |
481 | kgdb I/O driver. | |
482 | ||
483 | Connecting with gdb to a serial port | |
484 | ------------------------------------ | |
485 | ||
486 | 1. Configure kgdboc | |
487 | ||
821c6df8 | 488 | Configure kgdboc at boot using kernel parameters:: |
7fb2e8a4 | 489 | |
821c6df8 | 490 | kgdboc=ttyS0,115200 |
7fb2e8a4 MCC |
491 | |
492 | OR | |
493 | ||
821c6df8 | 494 | Configure kgdboc after the kernel has booted:: |
7fb2e8a4 | 495 | |
821c6df8 | 496 | echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc |
7fb2e8a4 MCC |
497 | |
498 | 2. Stop kernel execution (break into the debugger) | |
499 | ||
500 | In order to connect to gdb via kgdboc, the kernel must first be | |
501 | stopped. There are several ways to stop the kernel which include | |
821c6df8 | 502 | using kgdbwait as a boot argument, via a :kbd:`SysRq-G`, or running the |
7fb2e8a4 MCC |
503 | kernel until it takes an exception where it waits for the debugger to |
504 | attach. | |
505 | ||
821c6df8 | 506 | - When logged in as root or with a super user session you can run:: |
7fb2e8a4 | 507 | |
821c6df8 | 508 | echo g > /proc/sysrq-trigger |
7fb2e8a4 MCC |
509 | |
510 | - Example using minicom 2.2 | |
511 | ||
821c6df8 | 512 | Press: :kbd:`CTRL-A` :kbd:`f` :kbd:`g` |
7fb2e8a4 MCC |
513 | |
514 | - When you have telneted to a terminal server that supports sending | |
515 | a remote break | |
516 | ||
821c6df8 | 517 | Press: :kbd:`CTRL-]` |
7fb2e8a4 | 518 | |
821c6df8 | 519 | Type in: ``send break`` |
7fb2e8a4 | 520 | |
821c6df8 | 521 | Press: :kbd:`Enter` :kbd:`g` |
7fb2e8a4 MCC |
522 | |
523 | 3. Connect from gdb | |
524 | ||
821c6df8 | 525 | Example (using a directly connected port):: |
7fb2e8a4 MCC |
526 | |
527 | % gdb ./vmlinux | |
528 | (gdb) set remotebaud 115200 | |
529 | (gdb) target remote /dev/ttyS0 | |
530 | ||
531 | ||
821c6df8 | 532 | Example (kgdb to a terminal server on TCP port 2012):: |
7fb2e8a4 MCC |
533 | |
534 | % gdb ./vmlinux | |
535 | (gdb) target remote 192.168.2.2:2012 | |
536 | ||
537 | ||
538 | Once connected, you can debug a kernel the way you would debug an | |
539 | application program. | |
540 | ||
541 | If you are having problems connecting or something is going seriously | |
542 | wrong while debugging, it will most often be the case that you want | |
543 | to enable gdb to be verbose about its target communications. You do | |
821c6df8 MCC |
544 | this prior to issuing the ``target remote`` command by typing in:: |
545 | ||
546 | set debug remote 1 | |
7fb2e8a4 MCC |
547 | |
548 | Remember if you continue in gdb, and need to "break in" again, you need | |
821c6df8 MCC |
549 | to issue an other :kbd:`SysRq-G`. It is easy to create a simple entry point by |
550 | putting a breakpoint at ``sys_sync`` and then you can run ``sync`` from a | |
7fb2e8a4 MCC |
551 | shell or script to break into the debugger. |
552 | ||
553 | kgdb and kdb interoperability | |
554 | ============================= | |
555 | ||
556 | It is possible to transition between kdb and kgdb dynamically. The debug | |
557 | core will remember which you used the last time and automatically start | |
558 | in the same mode. | |
559 | ||
560 | Switching between kdb and kgdb | |
561 | ------------------------------ | |
562 | ||
563 | Switching from kgdb to kdb | |
564 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
565 | ||
566 | There are two ways to switch from kgdb to kdb: you can use gdb to issue | |
821c6df8 | 567 | a maintenance packet, or you can blindly type the command ``$3#33``. |
7fb2e8a4 MCC |
568 | Whenever the kernel debugger stops in kgdb mode it will print the |
569 | message ``KGDB or $3#33 for KDB``. It is important to note that you have | |
570 | to type the sequence correctly in one pass. You cannot type a backspace | |
571 | or delete because kgdb will interpret that as part of the debug stream. | |
572 | ||
821c6df8 | 573 | 1. Change from kgdb to kdb by blindly typing:: |
7fb2e8a4 | 574 | |
821c6df8 | 575 | $3#33 |
7fb2e8a4 | 576 | |
821c6df8 | 577 | 2. Change from kgdb to kdb with gdb:: |
7fb2e8a4 | 578 | |
821c6df8 | 579 | maintenance packet 3 |
7fb2e8a4 | 580 | |
821c6df8 MCC |
581 | .. note:: |
582 | ||
583 | Now you must kill gdb. Typically you press :kbd:`CTRL-Z` and issue | |
584 | the command:: | |
585 | ||
586 | kill -9 % | |
7fb2e8a4 MCC |
587 | |
588 | Change from kdb to kgdb | |
589 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
590 | ||
591 | There are two ways you can change from kdb to kgdb. You can manually | |
592 | enter kgdb mode by issuing the kgdb command from the kdb shell prompt, | |
593 | or you can connect gdb while the kdb shell prompt is active. The kdb | |
594 | shell looks for the typical first commands that gdb would issue with the | |
595 | gdb remote protocol and if it sees one of those commands it | |
596 | automatically changes into kgdb mode. | |
597 | ||
821c6df8 | 598 | 1. From kdb issue the command:: |
7fb2e8a4 | 599 | |
821c6df8 | 600 | kgdb |
7fb2e8a4 MCC |
601 | |
602 | Now disconnect your terminal program and connect gdb in its place | |
603 | ||
604 | 2. At the kdb prompt, disconnect the terminal program and connect gdb in | |
605 | its place. | |
606 | ||
607 | Running kdb commands from gdb | |
608 | ----------------------------- | |
609 | ||
610 | It is possible to run a limited set of kdb commands from gdb, using the | |
611 | gdb monitor command. You don't want to execute any of the run control or | |
612 | breakpoint operations, because it can disrupt the state of the kernel | |
613 | debugger. You should be using gdb for breakpoints and run control | |
614 | operations if you have gdb connected. The more useful commands to run | |
615 | are things like lsmod, dmesg, ps or possibly some of the memory | |
616 | information commands. To see all the kdb commands you can run | |
617 | ``monitor help``. | |
618 | ||
821c6df8 | 619 | Example:: |
7fb2e8a4 MCC |
620 | |
621 | (gdb) monitor ps | |
622 | 1 idle process (state I) and | |
623 | 27 sleeping system daemon (state M) processes suppressed, | |
624 | use 'ps A' to see all. | |
625 | Task Addr Pid Parent [*] cpu State Thread Command | |
626 | ||
627 | 0xc78291d0 1 0 0 0 S 0xc7829404 init | |
628 | 0xc7954150 942 1 0 0 S 0xc7954384 dropbear | |
629 | 0xc78789c0 944 1 0 0 S 0xc7878bf4 sh | |
630 | (gdb) | |
631 | ||
7fb2e8a4 MCC |
632 | kgdb Test Suite |
633 | =============== | |
634 | ||
635 | When kgdb is enabled in the kernel config you can also elect to enable | |
821c6df8 | 636 | the config parameter ``KGDB_TESTS``. Turning this on will enable a special |
7fb2e8a4 MCC |
637 | kgdb I/O module which is designed to test the kgdb internal functions. |
638 | ||
639 | The kgdb tests are mainly intended for developers to test the kgdb | |
640 | internals as well as a tool for developing a new kgdb architecture | |
641 | specific implementation. These tests are not really for end users of the | |
642 | Linux kernel. The primary source of documentation would be to look in | |
821c6df8 | 643 | the ``drivers/misc/kgdbts.c`` file. |
7fb2e8a4 MCC |
644 | |
645 | The kgdb test suite can also be configured at compile time to run the | |
646 | core set of tests by setting the kernel config parameter | |
821c6df8 | 647 | ``KGDB_TESTS_ON_BOOT``. This particular option is aimed at automated |
7fb2e8a4 MCC |
648 | regression testing and does not require modifying the kernel boot config |
649 | arguments. If this is turned on, the kgdb test suite can be disabled by | |
821c6df8 | 650 | specifying ``kgdbts=`` as a kernel boot argument. |
7fb2e8a4 MCC |
651 | |
652 | Kernel Debugger Internals | |
653 | ========================= | |
654 | ||
655 | Architecture Specifics | |
656 | ---------------------- | |
657 | ||
658 | The kernel debugger is organized into a number of components: | |
659 | ||
660 | 1. The debug core | |
661 | ||
821c6df8 | 662 | The debug core is found in ``kernel/debugger/debug_core.c``. It |
7fb2e8a4 MCC |
663 | contains: |
664 | ||
665 | - A generic OS exception handler which includes sync'ing the | |
666 | processors into a stopped state on an multi-CPU system. | |
667 | ||
668 | - The API to talk to the kgdb I/O drivers | |
669 | ||
670 | - The API to make calls to the arch-specific kgdb implementation | |
671 | ||
672 | - The logic to perform safe memory reads and writes to memory while | |
673 | using the debugger | |
674 | ||
675 | - A full implementation for software breakpoints unless overridden | |
676 | by the arch | |
677 | ||
678 | - The API to invoke either the kdb or kgdb frontend to the debug | |
679 | core. | |
680 | ||
681 | - The structures and callback API for atomic kernel mode setting. | |
682 | ||
821c6df8 | 683 | .. note:: kgdboc is where the kms callbacks are invoked. |
7fb2e8a4 MCC |
684 | |
685 | 2. kgdb arch-specific implementation | |
686 | ||
821c6df8 MCC |
687 | This implementation is generally found in ``arch/*/kernel/kgdb.c``. As |
688 | an example, ``arch/x86/kernel/kgdb.c`` contains the specifics to | |
7fb2e8a4 MCC |
689 | implement HW breakpoint as well as the initialization to dynamically |
690 | register and unregister for the trap handlers on this architecture. | |
691 | The arch-specific portion implements: | |
692 | ||
693 | - contains an arch-specific trap catcher which invokes | |
821c6df8 | 694 | :c:func:`kgdb_handle_exception` to start kgdb about doing its work |
7fb2e8a4 | 695 | |
821c6df8 | 696 | - translation to and from gdb specific packet format to :c:type:`pt_regs` |
7fb2e8a4 MCC |
697 | |
698 | - Registration and unregistration of architecture specific trap | |
699 | hooks | |
700 | ||
701 | - Any special exception handling and cleanup | |
702 | ||
703 | - NMI exception handling and cleanup | |
704 | ||
705 | - (optional) HW breakpoints | |
706 | ||
707 | 3. gdbstub frontend (aka kgdb) | |
708 | ||
821c6df8 | 709 | The gdbstub is located in ``kernel/debug/gdbstub.c``. It contains: |
7fb2e8a4 MCC |
710 | |
711 | - All the logic to implement the gdb serial protocol | |
712 | ||
713 | 4. kdb frontend | |
714 | ||
715 | The kdb debugger shell is broken down into a number of components. | |
716 | The kdb core is located in kernel/debug/kdb. There are a number of | |
717 | helper functions in some of the other kernel components to make it | |
718 | possible for kdb to examine and report information about the kernel | |
719 | without taking locks that could cause a kernel deadlock. The kdb core | |
720 | contains implements the following functionality. | |
721 | ||
722 | - A simple shell | |
723 | ||
724 | - The kdb core command set | |
725 | ||
726 | - A registration API to register additional kdb shell commands. | |
727 | ||
821c6df8 | 728 | - A good example of a self-contained kdb module is the ``ftdump`` |
7fb2e8a4 | 729 | command for dumping the ftrace buffer. See: |
821c6df8 | 730 | ``kernel/trace/trace_kdb.c`` |
7fb2e8a4 MCC |
731 | |
732 | - For an example of how to dynamically register a new kdb command | |
733 | you can build the kdb_hello.ko kernel module from | |
821c6df8 MCC |
734 | ``samples/kdb/kdb_hello.c``. To build this example you can set |
735 | ``CONFIG_SAMPLES=y`` and ``CONFIG_SAMPLE_KDB=m`` in your kernel | |
736 | config. Later run ``modprobe kdb_hello`` and the next time you | |
737 | enter the kdb shell, you can run the ``hello`` command. | |
7fb2e8a4 | 738 | |
821c6df8 | 739 | - The implementation for :c:func:`kdb_printf` which emits messages directly |
7fb2e8a4 MCC |
740 | to I/O drivers, bypassing the kernel log. |
741 | ||
742 | - SW / HW breakpoint management for the kdb shell | |
743 | ||
744 | 5. kgdb I/O driver | |
745 | ||
746 | Each kgdb I/O driver has to provide an implementation for the | |
747 | following: | |
748 | ||
749 | - configuration via built-in or module | |
750 | ||
751 | - dynamic configuration and kgdb hook registration calls | |
752 | ||
753 | - read and write character interface | |
754 | ||
755 | - A cleanup handler for unconfiguring from the kgdb core | |
756 | ||
757 | - (optional) Early debug methodology | |
758 | ||
759 | Any given kgdb I/O driver has to operate very closely with the | |
760 | hardware and must do it in such a way that does not enable interrupts | |
761 | or change other parts of the system context without completely | |
762 | restoring them. The kgdb core will repeatedly "poll" a kgdb I/O | |
763 | driver for characters when it needs input. The I/O driver is expected | |
764 | to return immediately if there is no data available. Doing so allows | |
765 | for the future possibility to touch watchdog hardware in such a way | |
766 | as to have a target system not reset when these are enabled. | |
767 | ||
768 | If you are intent on adding kgdb architecture specific support for a new | |
769 | architecture, the architecture should define ``HAVE_ARCH_KGDB`` in the | |
770 | architecture specific Kconfig file. This will enable kgdb for the | |
771 | architecture, and at that point you must create an architecture specific | |
772 | kgdb implementation. | |
773 | ||
774 | There are a few flags which must be set on every architecture in their | |
821c6df8 | 775 | ``asm/kgdb.h`` file. These are: |
7fb2e8a4 | 776 | |
821c6df8 MCC |
777 | - ``NUMREGBYTES``: |
778 | The size in bytes of all of the registers, so that we | |
779 | can ensure they will all fit into a packet. | |
7fb2e8a4 | 780 | |
821c6df8 MCC |
781 | - ``BUFMAX``: |
782 | The size in bytes of the buffer GDB will read into. This must | |
783 | be larger than NUMREGBYTES. | |
7fb2e8a4 | 784 | |
821c6df8 MCC |
785 | - ``CACHE_FLUSH_IS_SAFE``: |
786 | Set to 1 if it is always safe to call | |
787 | flush_cache_range or flush_icache_range. On some architectures, | |
788 | these functions may not be safe to call on SMP since we keep other | |
789 | CPUs in a holding pattern. | |
7fb2e8a4 MCC |
790 | |
791 | There are also the following functions for the common backend, found in | |
821c6df8 | 792 | ``kernel/kgdb.c``, that must be supplied by the architecture-specific |
7fb2e8a4 MCC |
793 | backend unless marked as (optional), in which case a default function |
794 | maybe used if the architecture does not need to provide a specific | |
795 | implementation. | |
796 | ||
797 | .. kernel-doc:: include/linux/kgdb.h | |
798 | :internal: | |
799 | ||
800 | kgdboc internals | |
801 | ---------------- | |
802 | ||
803 | kgdboc and uarts | |
804 | ~~~~~~~~~~~~~~~~ | |
805 | ||
806 | The kgdboc driver is actually a very thin driver that relies on the | |
807 | underlying low level to the hardware driver having "polling hooks" to | |
808 | which the tty driver is attached. In the initial implementation of | |
809 | kgdboc the serial_core was changed to expose a low level UART hook for | |
810 | doing polled mode reading and writing of a single character while in an | |
811 | atomic context. When kgdb makes an I/O request to the debugger, kgdboc | |
812 | invokes a callback in the serial core which in turn uses the callback in | |
813 | the UART driver. | |
814 | ||
815 | When using kgdboc with a UART, the UART driver must implement two | |
821c6df8 MCC |
816 | callbacks in the :c:type:`struct uart_ops <uart_ops>`. |
817 | Example from ``drivers/8250.c``:: | |
7fb2e8a4 | 818 | |
7fb2e8a4 MCC |
819 | |
820 | #ifdef CONFIG_CONSOLE_POLL | |
821 | .poll_get_char = serial8250_get_poll_char, | |
822 | .poll_put_char = serial8250_put_poll_char, | |
823 | #endif | |
824 | ||
825 | ||
826 | Any implementation specifics around creating a polling driver use the | |
827 | ``#ifdef CONFIG_CONSOLE_POLL``, as shown above. Keep in mind that | |
828 | polling hooks have to be implemented in such a way that they can be | |
829 | called from an atomic context and have to restore the state of the UART | |
830 | chip on return such that the system can return to normal when the | |
831 | debugger detaches. You need to be very careful with any kind of lock you | |
832 | consider, because failing here is most likely going to mean pressing the | |
833 | reset button. | |
834 | ||
835 | kgdboc and keyboards | |
821c6df8 | 836 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
7fb2e8a4 MCC |
837 | |
838 | The kgdboc driver contains logic to configure communications with an | |
839 | attached keyboard. The keyboard infrastructure is only compiled into the | |
821c6df8 | 840 | kernel when ``CONFIG_KDB_KEYBOARD=y`` is set in the kernel configuration. |
7fb2e8a4 MCC |
841 | |
842 | The core polled keyboard driver driver for PS/2 type keyboards is in | |
821c6df8 | 843 | ``drivers/char/kdb_keyboard.c``. This driver is hooked into the debug core |
7fb2e8a4 | 844 | when kgdboc populates the callback in the array called |
821c6df8 | 845 | :c:type:`kdb_poll_funcs[]`. The :c:func:`kdb_get_kbd_char` is the top-level |
7fb2e8a4 MCC |
846 | function which polls hardware for single character input. |
847 | ||
848 | kgdboc and kms | |
821c6df8 | 849 | ~~~~~~~~~~~~~~~~~~ |
7fb2e8a4 MCC |
850 | |
851 | The kgdboc driver contains logic to request the graphics display to | |
821c6df8 | 852 | switch to a text context when you are using ``kgdboc=kms,kbd``, provided |
7fb2e8a4 MCC |
853 | that you have a video driver which has a frame buffer console and atomic |
854 | kernel mode setting support. | |
855 | ||
856 | Every time the kernel debugger is entered it calls | |
821c6df8 MCC |
857 | :c:func:`kgdboc_pre_exp_handler` which in turn calls :c:func:`con_debug_enter` |
858 | in the virtual console layer. On resuming kernel execution, the kernel | |
859 | debugger calls :c:func:`kgdboc_post_exp_handler` which in turn calls | |
860 | :c:func:`con_debug_leave`. | |
7fb2e8a4 MCC |
861 | |
862 | Any video driver that wants to be compatible with the kernel debugger | |
821c6df8 MCC |
863 | and the atomic kms callbacks must implement the ``mode_set_base_atomic``, |
864 | ``fb_debug_enter`` and ``fb_debug_leave operations``. For the | |
865 | ``fb_debug_enter`` and ``fb_debug_leave`` the option exists to use the | |
7fb2e8a4 MCC |
866 | generic drm fb helper functions or implement something custom for the |
867 | hardware. The following example shows the initialization of the | |
868 | .mode_set_base_atomic operation in | |
821c6df8 | 869 | drivers/gpu/drm/i915/intel_display.c:: |
7fb2e8a4 | 870 | |
7fb2e8a4 MCC |
871 | |
872 | static const struct drm_crtc_helper_funcs intel_helper_funcs = { | |
873 | [...] | |
874 | .mode_set_base_atomic = intel_pipe_set_base_atomic, | |
875 | [...] | |
876 | }; | |
877 | ||
878 | ||
7fb2e8a4 MCC |
879 | Here is an example of how the i915 driver initializes the |
880 | fb_debug_enter and fb_debug_leave functions to use the generic drm | |
821c6df8 | 881 | helpers in ``drivers/gpu/drm/i915/intel_fb.c``:: |
7fb2e8a4 | 882 | |
7fb2e8a4 MCC |
883 | |
884 | static struct fb_ops intelfb_ops = { | |
885 | [...] | |
886 | .fb_debug_enter = drm_fb_helper_debug_enter, | |
887 | .fb_debug_leave = drm_fb_helper_debug_leave, | |
888 | [...] | |
889 | }; | |
890 | ||
891 | ||
7fb2e8a4 MCC |
892 | Credits |
893 | ======= | |
894 | ||
895 | The following people have contributed to this document: | |
896 | ||
821c6df8 | 897 | 1. Amit Kale <amitkale@linsyssoft.com> |
7fb2e8a4 | 898 | |
821c6df8 | 899 | 2. Tom Rini <trini@kernel.crashing.org> |
7fb2e8a4 MCC |
900 | |
901 | In March 2008 this document was completely rewritten by: | |
902 | ||
821c6df8 | 903 | - Jason Wessel <jason.wessel@windriver.com> |
7fb2e8a4 MCC |
904 | |
905 | In Jan 2010 this document was updated to include kdb. | |
906 | ||
821c6df8 | 907 | - Jason Wessel <jason.wessel@windriver.com> |