Kernel config options for kgdb¶

To enableCONFIG_KGDB you should look underKernel hacking ‣ Kernel debugging and selectKGDB: kernel debugger.

While it is not a hard requirement that you have symbols in your vmlinuxfile, gdb tends not to be very useful without the symbolic data, so youwill want to turn onCONFIG_DEBUG_INFO which is calledCompile the kernel with debug info in the config menu.

It is advised, but not required, that you turn on theCONFIG_FRAME_POINTER kernel option which is calledCompilethe kernel with frame pointers in the config menu. This option inserts codeto into the compiled executable which saves the frame information inregisters or on the stack at different points which allows a debuggersuch as gdb to more accurately construct stack back traces whiledebugging the kernel.

If the architecture that you are using supports the kernel optionCONFIG_STRICT_KERNEL_RWX, you should consider turning it off. Thisoption will prevent the use of software breakpoints because it markscertain regions of the kernel’s memory space as read-only. If kgdbsupports it for the architecture you are using, you can use hardwarebreakpoints if you desire to run with theCONFIG_STRICT_KERNEL_RWXoption turned on, else you need to turn off this option.

Next you should choose one of more I/O drivers to interconnect debugginghost and debugged target. Early boot debugging requires a KGDB I/Odriver that supports early debugging and the driver must be built intothe kernel directly. Kgdb I/O driver configuration takes place viakernel or module parameters which you can learn more about in the in thesection that describes the parameter kgdboc.

Here is an example set of.config symbols to enable or disable for kgdb:

# CONFIG_STRICT_KERNEL_RWX is not setCONFIG_FRAME_POINTER=yCONFIG_KGDB=yCONFIG_KGDB_SERIAL_CONSOLE=y

Kernel config options for kdb¶

Kdb is quite a bit more complex than the simple gdbstub sitting on topof the kernel’s debug core. Kdb must implement a shell, and also addssome helper functions in other parts of the kernel, responsible forprinting out interesting data such as what you would see if you ranlsmod, orps. In order to build kdb into the kernel you follow thesame steps as you would for kgdb.

The main config option for kdb isCONFIG_KGDB_KDB which is calledKGDB_KDB: include kdb frontend for kgdb in the config menu.In theory you would have already also selected an I/O driver such as theCONFIG_KGDB_SERIAL_CONSOLE interface if you plan on using kdb on aserial port, when you were configuring kgdb.

If you want to use a PS/2-style keyboard with kdb, you would selectCONFIG_KDB_KEYBOARD which is calledKGDB_KDB: keyboard asinput device in the config menu. TheCONFIG_KDB_KEYBOARD option is notused for anything in the gdb interface to kgdb. TheCONFIG_KDB_KEYBOARDoption only works with kdb.

Here is an example set of.config symbols to enable/disable kdb:

# CONFIG_STRICT_KERNEL_RWX is not setCONFIG_FRAME_POINTER=yCONFIG_KGDB=yCONFIG_KGDB_SERIAL_CONSOLE=yCONFIG_KGDB_KDB=yCONFIG_KDB_KEYBOARD=y

Kernel parameter: kgdboc¶

The kgdboc driver was originally an abbreviation meant to stand for“kgdb over console”. Today it is the primary mechanism to configure howto communicate from gdb to kgdb as well as the devices you want to useto interact with the kdb shell.

For kgdb/gdb, kgdboc is designed to work with a single serial port. Itis intended to cover the circumstance where you want to use a serialconsole as your primary console as well as using it to perform kerneldebugging. It is also possible to use kgdb on a serial port which is notdesignated as a system console. Kgdboc may be configured as a kernelbuilt-in or a kernel loadable module. You can only make use ofkgdbwait and early debugging if you build kgdboc into the kernel asa built-in.

Optionally you can elect to activate kms (Kernel Mode Setting)integration. When you use kms with kgdboc and you have a video driverthat has atomic mode setting hooks, it is possible to enter the debuggeron the graphics console. When the kernel execution is resumed, theprevious graphics mode will be restored. This integration can serve as auseful tool to aid in diagnosing crashes or doing analysis of memorywith kdb while allowing the full graphics console applications to run.

kgdboc=[kms][[,]kbd][[,]serial_device][,baud]

Kernel parameter:kgdboc_earlycon¶

If you specify the kernel parameterkgdboc_earlycon and your serialdriver registers a boot console that supports polling (doesn’t needinterrupts and implements a nonblocking read() function) kgdb will attemptto work using the boot console until it can transition to the regulartty driver specified by thekgdboc parameter.

Normally there is only one boot console (especially that implements theread() function) so just addingkgdboc_earlycon on its own issufficient to make this work. If you have more than one boot console youcan add the boot console’s name to differentiate. Note that names thatare registered through the boot console layer and the tty layer are notthe same for the same port.

For instance, on one board to be explicit you might do:

kgdboc_earlycon=qcom_geni kgdboc=ttyMSM0

If the only boot console on the device was “qcom_geni”, you could simplify:

kgdboc_earlycon kgdboc=ttyMSM0

Kernel parameter:kgdbwait¶

The Kernel command line optionkgdbwait makes kgdb wait for adebugger connection during booting of a kernel. You can only use thisoption if you compiled a kgdb I/O driver into the kernel and youspecified the I/O driver configuration as a kernel command line option.The kgdbwait parameter should always follow the configuration parameterfor the kgdb I/O driver in the kernel command line else the I/O driverwill not be configured prior to asking the kernel to use it to wait.

The kernel will stop and wait as early as the I/O driver andarchitecture allows when you use this option. If you build the kgdb I/Odriver as a loadable kernel module kgdbwait will not do anything.

Kernel parameter:kgdbcon¶

Thekgdbcon feature allows you to seeprintk() messages inside gdbwhile gdb is connected to the kernel. Kdb does not make use of the kgdbconfeature.

Kgdb supports using the gdb serial protocol to send console messages tothe debugger when the debugger is connected and running. There are twoways to activate this feature.

1. Activate with the kernel command line option:

   kgdbcon

2. Use sysfs before configuring an I/O driver:

   echo 1 > /sys/module/kgdb/parameters/kgdb_use_con

console=ttyS0,115200 kgdboc=ttyS0 kgdbcon

It is possible to use this option with kgdboc on a tty that is not asystem console.

Run time parameter:kgdbreboot¶

The kgdbreboot feature allows you to change how the debugger deals withthe reboot notification. You have 3 choices for the behavior. Thedefault behavior is always set to 0.

Kernel parameter:nokaslr¶

If the architecture that you are using enable KASLR by default,you should consider turning it off. KASLR randomizes thevirtual address where the kernel image is mapped and confusegdb which resolve kernel symbol address from symbol tableof vmlinux.

Quick start for kdb on a serial port¶

This is a quick example of how to use kdb.

1. Configure kgdboc at boot using kernel parameters:

   console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr

   OR

   Configure kgdboc after the kernel has booted; assuming you are usinga serial port console:

   echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc

2. Enter the kernel debugger manually or by waiting for an oops orfault. There are several ways you can enter the kernel debuggermanually; all involve using theSysRq-G, which means you must haveenabledCONFIG_MAGIC_SysRq=y in your kernel config.

   • When logged in as root or with a super user session you can run:

     echo g > /proc/sysrq-trigger

   • Example using minicom 2.2

     Press:CTRL-Afg

   • When you have telneted to a terminal server that supports sendinga remote break

     Press:CTRL-]

     Type in:sendbreak

     Press:Enterg

3. From the kdb prompt you can run thehelp command to see a completelist of the commands that are available.

   Some useful commands in kdb include:

   lsmod	Shows where kernel modules are loaded
   ps	Displays only the active processes
   psA	Shows all the processes
   summary	Shows kernel version info and memory usage
   bt	Get a backtrace of the current process usingdump_stack()
   dmesg	View the kernel syslog buffer
   go	Continue the system

4. When you are done using kdb you need to consider rebooting the systemor using thego command to resuming normal kernel execution. If youhave paused the kernel for a lengthy period of time, applicationsthat rely on timely networking or anything to do with real wall clocktime could be adversely affected, so you should take this intoconsideration when using the kernel debugger.

Quick start for kdb using a keyboard connected console¶

This is a quick example of how to use kdb with a keyboard.

1. Configure kgdboc at boot using kernel parameters:

   kgdboc=kbd

   OR

   Configure kgdboc after the kernel has booted:

   echo kbd > /sys/module/kgdboc/parameters/kgdboc

2. Enter the kernel debugger manually or by waiting for an oops orfault. There are several ways you can enter the kernel debuggermanually; all involve using theSysRq-G, which means you must haveenabledCONFIG_MAGIC_SysRq=y in your kernel config.

   • When logged in as root or with a super user session you can run:

     echo g > /proc/sysrq-trigger

   • Example using a laptop keyboard:

     Press and hold down:Alt

     Press and hold down:Fn

     Press and release the key with the label:SysRq

     Release:Fn

     Press and release:g

     Release:Alt

   • Example using a PS/2 101-key keyboard

     Press and hold down:Alt

     Press and release the key with the label:SysRq

     Press and release:g

     Release:Alt

3. Now type in a kdb command such ashelp,dmesg,bt orgo tocontinue kernel execution.

Connecting with gdb to a serial port¶

1. Configure kgdboc

   Configure kgdboc at boot using kernel parameters:

   kgdboc=ttyS0,115200

   OR

   Configure kgdboc after the kernel has booted:

   echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc

2. Stop kernel execution (break into the debugger)

   In order to connect to gdb via kgdboc, the kernel must first bestopped. There are several ways to stop the kernel which includeusing kgdbwait as a boot argument, via aSysRq-G, or running thekernel until it takes an exception where it waits for the debugger toattach.

   • When logged in as root or with a super user session you can run:

     echo g > /proc/sysrq-trigger

   • Example using minicom 2.2

     Press:CTRL-Afg

   • When you have telneted to a terminal server that supports sendinga remote break

     Press:CTRL-]

     Type in:sendbreak

     Press:Enterg

3. Connect from gdb

   Example (using a directly connected port):

   % gdb ./vmlinux(gdb) set remotebaud 115200(gdb) target remote /dev/ttyS0

   Example (kgdb to a terminal server on TCP port 2012):

   % gdb ./vmlinux(gdb) target remote 192.168.2.2:2012

   Once connected, you can debug a kernel the way you would debug anapplication program.

   If you are having problems connecting or something is going seriouslywrong while debugging, it will most often be the case that you wantto enable gdb to be verbose about its target communications. You dothis prior to issuing thetargetremote command by typing in:

   set debug remote 1

Remember if you continue in gdb, and need to “break in” again, you needto issue an otherSysRq-G. It is easy to create a simple entry point byputting a breakpoint atsys_sync and then you can runsync from ashell or script to break into the debugger.

Switching between kdb and kgdb¶

Running kdb commands from gdb¶

It is possible to run a limited set of kdb commands from gdb, using thegdb monitor command. You don’t want to execute any of the run control orbreakpoint operations, because it can disrupt the state of the kerneldebugger. You should be using gdb for breakpoints and run controloperations if you have gdb connected. The more useful commands to runare things like lsmod, dmesg, ps or possibly some of the memoryinformation commands. To see all the kdb commands you can runmonitorhelp.

Example:

(gdb) monitor ps1 idle process (state I) and27 sleeping system daemon (state M) processes suppressed,use 'ps A' to see all.Task Addr       Pid   Parent [*] cpu State Thread     Command0xc78291d0        1        0  0    0   S  0xc7829404  init0xc7954150      942        1  0    0   S  0xc7954384  dropbear0xc78789c0      944        1  0    0   S  0xc7878bf4  sh(gdb)

Architecture Specifics¶

The kernel debugger is organized into a number of components:

1. The debug core

   The debug core is found inkernel/debugger/debug_core.c. Itcontains:

   • A generic OS exception handler which includes sync’ing theprocessors into a stopped state on an multi-CPU system.

   • The API to talk to the kgdb I/O drivers

   • The API to make calls to the arch-specific kgdb implementation

   • The logic to perform safe memory reads and writes to memory whileusing the debugger

   • A full implementation for software breakpoints unless overriddenby the arch

   • The API to invoke either the kdb or kgdb frontend to the debugcore.

   • The structures and callback API for atomic kernel mode setting.

     Note

     kgdboc is where the kms callbacks are invoked.

2. kgdb arch-specific implementation

   This implementation is generally found inarch/*/kernel/kgdb.c. Asan example,arch/x86/kernel/kgdb.c contains the specifics toimplement HW breakpoint as well as the initialization to dynamicallyregister and unregister for the trap handlers on this architecture.The arch-specific portion implements:

   • contains an arch-specific trap catcher which invokeskgdb_handle_exception() to start kgdb about doing its work
   • translation to and from gdb specific packet format topt_regs
   • Registration and unregistration of architecture specific traphooks
   • Any special exception handling and cleanup
   • NMI exception handling and cleanup
   • (optional) HW breakpoints

3. gdbstub frontend (aka kgdb)

   The gdbstub is located inkernel/debug/gdbstub.c. It contains:

   • All the logic to implement the gdb serial protocol

4. kdb frontend

   The kdb debugger shell is broken down into a number of components.The kdb core is located in kernel/debug/kdb. There are a number ofhelper functions in some of the other kernel components to make itpossible for kdb to examine and report information about the kernelwithout taking locks that could cause a kernel deadlock. The kdb corecontains implements the following functionality.

   • A simple shell
   • The kdb core command set
   • A registration API to register additional kdb shell commands.
     • A good example of a self-contained kdb module is theftdumpcommand for dumping the ftrace buffer. See:kernel/trace/trace_kdb.c
     • For an example of how to dynamically register a new kdb commandyou can build the kdb_hello.ko kernel module fromsamples/kdb/kdb_hello.c. To build this example you can setCONFIG_SAMPLES=y andCONFIG_SAMPLE_KDB=m in your kernelconfig. Later runmodprobekdb_hello and the next time youenter the kdb shell, you can run thehello command.
   • The implementation forkdb_printf() which emits messages directlyto I/O drivers, bypassing the kernel log.
   • SW / HW breakpoint management for the kdb shell

5. kgdb I/O driver

   Each kgdb I/O driver has to provide an implementation for thefollowing:

   • configuration via built-in or module
   • dynamic configuration and kgdb hook registration calls
   • read and write character interface
   • A cleanup handler for unconfiguring from the kgdb core
   • (optional) Early debug methodology

   Any given kgdb I/O driver has to operate very closely with thehardware and must do it in such a way that does not enable interruptsor change other parts of the system context without completelyrestoring them. The kgdb core will repeatedly “poll” a kgdb I/Odriver for characters when it needs input. The I/O driver is expectedto return immediately if there is no data available. Doing so allowsfor the future possibility to touch watchdog hardware in such a wayas to have a target system not reset when these are enabled.

If you are intent on adding kgdb architecture specific support for a newarchitecture, the architecture should defineHAVE_ARCH_KGDB in thearchitecture specific Kconfig file. This will enable kgdb for thearchitecture, and at that point you must create an architecture specifickgdb implementation.

There are a few flags which must be set on every architecture in theirasm/kgdb.h file. These are:

• NUMREGBYTES:

  The size in bytes of all of the registers, so that wecan ensure they will all fit into a packet.

• BUFMAX:

  The size in bytes of the buffer GDB will read into. This mustbe larger than NUMREGBYTES.

• CACHE_FLUSH_IS_SAFE:

  Set to 1 if it is always safe to callflush_cache_range or flush_icache_range. On some architectures,these functions may not be safe to call on SMP since we keep otherCPUs in a holding pattern.

There are also the following functions for the common backend, found inkernel/kgdb.c, that must be supplied by the architecture-specificbackend unless marked as (optional), in which case a default functionmaybe used if the architecture does not need to provide a specificimplementation.

intkgdb_skipexception(int exception, struct pt_regs * regs)¶

(optional) exit kgdb_handle_exception early

Parameters

intexception

Exception vector number

structpt_regs*regs

Currentstructpt_regs.

On some architectures it is required to skip a breakpointexception when it occurs after a breakpoint has been removed.This can be implemented in the architecture specific portion of kgdb.

voidkgdb_breakpoint(void)¶

compiled in breakpoint

Parameters

void

no arguments

Description

This will be implemented as a static inline per architecture. Thisfunction is called by the kgdb core to execute an architecturespecific trap to cause kgdb to enter the exception processing.

intkgdb_arch_init(void)¶

Perform any architecture specific initalization.

Parameters

void

no arguments

Description

This function will handle the initalization of any architecturespecific callbacks.

voidkgdb_arch_exit(void)¶

Perform any architecture specific uninitalization.

Parameters

void

no arguments

Description

This function will handle the uninitalization of any architecturespecific callbacks, for dynamic registration and unregistration.

voidpt_regs_to_gdb_regs(unsigned long * gdb_regs, struct pt_regs * regs)¶

Convert ptrace regs to GDB regs

Parameters

unsignedlong*gdb_regs

A pointer to hold the registers in the order GDB wants.

structpt_regs*regs

Thestructpt_regs of the current process.

Convert the pt_regs inregs into the format for registers thatGDB expects, stored ingdb_regs.

voidsleeping_thread_to_gdb_regs(unsigned long * gdb_regs, struct task_struct * p)¶

Convert ptrace regs to GDB regs

Parameters

unsignedlong*gdb_regs

A pointer to hold the registers in the order GDB wants.

structtask_struct*p

Thestructtask_struct of the desired process.

Convert the register values of the sleeping process inp tothe format that GDB expects.This function is called when kgdb does not have access to thestructpt_regs and therefore it should fill the gdb registersgdb_regs with what has been saved instructthread_structthread field during switch_to.

voidgdb_regs_to_pt_regs(unsigned long * gdb_regs, struct pt_regs * regs)¶

Convert GDB regs to ptrace regs.

Parameters

unsignedlong*gdb_regs

A pointer to hold the registers we’ve received from GDB.

structpt_regs*regs

A pointer to astructpt_regs to hold these values in.

Convert the GDB regs ingdb_regs into the pt_regs, and store theminregs.

intkgdb_arch_handle_exception(int vector, int signo, int err_code, char * remcom_in_buffer, char * remcom_out_buffer, struct pt_regs * regs)¶

Handle architecture specific GDB packets.

Parameters

intvector

The error vector of the exception that happened.

intsigno

The signal number of the exception that happened.

interr_code

The error code of the exception that happened.

char*remcom_in_buffer

The buffer of the packet we have read.

char*remcom_out_buffer

The buffer ofBUFMAX bytes to write a packet into.

structpt_regs*regs

Thestructpt_regs of the current process.

This function MUST handle the ‘c’ and ‘s’ command packets,as well packets to set / remove a hardware breakpoint, if used.If there are additional packets which the hardware needs to handle,they are handled here. The code should return -1 if it wants toprocess more packets, and a0 or1 if it wants to exit from thekgdb callback.

voidkgdb_arch_handle_qxfer_pkt(char * remcom_in_buffer, char * remcom_out_buffer)¶

Handle architecture specific GDB XML packets.

Parameters

char*remcom_in_buffer

The buffer of the packet we have read.

char*remcom_out_buffer

The buffer ofBUFMAX bytes to write a packet into.

voidkgdb_call_nmi_hook(void * ignored)¶

Call kgdb_nmicallback() on the current CPU

Parameters

void*ignored

This parameter is only here to match the prototype.

If you’re using the default implementation ofkgdb_roundup_cpus()this function will be called per CPU. If you don’t implementkgdb_call_nmi_hook() a default will be used.

voidkgdb_roundup_cpus(void)¶

Get other CPUs into a holding pattern

Parameters

void

no arguments

Description

On SMP systems, we need to get the attention of the other CPUsand get them into a known state. This should do what is neededto get the other CPUs to call kgdb_wait(). Note that on some arches,the NMI approach is not used for rounding up all the CPUs. Normallythose architectures can just not implement this and get the default.

On non-SMP systems, this is not called.

voidkgdb_arch_set_pc(struct pt_regs * regs, unsigned long pc)¶

Generic call back to the program counter

Parameters

structpt_regs*regs

Currentstructpt_regs.

unsignedlongpc

The new value for the program counter

This function handles updating the program counter and requires anarchitecture specific implementation.

voidkgdb_arch_late(void)¶

Perform any architecture specific initalization.

Parameters

void

no arguments

Description

This function will handle the late initalization of anyarchitecture specific callbacks. This is an optional function forhandling things like late initialization of hw breakpoints. Thedefault implementation does nothing.

structkgdb_arch¶

Describe architecture specific values.

Definition

struct kgdb_arch {  unsigned char           gdb_bpt_instr[BREAK_INSTR_SIZE];  unsigned long           flags;  int (*set_breakpoint)(unsigned long, char *);  int (*remove_breakpoint)(unsigned long, char *);  int (*set_hw_breakpoint)(unsigned long, int, enum kgdb_bptype);  int (*remove_hw_breakpoint)(unsigned long, int, enum kgdb_bptype);  void (*disable_hw_break)(struct pt_regs *regs);  void (*remove_all_hw_break)(void);  void (*correct_hw_break)(void);  void (*enable_nmi)(bool on);};

Members

gdb_bpt_instr

The instruction to trigger a breakpoint.

flags

Flags for the breakpoint, currently justKGDB_HW_BREAKPOINT.

set_breakpoint

Allow an architecture to specify how to set a softwarebreakpoint.

remove_breakpoint

Allow an architecture to specify how to remove asoftware breakpoint.

set_hw_breakpoint

Allow an architecture to specify how to set a hardwarebreakpoint.

remove_hw_breakpoint

Allow an architecture to specify how to remove ahardware breakpoint.

disable_hw_break

Allow an architecture to specify how to disablehardware breakpoints for a single cpu.

remove_all_hw_break

Allow an architecture to specify how to remove allhardware breakpoints.

correct_hw_break

Allow an architecture to specify how to correct thehardware debug registers.

enable_nmi

Manage NMI-triggered entry to KGDB

structkgdb_io¶

Describe the interface for an I/O driver to talk with KGDB.

Definition

struct kgdb_io {  const char              *name;  int (*read_char) (void);  void (*write_char) (u8);  void (*flush) (void);  int (*init) (void);  void (*deinit) (void);  void (*pre_exception) (void);  void (*post_exception) (void);  struct console          *cons;};

Members

name

Name of the I/O driver.

read_char

Pointer to a function that will return one char.

write_char

Pointer to a function that will write one char.

flush

Pointer to a function that will flush any pending writes.

init

Pointer to a function that will initialize the device.

deinit

Pointer to a function that will deinit the device. Implies thatthis I/O driver is temporary and expects to be replaced. Called whenan I/O driver is replaced or explicitly unregistered.

pre_exception

Pointer to a function that will do any prep work forthe I/O driver.

post_exception

Pointer to a function that will do any cleanup workfor the I/O driver.

cons

valid if the I/O device is a console; else NULL.

kgdboc internals¶

#ifdef CONFIG_CONSOLE_POLL    .poll_get_char = serial8250_get_poll_char,    .poll_put_char = serial8250_put_poll_char,#endif

static const struct drm_crtc_helper_funcs intel_helper_funcs = {[...]        .mode_set_base_atomic = intel_pipe_set_base_atomic,[...]};

static struct fb_ops intelfb_ops = {[...]       .fb_debug_enter = drm_fb_helper_debug_enter,       .fb_debug_leave = drm_fb_helper_debug_leave,[...]};