Debugging JIT-ed Code

Background

Without special runtime support, debugging dynamically generated code can bequite painful. Debuggers generally read debug information from object files ondisk, but for JITed code there is no such file to look for.

In order to hand over the necessary debug info,GDB established aninterfacefor registering JITed code with debuggers. LLDB implements it in theJITLoaderGDB plugin. On the JIT side, LLVM MCJIT does implement the interfacefor ELF object files.

At a high level, whenever MCJIT generates new machine code, it does so in anin-memory object file that contains the debug information in DWARF format.MCJIT then adds this in-memory object file to a global list of dynamicallygenerated object files and calls a special function__jit_debug_register_code that the debugger knows about. When the debuggerattaches to a process, it puts a breakpoint in this function and associates aspecial handler with it. Once MCJIT calls the registration function, thedebugger catches the breakpoint signal, loads the new object file from theinferior’s memory and resumes execution. This way it can obtain debuginformation for pure in-memory object files.

GDB Version

In order to debug code JIT-ed by LLVM, you need GDB 7.0 or newer, which isavailable on most modern distributions of Linux. The version of GDB thatApple ships with Xcode has been frozen at 6.3 for a while.

LLDB Version

Due to a regression in release 6.0, LLDB didn’t support JITed code debugging fora while. The bug was fixed in mainline recently, so that debugging JITed ELFobjects should be possible again from the upcoming release 12.0 on. On macOS thefeature must be enabled explicitly using theplugin.jit-loader.gdb.enablesetting.

Debugging MCJIT-ed code

The emerging MCJIT component of LLVM allows full debugging of JIT-ed code withGDB. This is due to MCJIT’s ability to use the MC emitter to provide fullDWARF debugging information to GDB.

Note that lli has to be passed the--jit-kind=mcjit flag to JIT the codewith MCJIT instead of the newer ORC JIT.

Example

Consider the following C code (with line numbers added to make the exampleeasier to follow):

1intcompute_factorial(intn)2{3if(n<=1)4return1;56intf=n;7while(--n>1)8f*=n;9returnf;10}111213intmain(intargc,char**argv)14{15if(argc<2)16return-1;17charfirstletter=argv[1][0];18intresult=compute_factorial(firstletter-'0');1920// Returned result is clipped at 255...21returnresult;22}

Here is a sample command line session that shows how to build and run thiscode vialli inside LLDB:

>exportBINPATH=/workspaces/llvm-project/build/bin>$BINPATH/clang-g-S-emit-llvm--target=x86_64-unknown-unknown-elfshowdebug.c>lldb$BINPATH/lli(lldb)targetcreate"/workspaces/llvm-project/build/bin/lli"Currentexecutablesetto'/workspaces/llvm-project/build/bin/lli'(x86_64).(lldb)settingssetplugin.jit-loader.gdb.enableon(lldb)bcompute_factorialBreakpoint1:nolocations(pending).WARNING:Unabletoresolvebreakpointtoanyactuallocations.(lldb)run--jit-kind=mcjitshowdebug.ll51locationaddedtobreakpoint1Process21340stopped*thread#1, name = 'lli', stop reason = breakpoint 1.1frame#0: 0x00007ffff7fd0007 JIT(0x45c2cb0)`compute_factorial(n=5) at showdebug.c:3:111intcompute_factorial(intn)2{->3if(n<=1)4return1;5intf=n;6while(--n>1)7f*=n;(lldb)pn(int)$0=5(lldb)bshowdebug.c:9Breakpoint2:where=JIT(0x45c2cb0)`compute_factorial+60atshowdebug.c:9:1,address=0x00007ffff7fd003c(lldb)cProcess21340resumingProcess21340stopped*thread#1, name = 'lli', stop reason = breakpoint 2.1frame#0: 0x00007ffff7fd003c JIT(0x45c2cb0)`compute_factorial(n=1) at showdebug.c:9:16while(--n>1)7f*=n;8returnf;->9}1011intmain(intargc,char**argv)12{(lldb)pf(int)$1=120(lldb)bt*thread#1, name = 'lli', stop reason = breakpoint 2.1*frame#0: 0x00007ffff7fd003c JIT(0x45c2cb0)`compute_factorial(n=1) at showdebug.c:9:1frame#1: 0x00007ffff7fd0095 JIT(0x45c2cb0)`main(argc=2, argv=0x00000000046122f0) at showdebug.c:16:18frame#2: 0x0000000002a8306e lli`llvm::MCJIT::runFunction(this=0x000000000458ed10, F=0x0000000004589ff8, ArgValues=ArrayRef<llvm::GenericValue> @ 0x00007fffffffc798) at MCJIT.cpp:554:31frame#3: 0x00000000029bdb45 lli`llvm::ExecutionEngine::runFunctionAsMain(this=0x000000000458ed10, Fn=0x0000000004589ff8, argv=size=0, envp=0x00007fffffffe140) at ExecutionEngine.cpp:467:10frame#4: 0x0000000001f2fc2f lli`main(argc=4, argv=0x00007fffffffe118, envp=0x00007fffffffe140) at lli.cpp:643:18frame#5: 0x00007ffff788c09b libc.so.6`__libc_start_main(main=(lli`main at lli.cpp:387), argc=4, argv=0x00007fffffffe118, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffe108) at libc-start.c:308:16frame#6: 0x0000000001f2dc7a lli`_start + 42(lldb)finishProcess21340stopped*thread#1, name = 'lli', stop reason = step outReturnvalue:(int)$2=120frame#0: 0x00007ffff7fd0095 JIT(0x45c2cb0)`main(argc=2, argv=0x00000000046122f0) at showdebug.c:16:913if(argc<2)14return-1;15charfirstletter=argv[1][0];->16intresult=compute_factorial(firstletter-'0');1718//Returnedresultisclippedat255...19returnresult;(lldb)presult(int)$3=73670648(lldb)nProcess21340stopped*thread#1, name = 'lli', stop reason = step overframe#0: 0x00007ffff7fd0098 JIT(0x45c2cb0)`main(argc=2, argv=0x00000000046122f0) at showdebug.c:19:1216intresult=compute_factorial(firstletter-'0');1718//Returnedresultisclippedat255...->19returnresult;20}(lldb)presult(int)$4=120(lldb)exprresult=42(int)$5=42(lldb)presult(int)$6=42(lldb)cProcess21340resumingProcess21340exitedwithstatus=42(0x0000002a)(lldb)exit