Dev_tools

Nvim:help pages,generated fromsource using thetree-sitter-vimdoc parser.

Tools and techniques for developing Nvim

This is for developing or debugging Nvim itself.

Quickstart guide to developing Nvimdev-quickstart

You can start hacking on Nvim in less than 5 minutes:

1. Ensure you have the build prerequisites fromBUILD.md.2. Clone the source repo and "cd" into it:

git clone https://github.com/neovim/neovimcd neovim# (Optional) Build and run Nvim:makeVIMRUNTIME=./runtime ./build/bin/nvim --luamod-dev

3. Run a single test. We will start with "example_spec.lua", which is a real test that shows how tests are written:

make functionaltest TEST_FILE=test/functional/example_spec.lua

4. Notice thebefore_each block in the test file. Because it callsclear(), eachit() test will start a new Nvim instance.5. Tests will do stuff in the Nvim instance and make assertions usingeq(). Tests that want to check the UI can also usescreen:expect().6. Now make a code change in Nvim itself, then you can see the effects. The example test doesfeed('iline1…'), so let's make a change to the insert-mode code, which lives insrc/nvim/edit.c. In theinsert_handle_key function, just after thenormalchar label, add this code:

s->c = 'x';

7. Then run the "example_spec.lua" test again, and it should fail with something like this:

test/functional/example_spec.lua:31: Row 1 did not match.Expected:  |*line1               |  |*line^2               |  |{0:~                   }|  |{0:~                   }|  |                    |Actual:  |*xine1               |  |*xine^2               |  |{0:~                   }|  |{0:~                   }|  |                    |

You now understand how to modify the codebase, write tests, and run tests. Seedev-arch for details about the internal architecture.

Logsdev-tools-logs

Low-level log messages sink to$NVIM_LOG_FILE.

UI events are logged at DEBUG level.

rm -rf build/make CMAKE_EXTRA_FLAGS="-DLOG_DEBUG"

UseLOG_CALLSTACK() (Linux only) to log the current stacktrace. To log to analternate file (e.g. stderr) useLOG_CALLSTACK_TO_FILE(FILE*). Requires-no-pie ([ref](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860394#15)):

rm -rf build/make CMAKE_EXTRA_FLAGS="-DLOG_DEBUG -DCMAKE_C_FLAGS=-no-pie"

Many log messages have a shared prefix, such as "UI" or "RPC". Use the shell tofilter the log, e.g. at DEBUG level you might want to exclude UI messages:

tail -F ~/.local/state/nvim/log | cat -v | stdbuf -o0 grep -v UI | stdbuf -o0 tee -a log

Reproducible build

To make a reproducible build of Nvim, set cmake variableLUA_GEN_PRG toa LuaJIT binary built withLUAJIT_SECURITY_PRN=0. See commitcb757f2663e6950e655c6306d713338dfa66b18d.

Debug TUIdev-tools-tui

TUI INSPECT

Use the Ghosttyhttps://ghostty.org/ inspector tool to observe and query theoutput and events from any terminal application such as Nvim.

From the Ghostty inspector you can click the "Terminal IO" tab to get a trace.

TERMINFO LOGGING

At'verbose' level 3, Nvim logs its internal terminfo state, so you can seeexactly what terminfo values it is using on the current system.

nvim -V3log

TUI DEBUGGING WITH GDB LLDB

Launching the Nvim TUI involves two processes, one for main editor state and onefor rendering the TUI. Both of these processes use the nvim binary, so somewhatconfusingly setting a breakpoint in either will generally succeed but may not behit depending on which process the breakpoints were set in.

To debug the main process, you can debug the nvim binary with the--headlessflag which does not launch the TUI and will allow you to set breakpoints in codenot related to TUI rendering like so:

lldb -- ./build/bin/nvim --headless --listen ~/.cache/nvim/debug-server.pipe

While in lldb, enterrun. You can then attach to the headless process in anew terminal window to interact with the editor like so:

./build/bin/nvim --remote-ui --server ~/.cache/nvim/debug-server.pipe

Conversely for debugging TUI rendering, you can start a headless process anddebug the remote-ui process multiple times without losing editor state.

For details on using nvim-dap and automatically debugging the child (main)process, see [here](https://zignar.net/2023/02/17/debugging-neovim-with-neovim-and-nvim-dap/)

TUI REDRAW

For debugging Nvim TUI redraw behavior it is sometimes useful to slow down itsredraws. Set the'writedelay' and'redrawdebug' options to see where and whenthe UI is painted.

:set writedelay=50 rdb=compositor

Note: Nvim uses an internal screenbuffer to only send minimal updates even if a largeregion is repainted internally. To also highlight excess internal redraws, use

:set writedelay=50 rdb=compositor,nodelta

TUI TRACE

From the Ghostty inspector you can click the "Terminal IO" tab to get a trace.

Alternatively, the ancientscript command is the "state of the art". Thelibvtermvterm-dump utility formats the result for human-readability.

Record a Nvim terminal session and format it withvterm-dump:

script foo./build/bin/nvim -u NONE# Exit the script session with CTRL-d# Use `vterm-dump` utility to format the result../.deps/usr/bin/vterm-dump foo > bar

Then you can comparebar with another session, to debug TUI behavior.

TERMINAL REFERENCE

man terminfo

https://github.com/vim/vim/blob/0124320c97b0fbbb44613f42fc1c34fee6181fc8/src/libvterm/doc/seqs.txt

https://invisible-island.net/xterm/ctlseqs/ctlseqs.html

Debug Performancedev-tools-perf

PROFILING (EASY)

For debugging performance bottlenecks in any code, there is a simple (and veryeffective) approach:

1. Run the slow code in a loop.2. Break execution ~5 times and save the stacktrace.3. The cause of the bottleneck will (almost always) appear in most of the stacktraces.

PROFILING (FANCY)

For more advanced profiling, considerperf +flamegraph.

USDT PROFILING (POWERFUL)

Or you can use USDT probes viaNVIM_PROBE ([#12036](https://github.com/neovim/neovim/pull/12036)).

> USDT is basically a way to define stable probe points in userland binaries.> The benefit of bcc is the ability to define logic to go along with the probe> points.

Tools:

bpftrace provides an awk-like language to the kernel bytecode, BPF.

BCC provides a subset of C. Provides more complex logic than bpftrace, but takes a bit more effort.

Example using bpftrace to track slow vim functions, and print out any filesthat were opened during the trace. At the end, it prints a histogram offunction timing:

#!/usr/bin/env bpftraceBEGIN {  @depth = -1;}tracepoint:sched:sched_process_fork /@pidmap[args->parent_pid]/ {  @pidmap[args->child_pid] = 1;}tracepoint:sched:sched_process_exit /@pidmap[args->pid]/ {  delete(@pidmap[args->pid]);}usdt:build/bin/nvim:neovim:eval__call_func__entry {    @pidmap[pid] = 1;    @depth++;    @funcentry[@depth] = nsecs;}usdt:build/bin/nvim:neovim:eval__call_func__return {    $func = str(arg0);    $msecs = (nsecs - @funcentry[@depth]) / 1000000;    @time_histo = hist($msecs);    if ($msecs >= 1000) {      printf("%u ms for %s\n", $msecs, $func);      print(@files);    }    clear(@files);    delete(@funcentry[@depth]);    @depth--;}tracepoint:syscalls:sys_enter_open,tracepoint:syscalls:sys_enter_openat {  if (@pidmap[pid] == 1 && @depth >= 0) {    @files[str(args->filename)] = count();  }}END {  clear(@depth);}$ sudo bpftrace funcslower.bt1527 ms for Slower@files[/usr/lib/libstdc++.so.6]: 2@files[/etc/fish/config.fish]: 2<snip>^C@time_histo:[0]                71430 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|[1]                  346 |                                                    |[2, 4)               208 |                                                    |[4, 8)                91 |                                                    |[8, 16)               22 |                                                    |[16, 32)              85 |                                                    |[32, 64)               7 |                                                    |[64, 128)              0 |                                                    |[128, 256)             0 |                                                    |[256, 512)             6 |                                                    |[512, 1K)              1 |                                                    |[1K, 2K)               5 |                                                    |

Backtracesdev-tools-backtrace

LINUX

Core dumps are disabled by default on Ubuntu, CentOS and others.To enable core dumps:

ulimit -c unlimited

On systemd-based systems getting a backtrace is as easy as:

coredumpctl -1 gdb

coredumpctl is an optional tool, so you may need to install it:

sudo apt install systemd-coredump

The full backtrace is most useful; please send us thebacktrace.txt filewhen reporting a bug related to a crash:

2>&1 coredumpctl -1 gdb | tee -a backtrace.txt(gdb) thread apply all bt full

On systems withoutcoredumpctl, you may find acore dump file appearingin the current directory or in other locations. On Linux systems whereapport is installed (such as Ubuntu), the directory where core dump filesare saved can be/var/lib/apport/coredump or elsewhere, depending on thesystem configuration (see/proc/sys/kernel/core_pattern). See also:https://stackoverflow.com/a/18368068

To get a backtrace from the./core dump file:

gdb build/bin/nvim ./core 2>&1 | tee backtrace.txt(gdb) thread apply all bt full

MACOS

Ifnvim crashes, you can see the backtrace inConsole.app (under "CrashReports" or "User Diagnostic Reports" for older macOS versions).

open -a Console

You may also want to enable core dumps on macOS. To do this, first make surethe/cores/ directory exists and is writable:

sudo mkdir /coressudo chown root:admin /coressudo chmod 1775 /cores

Then set the core size limit tounlimited:

ulimit -c unlimited

Note that this is done per shell process. If you want to make this the defaultfor all shells, add the above line to your shell's init file (e.g.~/.bashrcor similar).

You can then open the core file inlldb:

lldb -c /cores/core.12345

Apple's documentation archive has some other useful informationhttps://developer.apple.com/library/archive/technotes/tn2124/_index.html#//apple_ref/doc/uid/DTS10003391-CH1-SECCOREDUMPS,but note that some of the things on this page are out of date (such as enablingcore dumps with/etc/launchd.conf).

WINDOWS

If the Windows version of Nvim crashes in a reproducible manner, you can takesome steps to provide a useful bug report.

You must obtain the debugger symbols (PDB) file for the Nvim executable: nvim.pdb.The PDB should be available from the same place that you obtained theexecutable (TODO: not currently provided by Nvim CI releases). Be sure to usethe PDB that matches the EXE (same build).

If you built the executable yourself with the Microsoft Visual C++ compiler,then the PDB was built with the EXE.

If you have Visual Studio, use that instead of the VC Toolkit and WinDbg.

For other compilers, always use the corresponding debugger: gdb or lldb.

Debugging Nvim crashes with Visual Studio 2005

First launch nvim.exe and then launch Visual Studio. (If you don't haveVisual Studio, get it fromhttps://visualstudio.microsoft.com/downloads/).

On the Tools menu, click Attach to Process. Choose the Nvim process.

In Nvim, reproduce the crash. A dialog will appear in Visual Studio, tellingyou about the unhandled exception in the Nvim process. Click Break to breakinto the process.

Visual Studio will pop up another dialog, telling you that no symbols areloaded and that the source code cannot be displayed. Click OK.

Several windows will open. Right-click in the Call Stack window. Choose LoadSymbols. The Find Symbols dialog will open, looking for (g)vim.pdb. Navigateto the directory where you have the PDB file and click Open.

At this point, you should have a full call stack with vim function names andline numbers. Double-click one of the lines and the Find Source dialog willnavigate to the directory where the Nvim source is (if you have it.)

If you don't know how to debug this any further, follow the instructionsat ":help bug-report". Paste the call stack into the bug report.

From Visual Studio you can also try saving a minidump via the Debug menu andsend it with the bug report. A minidump is a small file (<100KB), whichcontains information about the state of your process.

Gdbdev-tools-gdb

USING GDB TO STEP THROUGH FUNCTIONAL TESTS

UseTEST_TAG to run tests matching busted tags (of the form#foo e.g.it("test #foo ...", ...)):

GDB=1 TEST_TAG=foo make functionaltest

Then, in another terminal:

gdb build/bin/nvim(gdb) target remote localhost:7777

-- Seenvim_argv inhttps://github.com/neovim/neovim/blob/master/test/functional/testnvim.lua.

USING LLDB TO STEP THROUGH UNIT TESTS

lldb .deps/usr/bin/luajit -- .deps/usr/bin/busted --lpath="./build/?.lua" test/unit/

USING GDB

To attach to a runningnvim process with a pid of 1234 (Tip: the pid of arunning Nvim instance can be obtained by callinggetpid()), for instance:

gdb -tui -p 1234 build/bin/nvim

Thegdb interactive prompt will appear. At any time you can:

break foo to set a breakpoint on thefoo() function

n to step over the next statement

<Enter> to repeat the last command

s to step into the next statement

c to continue

finish to step out of the current function

p zub to print the value ofzub

bt to see a backtrace (callstack) from the current location

CTRL-x CTRL-a ortui enable to show a TUI view of the source file in the current debugging context. This can be extremely useful as it avoids the need for a gdb "frontend".

<up> and<down> to scroll the source file view

GDB REVERSE DEBUGGING

set record full insn-number-max unlimited

continue for a bit (at least untilmain() is executed

record

provoke the bug, then userevert-next,reverse-step, etc. to rewind the debugger

USING GDBSERVER

You may want to connect multiplegdb clients to the same runningnvimprocess, or you may want to connect to a remotenvim process with a localgdb. Usinggdbserver, you can attach to a single process and control itfrom multiplegdb clients.

Open a terminal and startgdbserver attached tonvim like this:

gdbserver :6666 build/bin/nvim 2> gdbserver.log

gdbserver is now listening on port 6666. You then need to attach to thisdebugging session in another terminal:

gdb build/bin/nvim

Once you've enteredgdb, you need to attach to the remote session:

(gdb) target remote localhost:6666

In case gdbserver puts the TUI as a background process, the TUI can becomeunable to read input from pty (and receives SIGTTIN signal) and/or output data(SIGTTOU signal). To force the TUI as the foreground process, you can add

signal (SIGTTOU, SIG_IGN);if (!tcsetpgrp(data->input.in_fd, getpid())) {    perror("tcsetpgrp failed");}

totui.c:terminfo_start.

USING GDBSERVER IN TMUX

Consider using a custom makefilehttps://github.com/neovim/neovim/blob/master/BUILD.md#custom-makefile toquickly start debugging sessions using thegdbserver method mentioned above.This examplelocal.mk will create the debugging session when you typemake debug.

.PHONY: dbg-start dbg-attach debug buildbuild:    @$(MAKE) nvimdbg-start: build    @tmux new-window -n 'dbg-neovim' 'gdbserver :6666 ./build/bin/nvim -D'dbg-attach:    @tmux new-window -n 'dbg-cgdb' 'cgdb -x gdb_start.sh ./build/bin/nvim'debug: dbg-start dbg-attach

Heregdb_start.sh includesgdb commands to be called when the debuggerstarts. It needs to attach to the server started by thedbg-start rule. Forexample:

(gdb) target remote localhost:6666(gdb) br main

Debugging crashes or memory leaksdev-tools-asan

BUILD WITH ASAN

Building Nvim with Clang sanitizers (Address Sanitizer: ASan, UndefinedBehavior Sanitizer: UBSan, Memory Sanitizer: MSan, Thread Sanitizer: TSan) isa good way to catch undefined behavior, leaks and other errors as soon as theyhappen. It's significantly faster than Valgrind.

Requires clang 3.4 or later, andllvm-symbolizer must be in$PATH:

clang --version

Build Nvim with sanitizer instrumentation (choose one):

CC=clang make CMAKE_EXTRA_FLAGS="-DENABLE_ASAN_UBSAN=ON"CC=clang make CMAKE_EXTRA_FLAGS="-DENABLE_MSAN=ON"CC=clang make CMAKE_EXTRA_FLAGS="-DENABLE_TSAN=ON"

Create a directory to store logs:

mkdir -p "$HOME/logs"

Configure the sanitizer(s) via these environment variables:

# Change to detect_leaks=1 to detect memory leaks (slower, noisier).export ASAN_OPTIONS="detect_leaks=0:log_path=$HOME/logs/asan"# Show backtraces in the logs.export MSAN_OPTIONS="log_path=${HOME}/logs/msan"export TSAN_OPTIONS="log_path=${HOME}/logs/tsan"

Logs will be written to${HOME}/logs/*san.PID then.

For more information:https://github.com/google/sanitizers/wiki/SanitizerCommonFlags

Main

Commands index

Quick reference

Quickstart guide to developing Nvim

TUI DEBUGGING WITH GDB LLDB