How to submit an LLVM bug report

Introduction - Got bugs?

If you’re working with LLVM and run into a bug, we definitely want to knowabout it. This document describes what you can do to increase the odds ofgetting it fixed quickly.

🔒 If you believe that the bug is security related, please followHow to report a security issue?. 🔒

Basically you have to do two things at a minimum. First, decide whether thebugcrashes the compiler or if the compiler ismiscompiling the program(i.e., the compiler successfully produces an executable, but it doesn’t runright). Based on what type of bug it is, follow the instructions in thelinked section to narrow down the bug so that the person who fixes it will beable to find the problem more easily.

Once you have a reduced test-case, go tothe LLVM Bug Tracking System and fill out the form with thenecessary details (note that you don’t need to pick a label, just use if you’renot sure). The bug description should contain the following information:

  • All information necessary to reproduce the problem.

  • The reduced test-case that triggers the bug.

  • The location where you obtained LLVM (if not from our Gitrepository).

Thanks for helping us make LLVM better!

Crashing Bugs

More often than not, bugs in the compiler cause it to crash—often due toan assertion failure of some sort. The most important piece of the puzzleis to figure out if it is crashing in the Clang front-end or if it is one ofthe LLVM libraries (e.g. the optimizer or code generator) that hasproblems.

To figure out which component is crashing (the front-end, middle-endoptimizer, or backend code generator), run theclang command line as youwere when the crash occurred, but with the following extra command lineoptions:

  • -emit-llvm-Xclang-disable-llvm-passes: Ifclang still crashes whenpassed these options (which disable the optimizer and code generator), thenthe crash is in the front-end. Jump ahead tofront-end bugs.

  • -emit-llvm: Ifclang crashes with this option (which disablesthe code generator), you found a middle-end optimizer bug. Jump ahead tomiddle-end bugs.

  • Otherwise, you have a backend code generator crash. Jump ahead tocodegenerator bugs.

Front-end bugs

On aclang crash, the compiler will dump a preprocessed file and a scriptto replay theclang command. For example, you should see something like

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:Preprocessed source(s) and associated run script(s) are located at:clang: note: diagnostic msg: /tmp/foo-xxxxxx.cclang: note: diagnostic msg: /tmp/foo-xxxxxx.sh

Thecreduce tool helps toreduce the preprocessed file down to the smallest amount of code that stillreplicates the problem. You’re encouraged to use creduce to reduce the codeto make the developers’ lives easier. Theclang/utils/creduce-clang-crash.py script can be used on the filesthat clang dumps to help with automating creating a test to check for thecompiler crash.

cvise is an alternative tocreduce.

Middle-end optimization bugs

If you find that a bug crashes in the optimizer, compile your test-case to a.bc file by passing “-emit-llvm-O1-Xclang-disable-llvm-passes-c-ofoo.bc”. The-O1 is important because-O0 adds theoptnonefunction attribute to all functions and many passes don’t run onoptnonefunctions. Then run:

opt-O3foo.bc-disable-output

If this doesn’t crash, please follow the instructions for afront-endbug.

If this does crash, then you should be able to debug this with the followingbugpoint command:

bugpointfoo.bc-O3

Run this, then file a bug with the instructions and reduced .bcfiles that bugpoint emits.

If bugpoint doesn’t reproduce the crash,llvm-reduce is an alternativeway to reduce LLVM IR. Create a script that repros the crash and run:

llvm-reduce--test=path/to/scriptfoo.bc

which should produce reduced IR that reproduces the crash. Be warned thellvm-reduce is still fairly immature and may crash.

If none of the above work, you can get the IR before a crash by running theopt command with the--print-before-all--print-module-scope flags todump the IR before every pass. Be warned that this is very verbose.

Backend code generator bugs

If you find a bug that crashes clang in the code generator, compile yoursource file to a .bc file by passing “-emit-llvm-c-ofoo.bc” toclang (in addition to the options you already pass). Once your havefoo.bc, one of the following commands should fail:

  1. llcfoo.bc

  2. llcfoo.bc-relocation-model=pic

  3. llcfoo.bc-relocation-model=static

If none of these crash, please follow the instructions for afront-endbug. If one of these do crash, you should be able to reducethis with one of the followingbugpoint command lines (usethe one corresponding to the command above that failed):

  1. bugpoint-run-llcfoo.bc

  2. bugpoint-run-llcfoo.bc--tool-args-relocation-model=pic

  3. bugpoint-run-llcfoo.bc--tool-args-relocation-model=static

Please run this, then file a bug with the instructions and reduced .bc filethat bugpoint emits. If something goes wrong with bugpoint, please submitthe “foo.bc” file and the option that llc crashes with.

LTO bugs

If you encounter a bug that leads to crashes in the LLVM LTO phase when usingthe-flto option, follow these steps to diagnose and report the issue:

Compile your source file to a.bc (Bitcode) file with the following options,in addition to your existing compilation options:

exportCFLAGS="-flto -fuse-ld=lld"CXXFLAGS="-flto -fuse-ld=lld"LDFLAGS="-Wl,-plugin-opt=save-temps"

These options enable LTO and save temporary files generated during compilationfor later analysis.

On Windows, you should be using lld-link as the linker. Adjust your compilationflags as follows:* Add/lldsavetemps to the linker flags.* When linking from the compiler driver, add/link/lldsavetemps in order to forward that flag to the linker.

Using the specified flags will generate four intermediate bytecode files:

  1. a.out.0.0.preopt.bc (Before any link-time optimizations (LTO) are applied)

  2. a.out.0.2.internalize.bc (After initial optimizations are applied)

  3. a.out.0.4.opt.bc (After an extensive set of optimizations)

  4. a.out.0.5.precodegen.bc (After LTO but before translating into machine code)

Execute one of the following commands to identify the source of the problem:

  1. opt"-passes=lto<O3>"a.out.0.2.internalize.bc

  2. llca.out.0.5.precodegen.bc

If one of these do crash, you should be able to reducethis withllvm-reducecommand line (use the bc file corresponding to the command above that failed):

llvm-reduce--testreduce.sha.out.0.2.internalize.bc

Example of reduce.sh script

$catreduce.sh#!/bin/bash -epath/to/not--crashpath/to/opt"-passes=lto<O3>"$1-otemp.bc2>err.loggrep-q"It->second == &Insn"err.log

Here we have grepped the failed assert message.

Please run this, then file a bug with the instructions and reduced .bc filethat llvm-reduce emits.

Miscompilations

If clang successfully produces an executable, but that executable doesn’t runright, this is either a bug in the code or a bug in the compiler. The firstthing to check is to make sure it is not using undefined behavior (e.g.reading a variable before it is defined). In particular, check to see if theprogram is clean under varioussanitizers (e.g.clang-fsanitize=undefined,address) andvalgrind. Many“LLVM bugs” that we have chased down ended up being bugs in the program beingcompiled, not LLVM.

Once you determine that the program itself is not buggy, you should choosewhich code generator you wish to compile the program with (e.g. LLC or the JIT)and optionally a series of LLVM passes to run. For example:

bugpoint-run-llc[...optznpasses...]file-to-test.bc--args--[programarguments]

bugpoint will try to narrow down your list of passes to the one pass thatcauses an error, and simplify the bitcode file as much as it can to assistyou. It will print a message letting you know how to reproduce theresulting error.

TheOptBisect page shows an alternative method for findingincorrect optimization passes.

Incorrect code generation

Similarly to debugging incorrect compilation by mis-behaving passes, youcan debug incorrect code generation by either LLC or the JIT, usingbugpoint. The processbugpoint follows in this case is to try tonarrow the code down to a function that is miscompiled by one or the othermethod, but since for correctness, the entire program must be run,bugpoint will compile the code it deems to not be affected with the CBackend, and then link in the shared object it generates.

To debug the JIT:

bugpoint-run-jit-output=[correctoutputfile][bitcodefile]\--tool-args--[argumentstopasstolli]\--args--[programarguments]

Similarly, to debug the LLC, one would run:

bugpoint-run-llc-output=[correctoutputfile][bitcodefile]\--tool-args--[argumentstopasstollc]\--args--[programarguments]

Special note: if you are debugging MultiSource or SPEC tests thatalready exist in thellvm/test hierarchy, there is an easier way todebug the JIT, LLC, and CBE, using the pre-written Makefile targets, whichwill pass the program options specified in the Makefiles:

cdllvm/test/../../programmakebugpoint-jit

At the end of a successfulbugpoint run, you will be presentedwith two bitcode files: asafe file which can be compiled with the Cbackend and thetest file which either LLC or the JITmis-codegenerates, and thus causes the error.

To reproduce the error thatbugpoint found, it is sufficient to dothe following:

  1. Regenerate the shared object from the safe bitcode file:

    llc-march=csafe.bc-osafe.cgcc-sharedsafe.c-osafe.so

  2. If debugging LLC, compile test bitcode native and link with the sharedobject:

    llctest.bc-otest.sgcctest.ssafe.so-otest.llc./test.llc[programoptions]

  3. If debugging the JIT, load the shared object and supply the testbitcode:

    lli-load=safe.sotest.bc[programoptions]