Sanitizer support

Sanitizers can be used in custom Julia builds to make it easier to detect certain kinds of errors in Julia's internal C/C++ code.

Address Sanitizer: easy build

From a source-checkout of Julia, you should be able to build a version supporting address sanitization in Julia and LLVM as follows:

$ mkdir /tmp/julia$ contrib/asan/build.sh /tmp/julia/

Here we've chosen/tmp/julia as a build directory, but you can choose whatever you wish. Once built, run the workload you wish to test with/tmp/julia/julia. Memory bugs will result in errors.

If you require customization or further detail, see the documentation below.

General considerations

Using Clang's sanitizers obviously requires you to use Clang (USECLANG=1), but there's another catch: most sanitizers require a run-time library, provided by the host compiler, while the instrumented code generated by Julia's JIT relies on functionality from that library. This implies that the LLVM version of your host compiler must match that of the LLVM library used within Julia.

An easy solution is to have a dedicated build folder for providing a matching toolchain, by building withBUILD_LLVM_CLANG=1. You can then refer to this toolchain from another build folder by specifyingUSECLANG=1 while overriding theCC andCXX variables.

The sanitizers error out when they detect a shared library being opened usingRTLD_DEEPBIND (ref:google/sanitizers#611). Sincelibblastrampoline by default usesRTLD_DEEPBIND, we need to set the environment variableLBT_USE_RTLD_DEEPBIND=0 when using a sanitizer.

To use one of of the sanitizers setSANITIZE=1 and then the appropriate flag for the sanitizer you want to use.

On macOS, this might need some extra flags also to work. Altogether, it might look like this, plus one or more of theSANITIZE_* flags listed below:

make -C deps USE_BINARYBUILDER_LLVM=0 LLVM_VER=svn stage-llvmmake -C src SANITIZE=1 USECLANG=1 \    CC=~+/deps/scratch/llvm-svn/build_Release/bin/clang \    CXX=~+/deps/scratch/llvm-svn/build_Release/bin/clang++ \    CPPFLAGS="-isysroot $(xcode-select -p)/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" \    CXXFLAGS="-isystem $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1"

(or put these into yourMake.user, so you don't need to remember them every time).

Address Sanitizer (ASAN)

For detecting or debugging memory bugs, you can use Clang'saddress sanitizer (ASAN). By compiling withSANITIZE_ADDRESS=1 you enable ASAN for the Julia compiler and its generated code. In addition, you can specifyLLVM_SANITIZE=1 to sanitize the LLVM library as well. Note that these options incur a high performance and memory cost. For example, using ASAN for Julia and LLVM makestestall1 take 8-10 times as long while using 20 times as much memory (this can be reduced to respectively a factor of 3 and 4 by using the options described below).

By default, Julia sets theallow_user_segv_handler=1 ASAN flag, which is required for signal delivery to work properly. You can define other options using theASAN_OPTIONS environment flag, in which case you'll need to repeat the default option mentioned before. For example, memory usage can be reduced by specifyingfast_unwind_on_malloc=0 andmalloc_context_size=2, at the cost of backtrace accuracy. For now, Julia also setsdetect_leaks=0, but this should be removed in the future.

Example setup

Step 1: Install toolchain

Checkout a Git worktree (or create out-of-tree build directory) at$TOOLCHAIN_WORKTREE and create a config file$TOOLCHAIN_WORKTREE/Make.user with

USE_BINARYBUILDER_LLVM=1BUILD_LLVM_CLANG=1

Run:

cd $TOOLCHAIN_WORKTREEmake -C deps install-llvm install-clang install-llvm-tools

to install toolchain binaries in$TOOLCHAIN_WORKTREE/usr/tools

Step 2: Build Julia with ASAN

Checkout a Git worktree (or create out-of-tree build directory) at$BUILD_WORKTREE and create a config file$BUILD_WORKTREE/Make.user with

TOOLCHAIN=$(TOOLCHAIN_WORKTREE)/usr/tools# use our new toolchainUSECLANG=1override CC=$(TOOLCHAIN)/clangoverride CXX=$(TOOLCHAIN)/clang++export ASAN_SYMBOLIZER_PATH=$(TOOLCHAIN)/llvm-symbolizerUSE_BINARYBUILDER_LLVM=1override SANITIZE=1override SANITIZE_ADDRESS=1# make the GC use regular malloc/frees, which are hooked by ASANoverride WITH_GC_DEBUG_ENV=1# default to a debug build for better line number reportingoverride JULIA_BUILD_MODE=debug# make ASAN consume less memoryexport ASAN_OPTIONS=detect_leaks=0:fast_unwind_on_malloc=0:allow_user_segv_handler=1:malloc_context_size=2JULIA_PRECOMPILE=1# tell libblastrampoline to not use RTLD_DEEPBINDexport LBT_USE_RTLD_DEEPBIND=0

Run:

cd $BUILD_WORKTREEmake debug

to buildjulia-debug with ASAN.

Memory Sanitizer (MSAN)

For detecting use of uninitialized memory, you can use Clang'smemory sanitizer (MSAN) by compiling withSANITIZE_MEMORY=1.

Thread Sanitizer (TSAN)

For debugging data-races and other threading related issues you can use Clang'sthread sanitizer (TSAN) by compiling withSANITIZE_THREAD=1.