Building Julia (Detailed)

Downloading the Julia source code

If you are behind a firewall, you may need to use thehttps protocol instead of thegit protocol:

git config --global url."https://".insteadOf git://

Be sure to also configure your system to use the appropriate proxy settings, e.g. by setting thehttps_proxy andhttp_proxy variables.

Building Julia

When compiled the first time, the build will automatically download pre-builtexternal dependencies. If you prefer to build all the dependencies on your own, or are building on a system that cannot access the network during the build process, add the following inMake.user:

USE_BINARYBUILDER=0

Building Julia requires 5GiB if building all dependencies and approximately 4GiB of virtual memory.

To perform a parallel build, usemake -j N and supply the maximum number of concurrent processes. If the defaults in the build do not work for you, and you need to set specific make parameters, you can save them inMake.user, and place the file in the root of your Julia source. The build will automatically check for the existence ofMake.user and use it if it exists.

You can create out-of-tree builds of Julia by specifyingmake O=<build-directory> configure on the command line. This will create a directory mirror, with all of the necessary Makefiles to build Julia, in the specified directory. These builds will share the source files in Julia anddeps/srccache. Each out-of-tree build directory can have its ownMake.user file to override the globalMake.user file in the top-level folder.

If everything works correctly, you will see a Julia banner and an interactive prompt into which you can enter expressions for evaluation. (Errors related to libraries might be caused by old, incompatible libraries sitting around in your PATH. In this case, try moving thejulia directory earlier in the PATH). Note that most of the instructions above apply to unix systems.

To run julia from anywhere you can:

• add an alias (inbash:echo "alias julia='/path/to/install/folder/bin/julia'" >> ~/.bashrc && source ~/.bashrc), or

• add a soft link to thejulia executable in thejulia directory to/usr/local/bin (or any suitable directory already in your path), or

• add thejulia directory to your executable path for this shell session (inbash:export PATH="$(pwd):$PATH" ; incsh ortcsh:

set path= ( $path $cwd ) ), or

• add thejulia directory to your executable path permanently (e.g. in.bash_profile), or

• writeprefix=/path/to/install/folder intoMake.user and then runmake install. If there is a version of Julia already installed in this folder, you should delete it before runningmake install.

Some of the options you can set to control the build of Julia are listed and documented at the beginning of the fileMake.inc, but you should never edit it for this purpose, useMake.user instead.

Julia's Makefiles define convenient automatic rules calledprint-<VARNAME> for printing the value of variables, replacing<VARNAME> with the name of the variable to print the value of. For example

$ make print-JULIA_PRECOMPILEJULIA_PRECOMPILE=1

These rules are useful for debugging purposes.

Now you should be able to run Julia like this:

julia

If you are building a Julia package for distribution on Linux, macOS, or Windows, take a look at the detailed notes indistributing.md.

Updating an existing source tree

If you have previously downloadedjulia usinggit clone, you can update the existing source tree usinggit pull rather than starting anew:

cd juliagit pull && make

Assuming that you had made no changes to the source tree that will conflict with upstream updates, these commands will trigger a build to update to the latest version.

General troubleshooting

1. Over time, the base library may accumulate enough changes such that the bootstrapping process in building the system image will fail. If this happens, the build may fail with an error like

    *** This error is usually fixed by running 'make clean'. If the error persists, try 'make cleanall' ***

   As described, runningmake clean && make is usually sufficient. Occasionally, the stronger cleanup done bymake cleanall is needed.

2. New versions of external dependencies may be introduced which may occasionally cause conflicts with existing builds of older versions.

   a. Specialmake targets exist to help wipe the existing build of a dependency. For example,make -C deps clean-llvm will clean out the existing build ofllvm so thatllvm will be rebuilt from the downloaded source distribution the next timemake is called.make -C deps distclean-llvm is a stronger wipe which will also delete the downloaded source distribution, ensuring that a fresh copy of the source distribution will be downloaded and that any new patches will be applied the next timemake is called.

   b. To delete existing binaries ofjulia and all its dependencies, delete the./usr directoryin the source tree.

3. If you've updated macOS recently, be sure to runxcode-select --install to update the command line tools. Otherwise, you could run into errors for missing headers and libraries, such asld: library not found for -lcrt1.10.6.o.

4. If you've moved the source directory, you might get errors such asCMake Error: The current CMakeCache.txt directory ... is different than the directory ... where CMakeCache.txt was created., in which case you may delete the offending dependency underdeps

5. In extreme cases, you may wish to reset the source tree to a pristine state. The following git commands may be helpful:

    git reset --hard #Forcibly remove any changes to any files under version control git clean -x -f -d #Forcibly remove any file or directory not under version control

   To avoid losing work, make sure you know what these commands do before you run them.git will not be able to undo these changes!

Platform-Specific Notes

Notes for various operating systems:

• Linux
• macOS
• Windows
• FreeBSD

Notes for various architectures:

• ARM

Required Build Tools and External Libraries

Building Julia requires that the following software be installed:

• [GNU make] — building dependencies.
• [gcc & g++][gcc] (>= 7.1) or[Clang][clang] (>= 5.0, >= 9.3 for Apple Clang) — compiling and linking C, C++.
• [libatomic][gcc] — provided by[gcc] and needed to support atomic operations.
• [python] (>=2.7) — needed to build LLVM.
• [gfortran] — compiling and linking Fortran libraries.
• [perl] — preprocessing of header files of libraries.
• [wget],[curl], or[fetch] (FreeBSD) — to automatically download external libraries.
• [m4] — needed to build GMP.
• [awk] — helper tool for Makefiles.
• [patch] — for modifying source code.
• [cmake] (>= 3.4.3) — needed to buildlibgit2.
• [pkg-config] — needed to buildlibgit2 correctly, especially for proxy support.
• [powershell] (>= 3.0) — necessary only on Windows.
• [which] — needed for checking build dependencies.

On Debian-based distributions (e.g. Ubuntu), you can easily install them withapt-get:

sudo apt-get install build-essential libatomic1 python gfortran perl wget m4 cmake pkg-config curl

Julia uses the following external libraries, which are automatically downloaded (or in a few cases, included in the Julia source repository) and then compiled from source the first time you runmake. The specific version numbers of these libraries that Julia uses are listed indeps/$(libname).version:

• [LLVM] (15.0 +patches) — compiler infrastructure (seenote below).
• [FemtoLisp] — packaged with Julia source, and used to implement the compiler front-end.
• [libuv] (custom fork) — portable, high-performance event-based I/O library.
• [OpenLibm] — portable libm library containing elementary math functions.
• [DSFMT] — fast Mersenne Twister pseudorandom number generator library.
• [OpenBLAS] — fast, open, and maintained [basic linear algebra subprograms (BLAS)]
• [LAPACK] — library of linear algebra routines for solving systems of simultaneous linear equations, least-squares solutions of linear systems of equations, eigenvalue problems, and singular value problems.
• [MKL] (optional) – OpenBLAS and LAPACK may be replaced by Intel's MKL library.
• [SuiteSparse] — library of linear algebra routines for sparse matrices.
• [PCRE] — Perl-compatible regular expressions library.
• [GMP] — GNU multiple precision arithmetic library, needed forBigInt support.
• [MPFR] — GNU multiple precision floating point library, needed for arbitrary precision floating point (BigFloat) support.
• [libgit2] — Git linkable library, used by Julia's package manager.
• [curl] — libcurl provides download and proxy support.
• [libssh2] — library for SSH transport, used by libgit2 for packages with SSH remotes.
• [mbedtls] — library used for cryptography and transport layer security, used by libssh2
• [utf8proc] — a library for processing UTF-8 encoded Unicode strings.
• [LLVM libunwind] — LLVM's fork of [libunwind], a library that determines the call-chain of a program.
• [ITTAPI] — Intel's Instrumentation and Tracing Technology and Just-In-Time API.

[GNU make]: https://www.gnu.org/software/make [patch]: https://www.gnu.org/software/patch [wget]: https://www.gnu.org/software/wget [m4]: https://www.gnu.org/software/m4 [awk]: https://www.gnu.org/software/gawk [gcc]: https://gcc.gnu.org [clang]: https://clang.llvm.org [python]: https://www.python.org/ [gfortran]: https://gcc.gnu.org/fortran/ [curl]: https://curl.haxx.se [fetch]: https://www.freebsd.org/cgi/man.cgi?fetch(1) [perl]: https://www.perl.org [cmake]: https://www.cmake.org [OpenLibm]: https://github.com/JuliaLang/openlibm [DSFMT]: https://github.com/MersenneTwister-Lab/dSFMT [OpenBLAS]: https://github.com/xianyi/OpenBLAS [LAPACK]: https://www.netlib.org/lapack [MKL]: https://software.intel.com/en-us/articles/intel-mkl [SuiteSparse]: https://people.engr.tamu.edu/davis/suitesparse.html [PCRE]: https://www.pcre.org [LLVM]: https://www.llvm.org [LLVM libunwind]: https://github.com/llvm/llvm-project/tree/main/libunwind [FemtoLisp]: https://github.com/JeffBezanson/femtolisp [GMP]: https://gmplib.org [MPFR]: https://www.mpfr.org [libuv]: https://github.com/JuliaLang/libuv [libgit2]: https://libgit2.org/ [utf8proc]: https://julialang.org/utf8proc/ [libunwind]: https://www.nongnu.org/libunwind [libssh2]: https://www.libssh2.org [mbedtls]: https://tls.mbed.org/ [pkg-config]: https://www.freedesktop.org/wiki/Software/pkg-config/ [powershell]: https://docs.microsoft.com/en-us/powershell/scripting/wmf/overview [which]: https://carlowood.github.io/which/ [ITTAPI]: https://github.com/intel/ittapi

Build dependencies

If you already have one or more of these packages installed on your system, you can prevent Julia from compiling duplicates of these libraries by passingUSE_SYSTEM_...=1 tomake or adding the line toMake.user. The complete list of possible flags can be found inMake.inc.

Please be aware that this procedure is not officially supported, as it introduces additional variability into the installation and versioning of the dependencies, and is recommended only for system package maintainers. Unexpected compile errors may result, as the build system will do no further checking to ensure the proper packages are installed.

LLVM

The most complicated dependency is LLVM, for which we require additional patches from upstream (LLVM is not backward compatible).

For packaging Julia with LLVM, we recommend either:

• bundling a Julia-only LLVM library inside the Julia package, or
• adding the patches to the LLVM package of the distribution.
  • A complete list of patches is available in onGithub see thejulia-release/15.x branch.
  • The only Julia-specific patch is the lib renaming (llvm7-symver-jlprefix.patch), which shouldnot be applied to a system LLVM.
  • The remaining patches are all upstream bug fixes, and have been contributed into upstream LLVM.

Using an unpatched or different version of LLVM will result in errors and/or poor performance. You can build a different version of LLVM from a remote Git repository with the following options in theMake.user file:

# Force source build of LLVMUSE_BINARYBUILDER_LLVM = 0# Use Git for fetching LLVM source code# this is either `1` to get all of themDEPS_GIT = 1# or a space-separated list of specific dependencies to download with gitDEPS_GIT = llvm# Other useful options:#URL of the Git repository you want to obtain LLVM from:#  LLVM_GIT_URL = ...#Name of the alternate branch to clone from git#  LLVM_BRANCH = julia-16.0.6-0#SHA hash of the alterate commit to check out automatically#  LLVM_SHA1 = $(LLVM_BRANCH)#List of LLVM targets to build.  It is strongly recommended to keep at least all the#default targets listed in `deps/llvm.mk`, even if you don't necessarily need all of them.#  LLVM_TARGETS = ...#Use ccache for faster recompilation in case you need to restart a build.#  USECCACHE = 1#  CMAKE_GENERATOR=Ninja#  LLVM_ASSERTIONS=1#  LLVM_DEBUG=Symbols

The various build phases are controlled by specific files:

• deps/llvm.version : touch or change to checkout a new version,make get-llvm check-llvm
• deps/srccache/llvm/source-extracted : result ofmake extract-llvm
• deps/llvm/build_Release*/build-configured : result ofmake configure-llvm
• deps/llvm/build_Release*/build-configured : result ofmake compile-llvm
• usr-staging/llvm/build_Release*.tgz : result ofmake stage-llvm (regenerate withmake reinstall-llvm)
• usr/manifest/llvm : result ofmake install-llvm (regenerate withmake uninstall-llvm)
• make version-check-llvm : runs every time to warn the user if there are local modifications

Though Julia can be built with newer LLVM versions, support for this should be regarded as experimental and not suitable for packaging.

libuv

Julia uses a custom fork of libuv. It is a small dependency, and can be safely bundled in the same package as Julia, and will not conflict with the system library. Julia builds shouldnot try to use the system libuv.

BLAS and LAPACK

As a high-performance numerical language, Julia should be linked to a multi-threaded BLAS and LAPACK, such as OpenBLAS or ATLAS, which will provide much better performance than the referencelibblas implementations which may be default on some systems.

Source distributions of releases

Each pre-release and release of Julia has a "full" source distribution and a "light" source distribution.

The full source distribution contains the source code for Julia and all dependencies so that it can be built from source without an internet connection. The light source distribution does not include the source code of dependencies.

For example,julia-1.0.0.tar.gz is the light source distribution for thev1.0.0 release of Julia, whilejulia-1.0.0-full.tar.gz is the full source distribution.

Building Julia from source with a Git checkout of a stdlib

If you need to build Julia from source with a Git checkout of a stdlib, then usemake DEPS_GIT=NAME_OF_STDLIB when building Julia.

For example, if you need to build Julia from source with a Git checkout of Pkg, then usemake DEPS_GIT=Pkg when building Julia. ThePkg repo is instdlib/Pkg, and created initially with a detachedHEAD. If you're doing this from a pre-existing Julia repository, you may need tomake clean beforehand.

If you need to build Julia from source with Git checkouts of more than one stdlib, thenDEPS_GIT should be a space-separated list of the stdlib names. For example, if you need to build Julia from source with a Git checkout of Pkg, Tar, and Downloads, then usemake DEPS_GIT='Pkg Tar Downloads' when building Julia.

Building an "assert build" of Julia

An "assert build" of Julia is a build that was built with bothFORCE_ASSERTIONS=1 andLLVM_ASSERTIONS=1. To build an assert build, define both of the following variables in yourMake.user file:

FORCE_ASSERTIONS=1LLVM_ASSERTIONS=1

Please note that assert builds of Julia will be slower than regular (non-assert) builds.

Building 32-bit Julia on a 64-bit machine

Occasionally, bugs specific to 32-bit architectures may arise, and when this happens it is useful to be able to debug the problem on your local machine. Since most modern 64-bit systems support running programs built for 32-bit ones, if you don't have to recompile Julia from source (e.g. you mainly need to inspect the behavior of a 32-bit Julia without having to touch the C code), you can likely use a 32-bit build of Julia for your system that you can obtain from theofficial downloads page. However, if you do need to recompile Julia from source one option is to use a Docker container of a 32-bit system. At least for now, building a 32-bit version of Julia is relatively straightforward usingubuntu 32-bit docker images. In brief, after setting updocker here are the required steps:

$ docker pull i386/ubuntu$ docker run --platform i386 -i -t i386/ubuntu /bin/bash

At this point you should be in a 32-bit machine console (note thatuname reports the host architecture, so will still say 64-bit, but this will not affect the Julia build). You can add packages and compile code; when youexit, all the changes will be lost, so be sure to finish your analysis in a single session or set up a copy/pastable script you can use to set up your environment.

From this point, you should

# apt update

(Note thatsudo isn't installed, but neither is it necessary since you are running asroot, so you can omitsudo from all commands.)

Then add all thebuild dependencies, a console-based editor of your choice,git, and anything else you'll need (e.g.,gdb,rr, etc). Pick a directory to work in andgit clone Julia, check out the branch you wish to debug, and build Julia as usual.

Update the version number of a dependency

There are two types of builds

1. Build everything (deps/ andsrc/) from source code. (AddUSE_BINARYBUILDER=0 toMake.user, seeBuilding Julia)
2. Build from source (src/) with pre-compiled dependencies (default)

When you want to update the version number of a dependency indeps/, you may want to use the following checklist:

### Check listVersion numbers:- [ ] `deps/$(libname).version`: `LIBNAME_VER`, `LIBNAME_BRANCH`, `LIBNAME_SHA1` and `LIBNAME_JLL_VER`- [ ] `stdlib/$(LIBNAME_JLL_NAME)_jll/Project.toml`: `version`Checksum:- [ ] `deps/checksums/$(libname)`- [ ] `deps/checksums/$(LIBNAME_JLL_NAME)-*/`: `md5` and `sha512`Patches:- [ ] `deps/$(libname).mk`- [ ] `deps/patches/$(libname)-*.patch`

Note:

• For specific dependencies, some items in the checklist may not exist.
• For checksum file, it may bea single file without a suffix, ora folder containing two files.

Example:OpenLibm

1. Update Version numbers indeps/openlibm.version
   • OPENLIBM_VER := 0.X.Y
   • OPENLIBM_BRANCH = v0.X.Y
   • OPENLIBM_SHA1 = new-sha1-hash
2. Update Version number instdlib/OpenLibm_jll/Project.toml
   • version = "0.X.Y+0"
3. Update checksums indeps/checksums/openlibm
   • make -f contrib/refresh_checksums.mk openlibm
4. Check if the patch filesdeps/patches/openlibm-*.patch exist
   • if patches don't exist, skip.
   • if patches exist, check if they have been merged into the new version and need to be removed. When deleting a patch, remember to modify the corresponding Makefile file (deps/openlibm.mk).