Binary distributions

These notes are for those wishing to compile a binary distribution of Julia for distribution on various platforms. We love users spreading Julia as far and wide as they can, trying it out on as wide an array of operating systems and hardware configurations as possible. As each platform has specific gotchas and processes that must be followed in order to create a portable, working Julia distribution, we have separated most of the notes by OS.

Note that while the code for Julia isMIT-licensed, with a few exceptions, the distribution created by the techniques described herein will be GPL licensed, as various dependent libraries such asSuiteSparse are GPL licensed. We do hope to have a non-GPL distribution of Julia in the future.

Versioning and Git

The Makefile uses both theVERSION file and commit hashes and tags from the git repository to generate thebase/version_git.jl with information we use to fill the splash screen and theversioninfo() output. If you for some reason don't want to have the git repository available when building you should pregenerate thebase/version_git.jl file with:

make -C base version_git.jl.phony

Julia has lots of build dependencies where we use patched versions that has not yet been included by the popular package managers. These dependencies will usually be automatically downloaded when you build, but if you want to be able to build Julia on a computer without internet access you should create a full-source-dist archive with the special make target

make full-source-dist

that creates a julia-version-commit.tar.gz archive with all required dependencies.

When compiling a tagged release in the git repository, we don't display the branch/commit hash info in the splash screen. You can use this line to show a release description of up to 45 characters. To set this line you have to create a Make.user file containing:

override TAGGED_RELEASE_BANNER = "my-package-repository build"

Target Architectures

By default, Julia optimizes its system image to the native architecture of the build machine. This is usually not what you want when building packages, as it will make Julia fail at startup on any machine with incompatible CPUs (in particular older ones with more restricted instruction sets).

We therefore recommend that you pass theMARCH variable when callingmake, setting it to the baseline target you intend to support. This will determine the target CPU for both the Julia executable and libraries, and the system image (the latter can also be set usingJULIA_CPU_TARGET). Typically useful values for x86 CPUs arex86-64 andcore2 (for 64-bit builds) andpentium4 (for 32-bit builds). Unfortunately, CPUs older than Pentium 4 are currently not supported (seethis issue).

The full list of CPU targets supported by LLVM can be obtained by runningllc -mattr=help.

Linux

On Linux,make binary-dist creates a tarball that contains a fully functional Julia installation. If you wish to create a distribution package such as a.deb, or.rpm, some extra effort is needed. See thejulia-debian repository for an example of what metadata is needed for creating.deb packages for Debian and Ubuntu-based systems. See theFedora package for RPM-based distributions. Although we have not yet experimented with it,Alien could be used to generate Julia packages for various Linux distributions.

Julia supports overriding standard installation directories viaprefix and other environment variables you can pass when callingmake andmake install. See Make.inc for their list.DESTDIR can also be used to force the installation into a temporary directory.

By default, Julia loads$prefix/etc/julia/startup.jl as an installation-wide initialization file. This file can be used by distribution managers to set up custom paths or initialization code. For Linux distribution packages, if$prefix is set to/usr, there is no/usr/etc to look into. This requires the path to Julia's privateetc directory to be changed. This can be done via thesysconfdir make variable when building. Simply passsysconfdir=/etc tomake when building and Julia will first check/etc/julia/startup.jl before trying$prefix/etc/julia/startup.jl.

OS X

To create a binary distribution on OSX, build Julia first, then cd tocontrib/mac/app, and runmake with the same makevars that were used withmake when building Julia proper. This will then create a.dmg file in thecontrib/mac/app directory holding a completely self-contained Julia.app.

Alternatively, Julia may be built as a framework by invokingmake with thedarwinframework target andDARWIN_FRAMEWORK=1 set. For example,make DARWIN_FRAMEWORK=1 darwinframework.

Windows

Instructions for reating a Julia distribution on Windows are described in thebuild devdocs for Windows.

Notes on BLAS and LAPACK

Julia builds OpenBLAS by default, which includes the BLAS and LAPACK libraries. On 32-bit architectures, Julia builds OpenBLAS to use 32-bit integers, while on 64-bit architectures, Julia builds OpenBLAS to use 64-bit integers (ILP64). It is essential that all Julia functions that call BLAS and LAPACK API routines use integers of the correct width.

Most BLAS and LAPACK distributions provided on linux distributions, and even commercial implementations ship libraries that use 32-bit APIs. In many cases, a 64-bit API is provided as a separate library.

When using vendor provided or OS provided libraries, amake option calledUSE_BLAS64 is available as part of the Julia build. When doingmake USE_BLAS64=0, Julia will call BLAS and LAPACK assuming a 32-bit API, where all integers are 32-bit wide, even on a 64-bit architecture.

Other libraries that Julia uses, such as SuiteSparse also use BLAS and LAPACK internally. The APIs need to be consistent across all libraries that depend on BLAS and LAPACK. The Julia build process will build all these libraries correctly, but when overriding defaults and using system provided libraries, this consistency must be ensured.

Also note that Linux distributions sometimes ship several versions of OpenBLAS, some of which enable multithreading, and others only working in a serial fashion. For example, in Fedora,libopenblasp.so is threaded, butlibopenblas.so is not. We recommend using the former for optimal performance. To choose an OpenBLAS library whose name is different from the defaultlibopenblas.so, passLIBBLAS=-l$(YOURBLAS) andLIBBLASNAME=lib$(YOURBLAS) tomake, replacing$(YOURBLAS) with the name of your library. You can also add.so.0 to the name of the library if you want your package to work without requiring the unversioned.so symlink.

Finally, OpenBLAS includes its own optimized version of LAPACK. If you setUSE_SYSTEM_BLAS=1 andUSE_SYSTEM_LAPACK=1, you should also setLIBLAPACK=-l$(YOURBLAS) andLIBLAPACKNAME=lib$(YOURBLAS). Else, the reference LAPACK will be used and performance will typically be much lower.

Starting with Julia 1.7, Julia useslibblastrampoline to pick a different BLAS at runtime.

Point releasing 101

Creating a point/patch release consists of several distinct steps.

Backporting commits

Some pull requests are labeled "backport pending x.y", e.g. "backport pending 0.6". This designates that the next subsequent release tagged from the release-x.y branch should include the commit(s) in that pull request. Once the pull request is merged into master, each of the commits should becherry picked to a dedicated branch that will ultimately be merged into release-x.y.

Creating a backports branch

First, create a new branch based on release-x.y. The typical convention for Julia branches is to prefix the branch name with your initials if it's intended to be a personal branch. For the sake of example, we'll say that the author of the branch is Jane Smith.

git fetch origingit checkout release-x.ygit rebase origin/release-x.ygit checkout -b js/backport-x.y

This ensures that your local copy of release-x.y is up to date with origin before you create a new branch from it.

Cherry picking commits

Now we do the actual backporting. Find all merged pull requests labeled "backport pending x.y" in the GitHub web UI. For each of these, scroll to the bottom where it says "someperson merged commit123abc intomaster XX minutes ago". Note that the commit name is a link; if you click it, you'll be shown the contents of the commit. If this page shows that123abc is a merge commit, go back to the PR page–-we don't want merge commits, we want the actual commits. However, if this does not show a merge commit, it means that the PR was squash-merged. In that case, use the git SHA of the commit, listed next to commit on this page.

Once you have the SHA of the commit, cherry-pick it onto the backporting branch:

git cherry-pick -x -e <sha>

There may be conflicts which need to be resolved manually. Once conflicts are resolved (if applicable), add a reference to the GitHub pull request that introduced the commit in the body of the commit message.

After all of the relevant commits are on the backports branch, push the branch to GitHub.

Checking for performance regressions

Point releases should never introduce performance regressions. Luckily the Julia benchmarking bot, Nanosoldier, can run benchmarks against any branch, not just master. In this case we want to check the benchmark results of js/backport-x.y against release-x.y. To do this, awaken the Nanosoldier from his robotic slumber using a comment on your backporting pull request:

@nanosoldier `runbenchmarks(ALL, vs=":release-x.y")`

This will run all registered benchmarks on release-x.y and js/backport-x.y and produce a summary of results, marking all improvements and regressions.

If Nanosoldier finds any regressions, try verifying locally and rerun Nanosoldier if necessary. If the regressions are deemed to be real rather than just noise, you'll have to find a commit on master to backport that fixes it if one exists, otherwise you should determine what caused the regression and submit a patch (or get someone who knows the code to submit a patch) to master, then backport the commit once that's merged. (Or submit a patch directly to the backport branch if appropriate.)

Building test binaries

After the backport PR has been merged into therelease-x.y branch, update your local clone of Julia, then get the SHA of the branch using

git rev-parse origin/release-x.y

Keep that handy, as it's what you'll enter in the "Revision" field in the buildbot UI.

For now, all you need are binaries for Linux x86-64, since this is what's used for running PackageEvaluator. Go to https://buildog.julialang.org, submit a job fornuke_linux64, then queue up a job forpackage_linux64, providing the SHA as the revision. When the packaging job completes, it will upload the binary to thejulialang2 bucket on AWS. Retrieve the URL, as it will be used for PackageEvaluator.

Checking for package breakages

Point releases should never break packages, with the possible exception of packages that are doing some seriously questionable hacks using Base internals that are not intended to be user-facing. (In those cases, maybe have a word with the package author.)

Checking whether changes made in the forthcoming new version will break packages can be accomplished usingPackageEvaluator, often called "PkgEval" for short. PkgEval is what populates the status badges on GitHub repos and on pkg.julialang.org. It typically runs on one of the non-benchmarking nodes of Nanosoldier and uses Vagrant to perform its duties in separate, parallel VirtualBox virtual machines.

Setting up PackageEvaluator

Clone PackageEvaluator and create a branch calledbackport-x.y.z, and check it out. Note that the required changes are a little hacky and confusing, and hopefully that will be addressed in a future version of PackageEvaluator. The changes to make will be modeled off ofthis commit.

The setup script takes its first argument as the version of Julia to run and the second as the range of package names (AK for packages named A-K, LZ for L-Z). The basic idea is that we're going to tweak that a bit to run only two versions of Julia, the current x.y release and our backport version, each with three ranges of packages.

In the linked diff, we're saying that if the second argument is LZ, use the binaries built from our backport branch, otherwise (AK) use the release binaries. Then we're using the first argument to run a section of the package list: A-F for input 0.4, G-N for 0.5, and O-Z for 0.6.

Running PackageEvaluator

To run PkgEval, find a hefty enough machine (such as Nanosoldier node 1), then run

git clone https://github.com/JuliaCI/PackageEvaluator.jl.gitcd PackageEvaluator.jl/scriptsgit checkout backport-x.y.z./runvagrant.sh

This produces some folders in the scripts/ directory. The folder names and their contents are decoded below:

Investigating results

Once that's done, you can use./summary.sh from that same directory to produce a summary report of the findings. We'll do so for each of the folders to aggregate overall results by version.

./summary.sh 0.4AK/*.json > summary_release.txt./summary.sh 0.5AK/*.json >> summary_release.txt./summary.sh 0.6AK/*.json >> summary_release.txt./summary.sh 0.4LZ/*.json > summary_backport.txt./summary.sh 0.5LZ/*.json >> summary_backport.txt./summary.sh 0.6LZ/*.json >> summary_backport.txt

Now we have two files,summary_release.txt andsummary_backport.txt, containing the PackageEvaluator test results (pass/fail) for each package for the two versions.

To make these easier to ingest into a Julia, we'll convert them into CSV files then use the DataFrames package to process the results. To convert to CSV, copy each .txt file to a corresponding .csv file, then enter Vim and executeggVGI"<esc> then:%s/\.json /",/g. (You don't have to use Vim; this just is one way to do it.) Now process the results with Julia code similar to the following.

using DataFramesrelease = readtable("summary_release.csv", header=false, names=[:package, :release])backport = readtable("summary_backport.csv", header=false, names=[:package, :backport])results = join(release, backport, on=:package, kind=:outer)for result in eachrow(results)    a = result[:release]    b = result[:backport]    if (isna(a) && !isna(b)) || (isna(b) && !isna(a))        color = :yellow    elseif a != b && occursin("pass", b)        color = :green    elseif a != b        color = :red    else        continue    end    printstyled(result[:package], ": Release ", a, " -> Backport ", b, "\n", color=color)end

This will write color-coded lines tostdout. All lines in red must be investigated as they signify potential breakages caused by the backport version. Lines in yellow should be looked into since it means a package ran on one version but not on the other for some reason. If you find that your backported branch is causing breakages, usegit bisect to identify the problematic commits,git revert those commits, and repeat the process.

Merging backports into the release branch

After you have ensured that

• the backported commits pass all of Julia's unit tests,
• there are no performance regressions introduced by the backported commits as compared to the release branch, and
• the backported commits do not break any registered packages,

then the backport branch is ready to be merged into release-x.y. Once it's merged, go through and remove the "backport pending x.y" label from all pull requests containing the commits that have been backported. Do not remove the label from PRs that have not been backported.

The release-x.y branch should now contain all of the new commits. The last thing we want to do to the branch is to adjust the version number. To do this, submit a PR against release-x.y that edits the VERSION file to remove-pre from the version number. Once that's merged, we're ready to tag.

Tagging the release

It's time! Check out the release-x.y branch and make sure that your local copy of the branch is up to date with the remote branch. At the command line, run

git tag v$(cat VERSION)git push --tags

This creates the tag locally and pushes it to GitHub.

After tagging the release, submit another PR to release-x.y to bump the patch number and add-pre back to the end. This denotes that the branch state reflects a prerelease version of the next point release in the x.y series.

Follow the remaining directions in the Makefile.

Signing binaries

Some of these steps will require secure passwords. To obtain the appropriate passwords, contact Elliot Saba (staticfloat) or Alex Arslan (ararslan). Note that code signing for each platform must be performed on that platform (e.g. Windows signing must be done on Windows, etc.).

Linux

Code signing must be done manually on Linux, but it's quite simple. First obtain the filejulia.key from the CodeSigning folder in thejuliasecure AWS bucket. Add this to your GnuPG keyring using

gpg --import julia.key

This will require entering a password that you must obtain from Elliot or Alex. Next, set the trust level for the key to maximum. Start by entering agpg session:

gpg --edit-key julia

At the prompt, typetrust, then when asked for a trust level, provide the maximum available (likely 5). Exit GnuPG.

Now, for each of the Linux tarballs that were built on the buildbots, enter

gpg -u julia --armor --detach-sig julia-x.y.z-linux-<arch>.tar.gz

This will produce a corresponding .asc file for each tarball. And that's it!

macOS

Code signing should happen automatically on the macOS buildbots. However, it's important to verify that it was successful. On a system or virtual machine running macOS, download the .dmg file that was built on the buildbots. For the sake of example, say that the .dmg file is calledjulia-x.y.z-osx.dmg. Run

mkdir ./jlmnthdiutil mount -readonly -mountpoint ./jlmnt julia-x.y.z-osx.dmgcodesign -v jlmnt/Julia-x.y.app

Be sure to note the name of the mounted disk listed when mounting! For the sake of example, we'll assume this isdisk3. If the code signing verification exited successfully, there will be no output from thecodesign step. If it was indeed successful, you can detach the .dmg now:

hdiutil eject /dev/disk3rm -rf ./jlmnt

If you get a message like

Julia-x.y.app: code object is not signed at all

then you'll need to sign manually.

To sign manually, first retrieve the OS X certificates from the CodeSigning folder in thejuliasecure bucket on AWS. Add the .p12 file to your keychain using Keychain.app. Ask Elliot Saba (staticfloat) or Alex Arslan (ararslan) for the password for the key. Now run

hdiutil convert julia-x.y.z-osx.dmg -format UDRW -o julia-x.y.z-osx_writable.dmgmkdir ./jlmnthdiutil mount -mountpoint julia-x.y.z-osx_writable.dmgcodesign -s "AFB379C0B4CBD9DB9A762797FC2AB5460A2B0DBE" --deep jlmnt/Julia-x.y.app

This may fail with a message like

Julia-x.y.app: resource fork, Finder information, or similar detritus not allowed

If that's the case, you'll need to remove extraneous attributes:

xattr -cr jlmnt/Julia-x.y.app

Then retry code signing. If that produces no errors, retry verification. If all is now well, unmount the writable .dmg and convert it back to read-only:

hdiutil eject /dev/disk3rm -rf ./jlmnthdiutil convert julia-x.y.z-osx_writable.dmg -format UDZO -o julia-x.y.z-osx_fixed.dmg

Verify that the resulting .dmg is in fact fixed by double clicking it. If everything looks good, eject it then drop the_fixed suffix from the name. And that's it!

Windows

Signing must be performed manually on Windows. First obtain the Windows 10 SDK, which contains the necessary signing utilities, from the Microsoft website. We need theSignTool utility which should have been installed somewhere likeC:\Program Files (x86)\Windows Kits\10\App Certification Kit. Grab the Windows certificate files from CodeSigning onjuliasecure and put them in the same directory as the executables. Open a Windows CMD window,cd to where all the files are, and run

set PATH=%PATH%;C:\Program Files (x86)\Windows Kits\10\App Certification Kit;signtool sign /f julia-windows-code-sign_2017.p12 /p "PASSWORD" ^   /t http://timestamp.verisign.com/scripts/timstamp.dll ^   /v julia-x.y.z-win32.exe

Note that^ is a line continuation character in Windows CMD andPASSWORD is a placeholder for the password for this certificate. As usual, contact Elliot or Alex for passwords. If there are no errors, we're all good!

Uploading binaries

Now that everything is signed, we need to upload the binaries to AWS. You can use a program like Cyberduck or theaws command line utility. The binaries should go in thejulialang2 bucket in the appropriate folders. For example, Linux x86-64 goes injulialang2/bin/linux/x.y. Be sure to delete the currentjulia-x.y-latest-linux-<arch>.tar.gz file and replace it with a duplicate ofjulia-x.y.z-linux-<arch>.tar.gz.

We also need to upload the checksums for everything we've built, including the source tarballs and all release binaries. This is simple:

shasum -a 256 julia-x.y.z* | grep -v -e sha256 -e md5 -e asc > julia-x.y.z.sha256md5sum julia-x.y.z* | grep -v -e sha256 -e md5 -e asc > julia-x.y.z.md5

Note that if you're running those commands on macOS, you'll get very slightly different output, which can be reformatted by looking at an existing file. Mac users will also need to usemd5 -r instead ofmd5sum. Upload the .md5 and .sha256 files tojulialang2/bin/checksums on AWS.

Ensure that the permissions on AWS for all uploaded files are set to "Everyone: READ."

For each file we've uploaded, we need to purge the Fastly cache so that the links on the website point to the updated files. As an example:

curl -X PURGE https://julialang-s3.julialang.org/bin/checksums/julia-x.y.z.sha256

Sometimes this isn't necessary but it's good to do anyway.