Repository files navigation

BringNix dependency management to your Rust project!

• Development Shell - knowing all the dependencies means easy creation ofcomplete shells. Runnix develop ordirenv allow in this repo and see!
• Caching - CI & CD pipelines move faster when purity guarantees allowskipping more work!
• Reproducibility - Pure builds. Access to all ofnixpkgs for repeatable environment setupacross multiple distributions and platforms

Withnix (with flake support) installed, generate aCargo.nix for your project:

# Use nix to get cargo2nix & rust toolchain on your pathnix develop github:cargo2nix/cargo2nix#bootstrap# In directory with Cargo.toml & Cargo.lock files (cargo generate-lockfile)cargo2nix# Or skip the shell and run it directlynix run github:cargo2nix/cargo2nix# You'll need this in version controlgit add Cargo.nix

To consume your newCargo.nix, write a nix expression like that found in thehello world example.

A bare minimum flake.nix:

{inputs={cargo2nix.url="github:cargo2nix/cargo2nix/release-0.11.0";flake-utils.follows="cargo2nix/flake-utils";nixpkgs.follows="cargo2nix/nixpkgs";};outputs=inputs:withinputs;flake-utils.lib.eachDefaultSystem(system:letpkgs=importnixpkgs{inheritsystem;overlays=[cargo2nix.overlays.default];};rustPkgs=pkgs.rustBuilder.makePackageSet{rustVersion="1.75.0";packageFun=import./Cargo.nix;};inrec{packages={# replace hello-world with your package namehello-world=(rustPkgs.workspace.hello-world{});default=packages.hello-world;};});}

For a more complete project with CI & CD mostly ready to go, check outUnixsocks or cargo2nix's ownCI workflow.

# these must be in version control!git add flake.nix Cargo.nixnix build........../result-bin/bin/hellohello world!

Check out our series ofexample projects which showcase how to usecargo2nix in more detail.

In this repo, simply usenix develop ordirenv allow. Even if you are ona bare NixOS system or fresh OSX environment with no dependencies or toolchainsinstalled, you will have everything you need to runcargo build. See thedevShell attribute inflake.nix to see how to prepare this kind of shell.

TheworkspaceShell function, created bymakePackagSet, acceptsall the same options as the nixmkShell function.

In your flake, you can choose your cargo2nix version by changing the URL.

Only use unstable for developing with the latest features. PR's against oldreleases can be accepted but no active support will be done.The defaultbranch for this repo is updated whenever a new release tag is made. Onlyspecific release branches are "stable."

Update your flake lock with the latest or a specific version of cargo2nix:

nix flake lock --update-input cargo2nixnix flake lock --update-input cargo2nix --override-input cargo2nix github:cargo2nix/cargo2nix/?rev=d45481420482fa7d9b0a62836555e24ec07d93be

If you need newer versions of Rust or the flake-utils inputs, just specify themusing URL instead of follows.

ThemakePackageSet function fromtheoverlay accepts arguments thatadjust how the workspace is built. Only thepackageFun argument is required.Cargo2nix's ownflake.nix has more information.

• rustVersion - is either a version string or YYYY-MM-DD date-string

• rustChannel -"nightly""beta""stable"

• rustProfile -"default" or"minimal" usually

• extraRustComponents -["clippy" "miri"] etc

• workspaceSrc - override where the source is supplied relative to theCargo.nix

• rootFeatures - a list offoo/feature strings for workspace crate features

• packageOverrides - control over the individual crate overrides used to makethem compatible on some platforms, for example to tweak C lib consumption

• target - setting an explicit target, useful when cross compiling to obtain aspecific Rust target that doesn't align with the nixpkgs target

rustPkgs contains all crates in the dependency graph and some extraconveniences for development. The workspace crates are also exposed via aworkspace attribute.

rustPkgs.<registry>.<crate>.<version> is an example of a cratefunctionpath. Calling the function results in a completed derivation, which can be usedas a flake output. They support all the normal behaviors such asoverride andoverrideAttrs. SeemkCrate.nix for the full set ofarguments the crate function supports.

rustPkgs.workspace.<crate> are usually the packages you will use. The otherpaths look like:

rustPkgs."registry+https://github.com/rust-lang/crates.io-index".openssl."0.10.30"

rustPkgs.workspaceShell is a derivation using Nix's standardmkShell,embellished with information we learned from the dependencies and theiroverrides, enabling vanillacargo build to work in anix develop shell.

This is for finished derivations, not for dependencies. Keep reading below forusingmakeOverride in the dependency tree.

workspaceShell and crates both supportoverride andoverrideAttrs like normal Nix derivations. This allows you to customizethe workspace shell or a build step in your workspace crate very easily. Seenix show-derivation andnix show-derivation #devShell for more information.

You can make overrides to packages in the dependency tree. See examples inoverrides.nix. Overriding thebuildPhase etc ispossible for a single crate without modifyingmkcrate.nix in cargo2nixdirectly. The output ofnix show-derivation can be valuable when determiningwhat the current output result is.

The most important function in cargo2nix source is mkcrate.nix because it'show we store information in dependents and replay them back whenbuilding dependents. It is vital for building crates in isolation.

• Thecargo2nix utility reads the Rust workspace configuration andCargo.lock and generates nix expressions that encode some of the feature,platform, and target logic into aCargo.nix

• The cargo2nixNixpkgsoverlayconsumes theCargo.nix, feeding it what you pass tomakePackageSet toprovide workspace outputs you can expose in your nix flake

• Because we know all of the dependencies, it's easy to create a shell from thosedependencies as environment setup using theworkspaceShell function andexposing the result in thedevShell flake output

Just like regularcargo builds, the Nix dependencies form aDAG, butpurity means we only expose essential information to dependencies and manuallyinvokecargo. Communication from dependencies to dependents is handled bywriting some extra outputs and then reading those outputs inside the nextdependent build.

There's two broad categories of information that need to be transmitted whenhand-building crates in isolation:

• Global information

  • target such asx86_64-unknown-linux-gnu
  • cargo actions such asbuild ortest
  • features which turn on optional dependencies & downstream features via logicin theCargo.nix expressions

  This information is known before any of the crates are built. It's used atevaluation time to decide what will be built. Seenix show-derivation results.

• Propagated information

  Each dependency writes information such as linker flags alongside its rlib andother outputs. When the dependent is going to consume the dependency, itreads this information back.

Derivations are evaluated in Nix with global information available. During thebuild, rlibs and dependency information are propagated back up the DAG. Eachderivation's build shell combines the linking, features, target, and otherinformation. You can see how it's used inmkcrate.nix

Evaluation of nix derivations doesn't require building anything. If you want tobuild a specific variant of a crate in a workspace with Nix, we would have toknow this when building all of its dependencies. This means certain behavior toswitch features and optional dependencies on or off depends on whatelse isbeing built. By defaultcargo2nix will build crates as if all other crates inthe workspacemight be build. This can be somewhat controlled with therootFeatures argument. (seerootFeatures inCargo.nix).This actually improves caching but may rarely result in a long build for anunneeded dependency (which your workspace should put behind a non-defaulttop-level feature). Cargo isn't any better at this aspect of caching vsrebuilding.

1. Flakes requireflake.nix andCargo.nix to be in version control.git add flake.nix Cargo.nix etc. Remember to keep them up to date! Beforebuilding the examples, you will usually need to update their pin ofcargo2nix:nix flake lock --update-input cargo2nix or nix may complainabout paths.

2. Old versions of thecargo2nix.overlay usually cannot consume newer versionsof theCargo.nix that an updated cargo2nix will produce. Update yourinputs withnix flake lock --update-input cargo2nix ornix build --update-input cargo2nix

3. When buildingsys crates,build.rs scripts may themselves attempt toprovide native dependencies that could be missing. See theoverlay/overrides.nix for patterns of common solutions for fixing upspecific deps.

   To provide your own override, pass a modifiedpackageOverrides topkgs.rustBuilder.makePackageSet:

   rustPkgs=pkgs.rustBuilder.makePackageSet{# ... required arguments not shown# Use the existing all list of overrides and append your overridepackageOverrides=pkgs:pkgs.rustBuilder.overrides.all++[# parentheses disambiguate each makeOverride call as a single list element(pkgs.rustBuilder.rustLib.makeOverride{name="fantasy-zlib-sys";overrideAttrs=drv:{propagatedBuildInputs=drv.propagatedBuildInputsor[]++[pkgs.zlib.dev];};})];};

4. Each derivation function inrustBuilder.makePackageSet has it's outputscreated withinmkcrate.nix, using some bashfunctions inmkcrate-utils.sh. We try not tocopy more than necessary to the outputs, but this also means sometimesskipping a necessary file. Each derivation ismultiple output, usingbinandout. The default output isbin and should contain just what'snecessary at runtime, possibly linked to other files in the Nix store. Thisoutput is for installation into a Nix profile or shell. Theout outputcontains all of this and extra information necessary for dependents toconsume the crate, usually linking information, which will collide if youattempt to install several such derivations.

5. Non-deterministic rustc or linker behavior can lead to binary-incompatiblecrates. Nix cannot protect from non-determinism, only impurity. Overrideyour builds withpreferLocalBuild = true;allowSubstitutes = false; forthe affected package. This has been seen more often because ofnon-deterministic macros. See #184 for more information.

6. Nixpkgs is a rolling release, and that means breakages occur but you havemany potential successful versions to choose from. View theCI logs andcheck theflake.lock for rev information from recent successes. Update toa specific input version with:

    nix flake lock --override-input nixpkgs github:nixpgks/nixpkgs?rev=a284564b7f75ac4db73607db02076e8da9d42c9d

7. Toml parsing / conversion issuesError: Cannot convert data to TOML (Invalid type <class 'NoneType'>)

   jq andremarshal are used to read & modify toml files in somecases. Lines of the form:[key."cfg(foo = \"a\", bar = \"b\"))".path]could produce breakage whenjq output was fed back toremarshal. Thereare workarounds in place to catch many cases. See #149 for more informationand report any newly found breakage until a total solution is in place.

8. Git dependencies and crates from alternative Cargo registries rely onbuiltins.fetchGit to support fetching from private Git repositories. Thismeans that such dependencies cannot be evaluated withrestrict-evalapplied.

   Also, if your Git dependency is tied to a Git branch, e.g.master, and youwould like to force it to update on upstream changes, you should append--option tarball-ttl 0 to yournix-build command.

You can load a nix shell for any crate derivation in the dependency tree. Thisis the same environment thecargo2nix overlay will build them in.

To do this, first find the.drv for your dependency by using, for example,nix show-derivation | grep colorify

nix show-derivation| rg -o"/nix.*crate.*colorify.*drv"nix/store/whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz# ignore environment to remove your shell's impuritiesnix develop --ignore-environment nix/store/whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz# the environment is now as it is when nix builds the packageecho$srcnix/store/whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gz# If you are working on a dependency and need the source (or a fresh copy) you# can unpack the $src variable. Through nix stdenv, tar is available in pure# shellsmkdir debugcp$src debugcd debugtar -xzfv whi3jprrpzlnvic9fsn5f69sddazp5sb-colorify-0.2.3.tar.gzcd<unpacked source>

You will need to override yourCargo.toml andCargo.lock in this shell, somake sure that you have them backed up if your are directly using your clone ofyour project instead of unpacking fresh sources like above.

Now you just need to run the$configurePhase and$buildPhase steps in order.You can find additional phases that may exist in overrides by runningenv | grep Phase

echo$configurePhase# runHook preConfigure runHook configureCargo runHook postConfigurerunHook preConfigure# usually does nothingrunHook findCraterunHook configureCargorunHook postConfigure# usually does nothingecho$buildPhase# runHook overrideCargoManifest runHook setBuildEnv runHook runCargorunHook overrideCargoManifest# This overrides your .cargo folder, e.g. for setting cross-compilersrunHook setBuildEnv# This sets up linker flags for the `rustc` invocationsrunHook runCargo

IfrunCargo succeeds, you will have a completed output ready for the (usually)less interesting$installPhase. If there's a problem, inspecting theenv orreading the generatedCargo.lock etc should yield clues. If you've unpacked afresh source and are using the--ignore-environment switch, everything isidentical to how the overlay builds the crate, cutting out guess work.

SeeContributing for potentially more information.

1. Fork this repository into the personal GitHub account
2. Select the appropriate branch, release- for stable changes, unstablefor breaking changes
3. Make changes on the personal fork
4. Make a Pull Request against this repository
5. Allow maintainers to make changes to your pull request (there's a checkbox)
6. Once the pull request has been approved, you will be thanked and observe yourchanges applied with authorship preserved (if we remember)

The design for the Nix overlay is inspired by the excellent work done by JamesKay, which is describedhere andhere. His source isavailablehere. This work would have been impossible withoutthese fantastic write-ups. Special thanks to James Kay!

cargo2nix is free and open source software distributed under the terms of theMIT License.