Repository files navigation

• Overview
• Getting Started
• Configuration
• External Packages
• Examples
• Contributing
• Known Issues

• r_pkg
• r_library
• r_unit_test
• r_pkg_test
• r_binary
• r_test
• r_markdown
• r_toolchain

• r_repository
• r_repository_list
• r_rules_dependencies
• r_coverage_dependencies
• r_register_toolchains

• r_library_image
• r_binary_image

These rules are used for buildingR packages with Bazel. Although R has anexcellent package management system, there is no continuous build andintegration system for entire R package repositories. An advantage of usingBazel, over a custom solution of tracking the package dependency graph andtriggering builds accordingly on each commit, is that R packages can be builtand tested as part of one build system in multi-language monorepos.

These rules are mature for production use.

The following assumes that you are familiar with how to use Bazel in general.

To begin, you can add the following or equivalent to your WORKSPACE file:

load("@bazel_tools//tools/build_defs/repo:http.bzl","http_archive")# Change master to the git tag you want.http_archive(name="rules_r",strip_prefix="rules_r-master",urls= ["https://github.com/grailbio/rules_r/archive/master.tar.gz"],)load("@rules_r//R:dependencies.bzl","r_register_toolchains","r_rules_dependencies")r_rules_dependencies()r_register_toolchains()

You can load the rules in your BUILD file like so:

load("@rules_r//R:defs.bzl","r_pkg","r_library","r_unit_test","r_pkg_test")

Advanced users can also set upGazelle to maintain the BUILD filesfor the R packages in their repo automatically.

The following software must be installed on your system:

1. bazel (v5.0.0 or above)2. R (4.1.2 or above; should be locatable using the `PATH` environment variable)

NOTE: After re-installing or upgrading R, please reset the registeredtoolchain withbazel sync --configure to rebuild your packages with the newinstallation.

NOTE: It is possible to use R from a bazel package instead of a systeminstallation. See the toolchainr-toolchain-nix in the tests directory as anexample.

For each package, you can also specify a different Makevars file that can beused to have finer control over native code compilation. The site-wideMakevars files are configured by default in the toolchains, and these definethe compiler toolchain to use and the flags needed for these toolchains forreproducible builds.

FormacOS, this setup will help you cover the requirements for a large numberof packages:

brew install gcc pkg-config icu4c openssl

ForUbuntu, this (or equivalent for other Unix systems) helps:

apt-get install pkgconf libssl-dev libxml2-dev libcurl4-openssl-dev

For no interference from other packages during the build (possibly otherversions installed manually by the user), it is recommended that packages otherthan those with recommended priority be installed in the directory pointed tobyR_LIBS_USER. The Bazel build process will then be able to hide all theother packages from R by setting a different value forR_LIBS_USER.

When moving to Bazel for installing R packages on your system, we recommendcleaning up existing machines:

sudo Rscript \  -e 'options("repos"="https://cloud.r-project.org")' \  -e 'lib <- c(.Library, .Library.site)' \  -e 'non_base_pkgs <- installed.packages(lib.loc=lib, priority=c("recommended", "NA"))[, "Package"]' \  -e 'remove.packages(non_base_pkgs, lib=lib)'# If not set up already, create the directory for R_LIBS_USER.Rscript \  -e 'dir.create(Sys.getenv("R_LIBS_USER"), recursive=TRUE, showWarnings=FALSE)'

For more details on how R searches different paths for packages, seelibPaths.

To depend on external packages from CRAN and other remote repos, you can define thepackages as a CSV with three columns -- Package, Version, and sha256. Then user_repository_list rule to define R repositories for eachpackage. For packages not in a CRAN like repo (e.g. github), you can user_repository rule directly. For packages on your local systembut outside your main repository, you will have to uselocal_repository witha saved BUILD file. Same for VCS repositories.

load("@rules_r//R:repositories.bzl", "r_repository", "r_repository_list")# R packages with non-standard sources.r_repository(    name = "R_plotly",    sha256 = "24c848fa2cbb6aed6a59fa94f8c9b917de5b777d14919268e88bff6c4562ed29",    strip_prefix = "plotly-a60510e4bbce5c6bed34ef6439d7a48cb54cad0a",    urls = [        "https://github.com/ropensci/plotly/archive/a60510e4bbce5c6bed34ef6439d7a48cb54cad0a.tar.gz",    ],)# R packages with standard sources.# See below for an example of how to generate the CSV package_list.r_repository_list(    name = "r_repositories_bzl",    build_file_overrides = "@myrepo//third-party/R:build_file_overrides.csv",    package_list = "@myrepo//third-party/R:packages.csv",    remote_repos = {        "BioCsoft": "https://bioconductor.org/packages/3.14/bioc",        "BioCann": "https://bioconductor.org/packages/3.14/data/annotation",        "BioCexp": "https://bioconductor.org/packages/3.14/data/experiment",        "CRAN": "https://cloud.r-project.org",    },)load("@r_repositories_bzl//:r_repositories.bzl", "r_repositories")r_repositories()

The list of all external R packages configured this way can be obtained fromyour shell with

$ bazel query 'filter(":R_", //external:*)'

NOTE: Periods ('.') in the package names are replaced with underscores('_') because bazel does not allow periods in repository names.

To generate and maintain a CSV file containing all your external dependenciesfor use withr_repository_list, you can use the functions in the scriptrepo_management.R.

For example:

script="/path/to/rules_r/scripts/repo_management.R"package_list_csv="/path/to/output/csv/file"packages="comma-separated list of packages you want to add to the local cache"bioc_version="bioc_version to use, e.g. 3.11"# This will be the cache directory for a local copy of all the packages.# The output CSV will always reflect the state of this directory.local_r_repo="${HOME}/.cache/r-repo"Rscript -<<EOFsource('${script}')pkgs <- strsplit('${packages}', ',')[[1]]# Set ForceDownload to TRUE when switching R or Bioc versions.# options("ForceDownload" = TRUE)# Keep in sync with r_repository_list in WORKSPACE.options(repos = c(    BioCsoft = "https://bioconductor.org/packages/${bioc_version}/bioc",    BioCann = "https://bioconductor.org/packages/${bioc_version}/data/annotation",    BioCexp = "https://bioconductor.org/packages/${bioc_version}/data/experiment",    CRAN = "https://cloud.r-project.org"))addPackagesToRepo(pkgs, repo_dir = '${local_r_repo}')packageList('${local_r_repo}', '${package_list_csv}')EOF

Some examples are available in the tests directory of this repo.

• Seetests/packages/exampleA for a barebones R package.
• Seetests/packages/exampleB for a barebones R package that depends on another package.
• Seetests/packages/exampleC for an R package that depends on external R packages anduser native code libraries.
• Seetests/packages/exampleD for an R package that depends on external R packages whichhave been built with bazel provided native code libraries instead of a system requirement.

Also seeRazel scripts that provide utility functions to generateBUILD filesandWORKSPACE rules.

Contributions are most welcome. Please submit a pull request giving the ownersof this github repo access to your branch for minor style related edits, etc. We recommendopening an issue first to discuss the nature of your change before beginning work on it.

Please check open issues at the github repo.

r_pkg(srcs,pkg_name,deps,cc_deps,build_args,install_args,config_override,roclets,roclets_deps,makevars,env_vars,inst_files,tools,build_tools,metadata)

Rule to install the package and its transitive dependencies in the Bazelsandbox, so it can be depended upon by other package builds.

The builds produced from this rule are tested to be byte-for-byte reproduciblewith the same R installation. For native code compilation, the compiler flagsfor reproducibility are defined in the default site Makevars file in the localtoolchain. If using your own toolchain, ensure that your site Makevars file hasthe right flags.

r_library(pkgs,library_path)

Executable rule to install the given packages and all dependencies to a userprovided or system default R library. Run the target with --help for usageinformation.

The rule used to provide a tar archive of the library as an implicit output.That feature is now it's own rule --r_library_tar. See documentation forr_library_tar rule andexample usage forcontainer_image rule.

r_unit_test(pkg,suggested_deps,env_vars,tools,data)

Rule to keep all deps in the sandbox, and run the provided R test scripts.

When run withbazel coverage, this rule will also produce a coverage reportin Cobertura XML format. The coverage report will contain coverage for R codein the package, and C/C++ code in thesrc directory of R packages.

r_pkg_test(pkg,suggested_deps,check_args,env_vars,tools,data)

Rule to keep all deps of the package in the sandbox, build a source archiveof this package, and run R CMD check on the package source archive in thesandbox.

r_binary(name,src,deps,data,env_vars,tools,rscript_args,script_args)

Build a wrapper shell script for running an executable which will have all thespecified R packages available.

The target can be executed standalone, withbazel run, or called from otherexecutables ifRUNFILES_DIR is exported in the environment withthe runfiles of the root executable.

r_test(name,src,deps,data,env_vars,tools,rscript_args,script_args)

This is identical tor_binary but is run as a test.

r_markdown(name,src,deps,data,env_vars,tools,rscript_args,script_args,render_function="rmarkdown::render",input_argument="input",output_dir_argument="output_dir",render_args)

This rule renders an R markdown through generating a stub to call the renderfunction. The render function and the argument names for the function aredefault set forrmarkdown::render but can be customized. Note thatrender_args will need to be quoted appropriately if set. This rule can beused wherever anr_binary rule can be used.

If arguments are given on the command line when running the target, flags ofthe form --arg=value are passed as keyword arguments to the renderfunction. The values can be arbitrary R expressions, and strings will need tobe quoted. The last argument without the prefix-- will be the outputdirectory, else the output directory will be the default outputdirectory of the render function, typically the same directory as the inputfile.

r_toolchain(r,rscript,version,args,makevars_site,env_vars,tools,files,system_state_file)

Toolchain to specify the tools and environment for performing build actions.Also seer_register_toolchains for howto configure the default registered toolchains.

r_repository(urls,strip_prefix,type,sha256,build_file,rscript)

Repository rule in place ofnew_http_archive that can run razel to generatethe BUILD file automatically. See section onexternal packages andRazel scripts.

r_repository_list(package_list,build_file_overrides,remote_repos,other_args,rscript)

Repository rule that will generate a bzl file containing a macro, to be calledasr_repositories(), forr_repository definitions for packages inpackage_list CSV. See section onexternal packages.

load("@rules_r//R:dependencies.bzl","r_rules_dependencies")r_rules_dependencies()

Repository rule that provides repository definitions for dependencies of theBUILD system. One such dependency is the site-wide Makevars file.

load("@rules_r//R:dependencies.bzl","r_coverage_dependencies")r_coverage_dependencies()load("@r_coverage_deps_bzl//:r_repositories.bzl",coverage_deps="r_repositories")coverage_deps()

Repository rule that provides repository definitions for dependencies incomputing code coverage for unit tests. Not needed if users already havea repository definition for thecovr package.

load("@rules_r//R:dependencies.bzl","r_register_toolchains")r_register_toolchains(r_home,strict,makevars_site,version,args,tools)

Repository rule that generates and registers a platform independent toolchainof typer_toolchain based on the user's system andenvironment. If you want to register your own toolchain for specific platforms,register them before calling this function in your WORKSPACE file to give thempreference.

NOTE: These toolchains read your system state and cache the findings forfuture runs. Whenever you install a new R version, or if you want to reset thetoolchain for any reason, run:

bazel sync --configure