Mocking allows you to temporarily replace the implementation of afunction with something that makes it easier to test. It’s useful whentesting failure scenarios that are hard to generate organically (e.g.,what happens if dependency X isn’t installed?), making tests morereliable, and making tests faster. It’s also a general escape hatch toresolve almost any challenging testing problem. That said, mocking comeswith downsides too: it’s an advanced technique that can lead to brittletests or tests that silently conceal problems. You should only use itwhen all other approaches fail.
(If, like me, you’re confused as to why you’d want to cruelly makefun of your tests, mocking here is used in the sense of making a fake orsimulated version of something, i.e., a mock-up.)
testthat’s primary mocking tool islocal_mocked_bindings() which is used to mock functions andis the focus of this vignette. But it also provides other tools forspecialized cases: you can uselocal_mocked_s3_method() tomock an S3 method,local_mocked_s4_method() to mock an S4method, andlocal_mocked_r6_class() to mock an R6 class.Once you understand the basic idea of mocking, it should bestraightforward to apply these other tools where needed.
In this vignette, we’ll start by illustrating the basics of mockingwith a few examples, continue to some real-world case studies fromthroughout the tidyverse, then finish up with the technical details soyou can understand the tradeoffs of the current implementation.
Let’s begin by motivating mocking with a simple example. Imagineyou’re writing a function likerlang::check_installed().The goal of this function is to check if a package is installed, and ifnot, give a nice error message. It also takes an optionalmin_version argument that you can use to enforce a versionconstraint. A simple base R implementation might look something likethis:
check_installed<-function(pkg,min_version =NULL) {if (!requireNamespace(pkg,quietly =TRUE)) {stop(sprintf("{%s} is not installed.", pkg)) }if (!is.null(min_version)) { pkg_version<-packageVersion(pkg)if (pkg_version< min_version) {stop(sprintf("{%s} version %s is installed, but %s is required.", pkg, pkg_version, min_version )) } }invisible()}Now that we’ve written this function, we want to test it. There aremany ways we might tackle this, but it’s reasonable to start by testingthe case where we don’t specify a minimum version. To do this, we needto come up with a package we know is installed and a package we knowisn’t installed:
test_that("check_installed() checks package is installed", {expect_no_error(check_installed("testthat"))expect_snapshot(check_installed("doesntexist"),error =TRUE)})#> ── Warning: check_installed() checks package is installed ──────────────────────#> Adding new snapshot:#> Code#> check_installed("doesntexist")#> Condition#> Error in `check_installed()`:#> ! {doesntexist} is not installed.#> Test passed with 2 successes 🎉.This is probably fine as we certainly know that testthat must beinstalled but it feels a little fragile as it depends on external statethat we don’t control. While it’s pretty unlikely, if someone doescreate adoesntexist package, this test will no longerwork. As a general principle, the less your tests rely on state outsideof your control, the more robust and reliable they’ll be.
Next we want to check the case where we specify a minimum version,and again we need to make up some inputs:
test_that("check_installed() checks minimum version", {expect_no_error(check_installed("testthat","1.0.0"))expect_snapshot(check_installed("testthat","99.99.999"),error =TRUE)})#> ── Warning: check_installed() checks minimum version ───────────────────────────#> Adding new snapshot:#> Code#> check_installed("testthat", "99.99.999")#> Condition#> Error in `check_installed()`:#> ! {testthat} version 3.3.1 is installed, but 99.99.999 is required.#> Test passed with 2 successes 🥳.Again, this is probably safe (since I’m unlikely to release 90+ newversions of testthat), but if you look at the snapshot messagecarefully, you’ll notice that it includes the current version oftestthat. That means every time a new version of testthat is released,we’ll have to update the snapshot. We could use thetransform argument to fix this:
test_that("check_installed() checks minimum version", {expect_no_error(check_installed("testthat","1.0.0"))expect_snapshot(check_installed("testthat","99.99.999"),error =TRUE,transform =function(lines)gsub(packageVersion("testthat"),"<version>", lines) )})#> ── Warning: check_installed() checks minimum version ───────────────────────────#> Adding new snapshot:#> Code#> check_installed("testthat", "99.99.999")#> Condition#> Error in `check_installed()`:#> ! {testthat} version <version> is installed, but 99.99.999 is required.#> Test passed with 2 successes 🎉.But it’s starting to feel like we’ve accumulating more and morehacks. So let’s take a fresh look and see how mocking might help us. Thebasic idea of mocking is to temporarily replace the implementation offunctions being used by the function we’re testing. Here we’re testingcheck_installed() and want to mockrequireNamespace() andpackageVersion() so wecan control their versions. There’s a small wrinkle here in thatrequireNamespace andpackageVersion are basefunctions, not our functions, so we need to make bindings in our packagenamespace so we can mock them (we’ll come back to why later).
For the first test, we mockrequireNamespace() twice:first to always returnTRUE (pretending every package isinstalled), and then to always returnFALSE (pretendingthat no packages are installed). Now the test is completelyself-contained and doesn’t depend on what packages happen to beinstalled.
test_that("check_installed() checks package is installed", {local_mocked_bindings(requireNamespace =function(...)TRUE)expect_no_error(check_installed("package-name"))local_mocked_bindings(requireNamespace =function(...)FALSE)expect_snapshot(check_installed("package-name"),error =TRUE)})#> ── Warning: check_installed() checks package is installed ──────────────────────#> Adding new snapshot:#> Code#> check_installed("package-name")#> Condition#> Error in `check_installed()`:#> ! {package-name} is not installed.#> Test passed with 2 successes 🎊.For the second test, we mockrequireNamespace() toreturnTRUE, and thenpackageVersion() toalways return version 2.0.0. This again ensures our test is independentof system state.
test_that("check_installed() checks minimum version", {local_mocked_bindings(requireNamespace =function(...)TRUE,packageVersion =function(...)numeric_version("2.0.0") )expect_no_error(check_installed("package-name","1.0.0"))expect_snapshot(check_installed("package-name","3.4.5"),error =TRUE)})#> ── Warning: check_installed() checks minimum version ───────────────────────────#> Adding new snapshot:#> Code#> check_installed("package-name", "3.4.5")#> Condition#> Error in `check_installed()`:#> ! {package-name} version 2.0.0 is installed, but 3.4.5 is required.#> Test passed with 2 successes 🎉.To give you more experience with mocking, this section looks at a fewplaces where we use mocking in the tidyverse:
testthat::skip_on_os() regardless of whatoperating system is running the test.usethis::use_release_issue().httr2::req_throttle().These situations are all a little complex, as this is the nature ofmocking: if you can use a simpler technique, you should. Mocking is onlyneeded for otherwise intractable problems.
testthat::skip_on_os() allows you to skip tests onspecific operating systems, using the internalsystem_os()function which is a thin wrapper aroundSys.info()[["sysname"]]. To test that this skip workscorrectly, we have to use mocking because there’s no other way topretend we’re running on a different operating system. This yields thefollowing test, where we using mocking to pretend that we’re always onWindows:
test_that("can skip on multiple oses", {local_mocked_bindings(system_os =function()"windows")expect_skip(skip_on_os("windows"))expect_skip(skip_on_os(c("windows","linux")))expect_no_skip(skip_on_os("linux"))})(The logic ofskip_on_os() is simple enough that I feelconfident we only need to simulate one platform.)
usethis::use_release_issue() creates a GitHub issue witha bulleted list of actions to follow when releasing a package. But someof the bullets depend on complex conditions that can take a while tocompute. So thetestsfor this function use mocks like this:
Here we pretend that there are no reverse dependencies (revdeps) forthe package, which is both slow to compute and will vary over time if weuse a real package. We also pretend that there are no related GitHubmilestones, which otherwise requires an GitHub API call, which is againslow and might vary over time. Together, these mocks keep the tests fastand self-contained, free from any state outside of our directcontrol.
httr2::req_throttle() prevents multiple requests frombeing made too quickly, using a technique called a leaky token bucket.This technique is inextricably tied to real time because you want toallow more requests as time elapses. So how do you test this? I startedby usingSys.sleep(), but this made my tests both slow(because I’d sleep for a second or two) and unreliable (becausesometimes more time elapsed than I expected). Eventually I figured outthat I could “manually control” time by using amockedfunction that returns the value of a variable I control. This allowsme to manually advance time and carefully test the implications.
You can see the basic idea with a simpler example. Let’s first beginwith a function that returns the “unix time”, the number of secondselapsed since midnight on Jan 1, 1970. This is easy to compute, but willmake some computations simpler later as well as providing a convenientfunction to mock.
Now I’m going to create a function factory that makes it easy tocompute how much time has elapsed since some fixed starting point:
elapsed<-function() { start<-unix_time()function() {unix_time()- start }}timer<-elapsed()Sys.sleep(0.5)timer()#> [1] 0.505657Imagine trying to test this function without mocking! You’d probablythink it’s not worth it. In fact, that’s what I thought originally, butI soon learned my lesson because I introduce bug because I’d forgottenthe complexities of computing the difference between two POSIXctvalues.
With mocking, however, I can “manipulate time” by mockingunix_time() so that it returns the value of a variable Icontrol. Now I can write a reliable test:
To finish up, it’s worth discussing how mocking works. Thefundamental challenge of mocking is that you want it to be “hygienic”,i.e. it should only affect the operation of your package code, not allrunning code. You can see why this might be problematic if you imaginemocking a function that testthat itself uses: you don’t want toaccidentally break testthat while trying to test your code! To achievethis goal,local_mocked_bindings() works by modifying yourpackage’snamespaceenvironment.
You can implement the basic idea using base R code like this:
old<-getFromNamespace("my_function","mypackage")assignInNamespace("my_function", new,"mypackage")# run the test...# restore the previous valueassignInNamespace("my_function", old,"mypackage")This implementation leads to two limitations oflocal_mocked_bindings():
The package namespace is locked, which means that you can’t addnew bindings to it. That means if you want to mock base functions, youhave to provide some binding that can be overridden. The easiest way todo this is with something likemean <- NULL. Thiscreates a binding thatlocal_mocked_bindings() can modify,but because of R’slexicalscoping rules doesn’t affect ordinary calls.
:: doesn’t use the package namespace, so if you wantto mock an explicitly namespaced function, you either have importfun into yourNAMESPACE (e.g., with@importFrom pkg fun) or create your own wrapper functionthat you can mock. Typically, one of these options will feel fairlynatural.
Overall, these limitations feel correct to me:local_mocked_bindings() makes it easy to temporarily changethe implementation of functions that you have written, while offeringworkarounds to override the implementations of functions that othershave written in the scope of your package.