So you've read theThreading and Tasks documentation, surveyed the associatedThreading and Tasks FAQ and have implemented a state-of-the-art component. Now you want to test it :). This document will explain how to write matching state-of-the-art tests.
In order tounit test a component which post tasks, you'll need to bring up the task environment in the scope of your test (or test fixture). It will need to outlive the majority of other members to ensure they have access to the task system throughout their lifetime. There are a rare exceptions, likebase::test::ScopedFeatureList, that need to outlive the task environment. For browser tests, see theBrowser tests section below.
Task environments come in various forms but share the same fundamental characteristics:
base::test::TaskEnvironment and support itsValidTraits and sometimes more.TheTaskEnvironment member is typically exposed in the protected section of the test fixture to allow tests to drive it directly (there's no need to expose public Run*() methods that merely forward to the private member).
Your component usesbase::SingleThreadTaskRunner::GetCurrentDefault() orbase::SequencedTaskRunner::GetCurrentDefault() to post tasks to the thread it was created on? You'll need at least abase::test::SingleThreadTaskEnvironment in order for these APIs to be functional andbase::RunLoop to run the posted tasks.
Typically this will look something like this:
foo.h
classFoo{public:Foo(): owning_sequence_(base::SequencedTaskRunner::GetCurrentDefault()){}DoSomethingAndReply(base::OnceClosure on_done){ DCHECK(owning_sequence_->RunsTasksInCurrentSequence()); something_was_done_=true; owning_sequence_->PostTask(on_done);}bool something_was_done()const{return something_was_done_;}private:bool something_was_done_=false; scoped_refptr<base::SequencedTaskRunner> owning_sequence_;};
foo_unittest.cc
TEST(FooTest,DoSomething){base::test::SingleThreadTaskEnvironment task_environment;Foo foo;RunLoop run_loop; foo.DoSomethingAndReply(run_loop.QuitClosure()); run_loop.Run(); EXPECT_TRUE(foo.something_was_done());}
Note thatRunLoop().RunUntilIdle() could be used instead of aQuitClosure() above butbest practices favor QuitClosure() over RunUntilIdle() as the latter can lead to flaky tests.
If your components depends onbase::ThreadPool (that‘s a good thing!), you’ll need a fullbase::test::TaskEnvironment. Don‘t be afraid to use a fullTaskEnvironment when appropriate: think of “SingleThread” as being a readability term like “const”, it documents that ThreadPool isn’t used when it‘s not but you shouldn’t be afraid to lift it.
Task runners are still obtained by the product code throughbase/task/thread_pool.h without necessitating a test-only task runner injection seam :).
Typical use case:
foo_service.h
classFooService{public:FooService(): backend_task_runner_(base::ThreadPool::CreateSequencedTaskRunner({base::MayBlock(),base::TaskPriority::BEST_EFFORT})), backend_(newFooBackend,base::OnTaskRunnerDeleter(backend_task_runner_)){}// Flushes state to disk async and replies.FlushAndReply(base::OnceClosure on_done){ DCHECK(owning_sequence_->RunsTasksInCurrentSequence()); backend_task_runner_->PostTaskAndReply(FROM_HERE,base::BindOnce(&FooBackend::Flush,Unretained(backend_.get()), std::move(on_done)));}private: scoped_refptr<base::SequencedTaskRunner> backend_task_runner_;// See https://youtu.be/m6Kz6pMaIxc?t=882 for memory management best// practices. std::unique_ptr<FooBackend,base::OnTaskRunnerDeleter> backend_;};
foo_service_unittest.cc
TEST(FooServiceTest,FlushAndReply){base::test::TaskEnvironment task_environment;FooService foo_service;RunLoop run_loop; foo_service.FlushAndReply(run_loop.QuitClosure()); run_loop.Run(); EXPECT_TRUE(VerifyFooStateOnDisk());}
This is the same thing asbase::test::TaskEnvironment with the addition ofcontent::BrowserThread support. You need this if-and-only-if the code under test is usingBrowserThread::UI orBrowserThread::IO. For determinism, both BrowserThreads will share the main thread and can be driven by RunLoop. By default the main thread will useMainThreadType::UI but you can override this via theMainThreadType trait to ask for an IO pump.
BrowserTaskEnvironment::REAL_IO_THREAD can be also used as a construction trait for rare instances that desire distinct physical BrowserThreads.
This is the //ios equivalent ofcontent::BrowserTaskEnvironment to simulateweb::WebThread.
This is the same thing as base::test::TaskEnvironment with the addition of blink::MainThreadScheduler and blink::MainThreadIsolate support. You need this if-and-only-if the code under test is using blink::Thread::Current() or needs v8::Isolate::GetCurrent() to be a blink Isolate.
All task environments support the following methods to run tasks:
base::RunLoop:Run(): run the main thread until theQuitClosure() is invoked (note: other threads also run in parallel by default).base::RunLoop::RunUntilIdle(): run the main thread until it is idle. This is typically not what you want in multi-threaded environments as it may resume beforeThreadPool is idle.TaskEnvironment::RunUntilIdle(): Runs everything the TaskEnvironment is aware of. This excludes system events and any threads outside of the main thread and ThreadPool. It should be used with care when such external factors can be involved.TaskEnvironment::FastForward*(): More on this in the TimeSource section below.By default tests run underTimeSource::SYSTEM_TIME which means delays are real-time andbase::Time::Now() andbase::TimeTicks::Now() return live system times (context).
Whenever testing code with delays, you should favorTimeSource::MOCK_TIME as a trait. This makes it such that delayed tasks andbase::Time::Now() +base::TimeTicks::Now() use a mock clock.
Under this mode, the mock clock will start at the current system time but will then only advance when explicitly requested byTaskEnvironment::FastForward*() andTaskEnvironment::AdvanceClock() methodsor whenRunLoop::Run() is running and all managed threads become idle (auto-advances to the soonest delayed task, if any, amongst all managed threads).
TaskEnvironment::FastForwardBy() repeatedly runs existing immediately executable tasks until idle and then advances the mock clock incrementally to run the next delayed task within the time delta. It may advance time by more than the requested amount if running the tasks causes nested time-advancing-method calls.
This makes it possible to test code with flush intervals, repeating timers, timeouts, etc. without any test-specific seams in the product code, e.g.:
foo_storage.h
classFooStorage{public:staticconstexprbase::TimeDelta::kFlushInterval=base::Seconds(30);// Sets |key| to |value|. Flushed to disk on the next flush interval.voidSet(base::StringPiece key,base::StringPiece value);};
foo_unittest.cc
classFooStorageTest{public:FooStorageTest()=default;// Test helper that returns true if |key| is found in the on disk storage.boolFindKeyInOnDiskStorage(base::StringPiece key);protected:base::test::TaskEnvironment task_environment{base::test::TaskEnvironment::TimeSource::MOCK_TIME};FooStorage foo_storage_;};TEST_F(FooStorageTest,Set){ foo_storage_.Set("mykey","myvalue"); EXPECT_FALSE(FindKeyInOnDiskStorage("mykey")); task_environment.FastForwardBy(FooStorage::kFlushInterval); EXPECT_TRUE(FindKeyInOnDiskStorage("mykey"));}
In contrast,TaskEnvironment::AdvanceClock() simply advances the mock time by the requested amount, and does not run tasks. This may be useful in cases whereTaskEnvironment::FastForwardBy() would result in a livelock. For example, if one task is blocked on aWaitableEvent and there is a delayed task that would signal the event (e.g., a timeout), thenTaskEnvironment::FastForwardBy() will never complete. In this case, you could advance the clock enough that the delayed task becomes runnable, and thenTaskEnvironment::RunUntilIdle() would run the delayed task, signalling the event.
TEST(FooTest,TimeoutExceeded){base::test::TaskEnvironment task_environment{base::test::TaskEnvironment::TimeSource::MOCK_TIME};base::WaitableEventevent;base::RunLoop run_loop;base::ThreadPool::PostTaskAndReply( FROM_HERE,{base::MayBlock()},base::BindOnce(&BlocksOnEvent,base::Unretained(&event)), run_loop.QuitClosure());base::ThreadPool::PostDelayedTask( FROM_HERE,{},base::BindOnce(&WaitableEvent::Signal,base::Unretained(&event)), kTimeout);// Can't use task_environment.FastForwardBy() since BlocksOnEvent blocks// and the task pool will not become idle.// Instead, advance time until the timeout task becomes runnable. task_environment.AdvanceClock(kTimeout);// Now the timeout task is runable. task_environment.RunUntilIdle();// The reply task should already have been executed, but run the run_loop to// verify. run_loop.Run();}
The average component only cares about running its tasks andMainThreadType::DEFAULT is sufficient. Components that care to interact asynchronously with the system will likely need aMainThreadType::UI to be able to receive system events (e.g,. UI or clipboard events).
Some components will prefer a main thread that handles asynchronous IO events and will useMainThreadType::IO. Such components are typically the ones living on BrowserThread::IO and being tested with aBrowserTaskEnvironment initialized withMainThreadType::IO.
Note: This is strictly about requesting a specificMessagePumpType for the main thread. It has nothing to do withBrowserThread::UI orBrowserThread::IO which are named threads in the //content/browser code.
By default non-delayed tasks posted tobase::ThreadPool may run at any point. Tests that require more determinism can requestThreadPoolExecutionMode::QUEUED to enforce that tasks posted tobase::ThreadPool only run whenTaskEnvironment::RunUntilIdle() orTaskEnvironment::FastForward*() are invoked. Note thatRunLoop::Run() doesnot unblock the ThreadPool in this mode and thus strictly runs only the main thread.
WhenThreadPoolExecutionMode::QUEUED is mixed withTimeSource::MOCK_TIME, time will auto-advance to the soonest taskthat is allowed to run when required (i.e. it will ignore delayed tasks in the thread pool while inRunLoop::Run()). SeeTaskEnvironmentTest.MultiThreadedMockTimeAndThreadPoolQueuedMode for an example.
This trait is of course irrelevant underSingleThreadTaskEnvironment.
Prefer an explicitSingleThreadTaskEnvironment over usingThreadingMode::MAIN_THREAD_ONLY. The only reason to useThreadingMode::MAIN_THREAD_ONLY explicitly is if the parent class of your test fixture manages theTaskEnvironment but takesTaskEnvironmentTraits to let its subclasses customize it and you really need aSingleThreadTaskEnvironment.
In some cases it makes sense to have the base fixture of an entire section of the codebase be managing theTaskEnvironment (e.g.ViewsTestBase). It's useful if such base fixture exposesTaskEnvironmentTraits to their subclasses so that individual tests within that domain can fine-tune their traits as desired.
This typically looks like this (in this caseFooTestBase opts to enforceMainThreadType::UI and leaves other traits to be specified as desired):
// Constructs a FooTestBase with |traits| being forwarded to its// TaskEnvironment. MainThreadType always defaults to UI and must not be// specified.template<typename...TaskEnvironmentTraits>NOINLINEexplicitFooTestBase(TaskEnvironmentTraits&&... traits): task_environment_(base::test::TaskEnvironment::MainThreadType::UI, std::forward<TaskEnvironmentTraits>(traits)...){}
Note, if you're not familiar with traits: TaskEnvironment traits usebase/traits_bag.h and will automatically complain at compile-time if an enum-based trait is specified more than once (i.e. subclasses will not compile if re-specifyingMainThreadType in the above example).
This is all nice and fancy for unit tests, but what about browser_tests, ui_integration_tests, etc? Tests that subclasscontent::BrowserTestBase bring up the entire environment (tasks & more) by default.
The downside is that you don't have fine-grained control over it like you would with all theTaskEnvironment methods.
The favored paradigm isRunLoop::Run() +QuitClosure(). The asynchronous nature of Chromium code makes this the most reliable way to wait for an event.
There are fancy versions of this to perform common actions, e.g.content/public/test/browser_test_utils.hcontent/public/test/content_browser_test_utils.h which will let you navigate, execute scripts, simulate UI interactions, etc.
But the idea is always the same :
RunLoop run_loop;run_loop.QuitClosure()run_loop.Run() until theQuitClosure() is called.So you fell in love withTimeSource::MOCK_TIME but now you're in a browser test... yeah, sorry about that...
The eventual goal is to make it possible to set up TaskEnvironmentTraits from your test fixture just like you can override command-line, etc. but we're not there yet...
In the mean time you can still use the oldbase::ScopedMockTimeMessageLoopTaskRunner to mock delayed tasks on the main thread (you're out of luck on other threads for now). And you can usebase::subtle::ScopedTimeClockOverrides if you want to overrideNow().
You think that‘s a mess? Just think that it used to be this way in unit tests too and you’ll be happy again :).
Here are some paradigms you might see throughout the code base and some insight on what to do about them (hint: copying them is not one!). Migration help is welcomecrbug.com/984323!
This is the ancestor ofSingleThreadTaskEnvironment +TimeSource::MOCK_TIME. It's sort of equivalent but prefer task environments for consistency.
The only case wherebase::TestMockTimeTaskRunner is still the only option is when writing regression tests that simulate a specific task execution order across multiple sequences. To do so, use twobase::TestMockTimeTaskRunner and have the racing components post their tasks to separate ones. You can then explicitly run tasks posted to each one from the main test thread in a way that deterministically exercises the race resolution under test. This only applies to task execution order races, data races still require parallel execution and this is the main reasonTaskEnvironment doesn't multiplex theThreadPool tasks onto the main thread (i.e. exercise data races, especially in the scope of TSAN).
Prefer usingSingleThreadTaskEnvironment overbase::TestSimpleTaskRunner.TestSimpleTaskRunner isn‘t as “simple” as it seems specifically because it runs tasks in a surprising order (delays aren’t respected and nesting doesn't behave as usual). Should you prefer to flush all tasks regardless of delays,TimeSource::MOCK_TIME andTaskEnvironment::FastForwardUntilNoTasksRemain() have you covered.
PreferSingleThreadTaskEnvironment orTaskEnvironment withThreadPoolExecutionMode::QUEUED overbase::NullTaskRunner. ANullTaskRunner might seem appealing, but not posting tasks is under-testing the potential side-effects of the code under tests. All tests should be okay if tasks born from their actions are run or deleted at a later point.
This is the ancestor ofbase::TestMockTimeTaskRunner which is itself mostly deprecated. As mentioned above in theTimeSource trait section: This should never be used anymore except to mock time when there's already a task system in place, e.g. in browser_tests.
Prior toTaskEnvironment::TimeSource::MOCK_TIME, many components hadSetClockForTesting() in their product code. And before modernThreading and Tasks, some components had SetTaskRunnerForTesting(). Neither of these test-only seams are required anymore now that task environments can mock those from under-the-hood. Cleanup in favor of modern TaskEnvironment paradigms is always appreciated (crbug.com/984323).
Different parts of the codebase have their own helper task runners. Please migrate away from them or document them above. Ultimately the goal is forTaskEnvironment and its subclasses to rule them all and to have a consistent task testing API surface once and for all.
It is still okay for specific areas to have a base fixture that configures a defaultTaskEnvironment appropriate for that area and use theTaskEnvironmentTraits paradigm outlined in theBase Fixture managed TaskEnvironment section above to let individual tests provide additional traits.