Running tests requires root access, which requires using a userdebug build on your device.
To use a userdebug build, seeRunning Builds. Googlers can refer tothis page.
If you can't run “adb root”, you will get an error when trying to install the test APKs like “adb: error: failed to copy” and “remote secure_mkdirs failed: Operation not permitted” (use “adb unroot” to return adb to normal).
The adb executable exists within the Android SDK:
third_party/android_sdk/public/platform-tools/adb
In order to allow the ADB to connect to the device, you must enable USB debugging:
Youmust ensure that the screen stays on while testing:adb shell svc power stayon usb Or do this manually on the device: Settings -> Developer options -> Stay Awake.
If this option is greyed out, stay awake is probably disabled by policy. In that case, get another device or log in with a normal, unmanaged account (because the tests will break in exciting ways if stay awake is off).
You may see a dialog likethis one, which states,Google may regularly check installed apps for potentially harmful behavior. This can interfere with the test runner. To disable this dialog, run:
adb shell settings putglobal package_verifier_enable0
Running tests on emulators is the same ason device. Refer toandroid_emulator.md for setting up emulators.
If you‘re adding a new test file, you’ll need to explicitly add it to a gn target. If you‘re adding a test to an existing file, you won’t need to make gn changes, but you may be interested in where your test winds up. In either case, here are some guidelines for where a test belongs:
C++ test files typically belong in<top-level directory>_unittests (e.g.base_unittests for//base). There are a few exceptions -- browser tests are typically their own target (e.g.content_browsertests for//content, orbrowser_tests for//chrome), and some unit test suites are broken at the second directory rather than the top-level one.
Java test files vary a bit more widely than their C++ counterparts:
<top-level directory>_javatests or<top-level directory>_test_java. Regardless, they'll wind up getting packaged into one of a few test APKs:webview_instrumentation_test_apk for anything in//android_webviewcontent_shell_test_apk for anything in//content or belowchrome_public_test_apk for most things in//chrome<top-level directory>_junit_tests (e.g.base_junit_tests for//base), though here again there are cases (particularly in//components) where suites are split at the second directory rather than the top-level one.Once you know what to build, just do it like you normally would build anything else, e.g.:ninja -C out/Release chrome_public_test_apk
If you do not know what target a test file belongs to, you can use//tools/autotest.py to figure it out fo you:
# Builds relevant test target and then runs the test:tools/autotest.py-C<output directory>TestClassName
All functional tests should be runnable via the wrapper scripts generated at build time:
<output directory>/bin/run_<target_name>[options]
Note that tests are sharded across all attached devices unless explicitly told to do otherwise by-d/--device.
The commands used by the buildbots are printed in the logs. Look athttps://build.chromium.org/ to duplicate the same test command as a particular builder.
Use--list-tests to list what tests are available.
# Prints out all available tests:<output directory>/bin/run_<target_name>--list-tests# Prints out all available tests that match a filter:<output directory>/bin/run_<target_name>--list-tests-f"*MyFilter*"
If you see this error when the test runner is attempting to deploy the test binaries to the AVD emulator, you may need to resize your userdata partition with the following commands:
# Resize userdata partition to be 1Gresize2fs android_emulator_sdk/sdk/system-images/android-25/x86/userdata.img1G# Set filesystem parameter to continue on errors; Android doesn't like some# things e2fsprogs does.tune2fs-econtinue android_emulator_sdk/sdk/system-images/android-25/x86/userdata.img
There's a known issue (https://crbug.com/1094062) where the unit test binaries can fail on Android R and later: if you see this error, try rerunning on an Android version with API level <= 29 (Android <= Q).
Crash stacks are logged and can be viewed usingadb logcat. To symbolize the traces, defineCHROMIUM_OUTPUT_DIR=$OUTDIR where$OUTDIR is the argument you pass toninja -C, and pipe the output throughthird_party/android_platform/development/scripts/stack. If$CHROMIUM_OUTPUT_DIR is unset, the script will searchout/Debug andout/Release. For example:
# If you build withninja-Cout/Debug chrome_public_test_apk# You can run:adb logcat-d| third_party/android_platform/development/scripts/stack# If you build withninja-Cout/android chrome_public_test_apk# You can run:adb logcat-d| CHROMIUM_OUTPUT_DIR=out/android third_party/android_platform/development/scripts/stack# orexport CHROMIUM_OUTPUT_DIR=out/androidadb logcat-d| third_party/android_platform/development/scripts/stack
JUnit tests are Java unittests running on the host instead of the target device. They are faster to run and therefore are recommended over instrumentation tests when possible.
The JUnits tests are usually following the pattern oftarget_junit_tests, for example,content_junit_tests andchrome_junit_tests.
When adding a new JUnit test, the associatedBUILD.gn file must be updated. For example, adding a test tochrome_junit_tests requires to updatechrome/android/BUILD.gn.
# Build the test suite.ninja-Cout/Default chrome_junit_tests# Run the test suite.out/Default/bin/run_chrome_junit_tests# Run a subset of tests. You might need to pass the package name for some tests.out/Default/bin/run_chrome_junit_tests-f"org.chromium.chrome.browser.media.*"
Similar todebugging apk targets:
out/Default/bin/run_chrome_junit_tests--wait-for-java-debuggerout/Default/bin/run_chrome_junit_tests--wait-for-java-debugger# Specify custom port via --debug-socket=9999
# Build a test suiteninja-Cout/Release content_unittests# Run a test suiteout/Release/bin/run_content_unittests[-vv]# Run a subset of tests and enable some "please go faster" options:out/Release/bin/run_content_unittests--fast-local-dev-f"ByteStreamTest.*"
In order to run instrumentation tests, you must leave your device screen ON and UNLOCKED. Otherwise, the test will timeout trying to launch an intent. Optionally you can disable screen lock under Settings -> Security -> Screen Lock -> None.
Next, you need to build the app, build your tests, and then run your tests (which will install the APK under test and the test APK automatically).
Examples:
ContentShell tests:
# Build the tests:ninja-Cout/Release content_shell_test_apk# Run the test suite:out/Release/bin/run_content_shell_test_apk[-vv]# Run a subset of tests and enable some "please go faster" options:out/Release/bin/run_content_shell_test_apk--fast-local-dev-f"*TestClass*"
Android WebView tests:
In order to run a subset of tests, use -f to filter based on test class/method or -A/-E to filter using annotations.
More Filtering examples:
# Run a specific test classout/Debug/bin/run_content_shell_test_apk-f"AddressDetectionTest.*"# Run a specific test methodout/Debug/bin/run_content_shell_test_apk-fAddressDetectionTest#testAddressLimits# Run a subset of tests by size (Smoke, SmallTest, MediumTest, LargeTest,# EnormousTest)out/Debug/bin/run_content_shell_test_apk-ASmoke# Run a subset of tests by annotation, such as filtering by Featureout/Debug/bin/run_content_shell_test_apk-AFeature=Navigation
You might want to add stars* to each as a regular expression, e.g.*AddressDetectionTest*
Similar todebugging apk targets:
out/Debug/bin/run_content_shell_test_apk--wait-for-java-debugger
If running withis_debug=false, Java stacks from logcat need to be fixed up:
build/android/stacktrace/java_deobfuscate.pyout/Release/apks/ChromePublicTest.apk.mapping< stacktrace.txt
Any stacks produced by test runner output will already be deobfuscated.
SeeWeb Tests.
SeeTelemetry.
(e.g. the “Android Debug (Nexus 7)” bot on the chromium.gpu waterfall)
Seehttps://www.chromium.org/developers/testing/gpu-testing for details. Use--browser=android-content-shell. Examine the stdio from the test invocation on the bots to see arguments to pass tosrc/content/test/gpu/run_gpu_test.py.