Adding and running tests

A high-quality suite of tests is crucial in ensuring correctness and robustness of the codebase. Here, we provide instructions how to run unit tests, and also how to add a new one.

Contents

Adding a new unit test

Python package: pytest

Add your test under the directories

Refer tothe PyTest tutorialto learn how to write tests for Python code.

You may try running your test by following instructions inthis section.

C++: Google Test

Add your test under the directorytests/cpp/. Refer tothis excellent tutorial on using Google Test.

You may try running your test by following instructions inthis section. Note. Google Test version 1.8.1 or later is required.

JVM packages: JUnit / scalatest

The JVM packages for XGBoost (XGBoost4J / XGBoost4J-Spark) usethe Maven Standard Directory Layout. Specifically, the tests for the JVM packages are located in the following locations:

To write a test for Java code, seeJUnit 5 tutorial.To write a test for Scala, seeScalatest tutorial.

You may try running your test by following instructions inthis section.

R package: testthat

Add your test under the directoryR-package/tests/testthat. Refer tothis excellent tutorial on testthat.

You may try running your test by following instructions inthis section.

Running Unit Tests Locally

R package

Run

python./ops/script/test_r_package.py--task=check

at the root of the project directory. The command builds and checks the XGBoostr-package. Alternatively, if you want to just run the tests, you can use the followingcommands after installing XGBoost:

cdR-package/tests/Rscripttestthat.R

JVM packages

Maven is used

mvntest

Python package: pytest

To run Python unit tests, first installpytest package:

pip3installpytest

Then compile XGBoost according to instructions inBuilding the Shared Library. Finally, invoke pytest at the project root directory:

# Tell Python where to find XGBoost moduleexportPYTHONPATH=./python-packagepytest-v-s--fulltracetests/python

In addition, to test CUDA code, run:

# Tell Python where to find XGBoost moduleexportPYTHONPATH=./python-packagepytest-v-s--fulltracetests/python-gpu

(For this step, you should have compiled XGBoost with CUDA enabled.)

For testing with distributed frameworks likeDask andPySpark:

# Tell Python where to find XGBoost moduleexportPYTHONPATH=./python-packagepytest-v-s--fulltracetests/test_distributed

C++: Google Test

To build and run C++ unit tests enable tests while running CMake:

cmake-Bbuild-S.-GNinja-DGOOGLE_TEST=ON-DUSE_DMLC_GTEST=ON-DUSE_CUDA=ON-DUSE_NCCL=ONcmake--buildbuildcd./build./testxgboost

Flags likeUSE_CUDA,USE_DMLC_GTEST are optional. For more info about how to buildXGBoost from source, seeBuilding From Source. One can also run all unit tests using ctest toolwhich provides higher flexibility. For example:

ctest--verbose

If you need to debug errors on Windows using the debugger from VS, you can append the gtest flags intest_main.cc:

::testing::GTEST_FLAG(filter)="Suite.Test";::testing::GTEST_FLAG(repeat)=10;

Sanitizers: Detect memory errors and data races

By default, sanitizers are bundled in GCC and Clang/LLVM. One can enable sanitizers withGCC >= 4.8 or LLVM >= 3.1, But some distributions might package sanitizers separately.Here is a list of supported sanitizers with corresponding library names:

  • Address sanitizer: libasan

  • Undefined sanitizer: libubsan

  • Leak sanitizer: liblsan

  • Thread sanitizer: libtsan

Memory sanitizer is exclusive to LLVM, hence not supported in XGBoost. With latestcompilers like gcc-9, when sanitizer flags are specified, the compiler driver should beable to link the runtime libraries automatically.

How to build XGBoost with sanitizers

One can build XGBoost with sanitizer support by specifying -DUSE_SANITIZER=ON.By default, address sanitizer and leak sanitizer are used when you turn theUSE_SANITIZER flag on. You can always change the default by providing asemicolon separated list of sanitizers to ENABLED_SANITIZERS. Note that threadsanitizer is not compatible with the other two sanitizers.

cmake-DUSE_SANITIZER=ON-DENABLED_SANITIZERS="address;undefined"/path/to/xgboost

By default, CMake will search regular system paths for sanitizers, you can alsosupply a specified SANITIZER_PATH.

cmake-DUSE_SANITIZER=ON-DENABLED_SANITIZERS="address;undefined"\-DSANITIZER_PATH=/path/to/sanitizers/path/to/xgboost

How to use sanitizers with CUDA support

Running XGBoost on CUDA with address sanitizer (asan) will raise memory error.To use asan with CUDA correctly, you need to configure asan via ASAN_OPTIONSenvironment variable:

ASAN_OPTIONS=protect_shadow_gap=0${BUILD_DIR}/testxgboost

Other sanitizer runtime options

By default undefined sanitizer doesn’t print out the backtrace. You can enable it byexporting environment variable:

UBSAN_OPTIONS=print_stacktrace=1 ${BUILD_DIR}/testxgboost

For details, please consultofficial documentation for sanitizers.