Frequently Asked Questions

How is this different from Autotest, kselftest, and so on?

KUnit is a unit testing framework. Autotest, kselftest (and some others) arenot.

Aunit test is supposed totest a single unit of code in isolation and hence the nameunit test. A unittest should be the finest granularity of testing and should allow all possiblecode paths to be tested in the code under test. This is only possible if thecode under test is small and does not have any external dependencies outside ofthe test’s control like hardware.

There are no testing frameworks currently available for the kernel that do notrequire installing the kernel on a test machine or in a virtual machine. Alltesting frameworks require tests to be written in userspace and run on thekernel under test. This is true for Autotest, kselftest, and some others,disqualifying any of them from being considered unit testing frameworks.

Does KUnit support running on architectures other than UML?

Yes, mostly.

For the most part, the KUnit core framework (what we use to write the tests)can compile to any architecture. It compiles like just another part of thekernel and runs when the kernel boots, or when built as a module, when themodule is loaded. However, there is infrastructure, like the KUnit Wrapper(tools/testing/kunit/kunit.py) that might not support some architectures(seeRunning tests on QEMU).

In short, yes, you can run KUnit on other architectures, but it might requiremore work than using KUnit on UML.

For more information, seeWriting Tests For Other Architectures.

What is the difference between a unit test and other kinds of tests?

Most existing tests for the Linux kernel would be categorized as an integrationtest, or an end-to-end test.

  • A unit test is supposed to test a single unit of code in isolation. A unittest should be the finest granularity of testing and, as such, allows allpossible code paths to be tested in the code under test. This is only possibleif the code under test is small and does not have any external dependenciesoutside of the test’s control like hardware.

  • An integration test tests the interaction between a minimal set of components,usually just two or three. For example, someone might write an integrationtest to test the interaction between a driver and a piece of hardware, or totest the interaction between the userspace libraries the kernel provides andthe kernel itself. However, one of these tests would probably not test theentire kernel along with hardware interactions and interactions with theuserspace.

  • An end-to-end test usually tests the entire system from the perspective of thecode under test. For example, someone might write an end-to-end test for thekernel by installing a production configuration of the kernel on productionhardware with a production userspace and then trying to exercise some behaviorthat depends on interactions between the hardware, the kernel, and userspace.

KUnit is not working, what should I do?

Unfortunately, there are a number of things which can break, but here are somethings to try.

  1. Run./tools/testing/kunit/kunit.pyrun with the--raw_outputparameter. This might show details or error messages hidden by the kunit_toolparser.

  2. Instead of runningkunit.pyrun, try runningkunit.pyconfig,kunit.pybuild, andkunit.pyexec independently. This can help trackdown where an issue is occurring. (If you think the parser is at fault, youcan run it manually againststdin or a file withkunit.pyparse.)

  3. Running the UML kernel directly can often reveal issues or error messages,kunit_tool ignores. This should be as simple as running./vmlinuxafter building the UML kernel (for example, by usingkunit.pybuild).Note that UML has some unusual requirements (such as the host having a tmpfsfilesystem mounted), and has had issues in the past when built statically andthe host has KASLR enabled. (On older host kernels, you may need to runsetarch`uname-m`-R./vmlinux to disable KASLR.)

  4. Make sure the kernel .config hasCONFIG_KUNIT=y and at least one test(e.g.CONFIG_KUNIT_EXAMPLE_TEST=y). kunit_tool will keep its .configaround, so you can see what config was used after runningkunit.pyrun.It also preserves any config changes you might make, so you canenable/disable things withmakeARCH=ummenuconfig or similar, and thenre-run kunit_tool.

  5. Try to runmakeARCH=umdefconfig before runningkunit.pyrun. Thismay help clean up any residual config items which could be causing problems.

  6. Finally, try running KUnit outside UML. KUnit and KUnit tests can bebuilt into any kernel, or can be built as a module and loaded at runtime.Doing so should allow you to determine if UML is causing the issue you’reseeing. When tests are built-in, they will execute when the kernel boots, andmodules will automatically execute associated tests when loaded. Test resultscan be collected from/sys/kernel/debug/kunit/<testsuite>/results, andcan be parsed withkunit.pyparse. For more details, seeRunning tests on QEMU.

If none of the above tricks help, you are always welcome to email any issues tokunit-dev@googlegroups.com.