5 The kernel contains a set of "self tests" under the tools/testing/selftests/
6 directory. These are intended to be small tests to exercise individual code
7 paths in the kernel. Tests are intended to be run after building, installing
10 Kselftest from mainline can be run on older stable kernels. Running tests
11 from mainline offers the best coverage. Several test rings run mainline
12 kselftest suite on stable releases. The reason is that when a new test
13 gets added to test existing code to regression test a bug, we should be
14 able to run that test on an older kernel. Hence, it is important to keep
15 code that can still test an older kernel and make sure it skips the test
16 gracefully on newer releases.
18 You can find additional information on Kselftest framework, how to
19 write new tests using the framework on Kselftest wiki:
21 https://kselftest.wiki.kernel.org/
23 On some systems, hot-plug tests could hang forever waiting for cpu and
24 memory to be ready to be offlined. A special hot-plug target is created
25 to run the full range of hot-plug tests. In default mode, hot-plug tests run
26 in safe mode with a limited scope. In limited mode, cpu-hotplug test is
27 run on a single cpu as opposed to all hotplug capable cpus, and memory
28 hotplug test is run on 2% of hotplug capable memory instead of 10%.
30 kselftest runs as a userspace process. Tests that can be written/run in
31 userspace may wish to use the `Test Harness`_. Tests that need to be
32 run in kernel space may wish to use a `Test Module`_.
34 Running the selftests (hotplug tests are run in limited mode)
35 =============================================================
40 $ make -C tools/testing/selftests
44 $ make -C tools/testing/selftests run_tests
46 To build and run the tests with a single command, use::
50 Note that some tests will require root privileges.
52 Kselftest supports saving output files in a separate directory and then
53 running tests. To locate output files in a separate directory two syntaxes
54 are supported. In both cases the working directory must be the root of the
55 kernel src. This is applicable to "Running a subset of selftests" section
58 To build, save output files in a separate directory with O= ::
60 $ make O=/tmp/kselftest kselftest
62 To build, save output files in a separate directory with KBUILD_OUTPUT ::
64 $ export KBUILD_OUTPUT=/tmp/kselftest; make kselftest
66 The O= assignment takes precedence over the KBUILD_OUTPUT environment
69 The above commands by default run the tests and print full pass/fail report.
70 Kselftest supports "summary" option to make it easier to understand the test
71 results. Please find the detailed individual test results for each test in
72 /tmp/testname file(s) when summary option is specified. This is applicable
73 to "Running a subset of selftests" section below.
75 To run kselftest with summary option enabled ::
77 $ make summary=1 kselftest
79 Running a subset of selftests
80 =============================
82 You can use the "TARGETS" variable on the make command line to specify
83 single test to run, or a list of tests to run.
85 To run only tests targeted for a single subsystem::
87 $ make -C tools/testing/selftests TARGETS=ptrace run_tests
89 You can specify multiple tests to build and run::
91 $ make TARGETS="size timers" kselftest
93 To build, save output files in a separate directory with O= ::
95 $ make O=/tmp/kselftest TARGETS="size timers" kselftest
97 To build, save output files in a separate directory with KBUILD_OUTPUT ::
99 $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest
101 Additionally you can use the "SKIP_TARGETS" variable on the make command
102 line to specify one or more targets to exclude from the TARGETS list.
104 To run all tests but a single subsystem::
106 $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests
108 You can specify multiple tests to skip::
110 $ make SKIP_TARGETS="size timers" kselftest
112 You can also specify a restricted list of tests to run together with a
115 $ make TARGETS="breakpoints size timers" SKIP_TARGETS=size kselftest
117 See the top-level tools/testing/selftests/Makefile for the list of all
120 Running the full range hotplug selftests
121 ========================================
123 To build the hotplug tests::
125 $ make -C tools/testing/selftests hotplug
127 To run the hotplug tests::
129 $ make -C tools/testing/selftests run_hotplug
131 Note that some tests will require root privileges.
137 You can use the "install" target of "make" (which calls the `kselftest_install.sh`
138 tool) to install selftests in the default location (`tools/testing/selftests/kselftest_install`),
139 or in a user specified location via the `INSTALL_PATH` "make" variable.
141 To install selftests in default location::
143 $ make -C tools/testing/selftests install
145 To install selftests in a user specified location::
147 $ make -C tools/testing/selftests install INSTALL_PATH=/some/other/path
149 Running installed selftests
150 ===========================
152 Found in the install directory, as well as in the Kselftest tarball,
153 is a script named `run_kselftest.sh` to run the tests.
155 You can simply do the following to run the installed Kselftests. Please
156 note some tests will require root privileges::
158 $ cd kselftest_install
161 To see the list of available tests, the `-l` option can be used::
163 $ ./run_kselftest.sh -l
165 The `-c` option can be used to run all the tests from a test collection, or
166 the `-t` option for specific single tests. Either can be used multiple times::
168 $ ./run_kselftest.sh -c size -c seccomp -t timers:posix_timers -t timer:nanosleep
170 For other features see the script usage output, seen with the `-h` option.
172 Timeout for selftests
173 =====================
175 Selftests are designed to be quick and so a default timeout is used of 45
176 seconds for each test. Tests can override the default timeout by adding
177 a settings file in their directory and set a timeout variable there to the
178 configured a desired upper timeout for the test. Only a few tests override
179 the timeout with a value higher than 45 seconds, selftests strives to keep
180 it that way. Timeouts in selftests are not considered fatal because the
181 system under which a test runs may change and this can also modify the
182 expected time it takes to run a test. If you have control over the systems
183 which will run the tests you can configure a test runner on those systems to
184 use a greater or lower timeout on the command line as with the `-o` or
185 the `--override-timeout` argument. For example to use 165 seconds instead
188 $ ./run_kselftest.sh --override-timeout 165
190 You can look at the TAP output to see if you ran into the timeout. Test
191 runners which know a test must run under a specific time can then optionally
192 treat these timeouts then as fatal.
197 In some cases packaging is desired, such as when tests need to run on a
198 different system. To package selftests, run::
200 $ make -C tools/testing/selftests gen_tar
202 This generates a tarball in the `INSTALL_PATH/kselftest-packages` directory. By
203 default, `.gz` format is used. The tar compression format can be overridden by
204 specifying a `FORMAT` make variable. Any value recognized by `tar's auto-compress`_
205 option is supported, such as::
207 $ make -C tools/testing/selftests gen_tar FORMAT=.xz
209 `make gen_tar` invokes `make install` so you can use it to package a subset of
210 tests by using variables specified in `Running a subset of selftests`_
213 $ make -C tools/testing/selftests gen_tar TARGETS="size" FORMAT=.xz
215 .. _tar's auto-compress: https://www.gnu.org/software/tar/manual/html_node/gzip.html#auto_002dcompress
217 Contributing new tests
218 ======================
220 In general, the rules for selftests are
222 * Do as much as you can if you're not root;
224 * Don't take too long;
226 * Don't break the build on any architecture, and
228 * Don't cause the top-level "make run_tests" to fail if your feature is
231 Contributing new tests (details)
232 ================================
234 * In your Makefile, use facilities from lib.mk by including it instead of
235 reinventing the wheel. Specify flags and binaries generation flags on
236 need basis before including lib.mk. ::
238 CFLAGS = $(KHDR_INCLUDES)
239 TEST_GEN_PROGS := close_range_test
242 * Use TEST_GEN_XXX if such binaries or files are generated during
245 TEST_PROGS, TEST_GEN_PROGS mean it is the executable tested by
248 TEST_GEN_MODS_DIR should be used by tests that require modules to be built
249 before the test starts. The variable will contain the name of the directory
250 containing the modules.
252 TEST_CUSTOM_PROGS should be used by tests that require custom build
253 rules and prevent common build rule use.
255 TEST_PROGS are for test shell scripts. Please ensure shell script has
256 its exec bit set. Otherwise, lib.mk run_tests will generate a warning.
258 TEST_CUSTOM_PROGS and TEST_PROGS will be run by common run_tests.
260 TEST_PROGS_EXTENDED, TEST_GEN_PROGS_EXTENDED mean it is the
261 executable which is not tested by default.
263 TEST_FILES, TEST_GEN_FILES mean it is the file which is used by
266 TEST_INCLUDES is similar to TEST_FILES, it lists files which should be
267 included when exporting or installing the tests, with the following
270 * symlinks to files in other directories are preserved
271 * the part of paths below tools/testing/selftests/ is preserved when
272 copying the files to the output directory
274 TEST_INCLUDES is meant to list dependencies located in other directories of
275 the selftests hierarchy.
277 * First use the headers inside the kernel source and/or git repo, and then the
278 system headers. Headers for the kernel release as opposed to headers
279 installed by the distro on the system should be the primary focus to be able
280 to find regressions. Use KHDR_INCLUDES in Makefile to include headers from
283 * If a test needs specific kernel config options enabled, add a config file in
284 the test directory to enable them.
286 e.g: tools/testing/selftests/android/config
288 * Create a .gitignore file inside test directory and add all generated objects
291 * Add new test name in TARGETS in selftests/Makefile::
295 * All changes should pass::
297 kselftest-{all,install,clean,gen_tar}
298 kselftest-{all,install,clean,gen_tar} O=abo_path
299 kselftest-{all,install,clean,gen_tar} O=rel_path
300 make -C tools/testing/selftests {all,install,clean,gen_tar}
301 make -C tools/testing/selftests {all,install,clean,gen_tar} O=abs_path
302 make -C tools/testing/selftests {all,install,clean,gen_tar} O=rel_path
307 Kselftest tests the kernel from userspace. Sometimes things need
308 testing from within the kernel, one method of doing this is to create a
309 test module. We can tie the module into the kselftest framework by
310 using a shell script test runner. ``kselftest/module.sh`` is designed
311 to facilitate this process. There is also a header file provided to
312 assist writing kernel modules that are for use with kselftest:
314 - ``tools/testing/selftests/kselftest_module.h``
315 - ``tools/testing/selftests/kselftest/module.sh``
317 Note that test modules should taint the kernel with TAINT_TEST. This will
318 happen automatically for modules which are in the ``tools/testing/``
319 directory, or for modules which use the ``kselftest_module.h`` header above.
320 Otherwise, you'll need to add ``MODULE_INFO(test, "Y")`` to your module
321 source. selftests which do not load modules typically should not taint the
322 kernel, but in cases where a non-test module is loaded, TEST_TAINT can be
323 applied from userspace by writing to ``/proc/sys/kernel/tainted``.
328 Here we show the typical steps to create a test module and tie it into
329 kselftest. We use kselftests for lib/ as an example.
331 1. Create the test module
333 2. Create the test script that will run (load/unload) the module
334 e.g. ``tools/testing/selftests/lib/printf.sh``
336 3. Add line to config file e.g. ``tools/testing/selftests/lib/config``
338 4. Add test script to makefile e.g. ``tools/testing/selftests/lib/Makefile``
344 # Assumes you have booted a fresh build of this kernel tree
345 cd /path/to/linux/tree
348 sudo make modules_install
349 make TARGETS=lib kselftest
354 A bare bones test module might look like this:
358 // SPDX-License-Identifier: GPL-2.0+
360 #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
362 #include "../tools/testing/selftests/kselftest_module.h"
364 KSTM_MODULE_GLOBALS();
367 * Kernel module for testing the foobinator
370 static int __init test_function()
375 static void __init selftest(void)
377 KSTM_CHECK_ZERO(do_test_case("", 0));
380 KSTM_MODULE_LOADERS(test_foo);
381 MODULE_AUTHOR("John Developer <jd@fooman.org>");
382 MODULE_LICENSE("GPL");
383 MODULE_INFO(test, "Y");
391 # SPDX-License-Identifier: GPL-2.0+
392 $(dirname $0)/../kselftest/module.sh "foo" test_foo
398 The kselftest_harness.h file contains useful helpers to build tests. The
399 test harness is for userspace testing, for kernel space testing see `Test
402 The tests from tools/testing/selftests/seccomp/seccomp_bpf.c can be used as
408 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
415 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
416 :functions: TH_LOG TEST TEST_SIGNAL FIXTURE FIXTURE_DATA FIXTURE_SETUP
417 FIXTURE_TEARDOWN TEST_F TEST_HARNESS_MAIN FIXTURE_VARIANT
423 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
426 .. kernel-doc:: tools/testing/selftests/kselftest_harness.h
427 :functions: ASSERT_EQ ASSERT_NE ASSERT_LT ASSERT_LE ASSERT_GT ASSERT_GE
428 ASSERT_NULL ASSERT_TRUE ASSERT_NULL ASSERT_TRUE ASSERT_FALSE
429 ASSERT_STREQ ASSERT_STRNE EXPECT_EQ EXPECT_NE EXPECT_LT
430 EXPECT_LE EXPECT_GT EXPECT_GE EXPECT_NULL EXPECT_TRUE
431 EXPECT_FALSE EXPECT_STREQ EXPECT_STRNE