pytest -rxXs# show extra info on xfailed, xpassed, and skipped tests

skipif¶

If you wish to skip something conditionally then you can useskipif instead.Here is an example of marking a test function to be skippedwhen run on an interpreter earlier than Python3.10:

importsys@pytest.mark.skipif(sys.version_info<(3,10),reason="requires python3.10 or higher")deftest_function():...

If the condition evaluates toTrue during collection, the test function will be skipped,with the specified reason appearing in the summary when using-rs.

You can shareskipif markers between modules. Consider this test module:

# content of test_mymodule.pyimportmymoduleminversion=pytest.mark.skipif(mymodule.__versioninfo__<(1,1),reason="at least mymodule-1.1 required")@minversiondeftest_function():...

You can import the marker and reuse it in another test module:

# test_myothermodule.pyfromtest_mymoduleimportminversion@minversiondeftest_anotherfunction():...

For larger test suites it’s usually a good idea to have one filewhere you define the markers which you then consistently applythroughout your test suite.

Alternatively, you can usecondition strings instead of booleans, but they can’t be shared between modules easilyso they are supported mainly for backward compatibility reasons.

Reference:pytest.mark.skipif

Skip all test functions of a class or module¶

You can use theskipif marker (as any other marker) on classes:

@pytest.mark.skipif(sys.platform=="win32",reason="does not run on windows")classTestPosixCalls:deftest_function(self):"will not be setup or run under 'win32' platform"

If the condition isTrue, this marker will produce a skip result foreach of the test methods of that class.

If you want to skip all test functions of a module, you may use thepytestmark global:

# test_module.pypytestmark=pytest.mark.skipif(...)

If multipleskipif decorators are applied to a test function, itwill be skipped if any of the skip conditions is true.

Skipping files or directories¶

Sometimes you may need to skip an entire file or directory, for example if thetests rely on Python version-specific features or contain code that you do notwish pytest to run. In this case, you must exclude the files and directoriesfrom collection. Refer toCustomizing test collection for moreinformation.

Skipping on a missing import dependency¶

You can skip tests on a missing import by usingpytest.importorskipat module level, within a test, or test setup function.

docutils=pytest.importorskip("docutils")

Ifdocutils cannot be imported here, this will lead to a skip outcome ofthe test. You can also skip based on the version number of a library:

docutils=pytest.importorskip("docutils",minversion="0.3")

The version will be read from the specifiedmodule’s__version__ attribute.

Summary¶

Here’s a quick guide on how to skip tests in a module in different situations:

1. Skip all tests in a module unconditionally:

pytestmark=pytest.mark.skip("all tests still WIP")

2. Skip all tests in a module based on some condition:

pytestmark=pytest.mark.skipif(sys.platform=="win32",reason="tests for linux only")

3. Skip all tests in a module if some import is missing:

pexpect=pytest.importorskip("pexpect")

condition parameter¶

If a test is only expected to fail under a certain condition, you can passthat condition as the first parameter:

@pytest.mark.xfail(sys.platform=="win32",reason="bug in a 3rd party library")deftest_function():...

Note that you have to pass a reason as well (see the parameter description atpytest.mark.xfail).

reason parameter¶

You can specify the motive of an expected failure with thereason parameter:

@pytest.mark.xfail(reason="known parser issue")deftest_function():...

raises parameter¶

If you want to be more specific as to why the test is failing, you can specifya single exception, or a tuple of exceptions, in theraises argument.

@pytest.mark.xfail(raises=RuntimeError)deftest_function():...

Then the test will be reported as a regular failure if it fails with anexception not mentioned inraises.

run parameter¶

If a test should be marked as xfail and reported as such but should not beeven executed, use therun parameter asFalse:

@pytest.mark.xfail(run=False)deftest_function():...

This is specially useful for xfailing tests that are crashing the interpreter and should beinvestigated later.

strict parameter¶

BothXFAIL andXPASS don’t fail the test suite by default.You can change this by setting thestrict keyword-only parameter toTrue:

@pytest.mark.xfail(strict=True)deftest_function():...

This will makeXPASS (“unexpectedly passing”) results from this test to fail the test suite.

You can change the default value of thestrict parameter using thexfail_strict ini option:

[pytest]xfail_strict=true

Ignoring xfail¶

By specifying on the commandline:

pytest --runxfail

you can force the running and reporting of anxfail marked testas if it weren’t marked at all. This also causespytest.xfail() to produce no effect.

Examples¶

Here is a simple test file with the several usages:

importpytestxfail=pytest.mark.xfail@xfaildeftest_hello():assert0@xfail(run=False)deftest_hello2():assert0@xfail("hasattr(os, 'sep')")deftest_hello3():assert0@xfail(reason="bug 110")deftest_hello4():assert0@xfail('pytest.__version__[0] != "17"')deftest_hello5():assert0deftest_hello6():pytest.xfail("reason")@xfail(raises=IndexError)deftest_hello7():x=[]x[1]=1

Running it with the report-on-xfail option gives this output:

! pytest -rx xfail_demo.py=========================== test session starts ============================platform linux -- Python 3.x.y, pytest-6.x.y, py-1.x.y, pluggy-1.x.ycachedir: $PYTHON_PREFIX/.pytest_cacherootdir: $REGENDOC_TMPDIR/examplecollected 7 itemsxfail_demo.pyxxxxxxx[100%]========================= short test summary info ==========================XFAIL xfail_demo.py::test_helloXFAIL xfail_demo.py::test_hello2  reason: [NOTRUN]XFAIL xfail_demo.py::test_hello3  condition: hasattr(os, 'sep')XFAIL xfail_demo.py::test_hello4  bug 110XFAIL xfail_demo.py::test_hello5  condition: pytest.__version__[0] != "17"XFAIL xfail_demo.py::test_hello6  reason: reasonXFAIL xfail_demo.py::test_hello7============================7 xfailed in 0.12s ============================