- Notifications
You must be signed in to change notification settings - Fork4
Make multi-threaded pytest test cases fail when they should
License
bjoluc/pytest-reraise
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Let's assume you are writing a pytest test case that includes assertions in another thread, roughly like this:
fromthreadingimportThreaddeftest_assert():defrun():assertFalseThread(target=run).start()
This test will pass as theAssertionError
is not raised in the main thread.pytest-reraise
is here to help you capture the exception and raise it in the main thread:
pip install pytest-reraise
fromthreadingimportThreaddeftest_assert(reraise):defrun():withreraise:assertFalseThread(target=run).start()
The above test will fail aspytest-reraise
captures the exception and raises it at the end of the test case.
Instead of using thereraise
context manager in a function, you can also wrap the entire function with it via thereraise.wrap()
method.Hence, the example
defrun():withreraise:assertFalseThread(target=run).start()
can also be written as
defrun():assertFalseThread(target=reraise.wrap(run)).start()
or even
@reraise.wrapdefrun():assertFalseThread(target=run).start()
By default, the captured exception (if any) is raised at the end of the test case.If you want to raise it before then, callreraise()
in your test case.If an exception has been raised within awith reraise
block by then,reraise()
will raise it right away:
deftest_assert(reraise):defrun():withreraise:assertFalsereraise()# This will not raise anything yett=Thread(target=run)t.start()t.join()reraise()# This will raise the assertion error
As seen in the example above,reraise()
can be called multiple times during a test case. Whenever an exception has been raised in awith reraise
block since the last call, it will be raised on the next call.
When thereraise
context manager is used multiple times in a single test case, only the first-raised exception will be re-raised in the end.In the below example, both threads raise an exception but only one of these exceptions will be re-raised.
deftest_assert(reraise):defrun():withreraise:assertFalsefor_inrange(2):Thread(target=run).start()
By default, thereraise
context manager does not catch exceptions, so they will not be hidden from the thread in which they are raised.If you want to change this, usereraise(catch=True)
instead ofreraise
:
deftest_assert(reraise):defrun():withreraise(catch=True):assertFalseprint("I'm alive!")Thread(target=run).start()
Note that you cannot usereraise()
(without thecatch
argument) as a context manager, as it is used to raise exceptions.
Ifreraise
captures an exception and the main thread raises an exception as well, the exception captured byreraise
will mask the main thread's exception unless that exception was already re-raised.The objective behind this is that the outcome of the main thread often depends on the work performed in other threads.Thus, failures in in other threads are likely to cause failures in the main thread, and other threads' exceptions (if any) are of greater importance for the developer than main thread exceptions.
The example below will reportassert False
, notassert "foo" == "bar"
.
deftest_assert(reraise):defrun():withreraise:assertFalse# This will be reportedt=Thread(target=run)t.start()t.join()assert"foo"=="bar"# This won't
reraise
provides anexception
property to retrieve the exception that was captured, if any.reraise.exception
can also be used to assign an exception if no exception has been captured yet.In addition to that,reraise.reset()
returns the value ofreraise.exception
and resets it toNone
so that the exception will not be raised anymore.
Here's a quick demonstration test case that passes:
deftest_assert(reraise):defrun():withreraise:assertFalset=Thread(target=run)t.start()t.join()# Return the captured exception:asserttype(reraise.exception)isAssertionError# This won't do anything, since an exception has already been captured:reraise.exception=Exception()# Return the exception and set `reraise.exception` to None:asserttype(reraise.reset())isAssertionError# `Reraise` will not fail the test case becauseassertreraise.exceptionisNone
About
Make multi-threaded pytest test cases fail when they should