Less Boilerplate

Most functional tests follow the Arrange-Act-Assert model:

1. Arrange, or set up, the conditions for the test
2. Act by calling some function or method
3. Assert that some end condition is true

Testing frameworks typically hook into your test’sassertions so that they can provide information when an assertion fails.unittest, for example, provides a number of helpful assertion utilities out of the box. However, even a small set of tests requires a fair amount ofboilerplate code.

Imagine you’d like to write a test suite just to make sure thatunittest is working properly in your project. You might want to write one test that always passes and one that always fails:

fromunittestimportTestCaseclassTryTesting(TestCase):deftest_always_passes(self):self.assertTrue(True)deftest_always_fails(self):self.assertTrue(False)

You can then run those tests from the command line using thediscover option ofunittest:

(venv)$python-munittestdiscoverF.======================================================================FAIL: test_always_fails (test_with_unittest.TryTesting)----------------------------------------------------------------------Traceback (most recent call last):  File "...\effective-python-testing-with-pytest\test_with_unittest.py",  line 10, in test_always_fails    self.assertTrue(False)AssertionError: False is not true----------------------------------------------------------------------Ran 2 tests in 0.006sFAILED (failures=1)

As expected, one test passed and one failed. You’ve proven thatunittest is working, but look at what you had to do:

1. Import theTestCase class fromunittest
2. CreateTryTesting, asubclass ofTestCase
3. Write a method inTryTesting for each test
4. Use one of theself.assert* methods fromunittest.TestCase to make assertions

That’s a significant amount of code to write, and because it’s the minimum you need forany test, you’d end up writing the same code over and over.pytest simplifies this workflow by allowing you to use normal functions and Python’sassert keyword directly:

deftest_always_passes():assertTruedeftest_always_fails():assertFalse

That’s it. You don’t have to deal with any imports or classes. All you need to do is include a function with thetest_ prefix. Because you can use theassert keyword, you don’t need to learn or remember all the differentself.assert* methods inunittest, either. If you can write an expression that you expect to evaluate toTrue, and thenpytest will test it for you.

Not only doespytest eliminate a lot of boilerplate, but it also provides you with a much more detailed and easy-to-read output.

Nicer Output

You can run your test suite using thepytest command from the top-level folder of your project:

(venv)$pytest============================= test session starts =============================platform win32 -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0rootdir: ...\effective-python-testing-with-pytestcollected 4 itemstest_with_pytest.py .F                                                   [ 50%]test_with_unittest.py F.                                                 [100%]================================== FAILURES ===================================______________________________ test_always_fails ______________________________    def test_always_fails():>       assert FalseE       assert Falsetest_with_pytest.py:7: AssertionError________________________ TryTesting.test_always_fails _________________________self = <test_with_unittest.TryTesting testMethod=test_always_fails>    def test_always_fails(self):>       self.assertTrue(False)E       AssertionError: False is not truetest_with_unittest.py:10: AssertionError=========================== short test summary info ===========================FAILED test_with_pytest.py::test_always_fails - assert FalseFAILED test_with_unittest.py::TryTesting::test_always_fails - AssertionError:...========================= 2 failed, 2 passed in 0.20s =========================

pytest presents the test results differently thanunittest, and thetest_with_unittest.py file was also automatically included. The report shows:

1. The system state, including which versions of Python,pytest, and any plugins you have installed
2. Therootdir, or the directory to search under for configuration and tests
3. The number of tests the runner discovered

These items are presented in the first section of the output:

============================= test session starts =============================platform win32 -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0rootdir: ...\effective-python-testing-with-pytestcollected 4 items

The output then indicates the status of each test using a syntax similar tounittest:

• A dot (.) means that the test passed.
• AnF means that the test has failed.
• AnE means that the test raised an unexpected exception.

The special characters are shown next to the name with the overall progress of the test suite shown on the right:

test_with_pytest.py .F                                                   [ 50%]test_with_unittest.py F.                                                 [100%]

For tests that fail, the report gives a detailed breakdown of the failure. In the example, the tests failed becauseassert False always fails:

================================== FAILURES ===================================______________________________ test_always_fails ______________________________    def test_always_fails():>       assert FalseE       assert Falsetest_with_pytest.py:7: AssertionError________________________ TryTesting.test_always_fails _________________________self = <test_with_unittest.TryTesting testMethod=test_always_fails>    def test_always_fails(self):>       self.assertTrue(False)E       AssertionError: False is not truetest_with_unittest.py:10: AssertionError

This extra output can come in extremely handy when debugging. Finally, the report gives an overall status report of the test suite:

=========================== short test summary info ===========================FAILED test_with_pytest.py::test_always_fails - assert FalseFAILED test_with_unittest.py::TryTesting::test_always_fails - AssertionError:...========================= 2 failed, 2 passed in 0.20s =========================

When compared to unittest, thepytest output is much more informative and readable.

In the next section, you’ll take a closer look at howpytest takes advantage of the existingassert keyword.

Less to Learn

Being able to use theassert keyword is also powerful. If you’ve used it before, then there’s nothing new to learn. Here are a few assertion examples so you can get an idea of the types of test you can make:

deftest_uppercase():assert"loud noises".upper()=="LOUD NOISES"deftest_reversed():assertlist(reversed([1,2,3,4]))==[4,3,2,1]deftest_some_primes():assert37in{numfornuminrange(2,50)ifnotany(num%div==0fordivinrange(2,num))}

They look very much like normal Python functions. All of this makes the learning curve forpytest shallower than it is forunittest because you don’t need to learn new constructs to get started.

Note that each test is quite small and self-contained. This is common—you’ll see long function names and not a lot going on within a function. This serves mainly to keep your tests isolated from each other, so if something breaks, you know exactly where the problem is. A nice side effect is that the labeling is much better in the output.

To see an example of a project that creates a test suite along with the main project, check out theBuild a Hash Table in Python With TDD tutorial. Additionally, you can work on Python practice problems to try test-driven development yourself while youget ready for your next interview orparse CSV files.

In the next section, you’re going to be examining fixtures, a great pytest feature to help you manage test input values.

Easier to Manage State and Dependencies

Your tests will often depend on types of data ortest doubles that mock objects your code is likely to encounter, such asdictionaries orJSON files.

Withunittest, you might extract these dependencies into.setUp() and.tearDown() methods so that each test in the class can make use of them. Using these special methods is fine, but as your test classes get larger, you may inadvertently make the test’s dependence entirelyimplicit. In other words, by looking at one of the many tests in isolation, you may not immediately see that it depends on something else.

Over time, implicit dependencies can lead to a complex tangle of code that you have to unwind to make sense of your tests. Tests should help to make your code more understandable. If the tests themselves are difficult to understand, then you may be in trouble!

pytest takes a different approach. It leads you towardexplicit dependency declarations that are still reusable thanks to the availability offixtures.pytest fixtures are functions that can create data, test doubles, or initialize system state for the test suite. Any test that wants to use a fixture must explicitly use this fixture function as an argument to the test function, so dependencies are always stated up front:

importpytest@pytest.fixturedefexample_fixture():return1deftest_with_fixture(example_fixture):assertexample_fixture==1

Looking at the test function, you can immediately tell that it depends on a fixture, without needing to check the whole file for fixture definitions.

Fixtures can also make use of other fixtures, again by declaring them explicitly as dependencies. That means that, over time, your fixtures can become bulky and modular. Although the ability to insert fixtures into other fixtures provides enormous flexibility, it can also make managing dependencies more challenging as your test suite grows.

Later in this tutorial, you’ll learnmore about fixtures and try a few techniques for handling these challenges.

Easy to Filter Tests

As your test suite grows, you may find that you want to run just a few tests on a feature and save the full suite for later.pytest provides a few ways of doing this:

• Name-based filtering: You can limitpytest to running only those tests whose fully qualified names match a particular expression. You can do this with the-k parameter.
• Directory scoping: By default,pytest will run only those tests that are in or under the current directory.
• Test categorization:pytest can include or exclude tests from particular categories that you define. You can do this with the-m parameter.

Test categorization in particular is a subtly powerful tool.pytest enables you to createmarks, or custom labels, for any test you like. A test may have multiple labels, and you can use them for granular control over which tests to run. Later in this tutorial, you’ll see an example ofhowpytest marks work and learn how to make use of them in a large test suite.

Allows Test Parametrization

When you’re testing functions that process data or perform generic transformations, you’ll find yourself writing many similar tests. They may differ only in theinput or output of the code being tested. This requires duplicating test code, and doing so can sometimes obscure the behavior that you’re trying to test.

unittest offers a way of collecting several tests into one, but they don’t show up as individual tests in result reports. If one test fails and the rest pass, then the entire group will still return a single failing result.pytest offers its own solution in which each test can pass or fail independently. You’ll seehow to parametrize tests withpytest later in this tutorial.

Has a Plugin-Based Architecture

One of the most beautiful features ofpytest is its openness to customization and new features. Almost every piece of the program can be cracked open and changed. As a result,pytest users have developed a rich ecosystem of helpful plugins.

Although somepytest plugins focus on specific frameworks likeDjango, others are applicable to most test suites. You’ll seedetails on some specific plugins later in this tutorial.

When to Create Fixtures

In this section, you’ll simulate a typicaltest-driven development (TDD) workflow.

Imagine you’re writing a function,format_data_for_display(), to process the data returned by an API endpoint. The data represents a list of people, each with a given name, family name, and job title. The function should output a list of strings that include each person’s full name (theirgiven_name followed by theirfamily_name), a colon, and theirtitle:

defformat_data_for_display(people):...# Implement this!

In good TDD fashion, you’ll want to first write a test for it. You might write the following code for that:

deftest_format_data_for_display():people=[{"given_name":"Alfonsa","family_name":"Ruiz","title":"Senior Software Engineer",},{"given_name":"Sayid","family_name":"Khan","title":"Project Manager",},]assertformat_data_for_display(people)==["Alfonsa Ruiz: Senior Software Engineer","Sayid Khan: Project Manager",]

While writing this test, it occurs to you that you may need to write another function to transform the data into comma-separated values for use inExcel:

defformat_data_for_display(people):...# Implement this!defformat_data_for_excel(people):...# Implement this!

Your to-do list grows! That’s good! One of the advantages of TDD is that it helps you plan out the work ahead. The test for theformat_data_for_excel() function would look awfully similar to theformat_data_for_display() function:

deftest_format_data_for_display():# ...deftest_format_data_for_excel():people=[{"given_name":"Alfonsa","family_name":"Ruiz","title":"Senior Software Engineer",},{"given_name":"Sayid","family_name":"Khan","title":"Project Manager",},]assertformat_data_for_excel(people)=="""given,family,titleAlfonsa,Ruiz,Senior Software EngineerSayid,Khan,Project Manager"""

Notably, both the tests have to repeat the definition of thepeople variable, which is quite a few lines of code.

If you find yourself writing several tests that all make use of the same underlying test data, then a fixture may be in your future. You can pull the repeated data into a single function decorated with@pytest.fixture to indicate that the function is apytest fixture:

importpytest@pytest.fixturedefexample_people_data():return[{"given_name":"Alfonsa","family_name":"Ruiz","title":"Senior Software Engineer",},{"given_name":"Sayid","family_name":"Khan","title":"Project Manager",},]# ...

You can use the fixture by adding the function reference as an argument to your tests. Note that you don’t call the fixture function.pytest takes care of that. You’ll be able to use the return value of the fixture function as the name of the fixture function:

# ...deftest_format_data_for_display(example_people_data):assertformat_data_for_display(example_people_data)==["Alfonsa Ruiz: Senior Software Engineer","Sayid Khan: Project Manager",]deftest_format_data_for_excel(example_people_data):assertformat_data_for_excel(example_people_data)=="""given,family,titleAlfonsa,Ruiz,Senior Software EngineerSayid,Khan,Project Manager"""

Each test is now notably shorter but still has a clear path back to the data it depends on. Be sure to name your fixture something specific. That way, you can quickly determine if you want to use it when writing new tests in the future!

When you first discover the power of fixtures, it can be tempting to use them all the time, but as with all things, there’s a balance to be maintained.

When to Avoid Fixtures

Fixtures are great for extracting data or objects that you use across multiple tests. However, they aren’t always as good for tests that require slight variations in the data. Littering your test suite with fixtures is no better than littering it with plain data or objects. It might even be worse because of the added layer of indirection.

As with most abstractions, it takes some practice and thought to find the right level of fixture use.

Nevertheless, fixtures will likely be an integral part of your test suite. As your project grows in scope, the challenge of scale starts to come into the picture. One of the challenges facing any kind of tool is how it handles being used at scale, and luckily,pytest has a bunch of useful features that can help you manage the complexity that comes with growth.

How to Use Fixtures at Scale

As you extract more fixtures from your tests, you might see that some fixtures could benefit from further abstraction. Inpytest, fixtures aremodular. Being modular means that fixtures can beimported, can import other modules, and they can depend on and import other fixtures. All this allows you to compose a suitable fixture abstraction for your use case.

For example, you may find that fixtures in two separate files, ormodules, share a common dependency. In this case, you can move fixtures from test modules into more general fixture-related modules. That way, you can import them back into any test modules that need them. This is a good approach when you find yourself using a fixture repeatedly throughout your project.

If you want to make a fixture available for your whole project without having to import it, a special configuration module calledconftest.py will allow you to do that.

pytest looks for aconftest.py module in each directory. If you add your general-purpose fixtures to theconftest.py module, then you’ll be able to use that fixture throughout the module’s parent directory and in any subdirectories without having to import it. This is a great place to put your most widely used fixtures.

Another interesting use case for fixtures andconftest.py is in guarding access to resources. Imagine that you’ve written a test suite for code that deals withAPI calls. You want to ensure that the test suite doesn’t make any real network calls even if someone accidentally writes a test that does so.

pytest provides amonkeypatch fixture to replace values and behaviors, which you can use to great effect:

importpytestimportrequests@pytest.fixture(autouse=True)defdisable_network_calls(monkeypatch):defstunted_get():raiseRuntimeError("Network access not allowed during testing!")monkeypatch.setattr(requests,"get",lambda*args,**kwargs:stunted_get())

By placingdisable_network_calls() inconftest.py and adding theautouse=True option, you ensure that network calls will be disabled in every test across the suite. Any test that executes code callingrequests.get() will raise aRuntimeError indicating that an unexpected network call would have occurred.

Your test suite is growing in numbers, which gives you a great feeling of confidence to make changes and not break things unexpectedly. That said, as your test suite grows, it might start taking a long time. Even if it doesn’t take that long, perhaps you’re focusing on some core behavior that trickles down and breaks most tests. In these cases, you might want to limit the test runner to only a certain category of tests.

pytest-randomly

Often the order of your tests is unimportant, but as your codebase grows, you may inadvertently introduce some side effects that could cause some tests to fail if they were run out of order.

pytest-randomly forces your tests to run in a random order.pytest always collects all the tests it can find before running them.pytest-randomly just shuffles that list of tests before execution.

This is a great way to uncover tests that depend on running in a specific order, which means they have astateful dependency on some other test. If you built your test suite from scratch inpytest, then this isn’t very likely. It’s more likely to happen in test suites that you migrate topytest.

The plugin will print a seed value in the configuration description. You can use that value to run the tests in the same order as you try to fix the issue.

pytest-cov

If you want to measure how well your tests cover your implementation code, then you can use thecoverage package.pytest-cov integrates coverage, so you can runpytest --cov to see the test coverage report and boast about it on your project front page.

pytest-django

pytest-django provides a handful of useful fixtures and marks for dealing with Django tests. You saw thedjango_db mark earlier in this tutorial. Therf fixture provides direct access to an instance of Django’sRequestFactory. Thesettings fixture provides a quick way to set or override Django settings. These plugins are a great boost to your Django testing productivity!

If you’re interested in learning more about usingpytest with Django, then check outHow to Provide Test Fixtures for Django Models in Pytest.

pytest-bdd

pytest can be used to run tests that fall outside the traditional scope of unit testing.Behavior-driven development (BDD) encourages writing plain-language descriptions of likely user actions and expectations, which you can then use to determine whether to implement a given feature.pytest-bdd helps you useGherkin to write feature tests for your code.

You can see which other plugins are available forpytest with this extensivelist of third-party plugins.