Testing Your Code with Python's pytest, Part II

Testing functions isn't hard, but how do you test user input and output?

In mylastarticle, I started looking at "pytest", a framework for testing Python programsthat's really changed the way I look at testing. For the first time, I reallyfeel like testing is something I can and should do on a regular basis; pytest makesthings so easy and straightforward.

One of the main topics I didn't cover in my last article is user input and output. Howcan you test programs that expect to get input from files or from the user? And, howcan you test programs that are supposed to display something on the screen?

So in this article, I describe how to test input and output in a variety of ways,allowing you to test programs that interact with the outside world. I try notonly to explain what you can do, but also show how it fits into the larger context of testingin general and pytest in particular.

Say you have a function that asks the user to enter an integer and thenreturns the value of that integer, doubled. You can imagine that the function wouldlook like this:

def double():    x = input("Enter an integer: ")    return int(x) * 2

How can you test that function with pytest? If the function were to take an argument,the answer would be easy. But in this case, the function is asking forinteractive input from the user. That's a bit harder to deal with. After all, howcan you, in your tests, pretend to ask the user for input?

In most programming languages, user input comes from a source known asstandard input (orstdin). In Python,sys.stdin is a read-only file object fromwhich you can grab the user's input.

So, if you want to test the "double" function from above, you can (should) replacesys.stdin with another file. There are two problems with this, however. First, youdon't really want to start opening files on disk. And second, do you really want toreplace the value ofsys.stdin in your tests? That'll affect more than just onetest.

The solution comes in two parts. First, you can use the pytest "monkey patching"facility to assign a value to a system object temporarily for the duration of thetest. This facility requires that you define your test function with a parameter namedmonkeypatch. The pytest system notices that you've defined it with that parameter,and then not only sets themonkeypatch local variable, but also sets it up to let youtemporarily set attribute names.

In theory, then, you could define your test like this:

def test_double(monkeypatch):    monkeypatch.setattr('sys.stdin', open('/etc/passwd'))    print(double())

In other words, this tells pytest that you want to open /etc/passwd and feed itscontents to pytest. This has numerous problems, starting with the fact that/etc/passwd contains multiple lines, and that each of its lines is non-numeric.The function thus chokes and exits with an error before it even gets to the (useless)call toprint.

But there's another problem here, one that I mentioned above. You don't really want tobe opening files during testing, if you can avoid it. Thus, one of the greattools in my testing toolbox is Python'sStringIO class. The beauty ofStringIO isits simplicity. It implements the API of a "file" object, but exists only in memoryand is effectively a string. If you can create aStringIO instance, you can pass itto the call tomonkeypatch.setattr, and thus make your tests precisely the way youwant.

Here's how to do that:

from io import StringIOfrom double import doublenumber_inputs = StringIO('1234\n')def test_double(monkeypatch):    monkeypatch.setattr('sys.stdin', number_inputs)    assert double() == 2468

You first create aStringIO object containing the input you want to simulate from theuser. Note that it must contain a newline (\n) to ensure that you'll see the end ofthe user's input and not hang.

You assign that to a global variable, which means you'll be able to access it fromwithin your test function. You then add the assertion to your test function, saying thatthe result should be 2468. And sure enough, it works.

I've used this technique to simulate much longer files, and I've been quite satisfiedby the speed and flexibility. Just remember that each line in the input "file"should end with a newline character. I've found that creating aStringIO with atriple-quoted string, which lets me include newlines and write in a more naturalfile-like way, works well.

You can usemonkeypatch to simulate calls to a variety of other objects as well. Ihaven't had much occasion to do that, but you can imagine all sorts ofnetwork-related objects that you don't actually want to use when testing. By monkey-patching those objects during testing, you can pretend to connect to a remote server,when in fact you're just getting pre-programmed text back.

What happens if you call thetest_double function with a string? You probablyshould test that too:

str_inputs = StringIO('abcd\n')def test_double_str(monkeypatch):    monkeypatch.setattr('sys.stdin', str_inputs)    assert double() == 'abcdabcd'

It looks great, right? Actually, not so much:

E   ValueError: invalid literal for int() with base 10: 'abcd'

The test failed, because the function exited with an exception. And that's okay; afterall, the functionshould raise an exception if the user gives input that isn'tnumeric. But, wouldn't it be nice to specify and test it?

The thing is, how can you test for an exception? You can't exactly use a usualassertstatement, much as you might like to. After all, you somehow need to trap the exception,and you can't simply say:

assert double() == ValueError

That's because exceptions aren't values that are returned. They are raised through adifferent mechanism.

Fortunately, pytest offers a good solution to this, albeit with slightly differentsyntax than you've seen before. It uses awith statement, which many Pythondevelopers recognize from its common use in ensuring that files are flushed andclosed when you write to them. Thewith statement opens a block, and if anexception occurs during that block, then the "context manager"—that is, the objectthat thewith runs on—has an opportunity to handle the exception. pytest takesadvantage of this with thepytest.raises context manager, which you can use in thefollowing way:

def test_double_str(monkeypatch):    with pytest.raises(ValueError):        monkeypatch.setattr('sys.stdin', str_inputs)        result = double()

Notice that you don't need anassert statement here, because thepytest.raises is,effectively, theassert statement. And, you do have to indicate the type of error(ValueError) that you're trying to trap, meaning what you expect to receive.

If you want to capture (or assert) something having to do with the exception that wasraised, you can use theas part of thewith statement. For example:

def test_double_str(monkeypatch):    with pytest.raises(ValueError) as e:        monkeypatch.setattr('sys.stdin', str_inputs)        results = double()    assert str(e.value) == "invalid literal for int()     ↪with base 10: 'abcd'"

Now you can be sure that not only was aValueError exception raised, but also whatmessage was raised.

I generally advise people not to useprint in their functions. After all, I'd liketo get some value back from a function; I don't really want to display something onthe screen. But at some point, you really do actually need to display things to theuser. How can you test for that?

The pytest solution is via thecapsys plugin. Similar tomonkeypatch, you declarecapsys as a parameter to your test function. You then run your function, allowingit to produce its output. Then you invoke thereadouterr function oncapsys,which returns a tuple of two strings containing the output tostdout and itserror-message counterpart,stderr. You then can run assertions on those strings.

For example, let's assume this function:

def hello(name):    print(f"Hello, {name}!")

You can test it in the following way:

def test_hello(capsys):    hello('world')    captured_stdout, captured_stderr = capsys.readouterr()    assert captured_stdout == 'Hello, world!'

But wait! This test actually fails:

E   AssertionError: assert 'Hello, world!\n' == 'Hello, world!'E     - Hello, world!E     ?              -E     + Hello, world!

Do you see the problem? The output, as written byprint, includes a trailingnewline (\n) character. But the test didn't check for that. Thus, you can check forthe trailing newline, or you can usestr.strip onstdout:

def test_hello(capsys):    hello('world')    captured_stdout, captured_stderr = capsys.readouterr()    assert captured_stdout.strip() == 'Hello, world!'

pytest continues to impress me as a testing framework, in no small part because itcombines a lot of small, simple ideas in ways that feel natural to me as a developer.It has gone a long way toward increasing my use of tests, both in generaldevelopment and in my teaching. My "Weekly Python Exercise" subscription service nowincludes tests, and I feel like it has improved a great deal as a result.

In my next article, I plan to take a third and final look at pytest, exploring some of the otherways it can interact with (and help) write robust and useful programs.

• Thepytest website.
• An excellent book on the subject is Brian Okken'sPython testing withpytest,published by Pragmatic Programmers. He also has many other resources, about pytestand code testing in general, athttp://pythontesting.net.
• "TestingYour Code with Python's pytest" by Reuven M. Lerner

Reuven Lerner teaches Python, data science and Git to companiesaround the world. You can subscribe to his free, weekly "betterdevelopers" e-mail list, and learn from his books and courses athttp://lerner.co.il. Reuven lives with his wife and children inModi'in, Israel.