Posted onMar 7, 2018 • Edited onOct 3, 2018 • Originally published atdmerej.info

Porting to pytest: a practical example

#python #django #testing

Originally published onmy blog.

Introduction

The other day I was following thedjango tutorial.

If you never read the tutorial, or don't want to, here's what you need to know:

We have a django project containing an application calledpolls.

We have two model objects representing questions and choices.

Each question has a publication date, a text, and a list of choices.

Each choice has a reference to an existing question (via a foreign key), a text, and a number of votes.

There's a view that shows a list of questions as links. Each link, when clicked displays the choices and has a form to let the user vote.

The code is pretty straightforward:

# polls/models.pyclassQuestion(models.Model):question_text=models.CharField(max_length=200)pub_date=models.DateTimeField('date published')defwas_published_recently(self):now=timezone.now()two_days_ago=now-datetime.timedelta(days=2)returntwo_days_ago<self.pub_date<=nowclassChoice(models.Model):question=models.ForeignKey(Question,on_delete=models.CASCADE)choice_text=models.CharField(max_length=200)

Everything went smoothly until I arrived at thepart 5, about automated testing, where I read the following:

Sometimes it may seem a chore to tear yourself away from your productive, creative programming work to face the unglamorous and unexciting business of writing tests, particularly when you know your code is working properly.

Well, allow me to retort!

Starting point: using tests from the documentation

Here's what the tests looks like when extracted from the django documentation:

importdatetimefromdjango.utilsimporttimezonefromdjango.testimportTestCasefrom.modelsimportQuestionclassQuestionModelTests(TestCase):deftest_was_published_recently_with_future_question(self):"""        was_published_recently() returns False for questions whose pub_date        is in the future.        """time=timezone.now()+datetime.timedelta(days=30)future_question=Question(pub_date=time)self.assertIs(future_question.was_published_recently(),False)deftest_was_published_recently_with_old_question(self);"""        was_published_recently() returns False for questions whose pub_date        is older than 1 day.        """time=timezone.now()-datetime.timedelta(days=1,seconds=1)old_question=Question(pub_date=time)self.assertIs(old_question.was_published_recently(),False)deftest_was_published_recently_with_recent_question(self):"""        was_published_recently() returns True for questions whose pub_date        is within the last day.        """time=timezone.now()-datetime.timedelta(hours=23,minutes=59,seconds=59)recent_question=Question(pub_date=time)self.assertIs(recent_question.was_published_recently(),True)defcreate_question(question_text,days):"""    Create a question with the given `question_text` and published the    given number of `days` offset to now (negative for questions published    in the past, positive for questions that have yet to be published).    """time=timezone.now()+datetime.timedelta(days=days)returnQuestion.objects.create(question_text=question_text,pub_date=time)classQuestionIndexViewTests(TestCase):deftest_no_questions(self):"""        If no questions exist, an appropriate message is displayed.        """response=self.client.get(reverse('polls:index'))self.assertEqual(response.status_code,200)self.assertContains(response,"No polls are available.")self.assertQuerysetEqual(response.context['latest_question_list'],[])deftest_past_question(self):"""        Questions with a pub_date in the past are displayed on the        index page.        """create_question(question_text="Past question.",days=-30)response=self.client.get(reverse('polls:index'))self.assertQuerysetEqual(response.context['latest_question_list'],['<Question: Past question.>'])deftest_future_question(self):"""        Questions with a pub_date in the future aren't displayed on        the index page.        """create_question(question_text="Future question.",days=30)response=self.client.get(reverse('polls:index'))self.assertContains(response,"No polls are available.")self.assertQuerysetEqual(response.context['latest_question_list'],[])deftest_future_question_and_past_question(self):"""        Even if both past and future questions exist, only past questions        are displayed.        """create_question(question_text="Past question.",days=-30)create_question(question_text="Future question.",days=30)response=self.client.get(reverse('polls:index'))self.assertQuerysetEqual(response.context['latest_question_list'],['<Question: Past question.>'])deftest_two_past_questions(self):"""        The questions index page may display multiple questions.        """create_question(question_text="Past question 1.",days=-30)create_question(question_text="Past question 2.",days=-5)response=self.client.get(reverse('polls:index'))self.assertQuerysetEqual(response.context['latest_question_list'],['<Question: Past question 2.>','<Question: Past question 1.>'])

We can run them using themanage.py script and check they all pass:

$python manage.pytestpollsCreating test database for alias 'default'...System check identified no issues (0 silenced).........---------------------------------------------------------------------------Ran 8 tests in 0.017sOKDestroying test database for alias 'default'...

OK, tests do pass. Let's try and improve them.

I've set upa GitHub repository where you can follow the following steps commit by commit if you wish.

Step one: setup pytest

I've already told youhow much I love pytest, so let's try to convert topytest.

The first step is to installpytest-django and configure it:

$pipinstallpytest pytest-django

# in pytest.ini[pytest]DJANGO_SETTINGS_MODULE=mysite.settingspython_files = tests.py test_*.py

We can now run tests usingpytest directly:

$pytest========== test session starts ========platform linux -- Python 3.5.3, pytest-3.3.1, py-1.5.2, pluggy-0.6.0Django settings: mysite.settings (from ini file)rootdir: /home/dmerej/src/dmerej/django-polls, inifile: pytest.iniplugins: django-3.1.2collected 8 itemspolls/tests.py ........   [100%]======== 8 passed in 0.18 seconds =======

Step two: rewrite assertions

We can now usepytest magic to rewrite all "easy" assertions such asassertFalse orassertEquals:

- self.assertFalse(future_question.was_published_recently())+ assert not future_question.was_published_recently()

Already we can see several improvements:

The code is more readable and follows PEP8
The error messages are more detailed:

#Before, with unittest$python manage.pytest    def test_was_published_recently_with_future_question(self):        ...>self.assertFalse(question.was_published_recently())E       AssertionError: True is not false#After, with pytest$pytest>assert not question.was_published_recently()E       AssertionError: assert not TrueE        +  where True = <bound method was_published_recently() of Question>

Then we have to deal withassertContains andassertQuerysetEqual which look a bit django-specific.

ForassertContains I quickly managed to find I could useresponse.rendered_content instead:

- self.assertContains(response, "No polls are available.")+ assert "No polls are available." in response.rendered_content

ForassertQuerysetEqual it was a bit harder.

deftest_past_question(self):create_question(question_text="Past question.",days=-30)response=self.client.get(reverse('polls:index'))self.assertQuerysetEqual(response.context['latest_question_list'],['<Question: Past question.>'])

This test checks that the context used to generate the response was passed correctlatest_question_list value.

But it does so by checking thestring representation of theQuestion object.

Thus, it will break as soon asQuestion.__str__ changes, which is not ideal.

So instead, we can write something like this and check for the content of thequestion_text attribute directly:

deftest_past_question(self):create_question(question_text="Past question.",days=-30)response=self.client.get(reverse('polls:index'))actual_questions=response.context['latest_question_list']assertlen(actual_questions)==1actual_question=actual_questions[0]assertactual_question.question_text=="Past question"

While we're at it, we can introduce small helper functions to make the tests easier to read:

For instance, the stringNo polls are available is hard-coded twice in the tests. Let's introduce aassert_no_polls helper:

defassert_no_polls(text):assert"No polls are available"intext

- assert "No polls are available." in response.rendered_content+ assert_no_polls(response.rendered_content)

An other hard-coded string ispolls:index, so let's introduceget_latest_list:

defget_latest_list(client):response=client.get(reverse('polls:index'))assertresponse.status_code==200returnresponse.context['latest_question_list']

Note how we embedded the status code check directly in our helper, so we don't have to repeat the check in each test.

Also, note that if the name of the route (polls:index) or the name of the context key used in the template (latest_question_list) ever changes, we'll just need to update the test code in one place.

Then, we can further simplify our assertions:

defassert_question_list_equals(actual_questions,expected_texts):assertlen(actual_questions)==len(expected_texts)foractual_question,expected_textinzip(actual_questions,expected_texts):assertactual_question.question_text==expected_textdeftest_past_question(self):...create_question(question_text="Past question.",days=-30)latest_list=get_latest_list(self.client)assert_question_list_equals(latest_list,["Past question."])

Step three: move code out of classes

The nice thing aboutpytest is that you don't need to put your tests as methods of a class, you can just write
test functions directly.

So we just remove theself parameter, indent back all the code, and we are (almost) good to go.

We already got rid of all theself.assert* methods, so the last thing to do is pass the Django test client as a parameter instead of usingself.client. (That's howpytest fixtures work):

-    def test_two_past_questions(self):-        ...-        latest_list = get_latest_list(self.client)+ def test_no_questions(client):+    latest_list = get_latest_list(client)

But then we encounter an unexpected failure:

Polls/tests.py:34: in create_question    return Question.objects.create(question_text=question_text, pub_date=time)    ...>       self.ensure_connection()E       Failed: Database access not allowed, use the "django_db" mark, or the "db" or "transactional_db" fixtures to enable it.

Back when we usedpython manage.py test, django'smanage.py script was implicitly creating a test database for us.

When we usepytest, we have to be explicit about it and add a special marker:

importpytest# No change here, no need for a DBdeftest_was_published_recently_with_old_question():...# We use create_question, which in turn calls Question.objects.create(),# so we need a database here:@pytest.mark.django_dbdeftest_no_questions(client):...

True, this is a bit annoying, but note that if we only want to test the models themselves (like thewas_published_recently() method), we can just use:

$pytest-k was_published_recently

and no database will be createdat all.

Step four: Get rid of doc strings

I don't like doc strings,except when I'm implementing a very public API. There, I've said it.

I very much prefer when the code is "self-documenting",especially when it's test code.

AsUncle Bob said, "tests should read like well-written specifications". So let's try some refactoring.

We can start with more meaningful variable names, and have more fun with the examples:

def test_was_published_recently_with_old_question():-   time = timezone.now() - datetime.timedelta(days=1, seconds=1)-   old_question = Question(pub_date=time)+   last_year = timezone.now() - datetime.timedelta(days=365)+   old_question = Question('Why is there something instead of nothing?',+                            pub_date=last_year)    assert not old_question.was_published_recently()def test_was_published_recently_with_recent_question():-   time = timezone.now() - datetime.timedelta(days=1, seconds=1)-   recent_question = Question(pub_date=time)+   last_night = timezone.now() - datetime.timedelta(hours=10)+   recent_question = Question('Dude, where is my car?', pub_date=last_night)

Time and date code is always tricky, and a negative number of days does not really make sense, so let's make things easier to reason about:

defn_days_ago(n):returntimezone.now()-datetime.timedelta(days=n)defn_days_later(n):returntimezone.now()+datetime.timedelta(days=n)

Alsocreate_question is coupled with theQuestion model, so let's use the same names for the parameter names and the model's attributes.

And since we may want to create question without caring about the publication date, let's make it an optional parameter:

defcreate_question(question_text,*,pub_date=None):ifnotpub_date:pub_date=timezone.now()...

Code becomes:

-    create_question(question_text="Past question.", days=-30)+    create_question(question_text="Past question.", pub_date=n_days_ago(30))

Finally, let's add a new test to see if our helpers really work:

@pytest.mark.django_dbdeftest_latest_five(client):foriinrange(0,10):pub_date=n_days_ago(i)create_question("Question #%s"%i,pub_date=pub_date)latest_list=get_latest_list(client)assertlen(actual_list)==5

Do you still think this test needs a docstring ?

Step five: fun with selenium

Selenium basics

Selenium deals with browser automation.

Here we are going to use the Python bindings, which allow us to startFirefox orChrome and control them with code.

(In both cases, you'll need to install a separate binary:geckodriver orchromdriver respectively)

Here's how you can useselenium do visit a web page and click the first link:

importselenium.webdriverdriver=selenium.webdriver.Firefox()# ordriver=selenium.webdriver.Chrome()driver.get("http://example.com")link=driver.find_element_by_tag_name('a')link.click()

The Live Server Test Case

Django exposes aLiveServerTestCase, but noLiveServer object or similar.

The code is a bit tricky because it needs to spawn a "real" server in a separate thread, make sure it uses a free port, and tell the selenium driver to use an URL likehttp://localhost:32456

Fear not,pytest also works fine in this case. We just have to be careful to usesuper() in the set up and tear down methods so that the code fromLiveServerTestCase gets executed properly:

importurllib.parseclassTestPolls(LiveServerTestCase):serialized_rollback=TruedefsetUp(self):super().setUp()self.driver=selenium.webdriver.Firefox()deftearDown(self):self.driver.close()super().tearDown()deftest_home_no_polls(self):url=urllib.parse.urljoin(self.live_server_url,"/polls")self.driver.get(url)assert_no_polls(self.browser.page_source)

If you're wondering why we needserialized_rollback=True, the answer is inthe documentation. Without it we may have weird database errors during tests.

Our first test is pretty basic: we ask the browser to visit the'polls/ URL and check no polls are shown, re-using ourassert_no_polls helper function from before.

Let's also check we are shown links to the questions if they are some, and can click on them:

classTestPolls(LiveServerTestCase):...deftest_home_list_polls(self):create_question("One?")create_question("Two?")create_question("Three?")url=urllib.parse.urljoin(self.live_server_url,"polls/")self.driver.get(url)first_link=self.driver.find_element_by_tag_name("a")first_link.click()assert"Three?"inself.driver.page_source

Let's build a facade

Thefind_element_by_* methods of the selenium API are a bit tedious to use: thery are calledfind_element_by_tag_name,find_element_by_class_name,find_element_by_id and so on

So let's write aBrowser class to hide those behind a more "Pythonic" API:

# oldlink=driver.find_element_by_tag_name("link")form=driver.find_element_by_id("form-id")# newlink=driver.find_element(tag_name="link")form=driver.find_element(id="form-id")

(This is known as the "facade" design attern)

classBrowser:""" A nice facade on top of selenium stuff """def__init__(self,driver):self.driver=driverdeffind_element(self,**kwargs):assertlen(kwargs)==1# we want exactly one named parameter herename,value=list(kwargs.items())[0]func_name="find_element_by_"+namefunc=getattr(self.driver,func_name)returnfunc(value)

Note how we have to convert theitems() to a real list just to get the first element... (In Python2,kwargs.items()[0] would have worked just fine). Please tell me if you find a better way ...

Note also how wedon't just inherit fromselenium.webdriver.Firefox. The goal is to expose adifferent API, so using composition here is better.

If we need access to attributes ofself.driver, we can just use a property, like this:

classBrowser...@propertydefpage_source(self):returnself.driver.page_source

And if we need to call a method directly to the underlying object, we can just forward tho call:

defclose(self):self.driver.close()

We can also hide the uglyurllib.parse.urljoin(self.live_server_url) implementation detail:

classBrowser:def__init__(self,driver):self.driver=driverself.live_server_url=None# will be set during test set updefget(self,url):full_url=urllib.parse.urljoin(self.live_server_url,url)self.driver.get(full_url)classTestPolls(LiveServerTestCase):defsetUp(self):super().setUp()driver=selenium.webdriver.Firefox()self.browser=Browser(driver)self.browser.live_server_url=self.live_server_url

Now the test reads:

deftest_home_no_polls(self):self.browser.get("/polls")assert_no_polls(self.browser.page_source)

Nice and short :)

Launching the driver only once

ThesetUp() method is called before each test, so if we add more tests we're going to create tons of instances of Firefox drivers.

Let's fix this by usingsetUpClass (and not forgetting thesuper() call)

classTestPolls(LiveServerTestCase):@classmethoddefsetUpClass(cls):super().setUpClass()driver=webdriver.Chrome()cls.browser=Browser(driver)defsetUp(self):self.browser.base_url=self.live_server_url@classmethoddeftearDownClass(cls):cls.browser.close()super().tearDownClass()

Now thebrowser is aclass attribute instead of being aninstance attribute. So there's only oneBrowser object for the whole test suite, which is what we wanted.

The rest of the code can still useself.browser, though.

Debugging tests

One last thing. You may think debugging such high-level tests would be painful.

But it's actually a pretty nice experience due to just one thing: the built-in Python debugger!

Just add something like:

deftest_login():self.browser.get("/login")importpdb;pdb.set_trace()

and then run the tests like this:

$pytest-k login-s

(The-s is required to avoid capturing output, whichpdp does not like)

And then, as soon as the tests reaches the line withpdb.set_trace() you will have:

A brand new Firefox instance running, with access to all the nice debugging tools (so you can quickly find out things like ids or CSS class names)
... and a nice REPL where you'll be able to try out the code usingself.browser

By the way, the REPL will be even nicer if you useipdb orpdbpp and enjoy auto-completion and syntax coloring right from the REPL :)

Conclusion

I hope I managed to show that you actuallycan get creative writing tests, and even have some fun.

See you next time!

Thanks for reading this far :)

I'd love to hear what you have to say, so please feel free to leave a comment below, or read thefeedback page for more ways to get in touch with me.