Writing Tests for github3.py
Unit Tests
In computer programming, unit testing is a method by which individualunits of source code, sets of one or more computer program modulestogether with associated control data, usage procedures, and operatingprocedures are tested to determine if they are fit for use. Intuitively,one can view a unit as the smallest testable part of an application.
In github3.py we use unit tests to make assertions about how the librarybehaves without making a request to the internet. For example, one assertionwe might write would check if custom information is sent along in a request toGitHub.
An existing test like this can be found intests/unit/test_repos_release.py:
deftest_delete(self):self.instance.delete()self.session.delete.assert_called_once_with(self.example_data['url'],headers={'Accept':'application/vnd.github.manifold-preview'})
In this test, we check that the library passes on important headers to the APIto ensure the request will work properly.self.instance is created for usand is an instance of theRelease class. The test then callsdelete tomake a request to the API.self.session is a mock object which fakes out anormal session. It does not allow the request through but allows us to verifyhow github3.py makes a request. We can see that github3.py calleddeleteon the session. We assert that it was only called once and that the onlyparameters sent were a URL and the custom headers that we are concerned with.
Mocks
Above we talked about mock objects. What are they?
In object-oriented programming, mock objects are simulated objects thatmimic the behavior of real objects in controlled ways. A programmertypically creates a mock object to test the behavior of some other object,in much the same way that a car designer uses a crash test dummy tosimulate the dynamic behavior of a human in vehicle impacts.
We use mocks in github3.py to prevent the library from talking directly withGitHub. The mocks we use intercept requests the library makes so we can verifythe parameters we use. In the example above, we were able to check thatcertain parameters were the only ones sent to a session method because wemocked out the session.
You may have noticed in the example above that we did not have to set up themock object. There is a convenient helper written intests/unit/helper.pyto do this for you.
Example - Testing the Release Object
Here’s a full example of how we test theRelease object intests/unit/test_repos_release.py.
Our first step is to import theUnitHelper class fromtests/unit/helper.py and theRelease object fromgithub3/repos/release.py.
from.helperimportUnitHelperfromgithub3.repos.releaseimportRelease
Then we construct our test class and indicate which class we will be testing(or describing).
classTestRelease(UnitHelper):described_class=Release
We can then use theGitHub API documentation about Releases to retrieve example releasedata. We then can use that as example data for our test like so:
classTestRelease(UnitHelper):described_class=Releaseexample_data={"url":releases_url("/1"),"html_url":"https://github.com/octocat/Hello-World/releases/v1.0.0","assets_url":releases_url("/1/assets"),"upload_url":releases_url("/1/assets{?name}"),"id":1,"tag_name":"v1.0.0","target_commitish":"master","name":"v1.0.0","body":"Description of the release","draft":False,"prerelease":False,"created_at":"2013-02-27T19:35:32Z","published_at":"2013-02-27T19:35:32Z"}
The above code now will handle making clean and brand new instances of theRelease object with the example data and a faked out session. We can nowconstruct our first test.
deftest_delete(self):self.instance.delete()self.session.delete.assert_called_once_with(self.example_data['url'],headers={'Accept':'application/vnd.github.manifold-preview'})
Integration Tests
Integration testing is the phase in software testing in which individualsoftware modules are combined and tested as a group.
The purpose of integration testing is to verify functional, performance,and reliability requirements placed on major design items.
In github3.py we use integration tests to ensure that when we make what shouldbe a valid request to GitHub, it is in fact valid. For example, if we weretesting how github3.py requests a user’s information, we would expect arequest for a real user’s data to be valid. If the test fails we know eitherwhat the library is doing is wrong or the data requested does not exist.
An existing test that demonstrates integration testing can be found intests/integration/test_repos_release.py:
deftest_iter_assets(self):"""Test the ability to iterate over the assets of a release."""cassette_name=self.cassette_name('iter_assets')withself.recorder.use_cassette(cassette_name):repository=self.gh.repository('sigmavirus24','github3.py')release=repository.release(76677)forassetinrelease.iter_assets():assertisinstance(asset,github3.repos.release.Asset)assertassetisnotNone
In this test we useself.recorder to record our interaction with GitHub.We then proceed to make the request to GitHub that will exercise the code wewish to test. First we request aRepository object from GitHub and thenusing that we request aRelease object. After receiving that release, weexercise the code that lists the assets of aRelease. We verify that eachasset is an instance of theAsset class and that at the end theassetvariable is notNone. Ifasset wasNone, that would indicate thatGitHub did not return any data and it did not exercise the code we are tryingto test.
Betamax
Betamax is the library that we use to create the recorder above. It sets upthe session object to intercept every request and corresponding response andsave them to what it callscassettes. After you record the interaction itnever has to speak to the internet again for that request.
In github3.py there is a helper class (much likeUnitHelper) intests/integration/helper.py which sets everything up for us.
Example - Testing the Release Object
Here’s an example of how we write an integration test for github3.py. Theexample can be found intests/integration/test_repos_release.py.
Our first steps are the necessary imports.
importgithub3from.helperimportIntegrationHelper
Then we start writing our test right away.
classTestRelease(IntegrationHelper):deftest_delete(self):"""Test the ability to delete a release."""self.token_login()cassette_name=self.cassette_name('delete')withself.recorder.use_cassette(cassette_name):repository=self.gh.repository('github3py','github3.py')release=repository.create_release('0.8.0.pre','develop','0.8.0 fake release','To be deleted')assertreleaseisnotNoneassertrelease.delete()isTrue
Every test has access toself.gh which is an instance ofGitHub.IntegrationHelper provides a lot of methods that allow you to focus onwhat we are testing instead of setting up for the test. The first of thosemethods we see in use isself.token_login which handles authenticatingwith a token. It’s sister method isself.basic_login which handlesauthentication with basic credentials. Both of these methods will set up theauthentication for you onself.gh.
The next convenience method we see isself.cassette_name. It constructs acassette name for you based on the test class name and the string you provideit.
Every test also has access toself.recorder. This is the Betamax recorderthat has been set up for you to record your interactions. The recorder isstarted when you write
withself.recorder.use_cassette(cassette_name):# …
Everything that talks to GitHub should be written inside of the contextcreated by the context manager there. No requests to GitHub should be madeoutside of that context.
In that context, we then retrieve a repository and create a release for it. Wewant to be sure that we will be deleting something that exists so we assertthat what we received back from GitHub is notNone. Finally we calldelete and assert that it returnsTrue.
When you write your new test and record a new cassette, be sure to add the newcassette file to the repository, like so:
gitaddtests/cassettes/Release_delete.json
Recording Cassettes that Require Authentication/Authorization
If you need to write a test that requires an Authorization (i.e., OAuth token)or Authentication (i.e., username and password), all you need to do is setenvironment variables when runningpy.test, e.g.,
GH_AUTH="abc123"py.testGH_USER="sigmavirus24"GH_PASSWORD="super-secure-password-plz-kthxbai"py.test
If you are concerned that your credentials will be saved, you need not worry.Betamax sanitizes information like that before saving the cassette. It neverdoes hurt to double check though.