Automated vs. Manual Testing

The good news is, you’ve probably already created a test without realizing it. Remember when you ran your application and used it for the first time? Did you check the features and experiment using them? That’s known asexploratory testing and is a form of manual testing.

Exploratory testing is a form of testing that is done without a plan. In an exploratory test, you’re just exploring the application.

To have a complete set of manual tests, all you need to do is make a list of all the features your application has, the different types of input it can accept, and the expected results. Now, every time you make a change to your code, you need to go through every single item on that list and check it.

That doesn’t sound like much fun, does it?

This is where automated testing comes in. Automated testing is the execution of your test plan (the parts of your application you want to test, the order in which you want to test them, and the expected responses) by a script instead of a human. Python already comes with a set of tools and libraries to help you create automated tests for your application. We’ll explore those tools and libraries in this tutorial.

Unit Tests vs. Integration Tests

The world of testing has no shortage of terminology, and now that you know the difference between automated and manual testing, it’s time to go a level deeper.

Think of how you might test the lights on a car. You would turn on the lights (known as thetest step) and go outside the car or ask a friend to check that the lights are on (known as thetest assertion). Testing multiple components is known asintegration testing.

Think of all the things that need to work correctly in order for a simple task to give the right result. These components are like the parts to your application, all of those classes, functions, and modules you’ve written.

A major challenge with integration testing is when an integration test doesn’t give the right result. It’s very hard to diagnose the issue without being able to isolate which part of the system is failing. If the lights didn’t turn on, then maybe the bulbs are broken. Is the battery dead? What about the alternator? Is the car’s computer failing?

If you have a fancy modern car, it will tell you when your light bulbs have gone. It does this using a form ofunit test.

A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.

You have just seen two types of tests:

1. An integration test checks that components in your application operate with each other.
2. A unit test checks a small component in your application.

You can write both integration tests and unit tests in Python. To write a unit test for the built-in functionsum(), you would check the output ofsum() against a known output.

For example, here’s how you check that thesum() of the numbers(1, 2, 3) equals6:

>>>assertsum([1,2,3])==6,"Should be 6"

This will not output anything on the REPL because the values are correct.

If the result fromsum() is incorrect, this will fail with anAssertionError and the message"Should be 6". Try an assertion statement again with the wrong values to see anAssertionError:

>>>assertsum([1,1,1])==6,"Should be 6"Traceback (most recent call last):  File"<stdin>", line1, in<module>AssertionError:Should be 6

In the REPL, you are seeing the raisedAssertionError because the result ofsum() does not match6.

Instead of testing on the REPL, you’ll want to put this into a new Python file calledtest_sum.py and execute it again:

deftest_sum():assertsum([1,2,3])==6,"Should be 6"if__name__=="__main__":test_sum()print("Everything passed")

Now you have written atest case, an assertion, and an entry point (the command line). You can now execute this at the command line:

$pythontest_sum.pyEverything passed

You can see the successful result,Everything passed.

In Python,sum() accepts any iterable as its first argument. You tested with a list. Now test with a tuple as well. Create a new file calledtest_sum_2.py with the following code:

deftest_sum():assertsum([1,2,3])==6,"Should be 6"deftest_sum_tuple():assertsum((1,2,2))==6,"Should be 6"if__name__=="__main__":test_sum()test_sum_tuple()print("Everything passed")

When you executetest_sum_2.py, the script will give an error because thesum() of(1, 2, 2) is5, not6. The result of the script gives you the error message, the line of code, and the traceback:

$pythontest_sum_2.pyTraceback (most recent call last):  File "test_sum_2.py", line 9, in <module>    test_sum_tuple()  File "test_sum_2.py", line 5, in test_sum_tuple    assert sum((1, 2, 2)) == 6, "Should be 6"AssertionError: Should be 6

Here you can see how a mistake in your code gives an error on the console with some information on where the error was and what the expected result was.

Writing tests in this way is okay for a simple check, but what if more than one fails? This is where test runners come in. The test runner is a special application designed for running tests, checking the output, and giving you tools for debugging and diagnosing tests and applications.

Choosing a Python Test Runner

There are many test runners available for Python. The one built into the Python standard library is calledunittest. In this tutorial, you will be usingunittest test cases and theunittest test runner. The principles ofunittest are easily portable to other frameworks. The three most popular test runners are:

• unittest
• nose ornose2
• pytest

Choosing the best test runner for your requirements and level of experience is important.

importunittestclassTestSum(unittest.TestCase):deftest_sum(self):self.assertEqual(sum([1,2,3]),6,"Should be 6")deftest_sum_tuple(self):self.assertEqual(sum((1,2,2)),6,"Should be 6")if__name__=="__main_":unittest.main()

$pythontest_sum_unittest.py.F======================================================================FAIL: test_sum_tuple (__main__.TestSum)----------------------------------------------------------------------Traceback (most recent call last):  File "test_sum_unittest.py", line 9, in test_sum_tuple    self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")AssertionError: Should be 6----------------------------------------------------------------------Ran 2 tests in 0.001sFAILED (failures=1)

$pipinstallnose2$python-mnose2.F======================================================================FAIL: test_sum_tuple (__main__.TestSum)----------------------------------------------------------------------Traceback (most recent call last):  File "test_sum_unittest.py", line 9, in test_sum_tuple    self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")AssertionError: Should be 6----------------------------------------------------------------------Ran 2 tests in 0.001sFAILED (failures=1)

deftest_sum():assertsum([1,2,3])==6,"Should be 6"deftest_sum_tuple():assertsum((1,2,2))==6,"Should be 6"

Where to Write the Test

To get started writing tests, you can simply create a file calledtest.py, which will contain your first test case. Because the file will need to be able to import your application to be able to test it, you want to placetest.py above the package folder, so your directory tree will look something like this:

project/│├── my_sum/│   └── __init__.py|└── test.py

You’ll find that, as you add more and more tests, your single file will become cluttered and hard to maintain, so you can create a folder calledtests/ and split the tests into multiple files. It is convention to ensure each file starts withtest_ so all test runners will assume that Python file contains tests to be executed. Some very large projects split tests into more subdirectories based on their purpose or usage.

target=__import__("my_sum.py")sum=target.sum

How to Structure a Simple Python Test

Before you dive into writing tests, you’ll want to first make a couple of decisions:

1. What do you want to test?
2. Are you writing a unit test or an integration test?

Then the structure of a test should loosely follow this workflow:

1. Create your inputs
2. Execute the code being tested, capturing the output
3. Compare the output with an expected result

For this application, you’re testingsum(). There are many behaviors insum() you could check, such as:

• Can it sum a list of whole numbers (integers)?
• Can it sum a tuple or set?
• Can it sum a list of floats?
• What happens when you provide it with a bad value, such as a single integer or a string?
• What happens when one of the values is negative?

The most simple test would be a list of integers. Create a file,test.py with the following Python code:

importunittestfrommy_sumimportsumclassTestSum(unittest.TestCase):deftest_list_int(self):"""        Test that it can sum a list of integers        """data=[1,2,3]result=sum(data)self.assertEqual(result,6)if__name__=="__main_":unittest.main()

This code example:

1. Importssum() from themy_sum package you created

2. Defines a new test case class calledTestSum, which inherits fromunittest.TestCase

3. Defines a test method,.test_list_int(), to test a list of integers. The method.test_list_int() will:

   • Declare a variabledata with a list of numbers(1, 2, 3)
   • Assign the result ofmy_sum.sum(data) to aresult variable
   • Assert that the value ofresult equals6 by using the.assertEqual() method on theunittest.TestCase class

4. Defines a command-line entry point, which runs theunittest test-runner.main()

If you’re unsure whatself is or how.assertEqual() is defined, you can brush up on your object-oriented programming withPython Object-Oriented Programming.

How to Write Assertions in Python

The last step of writing a test is to validate the output against a known response. This is known as anassertion. There are some general best practices around how to write assertions:

• Make sure tests are repeatable and run your test multiple times to make sure it gives the same result every time
• Try and assert results that relate to your input data, such as checking that the result is the actual sum of values in thesum() example

unittest comes with lots of methods to assert on the values, types, and existence of variables. Here are some of the most commonly used methods:

.assertIs(),.assertIsNone(),.assertIn(), and.assertIsInstance() all have opposite methods, named.assertIsNot(), and so forth.

Side Effects

When you’re writing tests, it’s often not as simple as looking at the return value of a function. Often, executing a piece of code will alter other things in the environment, such as the attribute of a class, a file on the filesystem, or a value in a database. These are known asside effects and are an important part of testing. Decide if the side effect is being tested before including it in your list of assertions.

If you find that the unit of code you want to test has lots of side effects, you might be breaking theSingle Responsibility Principle. Breaking the Single Responsibility Principle means the piece of code is doing too many things and would be better off being refactored. Following the Single Responsibility Principle is a great way to design code that it is easy to write repeatable and simple unit tests for, and ultimately, reliable applications.

Executing Python Test Runners

The Python application that executes your test code, checks the assertions, and gives you test results in your console is called thetest runner.

At the bottom oftest.py, you added this small snippet of code:

if__name__=="__main_":unittest.main()

This is a command line entry point. It means that if you execute the script alone by runningpython test.py at the command line, it will callunittest.main(). This executes the test runner by discovering all classes in this file that inherit fromunittest.TestCase.

This is one of many ways to execute theunittest test runner. When you have a single test file namedtest.py, callingpython test.py is a great way to get started.

Another way is using theunittest command line. Try this:

$python-munittesttest

This will execute the same test module (calledtest) via the command line.

You can provide additional options to change the output. One of those is-v for verbose. Try that next:

$python-munittest-vtesttest_list_int (test.TestSum) ... ok----------------------------------------------------------------------Ran 1 tests in 0.000s

This executed the one test insidetest.py and printed the results to the console. Verbose mode listed the names of the tests it executed first, along with the result of each test.

Instead of providing the name of a module containing tests, you can request an auto-discovery using the following:

$python-munittestdiscover

This will search the current directory for any files namedtest*.py and attempt to test them.

Once you have multiple test files, as long as you follow thetest*.py naming pattern, you can provide the name of the directory instead by using the-s flag and the name of the directory:

$python-munittestdiscover-stests

unittest will run all tests in a single test plan and give you the results.

Lastly, if your source code is not in the directory root and contained in a subdirectory, for example in a folder calledsrc/, you can tellunittest where to execute the tests so that it can import the modules correctly with the-t flag:

$python-munittestdiscover-stests-tsrc

unittest will change to thesrc/ directory, scan for alltest*.py files inside the thetests directory, and execute them.

Understanding Test Output

That was a very simple example where everything passes, so now you’re going to try a failing test and interpret the output.

sum() should be able to accept other lists of numeric types, like fractions.

At the top of thetest.py file, add an import statement to import theFraction type from thefractions module in the standard library:

fromfractionsimportFraction

Now add a test with an assertion expecting the incorrect value, in this case expecting the sum of 1/4, 1/4, and 2/5 to be 1:

importunittestfrommy_sumimportsumclassTestSum(unittest.TestCase):deftest_list_int(self):"""        Test that it can sum a list of integers        """data=[1,2,3]result=sum(data)self.assertEqual(result,6)deftest_list_fraction(self):"""        Test that it can sum a list of fractions        """data=[Fraction(1,4),Fraction(1,4),Fraction(2,5)]result=sum(data)self.assertEqual(result,1)if__name__=="__main_":unittest.main()

If you execute the tests again withpython -m unittest test, you should see the following output:

$python-munittesttestF.======================================================================FAIL: test_list_fraction (test.TestSum)----------------------------------------------------------------------Traceback (most recent call last):  File "test.py", line 21, in test_list_fraction    self.assertEqual(result, 1)AssertionError: Fraction(9, 10) != 1----------------------------------------------------------------------Ran 2 tests in 0.001sFAILED (failures=1)

In the output, you’ll see the following information:

1. The first line shows the execution results of all the tests, one failed (F) and one passed (.).

2. TheFAIL entry shows some details about the failed test:

   • The test method name (test_list_fraction)
   • The test module (test) and the test case (TestSum)
   • A traceback to the failing line
   • The details of the assertion with the expected result (1) and the actual result (Fraction(9, 10))

Remember, you can add extra information to the test output by adding the-v flag to thepython -m unittest command.

Running Your Tests From PyCharm

If you’re using thePyCharm IDE, you can rununittest orpytest by following these steps:

1. In the Project tool window, select thetests directory.
2. On the context menu, choose the run command forunittest. For example, chooseRun ‘Unittests in my Tests…’.

This will executeunittest in a test window and give you the results within PyCharm:

More information is available on thePyCharm Website.

Running Your Tests From Visual Studio Code

If you’re using the Microsoft Visual Studio Code IDE, support forunittest,nose, andpytest execution is built into the Python plugin.

If you have the Python plugin installed, you can set up the configuration of your tests by opening the Command Palette withCtrl+Shift+P and typing “Python test”. You will see a range of options:

ChooseDebug All Unit Tests, and VSCode will then raise a prompt to configure the test framework. Click on the cog to select the test runner (unittest) and the home directory (.).

Once this is set up, you will see the status of your tests at the bottom of the window, and you can quickly access the test logs and run the tests again by clicking on these icons:

This shows the tests are executing, but some of them are failing.

Why They’re Different From Other Applications

Think of all the code you’re going to be testing in a web application. The routes, views, and models all require lots of imports and knowledge about the frameworks being used.

This is similar to the car test at the beginning of the tutorial: you have to start up the car’s computer before you can run a simple test like checking the lights.

Django and Flask both make this easy for you by providing a test framework based onunittest. You can continue writing tests in the way you’ve been learning but execute them slightly differently.

How to Use the Django Test Runner

The Djangostartapp template will have created atests.py file inside your application directory. If you don’t have that already, you can create it with the following contents:

fromdjango.testimportTestCaseclassMyTestCase(TestCase):# Your test methods

The major difference with the examples so far is that you need to inherit from thedjango.test.TestCase instead ofunittest.TestCase. These classes have the same API, but the DjangoTestCase class sets up all the required state to test.

To execute your test suite, instead of usingunittest at the command line, you usemanage.py test:

$pythonmanage.pytest

If you want multiple test files, replacetests.py with a folder calledtests, insert an empty file inside called__init__.py, and create yourtest_*.py files. Django will discover and execute these.

More information is available at theDjango Documentation Website.

How to Useunittest and Flask

Flask requires that the app be imported and then set in test mode. You can instantiate a test client and use the test client to make requests to any routes in your application.

All of the test client instantiation is done in thesetUp method of your test case. In the following example,my_app is the name of the application. Don’t worry if you don’t know whatsetUp does. You’ll learn about that in theMore Advanced Testing Scenarios section.

The code within your test file should look like this:

importmy_appimportunittestclassMyTestCase(unittest.TestCase):defsetUp(self):my_app.app.testing=Trueself.app=my_app.app.test_client()deftest_home(self):result=self.app.get("/")# Make your assertions

You can then execute the test cases using thepython -m unittest discover command.

More information is available at Flask’s officialdocumentation on testing.

Handling Expected Failures

Earlier, when you made a list of scenarios to testsum(), a question came up: What happens when you provide it with a bad value, such as a single integer or a string?

In this case, you would expectsum() to throw an error. When it does throw an error, that would cause the test to fail.

There’s a special way to handle expected errors. You can use.assertRaises() as a context-manager, then inside thewith block execute the test steps:

importunittestfrommy_sumimportsumclassTestSum(unittest.TestCase):deftest_list_int(self):"""        Test that it can sum a list of integers        """data=[1,2,3]result=sum(data)self.assertEqual(result,6)deftest_list_fraction(self):"""        Test that it can sum a list of fractions        """data=[Fraction(1,4),Fraction(1,4),Fraction(2,5)]result=sum(data)self.assertEqual(result,1)deftest_bad_type(self):data="banana"withself.assertRaises(TypeError):result=sum(data)if__name__=="__main_":unittest.main()

This test case will now only pass ifsum(data) raises aTypeError. You can replaceTypeError with any exception type you choose.

Isolating Behaviors in Your Application

Earlier in the tutorial, you learned what a side effect is. Side effects make unit testing harder since, each time a test is run, it might give a different result, or even worse, one test could impact the state of the application and cause another test to fail:

There are some simple techniques you can use to test parts of your application that have many side effects:

• Refactoring code to follow the Single Responsibility Principle
• Mocking out any method or function calls to remove side effects
• Using integration testing instead of unit testing for this piece of the application

If you’re not familiar with mocking, seePython CLI Testing for some great examples.

Writing Integration Tests in Python

So far, you’ve been learning mainly about unit testing. Unit testing is a great way to build predictable and stable code. But at the end of the day, your application needs to work when it starts!

Integration testing is the testing of multiple components of the application to check that they work together. Integration testing might require acting like a consumer or user of the application by:

• Calling an HTTP REST API
• Calling a Python API
• Calling a web service
• Running a command line

Each of these types of integration tests can be written in the same way as a unit test, following the Input, Execute, and Assert pattern. The most significant difference is that integration tests are checking more components at once and therefore will have more side effects than a unit test. Also, integration tests will require more fixtures to be in place, like a database, a network socket, or a configuration file.

This is why it’s good practice to separate your unit tests and your integration tests. The creation of fixtures required for an integration like a test database and the test cases themselves often take a lot longer to execute than unit tests. Therefore, you may only want to run integration tests before you push to production instead of once on every commit.

A simple way to separate unit and integration tests is simply to put them in different folders:

project/│├── my_app/│   └── __init__.py│└── tests/    |    ├── unit/    |   ├── __init__.py    |   └── test_sum.py    |    └── integration/        ├── __init__.py        └── test_integration.py

There are many ways to execute only a select group of tests. The specify source directory flag,-s, can be added tounittest discover with the path containing the tests:

$python-munittestdiscover-stests/integration

unittest will have given you the results of all the tests within thetests/integration directory.

Testing Data-Driven Applications

Many integration tests will require backend data like a database to exist with certain values. For example, you might want to have a test that checks that the application displays correctly with more than 100 customers in the database, or the order page works even if the product names are displayed in Japanese.

These types of integration tests will depend on different test fixtures to make sure they are repeatable and predictable.

A good technique to use is to store the test data in a folder within your integration testing folder calledfixtures to indicate that it contains test data. Then, within your tests, you can load the data and run the test.

Here’s an example of that structure if the data consisted of JSON files:

project/│├── my_app/│   └── __init__.py│└── tests/    |    └── unit/    |   ├── __init__.py    |   └── test_sum.py    |    └── integration/        |        ├── fixtures/        |   ├── test_basic.json        |   └── test_complex.json        |        ├── __init__.py        └── test_integration.py

Within your test case, you can use the.setUp() method to load the test data from a fixture file in a known path and execute many tests against that test data. Remember you can have multiple test cases in a single Python file, and theunittest discovery will execute both. You can have one test case for each set of test data:

importunittestclassTestBasic(unittest.TestCase):defsetUp(self):# Load test dataself.app=App(database="fixtures/test_basic.json")deftest_customer_count(self):self.assertEqual(len(self.app.customers),100)deftest_existence_of_customer(self):customer=self.app.get_customer(id=10)self.assertEqual(customer.name,"Org XYZ")self.assertEqual(customer.address,"10 Red Road, Reading")classTestComplexData(unittest.TestCase):defsetUp(self):# Load test dataself.app=App(database="fixtures/test_complex.json")deftest_customer_count(self):self.assertEqual(len(self.app.customers),10000)deftest_existence_of_customer(self):customer=self.app.get_customer(id=9999)self.assertEqual(customer.name,u"バナナ")self.assertEqual(customer.address,"10 Red Road, Akihabara, Tokyo")if__name__=="__main_":unittest.main()

If your application depends on data from a remote location, like aremote API, you’ll want to ensure your tests are repeatable. Having your tests fail because the API is offline or there is aconnectivity issue could slow down development. In these types of situations, it is best practice to store remote fixtures locally so they can be recalled and sent to the application.

Therequests library has a complimentary package calledresponses that gives you ways to create response fixtures and save them in your test folders. Find out moreon their GitHub Page.

Installing Tox

Tox is available on PyPI as a package to install viapip:

$pipinstalltox

Now that you have Tox installed, it needs to be configured.

Configuring Tox for Your Dependencies

Tox is configured via a configuration file in your project directory. The Tox configuration file contains the following:

• The command to run in order to execute tests
• Any additional packages required before executing
• The target Python versions to test against

Instead of having to learn the Tox configuration syntax, you can get a head start by running the quickstart application:

$tox-quickstart

The Tox configuration tool will ask you those questions and create a file similar to the following intox.ini:

[tox]envlist=py27, py36[testenv]deps=commands=python -m unittest discover

Before you can run Tox, it requires that you have asetup.py file in your application folder containing the steps to install your package. If you don’t have one, you can followthis guide on how to create asetup.py before you continue.

Alternatively, if your project is not for distribution on PyPI, you can skip this requirement by adding the following line in thetox.ini file under the[tox] heading:

[tox]envlist=py27, py36skipsdist=True

If you don’t create asetup.py, and your application has some dependencies from PyPI, you’ll need to specify those on a number of lines under the[testenv] section. For example, Django would require the following:

[testenv]deps=django

Once you have completed that stage, you’re ready to run the tests.

You can now execute Tox, and it will create two virtual environments: one for Python 2.7 and one for Python 3.6. The Tox directory is called.tox/. Within the.tox/ directory, Tox will executepython -m unittest discover against each virtual environment.

You can run this process by calling Tox at the command line:

$tox

Tox will output the results of your tests against each environment. The first time it runs, Tox takes a little bit of time to create the virtual environments, but once it has, the second execution will be a lot faster.

Executing Tox

The output of Tox is quite straightforward. It creates an environment for each version, installs your dependencies, and then runs the test commands.

There are some additional command line options that are great to remember.

Run only a single environment, such as Python 3.6:

$tox-epy36

Recreate the virtual environments, in case your dependencies have changed orsite-packages/ is corrupt:

$tox-r

Run Tox with less verbose output:

$tox-q

Running Tox with more verbose output:

$tox-v

More information on Tox can be found at theTox Documentation Website.

Introducing Linters Into Your Application

Tox and Travis CI have configuration for a test command. The test command you have been using throughout this tutorial ispython -m unittest discover.

You can provide one or many commands in all of these tools, and this option is there to enable you to add more tools that improve the quality of your application.

One such type of application is called a linter. A linter will look at your code and comment on it. It could give you tips about mistakes you’ve made, correct trailing spaces, and even predict bugs you may have introduced.

For more information on linters, read thePython Code Quality tutorial.

$pipinstallflake8

$flake8test.pytest.py:6:1: E302 expected 2 blank lines, found 1test.py:23:1: E305 expected 2 blank lines after class or function definition, found 1test.py:24:20: W292 no newline at end of file

[flake8]ignore=E305exclude=.git,__pycache__max-line-length=90

$flake8--ignoreE305--exclude.git,__pycache__--max-line-length=90

matrix:include:-python:"2.7"script:"flake8"

$pipinstallblack

$blacktest.py

Keeping Your Test Code Clean

When writing tests, you may find that you end up copying and pasting code a lot more than you would in regular applications. Tests can be very repetitive at times, but that is by no means a reason to leave your code sloppy and hard to maintain.

Over time, you will develop a lot oftechnical debt in your test code, and if you have significant changes to your application that require changes to your tests, it can be a more cumbersome task than necessary because of the way you structured them.

Try to follow theDRY principle when writing tests:Don’tRepeatYourself.

Test fixtures and functions are a great way to produce test code that is easier to maintain. Also, readability counts. Consider deploying a linting tool likeflake8 over your test code:

$flake8--max-line-length=120tests/

Testing for Performance Degradation Between Changes

There are many ways to benchmark code in Python. The standard library provides thetimeit module, which can time functions a number of times and give you the distribution. This example will executetest() 100 times andprint() the output:

deftest():# ... your codeif__name__=="__main_":importtimeitprint(timeit.timeit("test()",setup="from __main__ import test",number=100))

Another option, if you decided to usepytest as a test runner, is thepytest-benchmark plugin. This provides apytest fixture calledbenchmark. You can passbenchmark() any callable, and it will log the timing of the callable to the results ofpytest.

You can installpytest-benchmark from PyPI usingpip:

$pipinstallpytest-benchmark

Then, you can add a test that uses the fixture and passes the callable to be executed:

deftest_my_function(benchmark):result=benchmark(test)

Execution ofpytest will now give you benchmark results:

More information is available at theDocumentation Website.

Testing for Security Flaws in Your Application

Another test you will want to run on your application is checking for common security mistakes or vulnerabilities.

You can installbandit from PyPI usingpip:

$pipinstallbandit

You can then pass the name of your application module with the-r flag, and it will give you a summary:

$bandit-rmy_sum[main]  INFO    profile include tests: None[main]  INFO    profile exclude tests: None[main]  INFO    cli include tests: None[main]  INFO    cli exclude tests: None[main]  INFO    running on Python 3.5.2Run started:2018-10-08 00:35:02.669550Test results:        No issues identified.Code scanned:        Total lines of code: 5        Total lines skipped (#nosec): 0Run metrics:        Total issues (by severity):                Undefined: 0.0                Low: 0.0                Medium: 0.0                High: 0.0        Total issues (by confidence):                Undefined: 0.0                Low: 0.0                Medium: 0.0                High: 0.0Files skipped (0):

As withflake8, the rules thatbandit flags are configurable, and if there are any you wish to ignore, you can add the following section to yoursetup.cfg file with the options:

[bandit]exclude:/testtests:B101,B102,B301

More details are available at theGitHub Website.