Repository files navigation

Apytest plugin for asserting data againstvoluptuous schema.

Common use case is to validate HTTP API responses (in your functional tests):

importrequestsfrompytest_voluptuousimportS,Partial,Exactfromvoluptuous.validatorsimportAll,Lengthdeftest_pypi():resp=requests.get('https://pypi.org/pypi/pytest/json')assertS({'info':Partial({'package_url':'https://pypi.org/project/pytest/','platform':'INVALID VALUE','description':Length(max=10),'downloads':list,'classifiers':dict,      }),'releases':dict,'urls':int   })==resp.json()

If validation fails, comparison returnsFalse and assert fails, printing error details like:

E       AssertionError: assert failed due to validation error(s):E         - info.platform: not a valid value for dictionary value (actual: 'unix')E         - info.description: length of value must be at most 10 for dictionary value (actual: ".. image:: https://...")E         - info.downloads: expected list for dictionary value (actual: {'last_month': -1, 'last_week': -1, 'last_day': -1})E         - info.classifiers: expected dict for dictionary value (actual: [u'Development Status :: 6 - Mature', ...])E         - last_serial: extra keys not allowed (actual: 4422291)E         - urls: expected int (actual: [{u'has_sig': False, u'upload_time': u'2018-10-27T16:31:24', ...}])

Works on python 2.7 and 3.4+:

pip install pytest-voluptuous

SeeCHANGELOG.

• Providesutility schemas (S,Exact andPartial) to cut down boilerplate.
• Implement apytest hook to provide error details onassert failure.
• Print descriptive validationfailure messages.
• Equal andUnordered validators (contributed to voluptuous project, available in 0.10+).

Because writing:

>>> r= {'info': {'package_url':'https://pypi.org/pypi/pytest'}}>>>assert'info'in r>>>assert'package_url'in r['info']>>>assert r['info']['package_url']=='https://pypi.org/pypi/pytest'

...is justway too annoying.

Why notJSON schema? It'stoo verbose, too inconvenient. JSON schema will nevermatch the convenience of a validation library that can utilize the goodies of the platform.

Why voluptuous and not some other library? No special reason - but it's pretty easy to use and understand. Also, thesyntax is quite compact.

Start by specifying a schema:

>>>from pytest_voluptuousimport S, Partial, Exact>>>from voluptuous.validatorsimport All, Length>>> schema= S({...'info': Partial({...'package_url':'https://pypi.org/project/pytest/',...'platform':'unix',...'description': Length(min=100),...'downloads':dict,...'classifiers':list,...     }),...'urls':list... })

Then load up the data to validate:

>>>import requests>>> data= requests.get('https://pypi.org/pypi/pytest/json').json()

Now if you assert this, the data will be validated against the schema, but instead of raising an error, the comparisonwill just evaluate toFalse which fails the assert:

>>>assert data== schemaTraceback (most recent call last):    ...AssertionError

Now gettingAssertionError in case the data doesn't match the schema is not very nice but don't worry - there'sno pytest magic in play here but once you run through pytest you'll rather get:

E       AssertionError: assert failed due to validation error(s):E         - info.platform: not a valid value for dictionary value (actual: 'unix')E         - info.description: length of value must be at most 10 for dictionary value (actual: ".. image:: https://docs.pytest.org/en/latest/_static/pytest1.png\n   :target: https://docs.pytest.org/en/latest/\n   :align: center\n   :alt: pytest\n\n\n------\n\n.. image:: https://img.shields.io/pypi/v/pytest.svg\n    :target: https://pypi.org/project/pytest/\n\n.. image:: https://img.shields.io/conda/vn/conda-forge/pytest.svg\n    :target: https://anaconda.org/conda-forge/pytest\n\n.. image:: https://img.shields.io/pypi/pyversions/pytest.svg\n    :target: https://pypi.org/project/pytest/\n\n.. image:: https://codecov.io/gh/pytest-dev/pytest/branch/master/graph/badge.svg\n    :target: https://codecov.io/gh/pytest-dev/pytest\n    :alt: Code coverage Status\n\n.. image:: https://travis-ci.org/pytest-dev/pytest.svg?branch=master\n    :target: https://travis-ci.org/pytest-dev/pytest\n\n.. image:: https://ci.appveyor.com/api/projects/status/mrgbjaua7t33pg6b?svg=true\n    :target: https://ci.appveyor.com/project/pytestbot/pytest\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n    :target: https://github.com/ambv/black\n\n.. image:: https://www.codetriage.com/pytest-dev/pytest/badges/users.svg\n    :target: https://www.codetriage.com/pytest-dev/pytest\n\nThe ``pytest`` framework makes it easy to write small tests, yet\nscales to support complex functional testing for applications and libraries.\n\nAn example of a simple test:\n\n.. code-block:: python\n\n    # content of test_sample.py\n    def inc(x):\n        return x + 1\n\n\n    def test_answer():\n        assert inc(3) == 5\n\n\nTo execute it::\n\n    $ pytest\n    ============================= test session starts =============================\n    collected 1 items\n\n    test_sample.py F\n\n    ================================== FAILURES ===================================\n    _________________________________ test_answer _________________________________\n\n        def test_answer():\n    >       assert inc(3) == 5\n    E       assert 4 == 5\n    E        +  where 4 = inc(3)\n\n    test_sample.py:5: AssertionError\n    ========================== 1 failed in 0.04 seconds ===========================\n\n\nDue to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started <https://docs.pytest.org/en/latest/getting-started.html#our-first-test-run>`_ for more examples.\n\n\nFeatures\n--------\n\n- Detailed info on failing `assert statements <https://docs.pytest.org/en/latest/assert.html>`_ (no need to remember ``self.assert*`` names);\n\n- `Auto-discovery\n  <https://docs.pytest.org/en/latest/goodpractices.html#python-test-discovery>`_\n  of test modules and functions;\n\n- `Modular fixtures <https://docs.pytest.org/en/latest/fixture.html>`_ for\n  managing small or parametrized long-lived test resources;\n\n- Can run `unittest <https://docs.pytest.org/en/latest/unittest.html>`_ (or trial),\n  `nose <https://docs.pytest.org/en/latest/nose.html>`_ test suites out of the box;\n\n- Python 2.7, Python 3.4+, PyPy 2.3, Jython 2.5 (untested);\n\n- Rich plugin architecture, with over 315+ `external plugins <http://plugincompat.herokuapp.com>`_ and thriving community;\n\n\nDocumentation\n-------------\n\nFor full documentation, including installation, tutorials and PDF documents, please see https://docs.pytest.org/en/latest/.\n\n\nBugs/Requests\n-------------\n\nPlease use the `GitHub issue tracker <https://github.com/pytest-dev/pytest/issues>`_ to submit bugs or request features.\n\n\nChangelog\n---------\n\nConsult the `Changelog <https://docs.pytest.org/en/latest/changelog.html>`__ page for fixes and enhancements of each version.\n\n\nLicense\n-------\n\nCopyright Holger Krekel and others, 2004-2018.\n\nDistributed under the terms of the `MIT`_ license, pytest is free and open source software.\n\n.. _`MIT`: https://github.com/pytest-dev/pytest/blob/master/LICENSE\n\n\n")E         - info.downloads: expected list for dictionary value (actual: {'last_month': -1, 'last_week': -1, 'last_day': -1})E         - info.classifiers: expected dict for dictionary value (actual: [u'Development Status :: 6 - Mature', u'Intended Audience :: Developers', u'License :: OSI Approved :: MIT License', u'Operating System :: MacOS :: MacOS X', u'Operating System :: Microsoft :: Windows', u'Operating System :: POSIX', u'Programming Language :: Python :: 2', u'Programming Language :: Python :: 2.7', u'Programming Language :: Python :: 3', u'Programming Language :: Python :: 3.4', u'Programming Language :: Python :: 3.5', u'Programming Language :: Python :: 3.6', u'Programming Language :: Python :: 3.7', u'Topic :: Software Development :: Libraries', u'Topic :: Software Development :: Testing', u'Topic :: Utilities'])E         - last_serial: extra keys not allowed (actual: 4422291)E         - urls: expected int (actual: [{u'has_sig': False, u'upload_time': u'2018-10-27T16:31:24', u'comment_text': u'', u'python_version': u'py2.py3', u'url': u'https://files.pythonhosted.org/packages/02/75/d041ed00994fbac4c5183e6f4bf6c906506bef8da7a57ef3fc825f171020/pytest-3.9.3-py2.py3-none-any.whl', u'md5_digest': u'150289b7b6658b62b3eddb96c4474e9d', u'downloads': -1, u'requires_python': u'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*', u'filename': u'pytest-3.9.3-py2.py3-none-any.whl', u'packagetype': u'bdist_wheel', u'digests': {u'sha256': u'bf47e8ed20d03764f963f0070ff1c8fda6e2671fc5dd562a4d3b7148ad60f5ca', u'md5': u'150289b7b6658b62b3eddb96c4474e9d'}, u'size': 214163}, {u'has_sig': False, u'upload_time': u'2018-10-27T16:31:26', u'comment_text': u'', u'python_version': u'source', u'url': u'https://files.pythonhosted.org/packages/28/09/f73d49a5b0b714e2d4712f044686cb8fa954aac15f4b7ea557049210179f/pytest-3.9.3.tar.gz', u'md5_digest': u'32ca214ba15bbd8680d9d807a371c385', u'downloads': -1, u'requires_python': u'>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*', u'filename': u'pytest-3.9.3.tar.gz', u'packagetype': u'sdist', u'digests': {u'sha256': u'a9e5e8d7ab9d5b0747f37740276eb362e6a76275d76cebbb52c6049d93b475db', u'md5': u'32ca214ba15bbd8680d9d807a371c385'}, u'size': 882503}])

Use== operator to do exact validation:

>>> data= {'foo':1,'bar':True}>>> S({'foo':1,'bar':True})== dataTrue

We omitassert in these examples (for easier doctesting).

Use<= to dopartial validation (to allow extra keys, that is):

>>> S({'foo':1})== data# not validFalse>>> S({'foo':1})<= data# validTrue

The operator you choose gets inherited, so with test data of:

>>> data= {...'outer1': {...'inner1':1,...'inner2':True...     },...'outer2':'foo'... }

With== you must provide exact valuealso in nested context:

>>> S({...'outer1': {...'inner1':1,# this would be valid but......# missing 'inner2'...     },...'outer2':'foo'... })== dataFalse>>> S({...'outer1': {...'inner1':int,# exact/partial matching...'inner2':bool# is for keys only...     },...'outer2':'foo'... })== dataTrue

<= implies partial matching:

>>> S({...'outer1': {...'inner1':int,...# 'inner2' missing but that's ok...     },...# 'outer2' is missing too... })<= dataTrue

When you need to mix and match operators, you can loosen matching withPartial:

>>> S({...'outer1': Partial({...'inner1':int...# 'inner2' ok to omit as scope is partial...     }),...'outer2':'foo'# can't be missing as outer scope is exact... })== dataTrue

And stricten withExact:

>>> S({...'outer1': Exact({...'inner1':int,...'inner2':bool...     }),...# 'outer2' can be missing as outer scope is partial... })<= dataTrue

Remember, matching mode is inherited, so you may end up doing stuff like this:

>>> data['outer1']['inner1']= {'prop':1}>>> S({...'outer1': Partial({...'inner1': Exact({...'prop':1...         })...     }),...'outer2':'foo'... })== dataTrue

There is no>=. If you want to declareschema keys that may be missing, useOptional:

>>>from voluptuous.schema_builderimport Optional>>> S({Optional('foo'):str})== {'extra':1}False>>> S({'foo':str})== {}False>>> S({'foo':str})<= {}False>>> S({Optional('foo'):str})== {}True>>> S({Optional('foo'):str})<= {'extra':1}True

Or, if you want to make all keys optional, overriderequired:

>>>from voluptuous.schema_builderimport Required>>> S({'foo':str},required=False)== {}True

In these cases, if you want torequire a key:

>>> S({'foo':str, Required('bar'):1},required=False)== {}False>>> S({'foo':str, Required('bar'):1},required=False)== {'bar':1}True

That's it. For available validators, look intovoluptuous docs.

Voluptuous 0.9.3 and earlier:

In voluptuous pre-0.10.2[] matchesany list, not an empty list. To declare an empty list, useEqual([]).

Similarly, in voluptuous pre-0.10.2,{} doesn'talways match an empty dict. If you're inside aSchema({...}, extra=PREVENT_EXTRA) (orExact),{} does indeed match exactly{}. However, insideSchema({...}, extra=ALLOW_EXTRA) (or ``Partial), it matchesany dict (because any extra keys are allowed).To declare an empty dict, useEqual({}).

Voluptuous 0.10.0+:

In voluptuous 0.10.0+{} and[] evaluate asempty dict andempty list, so you don't need above workarounds.

Always usedict andlist to validate dict or list of any size. It works despite voluptuous version.

Any version:

[str, int] matches any list that contains both strings and ints (in any order and 1-n times). To validatea list of fixed length with those types in it, useExactSequence([str, int]) andUnordered([str, int])when the order has no meaning. You can also use values inside these as inExactSequence([2, 3]).

Apache 2.0 licensed. SeeLICENSE formore details.