This task has been started in PR#5405, but ended without success, so I have decided to split it to multiple stages:

1. Makepytest compatible with the current test suite
   • Add support of unitest-like fixtures (setUp and other methods) on classes not derived fromunitest.UnitTest (fixed with8b64294)
   • Add support ofimage_comparison andknownfailif decorator (fixed withbbaf7dc)
   • Fixraise KnownFailureTest problems (PRAdd Py.test testing framework support #6730)
     • ImageComparisonTest.test
     • test_axes.test_formatter_large_small
     • test_backend_pgf.compare_figure
     • test_animation.check_save_animation
     • test_coding_standards.test_pep8_conformance_examples
   • test_basic leaks tests fromnumpy into global scope withfrom pylab import * (fixed with30caf00)
   • Solve strange failures on several tests
     • test_subplots.test_subplots_offsettext (PRMissingcleanup decorator intest_subplots.test_exceptions #6741)
     • test_axes.test_eventplot_problem_kwargs (PRFixed warnings catching and counting withwarnings.catch_warnings #6761)
     • test_pickle.test_complete (PRFixed brokentest_pickle.test_complete test #6840)
   • Ensure that pytest collects all the tests (6150/6150) (actually nose shows 6151 because it countslib/matplotlib/tests/test_delaunay.py::make_all_2d_testfuncs as a test)
2. Migration to pytest
   • Add new build underallow_failures with pytest
     • Appveyor
     • Travis
     • Usepytest-cov for coverage reports
   • Replace nose in all builds with pytest
3. Post-migration
   • RenamesetUp ->setup_method and other methods topytest analogs
     • Do not inherit tests fromunitest.UnitTest class
     • RemovesetUp/tearDown workaround
   • Rewriteimage_comparison decorator (currently it slowdowns the collection phase due to baseline images copying) or usepytest-mpl plugin (needs some improvements, see paragraph below)
   • Rewrite all assertations to pureassert
   • Get away fromcleanup decorator
   • Replaceknownfailif withmark.xfail
   • Replaceraise nose.SkipTest(...) withxfail(...) call ormark.skipif decorator
4. Enhancements
   • Replacesetup_method and etc methods with fixtures
   • Rewritetest_delaunay.make_all_2d_testfuncs
   • Usepytest-pep8 plugin instead oftest_coding_standards
   • Usepytest-mock plugin instead ofmock andunittest.mock

Tests entry point (raised in#5405 (comment))

I think that running tests with simplepy.test command is better thanpython tests.py, but there are unused tests which will run that way. Solutions:

• A. Get them away from pytest's sight with:
  1. Remove unused tests (test_sankey,test_skew,test_ttconv, andtest_usetex fromlib/matplotlib/tests)
  2. Renametest_only.py (tosetup_test_only.py probably)
• B. Make filter for collection phase.
  1. Implement whitelist
  2. Implement blacklist

I have tried both ways and they work, so what to choose is up to you.

• py.test --collect-filter=none
• py.test --collect-filter=blacklist
• py.test --collect-filter=whitelist

Currently--collect-filter=whitelist is default (to follow same politics as nose does currently).

Image comparison decorator

Why do we have to rewriteimage_comparison decorator? Becausepytest-mpl plugin in the current state ("as is") is not usable for matplotlib:

• + supportssavefig_kwargs
• + supportstolerance
• ‒ need to supplybaseline_dir in every decorator
• ‒ does not support image format conversion
• ‒ need something to do withfreetype_version,remove_text, andstyle