I've used thesubTest method ofunittest.TestCase intest_env_vars_for_windows_tests. Thispytest plugin givespytest full support for that mechanism. It letspytest continue with the other subtests even after one fails, as theunittest test runner would, and show separate passes and failures for the individual subtests. Separate results are shown only when at least one fails, so when everything passes the report remains uncluttered. (It also provides asubtests pytest fixture, though since this is not an autouse fixture it is not usable from within a method of a class that inherits fromunittest.TestCase.)

I think this is useful to have going forward, since we have many test cases that are large with many separate assertions of separate facts about the system under test, and as they are updated, some of them could be improved by having their separate claims divided into subtests so they can be individually described and so failures don't unnecessarily block later subtests.

However, if you'd rather this plugin be used, it can be removed.test_env_vars_for_windows_tests could retain subtests--they will still each run if none fails, just like multiple assertions in a test case without subtests. Or I could replace the subtests with more@ddt parameterization, or manually, etc.

I all sounds reasonable and in particular, since you are the one doing the work it seems fair that you can use the tooling you see as best fit. I also have no experience here and no preferences, and believe that anything that improves the tests in any way is very welcome. Thank you!

Sounds good. For the conceptually unrelated reason that it would facilitate more fine-grainedxfail marking--so we don't habitually carry XPASSing cases that aren't actually expected to have failed--I think the test class involved here,test.test_util.TestUtils, should be split, so that thecygpath-related tests can be purepytest tests. The reason tosplit it is:

• @pytest.mark.parametrize supports fine-grained marking of individual test cases, including asskip orxfail, but that form of parameterization cannot be applied to test methods inunittest.TestCase subclasses. So at least those tests would currently (i.e., until the underlying causes of some cases failing are addressed) benefit from being purepytest tests.
• Some of the test methods, not forcygpath, use therorepo fixture provided bytest.lib.helper.TestBase, which inherits fromunittest.TestCase. Although this could be converted to apytest fixture, I'd rather wait to do that until after more operating systems, at least Windows, are tested on CI, and also until I have more insight into whether it makes sense to do that at all, rather thanreplacingrorepo and other fixtures with new corresponding fixtures that use isolated repositories (#914,#1693 (review)). So at least those tests should currently remain in aunittest.TestCase subclass.

So long as it's acceptable to have multiple test classes in the same test module, this could be done at any time, and it may facilitate some other simplifications.I mention it here because I think it might lead to the elimination of subtests in this particular module, either by using@pytest.mark.parametrize for this too or for other reasons.

If that happens, I might remove thepytest-subtests test dependency, even though it might be re-added later, because the alternative ofpytest-check may be preferable in some of the large test methodsif they can also be converted to be purepytest tests (becausepytest-check supports a more compact syntax).