Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-108885: Use subtests for doctest examples run by unittest#134890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
NextNext commit
gh-108885: Use subtests for doctest examples run by unittest
Run each example as a subtest in unit tests synthesized bydoctest.DocFileSuite() and doctest.DocTestSuite().
  • Loading branch information
@serhiy-storchaka
serhiy-storchaka committedMay 29, 2025
commit55da0a16989b5533ae0e6d2191170001fb3347b3
61 changes: 40 additions & 21 deletionsDoc/library/doctest.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1043,12 +1043,15 @@ from text files and modules with doctests:
Convert doctest tests from one or more text files to a
:class:`unittest.TestSuite`.

The returned :class:`unittest.TestSuite` is to be run by the unittest framework
and runs the interactive examples in each file. If an example in any file
fails, then the synthesized unit test fails, and a :exc:`failureException`
exception is raised showing the name of the file containing the test and a
(sometimes approximate) line number. If all the examples in a file are
skipped, then the synthesized unit test is also marked as skipped.
The returned :class:`unittest.TestSuite` is to be run by the unittest
framework and runs the interactive examples in each file.
Each file is run as a separate unit test, and each example in a file
is run as a :ref:`subtest <subtests>`.
If any example in a file fails, then the synthesized unit test fails.
The traceback for failure or error contains the name of the file
containing the test and a (sometimes approximate) line number.
If all the examples in a file are skipped, then the synthesized unit
test is also marked as skipped.

Pass one or more paths (as strings) to text files to be examined.

Expand DownExpand Up@@ -1078,12 +1081,12 @@ from text files and modules with doctests:

Optional argument *setUp* specifies a set-up function for the test suite.
This is called before running the tests in each file. The *setUp* function
will be passed a :class:`DocTest` object. The setUp function can access the
will be passed a :class:`DocTest` object. The*setUp* function can access the
test globals as the *globs* attribute of the test passed.

Optional argument *tearDown* specifies a tear-down function for the test
suite. This is called after running the tests in each file. The *tearDown*
function will be passed a :class:`DocTest` object. ThesetUp function can
function will be passed a :class:`DocTest` object. The*tearDown* function can
access the test globals as the *globs* attribute of the test passed.

Optional argument *globs* is a dictionary containing the initial global
Expand All@@ -1105,16 +1108,22 @@ from text files and modules with doctests:
The global ``__file__`` is added to the globals provided to doctests loaded
from a text file using :func:`DocFileSuite`.

.. versionchanged:: next
Run each example as a :ref:`subtest <subtests>`.


.. function:: DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None, setUp=None, tearDown=None, optionflags=0, checker=None)

Convert doctest tests for a module to a :class:`unittest.TestSuite`.

The returned :class:`unittest.TestSuite` is to be run by the unittest framework
and runs each doctest in the module. If any of the doctests fail, then the
synthesized unit test fails, and a :exc:`failureException` exception is raised
showing the name of the file containing the test and a (sometimes approximate)
line number. If all the examples in a docstring are skipped, then the
The returned :class:`unittest.TestSuite` is to be run by the unittest
framework and runs each doctest in the module.
Each docstring is run as a separate unit test, and each example in
a docstring is run as a :ref:`subtest <subtests>`.
If any of the doctests fail, then the synthesized unit test fails.
The traceback for failure or error contains the name of the file
containing the test and a (sometimes approximate) line number.
If all the examples in a docstring are skipped, then the
synthesized unit test is also marked as skipped.

Optional argument *module* provides the module to be tested. It can be a module
Expand All@@ -1132,19 +1141,16 @@ from text files and modules with doctests:
drop-in replacement) that is used to extract doctests from the module.

Optional arguments *setUp*, *tearDown*, and *optionflags* are the same as for
function :func:`DocFileSuite` above.
function :func:`DocFileSuite` above, but they are called for each docstring.

This function uses the same search technique as :func:`testmod`.

.. versionchanged:: 3.5
:func:`DocTestSuite` returns an empty :class:`unittest.TestSuite` if *module*
contains no docstrings instead of raising :exc:`ValueError`.

.. exception:: failureException

When doctests which have been converted to unit tests by :func:`DocFileSuite`
or :func:`DocTestSuite` fail, this exception is raised showing the name of
the file containing the test and a (sometimes approximate) line number.
.. versionchanged:: next
Run each example as a :ref:`subtest <subtests>`.

Under the covers, :func:`DocTestSuite` creates a :class:`unittest.TestSuite` out
of :class:`!doctest.DocTestCase` instances, and :class:`!DocTestCase` is a
Expand DownExpand Up@@ -1508,7 +1514,7 @@ DocTestRunner objects
with strings that should be displayed. It defaults to ``sys.stdout.write``. If
capturing the output is not sufficient, then the display output can be also
customized by subclassing DocTestRunner, and overriding the methods
:meth:`report_start`, :meth:`report_success`,
:meth:`report_skip`, :meth:`report_start`, :meth:`report_success`,
:meth:`report_unexpected_exception`, and :meth:`report_failure`.

The optional keyword argument *checker* specifies the :class:`OutputChecker`
Expand All@@ -1533,14 +1539,27 @@ DocTestRunner objects
:class:`DocTestRunner` defines the following methods:


.. method:: report_skip(out, test, example)

Report that the given example was skipped. This method is provided to
allow subclasses of :class:`DocTestRunner` to customize their output; it
should not be called directly.

*example* is the example about to be processed. *test* is the test
containing *example*. *out* is the output function that was passed to
:meth:`DocTestRunner.run`.

.. versionadded:: next


.. method:: report_start(out, test, example)

Report that the test runner is about to process the given example. This method
is provided to allow subclasses of :class:`DocTestRunner` to customize their
output; it should not be called directly.

*example* is the example about to be processed. *test* is the test
*containing example*. *out* is the output function that was passed to
containing*example*. *out* is the output function that was passed to
:meth:`DocTestRunner.run`.


Expand Down
95 changes: 75 additions & 20 deletionsLib/doctest.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -101,15 +101,14 @@ def _test():
import re
import sys
import traceback
import types
import unittest
from io import StringIO, IncrementalNewlineDecoder
from collections import namedtuple
import _colorize # Used in doctests
from _colorize import ANSIColors, can_colorize


__unittest = True

class TestResults(namedtuple('TestResults', 'failed attempted')):
def __new__(cls, failed, attempted, *, skipped=0):
results = super().__new__(cls, failed, attempted)
Expand DownExpand Up@@ -387,7 +386,7 @@ def __init__(self, out):
self.__out = out
self.__debugger_used = False
# do not play signal games in the pdb
pdb.Pdb.__init__(self,stdout=out, nosigint=True)
super().__init__(stdout=out, nosigint=True)
# still use input() to get user input
self.use_rawinput = 1

Expand DownExpand Up@@ -1280,6 +1279,11 @@ def __init__(self, checker=None, verbose=None, optionflags=0):
# Reporting methods
#/////////////////////////////////////////////////////////////////

def report_skip(self, out, test, example):
"""
Report that the given example was skipped.
"""

def report_start(self, out, test, example):
"""
Report that the test runner is about to process the given
Expand DownExpand Up@@ -1377,6 +1381,8 @@ def __run(self, test, compileflags, out):

# If 'SKIP' is set, then skip this example.
if self.optionflags & SKIP:
if not quiet:
self.report_skip(out, test, example)
skips += 1
continue

Expand DownExpand Up@@ -2274,12 +2280,63 @@ def set_unittest_reportflags(flags):
return old


class _DocTestCaseRunner(DocTestRunner):

def __init__(self, *args, test_case, test_result, **kwargs):
super().__init__(*args, **kwargs)
self._test_case = test_case
self._test_result = test_result
self._examplenum = 0

def _subTest(self):
subtest = unittest.case._SubTest(self._test_case, str(self._examplenum), {})
self._examplenum += 1
return subtest

def report_skip(self, out, test, example):
unittest.case._addSkip(self._test_result, self._subTest(), '')

def report_success(self, out, test, example, got):
self._test_result.addSubTest(self._test_case, self._subTest(), None)

def report_unexpected_exception(self, out, test, example, exc_info):
tb = self._add_traceback(exc_info[2], test, example)
exc_info = (*exc_info[:2], tb)
self._test_result.addSubTest(self._test_case, self._subTest(), exc_info)

def report_failure(self, out, test, example, got):
msg = ('Failed example:\n' + _indent(example.source) +
self._checker.output_difference(example, got, self.optionflags).rstrip('\n'))
exc = self._test_case.failureException(msg)
tb = self._add_traceback(None, test, example)
exc_info = (type(exc), exc, tb)
self._test_result.addSubTest(self._test_case, self._subTest(), exc_info)

def _add_traceback(self, traceback, test, example):
if test.lineno is None or example.lineno is None:
lineno = None
else:
lineno = test.lineno + example.lineno + 1
return types.SimpleNamespace(
tb_frame = types.SimpleNamespace(
f_globals=test.globs,
f_code=types.SimpleNamespace(
co_filename=test.filename,
co_name=test.name,
),
),
tb_next = traceback,
tb_lasti = -1,
tb_lineno = lineno,
)


class DocTestCase(unittest.TestCase):

def __init__(self, test, optionflags=0, setUp=None, tearDown=None,
checker=None):

unittest.TestCase.__init__(self)
super().__init__()
self._dt_optionflags = optionflags
self._dt_checker = checker
self._dt_test = test
Expand All@@ -2303,30 +2360,28 @@ def tearDown(self):
test.globs.clear()
test.globs.update(self._dt_globs)

def run(self, result=None):
self._test_result = result
return super().run(result)

def runTest(self):
test = self._dt_test
old = sys.stdout
new = StringIO()
optionflags = self._dt_optionflags
result = self._test_result

if not (optionflags & REPORTING_FLAGS):
# The option flags don't include any reporting flags,
# so add the default reporting flags
optionflags |= _unittest_reportflags
if getattr(result, 'failfast', False):
optionflags |= FAIL_FAST

runner = DocTestRunner(optionflags=optionflags,
checker=self._dt_checker, verbose=False)

try:
runner.DIVIDER = "-"*70
results = runner.run(test, out=new.write, clear_globs=False)
if results.skipped == results.attempted:
raise unittest.SkipTest("all examples were skipped")
finally:
sys.stdout = old

if results.failed:
raise self.failureException(self.format_failure(new.getvalue().rstrip('\n')))
runner = _DocTestCaseRunner(optionflags=optionflags,
checker=self._dt_checker, verbose=False,
test_case=self, test_result=result)
results = runner.run(test, clear_globs=False)
if results.skipped == results.attempted:
raise unittest.SkipTest("all examples were skipped")

def format_failure(self, err):
test = self._dt_test
Expand DownExpand Up@@ -2441,7 +2496,7 @@ def shortDescription(self):
class SkipDocTestCase(DocTestCase):
def __init__(self, module):
self.module = module
DocTestCase.__init__(self,None)
super().__init__(None)

def setUp(self):
self.skipTest("DocTestSuite will not work with -O2 and above")
Expand Down
Loading
Loading

[8]ページ先頭

©2009-2025 Movatter.jp