In Python 2.7 and Python 3.2, the default warning filters were updated to hideDeprecationWarning by default, such that deprecation warnings in developmenttools that were themselves written in Python (e.g. linters, static analysers,test runners, code generators), as well as any other applications that merelyhappened to be written in Python, wouldn’t be visible to their users unlessthose users explicitly opted in to seeing them.
However, this change has had the unfortunate side effect of makingDeprecationWarning markedly less effective at its primary intended purpose:providing advance notice of breaking changes in APIs (whether in CPython, thestandard library, or in third party libraries) to users of those APIs.
To improve this situation, this PEP proposes a single adjustment to thedefault warnings filter: displaying deprecation warnings attributed to the mainmodule by default.
This change will mean that code entered at the interactive prompt and code insingle file scripts will revert to reporting these warnings by default, whilethey will continue to be silenced by default for packaged code distributed aspart of an importable module.
The PEP also proposes a number of small adjustments to the referenceinterpreter and standard library documentation to help make the warningssubsystem more approachable for new Python developers.
As part of the documentation updates, it will be made clearer that theunittest test runner displays all warnings by default when executingtest cases, and that other test runners are advised to follow that example.
The current set of default warnings filters consists of:
ignore::DeprecationWarningignore::PendingDeprecationWarningignore::ImportWarningignore::BytesWarningignore::ResourceWarning
The defaultunittest test runner then useswarnings.catch_warnings()warnings.simplefilter('default') to override the default filters whilerunning test cases.
The change proposed in this PEP is to update the default warning filter listto be:
default::DeprecationWarning:__main__ignore::DeprecationWarningignore::PendingDeprecationWarningignore::ImportWarningignore::BytesWarningignore::ResourceWarning
This means that in cases where the nominal location of the warning (asdetermined by thestacklevel parameter towarnings.warn) is in the__main__ module, the first occurrence of eachDeprecationWarning will onceagain be reported.
This change will lead toDeprecationWarning being displayed by default for:
While continuing to be hidden by default for:
zipapp archive’s__main__.pyfile__main__submoduleconsole_scripts orgui_scripts entry point definitionThis means that tool developers that create an installable or executableartifact (such as azipapp archive) for distribution to their usersshouldn’t see any change from the status quo, while users of more ad hocpersonal or locally distributed scripts are likely to start seeing relevantdeprecation warnings again (as they did in Python 2.6 and earlier).
FutureWarningThe standard library documentation will be updated to explicitly recommend theuse ofFutureWarning (rather thanDeprecationWarning) for backwardscompatibility warnings that are intended to be seen byusers of anapplication. (This will be in addition to the existing use ofFutureWarningto warn about constructs that will remain valid code in the future,but will have different semantics).
This will give the following three distinct categories of backwardscompatibility warning, with three different intended audiences:
PendingDeprecationWarning: hidden by default for all code.The intended audience is Python developers that take an active interest inensuring the future compatibility of their software (e.g. professionalPython application developers with specific support obligations).DeprecationWarning: reported by default for code that runs directly inthe__main__ module (as such code is considered relatively unlikely tohave a dedicated test suite), but hidden by default for code in other modules.The intended audience is Python developers that are at risk of upgrades totheir dependencies (including upgrades to Python itself) breaking theirsoftware (e.g. developers using Python to script environments where someoneelse is in control of the timing of dependency upgrades).FutureWarning: reported by default for all code.The intended audience is users of applications written in Python, rather thanother Python developers (e.g. warning about use of a deprecated setting in aconfiguration file format).For library and framework authors that want to ensure their API compatibilitywarnings are more reliably seen by their users, the recommendation is to use acustom warning class that derives fromDeprecationWarning in Python 3.7+,and fromFutureWarning in earlier versions.
Developers of test runners are advised to implement logic equivalent to thefollowing when determining their default warnings filters:
ifnotsys.warnoptions:warnings.simplefilter("default")
This effectively enables all warnings by default, as if the-Wd commandline option had been passed.
Note that actually enablingBytesWarning in a test suite still requirespassing the-b option to the interpreter at the command line. For implicitbytes conversion and bytes comparison warnings, the warnings filter machineryis only used to determine whether they should be printed as warnings or raisedas exceptions - when the command line flag isn’t set, the interpreter doesn’teven emit the warning in the first place.
Developers of interactive shells are advised to add a filter that enablesDeprecationWarning in the namespace where user code is entered and executed.
If that namespace is__main__ (as it is for the default CPython REPL), thenno changes are needed beyond those in this PEP.
Interactive shell implementations which use a namespace other than__main__ will need to add their own filter. For example, IPython uses thefollowing command ([6]) to set up a suitable filter:
warnings.filterwarnings("default",category=DeprecationWarning,module=self.user_ns.get("__name__"))
The current reference documentation for the warnings system is relatively shorton specificexamples of possible settings for the-W command line optionor thePYTHONWARNINGS environment variably that achieve particular endresults.
The following improvements are proposed as part of the implementation of thisPEP:
PYTHONWARNINGS environment variable:PYTHONWARNINGS=error# Convert to exceptionsPYTHONWARNINGS=always# Warn every timePYTHONWARNINGS=default# Warn once per call locationPYTHONWARNINGS=module# Warn once per calling modulePYTHONWARNINGS=once# Warn once per Python processPYTHONWARNINGS=ignore# Never warn
-We,-Wa,-Wd,-Wm,-Wo,-Wi) for each of thewarning actions listed under the-W command line switch documentationwarnings moduledocumentation, using theaction::category andaction::category:modulenotationwarnings.simplefilterdocumentation as a recommended approach to turning off all warnings bydefault in a Python application while still allowing them to be turnedback on viaPYTHONWARNINGS or the-W command line switch:ifnotsys.warnoptions:warnings.simplefilter("ignore")
None of these arenew (they already work in all still supported Pythonversions), but they’re not especially obvious given the current structureof the related documentation.
A reference implementation is available in the PR[4] linked from therelated tracker issue for this PEP[5].
As a side-effect of implementing this PEP, the internal warnings filter listwill start allowing the use of plain strings as part of filter definitions (inaddition to the existing use of compiled regular expressions). When present,the plain strings will be compared for exact matches only. This approach allowsthe new default filter to be added during interpreter startup without requiringearly access to there module.
As discussed in[1] and mentioned in[2], Python 2.7 and Python 3.2 changedthe default handling ofDeprecationWarning such that:
unittest test runner was updated to re-enable it when running testsThe intent was to avoid cases of tooling output like the following:
$ devtool mycode//usr/lib/python3.6/site-packages/devtool/cli.py:1: DeprecationWarning: 'async' and 'await' will become reserved keywords in Python 3.7 async = True... actual tool output ...
Even whendevtool is a tool specifically for Python programmers, this is nota particularly useful warning, as it will be shown on every invocation, eventhough the main helpful step an end user can take is to report a bug to thedevelopers ofdevtool.
The warning is even less helpful for general purpose developer tools that areused across more languages than just Python, and almost entirely *un*helpfulfor applications that simply happen to be written in Python, and aren’tnecessarily intended for a developer audience at all.
However, this change proved to have unintended consequences for the followingaudiences:
unittest(the request for third party test runners to change their default warningsfilters was never made explicitly, so many of them still rely on theinterpreter defaults that are designed to suit deployed applications)unittest test runner to test their Python codein a subprocess (since evenunittest only adjusts the warnings settingsin the current process)In these cases,DeprecationWarning ended up become almost entirelyequivalent toPendingDeprecationWarning: it was simply never seen at all.
This PEP exists specifically to explain both the proposed addition to thedefault warnings filter for 3.7,and to more clearly articulate the rationalefor the original change to the handling ofDeprecationWarning back in Python 2.7and 3.2.
This PEP does not solve all known problems with the current approach to handlingdeprecation warnings. Most notably:
unittest test runner does not currently report deprecationwarnings emitted at module import time, as the warnings filter override is onlyput in place during test execution, not during test discovery and loading.unittest test runner does not currently report deprecationwarnings in subprocesses, as the warnings filter override is applied directlyto the loadedwarnings module, not to thePYTHONWARNINGS environmentvariable.warn module[3] does provide this, but enabling itinvolves monkeypatching the standard library’swarnings module).__main__ isn’t likely to help find API compatibilityproblems. Near term, the best currently available answer is to run affectedapplications withPYTHONWARNINGS=default::DeprecationWarning orpython-Wdefault::DeprecationWarning and pay attention to theirstderr output. Longer term, this is really a question for researchersworking on static analysis of Python code: how to reliably find usage ofdeprecated APIs, and how to infer that an API or parameter is deprecatedbased onwarnings.warn calls, without actually running either the codeproviding the API or the code accessing it.While these are real problems with the status quo, they’re excluded fromconsideration in this PEP because they’re going to require more complexsolutions than a single additional entry in the default warnings filter,and resolving them at least potentially won’t require going through the PEPprocess.
For anyone interested in pursuing them further, the first two would beunittest module enhancement requests, the third would be awarningsmodule enhancement request, while the last would only require a PEP ifinferring API deprecations from their contents was deemed to be an intractablecode analysis problem, and an explicit function and parameter marker syntax inannotations was proposed instead.
The CPython reference implementation will also include the following relatedchanges in 3.7:
-Xdev command line option that combines several developer centricsettings (including-Wd) into one command line flag:https://github.com/python/cpython/issues/76224Independently of the proposed changes to the default filters in this PEP,issue 32229[7] is a proposal to add awarnings.hide_warnings API tomake it simpler for application developers to hide warnings during normaloperation, while easily making them visible when testing.
DeprecationWarning auto-configuration(https://github.com/ipython/ipython/blob/6.2.x/IPython/core/interactiveshell.py#L619)warnings.hide_warnings API proposal(https://github.com/python/cpython/issues/76410)This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0565.rst
Last modified:2025-02-01 08:55:40 GMT