The steering council decided that the__past__ aspect of this proposalwas too complicated for the potential benefit. The other aspect of strongerrequirements for backwards-compatibility should be addressed byPEP 387.
This PEP is complementary to PEPs 5, 236, and 387, and shares similargoals.
This PEP explains the need for an additional compatibility mechanismin support ofPEP 5, “Guidelines for Language Evolution”.PEP 236,“Back to the__future__”, introduced a mechanism for forwardcompatibility in support ofPEP 5 but noted that a new mechanism forbackward compatibility was outside the scope of that PEP. A relatedPEP (in progress) introduces such a mechanism for backwardcompatibility.
PEP 5, “Guidelines for Language Evolution”, notes that “This PEP [PEP 5]does not replace or preclude other compatibility strategies such asdynamic loading of backwards-compatible parsers.”
FromPEP 236: “From time to time, Python makes an incompatible changeto the advertised semantics of core language constructs, or changestheir accidental (implementation-dependent) behavior in some way.While this is never done capriciously, and is always done with the aimof improving the language over the long term, over the short term it’scontentious and disrupting.PEP 5, Guidelines for Language Evolution,suggests ways to ease the pain, and this PEP [PEP 236] introduces somemachinery in support of that.”
Also fromPEP 236: “The purpose of future_statement is to make lifeeasier for people who keep current with the latest release in a timelyfashion. We don’t hate you if you don’t, but your problems are muchharder to solve, and somebody with those problems will need to write aPEP addressing them. future_statement is aimed at a differentaudience.”
When an incompatible change to core language syntax or semantics isbeing made, Python currently provides the future_statement mechanismfor providing forward compatibility until the release that enforcesthe new syntax or semantics, but provides no corresponding standardmechanism for providing backward compatibility after this release.
A consequence of this asymmetry is that, with respect to a breakingchange, the older (pre-breaking) version of the Python interpreter ismore capable than the newer (breaking) version; the older interpretercan use both code designed prior to the change and newer code, whereasthe newer interpreter is only capable of using code that has beenupgraded to support the changed feature.
As an example, consider the changes to the division operatorintroduced inPEP 238 in 2001, soon afterPEP 236 introduced thefuture_statement mechanism.PEP 238 outlines a suite of usefulforward-compatibility mechanisms for “true division” in the Python 2.xseries but omits to include any backward-compatibility mechanisms forafter “true division” was first enforced in Python 3.0. Python versionssince 3.0 do not provide a backward compatibility mechanism such asfrom__past__importdivision for code that expects the old“classic division” semantics, whereas Python versions prior to 3.0 dosupport both “classic division” code and also forward compatibilitywith code expecting “true division”. A further consequence of this isthat the “most compatible” interpreter with respect to the variety ofdivision-related Python code in the wild is Python 2.7, the versionbefore the breaking change was first enforced.
In contrast to this situation, newer versions of application softwaresuch as office suites tend to be more capable than earlier versionswith respect to their support for loading different versions of theirdata file formats. The pattern is usually that the newer applicationversions can transparently load data from either their newer or theirolder data formats, and that the newer version defaults to saving datain the newer format. Newer application software versions tend to bebackward-compatible by default. Forward compatibility is relativelyrare.
This policy puts the user of the newer application software at anadvantage over the user of the older software, which is usuallyincapable of loading data in the newer format. Sometimes it ispossible for a user of a newer software application version to exportdata in an older version by choosing this option explicitly. In thesecases, the forward-compatibility this enables may or may not beperfect; some features may be missing or the results may be otherwisesuboptimal. Upgrading is therefore easy, whereas downgrading isharder.
The emergent behaviour over many users from such a policy of newattractive features plus backward compatibility features is that anatural pressure builds up on each individual user to upgrade his orher own application version, and, the more other users an individualexchanges data files with, the more acute this pressure becomes.
This PEP makes two specific, related proposals. The first is that:
PEP 5 be augmented with a 6th step in the section “Steps forIntroducing Backwards-Incompatible Features” to indicate that, when anincompatible change to core language syntax or semantics is beingmade, Python-dev’s policy is to prefer and expect that, whereverpossible, a mechanism for backward compatibility be considered andprovided for future Python versions after the breaking change isadopted by default, in addition to any mechanisms proposed for forwardcompatibility such as new future_statements. Furthermore,PEP 387,“Backwards Compatibility Policy” (if accepted) would beaugmented with the same 6th step.
As an example of how this PEP is to be applied, if the latest revisionof the “true division” PEP (238) were proposed today, it would beconsidered incomplete.PEP 238 notes the “severe backwardscompatibility issues” raised by the proposal and describes severalmeasures for forward compatibility in the Abstract and API Changessections. It also mentions some backward compatibility ideas raised onc.l.py, including “Usefrom__past__importdivision to useclassic division semantics in a module”, but it does not put forwardany backward compatibility plan as part of the proposal.
If this PEP is accepted, it would be expected that a proposal such asPEP 238, because of its large-scale compatibility implications, wouldalso be accompanied by a backward compatibility plan that enablesusers of future Python versions after the breaking change has comeinto effect to re-enable the classic division behaviour easily intheir code.
The second proposal is that:
Python provide a standard backward compatibility mechanism inparallel to the__future__module mechanism for forwardcompatibility.
For reference, this document will refer to this as a “__past__”mechanism hereon, although it need not have all the characteristicsof the__future__ module andfuture_statement mechanism.
The specific form and implementation of the__past__ mechanism isthe subject of a separate PEP (in progress). However, this PEPrecommends that this__past__ mechanism be designed to meetsimilar criteria to those outlined inPEP 296 for__future__.Specifically:
a. It should enable individual modules to specify obsolete behavioursto re-enable from older Python versions on a module-by-module basis.
b. It should be flexible enough for both Python 3.6+ and pointreleases of earlier versions to reintroduce backward compatibilitywith older Python syntax or semantics for user modules that invoke the__past__ mechanism.
c. It should be possible to run older code augmented to invoke__past__ behaviours on older Python versions such as 2.x that haveno knowledge of the specific__past__ features invoked, or eventhat the__past__ mechanism for backward-compatibility exists.
Some implementations of__past__ mechanisms that would violatethese criteria are:
a. Import hooks. These would normally fail to work on amodule-by-module basis; instead they apply recursively to all newmodules imported from within a module.
b. A new piece of syntax or new semantics for Python 3.6 that isincompatible with prior versions.
c. A function added in Python 3.6 to a module in the Python standardlibrary that exists under the same name in prior Python versions.
The benefit to Python-dev of adopting this proposal is that futurebackward-incompatible changes can be less disruptive if these changeseach have a corresponding__past__ feature that has beenimplemented and can be invoked easily by users of future Pythonversions. This can help the language to evolve more quickly and moreeffectively to correct for design mistakes.
The benefit to conservative users is obvious: they can add support forthe latest shiny compatibility-breaking Python version to their codemerely by adding a__past__ incantation (perhaps a single line) toeach module, and that this can be automated. They can then upgradetheir interpreter to the latest version and gain access to the latestshiny Python features.
The benefit to the community is that, if ten thousand users rely onpackage XYZ, and package XYZ can trivially add support for the latestPython version, those ten thousand users can also upgrade to thelatest Python version quickly, without being held back waiting forpackage XYZ to do this.
Q1: Does this PEP require that Python keep two possible sets of semanticsfor each backward-incompatible feature forever?
A1: Definitely not. Legacy features can still be phased out whenappropriate – that is, when the majority of the user-base hasmigrated to the newer Python version. This PEP merely proposes toshift the emphasis of the development effort directed at compatibilityfrom 100% forwards to at least 50% backwards. Backwards compatibilityis the more powerful of the two concepts for allowing a user-base toadopt the latest Python interpreter version.
Notice that it has been a long time since most users have cared aboutbackwards compatibility for non-nested scopes, because most users havemoved comfortably past Python 2.1.
Q2: But Python-dev is already overwhelmed and doesn’t have thebandwidth to implement / maintain the additional complexity!
A2: Python-dev can ask the community of developers to step up andmaintain backward compatibility in Python for legacy language featuresthey care about. When the community stops caring about a particularobsolete behaviour, Python-dev can stop caring too.
The__past__ mechanism could possibly be designed to be extensibleby the community, e.g. as a standard but “blessed” PyPI package, toreduce the load on the core developers.
Q3: Won’t backward compatibility features lead to lots of cruft andbloat and baggage in Python?
A3: Not necessarily. First, proposals for new compatibility-breakingfeatures in Python could be evaluated partly on the simplicity andmaintainability of the implementation of their associated__past__feature up-front.
Second, some old features are simple to provide backward compatibilityfor. Consider the “classic division” behaviour before Python 3.0. Thepython-future project contains a compatible implementation ofclassic division in the functionfuture.utils.old_div:
defold_div(a,b):""" Equivalent to ``a / b`` on Python 2 without ``from __future__ import division``. """ifisinstance(a,numbers.Integral)andisinstance(b,numbers.Integral):returna//belse:returna/b
Bundling such a function with Python 3.x versions, together witha simple mechanism to invoke it for every appearance ofa/b after an appropriate__past__ invocation, need not beonerous.
Q4: What about performance? Won’t the performance of newer Pythonversions suffer under the weight of legacy features?
A4: This can be evaluated on a case-by-case basis. The major potentialconcern is that the performance with the new default behaviour doesnot suffer unduly because of the presence of the legacy option. Theperformance under the influence of the__past__ invocation is ofsecondary importance.
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0497.rst
Last modified:2025-02-01 08:59:27 GMT