The process of including a new module into the Python standard library ishindered by the API lock-in and promise of backward compatibility implied bya module being formally part of Python. This PEP proposes a transitionalstate for modules - inclusion in a special__preview__ package for theduration of a minor release (roughly 18 months) prior to full acceptance intothe standard library. On one hand, this state provides the module with thebenefits of being formally part of the Python distribution. On the other hand,the core development team explicitly states that no promises are made withregards to the module’s eventual full inclusion into the standard library,or to the stability of its API, which may change for the next release.
Based on his experience with a similar “labs” namespace in Google App Engine,Guido has rejected this PEP[3] in favour of the simpler alternative ofexplicitly marking provisional modules as such in their documentation.
If a module is otherwise considered suitable for standard library inclusion,but some concerns remain regarding maintainability or certain API details,then the module can be accepted on a provisional basis. While it is consideredan unlikely outcome, such modulesmay be removed from the standard librarywithout a deprecation period if the lingering concerns prove well-founded.
As part of the same announcement, Guido explicitly accepted MatthewBarnett’s ‘regex’ module[4] as a provisional addition to the standardlibrary for Python 3.3 (using the ‘regex’ name, rather than as a drop-inreplacement for the existing ‘re’ module).
Whenever the Python core development team decides that a new module should beincluded into the standard library, but isn’t entirely sure about whether themodule’s API is optimal, the module can be placed in a special package named__preview__ for a single minor release.
In the next minor release, the module may either be “graduated” into thestandard library (and occupy its natural place within its namespace, leaving the__preview__ package), or be rejected and removed entirely from the Pythonsource tree. If the module ends up graduating into the standard library afterspending a minor release in__preview__, its API may be changed accordingto accumulated feedback. The core development team explicitly makes noguarantees about API stability and backward compatibility of modules in__preview__.
Entry into the__preview__ package marks the start of a transition of themodule into the standard library. It means that the core development teamassumes responsibility of the module, similarly to any other module in thestandard library.
__preview__We expect most modules proposed for addition into the Python standard libraryto go through a minor release in__preview__. There may, however, be someexceptions, such as modules that use a pre-defined API (for examplelzma,which generally follows the API of the existingbz2 module), or moduleswith an API that has wide acceptance in the Python development community.
In any case, modules that are proposed to be added to the standard library,whether via__preview__ or directly, must fulfill the acceptance conditionsset byPEP 2.
It is important to stress that the aim of this proposal is not to make theprocess of adding new modules to the standard library more difficult. On thecontrary, it tries to provide a means to addmore useful libraries. Moduleswhich are obvious candidates for entry can be added as before. Modules whichdue to uncertainties about the API could be stalled for a long time now havea means to still be distributed with Python, via an incubation period in the__preview__ package.
In principle, most modules in the__preview__ package should eventuallygraduate to the stable standard library. Some reasons for not graduating are:
Essentially, the decision will be made by the core developers on a per-casebasis. The point to emphasize here is that a module’s appearance in the__preview__ package in some release does not guarantee it will continuebeing part of Python in the next release.
Suppose theexample module is a candidate for inclusion in the standardlibrary, but some Python developers aren’t convinced that it presents the bestAPI for the problem it intends to solve. The module can then be added to the__preview__ package in release3.X, importable via:
from__preview__importexample
Assuming the module is then promoted to the standard library proper inrelease3.X+1, it will be moved to a permanent location in the library:
importexample
And importing it from__preview__ will no longer work.
Currently, the core developers are really reluctant to add new interfaces tothe standard library. This is because as soon as they’re published in arelease, API design mistakes get locked in due to backward compatibilityconcerns.
By gating all major API additions through some kind of a preview mechanismfor a full release, we get one full release cycle of community feedbackbefore we lock in the APIs with our standard backward compatibility guarantee.
We can also start integrating preview modules with the rest of the standardlibrary early, so long as we make it clear to packagers that the previewmodules should not be considered optional. The only difference between previewAPIs and the rest of the standard library is that preview APIs are explicitlyexempted from the usual backward compatibility guarantees.
Essentially, the__preview__ package is intended to lower the risk oflocking in minor API design mistakes for extended periods of time. Currently,this concern can block new additions, even when the core development teamconsensus is that a particular addition is a good idea in principle.
For future end users, the broadest benefit lies in a better “out-of-the-box”experience - rather than being told “oh, the standard library tools for task Xare horrible, download this 3rd party library instead”, those superior toolsare more likely to be just be an import away.
For environments where developers are required to conduct due diligence ontheir upstream dependencies (severely harming the cost-effectiveness of, oreven ruling out entirely, much of the material on PyPI), the key benefit liesin ensuring that anything in the__preview__ package is clearly underpython-dev’s aegis from at least the following perspectives:
For Python 3.3, there are a number of clear current candidates:
regex (http://pypi.python.org/pypi/regex)daemon (PEP 3143)ipaddr (PEP 3144)Other possible future use cases include:
requests)html5lib)simplegeneric)PEP 407 proposes a change to the core Python release cycle to permit interimreleases every 6 months (perhaps limited to standard library updates). Ifsuch a change to the release cycle is made, the following policy for the__preview__ namespace is suggested:
__preview__ namespace would alwaysbe empty.__preview__ namespace only ininterim releases that immediately follow a long-term support release.__future__Python already has a “forward-looking” namespace in the form of the__future__ module, so it’s reasonable to ask why that can’t be re-used forthis new purpose.
There are two reasons why doing so not appropriate:
1. The__future__ module is actually linked to a separate compilerdirectives feature that can actually change the way the Python interpretercompiles a module. We don’t want that for the preview package - we just wantan ordinary Python package.
2. The__future__ module comes with an express promise that names will bemaintained in perpetuity, long after the associated features have become thecompiler’s default behaviour. Again, this is precisely the opposite of what isintended for the preview package - it is almost certain that all names added tothe preview will be removed at some point, most likely due to their being movedto a permanent home in the standard library, but also potentially due to theirbeing reverted to third party package status (if community feedback suggests theproposed addition is irredeemably broken).
One proposed alternative[1] was to add explicit versioning to the__preview__ package, i.e.__preview34__. We think that it’s better tosimply define that a module being in__preview__ in Python 3.X will eithergraduate to the normal standard library namespace in Python 3.X+1 or willdisappear from the Python source tree altogether. Versioning the_preview__package complicates the process and does not align well with the main intent ofthis proposal.
It was proposed[1] to use a package name likepreview orexp, insteadof__preview__. This was rejected in the discussion due to the specialmeaning a “dunder” package name (that is, a namewith leading andtrailing double-underscores) conveys in Python. Besides, a non-dunder namewould suggest normal standard library API stability guarantees, which is notthe intention of the__preview__ package.
A pickled class instance based on a module in__preview__ in release 3.Xwon’t be unpickle-able in release 3.X+1, where the module won’t be in__preview__. Special code may be added to make this work, but this goesagainst the intent of this proposal, since it implies backward compatibility.Therefore, this PEP does not propose to preserve pickle compatibility.
Dj Gilcrease initially proposed the idea of having a__preview__ packagein Python[2]. Although his original proposal uses the name__experimental__, we feel that__preview__ conveys the meaning of thispackage in a better way.
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0408.rst
Last modified:2025-02-01 08:59:27 GMT