This PEP proposes a new way to govern the Python type system: a council that is responsiblefor maintaining and developing the Python type system. The council will maintain aspecification and conformance test suite and will initially be appointed by the Python Steering Council.
The Python type system was created byPEP 484, almost ten years ago. The typesystem is now widely used, and typing has become an important tool for writinggood, maintainable Python code. Many changes have been made to the type system to covermore use cases and improve usability. Several type checkers have been created, eachwith their own strengths. The type annotation syntax has driven several major innovationsin the Python ecosystem, such as the populardataclasses module, runtime typechecking and validation by packages such asPydantic,and static compilation by tools such asmypyc.
However, as the type system has grown, several interrelated problems with the currentway to manage the type system have become apparent.
The Python type system was initially created by a PEP (PEP 484), andchanges to the type system are still made by PEPs. The specification forthe Python type system, to the extent there is one, consists of this seriesof PEPs. But Standards Track PEPs aren’t meant to be living documentsor specifications; they are change proposals.
An example may illustrate the problem here. Around the same timeas the introduction of thetyping module byPEP 484,PEP 3156introduced theasyncio module, another major new feature that hasbeen instrumental to the success of Python 3. Both moduleshave evolved greatly since their initial creation and inspired changes to thecore language.
However,asyncio andtyping are different in an essential aspect:a user who usesasyncio interacts only with the standard library itself,while a user oftyping has to also think about an external tool, the typechecker. The Python language reference covers the symbols in the typing module, but doesnot (and should not) go into detail on how the full type system should beinterpreted by type checkers. That material currently exists only in the PEPs.
This problem is shared by the packaging ecosystem, which attempts to solve itby maintaining a separate set ofPyPA specifications.
Because the PEPs are the only specification we have, anything that could be seenas a change to the specification would theoretically require a new PEP. But thatis often too heavy a process for a small change. Sometimes changes are madedirectly to old PEPs instead, but that goes against the idea that accepted andimplemented PEPs become historical documents that should no longer be changed.
Some concrete examples include:
typing.NoReturn cannot be used inargument annotations. Nevertheless, type checkers have long accepted suchusage.typing_extensions package provides backports of newtype system features. Type checkers are expected to treat symbols in thismodule the same as symbols intyping, but this is not explicitlyspecified in any of the PEPs.While the PEPs provide a specification, they are often not sufficiently precise(sometimes intentionally so). This is especially true as the combinatorialcomplexity of the type system has grown.
It ends up falling to individual type checkers to decide how to navigateunderspecified areas. In cases where type checkers informally coordinate, thisresults in de facto standards that aren’t clearly recorded anywhere, makingthe type system less accessible to newcomers. For example:
@overload matching worksParamSpecshould workwith methods__exit__The SC has the entire language in its remit, and is not well-placed to makedecisions that are purely about the type system – if only because they don’t havethe time to deal with type system arcana alongside their other responsibilities.This is similar in spirit to the reasons why the Steering Council sometimes usesPEP delegation.
This PEP was endorsed by maintainers of all majortype checkers, includingRebecca Chen (pytype),Eric Traut (Pyright),and privately by maintainers of mypy and Pyre.
We propose the creation of a new group, the Typing Council. This group willbe responsible for developing and maintaining the Python type system, andfor solving the above problems.
The “operations and process” section describes how this group would operate andbe governed.
The more exciting “projects” section describes solutions to the above problemsthat the Typing Council could shepherd.
The Typing Council’s mandate is to ensure that the Python type system is:
typing module does followPython’s standard compatibility policy inPEP 387.The council would have three to five members, comprised of prominent community members,such as Python core developers and maintainers of major type checkers. The membersshould include people affiliated with a variety of projects related to type checking,which may include type checkers, CPython, typeshed, or other projects.
The initial members of the council are:
Current membership of the council is recorded in thepython/typing-council repository.
There is no termlimit for council members. Council members may resign their position at any time.There is an expectation that each member serves at most five consecutive yearsbefore resigning.
If there is a vacancy and there are three or more remaining members, it is upto the Council to decide whether to appoint a new member.To determine replacements, nominations will be collected from the typingcommunity. Self-nominations are allowed. The existing Typing Council will then decidethe replacement member(s) from the nominees. The expectation is that this wouldbe done by fiat, but the Typing Council can choose a replacement by any meansthey see fit, including a vote.
The Typing Council remains accountable to the Steering Council. At any point,for any reason, the Steering Council could (publicly or privately) make aspecific change or request a non-specific change to the composition of theTyping Council.
We acknowledge that this is a not particularly democratic structure and putsa lot of faith in the Typing Council. However, the Python community has a longhistory of success with not particularly democratic structures! We believeself-governance, cycling of membership, and accountability to theSteering Council will be sufficient to ensure that the Typing Council is meetingthe needs of the community.
The council would operate primarily through reviews of GitHub PRs. Regularmeetings are likely not necessary, but the council may set up video calls, aprivate chat, or whatever other mechanism they decide upon internally.
The council should aim for transparency, posting all decisions publicly ondiscuss.python.org, with arationale if possible. Before making a decision, the council should giveall interested community members a chance to weigh in. There should be atleast a week between the start of a discussion and the council’s decision.
Members of the council will be eligible to sponsor PEPs. If this PEP is accepted,PEP 1 should be amended to note this fact.
Just like today, the Python Steering Council would remain responsible for theoverall direction of the Python language and would continue to decide ontyping-related PEPs. The Typing Council would provide written opinions andrecommendations to the Steering Council on typing-related PEPs.
However, smaller changes to the type system could be madeby the Typing Council directly. The Steering Council could also chooseto delegate decisions on some PEPs to the Typing Council (exactly as any otherPEP delegation).
Some examples of how past and recent issues could have been handled under this model:
@override), which affects only users of typecheckers and does not change the overall language, would also by defaultbe decided upon by the Steering Council. However, such PEPs could bedelegated to the Typing Council for a decision (like any other PEP delegation).Other examples of PEPs that could potentially be delegated includePEP 647 (type guards),PEP 655 (individual requiredTypedDict items),PEP 673 (Self), andPEP 675 (Literal).typing.Never as an alias fortyping.NoReturn, would be done by means of a PR to the spec andconformance test suite. The TypingCouncil would then decide whether or not to merge the PR. They may ask for thefeature to be specified and discussed in a PEP if they feel that is warranted.The runtimetyping module will continue to be maintained by theCPython core developer team. However, any changes to the runtime module thataffect type checker behavior should be made in conjunction with a changeto the specification (see below) and should be approved by the Typing Council.For example, in Python 3.11 the core developers added the new functiontyping.assert_type(). If the Typing Council had been in place, thischange would require a matching change to the specification and approvalby the Typing Council. On the other hand, Python 3.11 also added thetyping.get_overloads() introspection helper. As this function does notaffect type checker behavior, it would not require approval by the TypingCouncil. However, as support for runtime type checkers is within the remitof the Council, they should monitor such changes and provide feedback whenappropriate.
The Typing Council has no direct authority over type checkers; it cannotforce them to implement particular features or make behavior changes. Typecheckers are incentivized to follow the specification set out by the Councilbecause it allows them to take advantage of shared resources, such aslibraries that expose typing information that follows the specification,the stub files in typeshed, thetyping standard library module, anduser documentation that covers the standard type system.Type checkers are free to extend the type system or deviate from thespecification, but they should document such differences clearly.
The fact that type checkers need to implement any decisions made by theTyping Council acts as a useful brake on the Council, ensuring that itsdecisions are conservative and well-considered. Individual type checkersremain free to innovate as they see fit, and successful innovations canbe incorporated into the standard type system.
Here are some efforts a Typing Council would be responsible for.
A conformance test suite would provide machine checkable documentation for howtype checkers should check Python code, accompanied by the results of major typechecker implementations on the test suite. A rough sketch for what this couldlook like wascreated by Shantanu.
This would contain prescriptive tests from behavior prescribed by previous PEPsand descriptive tests that let us document behavior of existing implementationsin areas that are not prescribed by any standard. These descriptions would beuseful to inform efforts below and to identify areas of focus forstandardization.
A specification would initially be created by stitching together thespecification sections from the existing PEPs, and then gradually improved toclarify points of confusion and cover more areas. A draft of such astitched-together spec wascreated by Jelle.
The specification has a few audiences:
Notably, the specification is not aimed at application developers who usetyping. Such users typically do notneed to worry about compatibility across type checkers. Theyare better served by a more informal user-facing reference, which is discussedin the next section.
There are different opinions within the community about how formal such aspecification should be. While this document recommends an incrementalapproach that builds off existing specification, it does not aim toprescribe a final state. The Typing Council would provide a mechanismto allow the specification to evolve to meet the level of formality thatthe community desires, for instance, by incorporating parts ofKevin Millikin’sdocument on “Python Static Types”as a means to achieve a better formalisation of the spec.
Proposed changes to the specification, including PEPs, shouldgenerally be accompanied by the following:
Documentation is important for the success of the Python type system, sothe Typing Council should ensure that there is good documentation for thetype system.
As mentioned previously, PEPs are point in time change proposals aimed atmultiple audiences that are hard to clarify. This makes them ill-suited as userdocumentation. The specification discussed in the previous section wouldbe a living document, but it would likely be too technical to serve asdocumentation for normal usage.
Therefore, a separate user-facing reference for the type system would beuseful. Such an effort could expand the documentation ontyping.python.org andreuse material from the documentation sections of individual type checkersand the CPython documentation.
This PEP serves as a charter for the Typing Council. Changes to its operationcan be made either through a new PEP or through a change to this PEP. In eithercase, the change would be decided upon by the Steering Council after discussionin the community.
This PEP proposes creating the typing specification by starting from theexisting PEPs, then clarifying and improving the specification as necessary.Some members of the community prefer to start from scratch, writing a new,more formal specification covering the entire type system. This couldprovide a more solid basis for the specification.
However, this would be a much larger undertaking. The existing formalizationeffort by Kevin Millikin is a good start, but so far covers only a subset ofPEP 484. Covering the rest of the type system would likely require severaltimes more effort when we consider that major type system features suchastyping.Protocol,typing.Literal, andtyping.TypedDictwere introduced only after PEP 484. It is not clear that there is even energyin the community for such a huge undertaking. Even if someone steps up todo all the work of putting together a specification, lots of effort would berequired from community members and type checker maintainers to considerwhether the specification accurately reflects current behavior, and if not,whether the specification or the type checkers should change.
Starting with the existing PEPs creates a lower-quality specification,but it means that the Typing Council can immediately start making a differenceanywhere in the type system by improving and clarifying the specification.A formalization effort can still proceed by gradually replacing sections of thespecification.
An earlier draft of this PEP suggested that the Steering Council appointmembers of the Typing Council each year. The current Steering Council suggestedthat it would be better to have the Typing Council self-organise and avoidthe need for the Steering Council to continuously supervise the Typing Council.
Alternate governance mechanisms are possible, including more democratic ones,but these typically raise several thorny questions, require much heavieramounts of process and are potentially more divisive. For example, see the PEP8000 series, or recent discussions about alternative governance in other Pythonsubcommunities. Ultimately, the Typing Council exists under the authority of theSteering Council, and so can rely on it to bootstrap governance and serve as anaccountability mechanism.
We are hopeful substantial progress will be made on projects that improve thetype system regardless of whether this PEP is accepted. We anticipate projectslike specification or the potential for PEP delegation would benefit more from aTyping Council, and projects like end user documentation would benefit less.Certainly the bottleneck is likely to be contributor effort, not governance.
However, currently the tools available to the community to resolve potentialcontention are either establishment of approximate consensus or the exercise ofpower by individual projects or contributors. While very valuable, the former isa slow process that can often end in inaction. The latter can result in a lessconsistent ecosystem. Finally, easily legible governance structures make thecommunity more accessible and equitable.
To ask the Typing Council for a decision,community members may open an issue in thepython/typing-councilrepository.
This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.
Source:https://github.com/python/peps/blob/main/peps/pep-0729.rst
Last modified:2025-03-05 16:28:34 GMT