Movatterモバイル変換


[0]ホーム

URL:


Following system colour schemeSelected dark colour schemeSelected light colour scheme

Python Enhancement Proposals

PEP 585 – Type Hinting Generics In Standard Collections

Author:
Łukasz Langa <lukasz at python.org>
Discussions-To:
Typing-SIG list
Status:
Final
Type:
Standards Track
Topic:
Typing
Created:
03-Mar-2019
Python-Version:
3.9
Resolution:
Python-Dev thread

Table of Contents

Important

This PEP is a historical document. The up-to-date, canonical documentation can now be found atGeneric Alias Typeand the documentation for__class_getitem__().

×

SeePEP 1 for how to propose changes.

Abstract

Static typing as defined by PEPs 484, 526, 544, 560, and 563 was builtincrementally on top of the existing Python runtime and constrained byexisting syntax and runtime behavior. This led to the existence ofa duplicated collection hierarchy in thetyping module due togenerics (for exampletyping.List and the built-inlist).

This PEP proposes to enable support for the generics syntax in allstandard collections currently available in thetyping module.

Rationale and Goals

This change removes the necessity for a parallel type hierarchy in thetyping module, making it easier for users to annotate their programsand easier for teachers to teach Python.

Terminology

Generic (n.) – a type that can be parameterized, typically a container.Also known as aparametric type or ageneric type. For example:dict.

parameterized generic – a specific instance of a generic with theexpected types for container elements provided. Also known asaparameterized type. For example:dict[str,int].

Backwards compatibility

Tooling, including type checkers and linters, will have to be adapted torecognize standard collections as generics.

On the source level, the newly described functionality requiresPython 3.9. For use cases restricted to type annotations, Python fileswith the “annotations” future-import (available since Python 3.7) canparameterize standard collections, including builtins. To reiterate,that depends on the external tools understanding that this is valid.

Implementation

Starting with Python 3.7, whenfrom__future__importannotations isused, function and variable annotations can parameterize standardcollections directly. Example:

from__future__importannotationsdeffind(haystack:dict[str,list[int]])->int:...

Usefulness of this syntax beforePEP 585 is limited as external toolinglike Mypy does not recognize standard collections as generic. Moreover,certain features of typing like type aliases or casting require puttingtypes outside of annotations, in runtime context. While these arerelatively less common than type annotations, it’s important to allowusing the same type syntax in all contexts. This is why starting withPython 3.9, the following collections become generic using__class_getitem__() to parameterize contained types:

  • tuple # typing.Tuple
  • list # typing.List
  • dict # typing.Dict
  • set # typing.Set
  • frozenset # typing.FrozenSet
  • type # typing.Type
  • collections.deque
  • collections.defaultdict
  • collections.OrderedDict
  • collections.Counter
  • collections.ChainMap
  • collections.abc.Awaitable
  • collections.abc.Coroutine
  • collections.abc.AsyncIterable
  • collections.abc.AsyncIterator
  • collections.abc.AsyncGenerator
  • collections.abc.Iterable
  • collections.abc.Iterator
  • collections.abc.Generator
  • collections.abc.Reversible
  • collections.abc.Container
  • collections.abc.Collection
  • collections.abc.Callable
  • collections.abc.Set # typing.AbstractSet
  • collections.abc.MutableSet
  • collections.abc.Mapping
  • collections.abc.MutableMapping
  • collections.abc.Sequence
  • collections.abc.MutableSequence
  • collections.abc.ByteString
  • collections.abc.MappingView
  • collections.abc.KeysView
  • collections.abc.ItemsView
  • collections.abc.ValuesView
  • contextlib.AbstractContextManager # typing.ContextManager
  • contextlib.AbstractAsyncContextManager # typing.AsyncContextManager
  • re.Pattern # typing.Pattern, typing.re.Pattern
  • re.Match # typing.Match, typing.re.Match

Importing those fromtyping is deprecated. Due toPEP 563 and theintention to minimize the runtime impact of typing, this deprecationwill not generate DeprecationWarnings. Instead, type checkers may warnabout such deprecated usage when the target version of the checkedprogram is signalled to be Python 3.9 or newer. It’s recommended toallow for those warnings to be silenced on a project-wide basis.

The deprecated functionality may eventually be removed from thetypingmodule. Removal will occur no sooner than Python 3.9’s end of life,scheduled for October 2025.

Parameters to generics are available at runtime

Preserving the generic type at runtime enables introspection of the typewhich can be used for API generation or runtime type checking. Suchusage is already present in the wild.

Just like with thetyping module today, the parameterized generictypes listed in the previous section all preserve their type parametersat runtime:

>>>list[str]list[str]>>>tuple[int,...]tuple[int, ...]>>>ChainMap[str,list[str]]collections.ChainMap[str, list[str]]

This is implemented using a thin proxy type that forwards all methodcalls and attribute accesses to the bare origin type with the followingexceptions:

  • the__repr__ shows the parameterized type;
  • the__origin__ attribute points at the non-parameterizedgeneric class;
  • the__args__ attribute is a tuple (possibly of length1) of generic types passed to the original__class_getitem__;
  • the__parameters__ attribute is a lazily computed tuple(possibly empty) of unique type variables found in__args__;
  • the__getitem__ raises an exception to disallow mistakeslikedict[str][str]. However it allows e.g.dict[str,T][int]and in that case returnsdict[str,int].

This design means that it is possible to create instances ofparameterized collections, like:

>>>l=list[str]()[]>>>listislist[str]False>>>list==list[str]False>>>list[str]==list[str]True>>>list[str]==list[int]False>>>isinstance([1,2,3],list[str])TypeError: isinstance() arg 2 cannot be a parameterized generic>>>issubclass(list,list[str])TypeError: issubclass() arg 2 cannot be a parameterized generic>>>isinstance(list[str],types.GenericAlias)True

Objects created with bare types and parameterized types are exactly thesame. The generic parameters are not preserved in instances createdwith parameterized types, in other words generic types erase typeparameters during object creation.

One important consequence of this is that the interpreter doesnotattempt to type check operations on the collection created witha parameterized type. This provides symmetry between:

l:list[str]=[]

and:

l=list[str]()

For accessing the proxy type from Python code, it will be exportedfrom thetypes module asGenericAlias.

Pickling or (shallow- or deep-) copying aGenericAlias instancewill preserve the type, origin, attributes and parameters.

Forward compatibility

Future standard collections must implement the same behavior.

Reference implementation

A proof-of-concept or prototypeimplementation exists.

Rejected alternatives

Do nothing

Keeping the status quo forces Python programmers to perform book-keepingof imports from thetyping module for standard collections, makingall but the simplest annotations cumbersome to maintain. The existenceof parallel types is confusing to newcomers (why is there bothlistandList?).

The above problems also don’t exist in user-built generic classes whichshare runtime functionality and the ability to use them as generic typeannotations. Making standard collections harder to use in type hintingfrom user classes hindered typing adoption and usability.

Generics erasure

It would be easier to implement__class_getitem__ on the listedstandard collections in a way that doesn’t preserve the generic type,in other words:

>>>list[str]<class 'list'>>>>tuple[int,...]<class 'tuple'>>>>collections.ChainMap[str,list[str]]<class 'collections.ChainMap'>

This is problematic as it breaks backwards compatibility: currentequivalents of those types in thetyping moduledo preservethe generic type:

>>>fromtypingimportList,Tuple,ChainMap>>>List[str]typing.List[str]>>>Tuple[int,...]typing.Tuple[int, ...]>>>ChainMap[str,List[str]]typing.ChainMap[str, typing.List[str]]

As mentioned in the “Implementation” section, preserving the generictype at runtime enables runtime introspection of the type which can beused for API generation or runtime type checking. Such usage is alreadypresent in the wild.

Additionally, implementing subscripts as identity functions would makePython less friendly to beginners. Say, if a user is mistakenly passinga list type instead of a list object to a function, and that function isindexing the received object, the code would no longer raise an error.

Today:

>>>l=list>>>l[-1]TypeError: 'type' object is not subscriptable

With__class_getitem__ as an identity function:

>>>l=list>>>l[-1]list

The indexing being successful here would likely end up raising anexception at a distance, confusing the user.

Disallowing instantiation of parameterized types

Given that the proxy type which preserves__origin__ and__args__ is mostly useful for runtime introspection purposes,we might have disallowed instantiation of parameterized types.

In fact, forbidding instantiation of parameterized types is what thetyping module does today for types which parallel builtincollections (instantiation of other parameterized types is allowed).

The original reason for this decision was to discourage spuriousparameterization which made object creation up to two orders of magnitudeslower compared to the special syntax available for those builtincollections.

This rationale is not strong enough to allow the exceptional treatmentof builtins. All other parameterized types can be instantiated,including parallels of collections in the standard library. Moreover,Python allows for instantiation of lists usinglist() and somebuiltin collections don’t provide special syntax for instantiation.

Makingisinstance(obj,list[str]) perform a check ignoring generics

An earlier version of this PEP suggested treating parameterized genericslikelist[str] as equivalent to their non-parameterized variantslikelist for purposes ofisinstance() andissubclass().This would be symmetrical to howlist[str]() creates a regular list.

This design was rejected becauseisinstance() andissubclass()checks with parameterized generics would read like element-by-elementruntime type checks. The result of those checks would be surprising,for example:

>>>isinstance([1,2,3],list[str])True

Note the object doesn’t match the provided generic type butisinstance() still returnsTrue because it only checks whetherthe object is a list.

If a library is faced with a parameterized generic and would like toperform anisinstance() check using the base type, that type canbe retrieved using the__origin__ attribute on the parameterizedgeneric.

Makingisinstance(obj,list[str]) perform a runtime type check

This functionality requires iterating over the collection which isa destructive operation in some of them. This functionality would havebeen useful, however implementing the type checker within Python thatwould deal with complex types, nested type checking, type variables,string forward references, and so on is out of scope for this PEP.

Naming the typeGenericType instead ofGenericAlias

We considered a different name for this type, but decidedGenericAlias is better – these aren’t real types, they arealiases for the corresponding container type with some extra metadataattached.

Note on the initial draft

An early version of this PEP discussed matters beyond generics instandard collections. Those unrelated topics were removed for clarity.

Acknowledgments

Thank you to Guido van Rossum for his work on Python, and theimplementation of this PEP specifically.

Copyright

This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.


Source:https://github.com/python/peps/blob/main/peps/pep-0585.rst

Last modified:2025-02-01 08:59:27 GMT


[8]ページ先頭

©2009-2025 Movatter.jp