Movatterモバイル変換


[0]ホーム

URL:


Following system colour schemeSelected dark colour schemeSelected light colour scheme

Python Enhancement Proposals

PEP 586 – Literal Types

Author:
Michael Lee <michael.lee.0x2a at gmail.com>, Ivan Levkivskyi <levkivskyi at gmail.com>, Jukka Lehtosalo <jukka.lehtosalo at iki.fi>
BDFL-Delegate:
Guido van Rossum <guido at python.org>
Discussions-To:
Typing-SIG list
Status:
Final
Type:
Standards Track
Topic:
Typing
Created:
14-Mar-2019
Python-Version:
3.8
Post-History:
14-Mar-2019
Resolution:
Typing-SIG message

Table of Contents

Important

This PEP is a historical document: seeLiterals andtyping.Literal for up-to-date specs and documentation. Canonical typing specs are maintained at thetyping specs site; runtime typing behaviour is described in the CPython documentation.

×

See thetyping specification update process for how to propose changes to the typing spec.

Abstract

This PEP proposes addingLiteral types to thePEP 484 ecosystem.Literal types indicate that some expression has literally aspecific value. For example, the following function will acceptonly expressions that have literally the value “4”:

fromtypingimportLiteraldefaccepts_only_four(x:Literal[4])->None:passaccepts_only_four(4)# OKaccepts_only_four(19)# Rejected

Motivation and Rationale

Python has many APIs that return different types depending on thevalue of some argument provided. For example:

  • open(filename,mode) returns eitherIO[bytes] orIO[Text]depending on whether the second argument is something liker orrb.
  • subprocess.check_output(...) returns either bytes or textdepending on whether theuniversal_newlines keyword argument isset toTrue or not.

This pattern is also fairly common in many popular 3rd party libraries.For example, here are just two examples from pandas and numpy respectively:

  • pandas.concat(...) will return eitherSeries orDataFrame depending on whether theaxis argument is set to0 or 1.
  • numpy.unique will return either a single array or a tuple containinganywhere from two to four arrays depending on three boolean flag values.

The typing issue tracker contains someadditional examples and discussion.

There is currently no way of expressing the type signatures of thesefunctions:PEP 484 does not include any mechanism for writing signatureswhere the return type varies depending on the value passed in.Note that this problem persists even if we redesign these APIs toinstead accept enums:MyEnum.FOO andMyEnum.BAR are bothconsidered to be of typeMyEnum.

Currently, type checkers work around this limitation by adding ad hocextensions for important builtins and standard library functions. Forexample, mypy comes bundled with a plugin that attempts to infer moreprecise types foropen(...). While this approach works for standardlibrary functions, it’s unsustainable in general: it’s not reasonable toexpect 3rd party library authors to maintain plugins for N differenttype checkers.

We propose addingLiteral types to address these gaps.

Core Semantics

This section outlines the baseline behavior of literal types.

Core behavior

Literal types indicate that a variable has a specific andconcrete value. For example, if we define some variablefoo to havetypeLiteral[3], we are declaring thatfoo must be exactly equalto3 and no other value.

Given some valuev that is a member of typeT, the typeLiteral[v] shall be treated as a subtype ofT. For example,Literal[3] is a subtype ofint.

All methods from the parent type will be directly inherited by theliteral type. So, if we have some variablefoo of typeLiteral[3]it’s safe to do things likefoo+5 sincefoo inherits int’s__add__ method. The resulting type offoo+5 isint.

This “inheriting” behavior is identical to how wehandle NewTypes.

Equivalence of two Literals

Two typesLiteral[v1] andLiteral[v2] are equivalent whenboth of the following conditions are true:

  1. type(v1)==type(v2)
  2. v1==v2

For example,Literal[20] andLiteral[0x14] are equivalent.However,Literal[0] andLiteral[False] isnot equivalentdespite that0==False evaluates to ‘true’ at runtime:0has typeint andFalse has typebool.

Shortening unions of literals

Literals are parameterized with one or more values. When a Literal isparameterized with more than one value, it’s treated as exactly equivalentto the union of those types. That is,Literal[v1,v2,v3] is equivalenttoUnion[Literal[v1],Literal[v2],Literal[v3]].

This shortcut helps make writing signatures for functions that acceptmany different literals more ergonomic — for example, functions likeopen(...):

# Note: this is a simplification of the true type signature._PathType=Union[str,bytes,int]@overloaddefopen(path:_PathType,mode:Literal["r","w","a","x","r+","w+","a+","x+"],)->IO[Text]:...@overloaddefopen(path:_PathType,mode:Literal["rb","wb","ab","xb","r+b","w+b","a+b","x+b"],)->IO[bytes]:...# Fallback overload for when the user isn't using literal types@overloaddefopen(path:_PathType,mode:str)->IO[Any]:...

The provided values do not all have to be members of the same type.For example,Literal[42,"foo",True] is a legal type.

However, Literalmust be parameterized with at least one type.Types likeLiteral[] orLiteral are illegal.

Legal and illegal parameterizations

This section describes what exactly constitutes a legalLiteral[...] type:what values may and may not be used as parameters.

In short, aLiteral[...] type may be parameterized by one or more literalexpressions, and nothing else.

Legal parameters forLiteral at type check time

Literal may be parameterized with literal ints, byte and unicode strings,bools, Enum values andNone. So for example, all ofthe following would be legal:

Literal[26]Literal[0x1A]# Exactly equivalent to Literal[26]Literal[-4]Literal["hello world"]Literal[b"hello world"]Literal[u"hello world"]Literal[True]Literal[Color.RED]# Assuming Color is some enumLiteral[None]

Note: Since the typeNone is inhabited by just a singlevalue, the typesNone andLiteral[None] are exactly equivalent.Type checkers may simplifyLiteral[None] into justNone.

Literal may also be parameterized by other literal types, or type aliasesto other literal types. For example, the following is legal:

ReadOnlyMode=Literal["r","r+"]WriteAndTruncateMode=Literal["w","w+","wt","w+t"]WriteNoTruncateMode=Literal["r+","r+t"]AppendMode=Literal["a","a+","at","a+t"]AllModes=Literal[ReadOnlyMode,WriteAndTruncateMode,WriteNoTruncateMode,AppendMode]

This feature is again intended to help make using and reusing literal typesmore ergonomic.

Note: As a consequence of the above rules, type checkers are also expectedto support types that look like the following:

Literal[Literal[Literal[1,2,3],"foo"],5,None]

This should be exactly equivalent to the following type:

Literal[1,2,3,"foo",5,None]

…and also to the following type:

Optional[Literal[1,2,3,"foo",5]]

Note: String literal types likeLiteral["foo"] should subtype eitherbytes or unicode in the same way regular string literals do at runtime.

For example, in Python 3, the typeLiteral["foo"] is equivalent toLiteral[u"foo"], since"foo" is equivalent tou"foo" in Python 3.

Similarly, in Python 2, the typeLiteral["foo"] is equivalent toLiteral[b"foo"] – unless the file includes afrom__future__importunicode_literals import, in which case it would beequivalent toLiteral[u"foo"].

Illegal parameters forLiteral at type check time

The following parameters are intentionally disallowed by design:

  • Arbitrary expressions likeLiteral[3+4] orLiteral["foo".replace("o","b")].
    • Rationale: Literal types are meant to be aminimal extension to thePEP 484 typing ecosystem and requiring typecheckers to interpret potentially expressions inside types adds toomuch complexity. Also seeRejected or out-of-scope ideas.
    • As a consequence, complex numbers likeLiteral[4+3j] andLiteral[-4+2j] are also prohibited. For consistency, literals likeLiteral[4j] that contain just a single complex number are alsoprohibited.
    • The only exception to this rule is the unary- (minus) for ints: typeslikeLiteral[-5] areaccepted.
  • Tuples containing valid literal types likeLiteral[(1,"foo","bar")].The user could always express this type asTuple[Literal[1],Literal["foo"],Literal["bar"]] instead. Also,tuples are likely to be confused with theLiteral[1,2,3]shortcut.
  • Mutable literal data structures like dict literals, list literals, orset literals: literals are always implicitly final and immutable. So,Literal[{"a":"b","c":"d"}] is illegal.
  • Any other types: for example,Literal[Path], orLiteral[some_object_instance] are illegal. This includes typevars: ifT is a typevar,Literal[T] is not allowed. Typevars can vary overonly types, never over values.

The following are provisionally disallowed for simplicity. We can considerallowing them in future extensions of this PEP.

  • Floats: e.g.Literal[3.14]. Representing Literals of infinity or NaNin a clean way is tricky; real-world APIs are unlikely to vary theirbehavior based on a float parameter.
  • Any: e.g.Literal[Any].Any is a type, andLiteral[...] ismeant to contain values only. It is also unclear whatLiteral[Any]would actually semantically mean.

Parameters at runtime

Although the set of parametersLiteral[...] may contain at type check timeis very small, the actual implementation oftyping.Literal will not performany checks at runtime. For example:

defmy_function(x:Literal[1+2])->int:returnx*3x:Literal=3y:Literal[my_function]=my_function

The type checker should reject this program: all three uses ofLiteral areinvalid according to this spec. However, Python itselfshould execute this program with no errors.

This is partly to help us preserve flexibility in case we want to expand thescope of whatLiteral can be used for in the future, and partly becauseit is not possible to detect all illegal parameters at runtime to begin with.For example, it is impossible to distinguish betweenLiteral[1+2] andLiteral[3] at runtime.

Literals, enums, and forward references

One potential ambiguity is between literal strings and forwardreferences to literal enum members. For example, suppose we have thetypeLiteral["Color.RED"]. Does this literal typecontain a string literal or a forward reference to someColor.REDenum member?

In cases like these, we always assume the user meant to construct aliteral string. If the user wants a forward reference, they must wrapthe entire literal type in a string – e.g."Literal[Color.RED]".

Type inference

This section describes a few rules regarding type inference andliterals, along with some examples.

Backwards compatibility

When type checkers add support for Literal, it’s important they do soin a way that maximizes backwards-compatibility. Type checkers shouldensure that code that used to type check continues to do so after supportfor Literal is added on a best-effort basis.

This is particularly important when performing type inference. Forexample, given the statementx="blue", should the inferredtype ofx bestr orLiteral["blue"]?

One naive strategy would be to always assume expressions are intendedto be Literal types. So,x would always have an inferred type ofLiteral["blue"] in the example above. This naive strategy is almostcertainly too disruptive – it would cause programs like the followingto start failing when they previously did not:

# If a type checker infers 'var' has type Literal[3]# and my_list has type List[Literal[3]]...var=3my_list=[var]# ...this call would be a type-error.my_list.append(4)

Another example of when this strategy would fail is when setting fieldsin objects:

classMyObject:def__init__(self)->None:# If a type checker infers MyObject.field has type Literal[3]...self.field=3m=MyObject()# ...this assignment would no longer type checkm.field=4

An alternative strategy thatdoes maintain compatibility in every case wouldbe to always assume expressions arenot Literal types unless they areexplicitly annotated otherwise. A type checker using this strategy wouldalways infer thatx is of typestr in the first example above.

This is not the only viable strategy: type checkers should feel free to experimentwith more sophisticated inference techniques. This PEP does not mandate anyparticular strategy; it only emphasizes the importance of backwards compatibility.

Using non-Literals in Literal contexts

Literal types follow the existing rules regarding subtyping with no additionalspecial-casing. For example, programs like the following are type safe:

defexpects_str(x:str)->None:...var:Literal["foo"]="foo"# Legal: Literal["foo"] is a subtype of strexpects_str(var)

This also means non-Literal expressions in general should not automaticallybe cast to Literal. For example:

defexpects_literal(x:Literal["foo"])->None:...defrunner(my_str:str)->None:# ILLEGAL: str is not a subclass of Literal["foo"]expects_literal(my_str)

Note: If the user wants their API to support accepting both literalsand the original type – perhaps for legacy purposes – they shouldimplement a fallback overload. SeeInteractions with overloads.

Interactions with other types and features

This section discusses how Literal types interact with other existing types.

Intelligent indexing of structured data

Literals can be used to “intelligently index” into structured types liketuples, NamedTuple, and classes. (Note: this is not an exhaustive list).

For example, type checkers should infer the correct value type whenindexing into a tuple using an int key that corresponds a valid index:

a:Literal[0]=0b:Literal[5]=5some_tuple:Tuple[int,str,List[bool]]=(3,"abc",[True,False])reveal_type(some_tuple[a])# Revealed type is 'int'some_tuple[b]# Error: 5 is not a valid index into the tuple

We expect similar behavior when using functions like getattr:

classTest:def__init__(self,param:int)->None:self.myfield=paramdefmymethod(self,val:int)->str:...a:Literal["myfield"]="myfield"b:Literal["mymethod"]="mymethod"c:Literal["blah"]="blah"t=Test()reveal_type(getattr(t,a))# Revealed type is 'int'reveal_type(getattr(t,b))# Revealed type is 'Callable[[int], str]'getattr(t,c)# Error: No attribute named 'blah' in Test

Note: SeeInteractions with Final for a proposal on how we canexpress the variable declarations above in a more compact manner.

Interactions with overloads

Literal types and overloads do not need to interact in a specialway: the existing rules work fine.

However, one important use case type checkers must take care tosupport is the ability to use afallback when the user is not using literaltypes. For example, consideropen:

_PathType=Union[str,bytes,int]@overloaddefopen(path:_PathType,mode:Literal["r","w","a","x","r+","w+","a+","x+"],)->IO[Text]:...@overloaddefopen(path:_PathType,mode:Literal["rb","wb","ab","xb","r+b","w+b","a+b","x+b"],)->IO[bytes]:...# Fallback overload for when the user isn't using literal types@overloaddefopen(path:_PathType,mode:str)->IO[Any]:...

If we were to change the signature ofopen to use just the first two overloads,we would break any code that does not pass in a literal string expression.For example, code like this would be broken:

mode:str=pick_file_mode(...)withopen(path,mode)asf:# f should continue to be of type IO[Any] here

A little more broadly: we propose adding a policy to typeshed thatmandates that whenever we add literal types to some existing API, we alsoalways include a fallback overload to maintain backwards-compatibility.

Interactions with generics

Types likeLiteral[3] are meant to be just plain old subclasses ofint. This means you can use types likeLiteral[3] anywhereyou could use normal types, such as with generics.

This means that it is legal to parameterize generic functions orclasses using Literal types:

A=TypeVar('A',bound=int)B=TypeVar('B',bound=int)C=TypeVar('C',bound=int)# A simplified definition for Matrix[row, column]classMatrix(Generic[A,B]):def__add__(self,other:Matrix[A,B])->Matrix[A,B]:...def__matmul__(self,other:Matrix[B,C])->Matrix[A,C]:...deftranspose(self)->Matrix[B,A]:...foo:Matrix[Literal[2],Literal[3]]=Matrix(...)bar:Matrix[Literal[3],Literal[7]]=Matrix(...)baz=foo@barreveal_type(baz)# Revealed type is 'Matrix[Literal[2], Literal[7]]'

Similarly, it is legal to construct TypeVars with value restrictionsor bounds involving Literal types:

T=TypeVar('T',Literal["a"],Literal["b"],Literal["c"])S=TypeVar('S',bound=Literal["foo"])

…although it is unclear when it would ever be useful to construct aTypeVar with a Literal upper bound. For example, theS TypeVar inthe above example is essentially pointless: we can get equivalent behaviorby usingS=Literal["foo"] instead.

Note: Literal types and generics deliberately interact in only verybasic and limited ways. In particular, libraries that want to type checkcode containing a heavy amount of numeric or numpy-style manipulation willalmost certainly likely find Literal types as proposed in this PEP to beinsufficient for their needs.

We considered several different proposals for fixing this, but ultimatelydecided to defer the problem of integer generics to a later date. SeeRejected or out-of-scope ideas for more details.

Interactions with enums and exhaustiveness checks

Type checkers should be capable of performing exhaustiveness checks whenworking Literal types that have a closed number of variants, such asenums. For example, the type checker should be capable of inferring thatthe finalelse statement must be of typestr, since all threevalues of theStatus enum have already been exhausted:

classStatus(Enum):SUCCESS=0INVALID_DATA=1FATAL_ERROR=2defparse_status(s:Union[str,Status])->None:ifsisStatus.SUCCESS:print("Success!")elifsisStatus.INVALID_DATA:print("The given data is invalid because...")elifsisStatus.FATAL_ERROR:print("Unexpected fatal error...")else:# 's' must be of type 'str' since all other options are exhaustedprint("Got custom status: "+s)

The interaction described above is not new: it’s alreadycodified within PEP 484.However, many typecheckers (such as mypy) do not yet implement this due to the expectedcomplexity of the implementation work.

Some of this complexity will be alleviated once Literal types are introduced:rather than entirely special-casing enums, we can instead treat them as beingapproximately equivalent to the union of their values and take advantage of anyexisting logic regarding unions, exhaustibility, type narrowing, reachability,and so forth the type checker might have already implemented.

So here, theStatus enum could be treated as being approximately equivalenttoLiteral[Status.SUCCESS,Status.INVALID_DATA,Status.FATAL_ERROR]and the type ofs narrowed accordingly.

Interactions with narrowing

Type checkers may optionally perform additional analysis for both enum andnon-enum Literal types beyond what is described in the section above.

For example, it may be useful to perform narrowing based on things likecontainment or equality checks:

defparse_status(status:str)->None:ifstatusin("MALFORMED","ABORTED"):# Type checker could narrow 'status' to type# Literal["MALFORMED", "ABORTED"] here.returnexpects_bad_status(status)# Similarly, type checker could narrow 'status' to Literal["PENDING"]ifstatus=="PENDING":expects_pending_status(status)

It may also be useful to perform narrowing taking into account expressionsinvolving Literal bools. For example, we can combineLiteral[True],Literal[False], and overloads to construct “custom type guards”:

@overloaddefis_int_like(x:Union[int,List[int]])->Literal[True]:...@overloaddefis_int_like(x:object)->bool:...defis_int_like(x):...vector:List[int]=[1,2,3]ifis_int_like(vector):vector.append(3)else:vector.append("bad")# This branch is inferred to be unreachablescalar:Union[int,str]ifis_int_like(scalar):scalar+=3# Type checks: type of 'scalar' is narrowed to 'int'else:scalar+="foo"# Type checks: type of 'scalar' is narrowed to 'str'

Interactions with Final

PEP 591 proposes adding a “Final” qualifier to the typingecosystem. This qualifier can be used to declare that some variable orattribute cannot be reassigned:

foo:Final=3foo=4# Error: 'foo' is declared to be Final

Note that in the example above, we know thatfoo will always be equal toexactly3. A type checker can use this information to deduce thatfoois valid to use in any context that expects aLiteral[3]:

defexpects_three(x:Literal[3])->None:...expects_three(foo)# Type checks, since 'foo' is Final and equal to 3

TheFinal qualifier serves as a shorthand for declaring that a variableiseffectively Literal.

If both this PEP andPEP 591 are accepted, type checkers are expected tosupport this shortcut. Specifically, given a variable or attribute assignmentof the formvar:Final=value wherevalue is a valid parameter forLiteral[...], type checkers should understand thatvar may be used inany context that expects aLiteral[value].

Type checkers are not obligated to understand any other uses of Final. Forexample, whether or not the following program type checks is left unspecified:

# Note: The assignment does not exactly match the form 'var: Final = value'.bar1:Final[int]=3expects_three(bar1)# May or may not be accepted by type checkers# Note: "Literal[1 + 2]" is not a legal type.bar2:Final=1+2expects_three(bar2)# May or may not be accepted by type checkers

Rejected or out-of-scope ideas

This section outlines some potential features that are explicitly out-of-scope.

True dependent types/integer generics

This proposal is essentially describing adding a very simplifieddependent type system to thePEP 484 ecosystem. One obvious extensionwould be to implement a full-fledged dependent type system that lets userspredicate types based on their values in arbitrary ways. That wouldlet us write signatures like the below:

# A vector has length 'n', containing elements of type 'T'classVector(Generic[N,T]):...# The type checker will statically verify our function genuinely does# construct a vector that is equal in length to "len(vec1) + len(vec2)"# and will throw an error if it does not.defconcat(vec1:Vector[A,T],vec2:Vector[B,T])->Vector[A+B,T]:# ...snip...

At the very least, it would be useful to add some form of integer generics.

Although such a type system would certainly be useful, it’s out of scopefor this PEP: it would require a far more substantial amount of implementationwork, discussion, and research to complete compared to the current proposal.

It’s entirely possible we’ll circle back and revisit this topic in the future:we very likely will need some form of dependent typing along with otherextensions like variadic generics to support popular libraries like numpy.

This PEP should be seen as a stepping stone towards this goal,rather than an attempt at providing a comprehensive solution.

Adding more concise syntax

One objection to this PEP is that having to explicitly writeLiteral[...]feels verbose. For example, instead of writing:

deffoobar(arg1:Literal[1],arg2:Literal[True])->None:pass

…it would be nice to instead write:

deffoobar(arg1:1,arg2:True)->None:pass

Unfortunately, these abbreviations simply will not work with theexisting implementation oftyping at runtime. For example, thefollowing snippet crashes when run using Python 3.7:

fromtypingimportTuple# Supposed to accept tuple containing the literals 1 and 2deffoo(x:Tuple[1,2])->None:pass

Running this yields the following exception:

TypeError:Tuple[t0,t1,...]:eachtmustbeatype.Got1.

We don’t want users to have to memorize exactly when it’s ok to elideLiteral, so we requireLiteral to always be present.

A little more broadly, we feel overhauling the syntax of types inPython is not within the scope of this PEP: it would be best to havethat discussion in a separate PEP, instead of attaching it to this one.So, this PEP deliberately does not try and innovate Python’s type syntax.

Backporting theLiteral type

Once this PEP is accepted, theLiteral type will need to be backported forPython versions that come bundled with older versions of thetyping module.We plan to do this by addingLiteral to thetyping_extensions 3rd partymodule, which contains a variety of other backported types.

Implementation

The mypy type checker currently has implemented a large subset of the behaviordescribed in this spec, with the exception of enum Literals and some of themore complex narrowing interactions described above.

Related work

This proposal was written based on the discussion that took place in thefollowing threads:

The overall design of this proposal also ended up converging intosomething similar to howliteral types are handled in TypeScript.

Acknowledgements

Thanks to Mark Mendoza, Ran Benita, Rebecca Chen, and the other members oftyping-sig for their comments on this PEP.

Additional thanks to the various participants in the mypy and typing issuetrackers, who helped provide a lot of the motivation and reasoning behindthis PEP.

Copyright

This document has been placed in the public domain.


Source:https://github.com/python/peps/blob/main/peps/pep-0586.rst

Last modified:2025-02-01 07:28:42 GMT


[8]ページ先頭

©2009-2025 Movatter.jp