from__future__importannotationsannotationlib module__annotations__VALUE_WITH_FAKE_GLOBALS format__annotations__SOURCE toSTRINGImportant
This PEP is a historical document. The up-to-date, canonical documentation can now be found atAnnotations andannotationlib.
×
SeePEP 1 for how to propose changes.
This PEP supplementsPEP 649 by providing various tweaks and additions to itsspecification:
from__future__importannotations (PEP 563) will continue to exist withits current behavior at least until Python 3.13 reaches its end-of-life. Subsequently,it will be deprecated and eventually removed.annotationlib, is added to provide tooling forannotations. It will include theget_annotations() function, an enum for annotationformats, aForwardRef class, and a helper function for calling__annotate__ functions.classmethod()and code that usesfunctools.wraps().__annotate__ functionsthat can be run in a “fake globals” environment. Instead, we add a fourth format,VALUE_WITH_FAKE_GLOBALS, to allow third-party implementors of annotate functions toindicate what formats they support.__annotations__ attribute directly will also clear__annotate__.SOURCE format is renamed toSTRING to improve clarity and reduce the risk ofuser confusion.PEP 649 provides an excellent framework for creating better semantics forannotations in Python. It solves a common pain point for users of annotations,including those using static type hints as well as those using runtime typing,and it makes the language more elegant and powerful.The PEP was originally proposed in 2021 for Python 3.10,and it was accepted in 2023. However, the implementation took longer than anticipated,and now the PEP is expected to be implemented in Python 3.14.
I have started working on the implementation of the PEP in CPython. I found thatthe PEP leaves some areas underspecified, and someof its decisions in corner cases are questionable. This new PEP proposes severalchanges and additions to the specification to address these issues.
This PEP supplements rather than supersedes PEP 649. The changes proposed hereshould make the overall user experience better, but they do not change thegeneral framework of the earlier PEP.
from__future__importannotationsPEP 563 previously introduced the future importfrom__future__importannotations,which changes all annotations to strings.PEP 649 proposes an alternative approachthat does not require this future import, and states:
If this PEP is accepted, PEP 563 will be deprecated and eventually removed.
However, the PEP does not provide a detailed plan for this deprecation.
There is some previous discussion of this topicon Discourse(note that in the linked post I proposed something different from what is proposed here).
We suggest the following deprecation plan:
from__future__importannotations will continue to work as itdid before, converting annotations into strings.__annotate__ function of objects withannotations will return the annotations as strings when called with theVALUEformat, reflecting the behavior of__annotations__.from__future__importannotations is deprecated. Compilingany code that uses the future import will emit aDeprecationWarning. This willhappen no sooner than the first release after Python 3.13 reaches its end-of-life, butthe community may decide to wait longer.SyntaxError, similar to any other undefined future import.Immediately make the future import a no-op: We considered applyingPEP 649 semanticsto all code in Python 3.14, making the future import a no-op. However, this would breakcode that works in 3.13 under the following set of conditions:
__future__importannotations is activetyping_extensions.TypedDict.This is expected to be a common pattern, so we cannot afford to break such code duringthe upgrade from 3.13 to 3.14.
Such code would still break when the future import is eventually removed. However, thisis many years in the future, giving affected libraries plenty of time to update their code.
Immediately deprecate the future import: Instead of waiting until Python 3.13 reachesits end-of-life, we could immediately start emitting warnings when the future import isused. However, many libraries are already usingfrom__future__importannotations asan elegant way to enable unrestricted forward references in their annotations. If we deprecatethe future import immediately, it would be impossible for these libraries to use unrestrictedforward references on all supported Python versions while avoiding deprecation warnings:unlike other features deprecated from the standard library, a__future__ import mustbe the first statement in a given module, meaning it would be impossible to onlyconditionally import__future__.annotations on Python 3.13 and lower. (The necessarysys.version_info check would count as a statement preceding the__future__ import.)
Keep the future import around forever: We could also decide to keep the future importindefinitely. However, this would permanently bifurcate the behavior of the Pythonlanguage. This is undesirable; the language should have only a single set of semantics,not two permanently different modes.
Make the future import a no-op in the future: Instead of eventually makingfrom__future__importannotations aSyntaxError, we could make it do nothinginstead at some point after Python 3.13 reaches its end-of-life. This still has someof the same issues outlined above around making it a no-op now, although the ecosystemwould have had much longer to adapt. It is better to have users explicitly removethe future import from their code in the future once they have confirmed they do notrely on stringized annotations.
annotationlib modulePEP 649 proposes to add tooling related to annotations to theinspectmodule. However, that module is rather large, has direct or indirect dependencieson at least 35 other standard library modules, and is so slow to import that otherstandard library modules are often discouraged from importing it. Furthermore, weanticipate adding more tools in addition to theinspect.get_annotations()function and theVALUE,FORWARDREF, andSOURCE formats.
A new standard library module provides a logical home for this functionality andalso enables us to add more tooling that is useful for consumers of annotations.
PEP 649 indicates thattyping.ForwardRef should be used to implement theFORWARDREF format ininspect.get_annotations(). However, the existing implementationoftyping.ForwardRef is intertwined with the rest of thetyping module,and it would not make sense to addtyping-specific behavior to the genericget_annotations()function. Furthermore,typing.ForwardRef is a problematicclass: it is public and documented, but the documentation lists no attributes or methodsfor it. Nonetheless, third-party libraries make use of some of its undocumentedattributes. For instance,PydanticandTypeguarduse the_evaluate method;beartypeandpyanalyzeuse the__forward_arg__ attribute.
We replace the existing but poorly specifiedtyping.ForwardRef with a new class,annotationlib.ForwardRef. It is designed to be mostly compatible with existing usesof thetyping.ForwardRef class, but without the behaviors specific to thetyping module. For compatibility with existing users, we keep the private_evaluate method, but mark it as deprecated. It delegates to a new public function inthetyping module,typing.evaluate_forward_ref, that is designed toevaluate forward references in a way that is specific to type hints.
We add a functionannotationlib.call_annotate_function as a helper for calling__annotate__ functions. This is a useful building block when implementing functionalitythat needs to partially evaluate annotations while a class is being constructed.For example, the implementation oftyping.NamedTuple needs to retrievethe annotations from a class namespace dictionary before the namedtuple class itselfcan be constructed, because the annotations determine what fields exist on the namedtuple.
A new module,annotationlib, is added to the standard library. Its aim is toprovide tooling for introspecting and wrapping annotations.
The design of the module is informed by the experience of updating the standardlibrary (e.g.,dataclasses andtyping.TypedDict) to usePEP 649 semantics.
The module will contain the following functionality:
get_annotations(): A function that returns the annotations of a function,module, or class. This will replaceinspect.get_annotations(). The latterwill delegate to the new function. It may eventually be deprecated, but tominimize disruption, we do not propose an immediate deprecation.get_annotate_from_class_namespace(namespace:Mapping[str,Any]): A function thatreturns the__annotate__ function from a class namespace dictionary, orNoneif there is none. This is useful in metaclasses during class construction. It isa separate function to avoid exposing implementation details about the internal storagefor the__annotate__ function (seebelow).Format: an enum that contains the possible formats of annotations. This willreplace theVALUE,FORWARDREF, andSOURCE formats inPEP 649.PEP 649 proposed to make these values global members of theinspectmodule; we prefer to place them within an enum. We propose to add a fourth format,VALUE_WITH_FAKE_GLOBALS (see below).ForwardRef: a class representing a forward reference; it may be returned byget_annotations() when the format isFORWARDREF. The existing classtyping.ForwardRef will become an alias of this class. Its members include:__forward_arg__: the string argument of the forward referenceevaluate(globals=None,locals=None,type_params=None,owner=None): a method that attempts to evaluatethe forward reference. TheForwardRef object may hold a reference to theglobals and other namespaces of the object that it originated from. If so, thesenamespaces may be used to evaluate the forward reference. Theowner argumentmay be the object that holds the original annotation, such as the class or moduleobject; it is used to extract the globals and locals namespaces if these are notprovided._evaluate(), with the same interface as the existingForwardRef._evaluatemethod. It will be undocumented and immediately deprecated. It is provided forcompatibility with existing users oftyping.ForwardRef.call_annotate_function(func:Callable,format:Format): a helper for callingan__annotate__ function with a given format. If the function does not supportthis format,call_annotate_function() will set up a “fake globals” environment,as described inPEP 649, and use that environment to return the desired annotationsformat.call_evaluate_function(func:Callable|None,format:Format): similar tocall_annotate_function, but does not rely on the function returning an annotationsdictionary. This is intended to be used for evaluating deferred attributes introduced byPEP 695 andPEP 696; see below for details.func may beNonefor convenience; ifNone is passed, the function also returnsNone.annotations_to_string(annotations:dict[str,object])->dict[str,str]: a function thatconverts each value in an annotations dictionary to a string representation.This is useful forimplementing theSOURCE format in cases where the original source is not available,such as in the functional syntax fortyping.TypedDict.type_repr(value:object)->str: a function that converts a single value to astring representation. This is used byannotations_to_string.It usesrepr() for most values, but for types it returns the fully qualified name.It is also useful as a helper for therepr() of a number of objects in thetyping andcollections.abc modules.A new function is also added to thetyping module,typing.evaluate_forward_ref.This function is a wrapper around theForwardRef.evaluate method, but it performsadditional work that is specific to type hints. For example, it recurses into complextypes and evaluates additional forward references within these types.
Contrary toPEP 649, the annotation formats (VALUE,FORWARDREF, andSOURCE)will not be added as global members of theinspect module. The only recommendedway to refer to these constants will be asannotationlib.Format.VALUE.
Use a different name: Naming is hard, and I considered several ideas:
annotations: The most obvious name, but it may cause confusion with the existingfrom__future__importannotations, because users may have bothimportannotationsandfrom__future__importannotations in the same module. The use of a common wordas the name will make the module harder to search for. There is a PyPI packageannotations,but it had only a single release in 2015 and looks abandoned.annotation (in the singular): Similar, but does not cause confusion with the futureimport. There is an abandoned PyPI packageannotation, but it apparently neverreleased any artifacts.annotools: Analogous toitertools andfunctools, but “anno” is a lessobvious abbreviation than “iter” or “func”. As of this writing, thereis no PyPI package with this name.annotationtools: A more explicit version. There is a PyPI packageannotationtools, which had a release in 2023.annotation_tools: A variation of the above but without a PyPI conflict. However,no other public standard library module has an underscore in its name.annotationslib: Analogous totomllib,pathlib, andimportlib.There is no PyPI package with this name.annotationlib: Similar to the above, but one character shorter and subjectively readsbetter. Also not taken on PyPI.annotationlib appears to be the best option.
Add the functionality to the inspect module: As described above, theinspect module is already quite large, and its import time is prohibitivefor some use cases.
Add the functionality to the typing module: While annotations are mostlyused for typing, they may also be used for other purposes. We prefer to keep a cleanseparation between functionality for introspecting annotations and functionality thatis exclusively meant for type hints.
Add the functionality to the types module: Thetypes module ismeant for functionality related totypes, and annotations can exist on functionsand modules, not only on types.
Develop this functionality in a third-party package: The functionality in this newmodule will be pure Python code, and it is possible to implement a third-party packagethat provides the same functionality by interacting directly with__annotate__functions generated by the interpreter. However, the functionality of the proposed newmodule will certainly be useful in the standard library itself (e.g., for implementingdataclasses andtyping.NamedTuple), so it makes sense to includeit in the standard library.
Add this functionality to a private module: It would be possible to initially developthe module in a private standard library module (e.g.,_annotations), and publicizeit only after we have gained more experience with the API. However, we already knowthat we will need parts of this module for the standard library itself (e.g., forimplementingdataclasses andtyping.NamedTuple). Even if we makeit private, the module will inevitably get used by third-party users. It is preferableto start with a clear, documented API from the beginning, to enable third-party users tosupportPEP 649 semantics as thoroughly as the standard library. The module willimmediately be used in other parts of the standard library, ensuring that it covers areasonable set of use cases.
PEP 649 specifies the following behavior of the interactive REPL:
For the sake of simplicity, in this case we forego delayed evaluation.Module-level annotations in the REPL shell will continue to work exactlyas they do with “stock semantics”, evaluating immediately and setting theresult directly inside the__annotations__dict.
There are several problems with this proposed behavior. It makes the REPL theonly context where annotations are still evaluated immediately, which isconfusing for users and complicates the language.
It also makes the implementation of the REPL more complex, as it needs toensure that all statements are compiled in “interactive” mode, even if theiroutput does not need to be displayed. (This matters if there are multiplestatements in a single line evaluated by the REPL.)
Most importantly, this breaks some plausible use cases that inexperiencedusers could run into. A user might write the following in a file:
a:X|None=NoneclassX:...
UnderPEP 649 this would work fine:X is not yet defined when it is usedin the annotation fora, but the annotation is lazily evaluated. However,if a user were to paste this same code into the REPL and execute it line byline, it would throw aNameError, because the nameX is not yet defined.
This topic was previously discussedon Discourse.
We propose to treat the interactive console like any other module-level code, andmake annotations lazily evaluated. This makes the language more consistent andavoids subtle behavior changes between modules and the REPL.
Because the REPL is evaluated line by line, we would generate a new__annotate__function for every evaluated statement in the global scope that contains annotations. Whenever a linecontaining annotations is evaluated, the previous__annotate__ function islost:
>>>x:int>>>__annotate__(1){'x': <class 'int'>}>>>y:str>>>__annotate__(1){'y': <class 'str'>}>>>z:doesntexist>>>__annotate__(1)Traceback (most recent call last):File "<python-input-5>", line 1, in <module> __annotate__(1) ~~~~~~~~~~~~^^^File "<python-input-4>", line 1, in __annotate__ z:doesntexist ^^^^^^^^^^^NameError: name 'doesntexist' is not defined
There will be no__annotations__ key in the global namespace of the REPL.In module namespaces, this key is created lazily when the__annotations__descriptor of the module object is accessed, but in the REPL there is no such moduleobject.
Classes and functions defined within the REPL will also work like any other classes,so evaluation of their annotations will be deferred. It is possible to access the__annotations__ and__annotate__ attributes or use theannotationlib moduleto introspect the annotations.
__annotations__Several objects in the standard library and elsewhere provide annotations for theirwrapped object.PEP 649 does not specify how such wrappers should behave.
Wrappers that provide annotations should be designed with the following goalsin mind:
__annotations__ should be deferred for as long as possible,consistent with the behavior of built-in functions, classes, and modules.__annotate__ and__annotations__ attributes should both be suppliedwith semantics consistent to those of the wrapped object.More specifically:
functools.update_wrapper() (and thereforefunctools.wraps())will copy only the__annotate__ attributefrom the wrapped object to the wrapper. The__annotations__ descriptor on thewrapper function will use the copied__annotate__.classmethod() andstaticmethod() currentlycopy the__annotations__ attribute from the wrapped object to the wrapper.They will instead have writable attributes for__annotate__ and__annotations__. Reading these attributes will retrievethe corresponding attribute from the underlying callable and cache it in the wrapper’s__dict__. Writing to these attributes will directly update the__dict__,without affecting the wrapped callable.Testing of the initial implementation of this PEP revealed serious problems withthe interaction between metaclasses and class annotations.
We found several bugs in the existing behavior of__annotations__ on classeswhile investigating the behaviors to be specified in this PEP. Fixing these bugson Python 3.13 and earlier is outside the scope of this PEP, but they are noted hereto explain the corner cases that need to be dealt with.
For context, on Python 3.10 through 3.13 the__annotations__ dictionary isplaced in the class namespace if the class has any annotations. If it does not,there is no__annotations__ class dictionary key when the class is created,but accessingcls.__annotations__ invokes a descriptor defined ontypethat returns an empty dictionary and stores it in the class dictionary.Static types are an exception: they never haveannotations, and accessing.__annotations__ raisesAttributeError.On Python 3.9 and earlier, the behavior was different; seegh-88067.
The following code fails identically on Python 3.10 through 3.13:
classMeta(type):passclassX(metaclass=Meta):a:strclassY(X):passMeta.__annotations__# importantassertY.__annotations__=={},Y.__annotations__# fails: {'a': <class 'str'>}
If the annotations on the metaclassMeta are accessed before the annotationsonY, then the annotations for the base classX are leaked toY.However, if the metaclass’s annotations arenot accessed (i.e., the lineMeta.__annotations__above is removed), then the annotations forY are correctly empty.
Similarly, annotations from annotated metaclasses leak to unannotatedclasses that are instances of the metaclass:
classMeta(type):a:strclassX(metaclass=Meta):passassertX.__annotations__=={},X.__annotations__# fails: {'a': <class 'str'>}
The reason for these behaviors is that if the metaclass contains an__annotations__ entry in its class dictionary, this preventsinstances of the metaclass from using the__annotations__ data descriptoron the basetype class. In the first case, accessingMeta.__annotations__setsMeta.__dict__["__annotations__"]={} as a side effect. Then, lookingup the__annotations__ attribute onY first sees the metaclass attribute,but skips it because it is a data descriptor. Next, it looks in the class dictionariesof the classes in its method resolution order (MRO), findsX.__annotations__,and returns it. In the second example, there are no annotationsanywhere in the MRO, sotype.__getattribute__ falls back toreturning the metaclass attribute.
WithPEP 649, the behavior of accessing the.__annotations__ attributeon classes when metaclasses are involved becomes even more erratic, because now__annotations__ is only lazily added to the class dictionary even for classeswith annotations. The new__annotate__ attribute is also lazily createdon classes without annotations, which causes further misbehaviors whenmetaclasses are involved.
The cause of these problems is that we set the__annotate__ and__annotations__class dictionary entries only under some circumstances, and rely on descriptorsdefined ontype to fill them in if they are not set. When normalattribute lookup is used, this approach breaks down in the presence ofmetaclasses, because entries in the metaclass’s own class dictionary can renderthe descriptors invisible.
We considered several solutions but landed on one where we store the__annotate__and__annotations__ objects in the class dictionary, but under a different,internal-only name. This means that the class dictionary entries will not interferewith the descriptors defined ontype.
This approach means that the.__annotate__ and.__annotations__ objects in classobjects will behave mostly intuitively, but there are a few downsides.
One concerns the interaction with classes defined underfrom__future__importannotations.Those will continue to have the__annotations__ entry in the class dictionary, meaningthat they will continue to display some buggy behavior. For example, if a metaclass is definedwith the__future__ import enabled and has annotations, and a class using that metaclass isdefined without the__future__ import, accessing.__annotations__ on that class will yieldthe wrong results. However, this bug already exists in previous versions of Python. It could befixed by setting the annotations at a different key in the class dict in this case too, but thatwould break users who directly access the class dictionary (e.g., during class construction).We prefer to keep the behavior under the__future__ import unchanged as much as possible.
Second, in previous versions of Python it was possible to access the__annotations__ attributeon instances of user-defined classes with annotations. However, this behavior was undocumentedand not supported byinspect.get_annotations(), and it cannot be preserved under thePEP 649 framework without bigger changes, such as a newobject.__annotations__ descriptor.This behavior change should be called out in porting guides.
The.__annotate__ and.__annotations__ attributes on class objectsshould reliably return the annotate function and the annotations dictionary,respectively, even in the presence of custom metaclasses.
Users should not access the class dictionary directly for accessing annotationsor the annotate function; the data stored in the class dictionary is an implementationdetail and its format may change in the future. If only the class namespacedictionary is available (e.g., while the class is being constructed),annotationlib.get_annotate_from_class_namespace may be used to retrieve the annotate functionfrom the class dictionary.
We considered three broad approaches for dealing with the behaviorof the__annotations__ and__annotate__ entries in classes:
type to fill in the field, andtherefore the metaclass’s attributes will not interfere. (Prototypeingh-120719.)__annotations__ and__annotate__ attributesdirectly. Instead, users should call function inannotationlib thatinvoke thetype descriptors directly. (Implemented ingh-122074.)type will always be used, without interference from the metaclass.(Initial prototype ingh-120816;later implemented ingh-132345.)Alex Waygood suggested an implementation using the first approach. When aheap type (such as a class created through theclass statement) is created,cls.__dict__["__annotations__"] is set to a special descriptor.On__get__, the descriptor evaluates the annotations by calling__annotate__and returning the result. The annotations dictionary is cached within thedescriptor instance. The descriptor also behaves like a mapping,so that code that usescls.__dict__["__annotations__"] will still usuallywork: treating the object as a mapping will evaluate the annotations and behaveas if the descriptor itself was the annotations dictionary. (Code that assumesthatcls.__dict__["__annotations__"] is specifically an instance ofdictmay break, however.)
This approach is also straightforward to implement for__annotate__: thisattribute is already always set for classes with annotations, and we can setit explicitly toNone for classes without annotations.
While this approach would fix the known edge cases with metaclasses, itintroduces significant complexity to all classes, including a new built-in type(for the annotations descriptor) with unusual behavior.
The second approach is simple to implement, but has the downside that directaccess tocls.__annotations__ remains prone to erratic behavior.
VALUE_WITH_FAKE_GLOBALS formatPEP 649 specifies:
This PEP assumes thatthird-party libraries may implement their own__annotate__methods, and those functions would almost certainly workincorrectly when run in this “fake globals” environment.For that reason, this PEP allocates a flag on code objects,one of the unused bits inco_flags, to mean “This codeobject can be run in a ‘fake globals’ environment.” Thismakes the “fake globals” environment strictly opt-in, andit’s expected that only__annotate__methods generatedby the Python compiler will set it.
However, this mechanism couples the implementation withlow-level details of the code object. The code object flags areCPython-specific and the documentationexplicitly warnsagainst relying on the values.
Larry Hastings suggested an alternative approach that does notrely on code flags: a fourth format,VALUE_WITH_FAKE_GLOBALS.Compiler-generated annotate functions would support only theVALUE andVALUE_WITH_FAKE_GLOBALS formats, both of which areimplemented identically. The standard library would use theVALUE_WITH_FAKE_GLOBALS format when invoking an annotate functionin one of the special “fake globals” environments.
This approach is useful as a forward-compatible mechanism foradding new annotation formats in the future. Users who manuallywrite annotate functions should raiseNotImplementedError iftheVALUE_WITH_FAKE_GLOBALS format is requested, so the standardlibrary will not call the manually written annotate function with“fake globals”, which could have unpredictable results.
The names of annotation formats indicate what kind of objects an__annotate__ function should return: with theSTRING format, itshould return strings; with theFORWARDREF format, it should returnforward references; and with theVALUE format, it should return values.The nameVALUE_WITH_FAKE_GLOBALS indicates that the function shouldstill return values, but is being executed in an unusual “fake globals” environment.
An additional format,VALUE_WITH_FAKE_GLOBALS, is added to theFormat enum in theannotationlib module, with value equal to 2. (As a result, the values of theother formats will shift relative to PEP 649:FORWARDREF will be 3 andSOURCEwill be 4.) The integer values of these formats are specified for use in places wherethe enum is not readily available, such as in__annotate__ functions implementedin C.
Compiler-generatedannotate functions will support this format and return the same value asthey would return for theVALUE format. The standard library will passthis format to the__annotate__ function when it is called in a “fake globals”environment, as used to implement theFORWARDREF andSOURCE formats.All public functions in theannotationlib module that accept a formatargument will raiseNotImplementedError if the format isVALUE_WITH_FAKE_GLOBALS.
Third-party code that implements__annotate__ functions should raiseNotImplementedError if theVALUE_WITH_FAKE_GLOBALS format is passedand the function is not prepared to be run in a “fake globals” environment.This should be mentioned in the data model documentation for__annotate__.
__annotations__PEP 649 specifies:
Settingo.__annotations__to a legal valueautomatically setso.__annotate__toNone.
However, the PEP does not say what happens if the__annotations__ attributeis deleted (usingdel). It seems most consistent that deleting the attributewill also delete__annotate__.
Deleting the__annotations__ attribute on functions, modules, and classesresults in setting__annotate__ to None.
SincePEP 649 was written, Python 3.12 and 3.13 gained support forseveral new features that also use deferred evaluation, similar to thebehavior this PEP proposes for annotations:
typestatement (PEP 695)typing.TypeVar objectscreated through the syntax for generics (PEP 695)typing.TypeVar,ParamSpec,andtyping.TypeVarTuple objects (PEP 696)Currently, these objects use deferred evaluation, but there is no directaccess to the function object used for deferred evaluation. To enablethe same kind of introspection that is now possible for annotations, we proposeto expose the internal function objects, allowing users to evaluate themusing the FORWARDREF and SOURCE formats.
We will add the following new attributes:
evaluate_value ontyping.TypeAliasTypeevaluate_bound,evaluate_constraints, andevaluate_default ontyping.TypeVarevaluate_default ontyping.ParamSpecevaluate_default ontyping.TypeVarTupleExcept forevaluate_value, these attributes may beNone if the objectdoes not have a bound, constraints, or default. Otherwise, the attribute is acallable, similar to an__annotate__ function, that takes a single integerargument and returns the evaluated value. Unlike__annotate__ functions,these callables return a single value, not a dictionary of annotations.These attributes are read-only.
Usually, users would use these attributes in combinations withannotationlib.call_evaluate_function. For example, to get aTypeVar’s boundin SOURCE format, one could writeannotationlib.call_evaluate_function(T.evaluate_bound,annotationlib.Format.SOURCE).
One consequence of the deferred evaluation of annotations is thatdataclasses can use forward references in their annotations:
>>>fromdataclassesimportdataclass>>>@dataclass...classD:...x:undefined...
However, theFORWARDREF format leaks into the field types of the dataclass:
>>>fields(D)[0].typeForwardRef('undefined')
We considered a change where the.type attribute of a field object wouldtrigger evaluation of annotations, so that the field type could contain actualvalues in the case of forward references that were defined after the dataclassitself was created, but before the field type is accessed.However, this would also mean that accessing.type could now run arbitrarycode in the annotation, and potentially throws errors such asNameError.
Therefore, we consider it more user-friendly to keep theForwardRef objectin the type, and document that users who want to resolve forward referencescan use theForwardRef.evaluate method.
If use cases come up in the future, we could add additional functionality,such as a new method that re-evaluates the annotation from scratch.
SOURCE toSTRINGTheSOURCE format is meant for tools that need to show a human-readableformat that is close to the original source code. However, we cannot retrievethe original source in__annotate__ functions, and in some cases, we have__annotate__ functions in Python code that do not have access to the originalcode. For example, this applies todataclasses.make_dataclass()and the call-based syntax fortyping.TypedDict.
This makes the nameSOURCE a bit of a misnomer. The goal of the formatshould indeed be to recreate the source, but the name is likely to misleadusers in practice. A more neutral name would emphasize that the format returnsan annotation dictionary with only strings. We suggestSTRING.
TheSOURCE format is renamed toSTRING. To reiterate the changes in thisPEP, the four supported formats are now:
VALUE: the default format, which evaluates the annotations and returns theresulting values.VALUE_WITH_FAKE_GLOBALS: for internal use; should be handled likeVALUEby annotate functions that support execution with fake globals.FORWARDREF: replaces undefined names withForwardRef objects.STRING: returns strings, attempts to recreate code close to the original source.PEP 649 does not support annotations that are conditionally definedin the body of a class or module:
It’s currently possible to set module and class attributes withannotations inside anifortrystatement, and it worksas one would expect. It’s untenable to support this behaviorwhen this PEP is active.
However, the maintainer of the widely used SQLAlchemy libraryreportedthat this pattern is actually common and important:
fromtypingimportTYPE_CHECKINGifTYPE_CHECKING:fromsome_moduleimportSpecialTypeclassMyClass:somevalue:strifTYPE_CHECKING:someothervalue:SpecialType
Under the behavior envisioned inPEP 649, the__annotations__ forMyClass would contain keys for bothsomevalue andsomeothervalue.
Fortunately, there is a tractable implementation strategy for makingthis code behave as expected again. This strategy relies on a few fortuitouscircumstances:
This allows the following implementation strategy:
__annotate__ function uses the set to determinewhich annotations were defined in the class or module body, and return only those.This was implemented inpython/cpython#130935.
For classes and modules, the__annotate__ function will return onlyannotations for those assignments that were executed when the class or module bodywas executed.
PEP 649 specifies that the value of the__annotations__ attributeon classes and modules is determined on first access by calling the__annotate__ function, and then it is cached for later access.This is correct in most cases and preserves compatibility, but there isone edge case where it can lead to surprising behavior: partially executedmodules.
Consider this example:
# recmod/__main__.pyfrom.importaprint("in __main__:",a.__annotations__)# recmod/a.pyv1:intfrom.importbv2:int# recmod/b.pyfrom.importaprint("in b:",a.__annotations__)
Note that whilerecmod/b.py executes, therecmod.a module is defined,but has not yet finished execution.
On 3.13, this produces:
$python3.13-mrecmodinb:{'v1':<class'int'>}in__main__:{'v1':<class'int'>,'v2':<class'int'>}
But withPEP 649 implemented as originally proposed, this wouldprint an empty dictionary twice, because the__annotate__ functionis set only when module execution is complete. This is obviouslyunintuitive.
Seepython/cpython#131550 for implementation.
Accessing__annotations__ on a partially executed module willcontinue to return the annotations that have been executed so far,similar to the behavior in earlier versions in Python. However, in thiscase the__annotations__ dictionary will not be cached, so lateraccesses to the__annotations__ attribute will return a fresh dictionary.This is necessary because__annotate__ must be called again in order toincorporate additional annotations.
PEP 649 goes into considerable detail on some aspects of the implementation.To avoid confusion, we describe a few aspects where the current implementationdiffers from that described in the PEP. However, these details are not guaranteedto hold in the future, and they may change without notice in the future, unlessthey are documented in the language reference.
ForwardRef objectsTheSOURCE format is implemented by the “stringizer” technique,where the globals dictionary of a function is augmented so that everylookup results in a special object that can be used to reconstruct theoperations that are performed on the object.
PEP 649 specifies:
In practice, the “stringizer” functionality will be implementedin theForwardRefobject currently defined in thetypingmodule.ForwardRefwill be extended toimplement all stringizer functionality; it will also beextended to support evaluating the string it contains,to produce the real value (assuming all symbols referencedare defined).
However, this is likely to lead to confusion in practice. An objectthat implements stringizer functionality must implement almost allspecial methods, including__getattr__ and__eq__, to returna new stringizer. Such an object is confusing to work with: all operationssucceed, but they are likely to return different objects than the userexpects.
The current implementation instead implements only a few useful methodson theForwardRef class. During the evaluation of annotations,an instance of a private stringizer class is used instead ofForwardRef.After evaluation completes, the implementation of the FORWARDREF formatconverts these internal objects intoForwardRef objects.
__annotate__ functionsPEP 649 specifies the signature of__annotate__ functions as:
__annotate__(format:int)->dict
However, usingformat as a parameter name could lead to collisionsif an annotation uses a symbol namedformat. To avoid this problem,the current implementation uses a positional-only parameter that is namedformat in the function signature, but that does not shadow use of the nameformat within the annotation.
PEP 649 provides a thorough discussion of the backwards compatibility implicationson existing code that uses either stock orPEP 563 semantics.
However, there is another set of compatibility problems: new code that is writtenassumingPEP 649 semantics, but uses existing tools that eagerly evaluate annotations.For example, consider adataclass-like class decorator@annotator that retrieves the annotatedfields in the class it decorates, either by accessing__annotations__ directlyor by callinginspect.get_annotations().
OncePEP 649 is implemented, code like this will work fine:
classX:y:YclassY:pass
But this will not, unless@annotator is changed to use the newFORWARDREFformat:
@annotatorclassX:y:YclassY:pass
This is not strictly a backwards compatibility issue, since no previously working codewould break; beforePEP 649, this code would have raisedNameError at runtime.In a sense, it is no different from any other new Python feature that needsto be supported by third-party libraries. Nevertheless, it is a serious issue for librariesthat perform introspection, and it is important that we make it as easy as possible forlibraries to support the new semantics in a straightforward, user-friendly way.
Several pieces of functionality in the standard library are affected by this issue,includingdataclasses,typing.TypedDict andtyping.NamedTuple.These have been updated to support this pattern using the functionality in the newannotationlib module.
One consequence ofPEP 649 is that accessing annotations on an object, even ifthe object is a function or a module, may now execute arbitrary code. This is trueeven if the STRING format is used, because the stringifier mechanism only overridesthe global namespace, and that is not enough to sandbox Python code completely.
In previous Python versions, accessing the annotations of functions or modulescould not execute arbitrary code, but classes and other objects could alreadyexecute arbitrary code on access of the__annotations__ attribute.Similarly, almost any further introspection on the annotations (e.g.,usingisinstance(), calling functions liketyping.get_origin, or evendisplaying the annotations withrepr()) could already execute arbitrary code.And of course, accessing annotations from untrusted code implies that the untrustedcode has already been imported.
The semantics ofPEP 649, as modified by this PEP, should largely be intuitive forusers who add annotations to their code. We eliminate the need for manually addingquotes around annotations that require forward references, a major source of confusionfor users.
For advanced users who need to introspect annotations, the story becomes more complex.The documentation of the newannotationlib module will serve as a reference for userswho need to interact programmatically with annotations.
The changes proposed in this PEP have been implemented on the main branchof the CPython repository.
First of all, I thank Larry Hastings for writingPEP 649. This PEP modifies some of hisinitial decisions, but the overall design is still his.
I thank Carl Meyer and Alex Waygood for feedback on early drafts of this PEP. Alex Waygood,Alyssa Coghlan, and David Ellis provided insightful feedback and suggestions on theinteraction between metaclasses and__annotations__. Larry Hastings also provided usefulfeedback on this PEP. Nikita Sobolev made various changes to the standard library that makeuse of PEP 649 functionality, and his experience helped improve the design.
PEP 649 acknowledges that the stringifier cannot handle all expressions. Now that wehave a draft implementation, we can be more precise about the expressions that can andcannot be handled. Below is a list of all expressions in the Python AST that can andcannot be recovered by the stringifier. The full list should probably not be added tothe documentation, but creating it is a useful exercise.
First, the stringifier of course cannot recover any information that is not present inthe compiled code, including comments, whitespace, parenthesization, and operations thatget simplified by the AST optimizer.
Second, the stringifier can intercept almost all operations that involve names lookedup in some scope, but it cannot intercept operations that operate fully on constants.As a corollary, this also means it is not safe to request theSOURCE format onuntrusted code: Python is powerful enough that it is possible to achieve arbitrarycode execution even with no access to any globals or builtins. For example:
>>>deff(x:(1).__class__.__base__.__subclasses__()[-1].__init__.__builtins__["print"]("Hello world")):pass...>>>annotationlib.get_annotations(f,format=annotationlib.Format.SOURCE)Hello world{'x': 'None'}
(This particular example worked for me on the current implementation of a draft of this PEP;the exact code may not keep working in the future.)
The following are supported (sometimes with caveats):
BinOpUnaryOpInvert (~),UAdd (+), andUSub (-) are supportedNot (not) is not supportedDict (except when using** unpacking)SetCompareEq andNotEq are supportedLt,LtE,Gt, andGtE are supported, but the operand may be flippedIs,IsNot,In, andNotIn are not supportedCall (except when using** unpacking)Constant (though not the exact representation of the constant; for example, escapesequences in strings are lost; hexadecimal numbers are converted to decimal)Attribute (assuming the value is not a constant)Subscript (assuming the value is not a constant)Starred (* unpacking)NameListTupleSliceThe following are unsupported, but throw an informative error when encountered by thestringifier:
FormattedValue (f-strings; error is not detected if conversion specifiers like!rare used)JoinedStr (f-strings)The following are unsupported and result in incorrect output:
BoolOp (and andor)IfExpLambdaListCompSetCompDictCompGeneratorExpThe following are disallowed in annotation scopes and therefore not relevant:
NamedExpr (:=)AwaitYieldYieldFromThis document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.
Source:https://github.com/python/peps/blob/main/peps/pep-0749.rst
Last modified:2025-10-06 14:23:25 GMT