Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-102433: Useinspect.getattr_static intyping._ProtocolMeta.__instancecheck__#103034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

AlexWaygood
Copy link
Member

@AlexWaygoodAlexWaygood commentedMar 25, 2023•
edited
Loading

Usegetattr_static rather thanhasattr in_ProtocolMeta.__instancecheck__, in order to avoid unexpectedly evaluating descriptors and__getattr__ methods in the process of callingisinstance() against runtime-checkable protocols.

This slows downisinstance() checks against runtime-checkable protocols by around 46% (on my machine), but the consensus inpython/typing#1363 is that the performance penalty is worth it because:

  • Anyisinstance() check causing an expensive (or otherwise badly behaved) property or__getattr__ method to be evaluated is pretty surprising behaviour.
  • Bringing the runtime's understanding of runtime-checkable protocols closer to the way static type checkers understand these classes is highly desirable.
  • Runtime-checkable protocols should not be used for performance-critical code anyway (this is now clearly documented followingTyping: Document performance pitfalls of protocols decorated with@runtime_checkable #102936.)

In various subtle edge cases, this change will mean that some classes that used to be considered structural "subclasses" of a certain runtime-checkable protocol will no longer be considered subclasses of that protocol (and vice versa). As such, this can't be backported. (Perhaps we should consider documenting the existing behaviour for 3.11 and 3.10.)

Note that most discussion has actually taken place in apython/typing issue rather than a CPython issue:

@AlexWaygoodAlexWaygood changed the titlegh-102344: Useinspect.getattr_static intyping._ProtocolMeta.__instancecheck__gh-102433: Useinspect.getattr_static intyping._ProtocolMeta.__instancecheck__Mar 25, 2023
@AlexWaygoodAlexWaygood added topic-typing 3.12only security fixes stdlibPython modules in the Lib dir labelsMar 25, 2023
@AlexWaygood
Copy link
MemberAuthor

AlexWaygood commentedMar 25, 2023•
edited
Loading

I'd be very interested to know if any maintainers of libraries that use type annotations at runtime have any concerns about this proposed change. Off the top of my head, I can think of:

@gvanrossum
Copy link
Member

I’m on vacation, please find another reviewer.

AlexWaygood reacted with thumbs up emoji

@gvanrossumgvanrossum removed their request for reviewMarch 25, 2023 21:09
@samuelcolvin
Copy link
Contributor

Thanks for tagging me, I'll discuss with the pydantic team and respond in a few days.

AlexWaygood reacted with thumbs up emoji

@leycec
Copy link

Thanks so much for the apropos ping,@AlexWaygood. You do the good typing work! Sadly, this...

This slows downisinstance() checks against runtime-checkable protocols by around 46% (on my machine)...

...is a non-starter. That's measurably unacceptable degradation. The existingProtocol API works. There's no demonstrable justification for both breaking that implementation in backwards-incompatible edge casesand imposing additional runtime costs while doing so.

Python ≥ 3.11's renewed focus on runtime performance is germane here. Efficiency is no longer orthogonal to our goals in the Python ecosystem. Efficiency is increasingly a goalin and of itself. Do not sacrifice quantifiable speed at the golden altar of well-intended but ultimately futile attempts to force parity between static and runtime type-checkers.

Static and runtime type-checkers will inevitably disagree. These are profoundly different understandings of what type-checking even means. Reducing the scope of that disagreement is useful – but preserving (or ideally improving) runtime performance is substantiallymore useful.

Static type-checking serves Python's runtime needs by improving the robustness and reliability of Python at runtime. Static analysis is subordinate to runtime. Runtime always comes first. Not last. We're all here for what Python accomplishes at runtime. To paraphrase President Truman:

The buck stops at runtime.

@Tinche
Copy link
Contributor

Should be fine for cattrs, we have separate build and run stages for the de/serializers.

AlexWaygood reacted with thumbs up emoji

@AlexWaygood
Copy link
MemberAuthor

AlexWaygood commentedMar 26, 2023•
edited
Loading

Thanks for the feedback@leycec! To be clear, I fully agree with your sentiment here, and have no wish to make the CPython runtime subservient to the needs of static typing:

Static type-checking serves Python's runtime needs by improving the robustness and reliability of Python at runtime. Static analysis is subordinate to runtime. Runtime always comes first. Not last. We're all here for what Python accomplishes at runtime.

I realise that in my summary of this PR at the top, I maybe should have been clearer about the bug that this PR is trying to fix. Ultimately this PR is first and foremost about fixing unexpected behaviourat runtime. Improving consistency between how static type checkers and the runtime see certain classes is a "nice to have" that comes along with the fix.

Here is a demonstration of the user-visible bug that this PR is trying to fix (which has been reported to the CPython repo multiple times, so is clearly surprising behaviour for users). Properties, other descriptors, and__getattr__ methods are all currently "called" if you compare an object against a runtime-checkable protocol usingisinstance(). This is the issue that this PR is trying to fix.

REPL session demonstrating the bug as it exists on CPython `main`
>>>import time>>>from typingimport runtime_checkable, Protocol>>>@runtime_checkable...classHasX(Protocol):...     x:int...>>>classA:...@property...defx(self):...raiseRuntimeError('no')...>>>classB:...@property...defx(self):...         time.sleep(3600)...return42...>>>classC:...def__getattr__(self,name):...raiseRuntimeError('no')...>>>isinstance(A(), HasX)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in __instancecheck__    if all(hasattr(instance, attr) and       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in <genexpr>    if all(hasattr(instance, attr) and           ^^^^^^^^^^^^^^^^^^^^^^^  File "<stdin>", line 4, in xRuntimeError: no>>>isinstance(C(), HasX)Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in __instancecheck__    if all(hasattr(instance, attr) and       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "C:\Users\alexw\AppData\Local\Programs\Python\Python311\Lib\typing.py", line 1968, in <genexpr>    if all(hasattr(instance, attr) and           ^^^^^^^^^^^^^^^^^^^^^^^  File "<stdin>", line 3, in __getattr__RuntimeError: no>>>isinstance(B(), HasX)# this will take an hour to evaluate

I don't like (at all) that this results in a 46% performance degradation for simpleisinstance() checks against runtime-checkable protocols, and I suggested a few workarounds inpython/typing#1363 that would still fix the bug for most users of runtime-checkable protocols, while having a much smaller performance degradation (e.g.python/typing#1363 (comment)). These were criticized, however, on the grounds that:

  • they weren't particularly "principled" solutions, which might have bugs in subtle edge cases
  • they felt like premature optimizations, sinceisinstance() checks against runtime-checkable protocols are abysmally slow anyway, so users should probably not be using runtime-checkable protocols for performance-critical code.

@AlexWaygood
Copy link
MemberAuthor

AlexWaygood commentedMar 26, 2023•
edited
Loading

There are three options here that I can see:

  1. Go with this solution (take the performance hit for now, but see if we can optimise_ProtocolMeta.__instancecheck__ and/orinspect.getattr_static in followup PRs).
  2. Go with a "less principled" solution that doesn't useinspect.getattr_static, and therefore has a smaller performance degradation, but might have bugs in subtle edge cases.
  3. Declare the bug as "wontfix"; document clearly in thetyping documentation that you should be careful about comparing instances of a classX against a runtime-checkable protocol if classX has properties with side effects, which would be triggered during the course of anisinstance() check against a runtime-checkable protocol.
carljm reacted with thumbs up emoji

@agronholm
Copy link
Contributor

agronholm commentedMar 26, 2023•
edited
Loading

IMHO, correct behavior should always come first, then maybe good performance.

As far astypeguard is concerned, I don't see a problem here.

AlexWaygood and carljm reacted with thumbs up emoji

@samuelcolvin
Copy link
Contributor

I'm pretty sure this shouldn't affect pydantic since we tend to avoidisinstance() checks unless they're unavoidable, we also don't have a validator forProtocol, and if we do, I guess we would need our own logic.

I'm therefore neutral on it.

FWIW (and it's not worth much since it's probably too late), my personal opinion is that torturingisinstance to provide partial runtime type checking is a mistake - it would be better ifisinstance just checked that the first arg's type inherits from the second arg - no magic, predicable behaviour, predictably fast. Then python could provide a second method for more powerful type checking with worse performance and more magic. I would argue that the decision to contortisinstance is the route cause headaches like this, and many more.

AlexWaygood and carljm reacted with thumbs up emoji

@AlexWaygood
Copy link
MemberAuthor

FWIW (and it's not worth much since it's probably too late), my personal opinion is that torturingisinstance to provide partial runtime type checking is a mistake - it would be better ifisinstance just checked that the first arg's type inherits from the second arg - no magic, predicable behaviour, predictably fast. Then python could provide a second method for more powerful type checking with worse performance and more magic. I would argue that the decision to contortisinstance is the route cause headaches like this, and many more.

FWIW I agree fully with this, and I'm quite glad that protocols are by defaultnot runtime-checkable. But, as you say, it's almost certainly too late to change this now without going through a painful deprecation period that (in my opinion) would cause more trouble than it's worth.

@agronholm
Copy link
Contributor

I agree with you both. The checking of run-time protocols would probably be best left to run-time type checkers, but getting rid of run-time checkable protocols would be a PITA at this point.

typeguard is lazy and usesinstance() on run-time protocols, and doesn't currently provide support for static protocols. That could be changed with enough motivation and free time though.

@carljm
Copy link
Member

Note the recent comment on the issue reinforcing that the current behavior of@runtime_checkable executing arbitrary properties causes (and hides) real bugs.

There are three options here that I can see:

  1. Go with this solution (take the performance hit for now, but see if we can optimise_ProtocolMeta.__instancecheck__ and/orinspect.getattr_static in followup PRs).
  2. Go with a "less principled" solution that doesn't useinspect.getattr_static, and therefore has a smaller performance degradation, but might have bugs in subtle edge cases.
  3. Declare the bug as "wontfix"; document clearly in thetyping documentation that you should be careful about comparing instances of a classX against a runtime-checkable protocol if classX has properties with side effects, which would be triggered during the course of anisinstance() check against a runtime-checkable protocol.

I agree that these are the options.

I think the advice we would have to document in the case of (3) is basically un-followable. If you are doingisinstance(x, SomeProtocol) in your code, it's because you don't already know the type ofx, so there's no way you can "be careful" thatx isn't of a type with dangerous or costly executable properties.

I agree with@agronholm that we should prioritize correct behavior first, and then seek to optimize it. IMO correct behavior (purely from a runtime perspective, ignoring static typing) is that we should not execute arbitrary descriptor code in anisinstance check, andinspect.getattr_static is how to reliably do that. So I favor (1).

AlexWaygood reacted with thumbs up emoji

@AlexWaygood
Copy link
MemberAuthor

I'm going to merge this once the CI passes.

We've already merged two optimisation PRs intomain, which, taken together, basically make up for the performance hit that this PR gives:

There are two more PRs (which are slightly riskier) that have yet to be decided on:

If anybody has any opinions on either of those, feedback is very welcome! Once decisions have been taken on the two open PRs, I'll create another PR adding a separate NEWS entry summarising all of the performance work that's been done (and the internal changes totyping.py that were needed for the performance optimisations).

I'd also like to look into optimisinginspect.getattr_static itself, as well, since the core reason for the performance hit here is obviously thatinspect.getattr_static is very slow compared tohasattr.

@AlexWaygoodAlexWaygood merged commit6d59c9e intopython:mainApr 2, 2023
@AlexWaygoodAlexWaygood deleted the runtime-protocols-getattr_static branchApril 2, 2023 13:22
@AlexWaygood
Copy link
MemberAuthor

Many thanks to everybody who took their time to provide feedback on this PR, I really appreciate it!

@tiangolo
Copy link
Contributor

Sorry for my delay replying here. 🙈 Thanks a lot for pinging me. 🙏

As FastAPI and my other tools heavily depend and interact with Pydantic (or copy its ideas), if this doesn't affect Pydantic (as Samuel confirmed above), it doesn't affect FastAPI and friends. 🚀

AlexWaygood reacted with heart emoji

gaogaotiantian pushed a commit to gaogaotiantian/cpython that referenced this pull requestApr 8, 2023
warsaw pushed a commit to warsaw/cpython that referenced this pull requestApr 11, 2023
kraj pushed a commit to YoeDistro/poky that referenced this pull requestJun 1, 2023
Changelog:============-  Fix use of@deprecated on classes with __new__ but no __init__.-  Fix regression in version 4.6.1 where comparing a generic class against a   runtime-checkable protocol using isinstance() would cause AttributeError to   be raised if using Python 3.7.-  Change deprecated@runtime to formal API @runtime_checkable in the error message.-  Fix regression in 4.6.0 where attempting to define a Protocol that was generic   over a ParamSpec or a TypeVarTuple would cause TypeError to be raised.-  typing_extensions is now documented athttps://typing-extensions.readthedocs.io/en/latest/.-  Add typing_extensions.Buffer, a marker class for buffer types, as proposed   by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12.-  Backport two CPython PRs fixing various issues with typing.Literal:python/cpython#23294 andpython/cpython#23383. Both CPython PRs were originally,   and both were backported to Python >=3.9.1, but no earlier.-  A side effect of one of the changes is that equality comparisons of Literal   objects will now raise a TypeError if one of the Literal objects being compared   has a mutable parameter. (Using mutable parameters with Literal is not   supported by PEP 586 or by any major static type checkers.)-  Literal is now reimplemented on all Python versions <= 3.10.0.-  Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise   TypeError when the protocol is not decorated with @runtime_checkable.-  Backport several significant performance improvements to runtime-checkable protocols   that have been made in Python 3.12 (seepython/cpython#74690 for details).-  A side effect of one of the performance improvements is that the members of a   runtime-checkable protocol are now considered "frozen" at runtime as soon as the   class has been created. Monkey-patching attributes onto a runtime-checkable   protocol will still work, but will have no impact on isinstance() checks comparing   objects to the protocol. See "What's New in Python 3.12" for more details.-  isinstance() checks against runtime-checkable protocols now use inspect.getattr_static()   rather than hasattr() to lookup whether attributes exist (backportingpython/cpython#103034).-  Backport the ability to define __init__ methods on Protocol classes, a change   made in Python 3.11 (originally implemented inpython/cpython#31628-  Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12.-  Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex,   SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as   the versions from the typing module, but isinstance() checks against the   typing_extensions versions are >10x faster at runtime on Python <3.12.-  Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples.-  Add typing_extensions.get_original_bases, a backport of types.get_original_bases,   introduced in Python 3.12 (CPython PRpython/cpython#101827, originally-  This function should always produce correct results when called on classes   constructed using features from typing_extensions.-  Constructing a call-based TypedDict using keyword arguments for the fields   now causes a DeprecationWarning to be emitted. This matches the behaviour   of typing.TypedDict on 3.11 and 3.12.-  Backport the implementation of NewType from 3.10 (where it is implemented as   a class rather than a function). This allows user-defined NewTypes to be pickled.-  Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed.-  Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695.-  Backport changes to the repr of typing.Unpack that were made in order   to implement PEP 692 (backport ofpython/cpython#104048).(From OE-Core rev: 2b1d07c7deb4f0247765bc737fb11a1747143edf)Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
rpurdie pushed a commit to yoctoproject/poky that referenced this pull requestJun 2, 2023
Changelog:============-  Fix use of@deprecated on classes with __new__ but no __init__.-  Fix regression in version 4.6.1 where comparing a generic class against a   runtime-checkable protocol using isinstance() would cause AttributeError to   be raised if using Python 3.7.-  Change deprecated@runtime to formal API @runtime_checkable in the error message.-  Fix regression in 4.6.0 where attempting to define a Protocol that was generic   over a ParamSpec or a TypeVarTuple would cause TypeError to be raised.-  typing_extensions is now documented athttps://typing-extensions.readthedocs.io/en/latest/.-  Add typing_extensions.Buffer, a marker class for buffer types, as proposed   by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12.-  Backport two CPython PRs fixing various issues with typing.Literal:python/cpython#23294 andpython/cpython#23383. Both CPython PRs were originally,   and both were backported to Python >=3.9.1, but no earlier.-  A side effect of one of the changes is that equality comparisons of Literal   objects will now raise a TypeError if one of the Literal objects being compared   has a mutable parameter. (Using mutable parameters with Literal is not   supported by PEP 586 or by any major static type checkers.)-  Literal is now reimplemented on all Python versions <= 3.10.0.-  Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise   TypeError when the protocol is not decorated with @runtime_checkable.-  Backport several significant performance improvements to runtime-checkable protocols   that have been made in Python 3.12 (seepython/cpython#74690 for details).-  A side effect of one of the performance improvements is that the members of a   runtime-checkable protocol are now considered "frozen" at runtime as soon as the   class has been created. Monkey-patching attributes onto a runtime-checkable   protocol will still work, but will have no impact on isinstance() checks comparing   objects to the protocol. See "What's New in Python 3.12" for more details.-  isinstance() checks against runtime-checkable protocols now use inspect.getattr_static()   rather than hasattr() to lookup whether attributes exist (backportingpython/cpython#103034).-  Backport the ability to define __init__ methods on Protocol classes, a change   made in Python 3.11 (originally implemented inpython/cpython#31628-  Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12.-  Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex,   SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as   the versions from the typing module, but isinstance() checks against the   typing_extensions versions are >10x faster at runtime on Python <3.12.-  Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples.-  Add typing_extensions.get_original_bases, a backport of types.get_original_bases,   introduced in Python 3.12 (CPython PRpython/cpython#101827, originally-  This function should always produce correct results when called on classes   constructed using features from typing_extensions.-  Constructing a call-based TypedDict using keyword arguments for the fields   now causes a DeprecationWarning to be emitted. This matches the behaviour   of typing.TypedDict on 3.11 and 3.12.-  Backport the implementation of NewType from 3.10 (where it is implemented as   a class rather than a function). This allows user-defined NewTypes to be pickled.-  Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed.-  Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695.-  Backport changes to the repr of typing.Unpack that were made in order   to implement PEP 692 (backport ofpython/cpython#104048).(From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893)Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this pull requestJun 2, 2023
Changelog:============-  Fix use of@deprecated on classes with __new__ but no __init__.-  Fix regression in version 4.6.1 where comparing a generic class against a   runtime-checkable protocol using isinstance() would cause AttributeError to   be raised if using Python 3.7.-  Change deprecated@runtime to formal API @runtime_checkable in the error message.-  Fix regression in 4.6.0 where attempting to define a Protocol that was generic   over a ParamSpec or a TypeVarTuple would cause TypeError to be raised.-  typing_extensions is now documented athttps://typing-extensions.readthedocs.io/en/latest/.-  Add typing_extensions.Buffer, a marker class for buffer types, as proposed   by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12.-  Backport two CPython PRs fixing various issues with typing.Literal:python/cpython#23294 andpython/cpython#23383. Both CPython PRs were originally,   and both were backported to Python >=3.9.1, but no earlier.-  A side effect of one of the changes is that equality comparisons of Literal   objects will now raise a TypeError if one of the Literal objects being compared   has a mutable parameter. (Using mutable parameters with Literal is not   supported by PEP 586 or by any major static type checkers.)-  Literal is now reimplemented on all Python versions <= 3.10.0.-  Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise   TypeError when the protocol is not decorated with @runtime_checkable.-  Backport several significant performance improvements to runtime-checkable protocols   that have been made in Python 3.12 (seepython/cpython#74690 for details).-  A side effect of one of the performance improvements is that the members of a   runtime-checkable protocol are now considered "frozen" at runtime as soon as the   class has been created. Monkey-patching attributes onto a runtime-checkable   protocol will still work, but will have no impact on isinstance() checks comparing   objects to the protocol. See "What's New in Python 3.12" for more details.-  isinstance() checks against runtime-checkable protocols now use inspect.getattr_static()   rather than hasattr() to lookup whether attributes exist (backportingpython/cpython#103034).-  Backport the ability to define __init__ methods on Protocol classes, a change   made in Python 3.11 (originally implemented inpython/cpython#31628-  Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12.-  Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex,   SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as   the versions from the typing module, but isinstance() checks against the   typing_extensions versions are >10x faster at runtime on Python <3.12.-  Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples.-  Add typing_extensions.get_original_bases, a backport of types.get_original_bases,   introduced in Python 3.12 (CPython PRpython/cpython#101827, originally-  This function should always produce correct results when called on classes   constructed using features from typing_extensions.-  Constructing a call-based TypedDict using keyword arguments for the fields   now causes a DeprecationWarning to be emitted. This matches the behaviour   of typing.TypedDict on 3.11 and 3.12.-  Backport the implementation of NewType from 3.10 (where it is implemented as   a class rather than a function). This allows user-defined NewTypes to be pickled.-  Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed.-  Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695.-  Backport changes to the repr of typing.Unpack that were made in order   to implement PEP 692 (backport ofpython/cpython#104048).Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
renovatebot referenced this pull request in allenporter/flux-localJun 3, 2023
[![MendRenovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)This PR contains the following updates:| Package | Change | Age | Adoption | Passing | Confidence ||---|---|---|---|---|---|| [typing-extensions](https://togithub.com/python/typing_extensions)([changelog](https://togithub.com/python/typing_extensions/blob/main/CHANGELOG.md))| `==4.5.0` -> `==4.6.3` |[![age](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/age-slim)](https://docs.renovatebot.com/merge-confidence/)|[![adoption](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)|[![passing](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/compatibility-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/)|[![confidence](https://badges.renovateapi.com/packages/pypi/typing-extensions/4.6.3/confidence-slim/4.5.0)](https://docs.renovatebot.com/merge-confidence/)|---### Release Notes<details><summary>python/typing_extensions</summary>###[`v4.6.3`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-463-June-1-2023)[CompareSource](https://togithub.com/python/typing_extensions/compare/4.6.2...4.6.3)-   Fix a regression introduced in v4.6.0 in the implementation of    runtime-checkable protocols. The regression meantthat doing `class Foo(X, typing_extensions.Protocol)`, where `X` was aclass that    had `abc.ABCMeta` as its metaclass, would then cause subsequent    `isinstance(1, X)` calls to erroneously raise `TypeError`. Patch byAlex Waygood (backporting theCPython[https://github.com/python/cpython/pull/105152](https://togithub.com/python/cpython/pull/105152)l/105152).-   Sync the repository's LICENSE file with that of CPython.    `typing_extensions` is distributed under the same license as    CPython itself.- Skip a problematic test on Python 3.12.0b1. The test fails on 3.12.0b1due to    a bug in CPython, which will be fixed in 3.12.0b2. The    `typing_extensions` test suite now passes on 3.12.0b1.###[`v4.6.2`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-462-May-25-2023)[CompareSource](https://togithub.com/python/typing_extensions/compare/4.6.1...4.6.2)- Fix use of `@deprecated` on classes with `__new__` but no `__init__`.    Patch by Jelle Zijlstra.- Fix regression in version 4.6.1 where comparing a generic classagainst aruntime-checkable protocol using `isinstance()` would cause`AttributeError`    to be raised if using Python 3.7.###[`v4.6.1`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-461-May-23-2023)[CompareSource](https://togithub.com/python/typing_extensions/compare/4.6.0...4.6.1)- Change deprecated `@runtime` to formal API `@runtime_checkable` in theerror    message. Patch by Xuehai Pan.- Fix regression in 4.6.0 where attempting to define a `Protocol` thatwasgeneric over a `ParamSpec` or a `TypeVarTuple` would cause `TypeError`to be    raised. Patch by Alex Waygood.###[`v4.6.0`](https://togithub.com/python/typing_extensions/blob/HEAD/CHANGELOG.md#Release-460-May-22-2023)[CompareSource](https://togithub.com/python/typing_extensions/compare/4.5.0...4.6.0)-   `typing_extensions` is now documented athttps://typing-extensions.readthedocs.io/en/latest/. Patch by JelleZijlstra.- Add `typing_extensions.Buffer`, a marker class for buffer types, asproposedby PEP 688. Equivalent to `collections.abc.Buffer` in Python 3.12. Patchby    Jelle Zijlstra.- Backport two CPython PRs fixing various issues with `typing.Literal`:[https://github.com/python/cpython/pull/23294](https://togithub.com/python/cpython/pull/23294)3294[https://github.com/python/cpython/pull/23383](https://togithub.com/python/cpython/pull/23383)ll/23383.Both CPython PRs wereoriginally by Yurii Karabas, and both were backported to Python >=3.9.1,but    no earlier. Patch by Alex Waygood.A side effect of one of the changes is that equality comparisons of`Literal`objects will now raise a `TypeError` if one of the `Literal` objectsbeingcompared has a mutable parameter. (Using mutable parameters with`Literal` is    not supported by PEP 586 or by any major static type checkers.)-   `Literal` is now reimplemented on all Python versions <= 3.10.0. The`typing_extensions` version does not suffer from the bug that was fixedin[https://github.com/python/cpython/pull/29334](https://togithub.com/python/cpython/pull/29334)9334.(The CPython bugfix was    backported to CPython 3.10.1 and 3.9.8, but no earlier.)- Backport [CPython PR26067](https://togithub.com/python/cpython/pull/26067)    (originally by Yurii Karabas), ensuring that `isinstance()` calls on    protocols raise `TypeError` when the protocol is not decorated with    `@runtime_checkable`. Patch by Alex Waygood.- Backport several significant performance improvements toruntime-checkableprotocols that have been made in Python 3.12([https://github.com/python/cpython/issues/74690](https://togithub.com/python/cpython/issues/74690)es/74690for details). Patch by Alex    Waygood.A side effect of one of the performance improvements is that the membersofa runtime-checkable protocol are now considered “frozen” at runtime assoon    as the class has been created. Monkey-patching attributes onto aruntime-checkable protocol will still work, but will have no impact on    `isinstance()` checks comparing objects to the protocol. See["What's New in Python3.12"](https://docs.python.org/3.12/whatsnew/3.12.html#typing)    for more details.-   `isinstance()` checks against runtime-checkable protocols now use    `inspect.getattr_static()` rather than `hasattr()` to lookup whetherattributes exist(backport[https://github.com/python/cpython/pull/103034](https://togithub.com/python/cpython/pull/103034)3034).    This means that descriptors and `__getattr__` methods are no longerunexpectedly evaluated during `isinstance()` checks againstruntime-checkableprotocols. However, it may also mean that some objects which used to beconsidered instances of a runtime-checkable protocol on older versionsof`typing_extensions` may no longer be considered instances of thatprotocolusing the new release, and vice versa. Most users are unlikely to beaffected    by this change. Patch by Alex Waygood.- Backport the ability to define `__init__` methods on Protocol classes,achange made in Python 3.11 (originallyimplemented[https://github.com/python/cpython/pull/31628](https://togithub.com/python/cpython/pull/31628)ll/31628by Adrian Garcia Badaracco).    Patch by Alex Waygood.- Speedup `isinstance(3, typing_extensions.SupportsIndex)` by >10x onPython    <3.12. Patch by Alex Waygood.-   Add `typing_extensions` versions of `SupportsInt`, `SupportsFloat`,`SupportsComplex`, `SupportsBytes`, `SupportsAbs` and `SupportsRound`.Thesehave the same semantics as the versions from the `typing` module, but`isinstance()` checks against the `typing_extensions` versions are >10xfaster    at runtime on Python <3.12. Patch by Alex Waygood.- Add `__orig_bases__` to non-generic TypedDicts, call-based TypedDicts,andcall-based NamedTuples. Other TypedDicts and NamedTuples already had theattribute.    Patch by Adrian Garcia Badaracco.-   Add `typing_extensions.get_original_bases`, a backport of[`types.get_original_bases`](https://docs.python.org/3.12/library/types.html#types.get_original_bases),introduced in Python 3.12(CPython[https://github.com/python/cpython/pull/101827](https://togithub.com/python/cpython/pull/101827)l/101827,originally by James    Hilton-Balfe). Patch by Alex Waygood.This function should always produce correct results when called onclasses    constructed using features from `typing_extensions`. However, it mayproduce incorrect results when called on some `NamedTuple` or`TypedDict`    classes that use `typing.{NamedTuple,TypedDict}` on Python <=3.11.- Constructing a call-based `TypedDict` using keyword arguments for thefieldsnow causes a `DeprecationWarning` to be emitted. This matches thebehaviour    of `typing.TypedDict` on 3.11 and 3.12.- Backport the implementation of `NewType` from 3.10 (where it isimplementedas a class rather than a function). This allows user-defined `NewType`sto be    pickled. Patch by Alex Waygood.- Fix tests and import on Python 3.12, where `typing.TypeVar` can nolonger be    subclassed. Patch by Jelle Zijlstra.- Add `typing_extensions.TypeAliasType`, a backport of`typing.TypeAliasType`    from PEP 695. Patch by Jelle Zijlstra.- Backport changes to the repr of `typing.Unpack` that were made inorder to    implement [PEP 692](https://peps.python.org/pep-0692/) (backport of[https://github.com/python/cpython/pull/104048](https://togithub.com/python/cpython/pull/104048)4048).Patch by Alex Waygood.</details>---### Configuration📅 **Schedule**: Branch creation - At any time (no schedule defined),Automerge - At any time (no schedule defined).🚦 **Automerge**: Enabled.♻ **Rebasing**: Whenever PR becomes conflicted, or you tick therebase/retry checkbox.🔕 **Ignore**: Close this PR and you won't be reminded about this updateagain.---- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, checkthis box---This PR has been generated by [MendRenovate](https://www.mend.io/free-developer-tools/renovate/). Viewrepository job log[here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS4xMDUuMiIsInVwZGF0ZWRJblZlciI6IjM1LjEwNS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9-->Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
pmhahn pushed a commit to univention/univention-corporate-server that referenced this pull requestJun 10, 2023
It was caused by the new [4.6 version of typing-extensions](https://github.com/python/typing_extensions/blob/main/CHANGELOG.md#release-460-may-22-2023) used by jsonschema-spec to validate protocol.>    isinstance() checks against runtime-checkable protocols now use inspect.getattr_static() rather than hasattr() to lookup whether attributes exist (backportingpython/cpython#103034). This means that descriptors and getattr methods are no longer unexpectedly evaluated during isinstance() checks against runtime-checkable protocols. However, it may also mean that some objects which used to be considered instances of a runtime-checkable protocol on older versions of typing_extensions may no longer be considered instances of that protocol using the new release, and vice versa. Most users are unlikely to be affected by this change. Patch by Alex Waygood.Temporary workaround will be downgrading typing-extensions to version below 4.6:    pip install "typing-extensions<4.6"Bug #50048
daregit pushed a commit to daregit/yocto-combined that referenced this pull requestMay 22, 2024
Changelog:============-  Fix use of@deprecated on classes with __new__ but no __init__.-  Fix regression in version 4.6.1 where comparing a generic class against a   runtime-checkable protocol using isinstance() would cause AttributeError to   be raised if using Python 3.7.-  Change deprecated@runtime to formal API @runtime_checkable in the error message.-  Fix regression in 4.6.0 where attempting to define a Protocol that was generic   over a ParamSpec or a TypeVarTuple would cause TypeError to be raised.-  typing_extensions is now documented athttps://typing-extensions.readthedocs.io/en/latest/.-  Add typing_extensions.Buffer, a marker class for buffer types, as proposed   by PEP 688. Equivalent to collections.abc.Buffer in Python 3.12.-  Backport two CPython PRs fixing various issues with typing.Literal:python/cpython#23294 andpython/cpython#23383. Both CPython PRs were originally,   and both were backported to Python >=3.9.1, but no earlier.-  A side effect of one of the changes is that equality comparisons of Literal   objects will now raise a TypeError if one of the Literal objects being compared   has a mutable parameter. (Using mutable parameters with Literal is not   supported by PEP 586 or by any major static type checkers.)-  Literal is now reimplemented on all Python versions <= 3.10.0.-  Backport CPython PR 26067, ensuring that isinstance() calls on protocols raise   TypeError when the protocol is not decorated with @runtime_checkable.-  Backport several significant performance improvements to runtime-checkable protocols   that have been made in Python 3.12 (seepython/cpython#74690 for details).-  A side effect of one of the performance improvements is that the members of a   runtime-checkable protocol are now considered "frozen" at runtime as soon as the   class has been created. Monkey-patching attributes onto a runtime-checkable   protocol will still work, but will have no impact on isinstance() checks comparing   objects to the protocol. See "What's New in Python 3.12" for more details.-  isinstance() checks against runtime-checkable protocols now use inspect.getattr_static()   rather than hasattr() to lookup whether attributes exist (backportingpython/cpython#103034).-  Backport the ability to define __init__ methods on Protocol classes, a change   made in Python 3.11 (originally implemented inpython/cpython#31628-  Speedup isinstance(3, typing_extensions.SupportsIndex) by >10x on Python <3.12.-  Add typing_extensions versions of SupportsInt, SupportsFloat, SupportsComplex,   SupportsBytes, SupportsAbs and SupportsRound. These have the same semantics as   the versions from the typing module, but isinstance() checks against the   typing_extensions versions are >10x faster at runtime on Python <3.12.-  Add __orig_bases__ to non-generic TypedDicts, call-based TypedDicts, and call-based NamedTuples.-  Add typing_extensions.get_original_bases, a backport of types.get_original_bases,   introduced in Python 3.12 (CPython PRpython/cpython#101827, originally-  This function should always produce correct results when called on classes   constructed using features from typing_extensions.-  Constructing a call-based TypedDict using keyword arguments for the fields   now causes a DeprecationWarning to be emitted. This matches the behaviour   of typing.TypedDict on 3.11 and 3.12.-  Backport the implementation of NewType from 3.10 (where it is implemented as   a class rather than a function). This allows user-defined NewTypes to be pickled.-  Fix tests and import on Python 3.12, where typing.TypeVar can no longer be subclassed.-  Add typing_extensions.TypeAliasType, a backport of typing.TypeAliasType from PEP 695.-  Backport changes to the repr of typing.Unpack that were made in order   to implement PEP 692 (backport ofpython/cpython#104048).(From OE-Core rev: a37154b9166323d05cca970ebb37bee0d5250893)Signed-off-by: Wang Mingyu <wangmy@fujitsu.com>Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
dangotbanned added a commit to dangotbanned/altair that referenced this pull requestMay 31, 2024
…python>=3.12.0`Any objects using `__getattr__`, e.g. `Parameter` break on older versions [see PR](python/cpython#103034)
@GDYendell
Copy link

GDYendell commentedNov 20, 2024•
edited
Loading

inspect.getattr_staticusesinspect._check_instance whichusesobject.__getattribute__(obj, "__dict__"). This fails for bound instance methods (althoughobj.__dict__ does work), so they cannot fulfillruntime_checkableProtocols anymore.

from typing import Protocol, runtime_checkable@runtime_checkableclass Labelled(Protocol):    label: strclass A:    def f(self):        passA.f.label = "f"a = A()print(a.f.__dict__)  # {'label': 'f'}print(object.__getattribute__(a.f, "__dict__"))  # AttributeErrorassert isinstance(a.f, Labelled)  # AssertionError

Is this an intended/known change? Could it be noted in the changeloghere as an example of case that will no longer work?

@AlexWaygood
Copy link
MemberAuthor

@GDYendell, I will admit that I did indeed fail to consider how this would change the behaviour of runtime-checkable protocols when used in conjunction with bound instance methods that have extra attributes monkeypatched onto them.

Can you possibly open a new issue about this? I haven't got the time to look at it right now, and comment threads on long-merged PRs are unfortunately likely to get forgotten about.

GDYendell reacted with thumbs up emoji

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@carljmcarljmcarljm approved these changes

@JelleZijlstraJelleZijlstraJelleZijlstra approved these changes

@Fidget-SpinnerFidget-SpinnerAwaiting requested review from Fidget-Spinner

Assignees
No one assigned
Labels
3.12only security fixesstdlibPython modules in the Lib dirtopic-typing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

11 participants
@AlexWaygood@gvanrossum@samuelcolvin@leycec@Tinche@agronholm@carljm@tiangolo@GDYendell@JelleZijlstra@bedevere-bot
[8]ページ先頭
©2009-2025 Movatter.jp