Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork2.2k
Use thetyping-inspection
library#11479
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This commit removes most of the introspection functions definedin the `_typing_extra` module, and replaces usage with the onesfrom `typing-inspection` instead.Small improvements/changes also have been applied.
cloudflare-workers-and-pagesbot commentedFeb 23, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Deploying pydantic-docs with |
Latest commit: | 3701115 |
Status: | ✅ Deploy successful! |
Preview URL: | https://4181d96a.pydantic-docs.pages.dev |
Branch Preview URL: | https://typing-inspection.pydantic-docs.pages.dev |
codspeed-hqbot commentedFeb 23, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
CodSpeed Performance ReportMerging#11479 willnot alter performanceComparing Summary
|
4170deb
tob343465
CompareThis error will be used as a wrapper to the `ForbiddenQualifier`exception from the `typing-inspection` library, with a clear errormessage.
This simplifies the logic on our end: we don't need to manuallyunwrap the annotated forms and qualifiers. The `inspect_annotation()`is also more robust, and support more cases (as per the added test).
The typed dict schema logic is left unchanged, as the next commitwill refactor how required keys are detected, and will also addlogic related to `ReadOnly`.
Instead of trying to find `Required`/`NotRequired` qualifiers (whichcould be nested under `Annotated` forms and/or `ReadOnly`), we relyon the `FieldInfo._qualifiers` private attribute, populated fromthe `inspect_annotation()` result which is guaranteed to correctlyhandle these cases.We also now don't error if we encounter `ReadOnly`, but raise awarning stating we don't enforce immutability of the dict item.
The source is provided, and the logic to detect final fieldswith a default value is moved *after* the `FieldInfo` instanceis created.The reason to do so is to again support more edge cases, e.g.When the annotation was wrapped with `Annotated` (e.g.`Annotated[Final[int], ...] = 1`. This previously wasn't consideredand as such wasn't treated as a classvar. As per the linked issuein this PR, this led to an inconsistency, so while technicallythis can be considered as a breaking change, we'll consider this asa bug.
882f11d
toffc1f3b
CompareUh oh!
There was an error while loading.Please reload this page.
pydantic/fields.py Outdated
default.metadata += annotation_metadata | ||
default = default.merge_field_infos( | ||
*[x for x in annotation_metadata if isinstance(x, FieldInfo)], default, annotation=default.annotation | ||
# TODO infer from the default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This can wait for v3, as if the field has a default it is currently treated as a class var anyway. This todo is thus linked in#11119.
github-actionsbot commentedFeb 26, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Coverage reportClick to see where and how coverage changed
This report was generated bypython-coverage-comment-action |
f'The annotation {_repr.display_as_type(annotation)!r} contains the {self._qualifier_repr_map[qualifier]!r} ' | ||
f'type qualifier, which is invalid in the context it is defined.' | ||
), | ||
code=None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Might need a new code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Might as well 🤷
from . import _repr | ||
from ._typing_extra import is_generic_alias, is_type_alias_type | ||
from ._typing_extra import is_generic_alias |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why isis_generic_alias
not included intyping_inspection
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
is_generic_alias
does anisinstance()
check againsttyping._GenericAlias
(e.g.List[int]
is an instance of such a class), which isn't documented and technically private (although I don't think it will change). So it's best to avoid relying on it.
It is also a footgun as whileis_generic_alias()
works for all parameterized typing objects, it doesn't check for new unions (is_generic_alias(int | str) == False
, butis_generic_alias(Union[int, str]) == True
). For instance, I'm not sure we expected new unions to be skipped here:
pydantic/pydantic/_internal/_core_utils.py
Lines 66 to 74 inacb0f10
defget_type_ref(type_:Any,args_override:tuple[type[Any], ...]|None=None)->str: | |
"""Produces the ref to be used for this type by pydantic_core's core schemas. | |
This `args_override` argument was added for the purpose of creating valid recursive references | |
when creating generic models without needing to create a concrete class. | |
""" | |
origin=get_origin(type_)ortype_ | |
args=get_args(type_)ifis_generic_alias(type_)else (args_overrideor ()) |
Similarly, I've used this function here as a way to check fortype[list[int]]
forms (heretype_param
islist[int]
):
pydantic/pydantic/_internal/_generate_schema.py
Lines 1711 to 1715 inacb0f10
if_typing_extra.is_generic_alias(type_param): | |
raisePydanticUserError( | |
'Subscripting `type[]` with an already parametrized type is not supported. ' | |
f'Instead of using type[{type_param!r}], use type[{_repr.display_as_type(get_origin(type_param))}].', | |
code=None, |
This would also matchtype[Union[int, str]]
, which we actually want to support! Thankfully there's a specific check for unions just before, but this could easily be missed.
I think there are still valid use cases where you want to check if something is a generic alias (and by that I don't meanisinstance(obj, (types.GenericAlias, typing._GenericAlias)
, but if theobj
is a parameterized generic class -- excluding unions, typing special forms likeLiteral
,Annotated
, etc), but it's probably best to rely onget_origin()
and thetyping_objects
check functions.
@@ -286,7 +287,7 @@ def _warn_on_nested_alias_in_annotation(ann_type: type[Any], ann_name: str) -> N | |||
args = getattr(ann_type, '__args__', None) | |||
if args: | |||
for anno_arg in args: | |||
if_typing_extra.is_annotated(anno_arg): | |||
iftyping_objects.is_annotated(get_origin(anno_arg)): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I presume it's expected that the origin is always fetched before feeding into a typing objects check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes, as per the guide, for performance reasons and also to have a cleaner approach, you should callget_origin()
once, and the use thetyping_objects.is_smth(origin)
functions to analyze the object.
In Pydantic, ouris_smth()
functions were already callingget_origin()
, but then we can end up with patterns like:
ifis_annotated(obj):# one get_origin() call under the hood ...elifis_literal(obj):# two get_origin() calls now ...
This was convenient as shown on thisif _typing_extra.is_annotated()
call, but can decrease performance overall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes, definitely good to minimizeget_origin
calls.
@@ -1104,9 +1107,9 @@ def _match_generic_type(self, obj: Any, origin: Any) -> CoreSchema: # noqa: C90 | |||
if schema is not None: | |||
return schema | |||
if_typing_extra.is_type_alias_type(origin): | |||
iftyping_objects.is_typealiastype(origin): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Wouldis_type_alias_type
make more sense?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I wanted to avoid having the name depending on the capitalization of the object, and simply haveis_<name>
, but both make sense I think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I'd preferis_type_alias_type
just bc it's easier to read, but it's up to you, doesn't really matter!
@@ -297,15 +297,15 @@ def replace_types(type_: Any, type_map: Mapping[TypeVar, Any] | None) -> Any: | |||
origin_type = getattr(typing, type_._name) | |||
assert origin_type is not None | |||
if _typing_extra.origin_is_union(origin_type): | |||
if any(_typing_extra.is_any(arg) for arg in resolved_type_args): | |||
if is_union_origin(origin_type): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why not justis_union
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
To not confuse it withtyping_object.is_union()
, which only checks fortyping.Union
(as it does for all the other functions intyping_objects
), I've tried finding a better name but couldn't find any
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Discussed at standup today, could be changed for clarity on thetyping-inspection
end, but I don't think this PR needs to be held up because of it.
Uh oh!
There was an error while loading.Please reload this page.
core_schema_types: list[CoreSchemaOrFieldType] = _typing_extra.literal_values( | ||
CoreSchemaOrFieldType # type: ignore | ||
) | ||
core_schema_types: list[CoreSchemaOrFieldType] = list(get_literal_values(CoreSchemaOrFieldType)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why are we yielding fromget_literal_values
? Seems like we always wrap in alist
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We (Pydantic) always wrap in a list, but others might not:
foreleminget_literal_values(Literal[1,2]): ...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Doesn't strike me as necessary, but also, very low stakes, feel free to leave it as you wish
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Looking good overall - I've left some nit picky questions + sent you some general feedback abouttyping-inspection
:)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Exciting to see a lot of these common concepts pulled out intotyping-inspection
. Even more exciting to see simplifications here.
pydantic-settings relies on it.
The library is incorrectly hooking into generic aliases.
6558a2b
intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Change Summary
Needs#11475 to be merged first.
Fixes#9904,fixes#11346,fixes#11510.
Related issue number
Checklist