Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Only evaluateFieldInfo annotations if required during schema building#10769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
sydney-runkle merged 2 commits intomainfromfieldinfo-evaluated
Nov 14, 2024

Conversation

Viicos
Copy link
Member

By adding a newevaluated attribute that can be set during model fields collection. Also refactor a bit the utility functions to eval such annotations.

Change Summary

Related issue number

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable
  • My PR is ready to review,please add a comment including the phrase "please review" to assign reviewers

@github-actionsgithub-actionsbot added the relnotes-fixUsed for bugfixes. labelNov 5, 2024
@@ -1215,11 +1215,15 @@ def _common_field_schema( # C901
) -> _CommonField:
# Update FieldInfo annotation if appropriate:
FieldInfo = import_cached_field_info()
if has_instance_in_type(field_info.annotation, (ForwardRef, str)):
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The interesting part is that we remove the call tohas_instance_in_type here.

sydney-runkle reacted with thumbs up emoji
@cloudflare-workers-and-pagesCloudflare Workers and Pages
Copy link

cloudflare-workers-and-pagesbot commentedNov 5, 2024
edited
Loading

Deploying pydantic-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit:076da3c
Status: ✅  Deploy successful!
Preview URL:https://bf7a63be.pydantic-docs.pages.dev
Branch Preview URL:https://fieldinfo-evaluated.pydantic-docs.pages.dev

View logs

@@ -1338,12 +1342,13 @@ def _type_alias_type_schema(self, obj: TypeAliasType) -> CoreSchema:
return maybe_schema

origin: TypeAliasType = get_origin(obj) or obj

annotation = origin.__value__
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Moved below, as accessing the__value__ attribute will trigger the evaluation of the annotation, and thus this can raise aNameError. Theeval_type call is still necessary though, as you can have explicit string references inside a a type alias type.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe I'm misunderstanding - was this moved below? Or do we not need this specific line anymore?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Basically, here is what can happen with type aliases:

typeA=Intannotation=origin.__value__# NameError, 'Int' is not definedtypeB='Int'annotation=origin.__value__assertannotation=='Int'eval_type(B, ..., ...)# NameError, 'Int' is not defined

So everything was moved in a singletry..except block:

try:annotation=_typing_extra.eval_type(origin.__value__,*self._types_namespace)except ...: ...

sydney-runkle reacted with thumbs up emoji
@codspeed-hqCodSpeed HQ
Copy link

codspeed-hqbot commentedNov 5, 2024
edited
Loading

CodSpeed Performance Report

Merging#10769 willimprove performances by 8.54%

Comparingfieldinfo-evaluated (076da3c) withmain (6e91707)

Summary

⚡ 5 improvements
✅ 39 untouched benchmarks

Benchmarks breakdown

Benchmarkmainfieldinfo-evaluatedChange
test_complex_model_schema_generation2.8 ms2.6 ms+6.33%
test_lots_of_models_with_lots_of_fields3.6 s3.3 s+8.54%
test_simple_model_schema_lots_of_fields_generation41.8 ms38.8 ms+7.73%
test_nested_recursive_generic_model_schema_generation2.3 ms2.2 ms+5.79%
test_nested_recursive_model_schema_generation2.3 ms2.2 ms+6.21%

@ViicosViicosforce-pushed thefieldinfo-evaluated branch 2 times, most recently from367f48a to88aa322CompareNovember 5, 2024 17:06
@github-actionsGitHub Actions
Copy link
Contributor

github-actionsbot commentedNov 5, 2024
edited
Loading

Coverage report

This PR does not seem to contain any modification to coverable code.

Copy link
Contributor

@sydney-runklesydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Alrighty, left some initial comments on the implementation.

The tuple return doesn't feel like the cleanest implementation, but I can't come up with anything better off the top of my head. Let's chat first thing tomorrow morning about alternatives. I'm not opposed to this approach, but want to think about other options before we move forward 👍

Comment on lines -283 to +559
'`eval_type_lenient` is deprecated, use `eval_type` with `lenient=True` instead.',
'`eval_type_lenient` is deprecated, use `try_eval_type` instead.',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hmm, this feels like an anti-pattern - if we already deprecated this, there's probably a reason why we didn't want to have two functions. Even if we disagree with this deprecation now, seems like a lot of churn to basically reintroduce the function and redirect for the deprecated case...

I believe my suggestion above re always returning a second value,evaluated, even in the case ofeval_type withlenient=False might help, and we can leave this be?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

A few more follow ups:

  1. This is a very internal function - can we just deprecate this?
  2. If we were to bring back a second function, to reduce churn, can we call iteval_type_lenient?

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I had to keep this function during the types namespaces refactor because fastapi makes use of it 🫤. I added thedeprecated decorator so that type checking fails in the fastapi CI and they can use the correct function (or best, do not rely on our internals because I don't think the_typing_extra module is stable enough, we might apply other changes here in the future).

I believe my suggestion above re always returning a second value,evaluated, even in the case ofeval_type withlenient=False might help, and we can leave this be?

Same issue ashttps://github.com/pydantic/pydantic/pull/10769/files#r1832947088, we can't have a single function because we want to catch theNameError to reraisePydanticUndefinedAnnotation

sydney-runkle reacted with thumbs up emoji
@@ -1338,12 +1342,13 @@ def _type_alias_type_schema(self, obj: TypeAliasType) -> CoreSchema:
return maybe_schema

origin: TypeAliasType = get_origin(obj) or obj

annotation = origin.__value__
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe I'm misunderstanding - was this moved below? Or do we not need this specific line anymore?

@sydney-runkle
Copy link
Contributor

Screenshot 2024-11-07 at 6 23 39 AM

Wow@Viicos, nice work here. Didn't realize this would have that significant of an impact.

We can chat about potential other APIs for this, but we should definitely move forward with this change :).

Does this unblock the_typing_extra.py refactor PR?

@sydney-runkle
Copy link
Contributor

Screenshot 2024-11-07 at 6 25 04 AM

This is great. I presume next we'll look at_apply_annotations, and we know there's a lot to gain there.

@sydney-runklesydney-runkle added topic-performance and removed relnotes-fixUsed for bugfixes. labelsNov 7, 2024
@sydney-runkle
Copy link
Contributor

sydney-runkle commentedNov 7, 2024
edited
Loading

Another follow up - what if we returned something like this:

frompythonimportTypedDictclassTypeEvaluationResult(TypedDict):evaluated:boolresult:Anyerror:NameError|None

Or something like:

frompythonimportTypedDict,TypeAliasclassTypeEvaluationSuccess(TypedDict):evaluated:Literal[True]result:AnyclassTypeEvaluationFailure(TypedDict):evaluated:Literal[False]error:NameErrortypeTypeEvaluationResult=TypeEvaluationSuccess|TypeEvaluationFailure

Doesn't have to be either of those exactly, but I think you get the point :). Then, we could have one function and preserve relevant errors, but also intuitively handle attempted evals?

Maybe this isn't better, but I think it does help us avoid the overloads, though admittedly there's more conditional logic required after a function call, as the result can be many things.

Could even do a named tuple.

@sydney-runklesydney-runkle mentioned this pull requestNov 7, 2024
5 tasks
@Viicos
Copy link
MemberAuthor

Seems like both the overload and structured return value have different drawbacks: what isresult if it failed to evaluate? How do you safely make use ofresult, even if you checked thaterror is None before ? (as type checkers will complain).

Maybe it's worth revisiting the API, if we want to make this module more stable and maybe expose it publicly, as it is used already by some other projects.

sydney-runkle reacted with thumbs up emoji

@sydney-runkle
Copy link
Contributor

Yeah, fair point. Let's not go with the structured return.

One other idea - could we revert to that single signature idea, and manufacture aNameError ifevaluated isFalse based on the input?

@Viicos
Copy link
MemberAuthor

You mean something likedict[str, tuple[Any, bool | NameError]] in the return type?

@sydney-runkle
Copy link
Contributor

Or even likedict[str, tuple[Any, bool], but when the second value isfalse and we want to raise on failure, just thenraise PydanticUndefinedAnnotation(<information about the failed eval similar to what the NameError would have provided>)

@Viicos
Copy link
MemberAuthor

but the issue is we currently raise usingPydanticUndefinedAnnotation.from_name_error so that's why we need the exception to be raised originally.

dict[str, tuple[Any, bool | NameError]] could work but just moves the burden of checking forbool (actuallyLiteral[True]) andNameError to the caller.

sydney-runkle reacted with thumbs up emoji

Copy link
Contributor

@sydney-runklesydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looking over this again, seems reasonable, given the constraints we've discussed re typing and overloads.

@sydney-runkle
Copy link
Contributor

Going to go ahead and merge. We can cherry pick relevant stuff for the v2.10 official release (not including this).

@sydney-runklesydney-runkle added relnotes-performanceUsed for performance improvements. and removed topic-performance labelsNov 14, 2024
@sydney-runklesydney-runkleenabled auto-merge (squash)November 14, 2024 16:07
By adding a new `evaluated` attribute that can be set during modelfields collection. Also refactor a bit the utility functions to evalsuch annotations.
@sydney-runklesydney-runkle merged commitbcfd413 intomainNov 14, 2024
52 checks passed
@sydney-runklesydney-runkle deleted the fieldinfo-evaluated branchNovember 14, 2024 16:53
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@sydney-runklesydney-runklesydney-runkle approved these changes

Assignees
No one assigned
Labels
relnotes-performanceUsed for performance improvements.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@Viicos@sydney-runkle

[8]ページ先頭

©2009-2025 Movatter.jp