Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Fixisinstance behavior for urls#10766

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
sydney-runkle merged 11 commits intomainfromurl-fix-i-think
Nov 5, 2024
Merged

Fixisinstance behavior for urls#10766

sydney-runkle merged 11 commits intomainfromurl-fix-i-think
Nov 5, 2024

Conversation

sydney-runkle
Copy link
Contributor

@sydney-runklesydney-runkle commentedNov 4, 2024
edited
Loading

This new implementation sort of starts to follow the pattern we'd use forRustModel like classes by accessing thepydantic-core structure from a wrapperpydantic type.

With this new pattern, isinstance checks work properly (they don't in the most recent version ofpydantic):

frompydanticimportHttpUrl,TypeAdapterhttp_url=TypeAdapter(HttpUrl).validate_python('http://example.com/something')print(http_url)#> http://example.com/somethingprint(repr(http_url))#> HttpUrl(http://example.com/something)print(http_url.__class__)#> <class 'pydantic.networks.HttpUrl'>print(http_url.host)#> example.com

I think some more refactoring should be done inpydantic-core, but this serves as thepydantic end of things, for the most part. In core (future PR), the main thing we can do is simplify theUrl andMultiHostUrl classes (remove docstrings, etc).

Eventually, we should get rid of validation on init, that's pretty gnarly - but that would constitute a breaking change. Should we do this now?

@github-actionsgithub-actionsbot added the relnotes-fixUsed for bugfixes. labelNov 4, 2024
@cloudflare-workers-and-pagesCloudflare Workers and Pages
Copy link

cloudflare-workers-and-pagesbot commentedNov 4, 2024
edited
Loading

Deploying pydantic-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit:b7c8485
Status: ✅  Deploy successful!
Preview URL:https://19f0349f.pydantic-docs.pages.dev
Branch Preview URL:https://url-fix-i-think.pydantic-docs.pages.dev

View logs

@sydney-runkle
Copy link
ContributorAuthor

sydney-runkle commentedNov 4, 2024
edited
Loading

The one testing issue that I'm running into currently is that nowHttpUrl, etc aren't direct subclasses ofUrl, so isinstance checks are failing validation...

Maybe, this can be fixed with a union against an isinstance schema, but I hate using unions for internally defined schemas.
We could also subclass fromUrl andMultiHostUrl, but that seems quite messy?

@codspeed-hqCodSpeed HQ
Copy link

codspeed-hqbot commentedNov 4, 2024
edited
Loading

CodSpeed Performance Report

Merging#10766 willnot alter performance

Comparingurl-fix-i-think (b7c8485) withmain (678ec30)

Summary

✅ 44 untouched benchmarks

@github-actionsGitHub Actions
Copy link
Contributor

github-actionsbot commentedNov 4, 2024
edited
Loading

Coverage report

This PR does not seem to contain any modification to coverable code.

@sydney-runkle
Copy link
ContributorAuthor

I've fixed the above concerns with a wrap validator, though this admittedly slows validation significantly (see regression above).

Copy link
Contributor

@davidhewittdavidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

To summarise:

  • We've made the decision to move to a Python object hierarchy, because this will produce results which match more what users expect
  • We keep the Rust state as an internal detail
  • We use a wrap validator so that core can handle these types appropriately

I feel that one day we might be able to do away with this Python wrapper if we wanted to squeeze every last drop of performance, but I also am happy with this 👍

We should be wary and expect there may be subtle breakages caused by this, and hence call this out in update notes.

sydney-runkle reacted with heart emoji
Copy link
Member

@samuelcolvinsamuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think constraints have to be enforced by__init__.

My suggestion would be a lazy type adapter (viafunctools.cache) which is called in__init__.

Then in core the functional validator, we do:

core_url=handler(input)instance=source.__new__(source)instance._url=core_url

@sydney-runkle
Copy link
ContributorAuthor

My suggestion would be a lazy type adapter (via functools.cache) which is called ininit.

Does this mean we can rip out validation logic on__init__ inpydantic-core?@samuelcolvin@davidhewitt

@sydney-runkle
Copy link
ContributorAuthor

Happy to fix validation on init in this pr, hopefully that moves this into the "don't need to deal with for a while" state.

@sydney-runkle
Copy link
ContributorAuthor

We might be able to pull this off with just an after validator, which could be good...

@davidhewitt
Copy link
Contributor

My suggestion would be a lazy type adapter (via functools.cache) which is called ininit.

Does this mean we can rip out validation logic on__init__ inpydantic-core?@samuelcolvin@davidhewitt

Probably not, because it currently needs to assume valid state internally. Maybe in the future, but I wouldn't rush to do it.

samuelcolvin and sydney-runkle reacted with thumbs up emoji

@samuelcolvin
Copy link
Member

I agree with@davidhewitt, and I don't see any particular advantage of ripping out that logic right now.

@sydney-runkle
Copy link
ContributorAuthor

Probably not, because it currently needs to assume valid state internally. Maybe in the future, but I wouldn't rush to do it.

I think this effectively results in double validation for some cases (see perf degradation above), but I'm ok with this as an intermediate state for now.

@samuelcolvin
Copy link
Member

we should definitely avoid double validation, I think that's doable.

sydney-runkle reacted with thumbs up emoji

@sydney-runkle
Copy link
ContributorAuthor

Here's where I'm having trouble with double validation:

frompydanticimportTypeAdapter,HttpUrlta=TypeAdapter(HttpUrl)http_url=HttpUrl('http://example.com/something')assertstr(http_url)=='http://example.com/something'assertrepr(http_url)=="HttpUrl('http://example.com/something')"asserthttp_url.__class__==HttpUrlasserthttp_url.host=='example.com'asserthttp_url.path=='/something'asserthttp_url.usernameisNoneasserthttp_url.passwordisNonehttp_url_validated=ta.validate_python(http_url)"""pydantic_core._pydantic_core.ValidationError: 1 validation error for function-wrap[wrap_val()]  URL input should be a string or URL [type=url_type, input_value=HttpUrl('http://example.com/something'), input_type=HttpUrl]    For further information visit https://errors.pydantic.dev/2.10/v/url_type"""

So, I think we need to have some sort of isinstnace check in the wrap validator. This is unfortunate though, as we'll no longer respectconfig.revalidate_instances...

I think this is where having acls passed topydantic-core'surl_schema could be helpful to help with this isinstance check that's failing, but I'd really prefer to keep that out of the scope of this fix, and make that change in v2.11. We already tried that once, and things went awry with duplicated validation.


@cached_property
def _validator(self) -> TypeAdapter:
return TypeAdapter(self.__class__)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It feels like it could be both simpler and more efficient if here thisTypeAdapter just built the underlying rustUrl, with appropriate constraints? Is there a reason why that might be problematic for subclasses? 🤔

Copy link
ContributorAuthor

@sydney-runklesydney-runkleNov 5, 2024
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hmm, I don't think there's an easy way to do this - you can't easily create aTypeAdapter with just a core schema. Even if we could hack that together, should we?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

well we could use aSchemaValidator instead of aTypeAdapter?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Found a relatively simple way withTypeAdapter. That's easiest for now, as we already have theisinstance conditional in the wrap validator in the custom core schema.

Down the line, maybe it does make sense to just useSchemaValidator inside these types to perform internal validation. I'm going to write up an issue with next steps based on the feedback here, and I'll include this 👍

Copy link
Member

@samuelcolvinsamuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Great work, LGTM.

@sydney-runkle
Copy link
ContributorAuthor

Thanks for the abundance of iterations here@samuelcolvin@davidhewitt.

I think we're moving in the right direction here, next steps include improving performance and optimizing / standardizing how we validate types like this on__init__.

@sydney-runklesydney-runkleenabled auto-merge (squash)November 5, 2024 17:47
@sydney-runklesydney-runkle merged commitfba836a intomainNov 5, 2024
52 checks passed
@sydney-runklesydney-runkle deleted the url-fix-i-think branchNovember 5, 2024 17:48
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@davidhewittdavidhewittdavidhewitt approved these changes

@samuelcolvinsamuelcolvinsamuelcolvin approved these changes

Assignees
No one assigned
Labels
relnotes-fixUsed for bugfixes.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@sydney-runkle@davidhewitt@samuelcolvin

[8]ページ先頭

©2009-2025 Movatter.jp