Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork2.2k
Fixisinstance
behavior for urls#10766
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Uh oh!
There was an error while loading.Please reload this page.
cloudflare-workers-and-pagesbot commentedNov 4, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Deploying pydantic-docs with |
Latest commit: | b7c8485 |
Status: | ✅ Deploy successful! |
Preview URL: | https://19f0349f.pydantic-docs.pages.dev |
Branch Preview URL: | https://url-fix-i-think.pydantic-docs.pages.dev |
sydney-runkle commentedNov 4, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
The one testing issue that I'm running into currently is that now Maybe, this can be fixed with a union against an isinstance schema, but I hate using unions for internally defined schemas. |
codspeed-hqbot commentedNov 4, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
CodSpeed Performance ReportMerging#10766 willnot alter performanceComparing Summary
|
github-actionsbot commentedNov 4, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I've fixed the above concerns with a wrap validator, though this admittedly slows validation significantly (see regression above). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
To summarise:
- We've made the decision to move to a Python object hierarchy, because this will produce results which match more what users expect
- We keep the Rust state as an internal detail
- We use a wrap validator so that core can handle these types appropriately
I feel that one day we might be able to do away with this Python wrapper if we wanted to squeeze every last drop of performance, but I also am happy with this 👍
We should be wary and expect there may be subtle breakages caused by this, and hence call this out in update notes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think constraints have to be enforced by__init__
.
My suggestion would be a lazy type adapter (viafunctools.cache
) which is called in__init__
.
Then in core the functional validator, we do:
core_url=handler(input)instance=source.__new__(source)instance._url=core_url
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Does this mean we can rip out validation logic on |
Happy to fix validation on init in this pr, hopefully that moves this into the "don't need to deal with for a while" state. |
We might be able to pull this off with just an after validator, which could be good... |
Probably not, because it currently needs to assume valid state internally. Maybe in the future, but I wouldn't rush to do it. |
I agree with@davidhewitt, and I don't see any particular advantage of ripping out that logic right now. |
I think this effectively results in double validation for some cases (see perf degradation above), but I'm ok with this as an intermediate state for now. |
we should definitely avoid double validation, I think that's doable. |
Here's where I'm having trouble with double validation: frompydanticimportTypeAdapter,HttpUrlta=TypeAdapter(HttpUrl)http_url=HttpUrl('http://example.com/something')assertstr(http_url)=='http://example.com/something'assertrepr(http_url)=="HttpUrl('http://example.com/something')"asserthttp_url.__class__==HttpUrlasserthttp_url.host=='example.com'asserthttp_url.path=='/something'asserthttp_url.usernameisNoneasserthttp_url.passwordisNonehttp_url_validated=ta.validate_python(http_url)"""pydantic_core._pydantic_core.ValidationError: 1 validation error for function-wrap[wrap_val()] URL input should be a string or URL [type=url_type, input_value=HttpUrl('http://example.com/something'), input_type=HttpUrl] For further information visit https://errors.pydantic.dev/2.10/v/url_type""" So, I think we need to have some sort of isinstnace check in the wrap validator. This is unfortunate though, as we'll no longer respect I think this is where having a |
pydantic/networks.py Outdated
@cached_property | ||
def _validator(self) -> TypeAdapter: | ||
return TypeAdapter(self.__class__) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It feels like it could be both simpler and more efficient if here thisTypeAdapter
just built the underlying rustUrl
, with appropriate constraints? Is there a reason why that might be problematic for subclasses? 🤔
sydney-runkleNov 5, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hmm, I don't think there's an easy way to do this - you can't easily create aTypeAdapter
with just a core schema. Even if we could hack that together, should we?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
well we could use aSchemaValidator
instead of aTypeAdapter
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Found a relatively simple way withTypeAdapter
. That's easiest for now, as we already have theisinstance
conditional in the wrap validator in the custom core schema.
Down the line, maybe it does make sense to just useSchemaValidator
inside these types to perform internal validation. I'm going to write up an issue with next steps based on the feedback here, and I'll include this 👍
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Great work, LGTM.
Thanks for the abundance of iterations here@samuelcolvin@davidhewitt. I think we're moving in the right direction here, next steps include improving performance and optimizing / standardizing how we validate types like this on |
fba836a
intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
This new implementation sort of starts to follow the pattern we'd use for
RustModel
like classes by accessing thepydantic-core
structure from a wrapperpydantic
type.With this new pattern, isinstance checks work properly (they don't in the most recent version of
pydantic
):I think some more refactoring should be done in
pydantic-core
, but this serves as thepydantic
end of things, for the most part. In core (future PR), the main thing we can do is simplify theUrl
andMultiHostUrl
classes (remove docstrings, etc).Eventually, we should get rid of validation on init, that's pretty gnarly - but that would constitute a breaking change. Should we do this now?