- Notifications
You must be signed in to change notification settings - Fork293
Do not call default factories taking the data argument if a validation error already occurred#1623
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Same needs to be done for DCs/TDs
CodSpeed Performance ReportMerging#1623 willnot alter performanceComparing Summary
|
…n error already occurred
a47bed9
toacce26b
Compareif matches!(self.default, DefaultType::DefaultFactory(_, true)) && state.has_field_error { | ||
// The default factory might use data from fields that failed to validate, and this results | ||
// in an unhelpul error. | ||
return Err(ValError::new(ErrorTypeDefaults::DefaultFactoryNotCalled, input)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It's a bit annoying because I don't have access to any input inside thedefault_value
method.@davidhewitt any idea how I could workaround this in a clean way? I could wrap up the return type in an enum of eitherOption<PyObject>
or a new singleton sentinel value, but the rtype is already complex enough 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Actually, the current behavior is a bit weird:
classModel(BaseModel):a:intb:intModel(b="fails")"""pydantic_core._pydantic_core.ValidationError: 2 validation errors for Modela Field required [type=missing, input_value={'b': 'fails'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.11/v/missingb Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='fails', input_type=str] For further information visit https://errors.pydantic.dev/2.11/v/int_parsing"""
Forb
, a value was provided and is used asinput_value
. Fora
, no value is provided and pydantic-core uses the whole input dict asinput_value
(on L214,input
is the provided dict):
pydantic-core/src/validators/model_fields.rs
Lines 205 to 217 in164b9ff
match field.validator.default_value(py,Some(field.name.as_str()), state){ | |
Ok(Some(value)) =>{ | |
// Default value exists, and passed validation if required | |
model_dict.set_item(&field.name_py, value)?; | |
} | |
Ok(None) =>{ | |
// This means there was no default value | |
errors.push(field.lookup_key.error( | |
ErrorTypeDefaults::Missing, | |
input, | |
self.loc_by_alias, | |
&field.name, | |
)); |
So perhaps we could just usePydanticUndefined
as an input value here, and then in the with_default validator, I can return:
Err(ValError::new(ErrorTypeDefaults::DefaultFactoryNotCalled,&self.undefined))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hmm interesting, I wonder if the error fora
should not includeinput_value
orinput_type
at all? Or maybe they should bePydanticUndefined
🤔
maybe we could special-casePydanticUndefined
in errors, so it would just show
Field required [type=missing]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I'll see if this can be done without too much special casing, I agree it would be better.
for err in line_errors { | ||
errors.push(lookup_path.apply_error_loc(err, self.loc_by_alias, &field.name)); | ||
Err(e) => { | ||
state.has_field_error = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why do we need to track this on the state? Should we adjust the algorithm to just never call.default_value
if there's any errors?
(We could go further and rearrange things a bit so that all default values only get filled in after the whole input is processed, there is some overlap with#1620 🤔)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Why do we need to track this on the state? Should we adjust the algorithm to just never call .default_value if there's any errors?
It's only relevant for default factories taking the data argument, so if you just have a "static" default it shouldn't matter. And because the logic to call the default value is nested under the current field validator, I can't really "influence" the logic from the model field.
This state approach doesn't look right, but I couldn't find a better way to do so 🤔
(We could go further and rearrange things a bit so that all default values only get filled in after the whole input is processed, there is some overlap with#1620 🤔)
Would this allow using other field values independently from the field order?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Ah, I see. I missed the default factories guarantee field order; this makes a lot of sense.
Because of the existence ofPydanticUndefined
as a way to ask for defaults, I think my original proposal actually doesn't make sense. 🤔
... but I do worry about when this flag is reset. Currently once you set it it's true for the rest of the validation. This seems problematic in unions, for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Indeed this is one of the reasons it doesn't look right; however, for unions I think it should be alright, as there's a boundary in the validation error.
The error we are handling here is the validation error of the field. If this field is defined like:
classModel(BaseModel):a:int|str
And a validation error happens onint
, the field validator will try validating againststr
. At this point, we don't know what happened, and we will only get a val error if validation againststr
failed as well.
Edit: Actually it might be wrong if we have:
classModel1(BaseModel):a:intclassModel2(BaseModel): ...classModel(BaseModel):m:Model1|Model2
if validation fails forModel1.a
, it will mutate the state (if it is the same state than when validatingModel
).
Now the question is: are unions the only concern? If so, we can make sure we somehow reset this state value on union validation, but there might be other concerns apart from unions :/
if matches!(self.default, DefaultType::DefaultFactory(_, true)) && state.has_field_error { | ||
// The default factory might use data from fields that failed to validate, and this results | ||
// in an unhelpul error. | ||
return Err(ValError::new(ErrorTypeDefaults::DefaultFactoryNotCalled, input)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hmm interesting, I wonder if the error fora
should not includeinput_value
orinput_type
at all? Or maybe they should bePydanticUndefined
🤔
maybe we could special-casePydanticUndefined
in errors, so it would just show
Field required [type=missing]
?
Change Summary
Fixespydantic/pydantic#11358.
@pydantic/oss, I'm not too satisfied with this approach, but I think it is reasonable? I just realized the same can also happen with validators:
But the default factory case strikes me as a much more common use case:
Related issue number
Checklist
pydantic-core
(except for expected changes)