CodSpeed Performance Report

Merging#1502 willnot alter performance

_{Comparingchanghc:10629-fix-enum (06ac56d) withmain (cd0346d)}

Summary

✅ 155 untouched benchmarks

Codecov Report

Attention: Patch coverage is97.29730% with1 line in your changes missing coverage. Please review.

Project coverage is 89.39%. Comparing base(ab503cb) to head(06ac56d).
Report is 226 commits behind head on main.

@@            Coverage Diff             @@##             main    #1502      +/-   ##==========================================- Coverage   90.21%   89.39%   -0.83%==========================================  Files         106      112       +6       Lines       16339    17966    +1627       Branches       36       40       +4     ==========================================+ Hits        14740    16060    +1320- Misses       1592     1886     +294- Partials        7       20      +13

This issue lies mostly in validating JSON values. Take the case in the issue as an example:

fromenumimportEnumclassA(Enum):a=1,2serialised=RootModel[A](A.a).model_dump_json()parsed=RootModel[A].model_validate_json(serialised)# fail

What happens here is thatA.a is serialised as[1, 2], but the validator is built with one possible enum value(1, 2) and thus cannot find any possible value that matches the input value[1, 2].

My initial commit does fix this issue by addinglist(tuple) to the validator, but this does not cover all cases. The complete solution is to load the validator with serialised possible enum values and use them when the validator deals with JSON input values. Think the following example:

classModel(BaseModel):x:int@model_serializerdefser_model(self):return [self.x]classA(Enum):a=2,3b=Model(x=1)serialised=RootModel[A](A.b).model_dump_json()# [1]parsed=RootModel[A].model_validate_json(serialised)# fail

To fix this issue completely, we need to know the serialised form of all enum values and add them to the validator. My initial fix solves the problem because tuples are serialised as JSON arrays and casting tuples to lists covers this case. The problem for me now is that I don't know how to find the corresponding serialisers given aPyAny instance. Since we know what possible enum values are and they are a limited set, I imagine this to be straightforward.@davidhewitt could you point me to the right place?

By the way, the following case is expected:

classMyEnum(Enum):a=1,2b= [1,2]v=SchemaValidator(core_schema.enum_schema(MyEnum,list(MyEnum.__members__.values())))assertv.validate_json('[1, 2]')isMyEnum.a

Since bothMyEnum.a andMyEnum.b are identical after serialisation, there is no way to determine what a JSON value represented before it was serialised, the first matched enum value is returned.

I think you might want to be looking ininfer.rs to see what we would serialize for an arbitrary object?

Hi@davidhewitt, I've finally come up with a seemingly generalisable solution, but I would like to check with you and see if I missed out anything.

The idea is basically that we create a separate lookup inEnumValidator that contains the JSON form of all enum values and use this lookup for validation when the input type is JSON. The JSON form is acquired by callingserializers::to_jsonable_python. So far my implementation works with the following scenarios:

# root model with enum round tripclassModel(BaseModel):x:int@model_serializerdefser_model(self):return [self.x]classA(Enum):e="1+2j"a=2,3b=complex(1,2)c=datetime.datetime.fromisoformat("2024-01-01T00:00:00Z")d=Model(x=2)f=float('inf')forvinA.__members__.values():RootModel[A].model_validate_json(RootModel[A](v).model_dump_json())# base model with enum round tripclassM(BaseModel):v:A#model_config = ConfigDict(ser_json_inf_nan='strings', use_enum_values=True)forvinA.__members__.values():M.model_validate_json(M(v=v).model_dump_json())

I'm not quite sure about callingto_jsonable_python because ideally we should compose the JSON lookup usingEnumSerializer so that it would be a round trip without any unexpected mismatch, e.g. think the complex value inference I missed out in my original implementation, but meanwhile that means exposing quite a few things from theserializer module and I'm not sure if that's the right thing to do.

At the same time, I have some questions regarding implementations aroundto_python(mode='JSON'), especially regardingfloat. Do you know whyhere it does not takeinf_nan_mode into consideration? This is a case where my implementation does not fully work. In the classM above, if you uncommentmodel_config, it gives an error when processingA.f, thatfloat('inf'):

pydantic_core._pydantic_core.ValidationError:1validationerrorforMvInputshouldbe'1+2j', (2,3), (1+2j),datetime.datetime(2024,1,1,0,0,tzinfo=datetime.timezone.utc),Model(x=2)orinf [type=enum,input_value='Infinity',input_type=str]

because the serialized value inmodel_dump_json ("Infinity") does not match the one acquired withto_jsonable_python (inf).

By the way, I'm still listing the original values in the error message instead of listing them in the JSON form because I'm not sure if showing the JSON form would be confusing.

Sorry if this is not very clear because I'm in a bit of a rush, but please let me know what you think. Thank you!

please review