Indirectlycloses#9443.

What we're trying to achieve and how

get_type_ref is pretty expensive, and we'll try to reduce the amount of calls to this function.

Currently, for every call toGenerateSchema.generate_schema, we unconditionally end up with the following call chain:_generate_schema_from_property ->get_schema_or_ref ->get_type_ref.

The idea is thus to remove this part (L750-752):

So that anyGenerateSchema method responsible for generating a core schema for a referenceable type handle it. For example,_model_schema, which already has this check:

Note that theif maybe_schema is not None branch is never reached (same forall the other referenceable types such as dataclasses, typeddict, etc) because the branch we're trying to remove handles it already.

So surely removing the branch from_generate_schema_from_property should work.

Challenges

However, this breaks for Pydantic models:

Let's say you have the following:

classSub(BaseModel):passclassModel(BaseModel):sub1:Subsub2:Sub

When we generate a schema forsub2, we follow the call chaingenerate_schema ->_generate_schema_from_property. If wedon't check forget_schema_or_ref, we would want it to be checked in_model_schema as shown above. However, the rest of the_generate_schema_from_property implementation will check for__get_pydantic_core_schema__(), and the__pydantic_core_schema__ property. In our example,Sub is already in the definitions thanks tosub1, so it would break by returning the complete core schema ofSub¹ instead of adefinition reference schema (produced byget_schema_or_ref).

So let's try to push things a bit further. Why not move the__get_pydantic_core_schema__()/__pydantic_core_schema__ logic to the_model_schema method? (same for dataclasses).

The issue is that while it would be fine for the__pydantic_core_schema__ property (only relevant for Pydantic models and dataclasses), the__get_pydantic_core_schema__() method can be set on pretty much anything.

Solution

That's why I had to remove the__get_pydantic_core_schema__ method fromBaseModel. Thecurrent implementation is really only returning the cached__pydantic_core_schema__ property or delegating the generation to theGenerateSchema class.

During model construction, we were doing something even more confusing:

The following call chain happens:cls.__get_pydantic_core_schema__(cls, handler) ->handler(cls) ~GenerateSchema().generate_schema(cls, from_dunder_get_core_schema=False).

I think havingfrom_dunder_get_core_schema to avoid an infinite loop is a sign that the current implementation is quite convoluted.

Instead, by:

• removing theBaseModel.__get_pydantic_core_schema__ method, and moving the check for the__pydantic_core_schema__ attribute toGenerateSchema._model_schema
• Only checking for a__get_pydantic_core_schema__ method (and not the property) when callingGenerateSchema.generate_schema (which isn't an issue for Pydantic models as we removed it).

We:

• allow for a more consistent approach when it comes to schema generation of models. We can now callGenerateSchema().generate_schema(some_model_cls), and the call chain is way simpler:generate_schema -> ... ->_model_schema.
• move the logic to potentially callget_schema_or_ref (and check for the__pydantic_core_schema__ property) to the responsible methods:_model_schema,_dataclass_schema, etc.

Breaking changes concerns

We had a couple reports from people overridingBaseModel.__get_pydantic_core_schema__:

• the first one I could find:type casting does not work with multiple attributes of the same custom type #7102. Thementioned reason isn't really compelling, the user was doing things not the intended way. It also led toa scary and hacky fix that I would really like not having.
• Strange ValidationErrors with ≥2 nested instances of model with customized __get_pydantic_core_schema__ (embedded schemas differ, field ordering dependent) #10160, a user facing unexpected issues. Iexplained in length why it was the case (and relates to my explanation here).
• __get_pydantic_core_schema__ ignored in self-referencing models #10582, a user also facing unexpected issues (maybe the same ones).

Feels like doing so always led to unexpected behavior, and a hacky fix. That's why I'm in support of having this pattern forBaseModel removed. It might break existing code, but we are probably helping the users in avoiding unexpected issues in the future.

We could try to still support the existingBaseModel.__get_pydantic_core_schema__ method, but honestly it's been quite hard to implement this already, and it might be that it's impossible to have both this perf improvement and support for the existing behavior at the same time