Fixes#11390,fixes#10695, works towards#9969 (we don't get theKeyError anymore, but we still get a clean schema error because of a missing definition).

Change Summary

#5305 made it so that unknown forward annotations would not stop field collection. The PR is huge and doesn't explain why this was done, but this introduced subtle bugs and code smells that this PR tries removing.

The first issue arising from this is that the model fields can be inaccurate when a model is not fully built yet:

fromtypingimportAnnotatedfrompydanticimportBaseModelclassModel(BaseModel):a:'Annotated[Forward, Field(alias="b")]'Model.model_fields['a'].alias#> None, should be 'b'

and I won't be surprised if some users currently make use ofmodel_fields on an incomplete model (especially as we implicitly rebuild on instance creation — meaning most users don't explicitly callmodel_rebuild()). A relatively big concern I also have is FastAPI relying onFieldInfo instances to validate data. They do manual stuff before calling the Pydantic validate methods, such as checking forFieldInfo.is_required(),alias.

When a model is being rebuilt, we reevaluate the annotation and recreate aFieldInfo instance during the core schema gen process (and we used to even to this in a less performant way; see#10769):

Doing so introduced a code smell where we track explicitly set attributes using_attributes_set. This is known to be a source of bugs as we need to keep track of such explicitly set attrs when you alter theFieldInfo instance:

• We forgot to update it when using... as the default value in some cases. See this added test case:
  

  pydantic/tests/test_edge_cases.py

  Lines 1919 to 1927 in929e8f4

  	deftest_ellipsis_forward_ref_annotated()->None:
  	"""This previously resulted in the ellipsis being used as a default value."""
  	classModel(BaseModel):
  	f:'Forward'
  	Forward=Annotated[int,Field(...)]
  	assertModel.model_fields['f'].defaultisPydanticUndefined

• We currently forget to updatedeprecated (first bug ofDeprecated fields bugs with forward annotations #11390).

And more generally:

• On_generate_schema.py#L1311, wedirectly mutate the original field info instance to take the newly evaluated annotation. This is the reason we get theKeyError on generics (KeyError when inhering from a self-referencing generic model #9969).
• We make use of the model fields (even though they might be incorrect) insideModelMetaclass.__new__, at model creation (i.e. stuff thatisn't reprocessed onmodel_rebuild()), such as setting deprecated descriptors (second bug of#11390: we set these descriptors based onFieldInfo.deprecated).

Instead, this PR does three things:

• Add a new__pydantic_fields_complete__ class property on models (for private use). It isTrue if at least one field annotation failed to evaluate duringcollect_model_fields(). On a related note: we still set__pydantic_fields__ for backwards compatibility, but a follow up PR can now easily add a user/deprecation warning when accessingmodel_fieldsif__pydantic_fields_complete__ == False. In V3, we could remove this backward compatibility logic (TBD).
  Also note that now wedon't enterGenerateSchema if the fields collection failed, as it is guaranteed to fail anyway (if an annotation failed to evaluate during fields collection, it will also immediately fail to be evaluated inGenerateSchema).
• Callingmodel_rebuild()will redo the fields collection if it wasn't successful the first time. Note that this aligns with Pydantic dataclasses.
• When an incomplete Pydantic model is encountered as an annotation (insideGenerateSchema), we first callrebuild_model_fields(); a new function that iterates over the existing (but potentially not complete) field infos, and recreate instances if the annotation needs to be evaluated again. This way, we avoid generating a schema for each field just to realize that one field still has a forward annotation:

  classSub(BaseModel):f1:int    ...# Many fields...f100:'Forward'classModel(BaseModel):sub:Sub# Note that Sub is incomplete, so in the process of generating the schema for# `Model`, we don't use the cached schema attribute# Thus previously, we would generate a schema for f1, f2, ..., f99. f100 would fail,# meaning we generated 99 fields core schemas for nothing.# In this PR, we first (re)collect fields for `Sub`, and immediately raise if fields collection did not suceed.

  On important thing to acknowledge: when encounteringsub: Sub, we don't callSub.model_rebuild(). Doing so might be the right thing to do, but is unfortunately not backwards compatible. The reason is that the NS Resolver used forModel might contain annotations (e.g. provided through theparent_namespace) that should be available forSub. This means that even ifsub: Sub successfully builds,Sub isn't complete yet — each model should be rebuilt independently, throughmodel_rebuild()).

TODO:

• Performance issues: I believe this is a tradeoff to have, as the couple affected benchmarks are models with one field defined, and the newrebuild_model_fields() might add a bit of overhead. However, it will be beneficial if many fields are defined (see the example withf1..100 above and thetest_failed_rebuild benchmark).

Related issue number

Checklist

• The pull request title is a good summary of the changes - it will be used in the changelog
• Unit tests for the changes exist
• Tests pass on CI
• Documentation reflects the changes where applicable
• My PR is ready to review,please add a comment including the phrase "please review" to assign reviewers