Refactor the_typing_extra module to:
- make use ofhttps://typing-extensions.readthedocs.io/en/latest/#runtime-use-of-types. In a lot of places, we are only checking forsome_obj is typing(_extensions).SomeType which can break at any time iftyping_extensions backports some changes from Python, in which casetyping.SomeType is not typing_extensions.SomeType.
- Cleanup/simplify someis_* functions (e.g.is_newtype,is_classvar).
- Add a newis_classvar_annotation function, that should be used in most cases. This function handlesClassVar being wrapped inAnnotated, and also stringified annotations.
- Renameall_literal_values toliteral_values, replacing the unusedliteral_values function. When we drop support for Python 3.8, replace usage with a plainget_args.
- Remove unecessaryis_dataclass function,dataclasses.is_dataclass provides a type guard return annotation.
Adapt the codebase to use the new constructs. In some cases, this does fix some actual bugs

Related issue number

Checklist

The pull request title is a good summary of the changes - it will be used in the changelog
Unit tests for the changes exist
Tests pass on CI
Documentation reflects the changes where applicable
My PR is ready to review,please add a comment including the phrase "please review" to assign reviewers

github-actionsbot added the relnotes-fixUsed for bugfixes. label

Oct 27, 2024

Copy link

cloudflare-workers-and-pagesbot commentedOct 27, 2024•
edited
Loading

Deploying pydantic-docs with Cloudflare Pages

Latest commit:	`5a43b8d`
Status:	✅ Deploy successful!
Preview URL:	https://b118d254.pydantic-docs.pages.dev
Branch Preview URL:	https://refactor-typing-extra.pydantic-docs.pages.dev

View logs

Copy link

codspeed-hqbot commentedOct 27, 2024•
edited
Loading

CodSpeed Performance Report

Merging#10725 willnot alter performance

_{Comparingrefactor-typing-extra (5a43b8d) withmain (a25e028)}

Summary

✅ 44 untouched benchmarks

Copy link

Contributor

github-actionsbot commentedOct 27, 2024•
edited
Loading

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
pydantic
dataclasses.py
json_schema.py
main.py
pydantic/_internal
_core_utils.py
_dataclasses.py
_fields.py
_generate_schema.py
_generics.py
_model_construction.py
_repr.py
_std_types_schema.py
_typing_extra.py					35,199
Project Total

_{This report was generated bypython-coverage-comment-action}

Viicos force-pushed therefactor-typing-extra branch 3 times, most recently from279daa8 toc68a904Compare

October 29, 2024 07:55

Refactor_typing_extra module

9517365

Viicos force-pushed therefactor-typing-extra branch fromc68a904 to591dde0Compare

October 29, 2024 08:29

Adapt codebase to use refactored module

0faf9b6

Viicos force-pushed therefactor-typing-extra branch from591dde0 to0faf9b6Compare

October 29, 2024 08:34

Perf improvements, 1

75647e0

Viicos force-pushed therefactor-typing-extra branch 4 times, most recently froma5ef806 to088b5c2Compare

October 30, 2024 08:44

Viicos commented

Oct 30, 2024

View reviewed changes

pydantic/_internal/_repr.py Outdated

Copy link

MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thisdisplay_as_type function needs to be refactored, as it is relatively expensive to recursively check forget_origin,get_args, etc. It is currently used:

InFieldInfo.__repr_args__, to make a string representation of theannotation attribute. This can be kept.
Inget_type_ref, called for each arg of a parametrized type. We should find a simpler way to generate a core schema reference.

Couple fixes

291dd9b

Viicos force-pushed therefactor-typing-extra branch from088b5c2 to291dd9bCompare

October 30, 2024 08:48

Viicos marked this pull request as ready for review

October 31, 2024 10:34

Copy link

Contributor

sydney-runkle commentedOct 31, 2024

Should we be concerned about some of the minor performance regressions we see for schema building here?

https://codspeed.io/pydantic/pydantic/branches/refactor-typing-extra

Copy link

MemberAuthor

Viicos commentedOct 31, 2024

Should we be concerned about some of the minor performance regressions we see for schema building here?

My assumption is that the small regression comes from the fact that in hot functions (display_as_type,get_type_ref,has_instance_in_type) we now properly check for the two variants (typing andtyping_extensions), e.g. forTypeAliasType.

I think the real problem is that we should find a way to avoid calling these functions so many times, this is what I'm working on currently as part of#10297

sydney-runkle reviewed

Nov 1, 2024

View reviewed changes

Copy link

Contributor

sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Wow, nice work@Viicos. Things feel much cleaner in the internal type management department with this PR. Left some general feedback 👍

pydantic/_internal/_generics.pyShow resolvedHide resolved

pydantic/_internal/_model_construction.pyShow resolvedHide resolved

pydantic/_internal/_generics.py

		@@ -184,7 +184,7 @@ def _get_caller_frame_info(depth: int = 2) -> tuple[str \| None, bool]:
		DictValues: type[Any] = {}.values().__class__


		def iter_contained_typevars(v: Any) -> Iterator[TypeVarType]:
		def iter_contained_typevars(v: Any) -> Iterator[TypeVar]: