Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Fix performance in union subtyping#15104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
JukkaL merged 1 commit intopython:masterfromhauntsaninja:union-perf
Apr 23, 2023

Conversation

@hauntsaninja
Copy link
Collaborator

@hauntsaninjahauntsaninja commentedApr 23, 2023
edited
Loading

This is a performance optimisation for subtyping between two unions that are largely the same.

Fixes#14034

This makes@adriangb's example in#14034 (comment) finish basically instantly. I could add it as a unit test?

Type checking pydantic core is still not fast — takes like four or five minutes with uncompiled mypy — but at least it's now feasible. I think there's room for doing some optimisation in make_simplified_union that would improve this.

adriangb, dmontagu, JukkaL, antonagestam, and CoolCat467 reacted with rocket emoji
This is a performance optimisation for subtyping between two unions thatare largely the same.Fixespython#14034
@adriangb
Copy link
Contributor

This is amazing! It's a huge (>99%) improvement.

@github-actions
Copy link
Contributor

According tomypy_primer, this change has no effect on the checked open source code. 🤖🎉

@dmontagu
Copy link

dmontagu commentedApr 23, 2023
edited
Loading

Massive improvement, I compiled this locally and was able to type check the main branches of pydantic and pydantic-core (with ourif not MYPY: hack removed inpydantic_core.core_schema) in ~26s on my machine (using python 3.11). And that was without the--disable-recursive-aliases flag that we used to need for it to finishat all; no one ever waited long enough to see how long it would take without it, but it was at least hours. (I'll note for anyone curious that with these changes, that flag no longer has any effect on performance while type-checking pydantic.)

For comparison, mypy 1.2.0with--disable-recursive-aliases takes ~2 minutes on my machine.

This is fast enough that it is viable once again for us to use mypy again for type-checking pydantic, but we'd definitely appreciate any further performance improvements that might be possible frommake_simplified_union!

@cdce8pcdce8p mentioned this pull requestApr 23, 2023
Copy link
Collaborator

@JukkaLJukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Awesome!

@JukkaLJukkaL merged commitbd6ce23 intopython:masterApr 23, 2023
@samuelcolvin
Copy link
Contributor

Awesome, thank you so much.

@hauntsaninjahauntsaninja deleted the union-perf branchApril 23, 2023 22:24
wesleywright pushed a commit that referenced this pull requestApr 24, 2023
This is a performance optimisation for subtyping between two unions thatare largely the same.Fixes#14034This makes@adriangb's example in#14034 (comment)finish basically instantly. I could add it as a unit test?Type checking pydantic core is still not fast — takes like four or fiveminutes with uncompiled mypy — but at least it's now feasible. I thinkthere's room for doing some optimisation in make_simplified_union thatwould improve this.
hauntsaninja added a commit to hauntsaninja/mypy that referenced this pull requestApr 25, 2023
The following code optimises make_simplified_union in the common casethat there are exact duplicates in the union. In this regard, this issimilar topython#15104To get this to work, I needed to use partial tuple fallbacks in a coupleplaces (these maybe had the potential to be latent crashes anyway?)There were some interesting things going on with recursive type aliasesand type state assumptionsThis is about a 25% speedup on the pydantic codebase and about a 2%speedup on self check (measured with uncompiled mypy)
hauntsaninja added a commit that referenced this pull requestMay 6, 2023
Fixes#15192The following code optimises make_simplified_union in the common casethat there are exact duplicates in the union. In this regard, this issimilar to#15104There's a behaviour change in one unit test. I think it's good? We'llsee what mypy_primer has to say.To get this to work, I needed to use partial tuple fallbacks in a coupleplaces. These could cause crashes anyway.There were some interesting things going on with recursive type aliasesand type state assumptionsThis is about a 25% speedup on the pydantic codebase and about a 2%speedup on self check (measured with uncompiled mypy)
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@JukkaLJukkaLJukkaL approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

mypy 0.990 is 1000x slow on pydantic codebase vs 0.982

5 participants

@hauntsaninja@adriangb@dmontagu@samuelcolvin@JukkaL

[8]ページ先頭

©2009-2025 Movatter.jp