Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

perf: try to cache inner contexts of overloads#19408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
sterliakov wants to merge10 commits intopython:master
base:master
Choose a base branch
Loading
fromsterliakov:experiments/st-overload-cache

Conversation

sterliakov
Copy link
Collaborator

@sterliakovsterliakov commentedJul 8, 2025
edited
Loading

Improves#14978 and marginally improves nested overloads checking in general. This is not ready for review, but I would appreciate any in-progress feedback on the idea itself.

If I understand correctly, this change should not change any behavior at all:infer_arg_types_in_empty_context is almost pure.

This change halvescolour check time.

Another very similar problem arises when checking overloaded binary operations (like in#14978), they do not translate directly into a series of overloads check but repeat the whole process for every node instead. Improving overload checking helps with that, but the result is still slow beyond reasonable. I'll try to push this further later in a separate PR.

@github-actionsGitHub Actions

This comment has been minimized.

@sterliakovsterliakov changed the titleWIP: try to cache inner contexts of overloads and opsWIP: try to cache inner contexts of overloadsJul 8, 2025
@github-actionsGitHub Actions

This comment has been minimized.

@sterliakovsterliakovforce-pushed theexperiments/st-overload-cache branch fromc7da595 toa1f8c77CompareJuly 9, 2025 15:00
@github-actionsGitHub Actions

This comment has been minimized.

@sterliakov
Copy link
CollaboratorAuthor

Now this is what I wanted to see: no behavior changes andcolour brought down to 27 minutes from 50 - almost halved!

This is not ready for code review, removing draft status to get some feedback on the idea itself. I do not particularly like usingid(expr) as a cache key, but it should be OK?TempNode is excluded, and everything else should not be subject to ID rot because objects actually remain in the parsed tree for the whole lifetime of checker, their IDs cannot get reused, and we should not construct any nodes other thanTempNode while checking function calls.

@sterliakovsterliakov marked this pull request as ready for reviewJuly 9, 2025 17:18
@github-actionsGitHub Actions

This comment has been minimized.

@sterliakovsterliakov changed the titleWIP: try to cache inner contexts of overloadsperf: try to cache inner contexts of overloadsJul 10, 2025
Copy link
Collaborator

@JukkaLJukkaL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think we need something along these lines, but it may be more complicated than this to maintain 100% compatibility with non-cached behavior. I suspect that we may need to consider the binder and possibly cache generated errors as well. There may be some other relevant state that is kind of implicit in the type checker. I can look into this in more detail and reason through all possible issues, but it will take some effort.

@sterliakov
Copy link
CollaboratorAuthor

That's why I only added caching toinfer_args_in_empty_context. It's a nice place to do that: diagnostics from this call arenever silenced and do not affect current overload resolution. We reject incompatible signatures later, accepting all arguments again anyway. Furthermore, it is only called when checking overload calls andAny calls, so I can safely remove overload stack check from it and replace it with a boolean flag. So I don't think that the generated errors can become relevant later. This might change overload resolution in presence of a globally invalid argument (foo(1 + 'a') wherefoo is overloaded), but any behavior is reasonable in such case IMO: we still emit an error that the argument is invalid itself. To prevent this, it is enough to record whether any errors were encountered, and skip the cache in such case.

Binder is indeed an implicit dependency, but it's still OK within a parent overload: re-accepting anAssignmentExpr should not change anything? Anyway, we can poison the cache invisit_assignment_expr to be extra sure, and that's probably a reasonable thing to do.

I'll add the changes mentioned above in a few hrs, but frankly my main "wtf" here is using IDs as cache keys.

@github-actionsGitHub Actions
Copy link
Contributor

According tomypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@sterliakov
Copy link
CollaboratorAuthor

Still quite nice in a single sample:

/tmp/mypy_primer/mypy_new/venv/bin/mypy on colour took 1988.16s/tmp/mypy_primer/mypy_old/venv/bin/mypy on colour took 2953.89s

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@JukkaLJukkaLJukkaL left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@sterliakov@JukkaL

[8]ページ先頭

©2009-2025 Movatter.jp