Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork3.1k
[PEP 696] Fix swapping TypeVars with defaults.#19449
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
According tomypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅ |
| repl=repl.accept(self) | ||
| ifisinstance(repl,TypeVarType): | ||
| repl.default=repl.default.accept(self) | ||
| repl.default=repl.default.accept(self) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Types should benever modified in place. I know you didn't introduce it, but this should be fixed ASAP. (I think there is also a chance some of the problems you try to fix may be caused by this).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Types should be never modified in place.
So, withcopy_modified then? If stuff should never be modified in place, it surprises me that the variables are not annotated asFinal in the first place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hm, If I change this then now 2 tests fail:testTypeVarDefaultsClassRecursive1 andtestTypeVarDefaultsClassRecursiveMultipleFiles
Details
____________________________________________________________________________________________________________________________________ testTypeVarDefaultsClassRecursive1 _____________________________________________________________________________________________________________________________________[gw0] linux -- Python 3.13.8 /home/rscholz/github/CONTRIBUTING/mypy/.venv/bin/python3data: /home/rscholz/github/CONTRIBUTING/mypy/test-data/unit/check-typevar-defaults.test:451:Failed: Unexpected type checker output (/home/rscholz/github/CONTRIBUTING/mypy/test-data/unit/check-typevar-defaults.test, line 451)------------------------------------------------------------------------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------------------------------------------------------------------------Expected: main:16: note: Revealed type is "__main__.ClassD1[builtins.str, builtins.str]" main:17: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.int]" main:18: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.float]" main:21: note: Revealed type is "__main__.ClassD1[builtins.str, builtins.str]" (diff) main:23: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.int]" main:25: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.float]" main:35: note: Revealed type is "__main__.ClassD2[builtins.str, builtins.str, builtins.str]" main:36: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.int, builtins.int]" main:37: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.float]" main:38: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.str]" main:41: note: Revealed type is "__main__.ClassD2[builtins.str, builtins.str, builtins.str]" (diff) main:43: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.int, builtins.int]" (diff) main:45: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.float]" main:47: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.str]" main:56: note: Revealed type is "__main__.ClassD3[builtins.str, builtins.list[builtins.str]]" ... main:58: note: Revealed type is "__main__.ClassD3[builtins.int, builtins.float]" main:63: note: Revealed type is "__main__.ClassD3[builtins.int, builtins.list[builtins.int]]" main:65: note: Revealed type is "__main__.ClassD3[builtins.int, builtins.float]"Actual: main:16: note: Revealed type is "__main__.ClassD1[builtins.str, builtins.str]" main:17: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.int]" main:18: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.float]" main:21: note: Revealed type is "__main__.ClassD1[builtins.str, T1`1 = builtins.str]" (diff) main:23: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.int]" main:25: note: Revealed type is "__main__.ClassD1[builtins.int, builtins.float]" main:35: note: Revealed type is "__main__.ClassD2[builtins.str, builtins.str, builtins.str]" main:36: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.int, builtins.int]" main:37: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.float]" main:38: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.str]" main:41: note: Revealed type is "__main__.ClassD2[builtins.str, T1`1 = builtins.str, T2`2 = T1`1 = builtins.str]" (diff) main:43: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.int, T2`2 = builtins.int]" (diff) main:45: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.float]" main:47: note: Revealed type is "__main__.ClassD2[builtins.int, builtins.float, builtins.str]" main:56: note: Revealed type is "__main__.ClassD3[builtins.str, builtins.list[builtins.str]]" ... main:58: note: Revealed type is "__main__.ClassD3[builtins.int, builtins.float]" main:63: note: Revealed type is "__main__.ClassD3[builtins.int, builtins.list[builtins.int]]" main:65: note: Revealed type is "__main__.ClassD3[builtins.int, builtins.float]"Alignment of first line difference: E: ..._.ClassD1[builtins.str, builtins.str]" A: ..._.ClassD1[builtins.str, T1`1 = builtins.str]" ^Update the test output using --update-data (implies -n0; you can additionally use the -k selector to update only specific tests)______________________________________________________________________________________________________________________________ testTypeVarDefaultsClassRecursiveMultipleFiles _______________________________________________________________________________________________________________________________[gw0] linux -- Python 3.13.8 /home/rscholz/github/CONTRIBUTING/mypy/.venv/bin/python3data: /home/rscholz/github/CONTRIBUTING/mypy/test-data/unit/check-typevar-defaults.test:518:Failed: Unexpected type checker output (/home/rscholz/github/CONTRIBUTING/mypy/test-data/unit/check-typevar-defaults.test, line 518)------------------------------------------------------------------------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------------------------------------------------------------------------Expected: main:15: note: Revealed type is "__main__.ClassG1[builtins.int, builtins.int]" main:16: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.str]" main:17: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.float]" main:20: note: Revealed type is "__main__.ClassG1[builtins.int, builtins.int]" (diff) main:22: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.str]" main:24: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.float]"Actual: main:15: note: Revealed type is "__main__.ClassG1[builtins.int, builtins.int]" main:16: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.str]" main:17: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.float]" main:20: note: Revealed type is "__main__.ClassG1[builtins.int, T2`1 = builtins.int]" (diff) main:22: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.str]" main:24: note: Revealed type is "__main__.ClassG1[builtins.str, builtins.float]"Alignment of first line difference: E: ..._.ClassG1[builtins.int, builtins.int]" A: ..._.ClassG1[builtins.int, T2`1 = builtins.int]" ^Update the test output using --update-data (implies -n0; you can additionally use the -k selector to update only specific tests)========================================================================================================================================== short test summary info ==========================================================================================================================================FAILED mypy/test/testcheck.py::TypeCheckSuite::check-typevar-defaults.test::testTypeVarDefaultsClassRecursive1FAILED mypy/test/testcheck.py::TypeCheckSuite::check-typevar-defaults.test::testTypeVarDefaultsClassRecursiveMultipleFilesI noticed thatTypeVarType.__hash__ is given byhash((self.id, self.upper_bound, tuple(self.values))), so changingdefault does not change the hash.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yes. For historical reasons some types are constructed "gradually", so we can't make type attributes final. Maybe we will at some point, but that would be a big refactoring. For now, mypy contributors should remember a simple rule: types always new, symbols always original.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If I change this then now 2 tests fail
Oh well, it means there is another bug. TBH I am not sure if default should be part of the hash conceptually (and what it even means to have a type variable with same id, but a different default). IMO the support for type variable defaults may need to be re-implemented from scratch, as it is problematic on a conceptual level: regular default is not a property of an argument, it is a property of a function; same for type variables, their defaults are not property of type variable type itself, but of the function/class that uses them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
So the problem seems to be with getting the correct ID's. In the test case, we haveclass D[T1=str, T2=T1].
>>>self.variables {1:T1`604 = builtins.str, 2: T2`605=T1`1=builtins.str}>>>self.recursive_tvar_guard{604:T1`604=builtins.str,605:None}>>>repl.default=repl.default.accept(self)>>>self.variables {1:T1`604 = builtins.str, 2: T2`605=T1`604=builtins.str}>>>self.recursive_tvar_guard{604:T1`604=builtins.str,605:None}
Modifyingrepl.default in-place ensures that in theExpandTypeVisitor.variables dictionary, the default of the dependent type variable gets updated to the new ID.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
TBH I don't understand why do we need to handle the defaults inexpand_type in the first place. As I explained, the default is not a property of a type variable, it is a property of a class/function, so the caller (e.g.applytype etc) should always provide acomplete type var -> replacement mapping. So thatexpand_type() should not be even allowed to look at the defaults.
Yeah, I just looked at the original PR that added this logic#16878, it looks broken. The full complete mapping should be computed either in:
semanal/typeanalfor "static" application likex: C[int]- in
applytypefor "dynamic" application likex = C[int]()
before we even callexpand_type. I think the only right way to do it is this way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I am not sure how to fix this at the moment, but given that this PR
- Fixes two very annoying bugs with PEP696 checking (essentially prevents typing bijections with defaults)
- Doesn't introduce inplace modification -- it was already there before (all we really do is make sure to update the default a-priori, which is crucial e.g. when the second typevar defaults to the first one and we expand the first one.)
- This issue probably needs to be addressed elsewhere
Maybe this can still be merged as-is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The downside is that this is papering over the problem instead of fixing the real root cause. But anyway, if you feel strongly about this, I can merge this, it will not be much more work to axe this all later (and the tests are useful).
0ece662 intopython:masterUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
testTypeVarDefaultsSwap)overrideerror when subclassing with multiple typevars with defaults #19362 (addedtestTypeVarDefaultsSwap2)Changed the logic for recursion guards of
TypeVarType: Instead of always substitutingrepl = repl.accept(self), and situationally updatingrepl.default = repl.default.accept(self)if the result is aTypeVarType, we now always updaterepl.default = repl.default.accept(self)a priori and then only choose the expandedrepl.accept(self)if the result is a concrete type.New Tests
testTypeVarDefaultsSwap(https://mypy-play.net/?mypy=1.17.0&python=3.12&gist=d5a025a31ae3c8b9e2a36f4738aa1991)testTypeVarDefaultsSwap2(https://mypy-play.net/?mypy=1.17.0&python=3.12&gist=d3ed42c82f7144967c97d846c4c041ef)PS: closed earlier PRs#19447, since it contained debugging changes, and#19448 because it didn't solve#19362.