Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Draft: Solve intermediate variable bug#19399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
randolf-scholz wants to merge16 commits intopython:master
base:master
Choose a base branch
Loading
fromrandolf-scholz:solve_intermediate_variable_bug

Conversation

randolf-scholz
Copy link
Contributor

@randolf-scholzrandolf-scholz commentedJul 8, 2025•
edited
Loading

Looking for feedback, as this was mostly a trial-and-error process.

  • Removedself.infer_function_type_arguments_using_context. Instead, we now use a functioninfer_constraints_from_context that only yields the constraints.
  • Modifiedinfer_function_type_arguments to now include the context.
    We compute 2 sets of constraints: the "outer constraints", determined by the context, and the "inner constraints" determined by the callable type. From this, we compute 2 solutions:
    • the "outer solution", which only uses the outer constraints.
    • the "joint solution", which uses both the outer and inner constraints.
  • We prefer the "joint solution", unless:
    • it failed to solve some variables
    • the return type of the outer solution is a (strict) subtype (i.e. more precise) of the return type of the joint solution
    • the return type of the outer solution is a union, and any member of that union is a subtype of the joint return type.1
  • if the outer solution was selected, we essentially fall back to the original code from the master branch, and first apply the outer solution, and then recompute the inner constraints and apply the inner solution.

New Tests

Modified Tests

  • testRecursiveAliasWithRecursiveInstance: improved inference2
  • testCallerTupleVarArgsAndGenericCalleeVarArg now revealsint | None rather thanLiteral[1]? | None3
  • testInferenceAgainstGenericCallableGenericProtocol now revealsF[T | None] rather thandef [T] (T|None) -> T|None4
  • testInferenceAgainstGenericParamSpecPopOn now revealsdef (U) -> U rather thandef [T] (T) -> T5

Footnotes

  1. This was needed to fixtestLiteralAndGenericWithUnion, because the outer solution wasint | Literal["foo"] whereas the joint solution wasLiteral["foo"]? which got converted intostr later.↩

  2. previously,a=b followed bya=[[b]] madea be inferred as__main__.B followed bybuiltins.list[Union[__main__.A, typing.Sequence[Union[__main__.A, ...]]]]. Now, it gets more precisely inferred asbuiltins.list[builtins.list[__main__.B]]. And afterwards,join(a,b) get more precisely inferred astyping.Sequence[typing.Sequence[__main__.B]] rather thantyping.Sequence[Union[__main__.A, typing.Sequence[Union[__main__.A, ...]]]] previously. See:https://mypy-play.net/?mypy=latest&python=3.12&gist=619a7fa6c3ec9d75bd1492bf8e68432c. The new result is also more in line with theinference by pyright↩

  3. same as pyright↩

  4. same as pyright↩

  5. same as pyright↩

lambda i: self.accept(args[i]),
)

# ??? QUESTION: Do we need to recompute arg_types and pass1_args here???
Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

It seems to work both with/without recomputingarg_types here, but I am not sure.

Comment on lines +2111 to +2125
# HACK: convert "Literal?" constraints to their non-literal versions.
inner_constraints: list[Constraint] = []
for constraint in _inner_constraints:
target = get_proper_type(constraint.target)
inner_constraints.append(
Constraint(
constraint.original_type_var,
constraint.op,
(
target.copy_modified(last_known_value=None)
if isinstance(target, Instance)
else target
),
)
)
Copy link
ContributorAuthor

@randolf-scholzrandolf-scholzJul 9, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This hack is needed fortestLiteralAndGenericWithUnion andtestLiteralMappingContext.

The only purpose is to convert constraints likeT ≷ Literal['foo"]? toT ≷ str, which, in these test cases, causes the joint solution to fail due to incompatible constraints likeT <: Literal["foo"] andT :> str, hence the longer route of using the outer solution first is used.

@github-actionsGitHub Actions
Copy link
Contributor

Diff frommypy_primer, showing the effect of this PR on open source code:

colour (https://github.com/colour-science/colour)+ colour/utilities/array.py:2428: error: Argument 1 to "__call__" of "_FloatOp" has incompatible type "floating[_16Bit]"; expected "integer[_32Bit] | floating[_32Bit]"  [arg-type]+ colour/utilities/array.py:2428: error: Argument 1 to "__call__" of "_FloatOp" has incompatible type "floating[_32Bit]"; expected "integer[_16Bit] | floating[_16Bit]"  [arg-type]+ colour/utilities/array.py:2428: note: Both left and right operands are unions+ colour/plotting/section.py:390: error: Argument 1 to "LineCollection" has incompatible type "ndarray[tuple[Any, ...], dtype[Any]]"; expected "Sequence[Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]]"  [arg-type]pydantic (https://github.com/pydantic/pydantic)- pydantic/aliases.py:29: error: Incompatible types in assignment (expression has type "list[str]", variable has type "list[int | str]")  [assignment]- pydantic/aliases.py:29: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance- pydantic/aliases.py:29: note: Consider using "Sequence" instead, which is covariant- pydantic/aliases.py:29: error: Argument 1 to "list" has incompatible type "tuple[str | int, ...]"; expected "Iterable[str]"  [arg-type]scikit-learn (https://github.com/scikit-learn/scikit-learn)- sklearn/linear_model/tests/test_ridge.py:744: error: Unsupported operand types for + ("list[None]" and "list[ABCMeta]")  [operator]operator (https://github.com/canonical/operator)- ops/model.py:2937: error: Argument 2 to "_list_recursive" of "Container" has incompatible type "str | Path"; expected "Path"  [arg-type]- ops/model.py:3020: error: Argument 2 to "_list_recursive" of "Container" has incompatible type "str | Path"; expected "Path"  [arg-type]- ops/_private/harness.py:4000: error: Argument 3 to "_notice_matches" of "_TestingPebbleClient" has incompatible type "set[NoticeType | str] | None"; expected "set[str] | None"  [arg-type]pandas (https://github.com/pandas-dev/pandas)+ pandas/core/arrays/datetimelike.py:2398: error: Unused "type: ignore" comment  [unused-ignore]+ pandas/core/arrays/sparse/accessor.py:443: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[ndarray[tuple[int], dtype[Any]]]")  [assignment]+ pandas/core/arrays/sparse/accessor.py:444: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[Any]")  [assignment]+ pandas/core/arrays/sparse/accessor.py:445: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[Any]")  [assignment]+ pandas/core/indexes/interval.py:801: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[ndarray[tuple[Any, ...], dtype[Any]] | ndarray[tuple[Any, ...], dtype[signedinteger[_32Bit | _64Bit]]]]")  [assignment]+ pandas/core/groupby/groupby.py:5650: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[Any]")  [assignment]+ pandas/io/formats/csvs.py:141: error: List item 0 has incompatible type "Hashable"; expected "str"  [list-item]+ pandas/tests/series/accessors/test_dt_accessor.py:441: error: Unused "type: ignore" comment  [unused-ignore]+ pandas/tests/scalar/timestamp/test_timestamp.py:124: error: Unused "type: ignore" comment  [unused-ignore]+ pandas/tests/indexes/datetimes/test_scalar_compat.py:141: error: Unused "type: ignore" comment  [unused-ignore]+ pandas/tests/dtypes/test_missing.py:806: error: Unused "type: ignore" comment  [unused-ignore]+ pandas/tests/dtypes/test_missing.py:828: error: Unused "type: ignore" comment  [unused-ignore]+ pandas/conftest.py:1664: error: Unused "type: ignore" comment  [unused-ignore]prefect (https://github.com/PrefectHQ/prefect)- src/prefect/utilities/callables.py:579: error: Incompatible types in assignment (expression has type "list[None]", variable has type "list[Optional[expr]]")  [assignment]- src/prefect/utilities/callables.py:579: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance- src/prefect/utilities/callables.py:579: note: Consider using "Sequence" instead, which is covariant- src/prefect/utilities/callables.py:581: error: Unsupported operand types for + ("list[None]" and "list[Optional[expr]]")  [operator]+ src/prefect/client/orchestration/__init__.py:1088: error: Function does not return a value (it only ever returns None)  [func-returns-value]artigraph (https://github.com/artigraph/artigraph)+ src/arti/graphs/__init__.py:181: error: Unused "type: ignore" comment  [unused-ignore]Expression (https://github.com/cognitedata/Expression)+ expression/collections/block.py:271: error: Incompatible return value type (got "int", expected "_TSourceSum | Literal[0]")  [return-value]+ expression/collections/block.py:271: error: Argument 1 to "sum" has incompatible type "tuple[_TSourceSum | Literal[0], ...]"; expected "Iterable[bool]"  [arg-type]+ expression/collections/block.py:934: error: Incompatible return value type (got "int", expected "_TSourceSum | Literal[0]")  [return-value]+ expression/collections/block.py:934: error: Argument 1 to "sum" has incompatible type "Block[_TSourceSum | Literal[0]]"; expected "Iterable[bool]"  [arg-type]jax (https://github.com/google/jax)+ jax/_src/numpy/lax_numpy.py:2224: error: Unsupported operand types for - ("Sequence[int | Any]" and "int")  [operator]+ jax/_src/numpy/lax_numpy.py:2224: note: Left operand is of type "Sequence[int | Any] | int | Any"hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)- src/hydra_zen/wrapper/_implementations.py:1756: error: Unsupported operand types for + ("list[None]" and "list[str]")  [operator]altair (https://github.com/vega/altair)+ tools/generate_schema_wrapper.py:488: error: Dict entry 0 has incompatible type "str": "OverridesItem[MethodSchemaGenerator]"; expected "str": "OverridesItem[SchemaGenerator]"  [dict-item]static-frame (https://github.com/static-frame/static-frame)+ static_frame/core/store.py:181: error: Argument 1 to "extend" of "list" has incompatible type "Sequence[TLabel]"; expected "Iterable[str]"  [arg-type]+ static_frame/core/store.py:181: error: Argument 1 to "extend" of "list" has incompatible type "Sequence[TLabel]"; expected "Iterable[tuple[str, ...]]"  [arg-type]+ static_frame/core/store.py:183: error: Argument 1 to "extend" of "list" has incompatible type "range"; expected "Iterable[str]"  [arg-type]+ static_frame/core/store.py:183: note: Following member(s) of "range" have conflicts:+ static_frame/core/store.py:183: note:     Expected:+ static_frame/core/store.py:183: note:         def __iter__(self) -> Iterator[str]+ static_frame/core/store.py:183: note:     Got:+ static_frame/core/store.py:183: note:         def __iter__(self) -> Iterator[int]+ static_frame/core/store.py:183: error: Argument 1 to "extend" of "list" has incompatible type "range"; expected "Iterable[tuple[str, ...]]"  [arg-type]+ static_frame/core/store.py:183: note: Following member(s) of "range" have conflicts:+ static_frame/core/store.py:183: note:     Expected:+ static_frame/core/store.py:183: note:         def __iter__(self) -> Iterator[tuple[str, ...]]+ static_frame/core/store.py:183: note:     Got:+ static_frame/core/store.py:183: note:         def __iter__(self) -> Iterator[int]steam.py (https://github.com/Gobot1234/steam.py)+ steam/abc.py:552: error: Value of type variable "AppT" of "FavouriteBadge" cannot be "PartialApp[str | None] | tuple[str, int]"  [type-var]- steam/state.py:1453: error: Argument 2 to "pop" of "dict" has incompatible type "None"; expected "ClanInvite | GroupInvite"  [arg-type]+ steam/state.py:1453: error: Incompatible types in assignment (expression has type "ClanInvite | GroupInvite | None", variable has type "ClanInvite | GroupInvite")  [assignment]core (https://github.com/home-assistant/core)+ homeassistant/helpers/service.py:982: error: Argument 1 to "HassJob" has incompatible type "Callable[[ServiceCall], Coroutine[Any, Any, ServiceResponse | EntityServiceResponse] | ServiceResponse | EntityServiceResponse | None]"; expected "Callable[[VarArg([ServiceCall] | [ServiceCall]), KwArg([ServiceCall] | [ServiceCall])], Coroutine[Any, Any, ServiceResponse | EntityServiceResponse] | JsonObjectType | EntityServiceResponse | JsonObjectType | EntityServiceResponse | None]"  [arg-type]+ homeassistant/components/bthome/coordinator.py:48: error: Returning Any from function declared to return "bool"  [no-any-return]+ homeassistant/components/xiaomi_ble/coordinator.py:70: error: Returning Any from function declared to return "bool"  [no-any-return]+ homeassistant/components/motioneye/media_source.py:96: error: Unused "type: ignore" comment  [unused-ignore]scipy-stubs (https://github.com/scipy/scipy-stubs)+ tests/sparse/test_construct.pyi:125: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[float64, tuple[int, int]]"  [assert-type]+ tests/sparse/test_construct.pyi:126: error: Expression is of type "csc_array[_ScalarT]", not "csc_array[float64]"  [assert-type]+ tests/sparse/test_construct.pyi:127: error: Expression is of type "csr_array[_ScalarT, tuple[int, int]]", not "csr_array[float64, tuple[int, int]]"  [assert-type]+ tests/sparse/test_construct.pyi:128: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[complex128, tuple[int, int]]"  [assert-type]+ tests/sparse/test_construct.pyi:129: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[complex128, tuple[int, int]]"  [assert-type]+ tests/sparse/test_construct.pyi:130: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[complex128, tuple[int, int]]"  [assert-type]spark (https://github.com/apache/spark)+ python/pyspark/ml/util.py:1166: error: Unused "type: ignore" comment  [unused-ignore]+ python/pyspark/ml/tuning.py:257: error: Value of type variable "JP" of "_from_java" of "JavaParams" cannot be "Estimator[Any]"  [type-var]+ python/pyspark/ml/tuning.py:440: error: Value of type variable "RL" of "loadParamsInstance" of "DefaultParamsReader" cannot be "Estimator[Any]"  [type-var]+ python/pyspark/ml/classification.py:3662: error: Value of type variable "JP" of "_from_java" of "JavaParams" cannot be "Classifier[Any]"  [type-var]+ python/pyspark/ml/classification.py:3920: error: Value of type variable "JP" of "_from_java" of "JavaParams" cannot be "Classifier[Any]"  [type-var]+ python/pyspark/ml/classification.py:3976: error: Unused "type: ignore" comment  [unused-ignore]antidote (https://github.com/Finistere/antidote)+ tests/lib/interface/test_custom.py:434: error: Unused "type: ignore" comment  [unused-ignore]+ tests/lib/interface/test_custom.py:437: error: Unused "type: ignore" comment  [unused-ignore]+ tests/lib/interface/test_custom.py:440: error: Unused "type: ignore" comment  [unused-ignore]+ tests/lib/interface/test_custom.py:443: error: Unused "type: ignore" comment  [unused-ignore]+ tests/lib/interface/test_custom.py:446: error: Unused "type: ignore" comment  [unused-ignore]+ tests/lib/interface/test_custom.py:449: error: Unused "type: ignore" comment  [unused-ignore]strawberry (https://github.com/strawberry-graphql/strawberry)+ strawberry/experimental/pydantic/error_type.py:118: error: Value of type variable "T" of "_wrap_dataclass" cannot be "WithStrawberryObjectDefinition"  [type-var]discord.py (https://github.com/Rapptz/discord.py)- discord/ext/commands/bot.py:1316: error: Argument 1 to "find" has incompatible type "Callable[[str], bool]"; expected "Callable[[list[str] | str], Any]"  [arg-type]schemathesis (https://github.com/schemathesis/schemathesis)+ src/schemathesis/specs/openapi/_hypothesis.py:397: error: Incompatible types in assignment (expression has type "SearchStrategy[dict[str, Any]]", variable has type "SearchStrategy[None]")  [assignment]- src/schemathesis/specs/openapi/_hypothesis.py:397: error: Argument 1 to "map" of "SearchStrategy" has incompatible type "Callable[[dict[str, Any]], dict[str, Any]]"; expected "Callable[[dict[str, Any]], None]"  [arg-type]+ src/schemathesis/specs/openapi/_hypothesis.py:399: error: Incompatible types in assignment (expression has type "SearchStrategy[dict[str, Any]]", variable has type "SearchStrategy[None]")  [assignment]- src/schemathesis/specs/openapi/_hypothesis.py:399: error: Argument 1 to "map" of "SearchStrategy" has incompatible type "Callable[[dict[str, Any]], dict[str, Any]]"; expected "Callable[[None], None]"  [arg-type]+ src/schemathesis/specs/openapi/_hypothesis.py:399: error: Argument 1 to "map" of "SearchStrategy" has incompatible type "Callable[[dict[str, Any]], dict[str, Any]]"; expected "Callable[[None], dict[str, Any]]"  [arg-type]xarray (https://github.com/pydata/xarray)+ xarray/core/dataset.py: note: In member "drop_dims" of class "Dataset":+ xarray/core/dataset.py:6108: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Iterable[str | None]"  [arg-type]+ xarray/core/dataset.py:6108: note: Left operand is of type "set[str] | set[Hashable]"+ xarray/core/dataset.py: note: At top level:+ xarray/core/dataarray.py: note: In function "_infer_coords_and_dims":+ xarray/core/dataarray.py:163: error: No overload variant of "__setitem__" of "list" matches argument types "int", "Hashable"  [call-overload]+ xarray/core/dataarray.py:163: note: Possible overload variants:+ xarray/core/dataarray.py:163: note:     def __setitem__(self, SupportsIndex, str, /) -> None+ xarray/core/dataarray.py:163: note:     def __setitem__(self, slice[Any, Any, Any], Iterable[str], /) -> None+ xarray/core/dataarray.py:186: error: Incompatible types in assignment (expression has type "Hashable", variable has type "str")  [assignment]+ xarray/core/dataarray.py: note: At top level:+ xarray/tests/test_dataarray.py:423: error: Unused "type: ignore" comment  [unused-ignore]zulip (https://github.com/zulip/zulip)+ zproject/config.py:38: error: Argument "fallback" to "get" of "RawConfigParser" has incompatible type "str | None"; expected "None"  [arg-type]sympy (https://github.com/sympy/sympy)+ sympy/core/add.py:384: error: Unused "type: ignore" comment  [unused-ignore]

@randolf-scholz
Copy link
ContributorAuthor

This still isn't quite correct; there are persisting issues with considering the outer context,

for example
fromtypingimportTypedDictclassA: ...classB(A): ...classOverridesItem[T](TypedDict):tp:type[T]d:dict[str,OverridesItem[A]]= {"foo":OverridesItem(tp=B)}

gives

error: Dict entry 0 has incompatible type "str": "OverridesItem[B]"; expected "str": "OverridesItem[A]"  [dict-item]

with the current version of the PR.

It seems that these issues stem from "solving" variables too eagerly.

In the example above,def [T] (*, tp: type[T`501]) -> TypedDict('tmp_b.OverridesItem', {'tp': type[T`501]}) yields the "solution"T=B, although the constraints only requireT :> B, which would allowT=A later on. So possibly the solution should be delayed until the interval(lower, upper) collapses to a single point.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
[8]ページ先頭
©2009-2025 Movatter.jp