Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-104357: fix inlined comprehensions that close over iteration var#104368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
JelleZijlstra merged 2 commits intopython:mainfromcarljm:argcell
May 11, 2023

Conversation

carljm
Copy link
Member

@carljmcarljm commentedMay 10, 2023
edited
Loading

This fixes the case where an inlined comprehension has an internal cellvar (i.e. it creates lambdas that close over the comprehension iteration variable), but the same name is also used a fast-local in the outer scope, and is read but never written within the body of the function (so e.g. an argument).

In code like this:

def f(x):    items = [lambda: x for x in range(2)]    return x

becausex is a cell inside the comprehension, after inlining it would appear inu_cellvars forf, and thus the compilation off would insert aMAKE_CELL forx at the start of the function. But we would still compile the non-comprehension part of the function treatingx as a fast-local (emittingLOAD_FAST for it), sof would return a cell object containing the value ofx, instead of returningx directly.

Similar cases were already tested, but only with a write (compiled asSTORE_FAST) tox before it was referenced, which stored the value directly and overwrote the wrongly-created cell.

One approach to fix this would be to track which names are cells only inside an inlined comprehension, and prevent the compiler from emittingMAKE_CELL for those names (the inlined comprehension will itself emitMAKE_CELL where it should, within the comprehension's isolated scope.) But this fix would add additional complexity to the compiler.

A simpler fix (implemented here) is to ensure that we always treat such a variable as a cell throughout the outer function too (emittingLOAD_DEREF andSTORE_DEREF for it). This maintains clearer semantics forco_cellvars (if a name is in there, it is a cell in the main function) and keeps the compiler simpler. The downside is that cells are slightly slower than fast locals, but comprehensions creating lambdas that close over the iteration variable are rare in real code (they don't behave as people usually want them to, given that all the created lambda functions share the same final value in their closure), so I don't think performance is a concern in this edge case.

Fixes#104357

@carljmcarljm changed the titlegh-104357: fix inlined comprehensions with cellsgh-104357: fix inlined comprehensions that close over iteration varMay 10, 2023
@JelleZijlstraJelleZijlstra merged commitfcd5fb4 intopython:mainMay 11, 2023
carljm added a commit to carljm/cpython that referenced this pull requestMay 11, 2023
* main: (27 commits)pythongh-87849: fix SEND specialization family definition (pythonGH-104268)pythongh-101819: Adapt _io.IOBase.seek and _io.IOBase.truncate to Argument Clinic (python#104384)pythongh-101819: Adapt _io._Buffered* methods to Argument Clinic (python#104367)pythongh-101819: Refactor `_io` futher in preparation for module isolation (python#104369)pythongh-101819: Adapt _io.TextIOBase methods to Argument Clinic (python#104383)pythongh-101117: Improve accuracy of sqlite3.Cursor.rowcount docs (python#104287)pythonGH-92184: Convert os.altsep to '/' in filenames when creating ZipInfo objects (python#92185)pythongh-104357: fix inlined comprehensions that close over iteration var (python#104368)pythonGH-90208: Suppress OSError exceptions from `pathlib.Path.glob()` (pythonGH-104141)pythonGH-102181: Improve specialization stats for SEND (pythonGH-102182)pythongh-103000: Optimise `dataclasses.asdict` for the common case (python#104364)pythongh-103538: Remove unused TK_AQUA code (pythonGH-103539)pythonGH-87695: Fix OSError from `pathlib.Path.glob()` (pythonGH-104292)pythongh-104263: Rely on Py_NAN and introduce Py_INFINITY (pythonGH-104202)pythongh-104010: Separate and improve docs for `typing.get_origin` and `typing.get_args` (python#104013)pythongh-101819: Adapt _io._BufferedIOBase_Type methods to Argument Clinic (python#104355)pythongh-103960: Dark mode: invert image brightness (python#103983)pythongh-104252: Immortalize Py_EMPTY_KEYS (pythongh-104253)pythongh-101819: Clean up _io windows console io afterpythongh-104197 (python#104354)pythongh-101819: Harden _io init (python#104352)  ...
carljm added a commit to carljm/cpython that referenced this pull requestMay 11, 2023
* main:pythongh-87849: fix SEND specialization family definition (pythonGH-104268)pythongh-101819: Adapt _io.IOBase.seek and _io.IOBase.truncate to Argument Clinic (python#104384)pythongh-101819: Adapt _io._Buffered* methods to Argument Clinic (python#104367)pythongh-101819: Refactor `_io` futher in preparation for module isolation (python#104369)pythongh-101819: Adapt _io.TextIOBase methods to Argument Clinic (python#104383)pythongh-101117: Improve accuracy of sqlite3.Cursor.rowcount docs (python#104287)pythonGH-92184: Convert os.altsep to '/' in filenames when creating ZipInfo objects (python#92185)pythongh-104357: fix inlined comprehensions that close over iteration var (python#104368)pythonGH-90208: Suppress OSError exceptions from `pathlib.Path.glob()` (pythonGH-104141)
@carljmcarljm deleted the argcell branchJune 16, 2023 17:41
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@JelleZijlstraJelleZijlstraJelleZijlstra approved these changes

Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Comprehension inlining: Bug if comprehension contains a lambda
3 participants
@carljm@JelleZijlstra@bedevere-bot

[8]ページ先頭

©2009-2025 Movatter.jp