I think we've changed the intent of the test here. It was previously checking thata was added to thef_locals snapshot during the listcomp, and now it's checking that it is not visible through the proxy afterwards.

To cover both aspects, I think we need two test cases (first one shows that"a" is accessible in thef_locals while the listcomp is running, second shows that it is gone afterwards):

deftest_frame_locals_in_listcomp(self):# Iteration variable is accessible via f_locals proxy while the listcomp is runningcode="""            val = [sys._getframe().f_locals["a"] for a in [0]][0]        """importsysself._check_in_scopes(code, {"val":0},ns={"sys":sys})deftest_frame_locals_after_listcomp(self):# Iteration variable is no longer accessible via f_locals proxy after listcomp finishescode="""            val = "a" in [sys._getframe().f_locals for a in [0]][0]        """importsysself._check_in_scopes(code, {"val":False},ns={"sys":sys})

That's a very interesting point because it introduced a new issue - should we include the hidden fast local inf_locals when it's module/class scope?

val= [sys._getframe().f_locals["a"]forain [0]][0]

What should thesys._getframe().f_locals inside the list comprehension return? It's it a dict or a proxy? Do we consider that "within" the function scope and return a proxy? Will that include the module-level variables as well? Do we return a dict? Should we include variablea? If so, writing to the key'a' won't have any effects because it's a fast variable, how do we deal with that inconsistency?

@markshannon

Hidden fast-locals were designed to mimic the prior behavior when comprehensions were previously implemented as one-shot functions. So as much as is feasible, the behavior off_locals with hidden fast-locals should mirror how it would behave if the comprehension were a function.

The second proposed test in@ncoghlan 's comment is not correct. Nor is the test as currently modified in the PR.

If you accessf_locals on a frame you got while inside the comprehension, it should include the hidden fast-local (i.e. the comprehension iteration variable), and that variable should still be present (in thef_locals of the frame you got while inside the comprehension) even after the comprehension is done. This behavior is the same as getting a frame (or itsf_locals) inside a function and returning it from the function. This is the current behavior inmain, and this behavior is important to keep in PEP 667.

Of course if you access the frame outside the comprehension, the comprehension's hidden fast locals should not be present in itsf_locals.

Currently in main, module globals are present in thef_locals of a frame accessed inside a module-level comprehension. This differs from a function, but this difference was called out in PEP 709 and determined to be OK, and I think it's still OK.

writing to the key 'a' won't have any effects because it's a fast variable

I suspect this is also not a problem in practice, though if we can reasonably return a proxy that can support it, it would be even better.

If you access f_locals on a frame you got while inside the comprehension, it should include the hidden fast-local (i.e. the comprehension iteration variable), and that variable should still be present (in the f_locals of the frame you got while inside the comprehension) even after the comprehension is done.

I don't think this is correct.f_locals is a proxy, so as the frame changes, so will the proxy. Inside the comprehension, the comprehension variables will be visible. After the comprehension has executed, the comprehension variables no longer exist.

I think the key here is "as if it's a one-shot function". That's infeasible with the current compiler because it's not a function anymore. For a function, we have a dedicated frame object that keeps the local variables, which can last longer than the intepreter frame as long as there are references on it. In that way, we can always access the variables on that frame even when the actual interpreter frame is gone.

However, with inline comprehension, it fakes the "function call" and clear the hidden variable - there's no mechanism currently to prevent the variable from being cleared. If we want this, we'll probably need to change the compiler and the interpreter which will definitely postpone this PEP to 3.14 (and would probably make the compiler code more complicated).

For class and module level comprehensions, can we return a proxy iff_locals is accessed inside the comprehension and a dict if it's accessed outside which does not have the hidden variable? I feel like that's more consistent.

I don't think this is correct. f_locals is a proxy, so as the frame changes, so will the proxy. Inside the comprehension, the comprehension variables will be visible. After the comprehension has executed, the comprehension variables no longer exist.

It depends whether we are aiming for "correctness" for the actual situation (that the comprehension doesn't have its own frame), or for the backwards-compatible fiction (which PEP 709 only partially maintained, but did jump through hoops to maintain in terms of visibility of class-scoped variables) that comprehensions still behave as if they have their own frame. But as@gaogaotiantian points out, it may be very hard or impossible to maintain that fiction in this case, while still havingf_locals be a proxy.

TBH in the long term I think it would be better to move away from that fiction anyway, but it will have backwards-compatibility implications. Today inmain,sys._getframe.f_locals() inside a module-level comprehension returns a dictionary that includes the comprehension iteration variable, and that variable is still visible in that dictionary after the comprehension is done.

I don't know why someone relying on that couldn't just uselocals() instead, though, andlocals() should be fine for backwards-compatibility; it remains a dict. So I don't know how significant that backwards-incompatibility will be in practice.

For class and module level comprehensions, can we return a proxy if f_locals is accessed inside the comprehension and a dict if it's accessed outside which does not have the hidden variable? I feel like that's more consistent.

Certainly the dict returned whenf_locals is accessed outside the comprehension should not have the hidden local in it.

I think returning a proxy inside the module- or class-scoped comprehension is more internally consistent for PEP 667; returning a dict (that includes the comprehension iteration variable) would be a bit more backward-compatible.

I'm not sure I follow the details, but I agree that people who accessf_locals are better off being shown implementation details that may vary by Python version than a costly fiction.