Thanks for doing this! I was hoping this would get done for 3.13 but hadn't found time to do it yet.

I don't agree thattest_frame_locals intest_listcomps should be expected to fail under PEP 667.f_locals is supposed to be a real-time proxy for the live variables in the frame. Within a comprehension, comprehension-scoped variables are live and should be present inf_locals.The fact that this test is failing suggests that the PR does not yet correctly handle "fast-hidden" locals.

EDIT: on second thought, I realized the problem is likely thatf_locals is now a live view instead of a snapshot, so by the time we look for the key, the comprehension is no longer running, anda is no longer a live name. So you are right that the test as written should be expected to fail; it should be updated to take a copy off_locals within the comprehension.

EDIT: on second thought, I realized the problem is likely thatf_locals is now a live view instead of a snapshot, so by the time we look for the key, the comprehension is no longer running, anda is no longer a live name. So you are right that the test as written should be expected to fail; it should be updated to take a copy off_locals within the comprehension.

Yes, that's why the test fails. The test isval = [sys._getframe().f_locals for a in [0]][0]["a"] so when it tries to access["a"], it's already out of the comprehension scope. This actually represents one of the changes we made tof_locals.

I cleaned up the code a bit to address the review and added the GC part in. This is still in draft state, which only aims to demostrate the possibility of PEP 667. There's plenty of work to be done before this can be merged in. The current goal is to make it enough for the PSF to decide whether to accept PEP 667. Let me know if there's still missing pieces (like do we want to actually remove all the FAST/LOCAL conversion calls to prove the point).

Removing the fast locals <--> slow locals conversion is key to fixing#74929. I think that is worth doing before submitting PEP 667 to the SC.

@gaogaotiantian since you are doing most of the work on this, would you like to be added as co-author of PEP 667?

Sure, I'm happy to be listed as the co-author. I'll work on the fast vs local thing and check if#74929 is fixed.

FWIW the discussion has been started athttps://discuss.python.org/t/pep-667-consistent-views-of-namespaces/46631.

We now need at least the following updates to the PEP:

• Add Discussion-To header pointing to that Discourse thread
• Add Tian Gao to list of Authors
• Link to implementation prototype PR from the Implementation section

I've already added myself to the list of authors with Mark's guidance. Let me know if you want me to make the PR to update the PEP.

Go right ahead.

@markshannon okay I made all fast/local related functions no-op. It did not break anything new and#74929 (comment) is not an issue anymore. The original test case in#74929 has been accidentally fixed in 3.11, is there any repro case that I can test against?

Is there a reason you're leaving the failing tests as failing? I think it is easier for reviewers to evaluate the compatibility impact on tests by seeing the required changes to tests in the PR, rather than by having to go look at the CI test results.

The current implementation is just a prototype and far from being done. I can make some modifications to the test so they can pass with the change if that's preferred.

We have two more months until feature freeze. If we want to land this on 3.13, we probably should push it forward soon.

How is the Discourse discussion on this topic going? Is it substantial enough to send the PEP to the SC? Do we need to make any PEP updates first?

I made a PR to the PEP and it's approved but not merged (I think Mark has to review and approve it?), if you meant the link changes mentioned before. I'm not sure if the discussion thread attracts enough attention. What point should we get for the Disclosure discussion?

We need to decide which mapping methods we wantFrameLocalsProxy to support.
Listing all the methods onUserDict:

Dunder methods

• Comparison operators,__le__, etc.
• __or__,__ior__,__ror__
• __contains__
• __copy__
• __getitem__
• __setitem__
• __delitem__ (see comment below under normal methods)
• __format__
• __len__
• __iter__
• __getitem__
• __reversed__
• __sizeof__
• __repr__,__str__
• Pickling support,__reduce__, etc

I think we need to support all the above, except pickling, and that__copy__ should return a normal__dict__, not a shallow copy.

Normal methods

• clear,
• copy,
• get,
• items,
• keys,
• pop,
• popitem,
• setdefault,
• update,
• values

Methods that remove items are tricky as we cannot properly delete fast locals; we just set them toNone and emit a warning.
So, I think it best not to implementclear,pop, orpopitem.

We don't need to implement them all before submitting the PEP though, just decide on which methods to support.

Should we follow some standard for the methods we should implement? The original PEP mentioned that we should do aMutableMapping, which probably lists all the methods that need to be implement.

For the remove related methods - should the debugger be able to decref the local variable? Just likedel val. I think making that equivalent tof.locals.pop("val") is reasonable. Should we even emit a warning for that? Did we do that now? I think we emit a warning when we have to assignNone to unassigned local variables.

Okay I resolved most of the comments, and I believe there's one thing left here - comprehensions.

I'm just going to jump to my proposal and see if that makes sense. If not, we can discuss it more.

importsysdeff():x=1    [print(sys._getframe().f_locals)forain [0]]print([sys._getframe().f_localsforain [0]][0])print(sys._getframe().f_locals)f()

will print

# Inside comprehension, show hidden variable a and locals in the outside frame{'x': 1, 'a': 0}# f_locals from inside but read outside, no hidden variable, only locals.# This does not match the fiction because the variable should "exist" if it's a frame, but in reality it's gone{'x': 1}# f_locals outside, just local variables{'x': 1}

Notice all the values above are proxies.

importsysclassC:x=1    [print(sys._getframe().f_locals)forain [0]]print([sys._getframe().f_localsforain [0]][0])print(sys._getframe().f_locals)

will print (ignore__module__ and__qualname__)

# This is a proxy without class-level variable{'a': 0}# Empty proxy{}# A dict with class level variable{'x': 1}

An alternative would be:

# A dict, but you can't write to variable a to change the local value{'a': 0, 'x': 1}# A dict{'x': 1}# Still a dict{'x': 1}

The latter seems more consistent with the function level result, and it makeslocals() backwards compatible, but it creates a horrible exemption where thef_locals is not "write through".

Maybe we could stick to a simple rule here -f_locals will always return something that's write through.locals() always returndict(proxy) or the dict itself (in class/module level, outside of the comprehension).

However, this will break the backwards compatibility forlocals() in comprehension in class/module level.

I think that proposal looks reasonable.

I think your first proposal for the class behavior is better. Comprehensions in class scope cannot see class-level variables (as if the comprehension were still a function, since functions defined in a class scope can't see variables in the enclosing class scope either), so those class variables should not show up whenf_locals is accessed inside the comprehension.

I think your first proposal for the class behavior is better. Comprehensions in class scope cannot see class-level variables (as if the comprehension were still a function, since functions defined in a class scope can't see variables in the enclosing class scope either), so those class variables should not show up whenf_locals is accessed inside the comprehension.

But we will probably treat module-level the same as class-level. Do you think it's reasonable to have a module-level comprehension to return a proxy without the global variables?

edit: I guess that makes sense as well. Even in a function in a module,f_locals should not return any global variables.

Okay I finished the last piece and cleaned up the code a bit.

Now we have some very consistent behavior forf_locals andlocals().

locals() will be either

• dict(f_locals) for function scope frames andall comprehensions
• orf_locals for class and module frames

f_locals will be

• a proxy for function scope frames andall comprehensions (comprehension variables are not accessible outside of the comprehension)
• or a dict for class and module frames

f_locals willalways be write through, no exceptions.

This breaks the behavior introduced by pep 709 that[locals() for a in [0]] will include outside variables if it's under a module or class level scope, but I think this specific behavior is not desired nor intentional, it's a compromise to begin with. If we consider the comprehension as a "function", thelocals() should never include the outside variables.

locals() in a function scope will include the cell variables as expected.

Let me know if there are missing pieces that I don't know of.

FYI, docs are missing here:https://docs.python.org/3.13/c-api/reflection.html

The back-compat functions were causing a few destructors to not be called properly in three tests in pybind11; switching to the new functions fixed it, though. Not sure if that's expected. I believe it’s because the back compat function for eval frame locals is basically the same as the new function and returns a new reference.