Closure on pdb has been an issue for years, and is often considered "to difficult to fix". PEP709 alleviated the issue by inlining all list, dict and set comprehensions, but the fundamental issue is still there - you can easily reproduce the issue with lambda functions ((lambda: x)()) or generators (any(x for x in lst if x != z)). Now thatpdb supports multi-line statement, you can even create your own function scope. Neither of them work in currentpdb.

The related (open) issues I can find are#83151,#80052,#76987,#70260 and#65360.

This PR fixes all the issues mentioned above, while keep (almost?) all the current correct behaviors.

Boiler alert, black magic involved.

The fundamental reason for the issue above, is that when python compiles code that generates nested functions, the inner function has no idea about the context. For example:

deff():z=1any([xforxinrange(2)ifx!=z])

The code above works fine because python compilesany([x for x in range(2) if x != z]) with the awareness thatz is a local variable off, so closure should be used. However, inpdb, if you execute/compileany([x for x in range(2) if x != z]), there's no way for Python to know thatz is an outer local variable and should be passed as a cell variable.

So, the solution is - trick the compiler.

What we can do, is to force compiler to compile the code we need in a closure-awareness environment. Take the code above as an example, instead of compilingany([x for x in range(2) if x != z]), we compile the following code:

defouter():z=Nonedef__pdb_scope():nonlocalzany([xforxinrange(2)ifx!=z])

Then we get the code object ofouter.__pdb_scope, and that is the code object we actually want - it considersz as a free variable. Then, we pass the value ofz withclosure argument ofexec:

exec(code,globals,locals,closure=(types.CellType(1),))

The solution is not done, as we still need to write the local variables back (locals won't be changed as all the variables are considered freevar). Also, it's possible for the code itself to create new variables and we want that in locals as well (k=1 for example). So, we just write all the local variables in the scope back by adding some new code at the end of the code.

To keep the original behavior as close as possible, if any exception is raised in the new method, we fallback to the original simpleexec solution (it might be a valid exception, but we will always have the same exception as before).

Why not ...

• just combineglobals() andlocals() as the global namespace?
  • result ofglobals() would be wrong
  • result ofglobal x; print(x) would be wrong ifx appears in both global and local namespace (if local is preferred, same issue if global is preferred).
  • impossible to do writeback correctly
• use the new method only?
  • the new method will give different exception on some cases (UnboundLocalError vsNameError)
  • the new method does not work well withglobal x; print(x) asx would be defined as bothglobal andnonlocal (it will fallback to the original method and work)

This is not the perfect solution, but it's pretty close. It passed all the existing tests and fixed all the cases in the issues mentioned above. I have not came up with an example that does not work with this solution (I would guess there will be some dark corners).

Yes, I know this is complicated, and it kind of depends on the implementation of CPython, but I still believe we should do this, because this issue has been repeatedly raised by users ofpdb for probably > 10 years.

• Issue:[PDB] NameError in list comprehension in PDB #83151