Feature or enhancement

Proposal:

I had a few proposals to utilizesys.monitoring for pdb and all of them were raising concerns. We had a discussion during language summit and I got feedbacks from several members that breaking backwards compatibility for bdb is not acceptible. I can totally understand the concern so I have a much more conservative proposal to do this. I would like to get opinions from people before start working on it.

TL; DR:

1. All existing code will work exactly as before.
2. sys.monitoring will be an opt-in feature for bdb.
3. Minimum changes are needed for existing debugger (including pdb) to onboard.

Details:

What I propose, is to keep all the code for bdb as it is, and add extra code forsys.monitoring without disturbing the existing methods. Everything will be hidden behind a feature gate so all the existing code will work exactly as before.

In order to usesys.monitoring instead ofsys.settrace, the user needs to initbdb.Bdb with an argument (use_monitoring=True?). Then the underlyingbdb.Bdb will work insys.monitoring mode. Ideally, that's theonly change the debugger needs to make.

Of course, in reality, if the debugger wants to onboard this feature, it may need a few tweaks. For example, in pdb,debug command togglessys.settrace to make it possible to debug something while tracing, that needs to be modified. However, the goal is to make it trivial for debugger developers to switch fromsys.settrace tosys.monitoring, if they knew howsys.monitoring works.

Let me re-emphasize it in case that's still a confusion - there's nothing the developer needs to do, if they just want the oldsys.settrace, everything will simply work because all the new stuff will be behind a feature gate.

If they chose to usesys.monitoring, there will be a few APIs inbdb.Bdb that does not even make sense anymore -trace_dispatch anddispatch_* functions. The documentation already says:

The following methods ofBdb normally don’t need to be overridden.

and a normal debugger should not override those methods anyway (pdb does not). Again, those APIs will still work insys.settrace mode.

As for pdb, we can also add a feature gate for it. The user can choose between thesys.settrace mode and thesys.monitoring mode. The behaviors in two modes should be identical, except for the performance. We can even make 3.14 a transition version, where the default mechanism is stillsys.settrace, and the user can opt-in thesys.monitoring mode by explicitly asking for it (through initialization argument or command line argument). This way, we can get feedbacks from the brave pioneers without disturbing most pdb users. We will have a full year to fix bugs introduced by the mechanism and stablize it.

In 3.15, we can makesys.monitoring the default and still keepsys.settrace as a fallback plan.

So, why bother? Because we can really gain 100x performance with breakpoints. Not only with breakpoints, even without breakpoints, there's a significant overhead to run with debugger attached:

deftrace(*args):returnNonedeff():start=time.perf_counter()fib(22)print(time.perf_counter()-start)f()

The overhead with trace attached is 4x-5x forf() because thecall event will still be triggered and even iff_trace == None, the instrumentation is still there and will be executed! We can have an almost zero-overhead debugger and people are very excited about the possibility.

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

#103103
https://discuss.python.org/t/make-pdb-faster-with-pep-669/37674

Linked PRs

• gh-120144: Make it possible to usesys.monitoring for bdb and make it default for pdb #124533
• gh-120144: Disable the CALL event when possible to achieve zero overhead pdb #131390
• gh-120144: Refactor bdb monitoring backend to match settrace behavior #132484