Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
Description
Feature or enhancement
Replace the currentsetprofile mechanism with PEP 669 API for cProfile.
Pitch
It's faster.
Before:
Hanoi: 0.342869599997357, 1.4579414000036195, 4.252174587699983fib: 0.15549560000363272, 0.7588815999988583, 4.88040561907301Hanoi: 0.33156009999947855, 1.394542599999113, 4.2060024713507635fib: 0.16397210000286577, 0.7469802000050549, 4.555532313070332Hanoi: 0.3341937000004691, 1.394005099995411, 4.171248889471747fib: 0.15704140000161715, 0.7399598000047263, 4.711877250184387Hanoi: 0.33133889999589883, 1.3821628000005148, 4.171447421409387fib: 0.15387790000386303, 0.754370700000436, 4.902397940064804Hanoi: 0.3403002000050037, 1.3969628000049852, 4.105089564991276fib: 0.15468130000226665, 0.761441399998148, 4.922646758121312After:
Hanoi: 0.3318088000014541, 1.2147841000041808, 3.661096691826309fib: 0.15522490000148537, 0.6360437999974238, 4.097562955372092Hanoi: 0.33085879999998724, 1.1502422000048682, 3.476535005279934fib: 0.15410060000431258, 0.6056611999956658, 3.9302974808580635Hanoi: 0.35002540000277804, 1.145935599997756, 3.273864125256799fib: 0.1540030999967712, 0.5974198000039905, 3.8792712615299036Hanoi: 0.3338971999983187, 1.1459307999975863, 3.431986851052829fib: 0.16891020000184653, 0.6197690999979386, 3.6692224625343126Hanoi: 0.3318254999976489, 1.1875411000000895, 3.578812056362467fib: 0.1544136999946204, 0.5971600999982911, 3.86727408266944920%+ speed up for overhead.
I guess the incentive is not in doubt, but I did have some issues when I implemented it.
- There is a very slight backward compatibility issue, or it's more like a design decision. The profiler has to disable its own profiling now.
# This works on main, but I don't think that's intuitive.pr=cProfile.Profile()pr.enable()pr=cProfile.Profile()pr.disable()
We can make it work as before, I just think this is not the right way to do it with PEP 669. Because of this, I changed an old (15 years) test intest_cprofile.
We need to get the actual c function from the descriptor, for which I used the code in the current legacy tracing. However,
_PyInstrumentation_MISSINGis not exposed so I had to store it locally (keep reading it fromsysis a bit expensive).On that matter, are we going to expose some APIs on C? It would be nice if I don't have to get
sys.monitoringand do stuff from there. We have some defined constants but some APIs could be handy. We may be able to reduce the overhead a bit if we have an interface likePyEval_SetProfile.
Addendum:
Benchmark Code
importtimeithanoi_setup="""import cProfiledef test(): def TowerOfHanoi(n, source, destination, auxiliary): if n == 1: return TowerOfHanoi(n - 1, source, auxiliary, destination) TowerOfHanoi(n - 1, auxiliary, destination, source) TowerOfHanoi(16, "A", "B", "C")pr = cProfile.Profile()"""fib_setup="""import cProfiledef test(): def fib(n): if n <= 1: return 1 return fib(n - 1) + fib(n - 2) fib(21)pr = cProfile.Profile()"""test_baseline="""test()"""test_profile="""pr.enable()test()pr.disable()"""baseline=timeit.timeit(test_baseline,setup=hanoi_setup,number=100)profile=timeit.timeit(test_profile,setup=hanoi_setup,number=100)print(f"Hanoi:{baseline},{profile},{profile/baseline}")baseline=timeit.timeit(test_baseline,setup=fib_setup,number=100)profile=timeit.timeit(test_profile,setup=fib_setup,number=100)print(f"fib:{baseline},{profile},{profile/baseline}")