Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-132042: Prebuild mro_dict for find_name_in_mro to speedup class creation#132618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
sergey-miryanov wants to merge22 commits intopython:main
base:main
Choose a base branch
Loading
fromsergey-miryanov:gh-132042-precalc-mro-dict

Conversation

sergey-miryanov
Copy link
Contributor

@sergey-miryanovsergey-miryanov commentedApr 16, 2025•
edited
Loading

This is one of the optimizations from#132156 that moved to separate PR.

All three optimizations from original PR give about 40% speedup on tests.
This optimization give about 15%-18% speedup.

Copy link
Member

@vstinnervstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

All optimizations give about 40% speedup on tests.

Would you mind to run benchmarks on this PR?

First, rebase the PR on the main branch to retrieve the "Do not lookup tp_dict each time to speedup class creation" change.

@sergey-miryanov
Copy link
ContributorAuthor

Would you mind to run benchmarks on this PR?

First, rebase the PR on the main branch to retrieve the "Do not lookup tp_dict each time to speedup class creation" change.

Yes, but I can do it on the weekends (at least I will try later today on my traveling laptop, but results will not be the same as from original PR).

@sergey-miryanov
Copy link
ContributorAuthor

sergey-miryanov commentedApr 18, 2025•
edited
Loading

@vstinner benchmark's results:

Details
+---------------------------------------------------------------+----------+------------------------+------------------------+| Benchmark                                                     | ref      | mro                    | mro2                   |+===============================================================+==========+========================+========================+| 1000-empty_cls                                                | 9.88 ms  | 10.5 ms: 1.06x slower  | 9.71 ms: 1.02x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_dunders                                         | 12.5 ms  | 13.8 ms: 1.10x slower  | not significant        |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                    | 12.8 ms  | 11.7 ms: 1.09x faster  | 10.7 ms: 1.20x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['A', 'B']                          | 14.1 ms  | 13.1 ms: 1.08x faster  | 12.3 ms: 1.15x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']               | 14.6 ms  | 11.9 ms: 1.23x faster  | 11.0 ms: 1.33x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                     | 16.1 ms  | 13.7 ms: 1.17x faster  | 12.9 ms: 1.25x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']            | 12.5 ms  | 11.2 ms: 1.11x faster  | 10.6 ms: 1.18x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                  | 14.2 ms  | 13.1 ms: 1.08x faster  | 12.5 ms: 1.13x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']   | 14.2 ms  | 11.8 ms: 1.20x faster  | 11.0 ms: 1.29x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']         | 16.0 ms  | 14.2 ms: 1.13x faster  | 13.1 ms: 1.22x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['Logger']                    | 12.2 ms  | 12.6 ms: 1.04x slower  | 11.7 ms: 1.04x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['Logger']                          | 13.7 ms  | 14.1 ms: 1.03x slower  | not significant        |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']           | 15.6 ms  | 12.9 ms: 1.20x faster  | 11.9 ms: 1.31x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['DatagramHandler']                 | 16.8 ms  | 14.8 ms: 1.13x faster  | 13.8 ms: 1.22x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']                 | 18.1 ms  | 14.3 ms: 1.27x faster  | 13.5 ms: 1.34x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['MagicMock']                       | 19.3 ms  | 17.2 ms: 1.12x faster  | 15.4 ms: 1.25x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                     | 23.0 ms  | 18.9 ms: 1.22x faster  | 17.1 ms: 1.35x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['Shelf']                           | 24.0 ms  | 21.1 ms: 1.14x faster  | 19.1 ms: 1.25x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['tuple']                     | 12.6 ms  | not significant        | 11.6 ms: 1.08x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['tuple']                           | 13.8 ms  | 14.3 ms: 1.04x slower  | 13.1 ms: 1.05x faster  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['dict']                      | 12.6 ms  | 14.2 ms: 1.13x slower  | 13.2 ms: 1.05x slower  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['dict']                            | 13.6 ms  | 15.6 ms: 1.15x slower  | 14.3 ms: 1.05x slower  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-empty_cls_with_bases-bases=['list']                      | 13.4 ms  | 15.2 ms: 1.13x slower  | 14.1 ms: 1.05x slower  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 1000-cls_with_bases-bases=['list']                            | 14.5 ms  | 16.9 ms: 1.16x slower  | 15.7 ms: 1.08x slower  |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls                                              | 985 ms   | 1.15 sec: 1.16x slower | not significant        |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_dunders                                       | 1.13 sec | 1.31 sec: 1.16x slower | 1.17 sec: 1.04x slower |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['A', 'B']                  | 1.29 sec | 1.20 sec: 1.08x faster | 1.06 sec: 1.22x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['A', 'B']                        | 1.42 sec | 1.37 sec: 1.03x faster | 1.24 sec: 1.15x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 1.48 sec | 1.20 sec: 1.23x faster | 1.10 sec: 1.34x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['A', 'B', 'D']                   | 1.64 sec | 1.39 sec: 1.18x faster | 1.28 sec: 1.28x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 1.23 sec | 1.14 sec: 1.08x faster | 1.05 sec: 1.18x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['A_dun', 'B_dun']                | 1.41 sec | 1.38 sec: 1.02x faster | 1.25 sec: 1.13x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 1.42 sec | 1.20 sec: 1.19x faster | 1.08 sec: 1.31x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 1.61 sec | 1.43 sec: 1.13x faster | 1.31 sec: 1.23x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['Logger']                  | 1.21 sec | 1.26 sec: 1.04x slower | 1.14 sec: 1.06x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['Logger']                        | 1.37 sec | 1.47 sec: 1.07x slower | not significant        |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['DatagramHandler']         | 1.54 sec | 1.29 sec: 1.19x faster | 1.20 sec: 1.28x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['DatagramHandler']               | 1.71 sec | 1.49 sec: 1.14x faster | 1.37 sec: 1.25x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['MagicMock']               | 1.80 sec | 1.47 sec: 1.23x faster | 1.33 sec: 1.35x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['MagicMock']                     | 1.94 sec | 1.66 sec: 1.16x faster | 1.53 sec: 1.26x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['Shelf']                   | 2.29 sec | 1.90 sec: 1.21x faster | 1.73 sec: 1.33x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['Shelf']                         | 2.42 sec | 2.17 sec: 1.12x faster | 1.99 sec: 1.22x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['tuple']                   | 1.25 sec | 1.27 sec: 1.02x slower | 1.16 sec: 1.08x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['tuple']                         | 1.40 sec | 1.43 sec: 1.02x slower | 1.32 sec: 1.06x faster |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['dict']                    | 1.25 sec | 1.41 sec: 1.13x slower | 1.29 sec: 1.03x slower |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['dict']                          | 1.39 sec | 1.58 sec: 1.14x slower | 1.45 sec: 1.04x slower |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-empty_cls_with_bases-bases=['list']                    | 1.32 sec | 1.50 sec: 1.13x slower | 1.42 sec: 1.07x slower |+---------------------------------------------------------------+----------+------------------------+------------------------+| 100000-cls_with_bases-bases=['list']                          | 1.47 sec | 1.67 sec: 1.14x slower | 1.58 sec: 1.07x slower |+---------------------------------------------------------------+----------+------------------------+------------------------+| Geometric mean                                                | (ref)    | 1.04x faster           | 1.14x faster           |+---------------------------------------------------------------+----------+------------------------+------------------------+

There are two mro column - for two runs. I believe results are not very stable due throttling (I ran benchmarks on macbook retina 2013 on windows, cpu - i5-4258U @ 2.40GHz)

@sergey-miryanov
Copy link
ContributorAuthor

@vstinner Please take a look.

@sergey-miryanov
Copy link
ContributorAuthor

sergey-miryanov commentedApr 20, 2025•
edited
Loading

Updated benchmarks (ran windows 11 x64 desktop, cpu- 11th Gen Intel(R) Core(TM) i5-11600K @ 3.90GHz):

Details
+---------------------------------------------------------------+---------+-----------------------+| Benchmark                                                     | ref     | mro                   |+===============================================================+=========+=======================+| 1000-empty_cls                                                | 4.41 ms | 4.04 ms: 1.09x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_dunders                                         | 5.03 ms | 4.74 ms: 1.06x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                    | 5.49 ms | 4.27 ms: 1.29x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                          | 6.00 ms | 4.90 ms: 1.23x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']               | 6.17 ms | 4.59 ms: 1.34x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                     | 6.71 ms | 5.18 ms: 1.30x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']            | 5.29 ms | 4.23 ms: 1.25x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                  | 5.99 ms | 4.96 ms: 1.21x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']   | 5.95 ms | 4.54 ms: 1.31x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']         | 6.86 ms | 5.29 ms: 1.30x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                    | 5.16 ms | 4.60 ms: 1.12x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Logger']                          | 5.86 ms | 5.28 ms: 1.11x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']           | 6.32 ms | 4.79 ms: 1.32x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']                 | 6.93 ms | 5.62 ms: 1.23x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']                 | 7.24 ms | 5.52 ms: 1.31x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                       | 7.87 ms | 6.20 ms: 1.27x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                     | 9.16 ms | 7.57 ms: 1.21x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                           | 9.84 ms | 8.52 ms: 1.15x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                     | 5.53 ms | 5.29 ms: 1.05x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                           | 6.17 ms | 6.00 ms: 1.03x faster |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                      | 5.48 ms | 5.69 ms: 1.04x slower |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['dict']                            | 6.14 ms | 6.43 ms: 1.05x slower |+---------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                      | 5.88 ms | 6.12 ms: 1.04x slower |+---------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['list']                            | 6.55 ms | 6.88 ms: 1.05x slower |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls                                              | 443 ms  | 430 ms: 1.03x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['A', 'B']                  | 547 ms  | 454 ms: 1.21x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['A', 'B']                        | 606 ms  | 527 ms: 1.15x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 615 ms  | 485 ms: 1.27x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['A', 'B', 'D']                   | 673 ms  | 558 ms: 1.20x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 531 ms  | 451 ms: 1.18x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['A_dun', 'B_dun']                | 605 ms  | 539 ms: 1.12x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 598 ms  | 489 ms: 1.22x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 671 ms  | 556 ms: 1.21x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['Logger']                  | 520 ms  | 472 ms: 1.10x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['Logger']                        | 588 ms  | 558 ms: 1.05x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['DatagramHandler']         | 630 ms  | 499 ms: 1.26x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['DatagramHandler']               | 693 ms  | 581 ms: 1.19x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['MagicMock']               | 721 ms  | 543 ms: 1.33x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['MagicMock']                     | 783 ms  | 609 ms: 1.29x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['Shelf']                   | 918 ms  | 758 ms: 1.21x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['Shelf']                         | 998 ms  | 843 ms: 1.18x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['tuple']                   | 552 ms  | 522 ms: 1.06x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['tuple']                         | 622 ms  | 593 ms: 1.05x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['dict']                    | 550 ms  | 558 ms: 1.01x slower  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['dict']                          | 616 ms  | 631 ms: 1.02x slower  |+---------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls_with_bases-bases=['list']                    | 595 ms  | 586 ms: 1.01x faster  |+---------------------------------------------------------------+---------+-----------------------+| 100000-cls_with_bases-bases=['list']                          | 656 ms  | 661 ms: 1.01x slower  |+---------------------------------------------------------------+---------+-----------------------+| Geometric mean                                                | (ref)   | 1.15x faster          |+---------------------------------------------------------------+---------+-----------------------+Benchmark hidden because not significant (1): 100000-cls_with_dunders

…hile type initialized- in the old realisation this exception swallowed and this base not checked  while finding in mro. so we don't change observed behavior with this  change.
@sergey-miryanov
Copy link
ContributorAuthor

Updated results
+-------------------------------------------------------------+---------+-----------------------+| Benchmark                                                   | ref2    | mro2                  |+=============================================================+=========+=======================+| 1000-empty_cls                                              | 4.41 ms | 4.11 ms: 1.07x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_dunders                                       | 5.08 ms | 4.86 ms: 1.05x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                  | 5.60 ms | 4.33 ms: 1.29x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                        | 6.14 ms | 5.03 ms: 1.22x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 6.31 ms | 4.65 ms: 1.36x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                   | 6.87 ms | 5.34 ms: 1.28x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 5.42 ms | 4.29 ms: 1.26x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                | 6.14 ms | 5.10 ms: 1.20x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 6.14 ms | 4.62 ms: 1.33x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 6.86 ms | 5.45 ms: 1.26x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                  | 5.33 ms | 4.67 ms: 1.14x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Logger']                        | 5.89 ms | 5.42 ms: 1.09x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']         | 6.46 ms | 4.84 ms: 1.33x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']               | 7.09 ms | 5.64 ms: 1.26x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']               | 7.38 ms | 5.22 ms: 1.41x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                     | 7.98 ms | 6.04 ms: 1.32x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                   | 9.26 ms | 7.27 ms: 1.27x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                         | 9.91 ms | 8.10 ms: 1.22x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                   | 5.60 ms | 5.07 ms: 1.10x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                         | 6.25 ms | 5.76 ms: 1.08x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                    | 5.58 ms | 5.45 ms: 1.02x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['dict']                          | 6.20 ms | 6.14 ms: 1.01x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                    | 5.99 ms | 5.85 ms: 1.03x faster |+-------------------------------------------------------------+---------+-----------------------+| 100000-empty_cls                                            | 442 ms  | 410 ms: 1.08x faster  |+-------------------------------------------------------------+---------+-----------------------+| Geometric mean                                              | (ref)   | 1.18x faster          |+-------------------------------------------------------------+---------+-----------------------+Benchmark hidden because not significant (1): 1000-cls_with_bases-bases=['list']

Copy link
Member

@vstinnervstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM. I just have a remark on a comment.

I'm surprised that the latest benchmark run no longer shows slower tests.

Co-authored-by: Victor Stinner <vstinner@python.org>
@sergey-miryanov
Copy link
ContributorAuthor

I'm surprised that the latest benchmark run no longer shows slower tests.

Yeah, ref2 became a bit slower than ref (ref is ran on main 3 days ago, ref2 on actual main).

Details
+-------------------------------------------------------------+---------+-----------------------+| Benchmark                                                   | ref     | ref2                  |+=============================================================+=========+=======================+| 1000-cls_with_dunders                                       | 5.03 ms | 5.08 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                  | 5.49 ms | 5.60 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                        | 6.00 ms | 6.14 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 6.17 ms | 6.31 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                   | 6.71 ms | 6.87 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 5.29 ms | 5.42 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                | 5.99 ms | 6.14 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 5.95 ms | 6.14 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                  | 5.16 ms | 5.33 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']         | 6.32 ms | 6.46 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']               | 6.93 ms | 7.09 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']               | 7.24 ms | 7.38 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                     | 7.87 ms | 7.98 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                   | 9.16 ms | 9.26 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                         | 9.84 ms | 9.91 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                   | 5.53 ms | 5.60 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                         | 6.17 ms | 6.25 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                    | 5.48 ms | 5.58 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['dict']                          | 6.14 ms | 6.20 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                    | 5.88 ms | 5.99 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['list']                          | 6.55 ms | 6.64 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+| Geometric mean                                              | (ref)   | 1.02x slower          |+-------------------------------------------------------------+---------+-----------------------+Benchmark hidden because not significant (4): 1000-empty_cls, 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'], 1000-cls_with_bases-bases=['Logger'], 100000-empty_cls

@sergey-miryanov
Copy link
ContributorAuthor

Merged with main due conflicts after#131174

@sergey-miryanov
Copy link
ContributorAuthor

Updated benchmarks with main (a bit slower again - ref3 today main)
+-------------------------------------------------------------+---------+-----------------------+-----------------------+| Benchmark                                                   | ref     | ref2                  | ref3                  |+=============================================================+=========+=======================+=======================+| 1000-empty_cls                                              | 4.41 ms | not significant       | 4.47 ms: 1.01x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_dunders                                       | 5.03 ms | 5.08 ms: 1.01x slower | 5.19 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                  | 5.49 ms | 5.60 ms: 1.02x slower | 5.69 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                        | 6.00 ms | 6.14 ms: 1.02x slower | 6.27 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 6.17 ms | 6.31 ms: 1.02x slower | 6.40 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                   | 6.71 ms | 6.87 ms: 1.02x slower | 7.03 ms: 1.05x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 5.29 ms | 5.42 ms: 1.02x slower | 5.52 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                | 5.99 ms | 6.14 ms: 1.02x slower | 6.29 ms: 1.05x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 5.95 ms | 6.14 ms: 1.03x slower | 6.34 ms: 1.07x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 6.86 ms | not significant       | 6.97 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                  | 5.16 ms | 5.33 ms: 1.03x slower | 5.38 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['Logger']                        | 5.86 ms | not significant       | 6.07 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']         | 6.32 ms | 6.46 ms: 1.02x slower | 6.58 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']               | 6.93 ms | 7.09 ms: 1.02x slower | 7.22 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']               | 7.24 ms | 7.38 ms: 1.02x slower | 7.48 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                     | 7.87 ms | 7.98 ms: 1.01x slower | 8.11 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                   | 9.16 ms | 9.26 ms: 1.01x slower | 9.46 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                         | 9.84 ms | 9.91 ms: 1.01x slower | 10.2 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                   | 5.53 ms | 5.60 ms: 1.01x slower | 5.64 ms: 1.02x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                         | 6.17 ms | 6.25 ms: 1.01x slower | 6.35 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                    | 5.48 ms | 5.58 ms: 1.02x slower | 5.68 ms: 1.04x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['dict']                          | 6.14 ms | 6.20 ms: 1.01x slower | 6.35 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                    | 5.88 ms | 5.99 ms: 1.02x slower | 6.04 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 1000-cls_with_bases-bases=['list']                          | 6.55 ms | 6.64 ms: 1.01x slower | 6.71 ms: 1.03x slower |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| 100000-empty_cls                                            | 443 ms  | not significant       | 449 ms: 1.02x slower  |+-------------------------------------------------------------+---------+-----------------------+-----------------------+| Geometric mean                                              | (ref)   | 1.02x slower          | 1.03x slower          |+-------------------------------------------------------------+---------+-----------------------+-----------------------+
This PR - 1.20x faster
+-------------------------------------------------------------+---------+-----------------------+| Benchmark                                                   | ref3    | mro3                  |+=============================================================+=========+=======================+| 1000-empty_cls                                              | 4.47 ms | 4.14 ms: 1.08x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_dunders                                       | 5.19 ms | 4.92 ms: 1.06x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                  | 5.69 ms | 4.43 ms: 1.29x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                        | 6.27 ms | 5.11 ms: 1.23x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 6.40 ms | 4.75 ms: 1.35x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                   | 7.03 ms | 5.41 ms: 1.30x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 5.52 ms | 4.36 ms: 1.27x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                | 6.29 ms | 5.17 ms: 1.22x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 6.34 ms | 4.72 ms: 1.34x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 6.97 ms | 5.51 ms: 1.27x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                  | 5.38 ms | 4.71 ms: 1.14x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Logger']                        | 6.07 ms | 5.47 ms: 1.11x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']         | 6.58 ms | 4.92 ms: 1.34x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']               | 7.22 ms | 5.69 ms: 1.27x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']               | 7.48 ms | 5.33 ms: 1.40x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                     | 8.11 ms | 6.07 ms: 1.34x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                   | 9.46 ms | 7.32 ms: 1.29x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                         | 10.2 ms | 8.12 ms: 1.26x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                   | 5.64 ms | 5.07 ms: 1.11x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                         | 6.35 ms | 5.72 ms: 1.11x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                    | 5.68 ms | 5.47 ms: 1.04x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['dict']                          | 6.35 ms | 6.12 ms: 1.04x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                    | 6.04 ms | 5.87 ms: 1.03x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['list']                          | 6.71 ms | 6.59 ms: 1.02x faster |+-------------------------------------------------------------+---------+-----------------------+| Geometric mean                                              | (ref)   | 1.20x faster          |+-------------------------------------------------------------+---------+-----------------------+

@sergey-miryanov
Copy link
ContributorAuthor

I also benchmarked with FT.

main vs main-ft (34% slower)
+-------------------------------------------------------------+---------+-----------------------+| Benchmark                                                   | ref4    | ref4-ft               |+=============================================================+=========+=======================+| 1000-empty_cls                                              | 4.57 ms | 6.41 ms: 1.40x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_dunders                                       | 5.17 ms | 7.50 ms: 1.45x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                  | 5.77 ms | 7.81 ms: 1.35x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                        | 6.40 ms | 8.29 ms: 1.30x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 6.52 ms | 8.88 ms: 1.36x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                   | 7.08 ms | 9.34 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 5.68 ms | 7.63 ms: 1.34x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                | 6.33 ms | 8.29 ms: 1.31x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 6.30 ms | 8.62 ms: 1.37x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 6.99 ms | 9.27 ms: 1.33x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                  | 5.55 ms | 7.34 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Logger']                        | 6.21 ms | 8.01 ms: 1.29x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']         | 6.72 ms | 8.89 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']               | 7.35 ms | 9.56 ms: 1.30x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']               | 7.65 ms | 10.1 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                     | 8.29 ms | 10.6 ms: 1.28x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                   | 9.55 ms | 12.6 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                         | 10.3 ms | 14.0 ms: 1.35x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                   | 5.70 ms | 7.72 ms: 1.35x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                         | 6.49 ms | 8.56 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                    | 5.72 ms | 7.54 ms: 1.32x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['dict']                          | 6.36 ms | 8.84 ms: 1.39x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                    | 6.07 ms | 7.96 ms: 1.31x slower |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['list']                          | 6.74 ms | 9.23 ms: 1.37x slower |+-------------------------------------------------------------+---------+-----------------------+| Geometric mean                                              | (ref)   | 1.34x slower          |+-------------------------------------------------------------+---------+-----------------------+
main-ft vs PR-ft (20% faster)
+-------------------------------------------------------------+---------+-----------------------+| Benchmark                                                   | ref4-ft | mro4-ft               |+=============================================================+=========+=======================+| 1000-empty_cls                                              | 6.41 ms | 5.73 ms: 1.12x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_dunders                                       | 7.50 ms | 6.86 ms: 1.09x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B']                  | 7.81 ms | 6.09 ms: 1.28x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B']                        | 8.29 ms | 6.95 ms: 1.19x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A', 'B', 'D']             | 8.88 ms | 6.62 ms: 1.34x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A', 'B', 'D']                   | 9.34 ms | 7.10 ms: 1.32x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun']          | 7.63 ms | 6.05 ms: 1.26x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun']                | 8.29 ms | 7.02 ms: 1.18x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun'] | 8.62 ms | 6.60 ms: 1.31x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['A_dun', 'B_dun', 'D_dun']       | 9.27 ms | 7.25 ms: 1.28x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Logger']                  | 7.34 ms | 6.22 ms: 1.18x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Logger']                        | 8.01 ms | 6.97 ms: 1.15x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['DatagramHandler']         | 8.89 ms | 6.61 ms: 1.35x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['DatagramHandler']               | 9.56 ms | 7.29 ms: 1.31x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['MagicMock']               | 10.1 ms | 6.89 ms: 1.46x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['MagicMock']                     | 10.6 ms | 7.68 ms: 1.38x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['Shelf']                   | 12.6 ms | 9.62 ms: 1.31x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['Shelf']                         | 14.0 ms | 11.1 ms: 1.25x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['tuple']                   | 7.72 ms | 6.79 ms: 1.14x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['tuple']                         | 8.56 ms | 7.62 ms: 1.12x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['dict']                    | 7.54 ms | 6.95 ms: 1.09x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['dict']                          | 8.84 ms | 8.40 ms: 1.05x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-empty_cls_with_bases-bases=['list']                    | 7.96 ms | 7.45 ms: 1.07x faster |+-------------------------------------------------------------+---------+-----------------------+| 1000-cls_with_bases-bases=['list']                          | 9.23 ms | 8.88 ms: 1.04x faster |+-------------------------------------------------------------+---------+-----------------------+| Geometric mean                                              | (ref)   | 1.21x faster          |+-------------------------------------------------------------+---------+-----------------------+

@Yhg1s
Copy link
Member

Yhg1s commentedApr 29, 2025•
edited
Loading

FYI, I've scheduled full benchmark runs (regular and free-threaded) onhttps://github.com/facebookexperimental/free-threading-benchmarking. (It'll be a couple of hours before they show up.)

I'm a little concerned about the need for the test change, but I guess it's rare enough to have custom__eq__/__hash__ on objectsused as keys in a class dictionary... as long as we're not backporting it to 3.13 :)

vstinner and sergey-miryanov reacted with thumbs up emoji

@sergey-miryanov
Copy link
ContributorAuthor

sergey-miryanov commentedApr 29, 2025•
edited
Loading

FYI, I've scheduled full benchmark runs (regular and free-threaded) onhttps://github.com/facebookexperimental/free-threading-benchmarking. (It'll be a couple of hours before they show up.)

Thanks for benchmarking. I'm looking forward to results :)

I'm a little concerned about the need for the test change, but I guess it's rare enough to have custom__eq__/__hash__ on objectsused as keys in a class dictionary... as long as we're not backporting it to 3.13 :)

IIUC,find_name_in_mro swallows this exception and this dict skipped while fixing dispatchers. If I swallow it when building MRO-dict without fixing test - then test passes. But IMHO we can't do it in general case (IIUC this is a@vstinner point too).

@Yhg1s
Copy link
Member

Basically no movement on any of the benchmarks:
https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20250429-3.14.0a7%2B-39f987b/bm-20250429-vultr-x86_64-sergey%252dmiryanov-gh_132042_precalc_mr-3.14.0a7%2B-39f987b-vs-base.md
https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20250429-3.14.0a7%2B-39f987b-NOGIL/bm-20250429-vultr-x86_64-sergey%252dmiryanov-gh_132042_precalc_mr-3.14.0a7%2B-39f987b-vs-base.md
https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20250429-3.14.0a7%2B-39f987b/bm-20250429-macm4pro-arm64-sergey%252dmiryanov-gh_132042_precalc_mr-3.14.0a7%2B-39f987b-vs-base.md
https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20250429-3.14.0a7%2B-39f987b-NOGIL/bm-20250429-macm4pro-arm64-sergey%252dmiryanov-gh_132042_precalc_mr-3.14.0a7%2B-39f987b-vs-base.md

(<1% geomean and <5% on individual benchmarks is within limits for noise. The slowdown in pidigits on linux may be real, but it's not a benchmark that does any type object creation, so I doubt it.)

@nascheme
Copy link
Member

This is an interesting optimization and worth considering IMHO. However, I am also concerned about unexpected behavioral differences caused by it. Doing a merge on each type dict is not the same has doing a dict lookup on each slot. As the broken unit test shows, you can have code that executes on the dict lookup and that can change the result of the operation (e.g. the hash method on a name value). To be fair, code that does that is probably "cursed" and deserves to break. However, I'd be a little worried about applying this optimization in the 3.14 release. Maybe we should defer and revisit in the 3.15 cycle?

@sergey-miryanov
Copy link
ContributorAuthor

sergey-miryanov commentedApr 30, 2025•
edited
Loading

(<1% geomean and <5% on individual benchmarks is within limits for noise. The slowdown in pidigits on linux may be real, but it's not a benchmark that does any type object creation, so I doubt it.)

Thanks! I'm crying! 😿

@sergey-miryanov
Copy link
ContributorAuthor

As the broken unit test shows, you can have code that executes on the dict lookup and that can change the result of the operation (e.g. the hash method on a name value). To be fair, code that does that is probably "cursed" and deserves to break.

Yeah, I agree with you. In the initial version I just stopped building of MRO-dict and fallback to original version offind_name_in_mro.@vstinner proposed to stop init of the type, so maybe it is worth to break such objects.

However, I'd be a little worried about applying this optimization in the 3.14 release. Maybe we should defer and revisit in the 3.15 cycle?

It is up to core-devs - I just write code a bit :)

@nascheme
Copy link
Member

nascheme commentedApr 30, 2025•
edited
Loading

Thanks! I'm crying! 😿

Don't cry, this is the nature of doing performance optimizations of Python. The low hanging fruit is pretty much gone and so optimizations are either quite complex or they only provide benefit to a specific subset of code. Just because the pyperformance benchmarks don't show a win, it doesn't mean this is not worth pursing. I'm sure some code out there does create more type objects and would see a significant benefit. So it's a matter of how costly the optimization is, in terms of code maintenance and risk of changing behavior (breaking currently working programs).

In this case, the maintenance cost looks pretty minor, the implementation is not complex. It's the change in behavior that's the concern and I think we are too close to the beta of 3.14 to risk it. Again, I think this is worth re-visiting and I hope you are not discouraged from looking for other kinds of optimizations.

sergey-miryanov, gpshead, and danielhollas reacted with heart emoji

@sergey-miryanov
Copy link
ContributorAuthor

@nascheme Thank you for kind words!

@sergey-miryanov
Copy link
ContributorAuthor

sergey-miryanov commentedMay 12, 2025•
edited
Loading

I have built table with results like in PR#132156.

The following tests callfixup_slot_dispatchers and may be affected by this PR.

testwarmup calls x slotdefs countloop calls x slotdefs countresult (base vs pr)
bench_mp_pool2701.03x faster (43.0 ms / 41.9 ms)
bench_thread_pool250not significant
deepcopy_reduce01not significant
docutils3470not significant
dulwich_log301.00x faster (19.1 ms / 19.0 ms)
genshi_text901.01x slower (11.4 ms / 11.5 ms)
genshi_xml90not significant
html5lib330not significant
pprint_pformat19601.01x faster (738 ms / 733 ms)
pylint8837not significant
regex_v8401.01x slower (9.94 ms / 10.0 ms)
sympy_expand330not significant
sympy_integrate3901.00x faster (8.16 ms / 8.14 ms)
sympy_sum391not significant
sympy_str300not significant
typing_runtime_protocols18701.01x slower (73.4 us / 74.3 us)
xml_etree_parse601.01x slower (56.4 ms / 56.8 ms)
xml_etree_iterparse26201.04x faster (37.8 ms / 36.3 ms)
xml_etree_generate601.00x faster (37.2 ms / 37.0 ms)
xml_etree_process601.00x faster (26.8 ms / 26.7 ms)

The only 'expected' 'faster' result is forxml_etree_iterparse but this is not stable. In another PR the result for this test is 'not significant'.

So, I repeat myself, IMHO this set of benchmarks is not suitable to catch differences from this PR.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@vstinnervstinnervstinner approved these changes

@markshannonmarkshannonAwaiting requested review from markshannonmarkshannon is a code owner

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@sergey-miryanov@Yhg1s@nascheme@vstinner
[8]ページ先頭
©2009-2025 Movatter.jp