Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-119258: Eliminate Type Guards in Tier 2 Optimizer with Watcher#119365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation

@saulshanabrook
Copy link
Contributor

@saulshanabrooksaulshanabrook commentedMay 21, 2024
edited by bedevere-appbot
Loading

This PR replaces#119259 to more accurately remove_GUARD_TYPE_VERSION.

With help and guidance from@brandtbucher, I added a global type version watcher to the interpreter, so that we can look at types by their versions.

Then, whenever we see a_GUARD_TYPE_VERSION we can add a watcher to that type, and if it isn't changed, the next time we see that type version we can omit the guard.

We should see if the performance is improved compared to the other PR, or at least see if it removes more guard type versions.

@brandtbucher
Copy link
Member

Looks like a print statement was left in there somewhere. I can try rerunning the benchmarks once you remove it!

@saulshanabrook
Copy link
ContributorAuthor

Hey@brandtbucher I may have removed it yesterday but just after you started the benchmark. I looked through the current diff and didn't see anymore. Thanks! Sorry about that.

to match `typ_version_tag`suggested by@brandtbucher
As per comment by@brandtbucher:> One that performs the mutation partway through execution like this. However, maybe change Foo.attr to 2 so we know it keeps running, but can see the effect of the changed value? I'd also make xxx just another class Bar, since I'm worried that our function watcher branch might invalidate this or something weird since it's just a function.
@saulshanabrook
Copy link
ContributorAuthor

Thank you@brandtbucher for the review! I resolved most of the comments and left a few responses for ones I was unclear about.

@saulshanabrooksaulshanabrook marked this pull request as draftJune 4, 2024 16:15
@brandtbucher
Copy link
Member

(You can’t assert it like that, since everything inside the assert is removed in debug builds.)

saulshanabrook reacted with thumbs up emoji

@brandtbucher
Copy link
Member

brandtbucher commentedJun 4, 2024
edited
Loading

Heh, now you’ll get an unused variable warning forres on non-debug builds.

It’s probably fine to just omit the assert. If you want to keep it, adding a(void)res; statement should silence the warning.

saulshanabrook reacted with thumbs up emoji

@saulshanabrook
Copy link
ContributorAuthor

I am not sure if the JIT aarch64 Linux failure is real or spurious:

2024-06-04T20:13:38.7145727Z ======================================================================2024-06-04T20:13:38.7146555Z ERROR: test_111615 (test.test_logging.ConfigDictTest.test_111615)2024-06-04T20:13:38.7147595Z ----------------------------------------------------------------------2024-06-04T20:13:38.7148245Z Traceback (most recent call last):2024-06-04T20:13:38.7231802Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 577, in configure2024-06-04T20:13:38.7233206Z     handler = self.configure_handler(handlers[name])2024-06-04T20:13:38.7234429Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 785, in configure_handler2024-06-04T20:13:38.7235394Z     proxy_queue = MM().Queue()2024-06-04T20:13:38.7236366Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 729, in temp2024-06-04T20:13:38.7237441Z     token, exp = self._create(typeid, *args, **kwds)2024-06-04T20:13:38.7238127Z                  ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^2024-06-04T20:13:38.7239281Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 609, in _create2024-06-04T20:13:38.7240419Z     conn = self._Client(self._address, authkey=self._authkey)2024-06-04T20:13:38.7241640Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 525, in Client2024-06-04T20:13:38.7242727Z     answer_challenge(c, authkey)2024-06-04T20:13:38.7243207Z     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^2024-06-04T20:13:38.7244264Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 953, in answer_challenge2024-06-04T20:13:38.7245667Z     message = connection.recv_bytes(256)         # reject large message2024-06-04T20:13:38.7247084Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 216, in recv_bytes2024-06-04T20:13:38.7248261Z     buf = self._recv_bytes(maxlength)2024-06-04T20:13:38.7251079Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 430, in _recv_bytes2024-06-04T20:13:38.7252234Z     buf = self._recv(4)2024-06-04T20:13:38.7253268Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 395, in _recv2024-06-04T20:13:38.7254313Z     chunk = read(handle, remaining)2024-06-04T20:13:38.7255282Z ConnectionResetError: [Errno 104] Connection reset by peer2024-06-04T20:13:38.7255840Z 2024-06-04T20:13:38.7256250Z The above exception was the direct cause of the following exception:2024-06-04T20:13:38.7256853Z 2024-06-04T20:13:38.7257038Z Traceback (most recent call last):2024-06-04T20:13:38.7258057Z   File "/home/runner/work/cpython/cpython/Lib/test/test_logging.py", line 4003, in test_1116152024-06-04T20:13:38.7259102Z     logging.config.dictConfig(config)2024-06-04T20:13:38.7259653Z     ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^2024-06-04T20:13:38.7260640Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 911, in dictConfig2024-06-04T20:13:38.7261682Z     dictConfigClass(config).configure()2024-06-04T20:13:38.7262245Z     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^2024-06-04T20:13:38.7263226Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 584, in configure2024-06-04T20:13:38.7264604Z     raise ValueError('Unable to configure handler '2024-06-04T20:13:38.7265383Z                      '%r' % name) from e2024-06-04T20:13:38.7266093Z ValueError: Unable to configure handler 'sink'2024-06-04T20:13:38.7266557Z 2024-06-04T20:13:38.7266775Z ======================================================================2024-06-04T20:13:38.7267777Z ERROR: test_config_queue_handler (test.test_logging.ConfigDictTest.test_config_queue_handler)2024-06-04T20:13:38.7268927Z ----------------------------------------------------------------------2024-06-04T20:13:38.7269587Z Traceback (most recent call last):2024-06-04T20:13:38.7270579Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 577, in configure2024-06-04T20:13:38.7271664Z     handler = self.configure_handler(handlers[name])2024-06-04T20:13:38.7273086Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 785, in configure_handler2024-06-04T20:13:38.7274148Z     proxy_queue = MM().Queue()2024-06-04T20:13:38.7275160Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 729, in temp2024-06-04T20:13:38.7276286Z     token, exp = self._create(typeid, *args, **kwds)2024-06-04T20:13:38.7276947Z                  ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^2024-06-04T20:13:38.7278049Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 609, in _create2024-06-04T20:13:38.7279252Z     conn = self._Client(self._address, authkey=self._authkey)2024-06-04T20:13:38.7280469Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 525, in Client2024-06-04T20:13:38.7281586Z     answer_challenge(c, authkey)2024-06-04T20:13:38.7282112Z     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^2024-06-04T20:13:38.7283263Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 953, in answer_challenge2024-06-04T20:13:38.7284679Z     message = connection.recv_bytes(256)         # reject large message2024-06-04T20:13:38.7286092Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 216, in recv_bytes2024-06-04T20:13:38.7287293Z     buf = self._recv_bytes(maxlength)2024-06-04T20:13:38.7288469Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 430, in _recv_bytes2024-06-04T20:13:38.7289647Z     buf = self._recv(4)2024-06-04T20:13:38.7290659Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 395, in _recv2024-06-04T20:13:38.7304889Z     chunk = read(handle, remaining)2024-06-04T20:13:38.7305627Z ConnectionResetError: [Errno 104] Connection reset by peer2024-06-04T20:13:38.7306205Z 2024-06-04T20:13:38.7306632Z The above exception was the direct cause of the following exception:2024-06-04T20:13:38.7307632Z 2024-06-04T20:13:38.7307826Z Traceback (most recent call last):2024-06-04T20:13:38.7309120Z   File "/home/runner/work/cpython/cpython/Lib/test/test_logging.py", line 3916, in test_config_queue_handler2024-06-04T20:13:38.7310373Z     self.do_queuehandler_configuration(qspec, lspec)2024-06-04T20:13:38.7311421Z     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^2024-06-04T20:13:38.7313016Z   File "/home/runner/work/cpython/cpython/Lib/test/test_logging.py", line 3870, in do_queuehandler_configuration2024-06-04T20:13:38.7314254Z     self.apply_config(cd)2024-06-04T20:13:38.7314700Z     ~~~~~~~~~~~~~~~~~^^^^2024-06-04T20:13:38.7315678Z   File "/home/runner/work/cpython/cpython/Lib/test/test_logging.py", line 3248, in apply_config2024-06-04T20:13:38.7316772Z     logging.config.dictConfig(conf)2024-06-04T20:13:38.7317298Z     ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^2024-06-04T20:13:38.7318293Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 911, in dictConfig2024-06-04T20:13:38.7319358Z     dictConfigClass(config).configure()2024-06-04T20:13:38.7319920Z     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^2024-06-04T20:13:38.7320917Z   File "/home/runner/work/cpython/cpython/Lib/logging/config.py", line 584, in configure2024-06-04T20:13:38.7322235Z     raise ValueError('Unable to configure handler '2024-06-04T20:13:38.7322979Z                      '%r' % name) from e2024-06-04T20:13:38.7323669Z ValueError: Unable to configure handler 'ah'2024-06-04T20:13:38.7324133Z 2024-06-04T20:13:38.7324350Z ======================================================================2024-06-04T20:13:38.7325425Z ERROR: test_multiprocessing_queues (test.test_logging.ConfigDictTest.test_multiprocessing_queues)2024-06-04T20:13:38.7326674Z ----------------------------------------------------------------------2024-06-04T20:13:38.7327355Z Traceback (most recent call last):2024-06-04T20:13:38.7328582Z   File "/home/runner/work/cpython/cpython/Lib/test/test_logging.py", line 3935, in test_multiprocessing_queues2024-06-04T20:13:38.7329952Z     q2 = MM().Queue()  # a proxy queue for use when pickling is needed2024-06-04T20:13:38.7331210Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 729, in temp2024-06-04T20:13:38.7332331Z     token, exp = self._create(typeid, *args, **kwds)2024-06-04T20:13:38.7333039Z                  ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^2024-06-04T20:13:38.7334175Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/managers.py", line 609, in _create2024-06-04T20:13:38.7335370Z     conn = self._Client(self._address, authkey=self._authkey)2024-06-04T20:13:38.7336607Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 525, in Client2024-06-04T20:13:38.7337708Z     answer_challenge(c, authkey)2024-06-04T20:13:38.7338203Z     ~~~~~~~~~~~~~~~~^^^^^^^^^^^^2024-06-04T20:13:38.7339348Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 953, in answer_challenge2024-06-04T20:13:38.7340744Z     message = connection.recv_bytes(256)         # reject large message2024-06-04T20:13:38.7342103Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 216, in recv_bytes2024-06-04T20:13:38.7343253Z     buf = self._recv_bytes(maxlength)2024-06-04T20:13:38.7344421Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 430, in _recv_bytes2024-06-04T20:13:38.7345553Z     buf = self._recv(4)2024-06-04T20:13:38.7346512Z   File "/home/runner/work/cpython/cpython/Lib/multiprocessing/connection.py", line 395, in _recv2024-06-04T20:13:38.7347506Z     chunk = read(handle, remaining)2024-06-04T20:13:38.7348256Z ConnectionResetError: [Errno 104] Connection reset by peer2024-06-04T20:13:38.7348871Z

@Fidget-Spinner
Copy link
Member

JIT aarch64 errors happen on the other PRs as well. So it's not this PR. Will merge this PR in 24 hours or so.

saulshanabrook reacted with heart emoji

@saulshanabrook
Copy link
ContributorAuthor

Sweet! Were the benchmarks ever re-run on comparing this, or is it not worth it b/c it's probably similar the first time?

@saulshanabrooksaulshanabrook marked this pull request as ready for reviewJune 7, 2024 15:34
@Fidget-Spinner
Copy link
Member

Sweet! Were the benchmarks ever re-run on comparing this, or is it not worth it b/c it's probably similar the first time?

The benchmarks the second time reported no speedup. But the noise range of a bench run is like 1%, so for all we know this couldve sped things up 0.5% but we cant tell.

saulshanabrook reacted with thumbs up emoji

@Fidget-SpinnerFidget-Spinner merged commit55402d3 intopython:mainJun 8, 2024
@Fidget-Spinner
Copy link
Member

Congrats@saulshanabrook !

saulshanabrook reacted with heart emojidpdani reacted with rocket emoji

@saulshanabrooksaulshanabrook deleted the optimizer-type-version-watcher branchJune 8, 2024 13:19
Fidget-Spinner added a commit that referenced this pull requestJun 20, 2024
noahbkim pushed a commit to hudson-trading/cpython that referenced this pull requestJul 11, 2024
…er (pythonGH-119365)Co-authored-by: parmeggiani <parmeggiani@spaziodati.eu>Co-authored-by: dpdani <git@danieleparmeggiani.me>Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>Co-authored-by: Ken Jin <kenjin@python.org>
estyxx pushed a commit to estyxx/cpython that referenced this pull requestJul 17, 2024
…er (pythonGH-119365)Co-authored-by: parmeggiani <parmeggiani@spaziodati.eu>Co-authored-by: dpdani <git@danieleparmeggiani.me>Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>Co-authored-by: Ken Jin <kenjin@python.org>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@Fidget-SpinnerFidget-SpinnerFidget-Spinner approved these changes

@brandtbucherbrandtbucherbrandtbucher left review comments

@markshannonmarkshannonAwaiting requested review from markshannonmarkshannon is a code owner

@gvanrossumgvanrossumAwaiting requested review from gvanrossum

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@saulshanabrook@brandtbucher@carljm@Fidget-Spinner@dpdani

[8]ページ先頭

©2009-2025 Movatter.jp