Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-82054: allow test runner to split test_asyncio#103859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
zitterbewegung wants to merge0 commits intopython:mainfromzitterbewegung:main

Conversation

@zitterbewegung
Copy link
Contributor

@zitterbewegungzitterbewegung commentedApr 25, 2023
edited
Loading

Summary:

This runs test_asyncio sub-tests in parallel using sharding by cinder. These two tests are typically the long-poles in runs because they are modules with a lot of further sub-tests run serially. By breaking out the sub-tests as independent modules we can run a lot more in parallel.

After porting we can see the direct impact is extremely large (15% increase in performance):

  • Without this change:
    • Running make test is 5 min 26 sec
  • With this change:
    • Running make test takes 3 min and 45 seconds

The drawbacks are that this implementation is hacky and due to the sorting of the tests it obscures when the asyncio tests occur and involves changing CPython test infrastructure but, the time saved it is worth it . It's not a complicated change and I think the win in productivity with the change above is significant.

@zitterbewegungzitterbewegung marked this pull request as draftApril 25, 2023 23:18
@zitterbewegungzitterbewegung marked this pull request as ready for reviewApril 25, 2023 23:24
@zitterbewegungzitterbewegung changed the titlegh-82054: Porting execution of tests in parallel by sharding (note this only is for asyncio and compiler)gh-82054: Porting parallel test execution from cinder by sharding (note this only is for asyncio and compiler)Apr 25, 2023
Copy link
Member

@carljmcarljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Looks good! Just a couple comments.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@zitterbewegungzitterbewegung changed the titlegh-82054: Porting parallel test execution from cinder by sharding (note this only is for asyncio and compiler)gh-82054: Porting parallel test execution from cinder by sharding (note this only is for asyncio)Apr 26, 2023
@carljmcarljm changed the titlegh-82054: Porting parallel test execution from cinder by sharding (note this only is for asyncio)gh-82054: allow test runner to split test_asyncio for better parallel executionApr 26, 2023
@carljm
Copy link
Member

carljm commentedApr 26, 2023
edited
Loading

@zitterbewegung in order to make the case for landing this, it would be useful to update the PR description with some current data. Copying the description from the Cinder diff isn't really useful, since it's from a much older version. What would be helpful is to runmake test with and without this change, and report any improvement in overall runtime.

It also may be that test_asyncio is no longer the longest pole, and we may need to apply the same treatment to some other modules (test_multiprocessing?) in order to see a significant benefit in overall runtime ofmake test.

@zitterbewegung
Copy link
ContributorAuthor

zitterbewegung commentedApr 26, 2023
edited
Loading

@carljm I can see an overall benefit of 100 seconds from running cpython's baseline. I have attached both runs.
To give explicit timings before this change running make test is 5 min 26 sec and with the change it goes to 3 min and 46 seconds (100 seconds saved). Adding multiprocessing gives us a 5 second improvement.

This was executed on a AMD Ryzen Threadripper 3970X 32-Core Processor with 128GB of ram.

Patched_run_of_tests.txt
original_time_cpython.txt

carljm reacted with heart emoji

@carljm
Copy link
Member

@zitterbewegung Awesome, those are great results! Can you directly update the PR description/summary to eliminate the text copied from the Cinder diff and instead just summarize these results?

@zitterbewegung
Copy link
ContributorAuthor

@zitterbewegung Awesome, those are great results! Can you directly update the PR description/summary to eliminate the text copied from the Cinder diff and instead just summarize these results?

Updated summary with new results.

carljm reacted with thumbs up emoji

@carljm
Copy link
Member

@zitterbewegung Ok, one last thing I see is thatmake patchcheck is failing on the Azure Pipelines CI job, due to something it doesn't like about the whitespace inruntests.py. Pretty nitpicky, but we do want to keep the CI green; can you runmake patchcheck locally and try to adjust the diff such that patchcheck is happy?

@zitterbewegung
Copy link
ContributorAuthor

@zitterbewegung Awesome, those are great results! Can you directly update the PR description/summary to eliminate the text copied from the Cinder diff and instead just summarize these results?

Done.

@zitterbewegung
Copy link
ContributorAuthor

zitterbewegung commentedApr 27, 2023
edited
Loading

@carljm I accidentally closed this PR while I was adding test_multiprocessing which saved 6 more seconds the new PR is#103927

@zitterbewegungzitterbewegung changed the titlegh-82054: allow test runner to split test_asyncio for better parallel executiongh-82054: allow test runner to split test_asyncio and test_multiprocessing for better parallel executionApr 27, 2023
@zitterbewegungzitterbewegung changed the titlegh-82054: allow test runner to split test_asyncio and test_multiprocessing for better parallel executiongh-82054: allow test runner to split test_asyncioApr 27, 2023
@zitterbewegung
Copy link
ContributorAuthor

zitterbewegung commentedApr 27, 2023
edited
Loading

@zitterbewegung Ok, one last thing I see is thatmake patchcheck is failing on the Azure Pipelines CI job, due to something it doesn't like about the whitespace inruntests.py. Pretty nitpicky, but we do want to keep the CI green; can you runmake patchcheck locally and try to adjust the diff such that patchcheck is happy?

I ran make patch check after I did the changes

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@carljmcarljmcarljm requested changes

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@zitterbewegung@bedevere-bot@carljm

[8]ページ先頭

©2009-2025 Movatter.jp