Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-132917: Use /proc/self/status for mem usage info.#133544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
nascheme merged 3 commits intopython:mainfromnascheme:gh-132917-use-proc-status
May 8, 2025

Conversation

nascheme
Copy link
Member

@naschemenascheme commentedMay 6, 2025•
edited by bedevere-appbot
Loading

Using smaps_rollup is slower and we can get the similar info from /proc/self/status. We don't need the extra accuracy that smaps_rollup is giving.

Profiling is showing that reading thesmaps_rollup file is taking on the order of 30 ms. Readingstatus is much faster. Some background detail on this difference:

https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/issues/10966#note_410194443

Using smaps_rollup is quite a lot slower and we can get the similar infofrom /proc/self/status.
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@nascheme for commit5b3621d 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F133544%2Fmerge

If you want to schedule another build, you need to add the🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelMay 6, 2025
@naschemenascheme added performancePerformance or resource usage topic-free-threading labelsMay 7, 2025
@nascheme
Copy link
MemberAuthor

nascheme commentedMay 7, 2025•
edited
Loading

Some more precise run time comparison. With the script fromGH-132917 running in 10 parallel threads, the/proc/self/smaps_rollup version takes 3000 us/per call and the/proc/self/status version uses 15 us/per call.

I also tested the MacOS version and it takes 4.5 us/per call. The full GC pass is taking roughly 70 ms.

@nascheme
Copy link
MemberAuthor

Comparing the info from /proc/self/smaps_rollup vs /proc/self/status. This is on a Linux 6.1.0 kernel with 16 GB of RAM and 16 GB of swap. I ran a Python program that just allocates a bunch of memory in a loop while printing the proc info.

Rss:Swap:VmRSS:VmSwap:
2,361,87602,361,6840
4,714,09204,713,9240
7,066,30807,066,1640
9,418,58809,418,4040
11,770,800011,770,6440
14,122,420014,122,2880
14,846,5521,625,61214,845,6041,625,612
15,439,2523,384,87215,478,7083,345,056

Copy link
Member

@Yhg1sYhg1s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This looks like a more efficient approach and the change is pretty simple... But unless@hugovk wants to reopen the release branch I think it should just go into beta 2. (Also I still have the same concerns I mentioned before, but they're not pressing enough to worry about for b1.)

@hugovk
Copy link
Member

Let's keep this for b2, thanks!

@naschemenaschemeenabled auto-merge (squash)May 7, 2025 16:53
@naschemenascheme merged commit751db4e intopython:mainMay 8, 2025
38 checks passed
@naschemenascheme added the needs backport to 3.14bugs and security fixes labelMay 8, 2025
@miss-islington-app
Copy link

Thanks@nascheme for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestMay 8, 2025
…133544)On Linux, use /proc/self/status for mem usage info.  Using smaps_rollup is quite a lot slower andwe can get the similar info from /proc/self/status.(cherry picked from commit751db4e)Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
@bedevere-app
Copy link

GH-133718 is a backport of this pull request to the3.14 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.14bugs and security fixes labelMay 8, 2025
nascheme added a commit that referenced this pull requestMay 8, 2025
… (gh-133718)On Linux, use /proc/self/status for mem usage info.  Using smaps_rollup is quite a lot slower andwe can get the similar info from /proc/self/status.(cherry picked from commit751db4e)Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@Yhg1sYhg1sYhg1s approved these changes

@pablogsalpablogsalAwaiting requested review from pablogsal

Assignees
No one assigned
Labels
performancePerformance or resource usageskip newstopic-free-threading
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@nascheme@bedevere-bot@hugovk@Yhg1s
[8]ページ先頭
©2009-2025 Movatter.jp