Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-82300: Add track parameter to shared memory#110778

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
gpshead merged 28 commits intopython:mainfrompan324:shmemuntrack
Dec 5, 2023

Conversation

@pan324
Copy link
Contributor

@pan324pan324 commentedOct 12, 2023
edited by github-actionsbot
Loading

torokati44 reacted with thumbs up emoji
@pan324pan324 requested a review fromtiran as acode ownerOctober 12, 2023 18:12
@ghost
Copy link

ghost commentedOct 12, 2023
edited by ghost
Loading

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

@pan324
Copy link
ContributorAuthor

Whoops I messed up the documentation part. The tracking part should be on unlink, not on close.

@gvanrossumgvanrossum removed the request for review fromtiranOctober 15, 2023 07:37
Copy link
Member

@gvanrossumgvanrossum left a comment
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks. This looks reasonable, I have mostly questions about the phrasing of the docs.

Do I understand correctly that the whole tracking mechanism only applies on POSIX? Maybe the docs should mention that?

@gvanrossum
Copy link
Member

@serhiy-storchaka Do you have any experience with this code? Could you help me with the review?

@gvanrossum
Copy link
Member

The new parameter needs tests.

pitrou and pan324 reacted with thumbs up emoji

@serhiy-storchaka
Copy link
Member

No, I have no experience with this code. I can help with technical review, but I can't say whether the proposed feature is the right way to solve the problem.

It looks liketrack=False only affects POSIX. What about Windows? Should it raise an exception fortrack=False? Or fortrack=True?

If in most casestrack should be False ifcreate is false, should nottrack be set tocreate by default?

gpshead reacted with heart emoji

@gvanrossum
Copy link
Member

Thanks,@serhiy-storchaka -- let's see what the PR author says.

Copy link
Member

@pitroupitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks@pan324 for submitting this! The proposal looks good on the principle, but please make sure you adress the review comments (including the request for additional unit tests).

@bedevere-app
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@pan324
Copy link
ContributorAuthor

Thanks everyone!

Yes, the tracking only happens on POSIX because Windows has its own reference counting. I was cautious about mentioning this because users shouldn't rely on such internal behavior. But I can see that Windows users might that need it as a word of caution to thoroughly test their shared memory deallocation on other systems.

Outside of multiprocessing, each Python process comes with its own resource tracker which (assumingtrack isTrue) cleans up on termination. If some process shares memory with some short-lived subprocess, then the subprocess process tracker will warn that the user failed to clean up and delete the shared memory. The main process can then either unlink to get a FileNotFoundError or do nothing and be warned by its own resource tracker that the memory wasn't cleaned up properly (right after, the resource tracker will warn about its own FileNotFoundError).

Surely the resource tracker itself would need some deeper changes to be more robust but it's not my area of expertise either which is why I have opted for these minimal changes.

Windows must not raise an exception for eithertrack value! That would make the code nonportable with no gains.

As far as I understand there is a clash between multiprocessing and subprocess. Users of subprocess always want to disable tracking when not creating. For multiprocessing I use the higher level constructs and therefore never had the need to access the SharedMemory module directly. So while users of SharedMemory typically wanttrack andcreate to have the same value, the internals of multiprocessing may depend ontrack always beingTrue.

I will look into adding suitable tests now.

pan324and others added3 commitsOctober 15, 2023 20:57
…-O38.rstCo-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
@gvanrossum
Copy link
Member

@pitrou Can you complete this review? I really don't know enough about this topic to review it properly.

@buzmeg
Copy link

@pitrou It looks like this is being held on your request from 3 weeks ago. I believe that@pan324 made the changes you requested.

Could you look at this and either release it or confirm that you still don't think the unit tests are sufficient?

Thanks.

pan324 reacted with heart emoji

@gpsheadgpshead self-assigned thisNov 30, 2023
@gpsheadgpshead added type-featureA feature request or enhancement 3.13bugs and security fixes and removed awaiting changes labelsDec 2, 2023
@gpsheadgpshead merged commit81ee026 intopython:mainDec 5, 2023
aisk pushed a commit to aisk/cpython that referenced this pull requestFeb 11, 2024
…ython#110778)Add a track parameter to shared memory to allow resource tracking via the side-launched resource tracker process to be disabled on platforms that use it (POSIX).This allows people who do not want automated cleanup at process exit because they are using the shared memory with processes not participating in Python's resource tracking to use the shared_memory API.Co-authored-by: Łukasz Langa <lukasz@langa.pl>Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>Co-authored-by: Antoine Pitrou <pitrou@free.fr>Co-authored-by: Gregory P. Smith <greg@krypto.org>
Glyphack pushed a commit to Glyphack/cpython that referenced this pull requestSep 2, 2024
…ython#110778)Add a track parameter to shared memory to allow resource tracking via the side-launched resource tracker process to be disabled on platforms that use it (POSIX).This allows people who do not want automated cleanup at process exit because they are using the shared memory with processes not participating in Python's resource tracking to use the shared_memory API.Co-authored-by: Łukasz Langa <lukasz@langa.pl>Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>Co-authored-by: Antoine Pitrou <pitrou@free.fr>Co-authored-by: Gregory P. Smith <greg@krypto.org>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@pitroupitroupitrou requested changes

@gvanrossumgvanrossumgvanrossum left review comments

@gpsheadgpsheadgpshead approved these changes

+1 more reviewer

@wtdcodewtdcodewtdcode left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@gpsheadgpshead

Labels

3.13bugs and security fixestype-featureA feature request or enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@pan324@gvanrossum@serhiy-storchaka@buzmeg@torokati44@gpshead@pitrou@wtdcode@ambv

[8]ページ先頭

©2009-2025 Movatter.jp