Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

GH-104510: Fix refleaks in _io base types#104516

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
kumaraditya303 merged 3 commits intopython:mainfromkumaraditya303:main
May 16, 2023

Conversation

kumaraditya303
Copy link
Contributor

@kumaraditya303kumaraditya303 commentedMay 15, 2023
edited by bedevere-bot
Loading

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@kumaraditya303 for commit5ee2358 🤖

If you want to schedule another build, you need to add the🔨 test-with-refleak-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-refleak-buildbotsTest PR w/ refleak buildbots; report in status section labelMay 15, 2023
Copy link
Member

@sunmy2019sunmy2019 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM

Copy link
Member

@Eclips4Eclips4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

All ref leaks are gone!
Thanks for the fix.

@erlend-aasland
Copy link
Contributor

The reason why this fix works is not obvious. Can you add a clarifying comment, alternatively expand on the C API docs fortp_traverse?

Note to self: we also need to document the GC peculiarities of heap-type-inherits-from-static-type (for exampledefdict in the _collections extension module).

Eclips4 reacted with thumbs up emoji

@erlend-aaslanderlend-aasland changed the titleGH-104510: fix refleaksGH-104510: Fix refleaks in \_io base typesMay 16, 2023
@erlend-aaslanderlend-aasland changed the titleGH-104510: Fix refleaks in \_io base typesGH-104510: Fix refleaks in _io base typesMay 16, 2023
@erlend-aaslanderlend-aasland linked an issueMay 16, 2023 that may beclosed by this pull request
@kumaraditya303kumaraditya303 added the 🔨 test-with-refleak-buildbotsTest PR w/ refleak buildbots; report in status section labelMay 16, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@kumaraditya303 for commit78c12ff 🤖

If you want to schedule another build, you need to add the🔨 test-with-refleak-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-refleak-buildbotsTest PR w/ refleak buildbots; report in status section labelMay 16, 2023
Copy link
Contributor

@erlend-aaslanderlend-aasland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

AFAICS, this is the correct solution.

We still need to write a sentence or two in the docs about our recent GC adventures (_io, _collections).

@sunmy2019
Copy link
Member

sunmy2019 commentedMay 16, 2023
edited
Loading

The heap type still has GC but it now inherits traverse and clear slots from its base becausePy_TPFLAGS_HAVE_GC isn't set. Seehttps://docs.python.org/3.12/c-api/gcsupport.html

The doc's a bit unclear here.

When callingPyType_Ready() or some of the APIs that indirectly call it likePyType_FromSpecWithBases() orPyType_FromSpec() the interpreter will automatically populate thetp_flags,tp_traverse andtp_clear fields if the type inherits from a class that implements the garbage collector protocol and the child class does not include thePy_TPFLAGS_HAVE_GC flag.

From the code, we see,

if (!(type->tp_flags&Py_TPFLAGS_HAVE_GC)&&
(base->tp_flags&Py_TPFLAGS_HAVE_GC)&&
(!type->tp_traverse&& !type->tp_clear)) {
type->tp_flags |=Py_TPFLAGS_HAVE_GC;
if (type->tp_traverse==NULL)
type->tp_traverse=base->tp_traverse;
if (type->tp_clear==NULL)
type->tp_clear=base->tp_clear;
}

To get thetp_traverse andtp_clear inherited, the child type must have them bothNULL.

This is against the doc. Also, the comparisons in lines 6674 and 6676 are unnecessary.

It was a 22-yr old code13d52f0. I guess the author (@gvanrossum) was trying to write

-      (!type->tp_traverse&& !type->tp_clear)) {+      (!type->tp_traverse|| !type->tp_clear)) {

Don't know what to do. Maybe we should only update the doc.

@kumaraditya303kumaraditya303 merged commit442a3e6 intopython:mainMay 16, 2023
@kumaraditya303
Copy link
ContributorAuthor

Don't know what to do. Maybe we should only update the doc.

The best we can do is to fix docs, the behavior here is ancient.

erlend-aasland reacted with thumbs up emoji

carljm added a commit to carljm/cpython that referenced this pull requestMay 16, 2023
* main:pythonGH-104510: Fix refleaks in `_io` base types (python#104516)pythongh-104539: Fix indentation error in logging.config.rst (python#104545)pythongh-104050: Don't star-import 'types' in Argument Clinic (python#104543)pythongh-104050: Add basic typing to CConverter in clinic.py (python#104538)pythongh-64595: Fix write file logic in Argument Clinic (python#104507)pythongh-104523: Inline minimal PGO rules (python#104524)pythongh-103861: Fix Zip64 extensions not being properly applied in some cases (python#103863)pythongh-69152: add method get_proxy_response_headers to HTTPConnection class (python#104248)pythongh-103763: Implement PEP 695 (python#103764)pythongh-104461: Run tkinter test_configure_screen on X11 only (pythonGH-104462)pythongh-104469: Convert _testcapi/watchers.c to use Argument Clinic (python#104503)pythongh-104482: Fix error handling bugs in ast.c (python#104483)pythongh-104341: Adjust tstate_must_exit() to Respect Interpreter Finalization (pythongh-104437)pythonGH-102613: Fix recursion error from `pathlib.Path.glob()` (pythonGH-104373)
@vstinner
Copy link
Member

Next time, it would be nice to elaborate the rationale for the fix in the commit, rather than just saying "fix" without any commit message ;-) It's really hard to handle the GC correctly with heap types. Better documentation is required.

The _io module is one of the few C extensions which use class inheritance with heap types implemented in C.

erlend-aasland reacted with thumbs up emoji

@vstinner
Copy link
Member

cc@pablogsal: GC issues are hard :-(

carljm added a commit to carljm/cpython that referenced this pull requestMay 17, 2023
* main: (26 commits)pythonGH-101520: Move tracemalloc functionality into core, leaving interface in Modules. (python#104508)  typing: Add more tests for TypeVar (python#104571)pythongh-104572: Improve error messages for invalid constructs in PEP 695 contexts (python#104573)  typing: Use PEP 695 syntax in typing.py (python#104553)pythongh-102153: Start stripping C0 control and space chars in `urlsplit` (python#102508)pythongh-104469: Update README.txt for _testcapi (pythongh-104529)pythonGH-103092: isolate `_elementtree` (python#104561)pythongh-104050: Add typing to Argument Clinic converters (python#104547)pythonGH-103906: Remove immortal refcounting in the interpreter (pythonGH-103909)pythongh-87474: Fix file descriptor leaks in subprocess.Popen (python#96351)pythonGH-103092: isolate `pyexpat`  (python#104506)pythongh-75367: Fix data descriptor detection in inspect.getattr_static (python#104517)pythongh-104050: Add more annotations to `Tools/clinic.py` (python#104544)pythongh-104555: Fix isinstance() and issubclass() for runtime-checkable protocols that use PEP 695 (python#104556)pythongh-103865: add monitoring support to LOAD_SUPER_ATTR (python#103866)  CODEOWNERS: Assign new PEP 695 files to myself (python#104551)pythonGH-104510: Fix refleaks in `_io` base types (python#104516)pythongh-104539: Fix indentation error in logging.config.rst (python#104545)pythongh-104050: Don't star-import 'types' in Argument Clinic (python#104543)pythongh-104050: Add basic typing to CConverter in clinic.py (python#104538)  ...
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@sunmy2019sunmy2019sunmy2019 approved these changes

@erlend-aaslanderlend-aaslanderlend-aasland approved these changes

@Eclips4Eclips4Eclips4 approved these changes

Assignees
No one assigned
Projects
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Ref leaks introduced by _io isolation (gh-101948)
6 participants
@kumaraditya303@bedevere-bot@erlend-aasland@sunmy2019@vstinner@Eclips4

[8]ページ先頭

©2009-2025 Movatter.jp