Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information#136790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
encukou merged 3 commits intopython:mainfromdisconnect3d:extend-pymalloc-doc
Jul 19, 2025

Conversation

disconnect3d
Copy link
Contributor

@disconnect3ddisconnect3d commentedJul 19, 2025
edited by github-actionsbot
Loading

This commit extends the documentation for disabling pymalloc with the--without-pymalloc flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).

I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run thepython -m test suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).

I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the--without-pymalloc flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as theaddress_in_range function used by pymalloc is annotated to be skipped from the ASan instrumentation.

This code can be seen here:

cpython/Objects/obmalloc.c

Lines 2096 to 2110 inacefb97

staticbool_Py_NO_SANITIZE_ADDRESS
_Py_NO_SANITIZE_THREAD
_Py_NO_SANITIZE_MEMORY
address_in_range(OMState*state,void*p,poolppool)
{
// Since address_in_range may be reading from memory which was not allocated
// by Python, it is important that pool->arenaindex is read only once, as
// another thread may be concurrently modifying the value without holding
// the GIL. The following dance forces the compiler to read pool->arenaindex
// only once.
uintarenaindex=*((volatileuint*)&pool->arenaindex);
returnarenaindex<maxarenas&&
(uintptr_t)p-allarenas[arenaindex].address<ARENA_SIZE&&
allarenas[arenaindex].address!=0;
}

While the annotation macro is defined here:

#if __has_feature(address_sanitizer)
#if !defined(_Py_ADDRESS_SANITIZER)
#define_Py_ADDRESS_SANITIZER
#define_Py_NO_SANITIZE_ADDRESS__attribute__((no_sanitize_address))
# endif
#endif
#if __has_feature(thread_sanitizer)
#if !defined(_Py_THREAD_SANITIZER)
#define_Py_THREAD_SANITIZER
#define_Py_NO_SANITIZE_THREAD__attribute__((no_sanitize_thread))
# endif
#endif
#elif defined(__GNUC__)
#if defined(__SANITIZE_ADDRESS__)
#define_Py_ADDRESS_SANITIZER
#define_Py_NO_SANITIZE_ADDRESS__attribute__((no_sanitize_address))
# endif

And the corresponding attribute is documented in:


📚 Documentation preview 📚:https://cpython-previews--136790.org.readthedocs.build/

…tionThis commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110While the annotation macro is defined here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598And the corresponding attribute is documented in:* for gcc:https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute* for clang:https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis
@encukou
Copy link
Member

Thank you the PR, and for taking the time to make it this short!

@encukouencukou merged commitd19bb44 intopython:mainJul 19, 2025
30 checks passed
@github-project-automationgithub-project-automationbot moved this fromTodo toDone inDocs PRsJul 19, 2025
@encukouencukou added needs backport to 3.13bugs and security fixes needs backport to 3.14bugs and security fixes labelsJul 19, 2025
@miss-islington-app
Copy link

Thanks@disconnect3d for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks@disconnect3d for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJul 19, 2025
…tion (pythonGH-136790)* Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan informationThis commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110While the annotation macro is defined here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598And the corresponding attribute is documented in:* for gcc:https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute* for clang:https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis* Update Doc/c-api/memory.rst* Improve --with-address-sanitizer and pymalloc docs---------(cherry picked from commitd19bb44)Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>Co-authored-by: Petr Viktorin <encukou@gmail.com>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJul 19, 2025
…tion (pythonGH-136790)* Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan informationThis commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110While the annotation macro is defined here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598And the corresponding attribute is documented in:* for gcc:https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute* for clang:https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis* Update Doc/c-api/memory.rst* Improve --with-address-sanitizer and pymalloc docs---------(cherry picked from commitd19bb44)Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>Co-authored-by: Petr Viktorin <encukou@gmail.com>
@bedevere-app
Copy link

GH-136798 is a backport of this pull request to the3.14 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.14bugs and security fixes labelJul 19, 2025
@bedevere-app
Copy link

GH-136799 is a backport of this pull request to the3.13 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.13bugs and security fixes labelJul 19, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@encukouencukouencukou approved these changes

@erlend-aaslanderlend-aaslandAwaiting requested review from erlend-aaslanderlend-aasland is a code owner

@corona10corona10Awaiting requested review from corona10corona10 is a code owner

Assignees
No one assigned
Labels
docsDocumentation in the Doc dirskip issueskip newssprint
Projects
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@disconnect3d@encukou@AA-Turner

[8]ページ先頭

©2009-2025 Movatter.jp