Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.4k
Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan information#136790
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation
…tionThis commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110While the annotation macro is defined here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598And the corresponding attribute is documented in:* for gcc:https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute* for clang:https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis
Uh oh!
There was an error while loading.Please reload this page.
Thank you the PR, and for taking the time to make it this short! |
d19bb44
intopython:mainUh oh!
There was an error while loading.Please reload this page.
Thanks@disconnect3d for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14. |
Thanks@disconnect3d for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13. |
…tion (pythonGH-136790)* Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan informationThis commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110While the annotation macro is defined here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598And the corresponding attribute is documented in:* for gcc:https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute* for clang:https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis* Update Doc/c-api/memory.rst* Improve --with-address-sanitizer and pymalloc docs---------(cherry picked from commitd19bb44)Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>Co-authored-by: Petr Viktorin <encukou@gmail.com>
…tion (pythonGH-136790)* Doc/c-api/memory.rst: extend --without-pymalloc doc with ASan informationThis commit extends the documentation for disabling pymalloc with the `--without-pymalloc` flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the `python -m test` suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the `--without-pymalloc` flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as the `address_in_range` function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Objects/obmalloc.c#L2096-L2110While the annotation macro is defined here:https://github.com/python/cpython/blob/acefb978dcb5dd554e3c49a3015ee5c2ad6bfda1/Include/pyport.h#L582-L598And the corresponding attribute is documented in:* for gcc:https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#index-no_005fsanitize_005faddress-function-attribute* for clang:https://clang.llvm.org/docs/AttributeReference.html#no-sanitize-address-no-address-safety-analysis* Update Doc/c-api/memory.rst* Improve --with-address-sanitizer and pymalloc docs---------(cherry picked from commitd19bb44)Co-authored-by: Disconnect3d <dominik.b.czarnota@gmail.com>Co-authored-by: Petr Viktorin <encukou@gmail.com>
GH-136798 is a backport of this pull request to the3.14 branch. |
GH-136799 is a backport of this pull request to the3.13 branch. |
Uh oh!
There was an error while loading.Please reload this page.
This commit extends the documentation for disabling pymalloc with the
--without-pymalloc
flag regarding why it is worth to use it when enabling AddressSanitizer for Python build (which is done, e.g., in CPython's CI builds).I have tested the CPython latest main build with both ASan and pymalloc enabled and it seems to work just fine. I did run the
python -m test
suite which didn't uncover any ASan crashes (though, it detected some memory leaks, which I believe are irrelevant here).I have discussed ASan and this flag with@encukou on the CPython Core sprint on EuroPython 2025. We initially thought that the
--without-pymalloc
flag is needed for ASan builds due to the fact pymalloc must hit the begining of page when determining if the memory to be freed comes from pymalloc or was allocated by the system malloc. In other words, we thought, that ASan would crash CPython during free of big objects (allocated by system malloc). It may be that this was the case in the past, but it is not the case anymore as theaddress_in_range
function used by pymalloc is annotated to be skipped from the ASan instrumentation.This code can be seen here:
cpython/Objects/obmalloc.c
Lines 2096 to 2110 inacefb97
While the annotation macro is defined here:
cpython/Include/pyport.h
Lines 582 to 598 inacefb97
And the corresponding attribute is documented in:
📚 Documentation preview 📚:https://cpython-previews--136790.org.readthedocs.build/