
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2017-09-14 17:36 byxiang.zhang, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Messages (7) | |||
|---|---|---|---|
| msg302187 -(view) | Author: Xiang Zhang (xiang.zhang)*![]() | Date: 2017-09-14 17:36 | |
PyMem_Raw* APIs should be thread safe. But in debug mode, they are not that simple and increment a global variable *serialno* and the incrementing action is not thread safe. | |||
| msg330947 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2018-12-03 15:19 | |
I looked at _Py_atomic_address to avoid atomic "serialno++", but we don't have atomic_fetch_add(). We could implement it using a loop and atomic_compare_exchange_strong()... but we don't have atomic_compare_exchange_strong() neither.I tried to add a mutex, but there are some pratical issues:*bpo-35388: question about calling Py_Initialize() / Py_Finalize() multiple times* I modified _PyRuntimeState_Init() to initialize the lock._PyRuntimeState_Init() calls PyThread_acquire_lock() which calls PyMem_RawMalloc(). Problem: PyMem_RawMalloc() requires the lock. I worked around the isuse using "if (_PyRuntime.mem.mutex != NULL) {". | |||
| msg330948 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2018-12-03 15:21 | |
By the way, the previous attempt to add _PyRuntime.mem had to be reverted. Seebpo-32096,bpo-30860 and the commit:commit9e87e7776f7ace66baaf7247233afdabd00c2b44Author: Victor Stinner <victor.stinner@gmail.com>Date: Fri Nov 24 12:09:24 2017 +0100bpo-32096: Remove obj and mem from _PyRuntime (#4532)bpo-32096,bpo-30860: Partially revert the commit2ebc5ce42a8a9e047e790aefbf9a94811569b2b6: * Move structures back fromInclude/internal/mem.h toObjects/obmalloc.c * Remove _PyObject_Initialize() and _PyMem_Initialize() * RemoveInclude/internal/pymalloc.h * Add test_capi.test_pre_initialization_api(): Make sure that it's possible to call Py_DecodeLocale(), and then call Py_SetProgramName() with the decoded string, before Py_Initialize(). PyMem_RawMalloc() and Py_DecodeLocale() can be called again before _PyRuntimeState_Init(). Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com> | |||
| msg330949 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2018-12-03 15:22 | |
See alsobpo-35265: "Internal C API: pass the memory allocator in a context".The most complex part is the Python initialization which changes the memory allocator *and* allocate memory. We have to remain which allocator has been used to allocate data, to be able to release memory at exit. | |||
| msg330957 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2018-12-03 16:41 | |
Note: PTHREAD_MUTEX_INITIALIZER can be used to statically initialize a mutex. | |||
| msg330959 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2018-12-03 16:45 | |
See alsobpo-35368: [2.7] Make PyMem_Malloc() thread-safe in debug mode. | |||
| msg342538 -(view) | Author: STINNER Victor (vstinner)*![]() | Date: 2019-05-15 02:45 | |
Python 3.8 has been fixed. I disabled serialno field by default: PYMEM_DEBUG_SERIALNO is not defined by default. You have to opt-in to get this bug :-) I don't see any easy fix older Python versions. I close the issue. | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:52 | admin | set | github: 75654 |
| 2019-05-15 02:45:20 | vstinner | set | status: open -> closed resolution: fixed messages: +msg342538 stage: needs patch -> resolved |
| 2018-12-03 16:45:31 | vstinner | set | messages: +msg330959 |
| 2018-12-03 16:43:57 | vstinner | set | title: PyMem_Raw* API in debug mode are not thread safe -> Debug hooks on memory allocators are not thread safe (serialno variable) versions: + Python 2.7, Python 3.8 |
| 2018-12-03 16:41:18 | vstinner | set | messages: +msg330957 |
| 2018-12-03 15:22:40 | vstinner | set | messages: +msg330949 |
| 2018-12-03 15:21:01 | vstinner | set | messages: +msg330948 |
| 2018-12-03 15:19:07 | vstinner | set | messages: +msg330947 |
| 2017-09-14 17:36:34 | xiang.zhang | create | |