Movatterモバイル変換


[0]ホーム

URL:


homepage

Issue31473

This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title:Debug hooks on memory allocators are not thread safe (serialno variable)
Type:behaviorStage:resolved
Components:Interpreter CoreVersions:Python 3.8, Python 3.7, Python 3.6, Python 2.7
process
Status:closedResolution:fixed
Dependencies:Superseder:
Assigned To:Nosy List: vstinner, xiang.zhang
Priority:normalKeywords:

Created on2017-09-14 17:36 byxiang.zhang, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.

Messages (7)
msg302187 -(view)Author: Xiang Zhang (xiang.zhang)*(Python committer)Date: 2017-09-14 17:36
PyMem_Raw* APIs should be thread safe. But in debug mode, they are not that simple and increment a global variable *serialno* and the incrementing action is not thread safe.
msg330947 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2018-12-03 15:19
I looked at _Py_atomic_address to avoid atomic "serialno++", but we don't have atomic_fetch_add(). We could implement it using a loop and atomic_compare_exchange_strong()... but we don't have atomic_compare_exchange_strong() neither.I tried to add a mutex, but there are some pratical issues:*bpo-35388: question about calling Py_Initialize() / Py_Finalize() multiple times* I modified _PyRuntimeState_Init() to initialize the lock._PyRuntimeState_Init() calls PyThread_acquire_lock() which calls PyMem_RawMalloc(). Problem: PyMem_RawMalloc() requires the lock. I worked around the isuse using "if (_PyRuntime.mem.mutex != NULL) {".
msg330948 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2018-12-03 15:21
By the way, the previous attempt to add _PyRuntime.mem had to be reverted. Seebpo-32096,bpo-30860 and the commit:commit9e87e7776f7ace66baaf7247233afdabd00c2b44Author: Victor Stinner <victor.stinner@gmail.com>Date:   Fri Nov 24 12:09:24 2017 +0100bpo-32096: Remove obj and mem from _PyRuntime (#4532)bpo-32096,bpo-30860:  Partially revert the commit2ebc5ce42a8a9e047e790aefbf9a94811569b2b6:        * Move structures back fromInclude/internal/mem.h toObjects/obmalloc.c    * Remove _PyObject_Initialize() and _PyMem_Initialize()    * RemoveInclude/internal/pymalloc.h    * Add test_capi.test_pre_initialization_api():       Make sure that it's possible to call Py_DecodeLocale(), and then call       Py_SetProgramName() with the decoded string, before Py_Initialize().        PyMem_RawMalloc() and Py_DecodeLocale() can be called again before    _PyRuntimeState_Init().        Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
msg330949 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2018-12-03 15:22
See alsobpo-35265: "Internal C API: pass the memory allocator in a context".The most complex part is the Python initialization which changes the memory allocator *and* allocate memory. We have to remain which allocator has been used to allocate data, to be able to release memory at exit.
msg330957 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2018-12-03 16:41
Note: PTHREAD_MUTEX_INITIALIZER can be used to statically initialize a mutex.
msg330959 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2018-12-03 16:45
See alsobpo-35368: [2.7] Make PyMem_Malloc() thread-safe in debug mode.
msg342538 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2019-05-15 02:45
Python 3.8 has been fixed. I disabled serialno field by default: PYMEM_DEBUG_SERIALNO is not defined by default. You have to opt-in to get this bug :-) I don't see any easy fix older Python versions. I close the issue.
History
DateUserActionArgs
2022-04-11 14:58:52adminsetgithub: 75654
2019-05-15 02:45:20vstinnersetstatus: open -> closed
resolution: fixed
messages: +msg342538

stage: needs patch -> resolved
2018-12-03 16:45:31vstinnersetmessages: +msg330959
2018-12-03 16:43:57vstinnersettitle: PyMem_Raw* API in debug mode are not thread safe -> Debug hooks on memory allocators are not thread safe (serialno variable)
versions: + Python 2.7, Python 3.8
2018-12-03 16:41:18vstinnersetmessages: +msg330957
2018-12-03 15:22:40vstinnersetmessages: +msg330949
2018-12-03 15:21:01vstinnersetmessages: +msg330948
2018-12-03 15:19:07vstinnersetmessages: +msg330947
2017-09-14 17:36:34xiang.zhangcreate
Supported byThe Python Software Foundation,
Powered byRoundup
Copyright © 1990-2022,Python Software Foundation
Legal Statements

[8]ページ先頭

©2009-2026 Movatter.jp