Feature or enhancement

Proposal:

We should audit every built-in module for thread safety and make any necessary fixes. This can be done separately from tagging the modules as safe usingPy_mod_gil; any data races in built-in modules are considered bugs in the free-threaded build and not incompatibilities.

Below is a list of all source files that contain at least one module definition, as defined by files that contain either a call toPyModule_Create() or an array ofPyModuleDef_Slots. If you'd like to help out with this task, take a look at one of the incomplete files. If the conversion is trivial, check it off and attach the PR to this issue. Otherwise, convert the line into an issue and assign it to yourself.

Every module is different, but here are a few high-level points to guide the work:

1. UsePyMutex as the basic unit of locking.
2. If you need to hold a lock for a longer period of time, especially across calls that may reenter Python code or acquire other locks, usecritical sections.
3. If the module you're looking at is a thin wrapper around related code elsewhere in CPython (e.g.,Modules/_codecsmodule.c andPython/codecs.c), you can also audit/convert the related non-module code. Otherwise, try to contain your work to just the module code, and create separate issues for any dependencies that aren't thread-safe.
4. Remember that C API functions are generally expected to handle thread-safety internally. C code that operates on Python objects using only C API calls is usually thread-safe by default (but look out for code that reads or modifiesPyObject*s in C structs, since that needs synchronization).
5. Watch out for functions/macros that return borrowed references, likePyList_GetItem() orPyDict_GetItem(). If other threads could have references to the object, prefer functions likePyList_GetItemRef() orPyDict_GetItemRef() that return owned references (and will safely raise an error if the index/key doesn't exist).

Files to audit

• Modules/_abc.c
• Modules/arraymodule.c -gh-128942: make arraymodule.c free-thread safe (lock-free) #130771
• Modules/_asynciomodule.c
• Modules/atexitmodule.c
• Modules/binascii.c
• Modules/_bisectmodule.c
• Modules/_blake2/blake2module.c
• Modules/_bz2module.c
• Modules/cjkcodecs/cjkcodecs.h
• Modules/cjkcodecs/multibytecodec.c
• Modules/cmathmodule.c
• Modules/_codecsmodule.c
• Modules/_collectionsmodule.c
• Modules/_contextvarsmodule.c
• Modules/_csv.c
• Modules/_ctypes/_ctypes.c
• Modules/_ctypes/_ctypes_test.c
• Modules/_cursesmodule.c
• Modules/_curses_panel.c
• Modules/_datetimemodule.c
• Modules/_dbmmodule.c
• Modules/_decimal/_decimal.c
• Modules/_elementtree.c
• Modules/errnomodule.c
• Modules/faulthandler.c
• Modules/fcntlmodule.c
• Modules/_functoolsmodule.c
• Modules/gcmodule.c
• Modules/_gdbmmodule.c
• Modules/grpmodule.c
• Modules/_hashopenssl.c
• Modules/_heapqmodule.c
• Modules/_io/_iomodule.c
• Modules/itertoolsmodule.c
• Modules/_json.c -gh-116738: Make _json module safe in the free-threading build #119438
• Modules/_localemodule.c
• Modules/_lsprof.c
• Modules/_lzmamodule.c
• Modules/mathmodule.c
• Modules/md5module.c
• Modules/mmapmodule.c
• Modules/_multiprocessing/multiprocessing.c
• Modules/_multiprocessing/posixshmem.c
• Modules/_opcode.c
• Modules/_operator.c
• Modules/overlapped.c
• Modules/_pickle.c
• Modules/posixmodule.c
• Modules/_posixsubprocess.c
• Modules/pwdmodule.c
• Modules/pyexpat.c
• Modules/_queuemodule.c
• Modules/_randommodule.c
• Modules/readline.c
• Modules/resource.c
• Modules/_scproxy.c
• Modules/selectmodule.c
• Modules/sha1module.c
• Modules/sha2module.c
• Modules/sha3module.c
• Modules/signalmodule.c
• Modules/socketmodule.c
• Modules/_sqlite/module.c
• Modules/_sre/sre.c
• Modules/_ssl.c
• Modules/_stat.c
• Modules/_statisticsmodule.c
• Modules/_struct.c
• Modules/_suggestions.c
• Modules/symtablemodule.c
• Modules/_sysconfig.c
• Modules/syslogmodule.c
• Modules/termios.c
• Modules/_testbuffer.c
• Modules/_testcapimodule.c
• Modules/_testclinic.c
• Modules/_testclinic_limited.c
• Modules/_testexternalinspection.c
• Modules/_testimportmultiple.c
• Modules/_testinternalcapi.c
• Modules/_testlimitedcapi.c
• Modules/_testmultiphase.c
• Modules/_testsinglephase.c
• Modules/_threadmodule.c
• Modules/timemodule.c
• Modules/_tkinter.c
• Modules/_tracemalloc.c
• Modules/_typingmodule.c
• Modules/unicodedata.c
• Modules/_uuidmodule.c
• Modules/_weakref.c
• Modules/_winapi.c
• Modules/_xxinterpchannelsmodule.c
• Modules/_xxinterpqueuesmodule.c
• Modules/xxlimited_35.c
• Modules/xxlimited.c
• Modules/xxmodule.c
• Modules/_xxsubinterpretersmodule.c
• Modules/xxsubtype.c
• Modules/_xxtestfuzz/_xxtestfuzz.c
• Modules/zlibmodule.c
• Modules/_zoneinfo.c
• Objects/genobject.c (including ag_running_async)
• PC/msvcrtmodule.c
• PC/python3dll.c
• PC/winreg.c
• PC/winsound.c
• Python/bltinmodule.c
• Python/import.c
• Python/instrumentation.c
• Python/marshal.c
• Python/Python-ast.c
• Python/Python-tokenize.c
• Python/sysmodule.c
• Python/_warnings.c

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://peps.python.org/pep-0703/, especiallythis section

Linked PRs

• gh-116738: Make _abc module thread-safe #117488
• gh-116738: Make_codecs module thread-safe #117530
• gh-116738: Make _csv module thread-safe #118344
• gh-116738: Make _json module safe in the free-threading build #119438
• gh-116738: Make the array built in module safe for free threaded python #120103
• [3.13] gh-116738: Make_csv module thread-safe (GH-118344) #125328
• gh-116738: remove multiprocessing suppressions #131319
• gh-116738: Make _heapq module thread-safe #135036
• gh-116738: Inline append for local PyList #135196
• [3.14] gh-116738: Make _heapq module thread-safe (GH-135036) #135309
• gh-116738: Make grp module thread-safe #135434
• GH-116738: document thread-safety of bisect #136555
• [3.14] gh-116738: Make grp module thread-safe (GH-135434) #136658
• gh-116738: Make pwd module thread-safe #136695
• gh-116738: Make syslog module thread-safe #136760
• gh-116738: Make bisect module thread-safe #137021
• [3.14] GH-116738: document thread-safety of bisect (GH-136555) #137221
• [3.13] GH-116738: document thread-safety of bisect (GH-136555) #137222
• gh-116738: Make cProfile module thread-safe #138229
• gh-116738: Fix test_json_mutating_exact_dict #138339
• gh-116738: Audit thread-safety for termios module #138395
• gh-116738: testdbm.gnu module on FT Python build #138467
• gh-116738: Test resource module on FT Python build #138504
• [3.14] gh-116738: makecProfile module thread-safe (GH-138229) #138575
• [3.14] gh-116738: add multi-threaded tests forresource module on free-threading builds (GH-138504) #138793
• gh-116738: Make mmap module thread-safe #139237
• [3.14] gh-116738: makemmap module thread-safe (GH-139237) #139825
• [3.14] gh-116738: testdbm.gnu module on FT Python build (GH-138467) #139987
• gh-116738: Test uuid module for free threading #140068
• [3.14] gh-116738: testuuid module thread safety in free-threading (GH-140068) #140229
• gh-116738: Make _suggestions module thread-safe #140321
• gh-116738: Add critical section to dbm/gdbm context manager #140391
• [3.14] gh-116738: Add critical section to dbm/gdbm context manager (gh-140391) #140459
• gh-116738: Use PyMutex for bz2 module #140555
• gh-116738: Use PyMutex for lzma module #140711
• gh-116738: Make csv module thread-safe #141365
• [3.14] gh-116738: Make csv module thread-safe (gh-141365) #141825
• gh-116738: Test re module for free threading #141923
• [3.14] gh-116738: Fix thread-safety issue in re module for free threading (gh-141923) #141990
• gh-116738: Make cmath module thread-safe #142161
• [3.14] gh-116738: Statically initialize special constants in cmath module (gh-142161) #142261
• gh-116738: Make zlib module thread-safe #142432
• gh-116738: Make _bz2 module thread-safe #142756