gh-145235: Make dict watcher API thread-safe for free-threaded builds#145233

Open

yoney wants to merge 3 commits intopython:mainfrom

yoney:watcher

Open

gh-145235: Make dict watcher API thread-safe for free-threaded builds#145233
yoney wants to merge 3 commits intopython:mainfrom
yoney:watcher

Conversation

Copy link

Contributor

yoney commentedFeb 25, 2026•
edited
Loading

In free-threaded builds, concurrent calls toPyDict_AddWatcher,PyDict_ClearWatcher,PyDict_Watch, andPyDict_Unwatch can race on the shared callback array and the per-dict watcher tags. This change adds a mutex to serialize watcher registration and removal, atomic operations for tag updates, and atomic acquire/release synchronization for callback dispatch in_PyDict_SendEvent.

Note: Before this change, runningtest.test_free_threading.test_dict_watcher emitted tsan warnings in the free-threaded tsan build.

cc:@colesbury @mpage @DinoV

Issue:Dict watcher API is not thread-safe in free-threaded builds #145235

Make dict watcher API thread-safe for free-threaded builds

da3c401

yoney changed the title~~Make dict watcher API thread-safe for free-threaded builds~~gh-145235: Make dict watcher API thread-safe for free-threaded builds

Feb 25, 2026

bedevere-appbot mentioned this pull request

Feb 25, 2026

Dict watcher API is not thread-safe in free-threaded builds#145235

Open

Add g_dict_watch_once to ignored

fd936f3

yoney marked this pull request as ready for review

February 25, 2026 22:13

yoney requested review fromFidget-Spinner,ZeroIntensity,ericsnowcurrently,markshannon,methane,savannahostrowski andtomasr8 ascode owners

February 25, 2026 22:13

bedevere-appbot added the awaiting review label

Feb 25, 2026

colesbury reviewed

Feb 25, 2026

View reviewed changes

Include/internal/pycore_dict_state.h OutdatedShow resolvedHide resolved

Objects/dictobject.c OutdatedShow resolvedHide resolved

Include/internal/pycore_dict_state.hShow resolvedHide resolved

Objects/dictobject.cShow resolvedHide resolved

Python/optimizer_analysis.c Outdated

		PyInterpreterState *interp = _PyInterpreterState_GET();
		if (interp->dict_state.watchers[GLOBALS_WATCHER_ID] == NULL) {
		interp->dict_state.watchers[GLOBALS_WATCHER_ID] = globals_watcher_callback;
		if (FT_ATOMIC_LOAD_PTR_RELAXED(interp->dict_state.watchers[GLOBALS_WATCHER_ID]) == NULL) {

Copy link

Contributor

colesburyFeb 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

A_PyOnceFlag (maybe in JitOptContext) might be a better fit here.

Copy link

ContributorAuthor

yoneyFeb 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

After looking into it (with Claude),JitOptContext seems to be per-optimization-run so_PyOnceFlag might not the right place. I put it in_Py_dict_state as I couldn't find a better place. It fits in the struct packing without extra space.