Myself,@lysnikolaou, and@rgommers are working on a new project in collaboration with the Python Runtime team at Meta to add support for Python 3.13 nogil builds in the scientific python software stack.

Our initial minimal goal is a build of NumPy that runs and passes all the tests in a nogil build of Python.

Resources:

• PEP 703

• Fork of numpy 1.24 with early nogil support

• Add changelog entry and docs describing the current state of using NumPy in a multithreaded environment.

  • DOC: add free-threading release notes #27200

• ConvertPyDict_GetItem andPyList_GetItem to use variants returning strong references (MNT: Add linter for thread-unsafe C API uses #26159)

  • Vendorpythoncapi-compat in a submodule (MNT: Adopt pythoncapi-compat? #26158)

• TSK: Make NumPy buildable on the nogil branch #26161

  • BUG: fix reference counting error in wrapping_method_resolve_descriptors #26190
  • TST: account for immortal objects in test_iter_refcount #26207

• Set up CPython 3.13 CI

  • 3.13 gil enabled build (TST: run the smoke tests on more python versions #26284)
  • --disable-gil build (TST: add basic free-threaded CI testing #26463)

• Make the test suite pass with the gil disabled

• Mark NumPy C extensions withPy_MOD_GIL_NOT_USED.

  • MAINT: declare that NumPy's C extensions support running without the GIL #26913
  • MAINT: add freethreading_compatible directive to cython build #26950

• Allow building f2py extensions that can be imported without the GIL.

  • ENH: add support in f2py to declare gil-disabled support #26981

• Use PyMutex instead of PyThread_type_lock

  • MAINT: replace PyThread_type_lock with PyMutex on Python >= 3.13.0b3 #27011

• Nightly wheels forcp313t ABI

  • manylinux and musllinuxCI: enable free-threaded wheel builds [wheel build] #26512
  • MacOSCI: build and upload free-threaded nightly wheels for macOS #26664
  • WindowsCI: start building Windows free-threaded wheels #27560

• Global state

  • C static variables in_multiarray_umath reorganized into structs:MNT: Reorganize non-constant global statics into structs #26607
  • Global caches need to be disabled, have locking, serialize creating read-only caches, or converted to thread-local caches.
    • Disabled:
      • MNT: disable the allocator cache for nogil builds #26251
      • MNT: disable the coercion cache for the nogil build #26294
      • NOGIL: Make loop data cache and dispatch cache thread-safe in nogil build #26348
    • Locking:
      • NOGIL: Make loop data cache and dispatch cache thread-safe in nogil build #26348
      • ENH: add locking around initializing the argparse cache #26430
      • ENH: Add locking to umath_linalg if no lapack is detected at build time #26750
    • Cache initialization moved to module initialization
      • MNT: Reorganize non-constant global statics into structs #26607
    • Some runtime caches need a single-init API.MAINT: use an atomic load/store and a mutex to initialize the argparse and runtime import caches #26780
    • Dragon4 scientific printing code uses a static global scratch space.
    • Triaged low priority:
      • wrapping_array_method.c uses a freelist that is certainly not thread safe. It has no internal consumers besides_scaled_float_dtype to test the implementation so ignoring it for now.
      • the number of legacy user dtypes numpy knows about is held in a global static variable:NPY_NUMUSERTYPES, which also unfortunately is part of the public C API. This one may just need to remain unfixed, given it's in a legacy API.
      • The check to force an error on module reloading isn't thread safe. Probably needs an atomic to guard initializing it.
  • Global settings and state may need to be made per-thread or synchronized across threads.
    • Converted to thread-local state:
      • [NOGIL] thread local promotion state #26368
      • MAINT: mark evil_global_disable_warn_O4O8_flag as thread-local #26676
    • np.set_string_function andPyArray_SetStringFunction are implemented using static global state.
      • These are removed, seeMNT:np.set_string_function implementation should be removed #26576,MNT: Removeset_string_function #26611
    • Make printoptions a context variable.MAINT: back printoptions with a true context variable #26846
    • hugepage support is controlled by a single static global. The only public interface to control this is an environment variable read at startup, so this isn't problematic unless someone is using private APIs.
    • NUMPY_WARN_IF_NO_MEMORY_POLICY is implemented using a global static. The only public interface is via an environment variable read at module initialization but it's tested using a thread-unsafe interface.