While this is probablydesirable, I'm not quite sure if it'sfeasible. With that said, several people (@vstinner at the sprint and@zooba during PR review) both expressed a desire to remove the LLVM build-time dependency for JIT builds. Let's have that conversation here.

Background

When building CPython with the JIT enabled,LLVM 16LLVM 18 is used to compileTools/jit/template.c many times, and process the resulting object files into a file calledjit_stencils.h in the build directory.

A useful analogy

Because this file depends onPython.h (and thuspyconfig.h and many build-specific configuration options, including things like_DEBUG/NDEBUG/Py_DEBUG/etc.) and contains binary code, it is probably most useful to think ofjit_stencils.h as a binary extension module.

If we could build, host, and manage compiled versions of, say,itertoolsmodule.c somewhere and have it work correctly for those who need it, then such a scheme would probably work forjit_stencils.h.

Open questions

• Can this be done in a way that actually works correctly and is worth the trouble (the status quo being "download LLVM 18 if you want to build the JIT").
• Should we just try to host these for the most common build configurations? Or "everything"?
• Should all platforms be in one file (with each platform guarded by#ifdefs), or many files?
• Should these files be checked in? Or hosted somewhere? Who/what builds them? How often?
• Does this introduce any new attack vectors?
• What should the workflow look like for:
  • ...those developing the JIT?
  • ...those changing header files that the JIT depends on?
  • ...those building CPython with a JIT from a random commit?
  • ...those building CPython with a JIT from a release tag?

Linked PRs

• GH-115869: Makejit_stencils.h reproducible #127166
• GH-115869: Reference implementation for hosting JIT stencils #129331
• GH-115869: Don't JIT zeroed bytes #130023