Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32.2k
gh-134273: Allow setting JIT compiler flags withCFLAGS_JIT
#134276
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks! These should also be part of the hash in_Target._compute_digest
.
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
BTW, JIT CI is broken on macOS and Windows currently, so if those fail (build failure on Windows and 2 failing tests on macOS), it's not your fault. |
Technically, this should probably also support |
It's up to you. If you want to add |
brandtbucher commentedMay 20, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
So, this breaks on universal builds, since that sets I think part of the problem is that we don't want to use an option for the JITjust because it's being used for the interpreter itself. We want very tight control over the code we're generating, especially since the parse step is sort of fragile. What flags are you needing to pass, exactly? Is it only include paths? |
zanieb commentedMay 20, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I onlyneed the include flag for my build to pass. But I only caught#134291 because all of my flags were passed through. It's possible there are other subtle differences. I'm totally fine with a |
CFLAGS_JIT
I've updated to use |
@brandtbucher gentle nudge, so I don't need to maintain patches in perpetuity :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for doing this! Just a couple of suggestions, then I'll land:
Tools/jit/_targets.py Outdated
@@ -120,6 +123,7 @@ async def _compile( | |||
) -> _stencils.StencilGroup: | |||
o = tempdir / f"{opname}.o" | |||
args = [ | |||
*shlex.split(self.cflags), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
You know what? I changed my mind.
Let's move these to the end, so people can override our options if they really need to. Anyone using this option should know what they're doing.
Uh oh!
There was an error while loading.Please reload this page.
Misc/NEWS.d/next/Build/2025-05-19-18-09-20.gh-issue-134273.ZAliyy.rst OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
Uh oh!
There was an error while loading.Please reload this page.
Thanks! I just pushed the last change myself. |
Sorry, I just pulled this down to play with it locally, and I have one question. It seems suboptimal to have to set the environment variable when actually running $ CFLAGS_JIT="-Icustom/include/dir" ./configure --enable-experimental-jit$# ...some time later...$ make What do you think about this patch, which will capture the variable at configure time and hardcode it in the Makefile? I think this is closer to what we do for similar things like diff --git a/configure.ac b/configure.acindex 9ab4b370f1c..f6091c21b2a 100644--- a/configure.ac+++ b/configure.ac@@ -2752,7 +2752,6 @@ AS_VAR_IF([ac_cv_gcc_compat], [yes], [ ]) # Check for --enable-experimental-jit:-AC_SUBST([CFLAGS_JIT]) AC_MSG_CHECKING([for --enable-experimental-jit]) AC_ARG_ENABLE([experimental-jit], [AS_HELP_STRING([--enable-experimental-jit@<:@=no|yes|yes-off|interpreter@:>@],@@ -2777,7 +2776,7 @@ AS_VAR_IF([jit_flags], [], [AS_VAR_APPEND([CFLAGS_NODIST], [" $jit_flags"]) AS_VAR_SET([REGEN_JIT_COMMAND],- ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"\$(CFLAGS_JIT)\""])+ ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\""]) AS_VAR_SET([JIT_STENCILS_H], ["jit_stencils.h"]) AS_VAR_IF([Py_DEBUG], [true],diff --git a/configure b/configureindex 68213817679..6170e67e6e2 100755--- a/configure+++ b/configure@@ -906,7 +906,6 @@ DSYMUTIL_PATH DSYMUTIL JIT_STENCILS_H REGEN_JIT_COMMAND-CFLAGS_JIT UNIVERSAL_ARCH_FLAGS WASM_STDLIB WASM_ASSETS_DIR@@ -10833,7 +10832,6 @@ esac fi # Check for --enable-experimental-jit:- { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --enable-experimental-jit" >&5 printf %s "checking for --enable-experimental-jit... " >&6; } # Check whether --enable-experimental-jit was given.@@ -10865,7 +10863,7 @@ then : else case e in #( e) as_fn_append CFLAGS_NODIST " $jit_flags"- REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"\$(CFLAGS_JIT)\""+ REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\"" JIT_STENCILS_H="jit_stencils.h" if test "x$Py_DEBUG" = xtrue then : |
Oh, that's how I thought itwas working originally and definitely my intent. I guess I broke that when I changed it to
If the patch works, that's great. |
(I can alsoactually use this patch in |
@zanieb did you get a chance to try this out? I'll land if so. |
I didn't try it yet, I can do so now. |
Seems happyastral-sh/python-build-standalone#659 |
2b0c684
intopython:mainUh oh!
There was an error while loading.Please reload this page.
@hugovk could we have this in 3.14 please? This is ostensively a bug in the build system and I have to maintain this patch downstream. |
@zanieb Yes, that's okay. |
Thanks@zanieb for the PR, and@brandtbucher for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14. |
Uh oh!
There was an error while loading.Please reload this page.