Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-134273: Allow setting JIT compiler flags withCFLAGS_JIT#134276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
brandtbucher merged 12 commits intopython:mainfromzanieb:zb/cflags-jit
Jun 12, 2025

Conversation

zanieb
Copy link
Contributor

@zaniebzanieb commentedMay 19, 2025
edited
Loading

Copy link
Member

@brandtbucherbrandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks! These should also be part of the hash in_Target._compute_digest.

zanieb reacted with thumbs up emoji
@bedevere-app
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@brandtbucher
Copy link
Member

BTW, JIT CI is broken on macOS and Windows currently, so if those fail (build failure on Windows and 2 failing tests on macOS), it's not your fault.

zanieb reacted with laugh emoji

@zanieb
Copy link
ContributorAuthor

Technically, this should probably also supportCFLAGS_NODIST? I'm not sure.

@brandtbucher
Copy link
Member

It's up to you. If you want to addCFLAGS_NODIST, just use the same--with-cflags option to pass them though (the build script doesn't care which is which).

@brandtbucher
Copy link
Member

brandtbucher commentedMay 20, 2025
edited
Loading

So, this breaks on universal builds, since that setsCONFIGURE_CFLAGS=-arch arm64 -arch x86_64. See the AArch64 macOS failures in CI (it's different from the expected test failures).

I think part of the problem is that we don't want to use an option for the JITjust because it's being used for the interpreter itself. We want very tight control over the code we're generating, especially since the parse step is sort of fragile.

What flags are you needing to pass, exactly? Is it only include paths?

@zanieb
Copy link
ContributorAuthor

zanieb commentedMay 20, 2025
edited
Loading

I onlyneed the include flag for my build to pass. But I only caught#134291 because all of my flags were passed through. It's possible there are other subtle differences. I'm totally fine with aCFLAGS_JIT, personally. I feel like people are opposed to new CFLAGS variants, but here it makes a lot of sense.

@zaniebzanieb changed the titlegh-134273: Propagate CFLAGS to the JITgh-134273: Allow setting JIT compiler flags withCFLAGS_JITMay 27, 2025
@zanieb
Copy link
ContributorAuthor

I've updated to useCFLAGS_JIT

@zanieb
Copy link
ContributorAuthor

@brandtbucher gentle nudge, so I don't need to maintain patches in perpetuity :)

Copy link
Member

@brandtbucherbrandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for doing this! Just a couple of suggestions, then I'll land:

@@ -120,6 +123,7 @@ async def _compile(
) -> _stencils.StencilGroup:
o = tempdir / f"{opname}.o"
args = [
*shlex.split(self.cflags),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

You know what? I changed my mind.

Let's move these to the end, so people can override our options if they really need to. Anyone using this option should know what they're doing.

zanieb reacted with thumbs up emoji
zanieband others added2 commitsJune 6, 2025 08:27
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
@brandtbucher
Copy link
Member

Thanks! I just pushed the last change myself.

zanieb reacted with thumbs up emoji

@brandtbucher
Copy link
Member

Sorry, I just pulled this down to play with it locally, and I have one question. It seems suboptimal to have to set the environment variable when actually runningmake. Itthink the way we prefer to do things like this is to "lock-in" configuration like this at configure time instead. For example, this won't currently work:

$ CFLAGS_JIT="-Icustom/include/dir" ./configure --enable-experimental-jit$# ...some time later...$ make

What do you think about this patch, which will capture the variable at configure time and hardcode it in the Makefile? I think this is closer to what we do for similar things likeCFLAGS_NODIST. I'm not an Autoconf or build system expert though, so I could be missing something:

diff --git a/configure.ac b/configure.acindex 9ab4b370f1c..f6091c21b2a 100644--- a/configure.ac+++ b/configure.ac@@ -2752,7 +2752,6 @@ AS_VAR_IF([ac_cv_gcc_compat], [yes], [ ])  # Check for --enable-experimental-jit:-AC_SUBST([CFLAGS_JIT]) AC_MSG_CHECKING([for --enable-experimental-jit]) AC_ARG_ENABLE([experimental-jit],               [AS_HELP_STRING([--enable-experimental-jit@<:@=no|yes|yes-off|interpreter@:>@],@@ -2777,7 +2776,7 @@ AS_VAR_IF([jit_flags],           [],           [AS_VAR_APPEND([CFLAGS_NODIST], [" $jit_flags"])            AS_VAR_SET([REGEN_JIT_COMMAND],-                      ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"\$(CFLAGS_JIT)\""])+                      ["\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\""])            AS_VAR_SET([JIT_STENCILS_H], ["jit_stencils.h"])            AS_VAR_IF([Py_DEBUG],                      [true],diff --git a/configure b/configureindex 68213817679..6170e67e6e2 100755--- a/configure+++ b/configure@@ -906,7 +906,6 @@ DSYMUTIL_PATH DSYMUTIL JIT_STENCILS_H REGEN_JIT_COMMAND-CFLAGS_JIT UNIVERSAL_ARCH_FLAGS WASM_STDLIB WASM_ASSETS_DIR@@ -10833,7 +10832,6 @@ esac fi  # Check for --enable-experimental-jit:- { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --enable-experimental-jit" >&5 printf %s "checking for --enable-experimental-jit... " >&6; } # Check whether --enable-experimental-jit was given.@@ -10865,7 +10863,7 @@ then :  else case e in #(   e) as_fn_append CFLAGS_NODIST " $jit_flags"-           REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"\$(CFLAGS_JIT)\""+           REGEN_JIT_COMMAND="\$(PYTHON_FOR_REGEN) \$(srcdir)/Tools/jit/build.py ${ARCH_TRIPLES:-$host} --output-dir . --pyconfig-dir . --cflags=\"$CFLAGS_JIT\""            JIT_STENCILS_H="jit_stencils.h"            if test "x$Py_DEBUG" = xtrue then :

@zanieb
Copy link
ContributorAuthor

t seems suboptimal to have to set the environment variable when actually running make. It think the way we prefer to do things like this is to "lock-in" configuration like this at configure time instead.

Oh, that's how I thought itwas working originally and definitely my intent. I guess I broke that when I changed it toCFLAGS_JIT, thanks for checking!

What do you think about this patch, which will capture the variable at configure time and hardcode it in the Makefile?

If the patch works, that's great.

@zanieb
Copy link
ContributorAuthor

(I can alsoactually use this patch inpython-build-standalone to confirm it's working as intended, right now I'm using something hackier)

@brandtbucher
Copy link
Member

@zanieb did you get a chance to try this out? I'll land if so.

@zanieb
Copy link
ContributorAuthor

I didn't try it yet, I can do so now.

@zanieb
Copy link
ContributorAuthor

@brandtbucherbrandtbucher merged commit2b0c684 intopython:mainJun 12, 2025
62 checks passed
@zanieb
Copy link
ContributorAuthor

@hugovk could we have this in 3.14 please? This is ostensively a bug in the build system and I have to maintain this patch downstream.

@hugovk
Copy link
Member

@zanieb Yes, that's okay.

@zaniebzanieb added the needs backport to 3.14bugs and security fixes labelJun 21, 2025
@miss-islington-app
Copy link

Thanks@zanieb for the PR, and@brandtbucher for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@brandtbucherbrandtbucherbrandtbucher approved these changes

@savannahostrowskisavannahostrowskiAwaiting requested review from savannahostrowskisavannahostrowski is a code owner

@erlend-aaslanderlend-aaslandAwaiting requested review from erlend-aaslanderlend-aasland is a code owner

@corona10corona10Awaiting requested review from corona10corona10 is a code owner

Assignees
No one assigned
Labels
needs backport to 3.14bugs and security fixes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@zanieb@brandtbucher@hugovk

[8]ページ先頭

©2009-2025 Movatter.jp