Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-139109: A new tracing JIT compiler frontend for CPython#140310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Fidget-Spinner merged 196 commits intopython:mainfromFidget-Spinner:tracing_jit
Nov 13, 2025

Conversation

@Fidget-Spinner
Copy link
Member

@Fidget-SpinnerFidget-Spinner commentedOct 18, 2025•
edited
Loading

This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus currenthttps://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT.Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR. The speedup in the merged version is about 1.1%.https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

Stats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long abortshttps://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .

This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this workgh-140277

Some tests for the optimizers are disabled for now because their invariants have changed. The main one is the stack space check, as it's no longer valid to deal with underflow.

The full test suite passes on my system.

Not close enough to a holiday, so no poem this time!

Pros:

  • Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.
  • optimizer.c is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.
  • The new JIT frontend is able to handle a lot more control-flow than the old one.
  • Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.
  • Better handling of polymorphism. We leverage the specializing interpreter for this.

Cons:

Design:

  • After each instruction, therecord_previous_inst function/label is executed. This does as the name suggests.
  • The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.
  • The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.
  • The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.
  • The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.
  • Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.
  • Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction ---_DYNAMIC_EXIT.

Future ideas:

  • _DYNAMIC_EXIT will cause a re-trace when hot enough starting from the target instruction.
  • As such, traces "grow" like a web/graph rather than a tree. It starts from a loop, then grows outwards naturally consuming more code around it as the hotspot gets hotter.

AA-Turner, Zheaoli, corona10, brettcannon, nmstoker, and tekknolagi reacted with hooray emojistonebig reacted with heart emojiemmatyping, dolfinus, Sacul0457, chris-eibl, Wulian233, savannahostrowski, efimov-mikhail, rafaeljcd, Zheaoli, sergey-miryanov, and 10 more reacted with rocket emoji
@Fidget-Spinner
Copy link
MemberAuthor

Fidget-Spinner commentedNov 12, 2025•
edited
Loading

The results after trusting the interpreter with specialization is slower.https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

This likely indicates bugs in the specializing interpreter. For example, deltablue is suffering from the CALL_LIST_APPEND bug.

I say we just merge the "slower" version though. We should fix the bugs in the interpreter separately.

@markshannon
Copy link
Member

I'm not worried about the deltablue slowdown. It subclasses a list for no reason, which I don't think we care about.

@markshannon
Copy link
Member

Is the iOS failure anything to do with this PR or is it failing elsewhere?

GOTO_TIER_ONE(target);
}

tier2op(_GUARD_IP__PUSH_FRAME, (ip/4--)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I thought the plan was to generate these exits, otherwise they get out of sync with the source instruction.

What isOFFSET_OF(x)? The offset of what?

If generating the whole uop is too cumbersome because of uop ids and such, then generate the body, leaving the declaration looking like this:

tier2op(_GUARD_IP__PUSH_FRAME, (ip/4--)) {IP_GUARD(_PUSH_FRAME);}

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Itis automatically generating it from the offset of the IP.

I've changed the name toIP_OFFSET_OF to make it clearer.

DISPATCH();
}

label(record_previous_inst) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

NOTE: To support the switch/case interpreter this will need to be an instruction.
No need to do so in this PR.

@mhsmith
Copy link
Member

Is the iOS failure anything to do with this PR or is it failing elsewhere?

It's been failing intermittently for a few weeks:#141358 (comment).

Copy link
Member

@markshannonmarkshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Final set of requests.

You seem to be usingcode_curr_size in various places for checking things other than just the uop count, such as whether tracing is active.
Can you add comments and/or names for the constants for all comparisons involvingcode_curr_size?

}
if (jump_target!=current_jump_target||current_exit_op!=exit_op) {
make_exit(&buffer[next_spare],exit_op,jump_target);
boolis_control_flow= (opcode==_GUARD_IS_FALSE_POP||opcode==_GUARD_IS_TRUE_POP||is_for_iter_test[opcode]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think this is the only use ofis_for_iter_test, in which case replace it withis_control_flow and simplify this test.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

No it's not. We also use it to decide to offset the target to pointer after the END_FOR too.

/* Special case the first instruction,
* so that we can guarantee forward progress */
if (progress_needed&&_tstate->jit_tracer_state.prev_state.code_curr_size <=3) {
if (progress_needed&&_tstate->jit_tracer_state.prev_state.code_curr_size <=CODE_SIZE_NO_PROGRESS) {
Copy link
Member

@markshannonmarkshannonNov 13, 2025•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is different. You've changed the rhs from 3 to 5.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Yes it should beCODE_SIZE_NO_PROGRESS. I realised the problem is that it's< not<= though!

@markshannonmarkshannon self-requested a reviewNovember 13, 2025 16:30
Copy link
Member

@markshannonmarkshannon left a comment•
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is a big step forward for the JIT.
@Fidget-Spinner thanks again for doing this.

sergey-miryanov, hugovk, tekknolagi, stonebig, Sacul0457, and AIdjis reacted with hooray emojiFidget-Spinner, efimov-mikhail, sergey-miryanov, ZeroIntensity, AlexWaygood, chris-eibl, Sacul0457, AIdjis, and Banicoder77 reacted with heart emoji
@Fidget-Spinner
Copy link
MemberAuthor

CI caught a bug in the dependency tracking. I fixed it in the latest commit.

@Fidget-SpinnerFidget-Spinner merged commit4fa80ce intopython:mainNov 13, 2025
72 checks passed
@Fidget-SpinnerFidget-Spinner deleted the tracing_jit branchNovember 13, 2025 18:08
@efimov-mikhail
Copy link
Member

This is a big step forward for the JIT.@Fidget-Spinner thanks again for doing this.

Congratulations,@Fidget-Spinner !

Fidget-Spinner, ZeroIntensity, rafalp, and Sacul0457 reacted with heart emojisavannahostrowski, Fidget-Spinner, ZeroIntensity, chris-eibl, rafalp, and Sacul0457 reacted with rocket emoji

CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull requestNov 13, 2025
* main: (463 commits)pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)pythongh-137109: refactor warning about threads when forking (python#141438)pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)pythongh-141442: Add escaping to iOS testbed arguments (python#141443)pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull requestNov 14, 2025
* 'main' of github.com:python/cpython: (464 commits)pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)pythongh-137109: refactor warning about threads when forking (python#141438)pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)pythongh-141442: Add escaping to iOS testbed arguments (python#141443)pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)  ...
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull requestNov 14, 2025
* 'main' of github.com:python/cpython: (464 commits)pythongh-140601: Add ResourceWarning to iterparse when not closed (pythonGH-140603)pythongh-137969: Fix double evaluation of `ForwardRef`s which rely on globals (python#140974)pythongh-139109: A new tracing JIT compiler frontend for CPython (pythonGH-140310)pythongh-141004: Document `PyErr_RangedSyntaxLocationObject` (python#141521)pythongh-140873: Add support of non-descriptor callables in functools.singledispatchmethod() (pythonGH-140884)pythongh-139653: Add PyUnstable_ThreadState_SetStackProtection() (python#139668)pythongh-141004: Document `PyCode_Optimize` (pythonGH-141378)pythongh-141004: Document C APIs for dictionary keys, values, and items (pythonGH-141009)pythongh-137959: Fix `TIER1_TO_TIER2` macro name in JIT InternalDocs (pythonGH-141496)pythongh-139871: Add `bytearray.take_bytes([n])` to efficiently extract `bytes` (pythonGH-140128)pythongh-140601: Refactor ElementTree.iterparse() tests (pythonGH-141499)pythongh-135801: Add the module parameter to compile() etc (pythonGH-139652)pythongh-140260: fix data race in `_struct` module initialization with subinterpreters (python#140909)pythongh-137109: refactor warning about threads when forking (python#141438)pythongh-141004: Document `PyRun_InteractiveOneObject` (pythonGH-141405)pythongh-124111: Fix TCL 9 thread detection (pythonGH-128103)pythongh-141442: Add escaping to iOS testbed arguments (python#141443)pythongh-140936: Fix JIT assertion crash at finalization if some generator is alive (pythonGH-140969)  Add details about JIT build infrastructure and updating dependencies to `Tools/jit` (python#141167)pythongh-141412: Use reliable target URL for urllib example (pythonGH-141428)  ...
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull requestDec 6, 2025
…honGH-140310)This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus currenthttps://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT. **Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR**. The speedup in the merged version is about 1.1%.https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svgStats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long abortshttps://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this workpythongh-140277The optimizer stack space check is disabled, as it's no longer valid to deal with underflow.Pros:* Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.* `optimizer.c` is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.* The new JIT frontend is able to handle a lot more control-flow than the old one.* Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.* Better handling of polymorphism. We leverage the specializing interpreter for this.Cons:* (For now) requires tail calling interpreter or computed gotos. This means no Windows JIT for now :(. Not to fret, tail calling is coming soon to Windows thoughpython#139962Design:* After each instruction, the `record_previous_inst` function/label is executed. This does as the name suggests.* The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.* The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.* The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.* The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.* Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.* Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction --- `_DYNAMIC_EXIT`.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@sergey-miryanovsergey-miryanovsergey-miryanov left review comments

@efimov-mikhailefimov-mikhailefimov-mikhail left review comments

@savannahostrowskisavannahostrowskisavannahostrowski left review comments

@kumaraditya303kumaraditya303kumaraditya303 left review comments

@chris-eiblchris-eiblchris-eibl left review comments

@markshannonmarkshannonmarkshannon approved these changes

@ericsnowcurrentlyericsnowcurrentlyAwaiting requested review from ericsnowcurrentlyericsnowcurrently is a code owner

@ZeroIntensityZeroIntensityAwaiting requested review from ZeroIntensityZeroIntensity is a code owner

@tomasr8tomasr8Awaiting requested review from tomasr8tomasr8 is a code owner

@brandtbucherbrandtbucherAwaiting requested review from brandtbucherbrandtbucher is a code owner

@diegorussodiegorussoAwaiting requested review from diegorussodiegorusso is a code owner

@erlend-aaslanderlend-aaslandAwaiting requested review from erlend-aasland

@AA-TurnerAA-TurnerAwaiting requested review from AA-TurnerAA-Turner is a code owner

@emmatypingemmatypingAwaiting requested review from emmatyping

@ezio-melottiezio-melottiAwaiting requested review from ezio-melottiezio-melotti is a code owner

@hugovkhugovkAwaiting requested review from hugovkhugovk is a code owner

@Eclips4Eclips4Awaiting requested review from Eclips4Eclips4 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@Fidget-Spinner@markshannon@Eclips4@mhsmith@efimov-mikhail@sergey-miryanov@hugovk@savannahostrowski@kumaraditya303@chris-eibl
[8]ページ先頭
©2009-2025 Movatter.jp