Dec 6, 2024 · Nov 22, 2024 · Nov 22, 2024 · Nov 23, 2024 · Nov 28, 2024 · Dec 5, 2024
diff --git a/InternalDocs/README.md b/InternalDocs/README.md

 - [The Specializing Interpreter](adaptive.md)

 - [The Tier 2 Interpreter](tier2.md)
 - [The Tier 2 Interpreter and JIT](tier2.md)

 - [Garbage Collector Design](garbage_collector.md)

diff --git a/InternalDocs/tier2.md b/InternalDocs/tier2.md
 [`--enable-experimental-jit=interpreter`](https://docs.python.org/dev/using/configure.html#cmdoption-enable-experimental-jit)),
 the executor jumps to `tier2_dispatch:` in
 [`Python/ceval.c`](../Python/ceval.c), where there is a loop that
 executes the micro-ops which are defined in
 [`Python/executor_cases.c.h`](../Python/executor_cases.c.h).
 executes the micro-ops. The micro-ops are are defined in
 [`Python/executor_cases.c.h`](../Python/executor_cases.c.h),
 which is generated by the build script
 [`Tools/cases_generator/tier2_generator.py`](../Tools/cases_generator/tier2_generator.py)
 from the bytecode definitions in
 [`Python/bytecodes.c`](../Python/bytecodes.c).
 This loop exits when an `_EXIT_TRACE` or `_DEOPT` uop is reached,
 and execution returns to teh tier 1 interpreter.

 state's `executor_list_head` field. This list is used when it is necessary
 to invalidate executors because values that their construction depended
 on may have changed.

 ## The JIT

 When the jit is enabled (python was configured with
 [`--enable-experimental-jit`](https://docs.python.org/dev/using/configure.html#cmdoption-enable-experimental-jit),
 the uop executor's `jit_code` field is populated with a pointer to a compiled
 C function that implement the executor logic. This function's signature is
 defined by `jit_func` in [`pycore_jit.h`](Include/internal/pycore_jit.h).
 When the executor is invoked by `ENTER_EXECUTOR`, instead of jumping to
 the uop interpreter at `tier2_dispatch`, the executor runs the function
 that `jit_code` points to. This function returns the instruction pointer
 of the next Tier 1 instruction that needs to execute.

 The generation of the jitted fuctions uses the copy-and-patch technique
 which is described in
 [Haoran Xu's article](https://sillycross.github.io/2023/05/12/2023-05-12/).
 At its core are statically generated `stencils` for the implementation
 of the micro ops, which are completed with runtime information while
 the jitted code is constructed for an executor by
 [`_PyJIT_Compile`](../Python/jit.c).

 The stencils are generated under the build target `regen-jit` by the scripts
 in [`/Tools/jit`](/Tools/jit). This script reads
 [`Python/executor_cases.c.h`](../Python/executor_cases.c.h) (which is
 generated from [`Python/bytecodes.c`](../Python/bytecodes.c)). For
 each opcode, it constructs a `.c` file that contains a function for
 implementing this opcode, with some runtime information injected.
 This is done by replacing `CASE` by the bytecode definition in the
 template file [`Tools/jit/template.c`](../Tools/jit/template.c).

 Each of the `.c` file is compiled by LLVM, to produce an object file
 that contains a function that executes the opcode. These compiled
 functions are used to generate the file
 [`jit_stencils.h`](../jit_stencils.h), which contains the functions
 that the JIT can use to emit code for each of the bytecodes.

 For Python maintainers this means that changes to the bytecodes and
 their implementations do not require changes related to the JIT,
 because everything the JIT needs is automatically generated from
 [`Python/bytecodes.c`](../Python/bytecodes.c) at build time.

 See Also:

 * [Copy-and-Patch Compilation: A fast compilation algorithm for high-level languages and bytecode](https://arxiv.org/abs/2011.13127)

 * [PyCon 2024: Building a JIT compiler for CPython](https://www.youtube.com/watch?v=kMO3Ju0QCDo)
Original file line number	Diff line number	Diff line change
Expand Up		@@ -38,7 +38,7 @@ Program Execution

		- [The Specializing Interpreter](adaptive.md)

		- [The Tier 2 Interpreter](tier2.md)
		- [The Tier 2 Interpreter and JIT](tier2.md)

		- [Garbage Collector Design](garbage_collector.md)

Expand Down
Original file line number	Diff line number	Diff line change
Expand Up		@@ -60,8 +60,12 @@ When tier 2 is enabled but the JIT is not (python was configured with
		[`--enable-experimental-jit=interpreter`](https://docs.python.org/dev/using/configure.html#cmdoption-enable-experimental-jit)),
		the executor jumps to `tier2_dispatch:` in
		[`Python/ceval.c`](../Python/ceval.c), where there is a loop that
		executes the micro-ops which are defined in
		[`Python/executor_cases.c.h`](../Python/executor_cases.c.h).
		executes the micro-ops. The micro-ops are are defined in
		[`Python/executor_cases.c.h`](../Python/executor_cases.c.h),
		which is generated by the build script
		[`Tools/cases_generator/tier2_generator.py`](../Tools/cases_generator/tier2_generator.py)
		from the bytecode definitions in
		[`Python/bytecodes.c`](../Python/bytecodes.c).
		This loop exits when an `_EXIT_TRACE` or `_DEOPT` uop is reached,
		and execution returns to teh tier 1 interpreter.

Expand All		@@ -72,3 +76,49 @@ inserted into a list of all executors which is stored in the interpreter
		state's `executor_list_head` field. This list is used when it is necessary
		to invalidate executors because values that their construction depended
		on may have changed.

		## The JIT

		When the jit is enabled (python was configured with
		[`--enable-experimental-jit`](https://docs.python.org/dev/using/configure.html#cmdoption-enable-experimental-jit),
		the uop executor's `jit_code` field is populated with a pointer to a compiled
		C function that implement the executor logic. This function's signature is
		defined by `jit_func` in [`pycore_jit.h`](Include/internal/pycore_jit.h).
		When the executor is invoked by `ENTER_EXECUTOR`, instead of jumping to
		the uop interpreter at `tier2_dispatch`, the executor runs the function
		that `jit_code` points to. This function returns the instruction pointer
		of the next Tier 1 instruction that needs to execute.

		The generation of the jitted fuctions uses the copy-and-patch technique
		which is described in
		[Haoran Xu's article](https://sillycross.github.io/2023/05/12/2023-05-12/).
		At its core are statically generated `stencils` for the implementation
		of the micro ops, which are completed with runtime information while
		the jitted code is constructed for an executor by
		[`_PyJIT_Compile`](../Python/jit.c).

		The stencils are generated under the build target `regen-jit` by the scripts
		in [`/Tools/jit`](/Tools/jit). This script reads
		[`Python/executor_cases.c.h`](../Python/executor_cases.c.h) (which is
		generated from [`Python/bytecodes.c`](../Python/bytecodes.c)). For
		each opcode, it constructs a `.c` file that contains a function for
		implementing this opcode, with some runtime information injected.
		This is done by replacing `CASE` by the bytecode definition in the
		template file [`Tools/jit/template.c`](../Tools/jit/template.c).

		Each of the `.c` file is compiled by LLVM, to produce an object file
		that contains a function that executes the opcode. These compiled
		functions are used to generate the file
		[`jit_stencils.h`](../jit_stencils.h), which contains the functions
		that the JIT can use to emit code for each of the bytecodes.

		For Python maintainers this means that changes to the bytecodes and
		their implementations do not require changes related to the JIT,
		because everything the JIT needs is automatically generated from
		[`Python/bytecodes.c`](../Python/bytecodes.c) at build time.

		See Also:

		* [Copy-and-Patch Compilation: A fast compilation algorithm for high-level languages and bytecode](https://arxiv.org/abs/2011.13127)

		* [PyCon 2024: Building a JIT compiler for CPython](https://www.youtube.com/watch?v=kMO3Ju0QCDo)