Movatterモバイル変換


[0]ホーム

URL:


ContentsMenuExpandLight modeDark modeAuto light/dark, in light modeAuto light/dark, in dark modeSkip to content
Python Developer's Guide
Logo
Python Developer's Guide
Back to top

Important

This is part of aDraft of the Python Contributor’s Guide.Text in square brackets are notes about content to fill in.Currently, the devguide and this new Contributor’s Guide co-exist in therepo. We are using Sphinx include directives to demonstrate the re-organization.The final Contributor’s Guide will replace the devguide with content in only oneplace.We welcome help with this!

The[Plan for the Contributor’s Guide] page has more details about the current state of this draftandhow you can help. See more info about the Contributor Guide in thediscussion forum:Refactoring the DevGuide.

Compile and build

Note

[This is theCompile and build section from the devguide. I think this pageis too long and could be split by build target, but we can leave that for alater time.]

CPython provides several compilation flags which help with debugging variousthings. While all of the known flags can be found in theMisc/SpecialBuilds.txt file, the most critical one is thePy_DEBUG flagwhich creates what is known as a “pydebug” build. This flag turns on variousextra sanity checks which help catch common issues. The use of the flag is socommon that turning on the flag is a basic compile option.

You should always develop under a pydebug build of CPython (the only instance ofwhen you shouldn’t is if you are taking performance measurements). Even whenworking only on pure Python code the pydebug build provides several usefulchecks that one should not skip.

See also

The effects of various configure and build flags are documented inthePython configure docs.

Unix

The core CPython interpreter only needs a C compiler to be built,however, some of the extension modules will need development headersfor additional libraries (such as thezlib library for compression).Depending on what you intend to work on, you might need to install theseadditional requirements so that the compiled interpreter supports thedesired features.

If you want to install these optional dependencies, consult theInstall dependencies section below.

If you don’t need to install them, the basic steps for building Pythonfor development is to configure it and then compile it.

Configuration is typically:

$ ./configure --with-pydebug

More flags are available toconfigure, but this is the minimum you shoulddo to get a pydebug build of CPython.

Note

You might need to runmakeclean before or after re-runningconfigurein a particular build directory.

Onceconfigure is done, you can then compile CPython with:

$ make -s -j $(nproc)

This will build CPython with only warnings and errors being printed tostderr. The-j argument means thatmake will concurrently runtasks, limiting the number of parallel jobs to the number of CPU cores in yourcomputer. You can adjust the number passed to the-j flag to changethe limit on parallel jobs, which can trade RAM usage versus compilation time.

At the end of the build you should see a success message, followedby a list of extension modules that haven’t been built because theirdependencies were missing:

The necessary bits to build these optional modules were not found:_gdbmTo find the necessary bits, look in configure.ac and config.log.Checked 106 modules (31 built-in, 74 shared, 0 n/a on macosx-13.4-arm64, 0 disabled, 1 missing, 0 failed on import)

If the build failed and you are using a C89 or C99-compliant compiler,please open a bug report on theissue tracker.

If you decide toInstall dependencies, you will need to re-run bothconfigure andmake.

Once CPython is done building you will then have a working buildthat can be run in-place;./python on most machines (and what is used inall examples),./python.exe wherever a case-insensitive filesystem is used(for example, on macOS by default), in order to avoid conflicts with thePythondirectory. There is normally no need to install your built copyof Python! The interpreter will realize where it is being run fromand thus use the files found in the working copy. If you are worriedyou might accidentally install your working copy build, you can add--prefix=/tmp/python to the configuration step. When running from yourworking directory, it is best to avoid using the--enable-shared flagtoconfigure; unless you are very careful, you may accidentally runwith code from an older, installed shared Python library rather than fromthe interpreter you just built.

Clang

If you are usingclang to build CPython, some flags you might want to set toquiet some standard warnings which are specifically superfluous to CPython are-Wno-unused-value-Wno-empty-body-Qunused-arguments. You can set yourCFLAGS environment variable to these flags when runningconfigure.

If you are usingclang withccache, turn off the noisyparentheses-equality warnings with the-Wno-parentheses-equality flag.These warnings are caused by clang not having enough information to detectthat extraneous parentheses in expanded macros are valid, because thepreprocessing is done separately by ccache.

If you are using LLVM 2.8, also use the-no-integrated-as flag in order tobuild thectypes module (without the flag the rest of CPython willstill build properly).

Optimization

If you are trying to improve CPython’s performance, you will probably wantto use an optimized build of CPython. It can take a lot longer to build CPythonwith optimizations enabled, and it’s usually not necessary to do so. However,it’s essential if you want accurate benchmark results for a proposed performanceoptimization.

For an optimized build of Python, useconfigure--enable-optimizations--with-lto.This sets the default make targets up to enable Profile Guided Optimization (PGO)and may be used to auto-enable Link Time Optimization (LTO) on some platforms.See--enable-optimizations and--with-ltoto learn more about these options.

$./configure--enable-optimizations--with-lto

Windows

Note

If you are using the Windows Subsystem for Linux (WSL),clone the repository from a native Windows shell programlike PowerShell or thecmd.exe command prompt,and use a build of Git targeted for Windows,for example, theGit for Windows download from the official Git website.Otherwise, Visual Studio will not be able to find all the project’s filesand will fail the build.

For a concise step by step summary of building Python on Windows,you can readVictor Stinner’s guide.

All supported versions of Python can be builtusing Microsoft Visual Studio 2017 or later.You can download and use any of the free or paid versions ofVisual Studio.

When installing it, select thePython development workloadand the optionalPython native development tools componentto obtain all of the necessary build tools.You can find Git for Windows on theIndividual components tabif you don’t already have it installed.

Note

If you want to build MSI installers, be aware that the build toolchainfor them has a dependency on the Microsoft .NET Framework Version 3.5(which may not be included on recent versions of Windows, such as Windows 10).If you are building on a recent Windows version, use the Control Panel(Programs ‣ Programs and Features ‣ Turn Windows Features on or off)and ensure that the entry.NET Framework 3.5 (includes .NET 2.0 and 3.0) is enabled.

Your first build should use the command line to ensure any external dependenciesare downloaded:

PCbuild\build.bat -c Debug

The above command line build uses the-cDebug argumentto build in theDebug configuration,which enables checks and assertions helpful for developing Python.By default, it builds in theRelease configurationand for the 64-bitx64 platform rather than 32-bitWin32;use-c and-p to control build config and platform, respectively.

After this build succeeds, you can open thePCbuild\pcbuild.sln solutionin the Visual Studio IDE to continue development, if you prefer.When building in Visual Studio,make sure to select build settings that match what you used with the script(theDebug configuration and thex64 platform)from the dropdown menus in the toolbar.

Note

If you need to change the build configuration or platform,build once with thebuild.bat script set to those options firstbefore building with them in VS to ensure all files are rebuilt properly,or you may encounter errors when loading modules that were not rebuilt.

Avoid selecting thePGInstrument andPGUpdate configurations,as these are intended for PGO builds and not for normal development.

You can run the build of Python you’ve compiled with:

PCbuild\amd64\python_d.exe

See thePCBuild readme for more details on what other software is necessaryand how to build.

WASI

WASI is a system interface standard forWebAssembly. Through a combination ofC compilers that can target WebAssembly andwasi-libc providingPOSIX-compatible shims for WASI, it’s possible for CPython to run on a WASIhost/runtime as aguest.

Note

The instructions below assume a Unix-based OS due to cross-compilation forCPython being designed for./configure /make.

To build for WASI, you will need to cross-compile CPython. This requires a Ccompiler just like building forUnix as well as:

  1. A C compiler that can target WebAssembly (for example,WASI SDK)

  2. A WASI host/runtime (for example,Wasmtime)

All of this is provided in the WASIdev container(which you can select as an alternative container when using acodespace). You can also use what’sinstalled in the container as a reference of what versions of these tools areknown to work.

Note

CPython has only been verified with the certain tools for WASI. Usingother compilers, hosts, or WASI versionsshould work, but the toolsand their versions specified in the container and build scripts aretested via abuildbot.

Building for WASI requires doing a cross-build where you have abuild Pythonto help produce a WASI build of CPython (technically it’s a “host x host”cross-build because the build Python is also the target Python while the hostbuild is the WASI build). This means you effectively build CPython twice: onceto have a version of Python for the build system to use and another that’s thebuild you ultimately care about (that is, the build Python is not meant for useby you directly, only the build system).

The easiest way to get a debug build of CPython for WASI is to use theTools/wasm/wasi.pybuild command (which should be run w/ a recent version ofPython you have installed on your machine):

python3Tools/wasm/wasibuild--quiet----config-cache--with-pydebug
python3Tools/wasm/wasi.pybuild--quiet----config-cache--with-pydebug

That single command will configure and build both the build Python and theWASI build incross-build/build andcross-build/wasm32-wasip1,respectively.

You can also do each configuration and build step separately; the command aboveis a convenience wrapper around the following commands:

$pythonTools/wasm/wasiconfigure-build-python--quiet----config-cache--with-pydebug$pythonTools/wasm/wasimake-build-python--quiet$pythonTools/wasm/wasiconfigure-host--quiet----config-cache$pythonTools/wasm/wasimake-host--quiet
$pythonTools/wasm/wasi.pyconfigure-build-python--quiet----config-cache--with-pydebug$pythonTools/wasm/wasi.pymake-build-python--quiet$pythonTools/wasm/wasi.pyconfigure-host--quiet----config-cache$pythonTools/wasm/wasi.pymake-host--quiet

Note

Theconfigure-host command infers the use of--with-pydebug from thebuild Python.

Running the separate commands afterwasibuild is useful if you, for example,only want to run themake-host step after making code changes.

Once everything is complete, there will be across-build/wasm32-wasip1/python.sh helper file which you can use to run thepython.wasm file (see the output from theconfigure-host subcommand):

cross-build/wasm32-wasip1/python.sh--version

You can also useMakefile targets and they will work as expected thanks totheHOSTRUNNER environment variable having been set to a similar value asused inpython.sh:

make-Ccross-build/wasm32-wasip1test

Note

WASI uses acapability-based security model. This means that the WASI hostdoes not give full access to your machine unless you tell it to. Thisalso means things like files can end up being mapped to a different pathinside the WASI host. So, if you try passing a file path topython.wasm/python.sh, it needs to match the pathinside theWASI host, not the path on your machine (much like using a container).

Emscripten

Emscripten is a complete open-source compiler toolchain. It compiles C/C++ codeintoWebAssembly/JavaScript executables, for use in JavaScript runtimes,including browsers and Node.js.

Note

The instructions below assume a Unix-based OS due to cross-compilation forCPython being designed for./configure /make.

To build for Emscripten, you will need to cross-compile CPython. This requires aC compiler just like building forUnix as well as:

  • The Emscripten compiler

  • Node.js

The simplest way to install the Emscripten compiler is:

# Install Emscriptengitclonehttps://github.com/emscripten-core/emsdk./emsdk/emsdkinstall4.0.12./emsdk/emsdkactivate4.0.12source./emsdk/emsdk_env.sh

Updating the Emscripten compiler version can cause breakages. For the bestcompatibility, use the appropriate Emscripten version based on the version ofCPython you’re building:

  • For building CPython 3.14, useemsdk version4.0.12.

  • For building the main branch of the CPython repository, you may wish to uselatest instead of a specific version.

It is possible (but not necessary) to enableccache for Emscripten buildsby setting theEM_COMPILER_WRAPPER environment, but this step will onlytake effect if it is doneafteremsdk_env.sh is sourced (otherwise, thesourced script removes the environment variable):

exportEM_COMPILER_WRAPPER=ccache

Building for Emscripten requires doing a cross-build where you have abuildPython to help produce an Emscripten build of CPython. This means you buildCPython twice: once to have a version of Python for the build system to use andanother that’s the build you ultimately care about (that is, the build Python isnot meant for use by you directly, only the build system).

The easiest way to get a debug build of CPython for Emscripten is to use theTools/wasm/emscriptenbuild command, which should be run with a recentversion of Python (3.13 or newer) already installed on your machine:

python3Tools/wasm/emscriptenbuild--quiet----config-cache--with-pydebug

That single command will configure and build both the build Python and theEmscripten build incross-build/build andcross-build/wasm32-emscripten/build/python/, respectively.

You can also do each configuration and build step separately; the command aboveis a convenience wrapper around the following commands:

pythonTools/wasm/emscriptenconfigure-build-python--quiet----config-cache--with-pydebugpythonTools/wasm/emscriptenmake-build-python--quietpythonTools/wasm/emscriptenmake-libffi--quietpythonTools/wasm/emscriptenmake-mpdec--quietpythonTools/wasm/emscriptenconfigure-host--quiet----config-cachepythonTools/wasm/emscriptenmake-host--quiet

Note

Theconfigure-host command infers the use of--with-pydebug from thebuild Python.

Running the separate commands afteremscriptenbuild is useful if you, forexample, only want to run themake-host step after making code changes.

Once everything is complete, there will be across-build/wasm32-emscripten/build/python/python.sh helper file which youcan use to run thepython.mjs file:

cross-build/wasm32-emscripten/build/python/python.sh--version

You can also useMakefile targets and they will work as expected thanks totheHOSTRUNNER environment variable having been set to a similar value asused inpython.sh:

make-Ccross-build/wasm32-emscripten/build/python/test

Additional instructions for running the resulting builds (through Node.js and/orthrough web browsers) are available in the CPython repository atTools/wasm/README.md.

Android

Build and test instructions for Android are maintained in the CPython repositoryatAndroid/README.md.

iOS

Compiling Python for iOS requires a macOS machine, on a recent version of macOS,running a recent version of Xcode. Apple expects developers to keep theiroperating systems and tools up-to-date; if your macOS version is more than onemajor release out of date, or your Xcode version is more than a couple of minorversions out of date, you’ll likely encounter difficulties. It is not possibleto compile for iOS using Windows or Linux as a build machine.

A complete build for Python on iOS requires compiling CPython four times: once formacOS; then once for each of the three underlying platforms used by iOS:

  • An ARM64 device (an iPhone or iPad);

  • An ARM64 simulator running on a recent macOS machine; and

  • An x86_64 simulator running on older macOS machine.

The macOS build is required because building Python involves running some Pythoncode. On a normal desktop build of Python, you can compile a Python interpreterand then use that interpreter to run Python code. However, the binaries producedfor iOS won’t run on macOS, so you need to provide an external Pythoninterpreter. From the root of a CPython code checkout, run the following:

$ ./configure --prefix=$(pwd)/cross-build/macOS$ make -j4 all$ make install

This will build and install Python for macOS into thecross-build/macOSdirectory.

The CPython build system can compile a single platform at a time. It is possibletotest a single platform at a time; however, for distribution purposes, youmust compile all three, and merge the results. See theiOS READMEfor details on this merging process.

The following instructions will build CPython for iOS with all extensionsenabled, provided you have installed the build dependencies XZ, BZip2, OpenSSLand libFFI in subfolders of thecross-build folder. Seethe iOSsection on installing build dependencies for details onhow to obtain these dependencies. These dependencies are all strictly optional,however, including libFFI ishighly recommended, as it is required by thectypes module which is used on iOS to support accessing native system APIs.

$exportPATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin"$./configure\LIBLZMA_CFLAGS="-I$(pwd)/cross-build/iphoneos.arm64/xz/include"\LIBLZMA_LIBS="-L$(pwd)/cross-build/iphoneos.arm64/xz/lib -llzma"\BZIP2_CFLAGS="-I$(pwd)/cross-build/iphoneos.arm64/bzip2/include"\BZIP2_LIBS="-L$(pwd)/cross-build/iphoneos.arm64/bzip2/lib -lbz2"\LIBFFI_CFLAGS="-I$(pwd)/cross-build/iphoneos.arm64/libffi/include"\LIBFFI_LIBS="-L$(pwd)/cross-build/iphoneos.arm64/libffi/lib -lffi"\--with-openssl="$(pwd)/cross-build/iphoneos.arm64/openssl"\--host=arm64-apple-ios12.0\--build=arm64-apple-darwin\--with-build-python=$(pwd)/cross-build/macOS/bin/python3.13\--enable-framework$make-j4all$makeinstall
$exportPATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin"$./configure\LIBLZMA_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.arm64/xz/include"\LIBLZMA_LIBS="-L$(pwd)/cross-build/iphonesimulator.arm64/xz/lib -llzma"\BZIP2_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.arm64/bzip2/include"\BZIP2_LIBS="-L$(pwd)/cross-build/iphonesimulator.arm64/bzip2/lib -lbz2"\LIBFFI_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.arm64/libffi/include"\LIBFFI_LIBS="-L$(pwd)/cross-build/iphonesimulator.arm64/libffi/lib -lffi"\--with-openssl="$(pwd)/cross-build/iphonesimulator.arm64/openssl"\--host=arm64-apple-ios12.0-simulator\--build=arm64-apple-darwin\--with-build-python=$(pwd)/cross-build/macOS/bin/python3.13\--enable-framework$make-j4all$makeinstall
$exportPATH="$(pwd)/iOS/Resources/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin"$./configure\LIBLZMA_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.x86_64/xz/include"\LIBLZMA_LIBS="-L$(pwd)/cross-build/iphonesimulator.x86_64/xz/lib -llzma"\BZIP2_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.x86_64/bzip2/include"\BZIP2_LIBS="-L$(pwd)/cross-build/iphonesimulator.x86_64/bzip2/lib -lbz2"\LIBFFI_CFLAGS="-I$(pwd)/cross-build/iphonesimulator.x86_64/libffi/include"\LIBFFI_LIBS="-L$(pwd)/cross-build/iphonesimulator.x86_64/libffi/lib -lffi"\--with-openssl="$(pwd)/cross-build/iphonesimulator.x86_64/openssl"\--host=x86_64-apple-ios12.0-simulator\--build=arm64-apple-darwin\--with-build-python=$(pwd)/cross-build/macOS/bin/python3.13\--enable-framework$make-j4all$makeinstall

These instructions modify yourPATH before the build. As iOS and macOS sharea hardware architecture (ARM64), it is easy for a macOS ARM64 binary to beaccidentally linked into your iOS build. This is especially common when Homebrewis present on the build system. The most reliable way to avoid this problem isto remove any potential source of other libraries from yourPATH.

However, thePATH is not completely bare — it includes theiOS/Resources/bin folder. This folder contains a collection of scripts thatwrap the invocation of the Xcodexcrun tool, removing user- andversion-specific paths from the values encoded in thesysconfigmodule. Copies of these scripts are included in the final build products.

Once this build completes, theiOS/Frameworks folder will contain aPython.framework that can be used for testing.

To run the test suite on iOS, complete a build for asimulator platform,ensure the path modifications from the build are still in effect, and run:

$ make testios

The full test suite takes approximately 12 minutes to run on a 2022 M1 MacBookPro, plus a couple of extra minutes to build the testbed application and bootthe simulator. There will be an initial burst of console output while the Xcodetest project is compiled; however, while the test suite is running, there is noconsole output or progress. This is a side effect of how Xcode operates whenexecuted at the command line. You should see an iOS simulator appear during thetesting process; the simulator will booth to an iOS landing screen, the testbedapp will be installed, and then started. The screen of the simulator will beblack while the test suite is running. When the test suite completes, success orfailure will be reported at the command line. In the case of failure, you willsee the full log of CPython test suite output.

You can also run the test suite in Xcode itself. This is required if you want torun on a physical device; it is also the easiest approach if you need to run asingle test, or a subset of tests. See theiOS READMEfor details.

On this page

[8]ページ先頭

©2009-2026 Movatter.jp