extensions:pip.bzl

This is the successor topip_parse for including third party PyPI dependencies into your bazel module usingbzlmod.

See also

Theenvironment markers specification for the explanation of theterms used in this extension.

Added in version 1.6.0.

Attributes:

config_settings –(list[label])
The list of labels toconfig_setting targets that need to be matched for the platform to beselected.
mandatory
arch_name –(str)(default“”)
The CPU architecture name to be used.You can use any cpu name from the@platforms//cpu: package.
Note
Either this orenvplatform_machine key should be specified.
optional
auth_patterns –(dict[str,str])(default{})
An optional dict mapping host names to custom authorization patterns.
If a URL’s host name is present in this dict the value will be used as a pattern whengenerating the authorization header for the http request. This enables the use of customauthorization schemes used in a lot of common cloud storage providers.
The pattern currently supports 2 tokens:<login> and<password>, which are replaced with their equivalent valuein the netrc file for the same host name. After formatting, the result is setas the value for theAuthorization field of the HTTP request.
Example attribute and netrc for a http download to an oauth2 enabled API using a bearer token:
```
auth_patterns = {    "storage.cloudprovider.com": "Bearer <password>"}
```
netrc:
```
machine storage.cloudprovider.com        password RANDOM-TOKEN
```
The final HTTP request would have the following header:
```
Authorization: Bearer RANDOM-TOKEN
```
optional
env –(dict[str,str])(default{})
The values to use for environment markers when evaluating an expression.
The keys and values should be compatible with thePyPA dependency specifiersspecification.
Missing values will be set to the specification’s defaults or computed usingavailable toolchain information.
Supported keys:
- implementation_name, defaults tocpython.
- os_name, defaults to a value inferred from theos_name.
- platform_machine, defaults to a value inferred from thearch_name.
- platform_release, defaults to an empty value.
- platform_system, defaults to a value inferred from theos_name.
- platform_version, defaults to0.
- sys_platform, defaults to a value inferred from theos_name.
Note
This is only used if theRULES_PYTHON_ENABLE_PIPSTAR is enabled.
optional
marker –(str)(default“”)
An environment marker expression that is used to enable/disable platforms for specific pythonversions, operating systems or CPU architectures.
If specified, the expression is evaluated during thebzlmod extension evaluation phase and if itevaluates toTrue, then the platform will be used to construct the hub repositories, otherwise, itwill be skipped.
This is especially useful for setting up freethreaded platform variants only for particular Pythonversions for which the interpreter builds are available. However, this could be also used for otherthings, such as setting up platforms for differentlibc variants.
optional
netrc –(str)(default“”)
Location of the .netrc file to use for authentication
optional
os_name –(str)(default“”)
The OS name to be used.You can use any OS name from the@platforms//os: package.
Note
Either this or the appropriateenv keys should be specified.
optional
platform –(str)(default“”)
A platform identifier which will be used as the unique identifier within the extension evaluation.If you are defining custom platforms in your project and don’t want things to clash, use extensionisolation feature.
optional
whl_abi_tags –(list[str])(default[])
A list of ABIs to select wheels for. The values can be either strings or include templateparameters like{major} and{minor} which will be replaced with python version parts. e.g.cp{major}{minor} will result incp313 given the full python version is3.13.5.Will always include"none" even if it is not specified.
Note
We select a single wheel and the last match will take precedence.
See also
See officialdocs for more information.
optional
whl_platform_tags –(list[str])(default[])
A list ofplatform_tag matchers so that we can select the best wheel based on the userpreference.Will always include"any" even if it is not specified.
The items in this list can contain a single* character that is equivalent to matching thelowest available version component in the platform_tag. If the wheel platform tag does nothave a version component, e.g.linux_x86_64 orwin_amd64, then* will act as a regular character.
Note
Normally, the* in the matcher means that we will target the lowest platform version that we canand will give preference to whls built targeting the older versions of the platform. If youspecify the version, then we will use the MVS (Minimal Version Selection) algorithm to select thecompatible wheel. As such, you need to keep in mind how to configure the target platforms toselect a particular wheel of your preference.
We select a single wheel and the last match will take precedence, if the platform_tag that wematch has a version component (e.g.android_x_arch, then the versionx will be used in theMVS matching algorithm).
Common patterns:
- To select any versioned wheel for an<os>,<arch>, use<os>_*_<arch>, e.g.manylinux_2_17_x86_64.
- To exclude versions up toX.Y -submit a PR supporting this feature.
- To exclude versions aboveX.Y, provide the full platform tag specifier, e.g.musllinux_1_2_x86_64, which will ensure that no wheels withmusllinux_1_3_x86_64 or higherare selected.
See also
See officialdocs for more information.
Changed in version 1.6.3:The matching of versioned platforms have been switched to MVS (Minimal Version Selection)algorithm for easier evaluation logic and fewer surprises. The legacy platform tags aresupported from this version without extra handling from the user.
optional

pip.override(file,patches,patch_strip=0)

Apply any overrides (e.g. patches) to a given Python distribution defined byother tags in this extension.

Attributes:

file –(str)
The Python distribution file name which needs to be patched. This will beapplied to all repositories that setup this distribution via the pip.parse tagclass.
mandatory
patches –(list[label])
A list of patches to apply to the repositoryafter ‘whl_library’ is extractedand BUILD.bazel file is generated.
mandatory
patch_strip –(int)(default0)
The number of leading path segments to be stripped from the file name in thepatches.
optional

pip.parse(hub_name,python_version,add_libdir_to_library_search_path=False,auth_patterns={},download_only=False,enable_implicit_namespace_pkgs=False,environment={},envsubst=[],experimental_extra_index_urls=[],experimental_index_url='',experimental_index_url_overrides={},experimental_requirement_cycles={},experimental_target_platforms=[],extra_hub_aliases={},extra_pip_args=[],isolated=True,netrc='',parallel_download=True,pip_data_exclude=[],python_interpreter='',python_interpreter_target=None,quiet=True,requirements_by_platform={},requirements_darwin=None,requirements_linux=None,requirements_lock=None,requirements_windows=None,simpleapi_skip=[],timeout=600,use_hub_alias_dependencies=False,whl_modifications={})

This tag class is used to create a pip hub and all of the spokes that are part of that hub.This tag class reuses most of the attributes found inpip_parse.The exception is it does not use the arg ‘repo_prefix’. We set the repositoryprefix for the user and the alias arg is always True in bzlmod.

Attributes:

hub_name –(str)
The name of the repo pip dependencies will be accessible from.
This name must be unique between modules; unless your module is guaranteed toalways be the root module, it’s highly recommended to include your module namein the hub name. Repo mapping,use_repo(...,pip="my_modules_pip_deps"), canbe used for shorter local names within your module.
Within a module, the samehub_name can be specified to group different Pythonversions of pip dependencies under one repository name. This allows using aPython version-agnostic name when referring to pip dependencies; thecorrect version will be automatically selected.
Typically, a module will only have a single hub of pip dependencies, but thisis not required. Each hub is a separate resolution of pip dependencies. Thismeans if different programs need different versions of some library, separatehubs can be created, and each program can use its respective hub’s targets.Targets from different hubs should not be used together.
mandatory
python_version –(str)
The Python version the dependencies are targetting, in Major.Minor format(e.g., “3.11”) or patch level granularity (e.g. “3.11.1”).
If an interpreter isn’t explicitly provided (usingpython_interpreter orpython_interpreter_target), then the version specified here must havea correspondingpython.toolchain() configured.
mandatory
add_libdir_to_library_search_path –(bool)(defaultFalse)
If true, add the lib dir of the bundled interpreter to the library search path viaLDFLAGS.
Added in version 1.3.0.
optional
auth_patterns –(dict[str,str])(default{})
An optional dict mapping host names to custom authorization patterns.
If a URL’s host name is present in this dict the value will be used as a pattern whengenerating the authorization header for the http request. This enables the use of customauthorization schemes used in a lot of common cloud storage providers.
The pattern currently supports 2 tokens:<login> and<password>, which are replaced with their equivalent valuein the netrc file for the same host name. After formatting, the result is setas the value for theAuthorization field of the HTTP request.
Example attribute and netrc for a http download to an oauth2 enabled API using a bearer token:
```
auth_patterns = {    "storage.cloudprovider.com": "Bearer <password>"}
```
netrc:
```
machine storage.cloudprovider.com        password RANDOM-TOKEN
```
The final HTTP request would have the following header:
```
Authorization: Bearer RANDOM-TOKEN
```
optional
download_only –(bool)(defaultFalse)
Whether to use “pip download” instead of “pip wheel”. Disables building wheels from source, but allows use of–platform, –python-version, –implementation, and –abi in –extra_pip_args to download wheels for a differentplatform from the host platform.
optional
enable_implicit_namespace_pkgs –(bool)(defaultFalse)
If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binaryand py_test targets must specify eitherlegacy_create_init=False or the global Bazel option--incompatible_default_to_explicit_init_py to prevent__init__.py being automatically generated in every directory.
This option is required to support some packages which cannot handle the conversion to pkg-util style.
optional
environment –(dict[str,str])(default{})
Environment variables to set in the pip subprocess.Can be used to set common variables such ashttp_proxy,https_proxy andno_proxyNote that pip is run with “–isolated” on the CLI soPIP_<VAR>_<NAME>style env vars are ignored, but env vars that control requests and urllib3can be passed. If you needPIP_<VAR>_<NAME>, take a look atextra_pip_argsandenvsubst.
optional
envsubst –(list[str])(default[])
A list of environment variables to substitute (e.g.["PIP_INDEX_URL","PIP_RETRIES"]). The corresponding variables are expanded inextra_pip_argsusing the syntax$VARNAME or${VARNAME} (expanding to empty string if unset)or${VARNAME:-default} (expanding to default if the variable is unset or emptyin the environment). Note: On Bazel 6 and Bazel 7.0 changes to the variables namedhere do not cause packages to be re-fetched. Don’t fetch different things basedon the value of these variables.
optional
experimental_extra_index_urls –(list[str])(default[])
The extra index URLs to use for downloading wheels using bazel downloader.Each value is going to be subject toenvsubst substitutions if necessary.
The indexes must support Simple API as described here:https://packaging.python.org/en/latest/specifications/simple-repository-api/
This is equivalent to--extra-index-urlspip option.
Changed in version 1.1.0:Starting with this version we will iterate over each index specified untilwe find metadata for all references distributions.
optional
experimental_index_url –(str)(default“”)
The index URL to use for downloading wheels using bazel downloader. This value is goingto be subject toenvsubst substitutions if necessary.
The indexes must support Simple API as described here:https://packaging.python.org/en/latest/specifications/simple-repository-api/
In the future this could be defaulted tohttps://pypi.org when this feature becomesstable.
This is equivalent to--index-urlpip option.
Changed in version 0.37.0:Ifdownload_only is set, thensdist archives will be discarded andpip.parse willoperate in wheel-only mode.
Changed in version 1.4.0:Index metadata will be used to deductsha256 values for packages even if thesha256 values are not present in the requirements.txt lock file.
optional
experimental_index_url_overrides –(dict[str,str])(default{})
The index URL overrides for each package to use for downloading wheels usingbazel downloader. This value is going to be subject toenvsubst substitutionsif necessary.
The key is the package name (will be normalized before usage) and the value is theindex URL.
This design pattern has been chosen in order to be fully deterministic about whichpackages come from which source. We want to avoid issues similar to what happened inhttps://pytorch.org/blog/compromised-nightly-dependency/.
The indexes must support Simple API as described here:https://packaging.python.org/en/latest/specifications/simple-repository-api/
optional
experimental_requirement_cycles –(dict[str,list[str]])(default{})
A mapping of dependency cycle names to a list of requirements which form that cycle.
Requirements which form cycles will be installed together and taken asdependencies together in order to ensure that the cycle is always satisified.
Example:sphinx depends onsphinxcontrib-serializinghtmlWhen listing both as requirements, ala
```
py_binary(name="doctool",...deps=["@pypi//sphinx:pkg","@pypi//sphinxcontrib_serializinghtml",])
```
Will produce a Bazel error such as
```
ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph:    //:doctool (...)    @pypi//sphinxcontrib_serializinghtml:pkg (...).-> @pypi_sphinxcontrib_serializinghtml//:pkg (...)|   @pypi_sphinxcontrib_serializinghtml//:_pkg (...)|   @pypi_sphinx//:pkg (...)|   @pypi_sphinx//:_pkg (...)`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...)
```
Which we can resolve by configuring these two requirements to be installed together as a cycle
```
pip_parse(...experimental_requirement_cycles={"sphinx":["sphinx","sphinxcontrib-serializinghtml",]},)
```
Warning:If a dependency participates in multiple cycles, all of those cycles must becollapsed down to one. For instancea<->b anda<->c cannot be listedas two separate cycles.
optional
experimental_target_platforms –(list[str])(default[])
NOTE: This will be removed in the next major version, so please consider migratingtobzlmod and rely onpip.parse.requirements_by_platform for this feature.
A list of platforms that we will generate the conditional dependency graph forcross platform wheels by parsing the wheel metadata. This will generate thecorrect dependencies for packages likesphinx orpylint, which includecolorama when installed and used on Windows platforms.
An empty list means falling back to the legacy behaviour where the hostplatform is the target platform.
WARNING: It may not work as expected in cases where the python interpreterimplementation that is being used at runtime is different between different platforms.This has been tested for CPython only.
For specific target platforms use values of the form<os>_<arch> where<os>is one oflinux,osx,windows and arch is one ofx86_64,x86_32,aarch64,s390x andppc64le.
You can also target a specific Python version by usingcp3<minor_version>_<os>_<arch>.If multiple python versions are specified as target platforms, then select statementsof thelib andwhl targets will include usage of version aware toolchain configsettings like@rules_python//python/config_settings:is_python_3.y.
Special values:host (for generating deps for the host platform only) and<prefix>_* values. For example,cp39_*,linux_*,cp39_linux_*.
NOTE: this is not for cross-compiling Python wheels but rather for parsing thewhl METADATA correctly.
optional
extra_hub_aliases –(dict[str,list[str]])(default{})
Extra aliases to make for specific wheels in the hub repo. This is useful whenpaired with thewhl_modifications.
Added in version 0.38.0:Forpip.parse with bzlmod
Added in version 1.0.0:Forpip_parse with workspace.
optional
extra_pip_args –(list[str])(default[])
Extra arguments to pass on to pip. Must not contain spaces.
Supports environment variables using the syntax$VARNAME or${VARNAME} (expanding to empty string if unset) or${VARNAME:-default} (expanding to default if the variable is unsetor empty in the environment), if"VARNAME" is listed in theenvsubst attribute. See alsoenvsubst.
optional
isolated –(bool)(defaultTrue)
Whether or not to pass the–isolated flag tothe underlying pip command. Alternatively, theRULES_PYTHON_PIP_ISOLATED environment variable can be usedto control this flag.
optional
netrc –(str)(default“”)
Location of the .netrc file to use for authentication
optional
parallel_download –(bool)(defaultTrue)
The flag allows to make use of parallel downloading feature in bazel 7.1 and abovewhen the bazel downloader is used. This is by default enabled as it improves theperformance by a lot, but in case the queries to the simple API are very expensiveor when debugging authentication issues one may want to disable this feature.
NOTE, This will download (potentially duplicate) data for multiple packages ifthere is more than one index available, but in general this should be negligiblebecause the simple API calls are very cheap and the user should not notice anyextra overhead.
If we are in synchronous mode, then we will use the first result that wefind in case extra indexes are specified.
optional
pip_data_exclude –(list[str])(default[])
Additional data exclusion parameters to add to the pip packages BUILD file.
optional
python_interpreter –(str)(default“”)
The python interpreter to use. This can either be an absolute path or the nameof a binary found on the host’sPATH environment variable. If no value is setpython3 is defaulted for Unix systems andpython.exe for Windows.
optional
python_interpreter_target –(label)(defaultNone)
If you are using a custom python interpreter built by another repository rule,use this attribute to specify its BUILD target. This allows pip_repository to invokepip using the same interpreter as your toolchain. If set, takes precedence overpython_interpreter. An example value: “@python3_x86_64-unknown-linux-gnu//:python”.
optional
quiet –(bool)(defaultTrue)
If True, suppress printing stdout and stderr output to the terminal.
If you would like to get more diagnostic output, setRULES_PYTHON_REPO_DEBUG=1orRULES_PYTHON_REPO_DEBUG_VERBOSITY=INFO|DEBUG|TRACE
optional
requirements_by_platform –(dict[label,str])(default{})
The requirements files and the comma delimited list of target platforms as values.
The keys are the requirement files and the values are comma-separated platformidentifiers. For now we only support<os>_<cpu> values that are present in@platforms//os and@platforms//cpu packages respectively.
optional
requirements_darwin –(label)(defaultNone)
Override the requirements_lock attribute when the host platform is Mac OS
optional
requirements_linux –(label)(defaultNone)
Override the requirements_lock attribute when the host platform is Linux
optional
requirements_lock –(label)(defaultNone)
A fully resolved ‘requirements.txt’ pip requirement file containing thetransitive set of your dependencies. If this file is passed instead of‘requirements’ no resolve will take place and pip_repository will createindividual repositories for each of your dependencies so that wheels arefetched/built only for the targets specified by ‘build/run/test’. Note that ifyour lockfile is platform-dependent, you can use therequirements_[platform]attributes.
Note, that in general requirements files are compiled for a specific platform,but sometimes they can work for multiple platforms.rules_python right nowsupports requirements files that are created for a particular platform withoutplatform markers.
optional
requirements_windows –(label)(defaultNone)
Override the requirements_lock attribute when the host platform is Windows
optional
simpleapi_skip –(list[str])(default[])
The list of packages to skip fetching metadata for from SimpleAPI index. You shouldnormally not need this attribute, but in case you do, please report this as a bugtorules_python and use this attribute until the bug is fixed.
EXPERIMENTAL: this may be removed without notice.
Added in version 1.4.0.
optional
timeout –(int)(default600)
Timeout (in seconds) on the rule’s execution duration.
optional
use_hub_alias_dependencies –(bool)(defaultFalse)
Controls if the hub alias dependencies are used. If set to true, then thegroup_library will be included in the hub repo.
True will become default in a subsequent release.
optional
whl_modifications –(dict[label,str])(default{})
A dict of labels to wheel names that is typically generated by the whl_modifications.The labels are JSON config files describing the modifications.
optional

pip.whl_mods(hub_name,whl_name,additive_build_content='',additive_build_content_file=None,copy_executables={},copy_files={},data=[],data_exclude_glob=[],srcs_exclude_glob=[])

This tag class is used to create JSON file that are used when calling wheel_builder.py. TheseJSON files contain instructions on how to modify a wheel’s project. Each of the attributescreate different modifications based on the type of attribute. Previously to bzlmod theseJSON files where referred to as annotations, and were renamed to whl_modifications in thisextension.

Attributes:

hub_name –(str)
Name of the whl modification, hub we use this name to set the modifications forpip.parse. If you have different pip hubs you can use a different name,otherwise it is best practice to just use one.
You cannot have the samehub_name in different modules. You can reuse the samename in the same module for different wheels that you put in the same hub, but youcannot have a child module that uses the samehub_name.
mandatory
whl_name –(str)
The whl name that the modifications are used for.
mandatory
additive_build_content –(str)(default“”)
(str, optional): Raw text to add to the generatedBUILD file of a package.
optional
additive_build_content_file –(label)(defaultNone)
(label, optional): path to a BUILD file to add to the generatedBUILD file of a package. You cannot use both additive_build_content and additive_build_content_filearguments at the same time.
optional
copy_executables –(dict[str,str])(default{})
(dict, optional): A mapping ofsrc andout files for[@bazel_skylib//rules:copy_file.bzl][cf]. Targets generated here will also be flagged asexecutable.
optional
copy_files –(dict[str,str])(default{})
(dict, optional): A mapping ofsrc andout files for[@bazel_skylib//rules:copy_file.bzl][cf]
optional
data –(list[str])(default[])
(list, optional): A list of labels to add asdata dependencies tothe generatedpy_library target.
optional
data_exclude_glob –(list[str])(default[])
(list, optional): A list of exclude glob patterns to add asdata tothe generatedpy_library target.
optional
srcs_exclude_glob –(list[str])(default[])
(list, optional): A list of labels to add assrcs to the generatedpy_library target.
optional

Movatterモバイル変換

//python/extensions:pip.bzl