//python:pip.bzl

Rules for pip integration.

This contains a set of rules that are used to support inclusion of third-partydependencies via fully lockedrequirements.txt files. Some of the exportedsymbols should not be used and they are either undocumented here or marked asfor internal use only.

If you are using a bazel version 7 or above withbzlmod, you should only careabout thecompile_pip_requirements macro exposed in this file. Therest of the symbols are for legacyWORKSPACE setups.

compile_pip_requirements(name,srcs=None,src=None,extra_args=[],extra_deps=[],generate_hashes=True,py_binary='<functionpy_binaryfrom//python:py_binary.bzl>',py_test='<functionpy_testfrom//python:py_test.bzl>',requirements_in=None,requirements_txt=None,requirements_darwin=None,requirements_linux=None,requirements_windows=None,visibility=['//visibility:private'],tags=None,constraints=[],**kwargs)

Generates targets for managing pip dependencies with pip-compile (piptools).

By default this rules generates a filegroup named “[name]” which can be included in the dataof some other compile_pip_requirements rule that references these requirements(e.g. with-r../other/requirements.txt).It also generates two targets for running pip-compile:

validate withbazeltest[name].test
update withbazelrun[name].update

If you are using a version control system, the requirements.txt generated by this rule shouldbe checked into it to ensure that all developers/users have the same dependency versions.

Args:

name – base name for generated targets, typically “requirements”.
srcs –(defaultNone)
a list of files containing inputs to dependency resolution. If not specified,defaults to["pyproject.toml"]. Supported formats are:
- a requirements text file, usually namedrequirements.in
- A.toml file, where theproject.dependencies list is used as perPEP621.
src –(defaultNone)
file containing inputs to dependency resolution. If not specified,defaults topyproject.toml. Supported formats are:
- a requirements text file, usually namedrequirements.in
- A.toml file, where theproject.dependencies list is used as perPEP621.
extra_args –(default[])
passed to pip-compile (akapiptools). See thepip-compile docsfor args and meaning (passing-h and/or--version can helpinform what args are available)
extra_deps –(default[])
extra dependencies passed to pip-compile.
generate_hashes –(defaultTrue)
whether to put hashes in the requirements_txt file.
py_binary –(default‘<function py_binary from //python:py_binary.bzl>’)
the py_binary rule to be used.
py_test –(default‘<function py_test from //python:py_test.bzl>’)
the py_test rule to be used.
requirements_in –(defaultNone)
file expressing desired dependencies. Deprecated, use src or srcs instead.
requirements_txt –(defaultNone)
result of “compiling” the requirements.in file.
requirements_darwin –(defaultNone)
File of darwin specific resolve output to check validate if requirement.in has changes.
requirements_linux –(defaultNone)
File of linux specific resolve output to check validate if requirement.in has changes.
requirements_windows –(defaultNone)
File of windows specific resolve output to check validate if requirement.in has changes.
visibility –(default[“//visibility:private”])
passed to both the _test and .update rules.
tags –(defaultNone)
tagging attribute common to all build rules, passed to both the _test and .update rules.
constraints –(default[])
a list of files containing constraints to pass to pip-compile with--constraint.
kwargs – other bazel attributes passed to the “_test” rule.

multi_pip_parse(name,default_version,python_versions,python_interpreter_target,requirements_lock,minor_mapping,**kwargs)

NOT INTENDED FOR DIRECT USE!

This is intended to be used by the multi_pip_parse implementation in the template of themulti_toolchain_aliases repository rule.

Args:

name – the name of the multi_pip_parse repository.
default_version –(str)
the default Python version.
python_versions –(list[str])
all Python toolchain versions currently registered.
python_interpreter_target –(dict[str,Label])
a dictionary which keys are Python versions and values are resolved host interpreters.
requirements_lock –(dict[str,Label])
a dictionary which keys are Python versions and values are locked requirements files.
minor_mapping –(dict[str,str])
mapping betweenX.Y toX.Y.Z format.
kwargs – extra arguments passed to all wrapped pip_parse.

Returns:

The internal implementation of multi_pip_parse repository rule.

package_annotation(additive_build_content=None,copy_files={},copy_executables={},data=[],data_exclude_glob=[],srcs_exclude_glob=[])

Annotations to apply to the BUILD file content from package generated from apip_repository rule.

Args:

additive_build_content –(defaultNone)
Raw text to add to the generatedBUILD file of a package.
copy_files –(default{})
A mapping ofsrc andout files for@bazel_skylib//rules:copy_file.bzl
copy_executables –(default{})
A mapping ofsrc andout files for@bazel_skylib//rules:copy_file.bzl. Targets generated here will also be flagged asexecutable.
data –(default[])
A list of labels to add asdata dependencies to the generatedpy_library target.
data_exclude_glob –(default[])
A list of exclude glob patterns to add asdata to the generatedpy_library target.
srcs_exclude_glob –(default[])
A list of labels to add assrcs to the generatedpy_library target.

Returns:

str: A json encoded string of the provided content.

reporulepip_parse(name,repo_mapping,add_libdir_to_library_search_path=False,annotations={},download_only=False,enable_implicit_namespace_pkgs=False,environment={},envsubst=[],experimental_requirement_cycles={},experimental_target_platforms=[],extra_hub_aliases={},extra_pip_args=[],isolated=True,pip_data_exclude=[],python_interpreter='',python_interpreter_target=None,quiet=True,requirements_by_platform={},requirements_darwin=None,requirements_linux=None,requirements_lock=None,requirements_windows=None,timeout=600,use_hub_alias_dependencies=False)

Accepts a locked/compiled requirements file and installs the dependencies listed within.

Those dependencies become available in a generatedrequirements.bzl file.You can instead check thisrequirements.bzl file into your repo, see the “vendoring” section below.

For advanced use-cases, such as handling multi-platform dependencies, see theHow-to: Multi-Platform PyPI Dependencies guide.

In your WORKSPACE file:

load("@rules_python//python:pip.bzl","pip_parse")pip_parse(name="pypi",requirements_lock=":requirements.txt",)load("@pypi//:requirements.bzl","install_deps")install_deps()

You can then reference installed dependencies from aBUILD file with the alias targets generated in the same repo, for example, forPyYAML we would have the following:

@pypi//pyyaml and@pypi//pyyaml:pkg both point to thepy_librarycreated after extracting thePyYAML package.
@pypi//pyyaml:data points to the extra data included in the package.
@pypi//pyyaml:dist_info points to thedist-info files in the package.
@pypi//pyyaml:whl points to the wheel file that was extracted.

py_library(name="bar",...deps=["//my/other:dep","@pypi//numpy","@pypi//requests",],)

load("@pypi//:requirements.bzl","requirement")py_library(name="bar",...deps=["//my/other:dep",requirement("numpy"),requirement("requests"),],)

In addition to therequirement macro, which is used to access the generatedpy_librarytarget generated from a package’s wheel, The generatedrequirements.bzl file containsfunctionality for exposingentry points aspy_binary targets as well.

load("@pypi//:requirements.bzl","entry_point")alias(name="pip-compile",actual=entry_point(pkg="pip-tools",script="pip-compile",),)

Note that for packages whose name and script are the same, only the name of the packageis needed when calling theentry_point macro.

load("@pip//:requirements.bzl","entry_point")alias(name="flake8",actual=entry_point("flake8"),)

Vendoring the requirements.bzl file

In some cases you may not want to generate the requirements.bzl file as a repository rulewhile Bazel is fetching dependencies. For example, if you produce a reusable Bazel modulesuch as a ruleset, you may want to include the requirements.bzl file rather than make your usersinstall the WORKSPACE setup to generate it.See https://github.com/bazel-contrib/rules_python/issues/608

This is the same workflow as Gazelle, which createsgo_repository rules withupdate-repos

To do this, use the “write to source file” pattern documented inhttps://blog.aspect.dev/bazel-can-write-to-the-source-folderto put a copy of the generated requirements.bzl into your project.Then load the requirements.bzl file directly rather than from the generated repository.See the example in rules_python/examples/pip_parse_vendored.

Attributes:

name –(Name)
A unique name for this repository.
mandatory
repo_mapping –(dict[str,str])
InWORKSPACE context only: a dictionary from local repository name to global repository name. This allows controls over workspace dependency resolution for dependencies of this repository.
For example, an entry"@foo":"@bar" declares that, for any time this repository depends on@foo (such as a dependency on@foo//some:target, it should actually resolve that dependency within globally-declared@bar (@bar//some:target).
This attribute isnot supported inMODULE.bazel context (when invoking a repository rule inside a module extension’s implementation function).
optional
add_libdir_to_library_search_path –(bool)(defaultFalse)
If true, add the lib dir of the bundled interpreter to the library search path viaLDFLAGS.
Added in version 1.3.0.
optional
annotations –(dict[str,str])(default{})
Optional annotations to apply to packages. Keys should be package names, withcapitalization matching the input requirements file, and values should begenerated using thepackage_name macro. For example usage, seethis WORKSPACEfile.
optional
download_only –(bool)(defaultFalse)
Whether to use “pip download” instead of “pip wheel”. Disables building wheels from source, but allows use of–platform, –python-version, –implementation, and –abi in –extra_pip_args to download wheels for a differentplatform from the host platform.
optional
enable_implicit_namespace_pkgs –(bool)(defaultFalse)
If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binaryand py_test targets must specify eitherlegacy_create_init=False or the global Bazel option--incompatible_default_to_explicit_init_py to prevent__init__.py being automatically generated in every directory.
This option is required to support some packages which cannot handle the conversion to pkg-util style.
optional
environment –(dict[str,str])(default{})
Environment variables to set in the pip subprocess.Can be used to set common variables such ashttp_proxy,https_proxy andno_proxyNote that pip is run with “–isolated” on the CLI soPIP_<VAR>_<NAME>style env vars are ignored, but env vars that control requests and urllib3can be passed. If you needPIP_<VAR>_<NAME>, take a look atextra_pip_argsandenvsubst.
optional
envsubst –(list[str])(default[])
A list of environment variables to substitute (e.g.["PIP_INDEX_URL","PIP_RETRIES"]). The corresponding variables are expanded inextra_pip_argsusing the syntax$VARNAME or${VARNAME} (expanding to empty string if unset)or${VARNAME:-default} (expanding to default if the variable is unset or emptyin the environment). Note: On Bazel 6 and Bazel 7.0 changes to the variables namedhere do not cause packages to be re-fetched. Don’t fetch different things basedon the value of these variables.
optional
experimental_requirement_cycles –(dict[str,list[str]])(default{})
A mapping of dependency cycle names to a list of requirements which form that cycle.
Requirements which form cycles will be installed together and taken asdependencies together in order to ensure that the cycle is always satisified.
Example:sphinx depends onsphinxcontrib-serializinghtmlWhen listing both as requirements, ala
```
py_binary(name="doctool",...deps=["@pypi//sphinx:pkg","@pypi//sphinxcontrib_serializinghtml",])
```
Will produce a Bazel error such as
```
ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph:    //:doctool (...)    @pypi//sphinxcontrib_serializinghtml:pkg (...).-> @pypi_sphinxcontrib_serializinghtml//:pkg (...)|   @pypi_sphinxcontrib_serializinghtml//:_pkg (...)|   @pypi_sphinx//:pkg (...)|   @pypi_sphinx//:_pkg (...)`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...)
```
Which we can resolve by configuring these two requirements to be installed together as a cycle
```
pip_parse(...experimental_requirement_cycles={"sphinx":["sphinx","sphinxcontrib-serializinghtml",]},)
```
Warning:If a dependency participates in multiple cycles, all of those cycles must becollapsed down to one. For instancea<->b anda<->c cannot be listedas two separate cycles.
optional
experimental_target_platforms –(list[str])(default[])
NOTE: This will be removed in the next major version, so please consider migratingtobzlmod and rely onpip.parse.requirements_by_platform for this feature.
A list of platforms that we will generate the conditional dependency graph forcross platform wheels by parsing the wheel metadata. This will generate thecorrect dependencies for packages likesphinx orpylint, which includecolorama when installed and used on Windows platforms.
An empty list means falling back to the legacy behaviour where the hostplatform is the target platform.
WARNING: It may not work as expected in cases where the python interpreterimplementation that is being used at runtime is different between different platforms.This has been tested for CPython only.
For specific target platforms use values of the form<os>_<arch> where<os>is one oflinux,osx,windows and arch is one ofx86_64,x86_32,aarch64,s390x andppc64le.
You can also target a specific Python version by usingcp3<minor_version>_<os>_<arch>.If multiple python versions are specified as target platforms, then select statementsof thelib andwhl targets will include usage of version aware toolchain configsettings like@rules_python//python/config_settings:is_python_3.y.
Special values:host (for generating deps for the host platform only) and<prefix>_* values. For example,cp39_*,linux_*,cp39_linux_*.
NOTE: this is not for cross-compiling Python wheels but rather for parsing thewhl METADATA correctly.
optional
extra_hub_aliases –(dict[str,list[str]])(default{})
Extra aliases to make for specific wheels in the hub repo. This is useful whenpaired with thewhl_modifications.
Added in version 0.38.0:Forpip.parse with bzlmod
Added in version 1.0.0:Forpip_parse with workspace.
optional
extra_pip_args –(list[str])(default[])
Extra arguments to pass on to pip. Must not contain spaces.
Supports environment variables using the syntax$VARNAME or${VARNAME} (expanding to empty string if unset) or${VARNAME:-default} (expanding to default if the variable is unsetor empty in the environment), if"VARNAME" is listed in theenvsubst attribute. See alsoenvsubst.
optional
isolated –(bool)(defaultTrue)
Whether or not to pass the–isolated flag tothe underlying pip command. Alternatively, theRULES_PYTHON_PIP_ISOLATED environment variable can be usedto control this flag.
optional
pip_data_exclude –(list[str])(default[])
Additional data exclusion parameters to add to the pip packages BUILD file.
optional
python_interpreter –(str)(default“”)
The python interpreter to use. This can either be an absolute path or the nameof a binary found on the host’sPATH environment variable. If no value is setpython3 is defaulted for Unix systems andpython.exe for Windows.
optional
python_interpreter_target –(label)(defaultNone)
If you are using a custom python interpreter built by another repository rule,use this attribute to specify its BUILD target. This allows pip_repository to invokepip using the same interpreter as your toolchain. If set, takes precedence overpython_interpreter. An example value: “@python3_x86_64-unknown-linux-gnu//:python”.
optional
quiet –(bool)(defaultTrue)
If True, suppress printing stdout and stderr output to the terminal.
If you would like to get more diagnostic output, setRULES_PYTHON_REPO_DEBUG=1orRULES_PYTHON_REPO_DEBUG_VERBOSITY=INFO|DEBUG|TRACE
optional
requirements_by_platform –(dict[label,str])(default{})
The requirements files and the comma delimited list of target platforms as values.
The keys are the requirement files and the values are comma-separated platformidentifiers. For now we only support<os>_<cpu> values that are present in@platforms//os and@platforms//cpu packages respectively.
optional
requirements_darwin –(label)(defaultNone)
Override the requirements_lock attribute when the host platform is Mac OS
optional
requirements_linux –(label)(defaultNone)
Override the requirements_lock attribute when the host platform is Linux
optional
requirements_lock –(label)(defaultNone)
A fully resolved ‘requirements.txt’ pip requirement file containing thetransitive set of your dependencies. If this file is passed instead of‘requirements’ no resolve will take place and pip_repository will createindividual repositories for each of your dependencies so that wheels arefetched/built only for the targets specified by ‘build/run/test’. Note that ifyour lockfile is platform-dependent, you can use therequirements_[platform]attributes.
Note, that in general requirements files are compiled for a specific platform,but sometimes they can work for multiple platforms.rules_python right nowsupports requirements files that are created for a particular platform withoutplatform markers.
optional
requirements_windows –(label)(defaultNone)
Override the requirements_lock attribute when the host platform is Windows
optional
timeout –(int)(default600)
Timeout (in seconds) on the rule’s execution duration.
optional
use_hub_alias_dependencies –(bool)(defaultFalse)
Controls if the hub alias dependencies are used. If set to true, then thegroup_library will be included in the hub repo.
True will become default in a subsequent release.
optional

Envvars:

RULES_PYTHON_PIP_ISOLATED, RULES_PYTHON_REPO_DEBUG

pip_utils.normalize_name(name)

normalize a PyPI package name and return a valid bazel label.

Args:

name – str, the PyPI package name.

Returns:

a normalized name as a string.

rulewhl_filegroup(name,whl,pattern='',runfiles=False)

Extract files matching a regular expression from a wheel file.

An empty pattern will match all files.

Example usage:

load("@rules_cc//cc:cc_library.bzl","cc_library")load("@rules_python//python:pip.bzl","whl_filegroup")whl_filegroup(name="numpy_includes",pattern="numpy/core/include/numpy",whl="@pypi//numpy:whl",)cc_library(name="numpy_headers",hdrs=[":numpy_includes"],includes=["numpy_includes/numpy/core/include"],deps=["@rules_python//python/cc:current_py_cc_headers"],)

Movatterモバイル変換

//python:pip.bzl

Vendoring the requirements.bzl file