Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork1.7k
PEP 796: Relative Virtual Environments#4476
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Changes from1 commit
ceeaa3bd0a928941d2ea865a415c4175b20bbc85b7a0c25298e6e21e37202cab81c50f55a5e28e09b00dc7a6a9bbc1ffbd2301d2ae14b01f175f37831cb756cd46f4db532f5c99434fc862142a1c8fb2dc5428903240227557cb710579623c9388b3b959f245b259cc560dadbf41ed515cbc9f50af0eea68dd5e0b50a80caf8f880ae5a67fFile filter
Filter by extension
Conversations
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
- Loading branch information
Uh oh!
There was an error while loading.Please reload this page.
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -24,55 +24,24 @@ become more portable. | ||
| Motivation | ||
| ========== | ||
| The ``home`` field in :file:`pyvenv.cfg` is used on interpreter startup to | ||
| determine the actual Python interpreter installation that is used to execute | ||
| code in that virtual environment. Currently, this path is required to be | ||
| absolute for correct virtual environment operation because the original | ||
| `PEP 405 <https://peps.python.org/pep-0405/>`__ | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page.
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| specifying virtual environments didn't cover any specific way of processing | ||
| relative paths, their behaviour is implementation dependent. CPython releases | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| up to and including CPython 3.14 resolve them relative to the current process | ||
| working directory, making them too unreliable to use in practice. | ||
| The reason to support a relative path is to support portable virtual | ||
| environments, which rely on using a host-agnostic relative path to point to | ||
| ``PYTHONHOME``. | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| A portable virtual environment is one that can be moved between | ||
| platform-compatible hosts, which is an important feature for some projects (see | ||
| "Why portable environments matter"). | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| The reason support for a relative ``home`` path needs to be | ||
| in the interpreter itself is because locating ``PYTHONHOME`` happens | ||
| very early in the interpreter startup process, which limits the options for | ||
| customizing how it's computed. Without the ability to specify where the | ||
| @@ -84,7 +53,66 @@ machines do so either by relying on undocumented interpreter behaviour | ||
| (Bazel, omitting the ``home`` key entirely to trigger an implementation | ||
| dependent fallback to resolving via a symlinked interpreter binary on | ||
| non-Windows systems, see `gh-135773`) or by requiring a post-installation script to be executed | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page.
| ||
| after the environment is placed in its target location ( | ||
| `venvstacks <https://lmstudio.ai/blog/venvstacks#publishing-environment-layer-archives>`__ | ||
| ). | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| While this PEP on its own isn't sufficient to enable portable virtual | ||
| environments, it allows tools like Bazel or venvstacks to more easily prepare | ||
| constrained environments that allow for such use cases. | ||
| Why portable virtual environments matter | ||
| ---------------------------------------- | ||
| Portable virtual environments are important for the efficiency and | ||
| reproducibility benefits they bring from being created once and reused multiple | ||
| times later in different locations. For example, a build farm can build a | ||
| virtual environment once, cache it, and then re-use it as-is to CI jobs. | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| Rationale | ||
| ========= | ||
| Defining semantics for a relative ``home`` path is the chosen design for the | ||
| following reasons. | ||
| First, it is a small change to interpreter startup, in particular of an | ||
| unreliable behavior that isn't specified. Currently, relative paths are | ||
| resolved to the process's current working directory, which makes them | ||
| unreliable for use in practice. | ||
| Second, for portable virtual environments, relative paths allow more | ||
| efficient, simple, and correct reproduction of virtual environments between | ||
| hosts. This is because they can be copied as-is to different locations. Some | ||
| example capabilities this allows are: | ||
| * A build farm creating (and caching) a virtual environment, which is then | ||
| served to developers (e.g. Bazel). | ||
| * Composing virtual environments together (e.g. venvstacks). | ||
| * Installing multiple arbitrary virtual environments into a container to | ||
| save disk space. | ||
| * Layering a virtual environment atop a container image for faster image | ||
| building. | ||
| * Not needing virtual environment creation tools on the host that uses a | ||
| virtual environment. | ||
| * Exact reproduction of an application's virtual environment between a | ||
| developer's host and a production host. | ||
| Third, requiring an absolute path is inherently overly proscriptive. The | ||
| interpreter itself doesn't care whether paths are relative or absolute, merely | ||
| that they point to valid locations, so users should be given the ability to use | ||
| a path of their choosing. Given a known anchor point, it's easy to transform a | ||
| relative path to an absolute path and still retain predictable and reliable | ||
| behavior that produces correct values. | ||
| Fullying designing portable virtual environments | ||
| ------------------------------------------------ | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| This PEP purposely only focuses on the interpreter startup behavior to limit | ||
| its scope. There are multiple implementations and many design questions for how | ||
| to implement portable virtual environments work (e.g. what installers should | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| do), but they are separate from the Python runtime initialization phase. | ||
| Specification | ||
| ============= | ||
| @@ -162,6 +190,22 @@ relative virtual environment paths will typically be aware of the underlying | ||
| base runtime Python version, and hence able to update the emitted relative path | ||
| accordingly. | ||
| Security Implications | ||
| ===================== | ||
| A relative path in :file:`pyvenv.cfg` may resolve differently depending on the | ||
| location of the virtual environment. This *could* point to a surprising, | ||
| potentially malicious, location. | ||
| However, this risk already exists today because a relative path isn't | ||
| _rejected_, but resolved relative to the current working directory. This PEP | ||
ncoghlan marked this conversation as resolved. OutdatedShow resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| just changes the anchor point to ``pyvenv.cfg`` itself. | ||
| Similarly, the same concern exists for absolute paths. The two are | ||
| fundamentally the same because they both rely on trusting whoever created | ||
| the ``pyvenv.cfg`` file, which requires having run another tool or downloaded | ||
| something from elsewhere. | ||
| How to Teach This | ||
| ================= | ||
| @@ -193,6 +237,11 @@ them. These questions are best addressed separately by tool owners. | ||
| References | ||
| ========== | ||
| portable virtual environment | ||
| A portable virtual environment is one that can be copied from | ||
| one host to another that is platform compatible (e.g. same OS, CPU | ||
| architecture, etc), with little or no modification or post processing. | ||
| * `rules_python <https://github.com/bazel-contrib/rules_python>`__: implements | ||
| host-relocatable virtual environments. | ||
| * `rules_py <https://github.com/aspect-build/rules_py>`__: implements | ||
| @@ -265,6 +314,7 @@ Code generally assumes that any virtual environment will be | ||
| automatically detected and activated by the presence of ``pyvenv.cfg``, so | ||
| things work better when alterations to the environment aren't a concern. | ||
| Copyright | ||
| ========= | ||
Uh oh!
There was an error while loading.Please reload this page.