Dev Guide

This document covers tips and guidance for working on therules_python codebase. Its primary audience is first-time contributors.

Running tests

Running tests is particularly easy thanks to Bazel, simply run:

bazeltest//...

And it will run all the tests it can find. The first time you do this, it willprobably take a long time because various dependencies will need to be downloadedand set up. Subsequent runs will be faster, but there are many tests, and some ofthem are slow. If you’re working on a particular area of code, you can run justthe tests in those directories instead, which can speed up your edit-run cycle.

Writing Tests

Most code should have tests of some sort. This helps us have confidence thatrefactors didn’t break anything and that releases won’t have regressions.

We don’t require 100% test coverage; testing certain Bazel functionality isdifficult, and some edge cases are simply too hard to test or not worth theextra complexity. We try to judiciously decide when not having tests is a goodidea.

Tests go undertests/. They are loosely organized into directories for theparticular subsystem or functionality they are testing. If an existing directorydoesn’t seem like a good match for the functionality being tested, then it’sfine to create a new directory.

Re-usable test helpers and support code go intests/support. Tests don’t needto be perfectly factored and not every common thing a test does needs to befactored into a more generally reusable piece. Copying and pasting is fine. It’smore important for tests to balance understandability and maintainability.

Test utilities

General code to support testing is intests/support. It has a varietyof functions, constants, rules etc, to make testing easier. Below are somecommon utilities that are frequently used.

sh_py_run_test

Thesh_py_run_test <tests/support/sh_py_run_test.bzl rule is a helper tomake it easy to run a Python program with custom build settings using a shellscript to perform setup and verification. This is best to use when verifyingbehavior needs certain environment variables or directory structures tocorrectly and reliably verify behavior.

When adding a test, you may find the flag you need to set isn’t supported bythe rule. To have it support setting a new flag, see the py_reconfig_test docsbelow.

py_reconfig_test

Thepy_reconfig_test andpy_reconfig_binary rules are helpers for runningPython binaries and tests with custom build flags. This is best to use whenverifying behavior that requires specific flags to be set and when the programitself can verify the desired state.

They are located intests/support/py_reconfig.bzl

When adding a test, you may find the flag you need to set isn’t supported bythe rule. To have it support setting a new flag:

  • Add an attribute to the rule. It should have the same name as the flagit’s for. It should be a string, string_list, or label attribute – thisallows distinguishing between if the value was specified or not.

  • Modify the transition and add the flag to both the inputs and outputslist, then modify the transition’s logic to check the attribute and setthe flag value if the attribute is set.

whl_from_dir_repo

Thewhl_from_dir_repo repository rule intests/support/whl_from_dirtakes a directory tree and turns it into a.whl file. This can be used tocreate arbitrary whl files to verify functionality.

Integration tests

An integration test is one that runs a separate Bazel instance inside the test.These tests are discouraged unless absolutely necessary because they are slow,require a lot of memory and CPU, and are generally harder to debug. Integrationtests are reserved for things that simply can’t be tested otherwise, or forsimple high-level verification tests.

Integration tests live intests/integration. When possible, add to an existingintegration test.

Updating internal dependencies

  1. Modify the./python/private/pypi/requirements.txt file and run:

    bazelrun//private:whl_library_requirements.update

  2. Run the following target to updatetwine dependencies:

    bazelrun//private:requirements.update

  3. Bump the coverage dependencies using the script using:

    bazelrun//tools/private/update_deps:update_coverage_deps<VERSION># for example:# bazel run //tools/private/update_deps:update_coverage_deps 7.6.1

Updating tool dependencies

It’s suggested to routinely update the tool versions within our repo. Some of thetools are using requirement files compiled byuv, and others use other means. In orderto have everything self-documented, we have a special target,//private:requirements.update, which usesrules_multirun to run allof the requirement-updating scripts in sequence in one go. This can be done once per release aswe prepare for releases.

Creating Backport PRs

The steps to create a backport PR are:

  1. Create an issue for the patch release; use thepatch releasetemplate.

  2. Create a fork ofrules_python.

  3. Checkout therelease/X.Y branch.

  4. Usegitcherry-pick-x to cherry pick the desired fixes.

  5. Update the release’sCHANGELOG.md file:

    • Add a Major.Minor.Patch section if one doesn’t exist

    • Copy the changelog text frommain to the release’s changelog.

  6. Send a PR with the backport’s changes.

    • The title should bebackport:PR#NtoMajor.Minor

    • The body must preserve the original PR’s number, commit hash, description,and authorship.Use the following format (gitcherry-pick will use this format):

      <originalPRtitle><originalPRbody>(cherrypickedfromcommit<commithash>)-----Co-authored-by:<originalPRauthor;separatelinesforeach>

    • If the PR contains multiple backport commits, separate each’s descriptionwith-----.

  7. Send a PR to update themain branch’sCHANGELOG.md to reflect thechanges done in the patched release.