Defining a Deliverable¶

A “deliverable” is a unit of distribution of a useful project. It maybe a single library or several server components that are packagedseparately but released and used together. Rather than base thedefinition on technical terms such as packaging, we use the socialorganization of the project to identify deliverables. If the contentsof two repositories share a bug reporting tool so that bugs for thecontents of both repositories are mixed together and use the sameversion numbers for all releases (e.g., one launchpad project), theyare both part of the same deliverable.

Within this repository, each deliverable is represented by a separatefile within the release series directory or the _independentdirectory. The data that needs to go into each file is described indetail below. All automated manipulation of the deliverable is managedthrough the data file, which is reviewed by the core team when thepatch is proposed to openstack/releases.

Adding Deliverables¶

In order to be considered to be included in the release for a givenseries, the project must be documented by adding a deliverable file tothis repository before the second milestone of the series. Exceptionswill be considered by the release team on a case-by-case basis.

Requesting a Release¶

The PTL or release liaison for a project may request a release frommaster by submitting a patch to this repository, appending the necessaryrelease metadata to the file describing the deliverable to bereleased. The release team will review the request and providefeedback about the version number.

The stable maintenance team, PTL, or release liaison for a project mayrequest a release from a stable branch by submitting a patch to thisrepository, appending the necessary release metadata to the filedescribing the deliverable to be released. The release team willreview the request and provide feedback about the version number. Ifthe stable release is requested by the stable maintenance team, itshould be acknowledged by the PTL or release liaison to ensure thatthe development team is aware of the coming change.

Prepare the release request by submitting a patch to thisrepository.

• Always add the new release to the end of the list being edited. Theversion numbers will be reordered for display.

• Always pick new version numbers for new releases. We do not updatethe contents of previously tagged releases, because that confusesusers who have already downloaded those packages.

• Make sure you follow semantic versioning rulessemver when picking the version number.

  In particular, if there is a change going into this release whichrequires a higher minimum version of a dependency, then theminor version should be incremented.

  Note

  The exception to this rule is when the versions of a project arepinned between minor versions in stable branches. In those caseswe frequently release global-requirements syncs with a patchversion to fix the target branch, e.g. stable/juno, but don’tincrement the minor version to avoid it being used in a differentbranch, like stable/kilo. Someone from thestable-maint-core teamshould +1 a change like this before it’s approved.

• Do not increment version numbers artificially to maintainconsistent versions between deliverables. We expect versions ofcompatible deliverables to drift apart over time, and made thedecision to embrace this by using other tools to document for userswhich combinations of packages go together.

  http://lists.openstack.org/pipermail/openstack-dev/2015-June/065992.html

  If two build artifacts always need to have the same version number,that implies strongly that they are part of the same deliverableand should not be released separately.

• Start version numbers with 0.1.0 for unstable early editions andprototypes. Switch to 1.0.0 for the first production-readyrelease. Do not release the first version of a deliverable with anumber that matches the version used by other existing relateddeliverables. This confuses consumers about the maturity of the newdeliverable and about where they should find “older” versions withlower numbers, which do not exist.

• Set the first line (summary) of the commit message to the packagename and version being requested.

• If you are not the release liaison or PTL, have the PTL of theproject acknowledge the request with a +1.

• Do not use the “Depends-On” feature of zuul to make a releaserequest depend on merging another patch in your project. Thedependency management does not work properly in the release checkjobs, and the validator requires that the patch listed in yourdeliverable file actually be merged into a proper branch.

• Do not submit multiple dependent patches for multiplereleases. Having a patch series with multiple releases means therelease team cannot properly prioritize processing them. Duringmilestone weeks, preference is given to milestonereleases. Releases from stable branches, independent projects, andother types of releases are processed later. If your milestonerelease request depends on a request that is deprioritized, you maymiss the deadline.

• RC1 tags and stable branches should be submitted together forprojects using the cycle-with-rc release model.

Using new-release command¶

The releases repository contains several tools to make working withthe data files easier. The new-release command, for example,calculates new version numbers based on the semantic versioninginformation given on the command line and determines the SHA of theHEAD of the appropriate branch.

Use thevenv tox environment to run the tool, like this:

$ tox -e venv -- new-release SERIES DELIVERABLE TYPE

The SERIES value should be the release series, such as “pike”.

The DELIVERABLE value should be the deliverable name, such as“oslo.config” or “cinder”.

The TYPE value should be one of:

bugfix – For a release containing only bug fixes.

feature – For a release with a new feature, a new dependency, ora change to the minimum allowed version of a dependency.

major – For a release with a backwards-incompatible change.

milestone – For a date-based milestone tag.

rc – For a release candidate.

new-series automatically includes a stable branch for the firstrelease candidate.

If the most recent release of cinder during the pike series is11.0.0.0b3 then running:

$ tox -e venv -- new-release pike cinder rc

detects that this is the first release candidate and updates the filedeliverables/pike/cinder.yaml with the new release and a new stablebranch.

If a deliverable includes multiple git repositories, all of therepositories are included in the new release unless their HEAD versionmatches the most recent release from that repository. To re-tag inthose cases, use the--force option.

Use the--stable-branch option to also create a stable branch for thenew release. Projects following the cycle-with-rc releasemodel automatically receive a new stable branch on their first releasecandidate.

Requesting a Branch¶

The PTL or release liaison for a project may request a new branch bysubmitting a patch to this repository, adding the necessary branchmetadata to the file describing the deliverable to be released. Therelease team will review the request and provide feedback about thebranch point and possibly the name.

Prepare the branch request by submitting a patch to this repository.

• RC1 tags and stable branches should be submitted together forprojects using the cycle-with-rc release model.

• Always add the new branch to the end of the list in the file beingedited.

• Branches should use one of the standard prefixes:

  stable/ – for stable series

  feature/ – for temporary feature branches

• stable/ branch names must match a valid series name.

• If you are not the release liaison or PTL, have the PTL of theproject acknowledge the request with a +1.

• Do not use the “Depends-On” feature of zuul to make a branchrequest depend on merging another patch in your project. Thedependency management does not work properly in the release checkjobs, and the validator requires that the patch listed in yourdeliverable file actually be merged into a proper branch.

Release Approval¶

Releases will only be denied during freeze weeks, periods where thereare known gate issues, or when releasing will introduce unwantedinstability. Releases made late in a week may be delayed until earlyin the next week unless there is a pressing need such as a gatefailure or security issue.

Who is Responsible for the Release?¶

The release team is responsible for helping to clearly signal thenature of the changes in the release through good version numberselection.

The project team is responsible for understanding the implications forconsuming projects when a new release is made, and ensuring thatreleases do not break other projects. When breaks occur, the projectteam is responsible for taking the necessary corrective action.

Deliverable Files¶

Deliverable repositories for projects using cycle_with_intermediaryor cycle_with_milestones should be placed in their respective releaseswithin the deliverables directory. Deliverable repositories forprojects using the independent release model should be placed in thedeliverables/_independent directory.

For a deliverable set of projects, we use one YAML file per releaseseries to hold all of the metadata for all releases and branches ofthat deliverable. For each deliverable, we need to track:

• the launchpad project name (such asoslo.config) or storyboardproject id (such as760)

• the series (Kilo, Liberty, etc.)

• the release model being used

• for each repository

  • the name (such asopenstack/oslo.config)

  • the hash of the commit to be tagged

  • the version number to use

• cycle highlights that will be published toreleases.openstack.org/$SERIES/highlights.html (optional, and forcycle-with-intermediary and cycle-with-rc projects only)

• the starting points of all branches

We track this metadata for the history of all releases of thedeliverable, so we can render a set of release history documentation.

The file should be named based on the deliverable to be tagged, soreleases forliberty from theopenstack/oslo.configrepository will have a file inopenstack/releases calleddeliverables/liberty/oslo.config.yaml. Releases of the samedeliverable from thestable/kilo branch will be described bydeliverables/kilo/oslo.config.yaml.

Deliverables File Schema¶

The top level of a deliverable file is a mapping with keys:

team

The name of the team that owns the deliverable, as listed in thegovernance repository data files.

launchpad

The slug name of the launchpad project, suitable for use in URLs.(Not needed for projects using storyboard.)

storyboard

The ID of the storyboard project, suitable for use in URLs and APIcalls. (Not needed for projects using launchpad.)

release-notes

The URL or URLs to the published release notes for the deliverablefor the series.

Deliverables contained a single repository should simply include theURL to the notes for that repository. Deliverables made up ofmultiple repositories should use a hash to map each repository nameto its notes URL.

include-pypi-link

Eithertrue orfalse, indicating whether the releaseannouncement should include the link to the package onPyPI. Defaults tofalse.

release-model

Identify the release model used by the deliverable. Seethe reference section of the documentation for descriptionsof the valid models.

type

Categorize the deliverable based on what it does. See the referencesection of the documentation for descriptions of the validdeliverable types.

artifact-link-mode

Describe how to link to artifacts produced by the project. Thedefault istarball. Valid values are:

tarball

Automatically generates links to version-specific files ontarballs.openstack.org.

none

Do not link to anything, just show the version number.

repository-settings

Mapping of special settings to control the behavior for each repository,keyed by the repository name.

flags

A list of flags attached to the repository.

no-artifact-build-job

This repository has no job for building an artifact, but shouldbe tagged anyway.

retired

This repository is no longer used, but was present in oldversions of a deliverable.

pypi-name

An optional name for the deliverable on pypi.python.org. Thisvalue is only needed if the name on PyPI does not match thecanonicalized output ofpythonsetup.py--name, such as if ituses capitalized letters (“DragonFlow” instead of “dragonflow”).

tarball-base

An optional name for the base of the tarball created by therelease. If no value is provided, it defaults to the repo base name oran overridden value set on a specific release entry underreleases.

release-type

This (optional) key sets the level of validation for the versions numbers.

python-service

Default: Enforces 3 digit semver version numbers in releases and allowsfor common alpha, beta and dev releases. This should be appropriate formost OpenStack component release requirements.

python-pypi

Likepython-service but requires the jobs to publish the componentto the Python Package Index (PyPI).

xstatic

Allows a more flexible versioning in line with xstatic package guidelinesand requirements.

fuel

The Fuel project manages its own packages.

puppet

All puppet modules should use this. If no release-type isspecified and the validation job can determine that a module is apuppet module, it assumes a release-type ofpuppet.

nodejs

All nodejs modules should use this. If no release-type isspecified and the validation job can determine that a module is anodejs module, it assumes a release-type ofnodejs.

neutron

For projects using the PyPI publishing variant that installsneutron in order to build the package. Typically used by neutronplugins.

horizon

For projects using the PyPI publishing variant that installshorizon in order to build the package. Typically used by horizonplugins.

releases

A list of the releases for the deliverable.

stable-branch-type

This (optional) key sets the validation for the location associatedwith each stable branch.

std

Default: Requires stable branches to be created from taggedreleases. This is the correct branch type for most projects.

The location must be either an existing version tag or the mostrecently added version number under the releases list (allowing atag and branch to be submitted together). All repositoriesassociated with the version (as identified by the deliverablefile) will be branched from that version using the name given.

std-with-versions

This mode has the same meaning as thestd branch type, with theaddition that version-based branches can be created as well.

These version-based branches are shorter term stable branches thatare named for the major and minor version number (e.g. bugfix/3.1).This is primarily used for Ironic releases.

tagless

This mode requires stable branch locations to be a mapping betweenrepository name and an existing commit, specified by thehash. This mode should only be used for projects that do not tagreleases, such as devstack and grenade.

upstream

Stable branch names track upstream release names, rather thanOpenStack series names.

none

This mode indicates that the deliverable should never have stablebranches. This is used for specific deliverables like tempest.

cycle-highlights

A list of plain-text bullet points describing some of the top newfeatures or changes you would like to point out for this releasecycle. Minimal RST markup is supported. These highlights arecompiled per team and published toreleases.openstack.org/$SERIES/highlights.html.

branches

A list of the branches for the deliverable.

Eachrelease entry is a mapping with keys:

version

The version tag for that release, to be applied to all of the memberprojects.

projects

A list of all of the projects making up the deliverable for thatrelease.

highlights

An optional message to be included in the release note emailannouncing the release. (Use| to indicate a multi-line,pre-formatted message.)

flags

A list of flags attached to the release.

forced

This release was applied by the release team, and not the projectteam.

skipped-sig

This independent release pre-dates the Ocata cycle and did notgenerate any signature. Signature link display should be skippedwhen the release website pages are generated.

Each entry in theprojects list is a mapping with keys:

repo

The name of the repository on git.openstack.org.

hash

The SHA1 hash for the commit to receive the version tag.

tarball-base

An optional name for the base of the tarball created by therelease. If no value is provided, it defaults to therepository-settingsvalue if set, else the repo base name.

Each entry in thebranches list is a mapping with keys:

name

The name of the branch.

location

The location value depends on the name.

If a branch name starts with stable/ then the location value dependson thestable-branch-type setting.

If a branch name starts with feature/ then the location must be amapping between the target repository name and the SHA of a commitalready in the target repository.

Examples¶

For example, one version ofdeliverables/liberty/oslo.config.yaml might contain:

---launchpad:oslo.configbranches:-name:feature/random-feature-worklocation:openstack/oslo.config:02a86d2eefeda5144ea8c39657aed24b8b0c9a39releases:-version:1.12.0projects:-repo:openstack/oslo.confighash:02a86d2eefeda5144ea8c39657aed24b8b0c9a39

and then for the subsequent release it would be updated to contain:

---launchpad:oslo.configbranches:-name:feature/random-feature-worklocation:openstack/oslo.config:02a86d2eefeda5144ea8c39657aed24b8b0c9a39-name:stable/newtonlocation:1.12.1releases:-version:1.12.0projects:-repo:openstack/oslo.confighash:02a86d2eefeda5144ea8c39657aed24b8b0c9a39-version:1.12.1projects:-repo:openstack/oslo.confighash:0c9113f68285f7b55ca01f0bbb5ce6cddada5023highlights:|Thisreleaseincludesthechangetostopimportingfromthe'oslo'namespacepackage.

For deliverables with multiple repositories, the list of projectswould contain all of them. For example, the Neutron deliverable mightbe described bydeliverables/mitaka/neutron.yaml containing:

---launchpad:neutronrelease-notes:openstack/neutron:https://docs.openstack.org/releasenotes/neutron/mitaka.htmlopenstack/neutron-lbaas:https://docs.openstack.org/releasenotes/neutron-lbaas/mitaka.htmlopenstack/neutron-fwaas:https://docs.openstack.org/releasenotes/neutron-fwaas/mitaka.htmlopenstack/neutron-vpnaas:https://docs.openstack.org/releasenotes/neutron-vpnaas/mitaka.htmlreleases:-version:8.0.0projects:-repo:openstack/neutronhash:3213eb124e40b130e174ac3a91067e2b196788dd-repo:openstack/neutron-fwaashash:ab5622891e2b1a7631f97471f55ffb9b5235e5ee-repo:openstack/neutron-lbaashash:19b18f05037dae4bbbada848aae6421da18ab490-repo:openstack/neutron-vpnaashash:a1b12601a64a2359b2224fd4406c5db008484700

To allow tagging for repositories without build artifacts, set theno-artifact-build-job flag.

---launchpad:astararepository-settings:openstack/astara-appliance:flags:-no-artifact-build-jobreleases:-version:9.0.0.0b1projects:-repo:openstack/astara-appliancehash:c21a64ea7b3b0fbdab8592afecdd31d9b8e64a6a

Helpers¶

In order to help build out these files there are various command linebased tools that come with this repository. To install these it is aseasy aspipinstall. in this repository directory.

• new-release takes arguments to describe a new release andupdates the deliverable file, automatically calculating the versionnumber

• edit-deliverable takes arguments to update the contents of asingle deliverable file

• list-changes that lists the changes in a given release file.

• interactive-release that goes through awizard style set ofquestions to produce a new or updated release of a given project orset of projects.

• missing-releases scans deliverable files and verifies that allof the releases that should have been tagged by hand have been

• init-series initializes a new deliverable directory with stubfiles based on the previous release.

• get-contacts Loads the governance and liaison data to print contactdeatils for a given team

tox-eaclmanager----seriesnewtonacls~/branches/openstack-infra/project-config

tox-eaclmanager--groupspre_releasettx

./tools/add_reviewers.shussuri-rc

tox-echeck_approval--695375

./tools/process_auto_releases.sh ussuri $(list-deliverables --unreleased --series ussuri)

tox -e venv -- list-deliverables --unreleased --series ussuri > deliverables.logvi deliverables.log  # edit contents as needed./tools/process_auto_releases.sh ussuri $(cat deliverables.log)

tools/add_release_note_links.shussuri

$ tools/search_emails.py

$ tools/search_emails.py --starting-date 2020-04-01 --ending-date 2020-4-1

$ tools/search_emails.py --authors "Herve Beraud" "Sean McGinnis"

$ tools/search_emails.py --starting-date 2020-04-01 --ending-date 2020-4-1 --authors "Herve Beraud" "Sean McGinnis"

$ tools/search_emails.py --topic ".?\[release\] Release countdown.*" --starting-date 2020-5-1

$ tools/search_emails.py --topic ".?openstack/watcher.*" --mailing-list http://lists.openstack.org/pipermail/release-job-failures/ --starting-date 2016-6-1

tools/list_eol_stale_branches.sh

GERRIT_USER=<gerrit_user_name>tools/list_eol_stale_branches.sh

tools/list_eom_stale_branches.sh<series>

tools/abandon_patches_on_branch.sh yoga $(list-deliverables --series yoga)

tools/list_unbranched_projects.sh

tox-emembership_freeze_test--stein~/branches/governance/reference/projects.yaml

tox-emembership_freeze_test--stein~/branches/governance/reference/projects.yaml--url--distgithttps://github.com/

tox-evenv--propose-final-releasesnewtonocata

tox-evenv--propose-library-branchestox-evenv--propose-library-branchespike

./tools/list_unreleased_changes.shmasteropenstack/oslo.config

./tools/list_unreleased_changes_for_team.shsteinoslo

./list_stable_unreleased_changes.shstable/liberty

./list_unreleased_changes.sh stable/liberty $(list-deliverables --repos --series liberty)

tox-evenv--list-eom-series