- Notifications
You must be signed in to change notification settings - Fork963
Document the new release process for Rustup#3844
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:master
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
The Rustup release process has historically been a manual process thatinvolved copying files from S3 to the local machine and back to S3. Thisintroduced a high risk of human error. When modifications to theexisting release script became necessary, the decision was made toautomate the release process (seerust-lang#3819 for details).The documentation in the dev-guide has been updated to cover the newrelease process, which is fully automated to produce `beta` releasesusing GitHub Actions and the [promote-release] tooling.[promote-release]:https://github.com/rust-lang/promote-release
jdno commentedMay 28, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Here's a rendered version of the file:https://github.com/jdno/rust-rustup/blob/new-release-process/doc/dev-guide/src/release-process.md |
Uh oh!
There was an error while loading.Please reload this page.
doc/dev-guide/src/release-process.md Outdated
| ### Release Artifacts | ||
| The release artifacts are produced by CI when a new commit is merged into the | ||
| `stable` branch, and they are uploaded to the `dev-static` bucket on S3. There, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think we should move this to a different bucket. CI shouldn't have direct access to the static buckets IMO, it also complicates any intermediate processing (e.g. signing or recompression).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
That's fine with me. How about we create arustup-builds bucket that stores the artifacts, similar to the bucket forrust-lang/rust?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Sounds perfect.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The config for the new bucket has been created here:rust-lang/simpleinfra#428
| After merging the PR in the previous step, update the commit SHA checksum in | ||
| `rustup-init.sh` to match the latest commit on `master`. Submit the change in a | ||
| PR and merge it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This step sounds surprising to me. If someone updates this to some other commit, how will we notice? Is there an opportunity to adjust the script as part of CI publishing it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@Mark-Simulacrum This is the most painful point for me personally during the past 2 releases (v1.27.0 and v1.27.1, for which I actually handled the release process), and I have no idea why this have to be the case.
Well, when we migrated to GitHub Merge Queue in v1.27.0, we have almost broken the whole workflow (which relied on merges instead of rebases and thus assumed shasum stability; to keep the coherence with the old workflow, a 2nd PR is required with GHMQ enabled for script traceability, see#3653 (comment) for more context). The current description is the closest I could get from the "spirit" of the original.
I'd love to see this step being automated. As long as the resultingrustup-init.sh script is traceable via the shasum when it's published ontorustup.sh etc., I personally don't see a particular reason we cannot leave a placeholder onmaster for example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We can look into automating this step, but I would do that after refactoring the release process as-is. This is already quite a big change and I don't want to slow down the implementation or add additional complexity to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Yeah, for sure. Mostly just flagging it since it sounded off.
| the right version number when run `--version`. | ||
| Once the beta release has happened, post a new topic named "Seeking beta testers | ||
| for Rustup $VER_NUM" on the [Internals Forum]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
We've automated this for rustc, maybe can reuse that for rustup...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Happy to create a tracking issue for this so that we can look into it once we've refactored the tooling.
| Once the official release has happened, prepare and push a tag on the latest | ||
| `stable` commit. | ||
| - `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Should we do this in promote release? We do for rust and cargo (also always signing the commits with the official key).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@Mark-Simulacrum Will be cool. So far I have to pingt-release before bedtime, and manually do the tagging the next morning 😓
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If I understand correctly what this mean, I agree it would be cool.
Currently I have an automated process to make sure the script that I download from rustup.rs is the same as a specific tagged release from this repository. Until a few hours ago it was v1.27.1. Now it should be a commit betweene2d9e7e and master that would be tagged v1.28.0. But there is no such tag yet.
If I understand correctly, this tagging step is supposed to happen after the file is distributed. And this comment suggests to do it at the same time. Is that right?
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: rami3l <rami3l@outlook.com>
rami3l left a comment• edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
@jdno Should we remove the manual release script, as I assume it's not needed anymore?
Uh oh!
There was an error while loading.Please reload this page.
doc/dev-guide/src/release-process.md Outdated
| Before making a release, ensure that `rustup-init.sh` is behaving correctly, | ||
| This document describes the process for making a new release of Rustup. It is | ||
| split into the two sections. The first section explains the release process, | ||
| while the second section documents for maintainers how to make a release. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
"documents how to make a release"? The "for maintainers" part reads a little awkwardly.
(Also "make" feels a little vague here, can we make that more precise?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I've rewritten the into a bit, but I'm not sure how much better it is now. 😅
Uh oh!
There was an error while loading.Please reload this page.
Thanks for working on this! |
Based on a discussion[^1] with the release team, the decision has beenmade to create a new S3 bucket for Rustup build artifacts. This enableus to revoke the access that Rustup's CI has to the release bucket,improving security and reducing the risk of malicious or accidentalcompromise of Rustup releases.[^1]:rust-lang#3844 (comment)
@jdno what's your plan for driving the new release process to completion? We're starting to think about how/when we want to publish our next release so it would be good to understand how much of a dependency we have on you and what that means for the schedule. |
I've started working on it, but I can't give a good estimate on how long it'll take. My hope is to get most of the implementation work done in the next two weeks. I don't want to be a blocker, though, so my work is strictly additive right now. The current process will continue to work until we have a full replacement. So regarding dependencies or the schedule, there shouldn't be any. |
joshka commentedMar 6, 2025
I'd strongly recommend adding roll back process for when a deployment causes failure like what happened in 1.28.0. I'd like to echo and amplify the comments in#4211 about rolling back to make sure that there's more than one voice calling for this practice. The 1.28.0 release highlighted that Rustup is a load bearing piece of software. Having a failed deployment cause many downstream failures like this should have resulted in a rollback within about an hour, not a roll forward a day later. I base this opinion having participated (as an owner and as someone affected by) in many post-mortems at a large internet company. The consensus engineering best practice is to rollback first, and think about how to solve the greater problem second. It's rare that this advice should be ignored, and there doesn't seem like there's a any good rationale that's been expressed here why that should have been so here. The users that migrated to using the new rustup changes would generally have been a very small number compared to those who were affected by this, and those users were actively aware of the changes. The impacted users were generally not made aware until this caused failure in their systems. I want to explicitly state this comment is not intended to throw shade at all on anyone involved in 1.28. It is only meant to constructively improve the situation going forward. |
rami3l commentedMar 7, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
@joshka I second this. In fact, Ihave already mentioned it on Zulip:
I added that rustup could be modified to make use of the Let's leave this thread for the release-related work: I've made#4240 for the rustup-related part of this problem. |
The Rustup release process has historically been a manual process that involved copying files from S3 to the local machine and back to S3. This introduced a high risk of human error. When modifications to the existing release script became necessary, the decision was made to automate the release process (see#3819 for details).
The documentation in the dev-guide has been updated to cover the new release process, which is fully automated to produce
betareleases using GitHub Actions and thepromote-release tooling.