Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings
/jujuPublic

feat: controller upgrades#21082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
tlm wants to merge61 commits intojuju:main
base:main
Choose a base branch
Loading
fromtlm:controller-upgrade-changes
Draft

Conversation

@tlm
Copy link
Member

@tlmtlm commentedNov 3, 2025•
edited by CodingCookieRookie
Loading

TODO: Extends on the work done in#21015

Checklist

  • Code style: imports ordered, good names, simple structure, etc
  • Comments saying why design decisions were made
  • Go unit tests, with comments saying what you're testing
  • Integration tests, with comments saying what you're testing
  • doc.go added or updated in changed packages

QA steps

IAAS

1. Upgrade controller without passing --agent-version and --stream.

Test case 1.1: upgrade to the highest version from the controller store. Upload agents 4.0-beta10, 4.0-beta11, 4.0-beta12, and 4.0-beta13. It should upgrade to 4.0-beta13 because it’s the highest version.

  1. juju bootstrap ps6 ctrl-upgrade1-1 --build-agent
  2. juju switch controller
  3. changeversion/core.go to4.0-beta10
  4. make simplestreams
  5. export JUJU_METADATA_SOURCE=<your own path>
  6. juju sync-agent-binary --agent-version 4.0-beta10
  7. repeat steps 3 to 6 replacing the version
  8. juju upgrade-controller --dry-run -- should show4.0-beta13 as the best version
  9. juju upgrade-controller -- should upgrade to4.0-beta13 because it's the highest version.
  10. the version injuju status andjuju controllers should show4.0-beta13

Test case 1.2: exclude non-patch versions. Upload agents 4.0-beta10, 4.0-beta11, 4.0-beta12, 4.0-beta13, 4.1-beta1, and 5.0-beta1. It should upgrade to 4.0-beta14 because it’s the highest patch version.

  1. juju bootstrap ps6 ctrl-upgrade1-2 --build-agent
  2. juju switch controller
  3. changeversion/core.go to4.0-beta10
  4. make simplestreams
  5. export JUJU_METADATA_SOURCE=<your own path>
  6. juju sync-agent-binary --agent-version 4.0-beta10
  7. repeat steps 3 to 6 replacing the version
  8. juju upgrade-controller --dry-run -- should show4.0-beta13 as the best version
  9. juju upgrade-controller -- should upgrade to4.0-beta13 because it's the highest version. Agents4.1-beta1 and5.0-beta1 are exluded because they're not a patch upgrade from the current agent4.0-beta8.
  10. the version injuju status andjuju controllers should show4.0-beta13

Test case 1.3: upgrade to the highest version from simplestream. It should upgrade to 4.0-beta7 because that is the highest one in theproposed stream.

  1. changeversion/core.go to4.0-beta5
  2. juju bootstrap ps6 ctrl-upgrade1-3 --build-agent --config agent-stream=proposed
  3. juju switch controller
  4. juju upgrade-controller --dry-run -- should show4.0-beta7 as the best version.
  5. juju ssh 0 thentail -f /var/log/juju/machine-0.log -- we want to see the logs for downloading and extracting4.0-beta7 from simplestreams
  6. in another terminaljuju upgrade-controller -- should say that it started upgrade4.0-beta7
  7. go back to the first terminal where it's inside the controller machine, then check that/var/lib/juju/tools/4.0-beta7-ubuntu-amd64 exists because that's where the agent binary is extracted
  8. the4.0-beta7 binary actually fails to start with... "api-caller" manifold worker returned unexpected error: [*] "machine-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused which is expected. The fix to that is done infeat: upgrader services #21070 but4.0-beta7 doesn't have those changes. As long as the binaries are successfully extracted we treat this test case as a pass.

2. Upgrade controller passing --agent-version

Test case 2.1: upgrades to a version from the controller store. Pass agent-version 4.0-beta9 that we uploaded.

  1. juju bootstrap ps6 ctrl-upgrade2-1 --build-agent
  2. juju switch controller
  3. changeversion/core.go to4.0-beta9
  4. make simplestreams
  5. export JUJU_METADATA_SOURCE=<your own path>
  6. juju sync-agent-binary --agent-version 4.0-beta9
  7. juju upgrade-controller --dry-run -- should show4.0-beta9 as the best version
  8. juju upgrade-controller -- should upgrade to4.0-beta9 because it's the highest version.
  9. the version injuju status andjuju controllers should show4.0-beta9

Test case 2.2: upgrades to a version from simplestream. Pass an agent-version 4.0-beta7 that exists in stream.

  1. changeversion/core.go to4.0-beta5
  2. juju bootstrap ps6 ctrl-upgrade2-2 --build-agent --config agent-stream=proposed
  3. juju switch controller
  4. juju upgrade-controller --agent-version 4.0-beta7 --dry-run -- should show4.0-beta7 as the best version.
  5. juju ssh 0 thentail -f /var/log/juju/machine-0.log -- we want to see the logs for downloading and extracting4.0-beta7 from simplestreams
  6. in another terminaljuju upgrade-controller --agent-version 4.0-beta7 -- should say that it started upgrade4.0-beta7
  7. go back to the first terminal where it's inside the controller machine, then check that/var/lib/juju/tools/4.0-beta7-ubuntu-amd64 exists because that's where the agent binary is extracted
  8. the4.0-beta7 binary actually fails to start with... "api-caller" manifold worker returned unexpected error: [*] "machine-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused which is expected. The fix to that is done infeat: upgrader services #21070 but4.0-beta7 doesn't have those changes. As long as the binaries are successfully extracted we treat this test case as a pass.

3. Upgrade controller passing --agent-stream

Test case 3.1: upgrades to the highest version from a supplied stream in the controller store. Upload agents 4.0-beta10, 4.0-beta11, 4.0-beta12, and 4.0-beta13. It should upgrade to 4.0-beta13 because it’s the highest version.

It doesn’t matter what stream is passed when we want to upgrade to an agent from controller store. This is because we haven’t modeled the stream for the controller store.

  1. juju bootstrap ps6 ctrl-upgrade3-1 --build-agent
  2. juju switch controller
  3. changeversion/core.go to4.0-beta10
  4. make simplestreams
  5. export JUJU_METADATA_SOURCE=<your own path>
  6. juju sync-agent-binary --agent-version 4.0-beta10
  7. repeat steps 3 to 6 replacing the version
  8. juju upgrade-controller --dry-run --agent-stream released -- should show4.0-beta13 as the best version
  9. juju upgrade-controller --agent-stream released -- should upgrade to4.0-beta13 because it's the highest version.
  10. the version injuju status andjuju controllers should show4.0-beta13

Test case 3.2: upgrade to the highest version from simplestream. It should upgrade to 4.0-beta7 because that is the highest one in theproposed stream.

  1. changeversion/core.go to4.0-beta5
  2. juju bootstrap ps6 ctrl-upgrade3-2 --build-agent
  3. juju switch controller
  4. juju upgrade-controller --dry-run --agent-stream proposed -- should show4.0-beta7 as the best version.
  5. juju ssh 0 thentail -f /var/log/juju/machine-0.log -- we want to see the logs for downloading and extracting4.0-beta7 from simplestreams
  6. in another terminaljuju upgrade-controller --agent-stream proposed -- should say that it started upgrade4.0-beta7
  7. go back to the first terminal where it's inside the controller machine, then check that/var/lib/juju/tools/4.0-beta7-ubuntu-amd64 exists because that's where the agent binary is extracted
  8. the4.0-beta7 binary actually fails to start with... "api-caller" manifold worker returned unexpected error: [*] "machine-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused which is expected. The fix to that is done infeat: upgrader services #21070 but4.0-beta7 doesn't have those changes. As long as the binaries are successfully extracted we treat this test case as a pass.

4. Upgrade controller passing both --agent-version and --stream

Test case 4.1: upgrade to the supplied version and stream from the controller store. Upload agents 4.0-beta10, 4.0-beta11, 4.0-beta12, and 4.0-beta13 to seed the controller store.

It doesn’t matter what stream is passed when we want to upgrade to an agent from controller store. This is because we haven’t modeled the stream for the controller store.

  1. juju bootstrap ps6 ctrl-upgrade3-1 --build-agent
  2. juju switch controller
  3. changeversion/core.go to4.0-beta10
  4. make simplestreams
  5. export JUJU_METADATA_SOURCE=<your own path>
  6. juju sync-agent-binary --agent-version 4.0-beta10
  7. repeat steps 3 to 6 replacing the version
  8. juju upgrade-controller --dry-run --agent-version 4.0-beta11 --agent-stream released -- should show4.0-beta11 as the best version
  9. juju upgrade-controller --agent-version 4.0-beta11 --agent-stream released -- should upgrade to4.0-beta11
  10. the version injuju status andjuju controllers should show4.0-beta11

Test case 4.2: upgrade to to the supplied –agent-stream and –agent-version

  1. changeversion/core.go to4.0-beta5
  2. juju bootstrap ps6 ctrl-upgrade4-2 --build-agent
  3. juju switch controller
  4. juju upgrade-controller --dry-run --agent-version 4.0-beta7 --agent-stream proposed -- should show4.0-beta7 as the best version.
  5. juju ssh 0 thentail -f /var/log/juju/machine-0.log -- we want to see the logs for downloading and extracting4.0-beta7 from simplestreams
  6. in another terminaljuju upgrade-controller --agent-version 4.0-beta7 --agent-stream proposed -- should say that it started upgrade4.0-beta7
  7. go back to the first terminal where it's inside the controller machine, then check that/var/lib/juju/tools/4.0-beta7-ubuntu-amd64 exists because that's where the agent binary is extracted
  8. the4.0-beta7 binary actually fails to start with... "api-caller" manifold worker returned unexpected error: [*] "machine-0" cannot open api: unable to connect to API: dial tcp 127.0.0.1:17070: connect: connection refused which is expected. The fix to that is done infeat: upgrader services #21070 but4.0-beta7 doesn't have those changes. As long as the binaries are successfully extracted we treat this test case as a pass.

CAAS

Perfect — here’s yourCAAS QA section rewritten in the same structured style as your IAAS example, but withExpected / Actual explicitly included for each test case.
Everything is formatted for GitHub Markdown (readable, consistent, and reviewer-friendly).

QA Steps

CAAS

1. Upgrade controller without passing--agent-version or--agent-stream

Test case 1.1: Multiple agents (<4.0-beta7) uploaded to the controller store, and HTTP simplestreams has4.0-beta7.
Should upgrade to4.0-beta7 because it’s the highest version available.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. const version="4.0-beta9"
  4. Check on dqlite that the controller state doesnot contain4.0-beta9.
  5. juju model-config agent-stream=proposed
  6. juju upgrade-controller --dry-run — should show4.0-beta7 as the best version.
  7. juju upgrade-controller — should upgrade to4.0-beta7.
  8. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta7 from HTTP simplestreams because it’s the highest version.
Actual: Finds and upgrades to4.0-beta7 from HTTP simplestreams because it’s the highest version.

Test case 1.2: Upload4.0-beta9 to the controller store.
Should upgrade to4.0-beta9 because it’s the highest version available locally.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. const version="4.0-beta9"
  4. Check on dqlite that the controller state doesnot contain4.0-beta9.
  5. make simplestreams
  6. export JUJU_METADATA_SOURCE="/home/alvin.chee@canonical.com/Desktop/repos/juju/_build/simplestreams"
  7. juju sync-agent-binary --agent-version=4.0-beta9
  8. unset JUJU_METADATA_SOURCE
  9. juju upgrade-controller --debug
  10. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta9 from the controller store because it’s the highest version.
Actual: Finds and upgrades to4.0-beta9 from the controller store because it’s the highest version.

2. Upgrade controller passing--agent-version only

Test case 2.1: HTTP simplestreams has4.0-beta5.
Should upgrade to4.0-beta5 explicitly from the stream.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. Check on dqlite that controller state doesnot contain4.0-beta5.
  4. juju model-config agent-stream=proposed
  5. juju upgrade-controller --agent-version=4.0-beta5
  6. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta5 from HTTP simplestreams.
Actual: Finds and upgrades to4.0-beta5 from HTTP simplestreams.

Test case 2.2:4.0-beta9 agent uploaded to the controller store.
Passing--agent-version=4.0-beta5 should upgrade to4.0-beta5 specifically.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. const version="4.0-beta9"
  4. Check on dqlite that the controller state doesnot contain4.0-beta9.
  5. make simplestreams
  6. export JUJU_METADATA_SOURCE="/home/alvin.chee@canonical.com/Desktop/repos/juju/_build/simplestreams"
  7. juju sync-agent-binary --agent-version=4.0-beta9
  8. unset JUJU_METADATA_SOURCE
  9. juju upgrade-controller --agent-version=4.0-beta5
  10. Verify controller version withjuju status andjuju controllers.

Expected: Upgrades to4.0-beta5 (explicitly specified version).
Actual: Upgrades to4.0-beta5 (explicitly specified version).

3. Upgrade controller passing--agent-stream only

Test case 3.1: HTTP simplestreams has4.0-beta7.
Should upgrade to4.0-beta7 because it’s the highest version in theproposed stream.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. Check on dqlite that controller state doesnot contain4.0-beta5.
  4. juju model-config agent-stream=proposed
  5. juju upgrade-controller --agent-stream=proposed
  6. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta7 from HTTP simplestreams because it’s the highest version.
Actual: Finds and upgrades to4.0-beta7 from HTTP simplestreams because it’s the highest version.
🛈 StatefulSet image version remains4.0-beta3 — expected due to known issue in4.0-beta7.

Test case 3.2:4.0-beta9 agent uploaded to controller store.
Should upgrade to4.0-beta9 because it’s the highest version available.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. const version="4.0-beta9"
  4. juju upgrade-controller --agent-stream=released --dry-run — verify controller store doesn’t yet have4.0-beta9.
  5. make simplestreams
  6. export JUJU_METADATA_SOURCE="/home/alvin.chee@canonical.com/Desktop/repos/juju/_build/simplestreams"
  7. juju sync-agent-binary --agent-version=4.0-beta9
  8. unset JUJU_METADATA_SOURCE
  9. juju upgrade-controller --agent-stream=released
  10. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta9 from the controller store because it’s the highest version.
Actual: Finds and upgrades to4.0-beta9 from the controller store because it’s the highest version.

4. Upgrade controller passing both--agent-version and--agent-stream

Test case 4.1: HTTP simplestreams has4.0-beta7.
Should upgrade to the explicitly supplied version4.0-beta7 from the stream.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. Check on dqlite that controller state doesnot contain4.0-beta5.
  4. juju model-config agent-stream=proposed
  5. juju upgrade-controller --agent-stream=proposed --agent-version=4.0-beta7
  6. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta7 from HTTP simplestreams because it’s the highest version.
Actual: Finds and upgrades to4.0-beta7 from HTTP simplestreams because it’s the highest version.

Test case 4.2:4.0-beta9 agent uploaded to controller store.
Should upgrade to4.0-beta9 as the explicitly supplied version from the released stream.

  1. const version="4.0-beta3"
  2. make minikube-operator-update && juju bootstrap minikube alvin3
  3. const version="4.0-beta9"
  4. juju upgrade-controller --agent-stream=released --agent-version=4.0-beta9 --dry-run — verify controller store doesn’t yet have4.0-beta9.
  5. make simplestreams
  6. export JUJU_METADATA_SOURCE="/home/alvin.chee@canonical.com/Desktop/repos/juju/_build/simplestreams"
  7. juju sync-agent-binary --agent-stream=released --agent-version=4.0-beta9
  8. unset JUJU_METADATA_SOURCE
  9. juju upgrade-controller --agent-stream=released --agent-version=4.0-beta9
  10. Verify controller version withjuju status andjuju controllers.

Expected: Finds and upgrades to4.0-beta9 from the controller store because it’s the highest version.
Actual: Finds and upgrades to4.0-beta9 from the controller store because it’s the highest version.

Documentation changes

Links

Issue:Fixes#19267.

Jira card:JUJU-

adisazhar123and others added25 commitsNovember 5, 2025 08:36
Lowercases the sqlair type used for agent stream as it doesn't need tobe exported.
New set of types to the agent binary domain for representing a singularagent binary. Have added tests as well for helpers when working withslices of agent binaries.
Implements in state the ability to request all agent binaries that existfor a stream in the controllers object store.
matchThis commit introduces a new agent binary helper for matching agentbinaries in a slice that don't match the given target version. This wasintroduced to work a slice down to versions that a caller cares about.
This commit adds to the agent binary domain two new functions forworking with multiple architecture values. The first func allowstransforming a slice of agent binaries to a slice of architectures.The second func helps perform a difference over to architecture slicesto get the values from a which do not appear in b.This work was done to support controller upgrades.
Documents error conditions on functions and adds a stream value check.
Moving logic from the controller upgrade to the agent binary domainaround simple streams. This is so that all agent binary sources can berepresented as a store.
This commit is about getting some renaming done to represent ourconcepts a little more. I renamed agent store to agent object store.This was done so I can take over the name agent store for interfaces.This will allow use to describe different types of stores underdifferent names.
This commit creates a new set of interfaces in the agent binary domainfor describing what a store of agent binaries looks like. The newstructure defines store for getting, putting and querying agentbinaries.They have been seperated like this because not all stores support allfunctionality.
Defines the functions for placing binaries into a store.
architecturesIntroduces a new func to the agent binary service for gett all availablesupported architectures.
Provides a new slice helper for filtering agent binaries that do notmatch a set of supplied architectures.
Adds a new method to the getter store for fetchin agent binaries by agiven sha. Ths is require for fetching and caching from externalsources.
This new type exists to replace the one available in core/agentbinary.This allows us to be in the domain and also use the correct arch enumtype.
Makes use of the new agent binary version type in the domain.
This commit adds a new func to the arch type for testing if anarchitecture is considered valid.
@tlmtlmforce-pushed thecontroller-upgrade-changes branch from58d0e50 to8c76b72CompareNovember 5, 2025 08:39
Comment on lines 599 to 600
// TODO (adisazhar123): replace with its respective New* method.
varcontrollerStore controllerupgraderservice.AgentBinaryQuerierStore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

TODO: need to wireup once controller store impl is available

tlmand others added24 commitsNovember 5, 2025 13:59
In practice they are not used and until all of our processing is over tousing these sha's they won't be.
The agent binary service was double caching downloads and then passingit off to the store which was doing the same. Remove this code anddelegates processing to the store.
After having been removed from the stores we can now just test insha256.
Updates agent binary state to match new interface requirements.
Updates to take a variadic argument set which makes the callers easierto write.
Updates the agent binary service to only consider finding a single agentbinary. This better represents what the caller requires.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@adisazhar123adisazhar123adisazhar123 left review comments

At least 2 approving reviews are required to merge this pull request.

Assignees

@tlmtlm

Labels

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Example issue

4 participants

@tlm@adisazhar123@jujubot@CodingCookieRookie
[8]ページ先頭
©2009-2025 Movatter.jp