- Notifications
You must be signed in to change notification settings - Fork1.1k
feat: delete pending canceled prebuilds#20499
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
d45515d to85f8107Compare85f8107 tob0cde3bCompare| database.ProvisionerJobStatusCanceling, | ||
| }, | ||
| templateVersionActive: []bool{true,false}, | ||
| templateVersionActive: []bool{true}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
There seems to be an issue with this test: when a template version is inactive (templateVersionActive = false), any prebuilds in thedatabase.ProvisionerJobStatusRunning state should be deleted.
From some quick debugging, it looks like the reconciliation loop isn’t picking up running prebuilds, theGetRunningPrebuiltWorkspaces query is returning 0 rows. I suspect this might be related to the agent not being properly marked as “ready”, which would cause those prebuilds to be excluded from the query results.
I’ll investigate this further and address the test in a follow-up PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Can you add a TODO for this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Can you add a TODO for this?
Added inb135381
johnstcn left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't have any blocking comments.
| workspace,err:=db.GetWorkspaceByID(ctx,workspaceID) | ||
| iferr!=nil { | ||
| returnxerrors.Errorf("get workspace by ID: %w",err) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I was going to mention that since you changed theUpdatePrebuildProvisionerJobWithCancel query to return workspace_id, we could also returnowner_id there as well so we can use it here. Unfortunately it looks likewsbuilder needs the workspace in any case.
| database.ProvisionerJobStatusCanceling, | ||
| }, | ||
| templateVersionActive: []bool{true,false}, | ||
| templateVersionActive: []bool{true}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Can you add a TODO for this?
Uh oh!
There was an error while loading.Please reload this page.
c3e3bb5 intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Description
PR#20387 introduced canceling pending prebuild jobs from inactive template versions to avoid provisioning obsolete workspaces. However, the associated prebuilds remained in the database with "Canceled" status, visible in the UI.
This PR now orphan-deletes these canceled prebuilt workspaces. Since the canceled jobs were never processed by a provisioner, no Terraform resources were created, making orphan deletion safe.
Orphan deletion always creates a provisioner job, but behaves differently based on provisioner availability:
The job cancellation and workspace deletion occur atomically in the same transaction. We don't split this into two separate reconciliation runs because there's no way to distinguish between system-canceled prebuilds and user-canceled workspaces. If we deleted canceled workspaces in a later run, we'd delete user-canceled workspaces that users may want to keep for troubleshooting.
Note: This only applies to system-generated prebuilds from inactive template versions.
Changes
UpdatePrebuildProvisionerJobWithCancelquery to return job ID, workspace ID, template ID, and template version preset IDDeprovisionModeenum to support orphan deletion in the provision flowActionTypeCancelPendinghandler to cancel jobs and orphan-delete associated workspaces atomically