Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: track resource replacements when claiming a prebuilt workspace#17571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
dannykopping merged 38 commits intomainfromdk/logreplacements
May 14, 2025
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
38 commits
Select commitHold shift + click to select a range
b32923a
feat: log resource replacements
dannykoppingApr 25, 2025
0b0830f
feat: show terraform state drift diff in build logs
dannykoppingApr 25, 2025
256395a
feat: only highlight lines which mention replacement
dannykoppingApr 25, 2025
61ef61a
feat: notify template admins when prebuild claim results in resource …
dannykoppingApr 25, 2025
a66559f
chore: appease linter
dannykoppingApr 25, 2025
222892b
chore: fix notifications test
dannykoppingApr 25, 2025
f34e011
fix: don't panic
dannykoppingApr 28, 2025
5168c01
fix: renaming type
dannykoppingApr 28, 2025
41e5e0c
chore: updating migration numbers
dannykoppingMay 6, 2025
b29e8fa
chore: minor touch-ups
dannykoppingMay 6, 2025
b31ed5e
feat: add resource replacements metric
dannykoppingMay 7, 2025
adf98d2
feat: add resource replacement notification
dannykoppingMay 7, 2025
f24aef0
make lint; make fmt
dannykoppingMay 7, 2025
70f9a53
chore: adding tests
dannykoppingMay 8, 2025
1e8385d
feat: pass flag to terraform provider when prebuilt workspace claimed
dannykoppingMay 9, 2025
d0f00ce
chore: update provider, add test for is_prebuild_claim
dannykoppingMay 12, 2025
11a2c5a
Merge branch 'main' of github.com:/coder/coder into dk/logreplacements
dannykoppingMay 12, 2025
ce63b24
Merge branch 'dk/is-prebuild-claim' of github.com:/coder/coder into d…
dannykoppingMay 12, 2025
d2c5d43
chore: replace GetTemplatePresetsByID with GetPresetByID
dannykoppingMay 12, 2025
22d82a4
chore: correcting docs link
dannykoppingMay 12, 2025
5209aae
Merge branch 'main' of github.com:/coder/coder into dk/logreplacement
dannykoppingMay 12, 2025
39ce658
Merge branch 'main' of github.com:/coder/coder into dk/logreplacements
dannykoppingMay 12, 2025
ac5655f
Merge branch 'main' of github.com:/coder/coder into dk/logreplacements
dannykoppingMay 12, 2025
82c3f58
chore: note provisioner API change
dannykoppingMay 12, 2025
7577a90
chore: fixups
dannykoppingMay 13, 2025
a893b79
chore: adding note about immutable resources
dannykoppingMay 13, 2025
d9c906a
chore: review feedback
dannykoppingMay 13, 2025
471198a
Merge branch 'main' of github.com:/coder/coder into dk/logreplacements
dannykoppingMay 13, 2025
7d694e6
chore: merge conflicts
dannykoppingMay 13, 2025
6b7a8b7
chore: fix 'is not iterable' bullshit
dannykoppingMay 13, 2025
5df2cb3
Merge branch 'main' of github.com:/coder/coder into dk/logreplacements
dannykoppingMay 14, 2025
6d1c3ea
chore: rename migrations
dannykoppingMay 14, 2025
5f62702
chore: set notifications manager before enterprise server initializes…
dannykoppingMay 14, 2025
f74d799
chore: completing refactor since https://github.com/coder/coder/pull/…
dannykoppingMay 14, 2025
971f65c
chore: remove unnecessary atomicity since map is protected by mutex a…
dannykoppingMay 14, 2025
bc362b0
chore: appeasing linter's Very Important Suggestion
dannykoppingMay 14, 2025
4fbd356
Merge branch 'main' of github.com:/coder/coder into dk/logreplacements
dannykoppingMay 14, 2025
b9eb8be
chore: remove old replacement logging
dannykoppingMay 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 31 additions & 31 deletionscli/server.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -928,6 +928,37 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.
options.StatsBatcher = batcher
defer closeBatcher()

// Manage notifications.
var (
notificationsCfg = options.DeploymentValues.Notifications
notificationsManager *notifications.Manager
)

metrics := notifications.NewMetrics(options.PrometheusRegistry)
helpers := templateHelpers(options)

// The enqueuer is responsible for enqueueing notifications to the given store.
enqueuer, err := notifications.NewStoreEnqueuer(notificationsCfg, options.Database, helpers, logger.Named("notifications.enqueuer"), quartz.NewReal())
if err != nil {
return xerrors.Errorf("failed to instantiate notification store enqueuer: %w", err)
}
options.NotificationsEnqueuer = enqueuer

// The notification manager is responsible for:
// - creating notifiers and managing their lifecycles (notifiers are responsible for dequeueing/sending notifications)
// - keeping the store updated with status updates
notificationsManager, err = notifications.NewManager(notificationsCfg, options.Database, options.Pubsub, helpers, metrics, logger.Named("notifications.manager"))
if err != nil {
return xerrors.Errorf("failed to instantiate notification manager: %w", err)
}

// nolint:gocritic // We need to run the manager in a notifier context.
notificationsManager.Run(dbauthz.AsNotifier(ctx))

// Run report generator to distribute periodic reports.
notificationReportGenerator := reports.NewReportGenerator(ctx, logger.Named("notifications.report_generator"), options.Database, options.NotificationsEnqueuer, quartz.NewReal())
defer notificationReportGenerator.Close()

// We use a separate coderAPICloser so the Enterprise API
// can have its own close functions. This is cleaner
// than abstracting the Coder API itself.
Expand DownExpand Up@@ -975,37 +1006,6 @@ func (r *RootCmd) Server(newAPI func(context.Context, *coderd.Options) (*coderd.
return xerrors.Errorf("write config url: %w", err)
}

// Manage notifications.
var (
notificationsCfg = options.DeploymentValues.Notifications
notificationsManager *notifications.Manager
)

metrics := notifications.NewMetrics(options.PrometheusRegistry)
helpers := templateHelpers(options)

// The enqueuer is responsible for enqueueing notifications to the given store.
enqueuer, err := notifications.NewStoreEnqueuer(notificationsCfg, options.Database, helpers, logger.Named("notifications.enqueuer"), quartz.NewReal())
if err != nil {
return xerrors.Errorf("failed to instantiate notification store enqueuer: %w", err)
}
options.NotificationsEnqueuer = enqueuer

// The notification manager is responsible for:
// - creating notifiers and managing their lifecycles (notifiers are responsible for dequeueing/sending notifications)
// - keeping the store updated with status updates
notificationsManager, err = notifications.NewManager(notificationsCfg, options.Database, options.Pubsub, helpers, metrics, logger.Named("notifications.manager"))
if err != nil {
return xerrors.Errorf("failed to instantiate notification manager: %w", err)
}

// nolint:gocritic // We need to run the manager in a notifier context.
notificationsManager.Run(dbauthz.AsNotifier(ctx))

// Run report generator to distribute periodic reports.
notificationReportGenerator := reports.NewReportGenerator(ctx, logger.Named("notifications.report_generator"), options.Database, options.NotificationsEnqueuer, quartz.NewReal())
defer notificationReportGenerator.Close()

// Since errCh only has one buffered slot, all routines
// sending on it must be wrapped in a select/default to
// avoid leaving dangling goroutines waiting for the
Expand Down
4 changes: 3 additions & 1 deletioncoderd/coderd.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -40,10 +40,11 @@ import (
"tailscale.com/util/singleflight"

"cdr.dev/slog"
"github.com/coder/coder/v2/codersdk/drpcsdk"
"github.com/coder/quartz"
"github.com/coder/serpent"

"github.com/coder/coder/v2/codersdk/drpcsdk"

"github.com/coder/coder/v2/coderd/ai"
"github.com/coder/coder/v2/coderd/cryptokeys"
"github.com/coder/coder/v2/coderd/entitlements"
Expand DownExpand Up@@ -1795,6 +1796,7 @@ func (api *API) CreateInMemoryTaggedProvisionerDaemon(dialCtx context.Context, n
Clock: api.Clock,
},
api.NotificationsEnqueuer,
&api.PrebuildsReconciler,
)
if err != nil {
return nil, err
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
DELETE FROM notification_templates WHERE id = '89d9745a-816e-4695-a17f-3d0a229e2b8d';
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
INSERT INTO notification_templates
(id, name, title_template, body_template, "group", actions)
VALUES ('89d9745a-816e-4695-a17f-3d0a229e2b8d',
'Prebuilt Workspace Resource Replaced',
E'There might be a problem with a recently claimed prebuilt workspace',
$$
Workspace **{{.Labels.workspace}}** was claimed from a prebuilt workspace by **{{.Labels.claimant}}**.

During the claim, Terraform destroyed and recreated the following resources
because one or more immutable attributes changed:

{{range $resource, $paths := .Data.replacements -}}
- _{{ $resource }}_ was replaced due to changes to _{{ $paths }}_
{{end}}

When Terraform must change an immutable attribute, it replaces the entire resource.
If you’re using prebuilds to speed up provisioning, unexpected replacements will slow down
workspace startup—even when claiming a prebuilt environment.

For tips on preventing replacements and improving claim performance, see [this guide](https://coder.com/docs/admin/templates/extending-templates/prebuilt-workspaces#preventing-resource-replacement).

NOTE: this prebuilt workspace used the **{{.Labels.preset}}** preset.
$$,
'Template Events',
'[
{
"label": "View workspace build",
"url": "{{base_url}}/@{{.Labels.claimant}}/{{.Labels.workspace}}/builds/{{.Labels.workspace_build_num}}"
},
{
"label": "View template version",
"url": "{{base_url}}/templates/{{.Labels.org}}/{{.Labels.template}}/versions/{{.Labels.template_version}}"
}
]'::jsonb);
1 change: 1 addition & 0 deletionscoderd/notifications/events.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -39,6 +39,7 @@ var (
TemplateTemplateDeprecated = uuid.MustParse("f40fae84-55a2-42cd-99fa-b41c1ca64894")

TemplateWorkspaceBuildsFailedReport = uuid.MustParse("34a20db2-e9cc-4a93-b0e4-8569699d7a00")
TemplateWorkspaceResourceReplaced = uuid.MustParse("89d9745a-816e-4695-a17f-3d0a229e2b8d")
)

// Notification-related events.
Expand Down
28 changes: 26 additions & 2 deletionscoderd/notifications/notifications_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -35,6 +35,9 @@ import (
"golang.org/x/xerrors"

"cdr.dev/slog"
"github.com/coder/quartz"
"github.com/coder/serpent"

"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/coderd/database"
"github.com/coder/coder/v2/coderd/database/dbauthz"
Expand All@@ -48,8 +51,6 @@ import (
"github.com/coder/coder/v2/coderd/util/syncmap"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/testutil"
"github.com/coder/quartz"
"github.com/coder/serpent"
)

// updateGoldenFiles is a flag that can be set to update golden files.
Expand DownExpand Up@@ -1226,6 +1227,29 @@ func TestNotificationTemplates_Golden(t *testing.T) {
Labels: map[string]string{},
},
},
{
name: "TemplateWorkspaceResourceReplaced",
id: notifications.TemplateWorkspaceResourceReplaced,
payload: types.MessagePayload{
UserName: "Bobby",
UserEmail: "bobby@coder.com",
UserUsername: "bobby",
Labels: map[string]string{
"org": "cern",
"workspace": "my-workspace",
"workspace_build_num": "2",
"template": "docker",
"template_version": "angry_torvalds",
"preset": "particle-accelerator",
"claimant": "prebuilds-claimer",
},
Data: map[string]any{
"replacements": map[string]string{
"docker_container[0]": "env, hostname",
},
},
},
},
}

// We must have a test case for every notification_template. This is enforced below:
Expand Down
7 changes: 7 additions & 0 deletionscoderd/notifications/notificationstest/fake_enqueuer.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -9,6 +9,7 @@ import (
"github.com/prometheus/client_golang/prometheus"

"github.com/coder/coder/v2/coderd/database/dbauthz"
"github.com/coder/coder/v2/coderd/notifications"
"github.com/coder/coder/v2/coderd/rbac"
"github.com/coder/coder/v2/coderd/rbac/policy"
)
Expand All@@ -19,6 +20,12 @@ type FakeEnqueuer struct {
sent []*FakeNotification
}

var _ notifications.Enqueuer = &FakeEnqueuer{}

func NewFakeEnqueuer() *FakeEnqueuer {
return &FakeEnqueuer{}
}

type FakeNotification struct {
UserID, TemplateID uuid.UUID
Labels map[string]string
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
From: system@coder.com
To: bobby@coder.com
Subject: There might be a problem with a recently claimed prebuilt workspace
Message-Id: 02ee4935-73be-4fa1-a290-ff9999026b13@blush-whale-48
Date: Fri, 11 Oct 2024 09:03:06 +0000
Content-Type: multipart/alternative; boundary=bbe61b741255b6098bb6b3c1f41b885773df633cb18d2a3002b68e4bc9c4
MIME-Version: 1.0

--bbe61b741255b6098bb6b3c1f41b885773df633cb18d2a3002b68e4bc9c4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

Hi Bobby,

Workspace my-workspace was claimed from a prebuilt workspace by prebuilds-c=
laimer.

During the claim, Terraform destroyed and recreated the following resources
because one or more immutable attributes changed:

docker_container[0] was replaced due to changes to env, hostname

When Terraform must change an immutable attribute, it replaces the entire r=
esource.
If you=E2=80=99re using prebuilds to speed up provisioning, unexpected repl=
acements will slow down
workspace startup=E2=80=94even when claiming a prebuilt environment.

For tips on preventing replacements and improving claim performance, see th=
is guide (https://coder.com/docs/admin/templates/extending-templates/prebui=
lt-workspaces#preventing-resource-replacement).

NOTE: this prebuilt workspace used the particle-accelerator preset.


View workspace build: http://test.com/@prebuilds-claimer/my-workspace/build=
s/2

View template version: http://test.com/templates/cern/docker/versions/angry=
_torvalds

--bbe61b741255b6098bb6b3c1f41b885773df633cb18d2a3002b68e4bc9c4
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8

<!doctype html>
<html lang=3D"en">
<head>
<meta charset=3D"UTF-8" />
<meta name=3D"viewport" content=3D"width=3Ddevice-width, initial-scale=
=3D1.0" />
<title>There might be a problem with a recently claimed prebuilt worksp=
ace</title>
</head>
<body style=3D"margin: 0; padding: 0; font-family: -apple-system, system-=
ui, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarel=
l', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif; color: #020617=
; background: #f8fafc;">
<div style=3D"max-width: 600px; margin: 20px auto; padding: 60px; borde=
r: 1px solid #e2e8f0; border-radius: 8px; background-color: #fff; text-alig=
n: left; font-size: 14px; line-height: 1.5;">
<div style=3D"text-align: center;">
<img src=3D"https://coder.com/coder-logo-horizontal.png" alt=3D"Cod=
er Logo" style=3D"height: 40px;" />
</div>
<h1 style=3D"text-align: center; font-size: 24px; font-weight: 400; m=
argin: 8px 0 32px; line-height: 1.5;">
There might be a problem with a recently claimed prebuilt workspace
</h1>
<div style=3D"line-height: 1.5;">
<p>Hi Bobby,</p>
<p>Workspace <strong>my-workspace</strong> was claimed from a prebu=
ilt workspace by <strong>prebuilds-claimer</strong>.</p>

<p>During the claim, Terraform destroyed and recreated the following resour=
ces<br>
because one or more immutable attributes changed:</p>

<ul>
<li>_docker<em>container[0]</em> was replaced due to changes to <em>env, h=
ostname</em><br>
</li>
</ul>

<p>When Terraform must change an immutable attribute, it replaces the entir=
e resource.<br>
If you=E2=80=99re using prebuilds to speed up provisioning, unexpected repl=
acements will slow down<br>
workspace startup=E2=80=94even when claiming a prebuilt environment.</p>

<p>For tips on preventing replacements and improving claim performance, see=
<a href=3D"https://coder.com/docs/admin/templates/extending-templates/preb=
uilt-workspaces#preventing-resource-replacement">this guide</a>.</p>

<p>NOTE: this prebuilt workspace used the <strong>particle-accelerator</str=
ong> preset.</p>
</div>
<div style=3D"text-align: center; margin-top: 32px;">
=20
<a href=3D"http://test.com/@prebuilds-claimer/my-workspace/builds/2=
" style=3D"display: inline-block; padding: 13px 24px; background-color: #02=
0617; color: #f8fafc; text-decoration: none; border-radius: 8px; margin: 0 =
4px;">
View workspace build
</a>
=20
<a href=3D"http://test.com/templates/cern/docker/versions/angry_tor=
valds" style=3D"display: inline-block; padding: 13px 24px; background-color=
: #020617; color: #f8fafc; text-decoration: none; border-radius: 8px; margi=
n: 0 4px;">
View template version
</a>
=20
</div>
<div style=3D"border-top: 1px solid #e2e8f0; color: #475569; font-siz=
e: 12px; margin-top: 64px; padding-top: 24px; line-height: 1.6;">
<p>&copy;&nbsp;2024&nbsp;Coder. All rights reserved&nbsp;-&nbsp;<a =
href=3D"http://test.com" style=3D"color: #2563eb; text-decoration: none;">h=
ttp://test.com</a></p>
<p><a href=3D"http://test.com/settings/notifications" style=3D"colo=
r: #2563eb; text-decoration: none;">Click here to manage your notification =
settings</a></p>
<p><a href=3D"http://test.com/settings/notifications?disabled=3D89d=
9745a-816e-4695-a17f-3d0a229e2b8d" style=3D"color: #2563eb; text-decoration=
: none;">Stop receiving emails like this</a></p>
</div>
</div>
</body>
</html>

--bbe61b741255b6098bb6b3c1f41b885773df633cb18d2a3002b68e4bc9c4--
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
{
"_version": "1.1",
"msg_id": "00000000-0000-0000-0000-000000000000",
"payload": {
"_version": "1.2",
"notification_name": "Prebuilt Workspace Resource Replaced",
"notification_template_id": "00000000-0000-0000-0000-000000000000",
"user_id": "00000000-0000-0000-0000-000000000000",
"user_email": "bobby@coder.com",
"user_name": "Bobby",
"user_username": "bobby",
"actions": [
{
"label": "View workspace build",
"url": "http://test.com/@prebuilds-claimer/my-workspace/builds/2"
},
{
"label": "View template version",
"url": "http://test.com/templates/cern/docker/versions/angry_torvalds"
}
],
"labels": {
"claimant": "prebuilds-claimer",
"org": "cern",
"preset": "particle-accelerator",
"template": "docker",
"template_version": "angry_torvalds",
"workspace": "my-workspace",
"workspace_build_num": "2"
},
"data": {
"replacements": {
"docker_container[0]": "env, hostname"
}
},
"targets": null
},
"title": "There might be a problem with a recently claimed prebuilt workspace",
"title_markdown": "There might be a problem with a recently claimed prebuilt workspace",
"body": "Workspace my-workspace was claimed from a prebuilt workspace by prebuilds-claimer.\n\nDuring the claim, Terraform destroyed and recreated the following resources\nbecause one or more immutable attributes changed:\n\ndocker_container[0] was replaced due to changes to env, hostname\n\nWhen Terraform must change an immutable attribute, it replaces the entire resource.\nIf you’re using prebuilds to speed up provisioning, unexpected replacements will slow down\nworkspace startup—even when claiming a prebuilt environment.\n\nFor tips on preventing replacements and improving claim performance, see this guide (https://coder.com/docs/admin/templates/extending-templates/prebuilt-workspaces#preventing-resource-replacement).\n\nNOTE: this prebuilt workspace used the particle-accelerator preset.",
"body_markdown": "\nWorkspace **my-workspace** was claimed from a prebuilt workspace by **prebuilds-claimer**.\n\nDuring the claim, Terraform destroyed and recreated the following resources\nbecause one or more immutable attributes changed:\n\n- _docker_container[0]_ was replaced due to changes to _env, hostname_\n\n\nWhen Terraform must change an immutable attribute, it replaces the entire resource.\nIf you’re using prebuilds to speed up provisioning, unexpected replacements will slow down\nworkspace startup—even when claiming a prebuilt environment.\n\nFor tips on preventing replacements and improving claim performance, see [this guide](https://coder.com/docs/admin/templates/extending-templates/prebuilt-workspaces#preventing-resource-replacement).\n\nNOTE: this prebuilt workspace used the **particle-accelerator** preset.\n"
}
Loading
Loading

[8]ページ先頭

©2009-2025 Movatter.jp