Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

fix: add constraint and runtime check for provisioner logs size limit#18893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
28 commits
Select commitHold shift + click to select a range
9f8cd50
add logs overflowed field to provisioner jobs and gen models
bcpeinhardtJul 15, 2025
68d8e5e
update queries and make gen
bcpeinhardtJul 15, 2025
476c6ff
add logs length field as well with constraint
bcpeinhardtJul 15, 2025
c8de633
update queries
bcpeinhardtJul 15, 2025
908c888
handle log size overflow on insertion
bcpeinhardtJul 15, 2025
c709a9e
update sdk
bcpeinhardtJul 15, 2025
f32ac68
implement dbauthz stubs
bcpeinhardtJul 15, 2025
29ad77c
couple tests
bcpeinhardtJul 15, 2025
f6134aa
initial frontend updates
bcpeinhardtJul 15, 2025
cc469f8
runtime check possible overflow
bcpeinhardtJul 21, 2025
29297d6
update log message to be less confusing
bcpeinhardtJul 21, 2025
cd891db
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 21, 2025
1398491
lint fixes
bcpeinhardtJul 21, 2025
395c9e1
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 21, 2025
630e2ac
rename migrations after conflicting merge
bcpeinhardtJul 21, 2025
e44cd47
run make gen after migration
bcpeinhardtJul 21, 2025
fa118db
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 21, 2025
105de84
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 29, 2025
2923e9d
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 30, 2025
61a58f5
adjust migration numbers
bcpeinhardtJul 30, 2025
e7f2ec6
update dbauthz and querymetrics for removed query
bcpeinhardtJul 30, 2025
4a2a7f8
add rbac tests for provisioner job logs length and overflowed queries
bcpeinhardtJul 30, 2025
8115e4a
reshuffling
bcpeinhardtJul 30, 2025
99d5090
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 30, 2025
c09f0f6
update migration numbers
bcpeinhardtJul 30, 2025
1ab1347
remove period from slog log
bcpeinhardtJul 30, 2025
6964cee
Merge branch 'main' into bcpeinhardt/17992-fix-startup-logs-size-limi…
bcpeinhardtJul 30, 2025
a03a956
migration rename
bcpeinhardtJul 30, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletioncli/testdata/coder_list_--output_json.golden
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -55,7 +55,8 @@
"template_name": "",
"template_display_name": "",
"template_icon": ""
}
},
"logs_overflowed": false
},
"reason": "initiator",
"resources": [],
Expand Down
2 changes: 1 addition & 1 deletioncli/testdata/coder_provisioner_jobs_list_--help.golden
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -11,7 +11,7 @@ OPTIONS:
-O, --org string, $CODER_ORGANIZATION
Select which organization (uuid or name) to use.

-c, --column [id|created at|started at|completed at|canceled at|error|error code|status|worker id|worker name|file id|tags|queue position|queue size|organization id|template version id|workspace build id|type|available workers|template version name|template id|template name|template display name|template icon|workspace id|workspace name|organization|queue] (default: created at,id,type,template display name,status,queue,tags)
-c, --column [id|created at|started at|completed at|canceled at|error|error code|status|worker id|worker name|file id|tags|queue position|queue size|organization id|template version id|workspace build id|type|available workers|template version name|template id|template name|template display name|template icon|workspace id|workspace name|logs overflowed|organization|queue] (default: created at,id,type,template display name,status,queue,tags)
Columns to display in table output.

-l, --limit int, $CODER_PROVISIONER_JOB_LIST_LIMIT (default: 50)
Expand Down
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -26,6 +26,7 @@
"template_display_name": "",
"template_icon": ""
},
"logs_overflowed": false,
"organization_name": "Coder"
},
{
Expand DownExpand Up@@ -57,6 +58,7 @@
"workspace_id": "===========[workspace ID]===========",
"workspace_name": "test-workspace"
},
"logs_overflowed": false,
"organization_name": "Coder"
}
]
3 changes: 3 additions & 0 deletionscoderd/apidoc/docs.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

3 changes: 3 additions & 0 deletionscoderd/apidoc/swagger.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

16 changes: 16 additions & 0 deletionscoderd/database/dbauthz/dbauthz.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4489,6 +4489,22 @@ func (q *querier) UpdateProvisionerJobByID(ctx context.Context, arg database.Upd
return q.db.UpdateProvisionerJobByID(ctx, arg)
}

func (q *querier) UpdateProvisionerJobLogsLength(ctx context.Context, arg database.UpdateProvisionerJobLogsLengthParams) error {
// Not sure what the rbac should be here, going with this for now
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is the only bit I'm unsure of, maybe include the same comment as the other provisioner related functions have?

// TODO: Remove this once we have a proper rbac check for provisioner jobs.// Details in https://github.com/coder/coder/issues/16160

or ask@mafredri since he opened that issue?

Copy link
ContributorAuthor

@bcpeinhardtbcpeinhardtJul 30, 2025
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Our manual testing was actually from the admin account, we should look at this again before merging I think.
Edit: I tested with a member account and everything works fine. I could be way off here but I think the action.Update on the ResourceProvisionerJob is coming from the template admin?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

There are several other queries which perform the same rbac check we do to update fields on the ProvisionerJobs table, so I feel comfortable merging this and revisiting the provisioner jobs rbac in the future.

if err := q.authorizeContext(ctx, policy.ActionUpdate, rbac.ResourceProvisionerJobs); err != nil {
return err
}
return q.db.UpdateProvisionerJobLogsLength(ctx, arg)
}

func (q *querier) UpdateProvisionerJobLogsOverflowed(ctx context.Context, arg database.UpdateProvisionerJobLogsOverflowedParams) error {
// Not sure what the rbac should be here, going with this for now
if err := q.authorizeContext(ctx, policy.ActionUpdate, rbac.ResourceProvisionerJobs); err != nil {
return err
}
return q.db.UpdateProvisionerJobLogsOverflowed(ctx, arg)
}

func (q *querier) UpdateProvisionerJobWithCancelByID(ctx context.Context, arg database.UpdateProvisionerJobWithCancelByIDParams) error {
// TODO: Remove this once we have a proper rbac check for provisioner jobs.
// Details in https://github.com/coder/coder/issues/16160
Expand Down
14 changes: 14 additions & 0 deletionscoderd/database/dbauthz/dbauthz_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4341,6 +4341,20 @@ func (s *MethodTestSuite) TestSystemFunctions() {
UpdatedAt: time.Now(),
}).Asserts(rbac.ResourceProvisionerJobs, policy.ActionUpdate)
}))
s.Run("UpdateProvisionerJobLogsLength", s.Subtest(func(db database.Store, check *expects) {
j := dbgen.ProvisionerJob(s.T(), db, nil, database.ProvisionerJob{})
check.Args(database.UpdateProvisionerJobLogsLengthParams{
ID: j.ID,
LogsLength: 100,
}).Asserts(rbac.ResourceProvisionerJobs, policy.ActionUpdate)
}))
s.Run("UpdateProvisionerJobLogsOverflowed", s.Subtest(func(db database.Store, check *expects) {
j := dbgen.ProvisionerJob(s.T(), db, nil, database.ProvisionerJob{})
check.Args(database.UpdateProvisionerJobLogsOverflowedParams{
ID: j.ID,
LogsOverflowed: true,
}).Asserts(rbac.ResourceProvisionerJobs, policy.ActionUpdate)
}))
s.Run("InsertProvisionerJob", s.Subtest(func(db database.Store, check *expects) {
dbtestutil.DisableForeignKeysAndTriggers(s.T(), db)
check.Args(database.InsertProvisionerJobParams{
Expand Down
1 change: 1 addition & 0 deletionscoderd/database/dbfake/dbfake.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -179,6 +179,7 @@ func (b WorkspaceBuildBuilder) Do() WorkspaceResponse {
Input: payload,
Tags: map[string]string{},
TraceMetadata: pqtype.NullRawMessage{},
LogsOverflowed: false,
})
require.NoError(b.t, err, "insert job")

Expand Down
1 change: 1 addition & 0 deletionscoderd/database/dbgen/dbgen.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -775,6 +775,7 @@ func ProvisionerJob(t testing.TB, db database.Store, ps pubsub.Pubsub, orig data
Input: takeFirstSlice(orig.Input, []byte("{}")),
Tags: tags,
TraceMetadata: pqtype.NullRawMessage{},
LogsOverflowed: false,
})
require.NoError(t, err, "insert job")
if ps != nil {
Expand Down
14 changes: 14 additions & 0 deletionscoderd/database/dbmetrics/querymetrics.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

28 changes: 28 additions & 0 deletionscoderd/database/dbmock/dbmock.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

9 changes: 8 additions & 1 deletioncoderd/database/dump.sql
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

8 changes: 8 additions & 0 deletionscoderd/database/errors.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -79,3 +79,11 @@ func IsWorkspaceAgentLogsLimitError(err error) bool {

return false
}

func IsProvisionerJobLogsLimitError(err error) bool {
var pqErr *pq.Error
if errors.As(err, &pqErr) {
return pqErr.Constraint == "max_provisioner_logs_length" && pqErr.Table == "provisioner_jobs"
}
return false
}
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
ALTER TABLE provisioner_jobs DROP COLUMN logs_length;
ALTER TABLE provisioner_jobs DROP COLUMN logs_overflowed;
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
-- Add logs length tracking and overflow flag, similar to workspace agents
ALTER TABLE provisioner_jobs ADD COLUMN logs_length integer NOT NULL DEFAULT 0 CONSTRAINT max_provisioner_logs_length CHECK (logs_length <= 1048576);
ALTER TABLE provisioner_jobs ADD COLUMN logs_overflowed boolean NOT NULL DEFAULT false;

COMMENT ON COLUMN provisioner_jobs.logs_length IS 'Total length of provisioner logs';
COMMENT ON COLUMN provisioner_jobs.logs_overflowed IS 'Whether the provisioner logs overflowed in length';
4 changes: 4 additions & 0 deletionscoderd/database/models.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

2 changes: 2 additions & 0 deletionscoderd/database/querier.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

Loading
Loading

[8]ページ先頭

©2009-2025 Movatter.jp