Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: persist prebuild definitions on template import#16951

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
SasSwart merged 88 commits intomainfromjjs/insert-prebuilds
Apr 7, 2025
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
Show all changes
88 commits
Select commitHold shift + click to select a range
5ecc277
add prebuilds system user database changes and associated changes
SasSwartMar 12, 2025
bc5f4f4
optionally prevent system users from counting to user count
dannykoppingMar 13, 2025
48c5372
appease the linter
dannykoppingMar 13, 2025
b16d126
add unit test for system user behaviour
dannykoppingMar 13, 2025
2c25542
reverting RBAC changes; not relevant here
dannykoppingMar 13, 2025
8e491d8
removing unnecessary changes
dannykoppingMar 13, 2025
514fdbf
exclude system user db tests from non-linux OSs
dannykoppingMar 13, 2025
390a1fd
feat: add migrations and queries to support prebuilds
SasSwartMar 12, 2025
07e9613
Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwartMar 17, 2025
a07c2b2
feat: persist prebuild definitions on template import
SasSwartMar 17, 2025
300e80f
add prebuilds system user database changes and associated changes
SasSwartMar 12, 2025
b788237
optionally prevent system users from counting to user count
dannykoppingMar 13, 2025
8122595
appease the linter
dannykoppingMar 13, 2025
bfb7c28
add unit test for system user behaviour
dannykoppingMar 13, 2025
6639167
reverting RBAC changes; not relevant here
dannykoppingMar 13, 2025
769ae1d
removing unnecessary changes
dannykoppingMar 13, 2025
e7e9c27
exclude system user db tests from non-linux OSs
dannykoppingMar 13, 2025
3936047
Rename prebuild system user reference
SasSwartMar 17, 2025
8bdcafb
ensure that users.IsSystem is not nullable
SasSwartMar 17, 2025
412d198
feat: add migrations and queries to support prebuilds
SasSwartMar 12, 2025
51773ec
Simplify workspace_latest_build view
dannykoppingMar 14, 2025
23773c2
Revert test change
dannykoppingMar 17, 2025
bc3ff44
make gen
dannykoppingMar 17, 2025
baa3076
refactor: add comments to SQL queries
evgeniy-scherbinaMar 19, 2025
ed14fb3
test: added get-presets-backoff test
evgeniy-scherbinaMar 20, 2025
3cc74fb
refactor: add comment to SQL query
evgeniy-scherbinaMar 20, 2025
fc32154
refactor: add comments + improve tests
evgeniy-scherbinaMar 21, 2025
d7b4ec4
fix: bug in SQL
evgeniy-scherbinaMar 21, 2025
e8b53f7
test: minor changes to the test
evgeniy-scherbinaMar 21, 2025
9df6554
refactor: remove job_status from SQL query
evgeniy-scherbinaMar 21, 2025
ccc309e
refactor: embed preset_prebuilds table into presets table
evgeniy-scherbinaMar 21, 2025
ee1f16a
refactor: rename sql table
evgeniy-scherbinaMar 21, 2025
d040ddd
refactor: remove unnecessary JOIN
evgeniy-scherbinaMar 23, 2025
83a6722
refactor: remove unnecessary JOIN
evgeniy-scherbinaMar 23, 2025
cd70710
refactor: use INNER JOIN for consistency
evgeniy-scherbinaMar 23, 2025
0f3bda0
Merge remote-tracking branch 'origin/prebuilds-db' into jjs/insert-pr…
SasSwartMar 24, 2025
a7c7cd2
make lint
SasSwartMar 24, 2025
97cc4ff
refactor: simplify GetPresetsBackoff SQL Query
evgeniy-scherbinaMar 24, 2025
4d59039
Revert "refactor: simplify GetPresetsBackoff SQL Query"
evgeniy-scherbinaMar 24, 2025
205d6af
refactor: improve GetPresetsBackoff query
evgeniy-scherbinaMar 24, 2025
e489e1b
Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbinaMar 25, 2025
1b29686
Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbinaMar 25, 2025
20470e4
fix: bump migration numbers
evgeniy-scherbinaMar 25, 2025
6fc1889
Merge remote-tracking branch 'origin/main' into jjs/insert-prebuilds
SasSwartMar 25, 2025
7b9c8ce
test: remove deprecated test
evgeniy-scherbinaMar 25, 2025
e189a0b
fix: fix linter
evgeniy-scherbinaMar 25, 2025
692c0e5
fix: fix 000310_prebuilds.down migration
evgeniy-scherbinaMar 25, 2025
f747db0
fix: fix fixture migration
evgeniy-scherbinaMar 25, 2025
3166a42
fix: fix get-presets-backoff test
evgeniy-scherbinaMar 25, 2025
aa6b490
fix: fix linter
evgeniy-scherbinaMar 25, 2025
bc4e7d2
fix: fix linter
evgeniy-scherbinaMar 25, 2025
f167b92
correctly select for the latest built with a preset in latest_prebuil…
SasSwartMar 26, 2025
8fd34ab
Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwartMar 26, 2025
7a8ec49
Properly label and filter metrics for prebuilds
SasSwartMar 26, 2025
a64d661
test: fix db tests
evgeniy-scherbinaMar 27, 2025
865998b
Merge branch 'prebuilds-db' into jjs/insert-prebuilds
SasSwartMar 27, 2025
c787cd2
test: added tests for workspaces with multiple agents
evgeniy-scherbinaMar 27, 2025
bd38603
refactor: avoid code duplication
evgeniy-scherbinaMar 27, 2025
097f9c3
clarify query clause
SasSwartMar 27, 2025
4cfdd6f
tidy up dbauthz_test.go
SasSwartMar 27, 2025
4a34d52
refactor: remove * usage from prebuilds.sql queries
evgeniy-scherbinaMar 27, 2025
8d9cd45
refactor: remove * usage from prebuilds views
evgeniy-scherbinaMar 27, 2025
f870d7e
refactor: join wlb with pj
evgeniy-scherbinaMar 27, 2025
18ad931
refactor: Rename SQL query
evgeniy-scherbinaMar 27, 2025
4667171
Added comments for SQL query
evgeniy-scherbinaMar 27, 2025
a26c094
refactor: fix down migration
evgeniy-scherbinaMar 27, 2025
bf4ab53
make lint
SasSwartMar 28, 2025
a84b1bb
Merge remote-tracking branch 'origin/main' into jjs/insert-prebuilds
SasSwartMar 28, 2025
6ed4121
Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwartMar 28, 2025
e8b1502
Merge branch 'prebuilds-db' into jjs/insert-prebuilds
SasSwartMar 28, 2025
2312f41
renumber migrations
SasSwartMar 28, 2025
4540a55
add tests for prebuilds in the provisionerdserver
SasSwartMar 28, 2025
5c41ba9
Merge branch 'prebuilds-db' into jjs/insert-prebuilds
SasSwartMar 28, 2025
d09b757
fix indent
SasSwartMar 28, 2025
61e86f6
make more use of dbgen
SasSwartMar 28, 2025
1419df0
update dbmemt
SasSwartMar 28, 2025
6589221
Add tests
SasSwartMar 28, 2025
b15b97a
Merge origin/main
SasSwartApr 3, 2025
dd656a7
Merge remote-tracking branch 'origin/main' into jjs/insert-prebuilds
SasSwartApr 3, 2025
508b244
fix tests
SasSwartApr 3, 2025
e1f585d
remove duplicate migrations fixture
SasSwartApr 3, 2025
7d4f1b9
clean up go.mod slightly
SasSwartApr 3, 2025
43f82b9
update dependency on terraform-provider-coder
SasSwartApr 7, 2025
5d8de71
simplify query
SasSwartApr 7, 2025
a87933a
Correct the prebuilds type
SasSwartApr 7, 2025
798cfa1
fix prebuild decoding
SasSwartApr 7, 2025
dc87f45
make fmt
SasSwartApr 7, 2025
7c17fcd
update our dependency on terraform-provider-coder
SasSwartApr 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 13 additions & 3 deletionscoderd/database/dbauthz/dbauthz.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -2187,14 +2187,24 @@ func (q *querier) GetPresetByWorkspaceBuildID(ctx context.Context, workspaceID u
return q.db.GetPresetByWorkspaceBuildID(ctx, workspaceID)
}

func (q *querier)GetPresetParametersByTemplateVersionID(ctx context.Context,templateVersionID uuid.UUID) ([]database.TemplateVersionPresetParameter, error) {
func (q *querier)GetPresetParametersByPresetID(ctx context.Context,presetID uuid.UUID) ([]database.TemplateVersionPresetParameter, error) {
// An actor can read template version presets if they can read the related template version.
_, err := q.GetTemplateVersionByID(ctx, templateVersionID)
_, err := q.GetPresetByID(ctx, presetID)
if err != nil {
return nil, err
}

return q.db.GetPresetParametersByPresetID(ctx, presetID)
}

func (q *querier) GetPresetParametersByTemplateVersionID(ctx context.Context, args uuid.UUID) ([]database.TemplateVersionPresetParameter, error) {
// An actor can read template version presets if they can read the related template version.
_, err := q.GetTemplateVersionByID(ctx, args)
if err != nil {
return nil, err
}

return q.db.GetPresetParametersByTemplateVersionID(ctx,templateVersionID)
return q.db.GetPresetParametersByTemplateVersionID(ctx,args)
}

func (q *querier) GetPresetsBackoff(ctx context.Context, lookback time.Time) ([]database.GetPresetsBackoffRow, error) {
Expand Down
220 changes: 124 additions & 96 deletionscoderd/database/dbauthz/dbauthz_test.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -182,7 +182,6 @@ func TestDBAuthzRecursive(t *testing.T) {
method.Name == "PGLocks" {
continue
}
// Log the name of the last method, so if there is a panic, it is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Intentional deletion?

// easy to know which method failed.
// t.Log(method.Name)
// Call the function. Any infinite recursion will stack overflow.
Expand DownExpand Up@@ -969,8 +968,7 @@ func (s *MethodTestSuite) TestOrganization() {
TemplateVersionID: workspaceBuild.TemplateVersionID,
Name: "test",
}
preset, err := db.InsertPreset(context.Background(), insertPresetParams)
require.NoError(s.T(), err)
preset := dbgen.Preset(s.T(), db, insertPresetParams)
insertPresetParametersParams := database.InsertPresetParametersParams{
TemplateVersionPresetID: preset.ID,
Names: []string{"test"},
Expand DownExpand Up@@ -1027,8 +1025,8 @@ func (s *MethodTestSuite) TestOrganization() {
})

check.Args(database.OrganizationMembersParams{
OrganizationID:uuid.UUID{},
UserID:uuid.UUID{},
OrganizationID:o.ID,
UserID:u.ID,
}).Asserts(
mem, policy.ActionRead,
)
Expand DownExpand Up@@ -3906,96 +3904,6 @@ func (s *MethodTestSuite) TestSystemFunctions() {
ErrorsWithInMemDB(sql.ErrNoRows).
Returns([]database.ParameterSchema{})
}))
s.Run("GetPresetByWorkspaceBuildID", s.Subtest(func(db database.Store, check *expects) {
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})
preset, err := db.InsertPreset(context.Background(), database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)
workspace := dbgen.Workspace(s.T(), db, database.WorkspaceTable{
OrganizationID: org.ID,
OwnerID: user.ID,
TemplateID: template.ID,
})
job := dbgen.ProvisionerJob(s.T(), db, nil, database.ProvisionerJob{
OrganizationID: org.ID,
})
workspaceBuild := dbgen.WorkspaceBuild(s.T(), db, database.WorkspaceBuild{
WorkspaceID: workspace.ID,
TemplateVersionID: templateVersion.ID,
TemplateVersionPresetID: uuid.NullUUID{UUID: preset.ID, Valid: true},
InitiatorID: user.ID,
JobID: job.ID,
})
_, err = db.GetPresetByWorkspaceBuildID(context.Background(), workspaceBuild.ID)
require.NoError(s.T(), err)
check.Args(workspaceBuild.ID).Asserts(rbac.ResourceTemplate, policy.ActionRead)
}))
s.Run("GetPresetParametersByTemplateVersionID", s.Subtest(func(db database.Store, check *expects) {
ctx := context.Background()
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})
preset, err := db.InsertPreset(ctx, database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)
_, err = db.InsertPresetParameters(ctx, database.InsertPresetParametersParams{
TemplateVersionPresetID: preset.ID,
Names: []string{"test"},
Values: []string{"test"},
})
require.NoError(s.T(), err)
presetParameters, err := db.GetPresetParametersByTemplateVersionID(ctx, templateVersion.ID)
require.NoError(s.T(), err)

check.Args(templateVersion.ID).Asserts(template.RBACObject(), policy.ActionRead).Returns(presetParameters)
}))
s.Run("GetPresetsByTemplateVersionID", s.Subtest(func(db database.Store, check *expects) {
ctx := context.Background()
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})

_, err := db.InsertPreset(ctx, database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)

presets, err := db.GetPresetsByTemplateVersionID(ctx, templateVersion.ID)
require.NoError(s.T(), err)

check.Args(templateVersion.ID).Asserts(template.RBACObject(), policy.ActionRead).Returns(presets)
}))
s.Run("GetWorkspaceAppsByAgentIDs", s.Subtest(func(db database.Store, check *expects) {
dbtestutil.DisableForeignKeysAndTriggers(s.T(), db)
aWs := dbgen.Workspace(s.T(), db, database.WorkspaceTable{})
Expand DownExpand Up@@ -4839,6 +4747,125 @@ func (s *MethodTestSuite) TestNotifications() {
}

func (s *MethodTestSuite) TestPrebuilds() {
s.Run("GetPresetByWorkspaceBuildID", s.Subtest(func(db database.Store, check *expects) {
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})
preset, err := db.InsertPreset(context.Background(), database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)
workspace := dbgen.Workspace(s.T(), db, database.WorkspaceTable{
OrganizationID: org.ID,
OwnerID: user.ID,
TemplateID: template.ID,
})
job := dbgen.ProvisionerJob(s.T(), db, nil, database.ProvisionerJob{
OrganizationID: org.ID,
})
workspaceBuild := dbgen.WorkspaceBuild(s.T(), db, database.WorkspaceBuild{
WorkspaceID: workspace.ID,
TemplateVersionID: templateVersion.ID,
TemplateVersionPresetID: uuid.NullUUID{UUID: preset.ID, Valid: true},
InitiatorID: user.ID,
JobID: job.ID,
})
_, err = db.GetPresetByWorkspaceBuildID(context.Background(), workspaceBuild.ID)
require.NoError(s.T(), err)
check.Args(workspaceBuild.ID).Asserts(rbac.ResourceTemplate, policy.ActionRead)
}))
s.Run("GetPresetParametersByTemplateVersionID", s.Subtest(func(db database.Store, check *expects) {
ctx := context.Background()
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})
preset, err := db.InsertPreset(ctx, database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)
insertedParameters, err := db.InsertPresetParameters(ctx, database.InsertPresetParametersParams{
TemplateVersionPresetID: preset.ID,
Names: []string{"test"},
Values: []string{"test"},
})
require.NoError(s.T(), err)
check.
Args(templateVersion.ID).
Asserts(template.RBACObject(), policy.ActionRead).
Returns(insertedParameters)
}))
s.Run("GetPresetParametersByPresetID", s.Subtest(func(db database.Store, check *expects) {
ctx := context.Background()
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})
preset, err := db.InsertPreset(ctx, database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)
insertedParameters, err := db.InsertPresetParameters(ctx, database.InsertPresetParametersParams{
TemplateVersionPresetID: preset.ID,
Names: []string{"test"},
Values: []string{"test"},
})
require.NoError(s.T(), err)
check.
Args(preset.ID).
Asserts(template.RBACObject(), policy.ActionRead).
Returns(insertedParameters)
}))
s.Run("GetPresetsByTemplateVersionID", s.Subtest(func(db database.Store, check *expects) {
ctx := context.Background()
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
template := dbgen.Template(s.T(), db, database.Template{
CreatedBy: user.ID,
OrganizationID: org.ID,
})
templateVersion := dbgen.TemplateVersion(s.T(), db, database.TemplateVersion{
TemplateID: uuid.NullUUID{UUID: template.ID, Valid: true},
OrganizationID: org.ID,
CreatedBy: user.ID,
})

_, err := db.InsertPreset(ctx, database.InsertPresetParams{
TemplateVersionID: templateVersion.ID,
Name: "test",
})
require.NoError(s.T(), err)

presets, err := db.GetPresetsByTemplateVersionID(ctx, templateVersion.ID)
require.NoError(s.T(), err)

check.Args(templateVersion.ID).Asserts(template.RBACObject(), policy.ActionRead).Returns(presets)
}))
s.Run("ClaimPrebuiltWorkspace", s.Subtest(func(db database.Store, check *expects) {
org := dbgen.Organization(s.T(), db, database.Organization{})
user := dbgen.User(s.T(), db, database.User{})
Expand DownExpand Up@@ -4923,7 +4950,8 @@ func (s *MethodTestSuite) TestPrebuilds() {
UUID: template.ID,
Valid: true,
},
OrganizationID: org.ID,
InvalidateAfterSecs: preset.InvalidateAfterSecs,
OrganizationID: org.ID,
})
}))
}
Expand Down
21 changes: 20 additions & 1 deletioncoderd/database/dbmem/dbmem.go
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -4275,6 +4275,21 @@ func (q *FakeQuerier) GetPresetByWorkspaceBuildID(_ context.Context, workspaceBu
return database.TemplateVersionPreset{}, sql.ErrNoRows
}

func (q *FakeQuerier) GetPresetParametersByPresetID(_ context.Context, presetID uuid.UUID) ([]database.TemplateVersionPresetParameter, error) {
q.mutex.RLock()
defer q.mutex.RUnlock()

parameters := make([]database.TemplateVersionPresetParameter, 0)
for _, parameter := range q.presetParameters {
if parameter.TemplateVersionPresetID != presetID {
continue
}
parameters = append(parameters, parameter)
}

return parameters, nil
}

func (q *FakeQuerier) GetPresetParametersByTemplateVersionID(_ context.Context, templateVersionID uuid.UUID) ([]database.TemplateVersionPresetParameter, error) {
q.mutex.RLock()
defer q.mutex.RUnlock()
Expand All@@ -4293,7 +4308,6 @@ func (q *FakeQuerier) GetPresetParametersByTemplateVersionID(_ context.Context,
continue
}
parameters = append(parameters, parameter)
break
}
}

Expand DownExpand Up@@ -8854,6 +8868,11 @@ func (q *FakeQuerier) InsertPreset(_ context.Context, arg database.InsertPresetP
TemplateVersionID: arg.TemplateVersionID,
Name: arg.Name,
CreatedAt: arg.CreatedAt,
DesiredInstances: arg.DesiredInstances,
InvalidateAfterSecs: sql.NullInt32{
Int32: 0,
Valid: true,
},
}
q.presets = append(q.presets, preset)
return preset, nil
Expand Down
7 changes: 7 additions & 0 deletionscoderd/database/dbmetrics/querymetrics.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

15 changes: 15 additions & 0 deletionscoderd/database/dbmock/dbmock.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

1 change: 1 addition & 0 deletionscoderd/database/querier.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.

Loading
Loading

[8]ページ先頭

©2009-2025 Movatter.jp