- Notifications
You must be signed in to change notification settings - Fork1.2k
feat: add migrations and queries to support prebuilds#16891
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Merged
Uh oh!
There was an error while loading.Please reload this page.
Merged
Changes from1 commit
Commits
Show all changes
77 commits Select commitHold shift + click to select a range
300e80f add prebuilds system user database changes and associated changes
SasSwartb788237 optionally prevent system users from counting to user count
dannykopping8122595 appease the linter
dannykoppingbfb7c28 add unit test for system user behaviour
dannykopping6639167 reverting RBAC changes; not relevant here
dannykopping769ae1d removing unnecessary changes
dannykoppinge7e9c27 exclude system user db tests from non-linux OSs
dannykopping3936047 Rename prebuild system user reference
SasSwart8bdcafb ensure that users.IsSystem is not nullable
SasSwart412d198 feat: add migrations and queries to support prebuilds
SasSwart51773ec Simplify workspace_latest_build view
dannykopping23773c2 Revert test change
dannykoppingbc3ff44 make gen
dannykoppingbaa3076 refactor: add comments to SQL queries
evgeniy-scherbinaed14fb3 test: added get-presets-backoff test
evgeniy-scherbina3cc74fb refactor: add comment to SQL query
evgeniy-scherbinafc32154 refactor: add comments + improve tests
evgeniy-scherbinad7b4ec4 fix: bug in SQL
evgeniy-scherbinae8b53f7 test: minor changes to the test
evgeniy-scherbina9df6554 refactor: remove job_status from SQL query
evgeniy-scherbinaccc309e refactor: embed preset_prebuilds table into presets table
evgeniy-scherbinaee1f16a refactor: rename sql table
evgeniy-scherbinad040ddd refactor: remove unnecessary JOIN
evgeniy-scherbina83a6722 refactor: remove unnecessary JOIN
evgeniy-scherbinacd70710 refactor: use INNER JOIN for consistency
evgeniy-scherbina97cc4ff refactor: simplify GetPresetsBackoff SQL Query
evgeniy-scherbina4d59039 Revert "refactor: simplify GetPresetsBackoff SQL Query"
evgeniy-scherbina205d6af refactor: improve GetPresetsBackoff query
evgeniy-scherbinae489e1b Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbina1b29686 Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbina20470e4 fix: bump migration numbers
evgeniy-scherbina7b9c8ce test: remove deprecated test
evgeniy-scherbinae189a0b fix: fix linter
evgeniy-scherbina692c0e5 fix: fix 000310_prebuilds.down migration
evgeniy-scherbinaf747db0 fix: fix fixture migration
evgeniy-scherbina3166a42 fix: fix get-presets-backoff test
evgeniy-scherbinaaa6b490 fix: fix linter
evgeniy-scherbinabc4e7d2 fix: fix linter
evgeniy-scherbinaf167b92 correctly select for the latest built with a preset in latest_prebuil…
SasSwart8fd34ab Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwart7a8ec49 Properly label and filter metrics for prebuilds
SasSwarta64d661 test: fix db tests
evgeniy-scherbinac787cd2 test: added tests for workspaces with multiple agents
evgeniy-scherbinabd38603 refactor: avoid code duplication
evgeniy-scherbina097f9c3 clarify query clause
SasSwart4cfdd6f tidy up dbauthz_test.go
SasSwart4a34d52 refactor: remove * usage from prebuilds.sql queries
evgeniy-scherbina8d9cd45 refactor: remove * usage from prebuilds views
evgeniy-scherbinaf870d7e refactor: join wlb with pj
evgeniy-scherbina18ad931 refactor: Rename SQL query
evgeniy-scherbina4667171 Added comments for SQL query
evgeniy-scherbinaa26c094 refactor: fix down migration
evgeniy-scherbina6ed4121 Merge remote-tracking branch 'origin/main' into prebuilds-db
SasSwart2312f41 renumber migrations
SasSwart5150a5c refactor: clarify comment for SQL query
evgeniy-scherbinabff34ea refactor: fix indentations
evgeniy-scherbinaef462b6 refactor: rename helper func in test package
evgeniy-scherbinadc45165 refactor: database level tests
evgeniy-scherbina9c8a352 refactor: database level tests
evgeniy-scherbinaeb80919 refactor: helper funcs in db-level tests
evgeniy-scherbina0b2bbee refactor: minor improvement in SQL query
evgeniy-scherbina3a97bf6 refactor: rename SQL queries
evgeniy-scherbina2eeb884 refactor: rename SQL queries
evgeniy-scherbina73f99e8 refactor: rename fields in SQL query
evgeniy-scherbina113e12b fix: improve rbac policies
evgeniy-scherbina217cae0 fix: minor fix in filtered_builds CTE
evgeniy-scherbinab0bf220 fix: formatting
evgeniy-scherbinaff72a00 refactor: minor refactoring
evgeniy-scherbinaa007d4a fix: handle presets with the same tv.id and name
evgeniy-scherbinaea9c53b fix: redefine RBAC permissions for prebuilds
evgeniy-scherbina66d44ed Merge remote-tracking branch 'origin/main' into prebuilds-db
evgeniy-scherbina29e121f fix: fix migration numbers
evgeniy-scherbinaf64754f test: fix dbmem tests
evgeniy-scherbina7e02397 test: fix dbmem tests
evgeniy-scherbina55d9827 fix: linter
evgeniy-scherbina866454b fix: linter
evgeniy-scherbina4fa959a fix: linter
evgeniy-scherbinaFile filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading.Please reload this page.
Diff view
Diff view
refactor: remove job_status from SQL query
- Loading branch information
Uh oh!
There was an error while loading.Please reload this page.
commit9df655403f70d984f37238344490973ac164ef01
There are no files selected for viewing
11 changes: 0 additions & 11 deletionscoderd/database/querier_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -3615,7 +3615,6 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := backoffs[0] | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmplV1.preset.ID) | ||
| require.Equal(t, int32(1), backoff.NumFailed) | ||
spikecurtis marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| }) | ||
| @@ -3644,7 +3643,6 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := backoffs[0] | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmplV1.preset.ID) | ||
| require.Equal(t, int32(3), backoff.NumFailed) | ||
| }) | ||
| @@ -3676,7 +3674,6 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := backoffs[0] | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmplV2.preset.ID) | ||
| require.Equal(t, int32(2), backoff.NumFailed) | ||
| }) | ||
| @@ -3708,14 +3705,12 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := findBackoffByTmplVersionId(backoffs, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl1V1.preset.ID) | ||
| require.Equal(t, int32(1), backoff.NumFailed) | ||
| } | ||
| { | ||
| backoff := findBackoffByTmplVersionId(backoffs, tmpl2.ActiveVersionID) | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl2.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl2V1.preset.ID) | ||
| require.Equal(t, int32(1), backoff.NumFailed) | ||
| } | ||
| }) | ||
| @@ -3758,21 +3753,18 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := findBackoffByTmplVersionId(backoffs, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl1V1.preset.ID) | ||
| require.Equal(t, int32(1), backoff.NumFailed) | ||
| } | ||
| { | ||
| backoff := findBackoffByTmplVersionId(backoffs, tmpl2.ActiveVersionID) | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl2.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl2V1.preset.ID) | ||
| require.Equal(t, int32(2), backoff.NumFailed) | ||
| } | ||
| { | ||
| backoff := findBackoffByTmplVersionId(backoffs, tmpl3.ActiveVersionID) | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl3.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl3V2.preset.ID) | ||
| require.Equal(t, int32(3), backoff.NumFailed) | ||
| } | ||
| }) | ||
| @@ -3931,7 +3923,6 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := backoffs[0] | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl1V1.preset.ID) | ||
| require.Equal(t, int32(1), backoff.NumFailed) | ||
| } | ||
| }) | ||
| @@ -3982,7 +3973,6 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := backoffs[0] | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl1V1.preset.ID) | ||
| require.Equal(t, int32(2), backoff.NumFailed) | ||
| } | ||
| }) | ||
| @@ -4037,7 +4027,6 @@ func TestGetPresetsBackoff(t *testing.T) { | ||
| backoff := backoffs[0] | ||
| require.Equal(t, backoff.TemplateVersionID, tmpl1.ActiveVersionID) | ||
| require.Equal(t, backoff.PresetID, tmpl1V1.preset.ID) | ||
| require.Equal(t, int32(5), backoff.NumFailed) | ||
| // make sure LastBuildAt is equal to latest failed build timestamp | ||
| require.Equal(t, 0, now.Compare(backoff.LastBuildAt.(time.Time))) | ||
17 changes: 7 additions & 10 deletionscoderd/database/queries.sql.go
Some generated files are not rendered by default. Learn more abouthow customized files appear on GitHub.
Oops, something went wrong.
Uh oh!
There was an error while loading.Please reload this page.
7 changes: 3 additions & 4 deletionscoderd/database/queries/prebuilds.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -86,14 +86,13 @@ WITH filtered_builds AS ( | ||
| GROUP BY preset_id) | ||
| SELECT tsb.template_version_id, | ||
| tsb.preset_id, | ||
| COALESCE(fc.num_failed, 0)::int AS num_failed, | ||
| MAX(tsb.created_at::timestamptz) AS last_build_at | ||
| FROM time_sorted_builds tsb | ||
| LEFT JOIN failed_count fc ON fc.preset_id = tsb.preset_id | ||
| WHERE tsb.rn <= tsb.desired_instances -- Fetch the last N builds, where N is the number of desired instances; if any fail, we backoff | ||
| AND tsb.job_status = 'failed'::provisioner_job_status | ||
| GROUP BY tsb.template_version_id, tsb.preset_id, fc.num_failed; | ||
evgeniy-scherbina marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading.Please reload this page. | ||
| -- name: ClaimPrebuild :one | ||
| UPDATE workspaces w | ||
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.