- Notifications
You must be signed in to change notification settings - Fork928
feat(coderd/database): keep only 1 day ofworkspace_agent_stats
after rollup#12674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
mafredri commentedMar 19, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
77ba7dc
to754fdd5
Compare5024af0
to9286a28
CompareThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Is there a maximum number of rows that could be deleted here?
Would it make sense to add a limit on the number of rows deleted?
My thinking is that we could choose a reasonably large upper limit and hopefully converge after a few iterations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Not a bad idea. Limiting would only be relevant for the first time(s) this query runs, after that there's no point but if we want to avoid the initial performance penalty of doing a huge delete, that would be sensible. There's no limit for deletes though so we could rely on selecting the min(created_at) and only delete 1 (or N days) of data.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I thinking deleting the oldest day of data every interval will probably work fine here. The magnitude of that data should scale with the size of the deployment. We'll have to assume that the database is correctly sized to handle that level of deletes, but if not that is likely a sizing issue with the database itself in relation to the rest of the deployment.
754fdd5
to7128953
Compare9286a28
to95cdaca
Comparecreated_at < ( | ||
SELECT | ||
COALESCE( | ||
-- When generating initial template usage stats, all the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This change may be little too soon to drop. If customers identify problems with the release, they will decide to roll back to the previous version. Boom! Noworkspace_agent_stats
anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I agree, I made this a separate PR so we can defer it for later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Rollbacks require db backups at present. We just do not support downgrades at present.
And not supported like, you can't even really do it. You have to run themigration downgrade
with the latest version, then rollback. We have 0 documentation for this, so no customer should do it.
workspace_agent_stats
after rollupworkspace_agent_stats
after rollup7128953
tob915c4c
Compare95cdaca
to8a2d9f8
Compare1477d70
tod982c5b
Compare8a2d9f8
toad2b3e7
Compared982c5b
to71cd6ff
Comparead2b3e7
to8442baa
Compare71cd6ff
tofb7f72c
Compare8442baa
to563ad78
Compare6c31bbb
to364b21e
Compare140017a
toeb01ba5
Compare364b21e
to3e356f2
Compareeb01ba5
to1ef1513
Compare3e356f2
toccb914c
Compare1ef1513
toe720b91
Compareccb914c
to1489cd6
Compare1489cd6
to859f9d4
Compare859f9d4
to89ecbf2
Compare@johnstcn I've added the batch deleting in case you want to take a look. |
89ecbf2
tob5b95d2
Compareworkspace_agent_stats
after rollupworkspace_agent_stats
after rollupThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
just needs a make gen
b5b95d2
to18b45c2
CompareThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
👍
Uh oh!
There was an error while loading.Please reload this page.
This will free up many gigabytes of data for many customers, more for those that have a lot of workspaces running daily.
Refs#12122