Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat(coderd/database): keep only 1 day ofworkspace_agent_stats after rollup#12674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
mafredri merged 2 commits intomainfrommafredri/keep-only-1-day-of-agent-stats
Apr 22, 2024

Conversation

mafredri
Copy link
Member

@mafredrimafredri commentedMar 19, 2024
edited
Loading

This will free up many gigabytes of data for many customers, more for those that have a lot of workspaces running daily.

Refs#12122

This was referencedMar 19, 2024
@mafredriGraphite App
Copy link
MemberAuthor

mafredri commentedMar 19, 2024
edited
Loading

This stack of pull requests is managed by Graphite.Learn more about stacking.

Join@mafredri and the rest of your teammates onGraphiteGraphite

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is there a maximum number of rows that could be deleted here?
Would it make sense to add a limit on the number of rows deleted?
My thinking is that we could choose a reasonably large upper limit and hopefully converge after a few iterations.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Not a bad idea. Limiting would only be relevant for the first time(s) this query runs, after that there's no point but if we want to avoid the initial performance penalty of doing a huge delete, that would be sensible. There's no limit for deletes though so we could rely on selecting the min(created_at) and only delete 1 (or N days) of data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I thinking deleting the oldest day of data every interval will probably work fine here. The magnitude of that data should scale with the size of the deployment. We'll have to assume that the database is correctly sized to handle that level of deletes, but if not that is likely a sizing issue with the database itself in relation to the rest of the deployment.

mafredri reacted with thumbs up emoji
@mafredrimafredriforce-pushed themafredri/fix-insights-tests-after-refactor branch from754fdd5 to7128953CompareMarch 20, 2024 10:25
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from9286a28 to95cdacaCompareMarch 20, 2024 10:25
created_at < (
SELECT
COALESCE(
-- When generating initial template usage stats, all the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This change may be little too soon to drop. If customers identify problems with the release, they will decide to roll back to the previous version. Boom! Noworkspace_agent_stats anymore.

Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I agree, I made this a separate PR so we can defer it for later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Rollbacks require db backups at present. We just do not support downgrades at present.

And not supported like, you can't even really do it. You have to run themigration downgrade with the latest version, then rollback. We have 0 documentation for this, so no customer should do it.

mafredri reacted with thumbs up emoji
@mafredrimafredri changed the titlefeat(coderd/database): keep only 1 day ofworkspace_agent_stats after rollupfeat(coderd/database): DO NOT MERGE: keep only 1 day ofworkspace_agent_stats after rollupMar 20, 2024
@mafredrimafredriforce-pushed themafredri/fix-insights-tests-after-refactor branch from7128953 tob915c4cCompareMarch 20, 2024 12:37
@mafredrimafredri changed the base branch frommafredri/fix-insights-tests-after-refactor tomafredri/add-sftp-to-insights-appsMarch 20, 2024 12:37
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from95cdaca to8a2d9f8CompareMarch 20, 2024 12:37
@mafredrimafredriforce-pushed themafredri/add-sftp-to-insights-apps branch from1477d70 tod982c5bCompareMarch 20, 2024 14:53
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from8a2d9f8 toad2b3e7CompareMarch 20, 2024 14:53
@mafredrimafredriforce-pushed themafredri/add-sftp-to-insights-apps branch fromd982c5b to71cd6ffCompareMarch 20, 2024 16:03
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch fromad2b3e7 to8442baaCompareMarch 20, 2024 16:03
@mafredrimafredriforce-pushed themafredri/add-sftp-to-insights-apps branch from71cd6ff tofb7f72cCompareMarch 20, 2024 19:04
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from8442baa to563ad78CompareMarch 20, 2024 19:04
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from6c31bbb to364b21eCompareMarch 25, 2024 12:53
@mafredrimafredriforce-pushed themafredri/add-sftp-to-insights-apps branch from140017a toeb01ba5CompareMarch 25, 2024 12:56
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from364b21e to3e356f2CompareMarch 25, 2024 12:56
@mafredrimafredriforce-pushed themafredri/add-sftp-to-insights-apps branch fromeb01ba5 to1ef1513CompareMarch 25, 2024 13:18
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from3e356f2 toccb914cCompareMarch 25, 2024 13:18
@mafredrimafredriforce-pushed themafredri/add-sftp-to-insights-apps branch from1ef1513 toe720b91CompareMarch 27, 2024 12:01
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch fromccb914c to1489cd6CompareMarch 27, 2024 12:01
Base automatically changed frommafredri/add-sftp-to-insights-apps tomainMarch 27, 2024 12:09
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from1489cd6 to859f9d4CompareMarch 28, 2024 11:02
@github-actionsgithub-actionsbot added the staleThis issue is like stale bread. labelApr 5, 2024
@mafredrimafredri reopened thisApr 8, 2024
@github-actionsgithub-actionsbot removed the staleThis issue is like stale bread. labelApr 9, 2024
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from859f9d4 to89ecbf2CompareApril 9, 2024 12:20
@mafredrimafredri marked this pull request as ready for reviewApril 9, 2024 12:21
@mafredri
Copy link
MemberAuthor

@johnstcn I've added the batch deleting in case you want to take a look.

@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch from89ecbf2 tob5b95d2CompareApril 9, 2024 12:22
@mafredrimafredri changed the titlefeat(coderd/database): DO NOT MERGE: keep only 1 day ofworkspace_agent_stats after rollupfeat(coderd/database): keep only 1 day ofworkspace_agent_stats after rollupApr 9, 2024
Copy link
Member

@johnstcnjohnstcn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

just needs a make gen

mafredri reacted with laugh emoji
@mafredrimafredriforce-pushed themafredri/keep-only-1-day-of-agent-stats branch fromb5b95d2 to18b45c2CompareApril 9, 2024 13:27
@github-actionsgithub-actionsbot added the staleThis issue is like stale bread. labelApr 17, 2024
@mafredrimafredri reopened thisApr 22, 2024
@mafredrimafredri removed the staleThis issue is like stale bread. labelApr 22, 2024
Copy link
Member

@mtojekmtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

👍

@mafredrimafredri merged commite17e8aa intomainApr 22, 2024
@mafredrimafredri deleted the mafredri/keep-only-1-day-of-agent-stats branchApril 22, 2024 10:11
@github-actionsgithub-actionsbot locked and limited conversation to collaboratorsApr 22, 2024
Sign up for freeto subscribe to this conversation on GitHub. Already have an account?Sign in.
Reviewers

@EmyrkEmyrkEmyrk left review comments

@johnstcnjohnstcnjohnstcn approved these changes

@mtojekmtojekmtojek approved these changes

Assignees

@mafredrimafredri

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

4 participants
@mafredri@johnstcn@Emyrk@mtojek

[8]ページ先頭

©2009-2025 Movatter.jp