Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

create db vacuum service to periodically delete old prefect resources#19280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
levzem wants to merge1 commit intoPrefectHQ:main
base:main
Choose a base branch
Loading
fromlevzem:lev/vacuum

Conversation

@levzem
Copy link
Contributor

Closes#16054

This PR adds a new LoopService called DBVacuum that allows for the automatic clean up of Prefect resources that are older than a configurable retention period. This is to help keep the Prefect DB from overflowing with old flow runs/task runs/etc.

The DBVacuum deletes the following:

  1. flow runs
  2. associated logs
  3. associated artifacts

Task runs are handled by the DELETE CASCADE rules setup on the flow run table.

The DBVacuum queries a limited amount of parent flow runs (those that do not have a parent_task_run_id) that are older than the retention and deletes them. It then deletes logs that no longer have any flow runs associated with them and finally performs the same deletion for artifacts.

The deletion of a parent flow run will delete child task runs and set their child flow runs parent_task_run_id to null, which will get picked up on the next round of the loop. This gives a guarantee that eventually all flow runs and their sub flow runs will be deleted (along with the associated logs and artifacts).

Checklist

  • This pull request references any related issue by including "closes<link to issue>"
    • If no issue exists and your change is not a small fix, pleasecreate an issue first.
  • If this pull request adds new functionality, it includes unit tests that cover the changes
  • If this pull request removes docs files, it includes redirect settings inmint.json.
  • If this pull request adds functions or classes, it includes helpful docstrings.

@github-actionsgithub-actionsbot added the performanceRelated to an optimization or performance improvement labelOct 24, 2025
@codspeed-hq
Copy link

codspeed-hqbot commentedOct 24, 2025
edited
Loading

CodSpeed Performance Report

Merging#19280 willnot alter performance

Comparinglevzem:lev/vacuum (7225337) withmain (7fcfb7e)

Summary

✅ 2 untouched

@levzem
Copy link
ContributorAuthor

@stephane-additive fyi

@zzstoatzz
Copy link
Collaborator

hi@levzem - i appreciate the PR and have a couple initial thoughts:

  • something like this should exist
  • we should not introduce it as default on, as that would cause significant changes in behavior for server operators
  • we should probably get aligned in a github issue (that proposes a design, not just the original motivation of cleaning up the DB) so as to reduce back and forth on the PR

can you create an issue where you lay out your design here and why? this would be a non-trivial introduction to the server

@levzem
Copy link
ContributorAuthor

@zzstoatzz thanks for the quick turnaround.

I agree it should be off by default, that was a copy pasta mistake on my end.

I have tagged you in a design proposal in#16054. Look forward to hearing your thoughts.

zzstoatzz reacted with heart emoji

@zzstoatzzzzstoatzz marked this pull request as draftOctober 27, 2025 18:04
@zzstoatzz
Copy link
Collaborator

great thank you@levzem - marking this as a draft for the time being! will give feedback on the issue

@github-actions
Copy link
Contributor

This pull request is stale because it has been open 14 days with no activity. To keep this pull request open remove stale label or comment.

@github-actions
Copy link
Contributor

This pull request was closed because it has been stale for 14 days with no activity. If this pull request is important or you have more to add feel free to re-open it.

@zzstoatzz
Copy link
Collaborator

hi@levzem - sorry for the delay here

if you're willing, can you check out thenew background servicesimplemented in docket?

notably,LoopService has been totally removed, and you can instead use a perpetual docket task

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@cicdwcicdwAwaiting requested review from cicdwcicdw is a code owner

@desertaxledesertaxleAwaiting requested review from desertaxledesertaxle is a code owner

@zzstoatzzzzstoatzzAwaiting requested review from zzstoatzzzzstoatzz is a code owner

@chrisguidrychrisguidryAwaiting requested review from chrisguidrychrisguidry will be requested when the pull request is marked ready for reviewchrisguidry is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

performanceRelated to an optimization or performance improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Auto clean-up feature for the Prefect internal database

2 participants

@levzem@zzstoatzz

[8]ページ先頭

©2009-2025 Movatter.jp