Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Classic workflow retention policy#698

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
NimRegev merged 7 commits intomasterfromclassic-workflow-retention-policy
May 17, 2023
Merged
Show file tree
Hide file tree
Changes fromall commits
Commits
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 13 additions & 11 deletions_data/nav.yml
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -370,6 +370,19 @@
url: "/introduction-to-codefresh-pipelines"
- title: Creating pipelines
url: "/pipelines"
- title: Configuration for pipelines
url: "/configuration"
sub-pages:
- title: Global pipeline settings
url: "/pipeline-settings"
- title: Shared configuration
url: "/shared-configuration"
- title: Secrets for pipelines
url: "/secrets-store"
- title: Public logs and status badges
url: "/build-status"
- title: Build retention policy
url: "/build-retention-policy"
- title: Steps in pipelines
url: "/steps"
sub-pages:
Expand DownExpand Up@@ -430,17 +443,6 @@
url: "/deployment-environments"
- title: Running pipelines locally
url: "/running-pipelines-locally"
- title: Configuration for pipelines
url: "/configuration"
sub-pages:
- title: Global pipeline settings
url: "/pipeline-settings"
- title: Shared configuration
url: "/shared-configuration"
- title: Secrets for pipelines
url: "/secrets-store"
- title: Public logs and status badges
url: "/build-status"
- title: Service containers
url: "/service-containers"
- title: Docker image metadata
Expand Down
38 changes: 34 additions & 4 deletions_docs/installation/codefresh-on-prem.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -520,12 +520,20 @@ Codefresh installation supports automatic storage provisioning based on the stan


### Retention policy for Codefresh builds
Define a retention policy to manage Codefresh builds. The retention settings are controlled through `cf-api` deployment environment variables, all of which have default settings which you can retain or customize. By default, Codefresh deletes builds older than six months, including offline logs.
Define a retention policy to manage Codefresh builds. The retention settings are controlled through `cf-api` deployment environment variables, all of which have default settings which you can retain or customize.

There are two mechanisms to define the retention policy. Both mechanisms are implemented as Cron jobs.
1. Legacy retention mechanism: Allows you to delete builds in chunks, and also, optionally, delete offline logs from the database.
1. New retention mechanism: Allows you delete builds by days, and does not delete offline logs.


#### Configure retention policy for builds and logs
With this method, Codefresh by default deletes builds older than six months, including offline logs for these builds.

The retention mechanism, implemented as a Cron Job, removes data from collections such as:
* workflowproccesses
* workflowrequests
* workflowrevisions
*`workflowproccesses`
*`workflowrequests`
*`workflowrevisions`

{: .table .table-bordered .table-hover}
| Env Variable | Description | Default |
Expand All@@ -535,6 +543,28 @@ The retention mechanism, implemented as a Cron Job, removes data from collection
|`RETENTION_POLICY_DAYS` | The number of days for which to retain builds. Builds older than the defined retention period are deleted. | `180` |
|`RUNTIME_MONGO_URI` | Optional. The URI of the Mongo database from which to remove MongoDB logs (in addition to the builds). | |

#### Configure TTL-based retention policy

The TTL-based retention mechanism is implemented as a Cron job, and deletes data from the `workflowprocesses` collection. Build logs are not deleted.

>**IMPORTANT**:
> * For existing environments, for the retention mechanism to work, you must first drop the index in MongoDB. This requires a maintenance window that depends on the number of builds to be deleted, approximately three hours per MongoDB node.
>* If you have more than one `cf-api`, you must update the configuration for all of them.

{: .table .table-bordered .table-hover}
| Env Variable | Description | Default |
|---------------|--------------------------- |---------------------- |
|`TTL_RETENTION_POLICY_IS_ENABLED` | Determines if automatic build deletion through the Cron job is enabled. | `false` |
|`TTL_RETENTION_POLICY_IN_DAYS` | The number of days for which to retain builds, and can be between `30`(minimum) and `365` (maximum). Builds older than the defined retention period are deleted. | `365` |


1. (Optional) For existing environments:
1. In MongoDB, drop the index on `created` field in `workflowprocesses` collection.
1. In `cf-api`, add to `env`:
1. `TTL_RETENTION_POLICY_IS_ENABLED` set to `true`.
1. `TTL_RETENTION_POLICY_IN_DAYS`.
1. Verify that the `created` field in the `workflowprocesses` collection has a new index.
1. Restart `cf-api`.

### Managing Codefresh backups

Expand Down
50 changes: 50 additions & 0 deletions_docs/pipelines/configuration/build-retention-policy.md
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
---
title: "Configure build retention policy for pipelines"
description: "Define workflow retention"
group: pipelines
sub_group: configuration
toc: true
---

Define the retention policy for pipeline builds in your environments.

For existing environments, save storage space by deleting older builds. Fewer pipeline builds to manage, make it easy to navigate to and find relevant builds for monitoring and troubleshooting.
For new environments, defining a retention policy to better control build environments and optimize storage space.

Retention settings are controlled through environment variables in `cf-api`. By default, when enabled, Codefresh implements a Cron job that deletes removes data from the `workflowproccesses` collection.

>**NOTE**:
>Build logs are _not_deleted.


## Pipeline build retention settings


{: .table .table-bordered .table-hover}
| Env Variable | Description | Default |
|---------------|--------------------------- |---------------------- |
|`TTL_RETENTION_POLICY_IS_ENABLED` | Determines if automatic build deletion through the Cron job is enabled. | `false` |
|`TTL_RETENTION_POLICY_IN_DAYS` | The number of days for which to retain builds, and can be between `30`(minimum) and `365` (maximum). Builds older than the defined retention period are deleted. | `365` |



## (Optional) Drop MongoDB index for existing environments

For existing environments, for the retention mechanism to work, you must first drop the existing index in MongoDB.
This action requires a maintenance window that depends on the number of workflows to delete, approximately three hours per MongoDB node.

1. In MongoDB, drop the index for `created` field in `workflowprocesses` collection.
1. Continue with _Configure build retention policy_.

## Configure build retention policy
If you have more than one `cf-api`, you must configure _all_ of them.

1. In `cf-api`, add `env.TTL_RETENTION_POLICY_IS_ENABLED`, and set it to `true`.
1. Add `TTL_RETENTION_POLICY_IN_DAYS` and set to the required value. If not set, uses the default of `365` days.
1. Verify that the `created` field in the `workflowprocesses` collection has a new index.
1. Restart `cf-api`.

## Related articles
[Creating pipelines]({{site.baseurl}}/docs/pipelines/pipelines/)
[Monitoring pipelines]({{site.baseurl}}/docs/pipelines/monitoring-pipelines/)
[Pipeline integrations with Codefresh API]({{site.baseurl}}/docs/integrations/codefresh-api/)
[8]ページ先頭
©2009-2025 Movatter.jp