Setup Object Lifecycle ManagementConfiguration samples

To support common use cases like setting a Time to Live (TTL) for objects,retaining noncurrent versions of objects, or "downgrading" storage classes ofobjects to help manage costs, Cloud Storage offers the ObjectLifecycle Management feature.

This page describes the feature as well as the options available when using it.For the general format of a lifecycle configuration file, see thebucket resource representation for JSON or thelifecycle configuration format for XML.

Introduction

In order to use Object Lifecycle Management, you define a lifecycleconfiguration, which must beset on a bucket. The configuration contains aset of rules which apply to current and future objects in the bucket. When anobject meets the criteria of one of the rules, Cloud Storage automaticallyperforms a specified action on the object. Here are some example use cases:

• Downgrade the storage class of objects older than 365 days toColdline storage.
• Delete objects created before January 1, 2019.
• Keep only the 3 most recent versions of each object in a bucket withversioning enabled.

Lifecycle configuration

Each lifecycle management configuration contains a set of rules. Each rulecontains oneaction and one or moreconditions.

• An object has to matchall of the conditions specified in a rule for theaction in the rule to be taken.

• If you specify multiple rules that contain the same action, the action istaken on an object when that object matches the conditions inany of therules.

• If multiple rules have their conditions satisfied simultaneously for a singleobject, Cloud Storage performs the action associated with only oneof the rules, based on the following considerations:

  • TheDelete action takes precedence over anySetStorageClass action.
  • TheSetStorageClass action that switches the object to the storage classwith the lowestat-rest storage pricing takes precedence.

  For example, if you have one rule that changes the object's class toNearline storage and another rule that changes the object's class toColdline storage, but both rules use the exact same condition, theobject's class always changes to Coldline storage when the condition ismet.

• You should test your lifecycle rules on development data before applying toproduction to ensure your rules don't perform actions under unintended sets ofconditions. If that's not possible, you should test on a small subset of yourproduction data by using thematchesPrefix ormatchesSuffix conditions in your rules.

• Changes to a bucket's lifecycle configuration can take up to 24 hours to gointo effect, and Object Lifecycle Management might still perform actionsbased on the old configuration during this time.

  For example, if you change anage condition from 10 days to 20 days, anobject that is 11 days old could be deleted by Object Lifecycle Managementup to 24 hours later, due to the criteria of the old configuration.

For use cases, seeConfiguration examples for Object Lifecycle Management.

Lifecycle actions

A lifecycle rule specifies exactly one of the following actions:

• Delete
• SetStorageClass
• AbortIncompleteMultipartUpload

Delete

TheDelete action deletes an object when the object meets all conditionsspecified in the lifecycle rule. By default, when youdelete a live object, it becomessoft-deleted, and Cloud Storageretains it for a duration of seven days. You canrestore this soft-deletedobject within the soft delete retention duration.

Exception: In buckets withObject Versioning enabled, deleting the liveversion of an object causes it to become anoncurrent version, while deletinga noncurrent version deletes that version from the bucket. See theconfiguration for deleting objects for an example of using theDeleteaction along with Object Versioning.

TheDelete action does not take effect on an object while the object has anobject hold placed on it or aretention policy that it has not yetfulfilled. As long as the conditions in theDelete action remain satisfied forthe object, theDelete action occurs after any object hold is removed and anyretention policy is fulfilled.

Warning: If you disable soft delete on your bucket, any objects youdelete are permanently removed and cannot be restored. You should take carewhen setting up your lifecycle rules so that you don't cause more data to bedeleted than you intend.

SetStorageClass

TheSetStorageClass action changes thestorage class of an object andupdates the object'smodification time when the object meets allconditions specified in the lifecycle rule.

SetStorageClass supports the following storage class transitions:

Original storage class	New storage class
Durable Reduced Availability (DRA) storage	Nearline storage Coldline storage Archive storage Multi-Regional storage/Regional storage1
Standard storage, Multi-Regional storage, or Regional storage	Nearline storage Coldline storage Archive storage
Nearline storage	Coldline storage Archive storage
Coldline storage	Archive storage

¹ For buckets in aregion, the new storage class cannot beMulti-Regional storage. For buckets in a multi-region or dual-region,the new storage class cannot be Regional storage.

Cloud Storage does not validate correctness of the storage classtransition. This means that you can specify a storage class transition notlisted in the above table, but the transition will not occur. You should verifythat your lifecycle rules use one of the listed storage class transitions.

Note: You cannot set theSetStorageClass action on buckets withAutoclass enabled.

Abort incomplete multipart uploads

TheAbortIncompleteMultipartUpload action aborts an incompletemultipart upload and deletes the associated parts when the multipartupload meets the conditions specified in the lifecycle rule.

Only the following lifecycle conditions can be used with this action:

• age
• matchesPrefix
• matchesSuffix

Attempting to create a rule that uses theAbortIncompleteMultipartUploadaction in combination with other conditions results in an error.

Lifecycle conditions

A lifecycle rule includes conditions which an object must meet before theaction defined in the rule occurs on the object. Lifecycle rules support thefollowing conditions:

• age
• createdBefore
• customTimeBefore
• daysSinceCustomTime
• daysSinceNoncurrentTime
• isLive
• matchesStorageClass
• matchesPrefix andmatchesSuffix
• noncurrentTimeBefore
• numNewerVersions

Note: Capitalization differs between JSON and XML. Conditions are represented aslower camel case (lowerCamelCase) in JSON and upper camel case(UpperCamelCase) in XML. Additionally, when specifying the "number of newerversions" condition in an XML configuration file, you must use the nameNumberOfNewerVersions instead ofNumNewerVersions.

All conditions are optional, but at least one condition is required. If youattempt to set an invalid lifecycle configuration, such as by using an action orcondition that does not exist, you receive a400 Bad request error response,and any existing lifecycle configuration remains in place.

age

Theage condition is satisfied when a resource reaches the specified age (indays). Age is measured from the resource's creation time.

• For objects, the creation time is the time when the object is successfullywritten to the bucket, such as when an upload completes.

  • The age of an object is unaffected by the object becoming anoncurrent version.

• Formultipart uploads, the creation time is the time when the uploadis initiated.

For example, if a resource is created at 2022/01/10 10:00 UTC and theagecondition is 10 days, then the condition is satisfied for the resource on andafter 2022/01/20 10:00 UTC.

createdBefore

ThecreatedBefore condition is satisfied when an object is created beforemidnight of the specified date in UTC.

customTimeBefore

ThecustomTimeBefore condition is satisfied when the date portion of anobject'sCustom-Time metadata is earlier than the date specifiedin this condition. This condition is set using the date formatYYYY-MM-DD.customTimeBefore is never satisfied for an object with noCustom-Timemetadata set.

daysSinceCustomTime

ThedaysSinceCustomTime condition is satisfied when the specified number ofdays have passed since the date and time specified in an object'sCustom-Time metadata field. For example, if an object'sCustom-Time is2020-05-16T10:00:00Z and thedaysSinceCustomTime condition is 10 days, thenthe condition is satisfied for the object on and after 2020/05/26 10:00 UTC.

daysSinceCustomTime is never satisfied for an object with noCustom-Timemetadata set.

daysSinceNoncurrentTime

ThedaysSinceNoncurrentTime condition is typically only used in conjunctionwithObject Versioning. The condition is satisfied when the specifiednumber of days have passed since the object became noncurrent, either becausethe live version was deleted or replaced. For example, if an object becamenoncurrent at 2020/07/08 15:00 UTC and thedaysSinceNoncurrentTime conditionis 10 days, then the condition is satisfied for the object on and after2020/07/18 15:00 UTC.

isLive

TheisLive condition is typically only used in conjunction withObject Versioning. When set tofalse, this condition is satisfied for anynoncurrent version of an object. When set totrue, this condition is satisfiedfor the live version of an object. If you don't use versioning, all yourobjects are considered live and match whenisLive istrue.

matchesPrefix andmatchesSuffix

ThematchesPrefix andmatchesSuffix conditions are satisfied when thebeginning or end of anobject's name is an exact case-sensitive match withthe specified prefix or suffix. You can specify multiple strings as a list (forexample,"matchesSuffix": [".jpg", ".png"]).

When usingmatchesPrefix, don't include the bucket name or the/ thatprecedes object names in most request paths. For example, in theGoogle Cloud CLI, the path to an object in a bucket namedmy_bucket has aformat similar togs://my_bucket/pictures/paris_2022.jpg. To match the object,you would use a condition such as"matchesPrefix":["pictures/paris_"].

You can have up to 1000 prefixes and suffixes in total specified across allrules. In the Google Cloud console, you can copy and paste up to 1000 prefixes orsuffixes in total. A prefix or suffix cannot be used twice in a single condition.

matchesStorageClass

ThematchesStorageClass condition is satisfied when an object in the bucket isstored as the specified storage class. You can use the following values formatchesStorageClass:STANDARD,NEARLINE,COLDLINE,ARCHIVE,MULTI_REGIONAL,REGIONAL, andDURABLE_REDUCED_AVAILABILITY.

Caution: Do not useMULTI_REGIONAL inmatchesStorageClass if the bucket islocated in aregion, and do not useREGIONAL inmatchesStorageClass ifthe bucket is located in a multi-region or dual-region. Doing so results in anerror when you attempt toset a lifecycle management configuration.

Generally, if you intend to use thematchesStorageClass condition onStandard storage objects, you should also include the following:

• If the bucket is in aregion, includeREGIONAL andDURABLE_REDUCED_AVAILABILITY in the condition.

• If the bucket is in a multi-region or dual-region, includeMULTI_REGIONAL andDURABLE_REDUCED_AVAILABILITY in the condition.

Including these additional classes ensures the lifecycle rule covers olderobjects in your buckets which might be set to legacy storage classes.

Note: You cannot use thematchesStorageClass condition on buckets withAutoclass enabled.

noncurrentTimeBefore

ThenoncurrentTimeBefore condition is typically only used in conjunctionwithObject Versioning. The condition is satisfied for objects that becamenoncurrent on a date prior to the one specified in this condition. Thecondition is set using the date formatYYYY-MM-DD.noncurrentTimeBefore isnever satisfied for a live object.

numNewerVersions

ThenumNewerVersions condition is typically only used in conjunction withObject Versioning. If the value of this condition is set toN, an objectversion satisfies the condition when there are at leastN versions (includingthe live version) newer than it. For a live object version, the number of newerversions is considered to be 0. For the most recent noncurrent version, thenumber of newer versions is 1 (or 0 if there is no live object version), and soon.

Important: If you're working with an XML configuration file, this condition usesthe nameNumberOfNewerVersions.

Object lifecycle behavior

When Object Lifecycle Management is configured for Cloud Storagebuckets, Cloud Storage regularly inspects all the objects and performs allactions applicable according to the bucket's rules. Cloud Storageperforms an action asynchronously, so there can be a lag between when theconditions are satisfied and when the action is taken. Your applicationsshouldn't rely on lifecycle actions occurring within a certain amount of timeafter a lifecycle condition is met.

For example, if an object meets the conditions for deletion, the object mightnot be deleted right away, and you see the object until the lifecycle action isexecuted on the object.

Applicable charges still apply while the object remains in its originalstate, with the exception of at-rest storage costs, which are waived if theobject meets all of the following criteria:

• The object is in a bucket with soft delete disabled.

• The object is subject to a rule with aDelete action.

• The only condition for the rule is either anage condition or a combinationof theage andmatchesStorageClass conditions.

• Theage condition is satisfied for the object in rules with only agecondition. Both theage andmatchesStorageClass conditions should besatisfied for the object if the delete rule has both conditions.

• The object does not have any object holds.

Object lifecycle behavior on versioned objects

In buckets withObject Versioning enabled, a live object that gets deletedaccording to lifecycle rules will exist in a noncurrent state for some amount oftime. If the noncurrent version of the object also satisfies the deleterule's conditions, the noncurrent version of the object will also get deletedafter the time elapses.

For example, say there's a lifecycle rule that deletes objects older than 180days. If a live object is 200 days old, it gets deleted and becomes noncurrent.The now noncurrent object is still older than 180 days, so the noncurrent objectalso gets deleted after some time passes.

SetStorageClass cost considerations

Similar tochanging an object's storage class manually, usingSetStorageClass counts as aClass A operation and is billed at the ratedetermined by the destination storage class.

Caution: The cost of Class A operations from changing the storage class of smallobjects can outweigh the cost savings of changing to a lower cost storage tier.If you don't need objects anymore considerdeleting them.

Unlike changing an object's storage class manually, usingSetStorageClass doesnot rewrite an object. This gives Object Lifecycle Management certain pricingadvantages:

• There are neverinter-region replication charges,retrieval fees,orearly deletion fees associated with the storage class change.However, early deletion fees apply if objects in a storage class with aspecified minimum storage duration are deleted before the duration ends.

• The object's time spent set at the original storage class counts towards anyminimum storage duration that applies for the new storage class.

For example, say you upload an object as Nearline storage, and 20 days lateryour lifecycle configuration changes the storage class of the object toColdline storage. This change incurs no retrieval or early deletion fees.If you then delete the object 60 days after the storage class change, there isonly a 10-day early deletion charge, since Coldline storage has a 90-dayminimum storage duration, and the object existed for a total of 80 days.

In comparison, say you upload an object as Nearline storage, and 20 dayslaterchange the storage class using a rewrite (againto Coldline storage). This change incurs both a retrieval fee and a 10-dayearly deletion charge. If you then delete the object 60 days after the rewrite,there is a 30-day early deletion charge.

In both these examples, ifsoft delete is enabled on the bucket,the storage charges increase, but the early deletion charges reduce based onthe length of the soft delete retention period.

Object creation time

In many cases, an object's upload completes soon after it begins; however, foruploads that occur over multiple requests, such asresumable uploads,there can be days between when the initial upload request is sent and when thefinal upload request is sent. In such cases, you should keep in mind thefollowing:

• An object is not subject to lifecycle rules until after its upload completes.
• An object's creation time is based on when the upload completes. This impactstheage andcreatedBefore lifecycle conditions.
• When you set aCustom-Time for the object, you do so at the beginning of theupload. If you set aCustom-Time based on the time of the request, theCustom-Time could be much earlier than the object's creation time. Thisimpacts thecustomTimeBefore anddaysSinceCustomTime lifecycle conditions.

Expiration time metadata

Important: You can only retrieve an object's expiration time byrequesting the object's metadata with the XML API. The expiration timeappears in thex-goog-expiration response header of such a request.

If aDelete action is specified for a bucket with theage condition (and noother conditions besidesmatchesStorageClass), then some objects might betagged with expiration time metadata. An object's expiration time indicates thetime at which the object becomes (or became) eligible for deletion byObject Lifecycle Management. The expiration time might change as the bucket'slifecycle configuration orretention policy change.

Note that the absence of expiration time metadata does not necessarily meanthe object will not be deleted, but rather that not enough information isavailable to determine when or if it will be deleted. For example, if the objectcreation time is 2020/01/10 10:00 UTC and theage condition is setto 10 days, then the object expiration time is 2020/01/20 10:00 UTC. However,the expiration time is not available for the object if:

• There are other conditions specified in theDelete rule, with the exceptionofmatchesStorageClass.

• You use amatchesStorageClass condition that does not include the object'sstorage class.

• The object is under ahold, because Cloud Storage cannot knowwhen the hold will be removed.

• Soft delete is enabled on your bucket.

You are not charged for storage after the object expiration time even if theobject is not deleted immediately. You can continue to access the object beforeit is deleted and are responsible for other charges (request, networkbandwidth). If the expiration time is not available for anobject, the object is charged for storage until the time it is deleted.

When working with expiration times, keep in mind the following:

• If the bucket has aretention policy, the expiration time is the laterof the Object Lifecycle Managementage condition and the time theobject satisfies the retention period specified by the retention policy.

• If there are multiple conflicting expiration times applicable for an objectdue to different lifecycle management rules, then the earliest applicableexpiration time is used.

Options for tracking Lifecycle actions

To track the lifecycle management actions that Cloud Storage takes, useone of the following options:

• UseCloud Storage usage logs. This feature logs both theaction and who performed the action. A value ofGCS Lifecycle Managementin thecs_user_agent field of the log entry indicates theaction was taken by Cloud Storage in accordance with a lifecycleconfiguration.

• EnablePub/Sub Notifications for Cloud Storagefor your bucket. This feature sends notifications to a Pub/Subtopic of your choice when specified actions occur. Note that this featuredoes not record who performed the actions.

What's next

• Enable Object Lifecycle Management.
• Explorelifecycle configuration examples.
• Review the general format of a lifecycle configuration inJSON API requests andXML API requests.