Manage

This page discusses the commonly-used metadata fields that are stored along withobjects in Cloud Storage.

Introduction

Objects stored in Cloud Storage havemetadata associated with them.Metadata identifies properties of the object, as well as specifies how theobject should be handled when it's accessed. Metadata exists askey:valuepairs. For example, thestorage class of an object is representedby the metadata entrystorageClass:STANDARD.storageClass is thekeyfor the metadata, and all objects have such a key associated with them.STANDARD specifies thevalue this specific object has, and the value variesfrom object to object.

The mutability of metadata varies: some metadata you can edit at any time, somemetadata you can only set at the time the object is created, and some metadatayou can only view. For example, you can edit the value of theCache-Controlmetadata at any time, but you can only assign thestorageClass metadata whenthe object is created or rewritten, and you cannot directly edit the valuefor thegeneration metadata, though thegeneration value changes whenthe object is replaced.

Editable metadata

There are two categories of metadata thatusers can change for objects:

• Fixed-key metadata: Metadata whose keys are set, but for which youcan specify a value.

• Custom metadata: Metadata that you add by specifying both a key anda value associated with the key.

When editing metadata, you should generally avoid non-ascii characters,because they are not permitted in HTTP headers, which the XML API uses.

Fixed-key metadata

You can edit the following metadata for objects, though you musthave sufficientpermission to do so:

• Access control metadata
• Cache-Control
• Content-Disposition
• Content-Encoding
• Content-Language
• Content-Type
• Custom-Time
• Object holds
• Retention configuration

Access control metadata

Cloud Storage usesIdentity and Access Management (IAM) andAccess Control Lists (ACLs) to control access to objects. Use these linksto learn about these access control methods and associated metadata.

Cache-Control

TheCache-Control metadata can specify two different aspectsof how data is served from Cloud Storage: whether the data can becached and whether the data can be transformed.

TheCache-Control metadata lets you control whether and for how long cachesare allowed to cache your objects, which can then be served to satisfy futurerequests. Caches can include browser and Internet caches, as well asCloud Storage built-in caching.

If an applicable object doesn't have aCache-Control metadata entry,Cloud Storage uses the default value of:

• public, max-age=3600, if the object is not encrypted using acustomer-managed encryption key or stored within aVirtual Private Cloud service perimeter.

• no-cache, no-store, max-age=0, if the object is encrypted using acustomer-managed encryption key.

• private, max-age=0, if the object is stored within aVirtual Private Cloud service perimeter.

• no-cache, no-store, max-age=0, must-revalidate, if the object is storedwithin aVirtual Private Cloud service perimeterand encrypted using acustomer-managed encryption key.

If you allow caching, downloads might continue to receive earlier versions ofan object, even after uploading a newer version. This is because such earlierversions remain "fresh" in caches for a period of time determined bymax-age. Additionally, because objects can be cached at various places on theInternet, there is no way to force a cached object to expire globally. Thismeans that if you revoke public access to an object, the object can still beserved from a cache, depending on when it was last accessed and itsCache-Control setting. For example, if your object was served with aCache-Control ofpublic, max-age=3600, it can persist in a cache for anhour. If you want to prevent serving cached versions of publicly readableobjects, setCache-Control: no-store on the object.

If you need greater control over cache behavior, you can configureCloud CDN in front of your bucket.

Transforming data

Cache-Control metadata also lets you serve objects as they are stored,without applying anytransformations to the data, such as removing gzipcontent-encoding for incompatible clients. To serve an object as-is, setCache-Control:no-transform.

Content-Disposition

TheContent-Disposition metadata specifies presentationinformation about the data being transmitted. SettingContent-Dispositionallows you to control presentation style of the content, for exampledetermining whether an attachment should be automatically displayed or whethersome form of action from the user should be required to open it. Seehttps://datatracker.ietf.org/doc/html/rfc6266 for theContent-Disposition specification.

Content-Encoding

TheContent-Encoding metadata can be used to indicate that an object iscompressed, while still maintaining the object's underlyingContent-Type.For example, a text file that is gzip compressed can have the fact thatit's a text file indicated inContent-Type and the fact that it'sgzip compressed indicated inContent-Encoding. You should ensure thatfiles are, in fact, compressed using the specifiedContent-Encoding beforeuploading them, or else unexpected behavior can occur when attempting todownload the objects. For more information, see theTranscoding page.

For compressible content, such as text, usingContent-Encoding: gzip savesnetwork and storage costs and improves content serving performance. However,for content that is already inherently compressed, such as archives and manymedia formats, applying another level of compression and marking it in theContent-Encoding metadata is typically detrimental to both object size andperformance and should be avoided.

Content-Language

TheContent-Language metadata indicates the language(s) thatthe object is intended for. Refer toISO 639-1 language codesfor typical values of this metadata.

Cloud Storage supportsContent-Language values up to 100 charactersin length.

Content-Type

The most commonly set metadata isContent-Type (also known as media type),which lets browsers render the object properly. All objects have avalue specified in theirContent-Type metadata, but this value does not haveto match the underlying type of the object. For example, if theContent-Typeis not specified by the uploader and cannot be determined, it is set toapplication/octet-stream orapplication/x-www-form-urlencoded, depending onhow you uploaded the object. For a list of valid content types, see theIANA Media Types page.

TheCustom-Time metadata is a user-specified date and time represented intheRFC 3339 formatYYYY-MM-DD'T'HH:MM:SS.SS'Z' orYYYY-MM-DD'T'HH:MM:SS'Z' when milliseconds are zero. This metadata istypically set in order to use theDaysSinceCustomTime condition inObject Lifecycle Management.

You cannot removeCustom-Time once it's been set on an object. Additionally,the value forCustom-Time cannot decrease. That is, you cannot setCustom-Time to be an earlier date/time than the existingCustom-Time. Youcan, however, effectively remove or reset theCustom-Time byrewriting the object.

Object holds

Use metadata flags to place object holds, which prevent objects from beingdeleted or replaced. For more information, see theObject holds page.

Retention configuration

When present, an object's retention configuration defines a date and time priorto which the object cannot be deleted or replaced. For more information, seeObject Retention Lock.

Custom metadata

Custom metadata is metadata for which you define both the key and the value. Tocreate custom metadata, you specify both a key and a value. After you havecreated a custom metadatakey:value pair, you can delete the key or change thevalue.

Custom metadata is subject to asize limit and incursstorage costs.

TheViewing and Editing Metadata page includes information about settingcustom metadata.

Thex-goog-meta- prefix

TheXML API sets and retrieves object metadata using request headers, andtheJSON API allows setting custom metadata in the final request of aresumable upload by using request headers. In order to clearly distinguishcustom metadata headers from standard request headers, both APIs prefix suchcustom metadata headers withx-goog-meta-.

Non-editable metadata

Some metadata cannot be edited directly. This metadata is set at the time ofobject creation or rewrite. As part of the object creation or rewrite,you can set some such metadata, such as thestorage class of the objectorcustomer-managed encryption keys. Other metadata is automaticallyadded and can only be viewed, such the generation number of the object orthe time of creation.

Generation and metageneration numbers

As part of its metadata, every Cloud Storage object has a numericgeneration property and a numericmetageneration property that uniquelyidentifies the object:

Thegeneration andmetageneration properties are used in the followingcircumstances:

• When usingpreconditions in requests: Preconditions cause the requestto fail if the precondition isn't satisfied. Failing in this manner preventsthe request from applying to an unexpected version of an object, such asretrieving the wrong object data or modifying the wrong state of an object'smetadata.

• Whenlisting, accessing, restoring, and deleting noncurrent object versions:Noncurrent object versions are specifically relevant in buckets that use orpreviously usedObject Versioning.

Checksums

Checksums are metadata calculated from the associated object's data. Checksumsare used forvalidating that object data isn't corrupted.Cloud Storage objects have several checksum metadata fields.

CRC32C

All Cloud Storage objects have aCRC32C hash. Libraries forcomputing CRC32C include:

• Google's CRC32C for C++
• hash/crc32 for Go
• GoogleAPIs Guava for Java
• google-crc32c for Python
• digest-crc in Ruby

• The object is not acomposite object.
• The object was not uploaded using anXML API multipart upload.

When a Content-MD5 header is present in the request, Cloud Storage validatesthe integrity of the object data by comparing the provided MD5 hash with thecalculated hash of the uploaded object. If the hashes don't match, the requestis rejected.

ETags

All Cloud Storage objects have an ETag. However, the same object can have adifferent ETag value when it's requested from the XML API compared to the JSONAPI. In most cases, users should make no assumptions about the value used in anETag except that it changes whenever the underlying data or metadata changes,per thespecification.

An object's ETag header returns the object's MD5 value if all the following aretrue:

• The request is being made through the XML API
• The object only usesGoogle-owned and Google-managed encryption keys for server-sideencryption
• The object is not acomposite object and was not uploaded using anXML API multipart upload

Modification time

As part of its metadata, every Cloud Storage object has anupdatedproperty that indicates the last time the object's metadata was modified. Theupdated time is set initially to the object's creation time and then changeswhenever any metadata of the object changes. This includes changes made by arequester, such as modifying custom metadata, as well as changes made byCloud Storage on behalf of a requester, such as changing the storageclass based on anObject Lifecycle Configuration.

What's next

• View and edit object metadata.
• Learn about availablestorage classes.
• For a detailed description of all object metadata fields available in theJSON API, see theObjects reference documentation for JSON.
• Learn aboutinventory reports, which allow you to getthe metadata for all objects in a bucket at once.