Metadata preservation

This document describes metadata that is preserved when you useStorage Transfer Service to transfer data between various sources and destinations.

Overview

Storage Transfer Service preserves the following metadata:

  • User-createdcustom metadata fortransfers that originate from Cloud Storage, Amazon S3, orMicrosoft Azure Blob Storage is preserved.

  • Transfers between Cloud Storage buckets can optionally preserveobject ACLs, customer-managed encryption keys, storage class, objectcreation time (as the value of acustomTime field), and temporary holds.

  • For transfers from any source to a Cloud Storage bucket, the object'sstorage class in the destination bucket can be set to any supported class aspart of the transfer.

  • File size and last modified time (mtime) are preserved fortransfers that originate from POSIX file systems.mtime is notpreserved for folders.

  • Optionally, symlinks, numeric UID, numeric GID, and numeric MODE can bepreserved for transfers to and from POSIX file systems.

  • For transfers between file systems only, if UID, GID, or MODE are preserved,that metadata is also preserved for folders. Cloud Storage recreatesfolders on the destination file system and restores UID, GID, and/or MODE.This includes empty folders.mtime is not preserved.

    Folder-level metadata is not preserved if the transfer is bymanifest.

Metadata fields that are not explicitly mentioned in this document are notpreserved.

Metadata preservation behavior

The following sections list metadata examples from different sourcestorage systems and how Storage Transfer Service preservesmetadata from each. For an exhaustive list of metadata, refer to the sourcestorage system's documentation.

Amazon S3 or S3-compatible storage to Cloud Storage

Metadata examplePreservation behavior
Amazon S3 fixed-key metadata fields, such as:Cache-Control,Content-Disposition, andContent-Type.Preserved as fixed-key metadata.
Amazon S3 user-defined metadata, formatted as key:value pairs. For more information, see the User-defined object metadata section of Object key and metadata.

Preserved as a custom metadata field in destination Cloud Storage objects, which you can edit later or remove.

ETag Preserved as a custom metadata field with the keyx-goog-source-etag, which you can edit later or remove.
Object size. Preserved assize.
Amazon S3 access control lists (ACLs). For a complete list, see theCondition Keys section of Access Control List (ACL) Overview.Not preserved.
Amazon S3 object tags, defined by you as key-value pairs. For more information, see Object Tags.Not preserved.
Amazon S3 system-defined metadata, except forETag and object size. For a complete list, see theSystem-defined object metadata section ofObject key and metadata.

Not preserved.

Timestamp metadata from the source isn't preserved. Creation time,timeCreated, reflects the time that an object is created in Cloud Storage. Similarly,updated reflects the time that metadata for an object is modified in Cloud Storage.

Storage class

There are multiple options for setting storage class during a transfer.

  • Set each object's storage class to that of the destination bucket. This is the default behavior.
  • Set a specific storage class on all objects being transferred.

See the metadataOptions reference documentation for details.

Microsoft Azure Storage to Cloud Storage

Metadata examplePreservation behavior
Microsoft Azure Storage fixed-key metadata fields, such as:Cache-Control,Content-Disposition, andContent-Type. Preserved as fixed-key metadata.
Microsoft Azure Storage user-defined metadata, formatted as key:value pairs. For more information, see Settings and retrieving properties and metadata for Blob service resources.

Preserved as a custom metadata field in destination Cloud Storage objects, which you can edit later or remove.

ETag Preserved as a custom metadata field with the keyx-goog-source-etag, which you can edit later or remove.
Object size. Preserved assize.
POSIX file system permissions supported by Azure Data Lake Storage (ADLS) Gen 2. Not preserved.
Microsoft Azure Storage access control, specificallyx-ms-blob-public-access. For more information, see the Response Headers section of Get Container ACL. Not preserved.
Microsoft Azure Storage index tags. For more information, see Manage and find Azure Blob data with blob index tags. Not preserved.
Microsoft Azure Storage timestamp metadata, such as:Last-Modified,x-ms-creation-time,x-ms-version,x-ms-request-server-encrypted, andx-ms-encryption-scope. For more information, see Set Blob Metadata.

Not preserved.

Timestamp metadata from the source isn't preserved. Creation time,timeCreated, reflects the time that an object is created in Cloud Storage. Similarly,updated reflects the time that metadata for an object is modified in Cloud Storage.

Storage class

There are multiple options for setting storage class during a transfer.

  • Set each object's storage class to that of the destination bucket. This is the default behavior.
  • Set a specific storage class on all objects being transferred.

See the metadataOptions reference documentation for details.

Transfers between Cloud Storage buckets

Metadata examplePreservation behavior

Cloud Storage fixed-key metadata fields, such as:Cache-Control,Content-Disposition, andContent-Type.

For more information, seeObject metadata

Preserved as fixed-key metadata.

Cloud Storage user-defined metadata, formatted as key:value pairs. For more information, seeCustom metadata.

Preserved as a custom metadata field in destination Cloud Storage objects, which you can edit later or remove.

Object size Preserved assize.
Object generation Preserved as a custom metadata field with the keyx-goog-reserved-source-generation, which you can edit later or remove.
Object holds

Event-based holds are not preserved. If the destination bucket has thedefault event-based hold property enabled, an event-based hold is applied to transferred objects.

Temporary holds are preserved by default. To discard temporary holds during the transfer, set thetemporaryHold field of the metadataOptions object toTEMPORARY_HOLD_SKIP.

Access Control Lists (ACLs)

ACLs can optionally be preserved. See the metadataOptions reference documentation for details.

When preserving ACLs be careful to avoid creatinginaccessible objects.

For more information, see the Cloud StorageAccess Control Lists documentation.

Storage class

There are multiple options for setting storage class during a transfer.

  • Set each object's storage class to that of the destination bucket. This is the default behavior.
  • Preserve the source object's storage class.
  • Set a specific storage class on all objects being transferred.

See the metadataOptions reference documentation for details.

Customer-managed encryption key

If a customer-managed encryption key (CMEK) is being used on an object, the object can optionally use the same key when it is written to the destination bucket.

The default behavior is to write the object to the destination bucket using the bucket's encryption method.

When preserving the original CMEK, be aware of the following limitations:

See the metadataOptions reference documentation for details.

Timestamp metadata

timeCreated can optionally be preserved. The preserved value is stored in thecustomTime field of the transferred object in Cloud Storage. See the metadataOptions reference documentation for details.

updated metadata is not preserved.

Other Cloud Storage non-editable metadata, such asetag andcomponentCount. Not Preserved.

For a list of metadata in Cloud Storage, seeObjects.

URL list transfer to Cloud Storage

For more information about URL lists, seeCreating a URLlist.

Metadata examplePreservation behavior
Fixed-key metadata fields, such as:Cache-Control,Content-Disposition, andContent-Type. Preserved as editable metadata.
Content-Length andMD5

Preserved as non-editable metadata.

If the source doesn't provide anMD5 hash value, then we don't preserve a value.

This preservation behavior is specific toContent-Length andMD5. Any other non-editable metadata not listed is not preserved.

Timestamp metadata, such as: Creation time, modified time, and other source-specific metadata.

Not preserved.

Timestamp metadata from the source isn't preserved. Creation time,timeCreated, reflects the time that an object is created in Cloud Storage. Similarly,updated reflects the time that metadata for an object is modified in Cloud Storage.

Storage class

There are multiple options for setting storage class during a transfer.

  • Set each object's storage class to that of the destination bucket. This is the default behavior.
  • Set a specific storage class on all objects being transferred.

See the metadataOptions reference documentation for details.

POSIX file system transfers

When transferring files from POSIX file systems, Storage Transfer Service can optionallypreserve certain attributes as custom metadata. If these files are later writtenback to a file system, Storage Transfer Service can convert the preserved metadataback to POSIX attributes.

Metadata examplePreservation behavior
Modified time (mtime)

Preserved.

mtime is preserved as custom metadata with the keygoog-reserved-file-mtime.

File size

Preserved.

File size is preserved assize.

Numeric UID
Numeric GID
Numeric MODE
Symbolic links

Optional.

Preservation behavior is specified with themetadataOptions object. SeePreserving optional POSIX metadata for details.

Default behavior is to not preserve any metadata.

Folder metadataFolder-level metadata is only preserved for transfers between file systems. The transfer's UID, GID, and MODE preservation settings apply to files and folders for these transfers.

mtime is not preserved for folders.mtime is set to the create time of the folder on the destination file system.

Folder metadata is not preserved formanifest transfers.

Storage class

There are multiple options for setting storage class during a transfer.

  • Set each object's storage class to that of the destination bucket. This is the default behavior.
  • Set a specific storage class on all objects being transferred.

See the metadataOptions reference documentation for details.

Preserving optional POSIX metadata

To preserve one or more of numeric UID, numeric GID, numeric MODE,and symbolic links, specify ametadataOptions object in the body of your transfer job.

These options apply to both POSIX-to-Cloud Storage transfers andCloud Storage-to-POSIX transfers. For the latter, the metadata must havebeen preserved when files were initially transferred to Cloud Storage.

{  "description": "metadata-example",  "projectId": "example-project-id"  "transferSpec": {    ...    "transferOptions": {      "metadataOptions": {        "gid":     "GID_NUMBER",       # Default is "GID_SKIP"        "uid":     "UID_NUMBER",       # Default is "UID_SKIP"        "mode":    "MODE_PRESERVE",    # Default is "MODE_SKIP"        "symlink": "SYMLINK_PRESERVE"  # Default is "SYMLINK_SKIP"      }    }  }}

POSIX to Cloud Storage

Preserved metadata is stored in Cloud Storage as custom metadatakey:value pairs.

  • Numeric GID is stored asgoog-reserved-posix-gid.
  • Numeric UID is stored asgoog-reserved-posix-uid.
  • Numeric MODE is stored asgoog-reserved-posix-mode.

For symbolic links, Storage Transfer Service preserves the target link as an object inCloud Storage with the following qualities:

  • Object key is composed of the destination prefix plus the path to the symlink,relative to theroot_directory.
  • Object metadata:
    • Any symlink metadata is preserved as Cloud Storage object metadata.
    • A custom metadata entry is made:goog-reserved-file-is-symlink:true.
  • The object content is the target of the symlink. For example, for a symlinksym-> dir1/target, the object's content is "dir1/target".

Storage Transfer Service does not validate the link or copy the target file.

Cloud Storage to POSIX

If metadata is preserved when files are transferred to Cloud Storage, thatmetadata can be written back to the files when they aretransferred back to a POSIX file system.

If a metadata option is set to preserve, Storage Transfer Service takes the followingactions:

  • Symbolic links: Storage Transfer Service creates a symlink file pointing to thetarget link. If the target file does not exist, the symlink will be broken.
  • GID, UID, and MODE: the values stored in Cloud Storage metadata are writtenback to the file.

POSIX to POSIX

Transfers between file systems can optionally preserve GID, UID, and MODE forfiles and folders.

Last modified time is saved for files, but not for folders.mtimeis set to the create time of the folder on the destination file system.

Storage Transfer Service saves folder metadata by creating 0-byte folder objects inthe intermediate bucket, then copying that metadata back to the folder on thedestination file system. For this reason, the number of objects created in theintermediate bucket may be greater than the number of files beingtransferred.

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-19 UTC.