Create

This page describes composite objects, which you create from existing objectswithout transferring additional object data. Composite objects are useful formakingappends to an existing object, as well as for recreatingobjects that youuploaded as multiple components in parallel.

Compose operation

The compose operation concatenates the data in a given sequence of sourceobjects to create a new object called acomposite object. The source objectsall must:

• Have the samestorage class.
• Be stored in the same Cloud Storage bucket.

When you perform a composition:

• The source objects are unaffected.
• You can use between 1 and 32 source objects.
• Source objects can themselves be composite objects.

The composite object that results from a composition:

• Has the same storage class as the source objects.
• Does not change if the source objects are subsequently replaced or deleted.

When usinggcloud storage to perform object composition, the object thatresults has aContent-Type set to match theContent-Type of the firstsource object.

Composite object metadata

There are several differences between the metadata of a composite object andthe metadata of other objects:

• Composite objectsdo not have an MD5 hash metadata field.

  • Any MD5 value you provide as part of a compose request is ignored.

• The ETag value of a composite object is not based on an MD5 hash, and clientcode should make no assumptions about composite object ETags except thatthey change whenever the underlying object changes per theIETF specification for HTTP/1.1.

  Caution: Exercise caution when first using composite objects, since anyclients expecting to find an MD5 digest within the ETag header may concludethat object data has been corrupted, which could trigger endless dataretransmission attempts.

• Each composite object has acomponent count metadata field, which counts thenumber of non-composite objects contained within the composite object.

  • If you rewrite a composite object to a different location or storageclass, the result is a composite object with a component count of 1.

Integrity Checking Composite Objects

Cloud Storage usesCRC32C for integrity checking eachsource object at upload time and for allowing the caller to perform an integritycheck of the resulting composite object when it is downloaded. CRC32C is anerror detecting code that can be efficiently calculated from the CRC32Cvalues of its components. Your application should use CRC32C as follows:

• When uploading source objects, you should calculate the CRC32C foreach object using a CRC32C library, such as one of those listed in theObject metadata page, and include that value in your request. Based onthe values you provide, Cloud Storagevalidates each upload.
• The compose operation automatically checks that the source objects arecorrectly assembled, and it ignores any CRC32C value you provide as part ofthe compose request. A CRC32C of the resulting composite object is returnedin the response.
• If your application could change source objects between the time ofuploading and composing those objects, you should specify generation-specificnames for the source objects to avoid race conditions. Alternatively, you canbuild a CRC32C value from the CRC32C values of the intended source objects andcompare it to the CRC32C value returned by the compose operation.
• At download time, you should calculate the CRC32C of the downloadedobject and compare that with the value included in the response.

Limited Append and Edit

You can use the compose operation to perform limited object appends and edits.

You accomplish appending by uploading data to a temporary new object, composingthe object you wish to append with this temporary object, optionally naming theoutput of the compose operation the same as the original object, and deletingthe temporary object.

For example, in thegcloud CLI, the series of commands toappend the stringnew data to an existing Cloud Storage object isthe following:

$ echo 'new data' | gcloud storage cp - gs://bucket/temporary_object$ gcloud storage objects compose gs://bucket/object_to_append gs://bucket/temporary_object \    gs://bucket/object_to_append$ gcloud storage rm gs://bucket/temporary_object

You can also use composition to support a basic flavor of object editing.For example, you could compose an objectX from the sequence{Y1, Y2, Y3},replace the contents ofY2, and recomposeX from those same components.Note that this requires thatY1,Y2, andY3 be left undeleted, so youwill be billed for those components as well as for the composite.

• When performing appends in a bucket withObject Versioning enabled, be sure that you properly manage the noncurrent version of the object that each append generates.
• Soft delete preserves objects that get deleted or overwritten. To avoid unexpected data storage fees, consider disabling soft delete on buckets with high rates of compose operations.

Composite object contexts

During acompose object operation, Cloud Storage merges allcontexts (preview) from the source objects and attaches thesecontexts to the destination object. The contexts are merged to handle bothunique and duplicate context keys, as described in the following sections.

Unique context keys

If source objects have unique context keys, Cloud Storage directlyattaches these keys and their corresponding values to the destination object.

Consider the following example:

Source object A contexts:Department: Engineering, Status: Active

Source object B contexts:Owner: m_jones, Version: 1.1

After the compose operation, the destination object has the following combinedcontexts:

{"contexts":{"custom":{"Department":{"value":"Engineering","createTime":"2023-10-26T10:00:00.000Z","updateTime":"2023-10-26T10:00:00.000Z"},"Status":{"value":"Active","createTime":"2023-10-26T10:00:00.000Z","updateTime":"2023-10-26T10:00:00.000Z"},"Owner":{"value":"m_jones","createTime":"2023-10-26T10:00:00.000Z","updateTime":"2023-10-26T10:00:00.000Z"},"Version":{"value":"1.1","createTime":"2023-10-26T10:00:00.000Z","updateTime":"2023-10-26T10:00:00.000Z"}}}}

Duplicate context keys

When multiple source objects have the same context key, the value from the lastobject processed by Cloud Storage overrides the values from any objectsprocessed earlier.

For example, consider source objects processed in the following order:

1. Source object A

2. Source object B

Source object A contexts:Version: 1.0, ReleaseDate: 2024-01-15

Source object B contexts:Version: 1.1, Owner: m_jones

Both source objects have aVersion key, but object A hasVersion: 1.0 andobject B hasVersion: 1.1. Because Cloud Storage processes sourceobject B after source object A, theVersion value from source object B takesprecedence and the final value is1.1.

The destination object combines these contexts as follows:

{"contexts":{"custom":{"Version":{"value":"1.1","createTime":"2025-01-01T00:00:00.000Z","updateTime":"2025-01-01T00:00:00.000Z"},"ReleaseDate":{"value":"2024-01-15","createTime":"2025-01-01T00:00:00.000Z","updateTime":"2025-01-01T00:00:00.000Z"},"Owner":{"value":"m_jones","createTime":"2025-01-01T00:00:00.000Z","updateTime":"2025-01-01T00:00:00.000Z"}}}}

What's next

• Compose an object.
• Learn aboutparallel composite uploads.
• Learn aboutMultipart uploads, an alternative way to upload objects inmultiple chunks for XML API users.
• Learn aboutobject contexts.