Setup

This page provides an overview of Pub/Sub notifications forCloud Storage.

Overview

Pub/Sub notifications sends information about changes toobjects in your buckets toPub/Sub, where the information is addedto a Pub/Sub topic of your choice in the form of messages. Forexample, you can track objects that are created and deleted in your bucket.Each notification contains information describing both the event that triggeredit and the object that changed.

You can send notifications to any Pub/Sub topic in any project forwhich you have sufficient permissions. Once received by thePub/Sub topic,subscribers to the topic can receive theassociated message. SeeConfigure Pub/Sub notifications for Cloud Storagefor information on connecting yourCloud Storage buckets to a Pub/Sub topic.

Note: When enabled,events that trigger notifications takeslightly longer to complete. This impact is on the order of 0.1 seconds.

Other notification options

Subscribing to Pub/Sub notifications is a versatile way totrigger alerts and actions in response to changes in a bucket. The followingoptions are also available:

• Cloud Run functions: If you only want to trigger a lightweight,stand-alone function in response to events and don't want to manage aPub/Sub topic, use Cloud Run functions.Cloud Run functions allow you to execute C#, Go, Java, Node.js, Python,PHP, and Ruby functions when an object in your bucket changes. Note that yourbucket must reside in the same project as Cloud Run functions.See the associated tutorial for a demonstration of usingCloud Run functions with Cloud Storage.

Notification configurations

Anotification configuration is a rule you attach to a bucket that specifies:

• The topic in Pub/Sub that receives notifications.
• The events that trigger a notification to be sent.
• The information contained within notifications.

You can attach multiple notification configurations to a bucket. A bucket canhave up to 100 total notification configurations and up to 10 notificationconfigurations set to trigger for a specific event.

For example, if you have a notification configuration that sends deletionnotifications to one Pub/Sub topic, you can add a secondnotification configuration to the bucket that sends deletion notifications toanother topic. However, if you try to create more than 10 notificationconfigurations that do this, you receive an error. In addition to thesenotification configurations, you can also create notification configurationsthat send notifications for other events, such as object creation, either toPub/Sub topics used by the deletion notifications, or todifferent topics.

Each notification configuration is identified by aninteger. This integer is returned:

• When you create the notification configuration.
• When you list the notification configurations attached to a bucket.
• In thenotificationConfig attribute of each notification triggered by thenotification configuration.

Creating and deleting notification configurations increment a bucket'smetageneration number.

Event types

The following is a list of event types supported by Cloud Storage:

Event type	Description
OBJECT_FINALIZE	Sent when a new object (or a new generation of an existing object) issuccessfully created in the bucket. This includes copying, rewriting, or restoring an existing object. A failed upload does not trigger this event.
OBJECT_METADATA_UPDATE	Sent when the metadata of an existing object changes.
OBJECT_DELETE	Sent when an object has been permanently deleted. This includes objects that are replaced or deleted as part of the bucket'slifecycle configuration. This does not include objects that become noncurrent (seeOBJECT_ARCHIVE) or abortedmultipart uploads.
OBJECT_ARCHIVE	Only sent when a bucket has enabledobject versioning. This event indicates that the live version of an object has become a noncurrent version, either because it was explicitly made noncurrent or because it was replaced by the upload of an object of the same name.

For other Cloud Storage events, such as bucket operations or objectreads, you can enable the appropriatetype of audit log in Cloud Audit Logsandroute the audit logs to Pub/Sub using afilter.

Replacing objects

Replacing an existing object with a new one of the same name triggers twoseparate events:OBJECT_FINALIZE for the new version of the object and eitherOBJECT_ARCHIVE orOBJECT_DELETE for the replaced object. TheOBJECT_FINALIZE event contains an additional attributeoverwroteGeneration,which provides the generation number of the object that was replaced. TheOBJECT_ARCHIVE orOBJECT_DELETE event contains an additional attributeoverwrittenByGeneration, which provides the generation number of the newobject.

Notification format

Notifications sent to the Pub/Sub topic consist of two parts:

• Attributes: A set of key:value pairs describing the event.
• Payload: A string that contains the metadata of the changed object.

Attributes

Attributes are key:value pairs contained in all notifications sent byCloud Storage to your Pub/Sub topic. Notifications alwayscontain the following set of key:value pairs, regardless of the notification'spayload:

Attribute name	Example	Description
notificationConfig	projects/_/buckets/foo/notificationConfigs/3	An identifier for the notification configuration that triggered this notification.
eventType	OBJECT_FINALIZE	The type of event that has just occurred. SeeEvent types for a list of possible values.
payloadFormat	JSON_API_V1	The format of the object payload. SeePayload for a list of possible values.
bucketId	foo	The name of the bucket that contains the changed object.
objectId	bar	The name of the changed object.
objectGeneration	123456	The generation number of the changed object.
eventTime	2021-01-15T01:30:15.01Z	The time that the event took place, expressed in theRFC 3339 format.

Notifications sometimes contain the following set of key:value pairs,regardless of the notification's payload:

Attribute name	Example	Description
overwrittenByGeneration	107458	The generation number of the object that replaced the object that this notification pertains to. This attribute only appears inOBJECT_ARCHIVE orOBJECT_DELETE events in the case of a replacement.
overwroteGeneration	352947	The generation number of the object that was replaced by the object that this notification pertains to. This attribute only appears inOBJECT_FINALIZE events in the case of a replacement.

In addition to the above attributes, a notification configuration can containup to 10 custom attributes. Custom attributes are defined when creating anotification configuration, using the--custom-attributes in agcloud storage command, or thecustom_attributes object in thebody of aPOST notificationConfigs JSON request.

Payload

The payload is a string that contains the metadata of the changed object. Whenyou create a notification configuration, you specify a type of payload toinclude in notifications triggered by that configuration. You can specifythe following types of payload:

Payload type	Description
NONE	No payload is included with the notification.
JSON_API_V1	The payload will be a UTF-8 string containing theresource representation of the object’s metadata.

ForOBJECT_DELETE notifications, the metadata contained in the payload representsthe object metadata as it was before the delete, along with an additionaltimeDeleted property. For all other notifications, the metadata included in thepayload represents the object metadataafter thechange occurs.

For example, say you have a notification configuration thattracksOBJECT_METADATA_UPDATE events. If a user changes thecontentTypeproperty of an object frombinary/octet-stream tovideo/mp4, anOBJECT_METADATA_UPDATE notification is sent, and the metadata in thepayload includes"contentType":"video/mp4".

Delivery guarantees

Important: There is no SLA for delivery time, but notifications are typicallydelivered within seconds. In some circumstances notifications might be delayedsubstantially longer.

When you add a notification configuration, Cloud Storage may take upto 30 seconds to begin sending notifications associated with it. Once started,Cloud Storage guarantees at-least-once delivery toPub/Sub. Pub/Sub also offers at-least-once deliveryto the recipient, which means that you could receive multiple messages, withmultiple IDs, that represent the same Cloud Storage event.

Notifications are not guaranteed to be published in the orderPub/Sub receives them. If you plan to modify the Cloud Storageobject based on a notification, it is recommended that you use the object'sgeneration and metageneration numbers as preconditions on your update request.

If a notification consistently cannot be delivered to a Pub/Subtopic, Cloud Storage may delete the notification after 7 days. Deliveryfailure can occur when the Pub/Sub topic no longer exists, whenCloud Storage no longer has permission to publish to the topic,or when the project that owns the topic exceeds its publishing quota.

What's next

• Configure Pub/Sub notifications for Cloud Storage.
• Learn more about Pub/Sub.
• Subscribe a bucket to receive notifications sent toPub/Sub.
• Use Cloud Run functions todeliver events with a Cloud Storage trigger.
• Use Pub/Sub notifications forevent-driven transfersbetween Cloud Storage buckets.