This document provides an overview of Dataplex Universal Catalog metadata change feeds.These metadata change feeds let you track metadata changes in yourDataplex Universal Catalog instance in near real-time and build event-drivenworkflows based on those changes.

Automated monitoring of metadata changes

In Dataplex Universal Catalog, an entry represents a data asset, such as aBigQuery table, and an aspect is a set of related metadata fieldsattached to an entry that describes it. When an entry or aspect is created,updated, or deleted, Dataplex Universal Catalog publishes a notification message toa Pub/Sub topic that you specify. These notifications, also calleda metadata change feed, contain information about the change. This includes whenthe change happened, what resource changed, and the type of change. For moreinformation about entries and aspects, seeAbout metadata management inDataplex Universal Catalog.

The following architecture diagram shows how Dataplex Universal Catalog capturesmetadata changes (create, update, delete) and pushes them toPub/Sub for downstream event-driven workflows.

To control which changes generate notifications, you can configure a metadatachange feed to monitor specific resources. You do this by specifying a scope,such as your entire organization, specific projects, or specific entry groups.While scope lets you definewhich resources to monitor, you can use filters tofurther refinewhen Dataplex Universal Catalog sends notifications. For example,you might want to receive notifications only when tables of typebigquery-table are updated, but not when they're created or deleted. To dothis, you can apply one or more filters to your metadata change feed based onentry type, aspect type, or change type (CREATE,UPDATE, orDELETE).

For example, an online retail company uses BigQuery to manage productinventory in a dedicated project. To monitor only schema changes in theirinventory tables, they create a metadata change feed with the project as scope,and apply filters forentry_type=bigquery-table andchange_type=UPDATE. Ifthe schema of a critical table, such asproduct_stock, updates, this changegenerates anUPDATE notification that matches the metadata change feed'sfilter. The metadata change feed then sends a notification to aPub/Sub topic. An automated workflow subscribed to thisPub/Sub topic can immediately pause downstream reporting pipelinesand alert the inventory management team to prevent decisions based oninconsistent data.

Use cases

You can use metadata change feeds for various purposes, including:

• Metadata sync: continuously sync Dataplex Universal Catalog metadatachanges to an external or third-party data catalog or search index.
• Policy enforcement: automatically apply or update security policies whena data classification aspect changes on an entry.
• Data quality automation: trigger a data quality scan or alert the dataowner when the schema of a table changes.
• ETL/ELT triggering: start a data transformation job when a new tableentry is created or updated.
• Auditing: log all metadata changes to an audit table for compliancepurposes.

Terminology

A metadata change feed is a Dataplex Universal Catalog resource that monitorsmetadata changes (create, update, delete) for entries and aspects, and sendsnotifications to a Pub/Sub topic. In the API, this resource iscalledmetadataFeeds (projects/PROJECT_ID/locations/LOCATION/metadataFeeds/FEED_ID).

You configure a metadata change feed by defining its scope, filters, and destination. When ametadata change occurs that matches the metadata change feed's scope and filters,Dataplex Universal Catalog publishes a notification message to the destinationPub/Sub topic.

Metadata change feed configuration

You can configure a metadata change feed by defining the following:

• Scope: the set of resources to monitor for changes, such as yourentire organization, specific projects, or specific entry groups. In theAPI, you specify the resource name. The following example shows theresource name format for an entry group:projects/PROJECT_ID/locations/LOCATION/entryGroups/ENTRY_GROUP_ID.

• Filters: criteria to filter which changes generate notifications basedon entry type, aspect type, or change type (CREATE,UPDATE, orDELETE). In the API, you specify the resource name. The followingexample shows the resource name format for an entry type:projects/PROJECT_ID/locations/global/entryTypes/ENTRY_TYPE. If you don't specify any filters,then all change types (CREATE,UPDATE, andDELETE) within thefeed's scope generate notifications.

• Destination: the Pub/Sub topic whereDataplex Universal Catalog publishes notification messages. In the API, youspecify the topic name. The following example shows the resource nameformat for a Pub/Sub topic:projects/PROJECT_ID/topics/TOPIC_ID.

The following example shows a metadata change feed configured to monitorprojectsPROJECT_ID_1 andPROJECT_ID_2 forCREATE events,sending notifications toTOPIC_ID:

{"scope":{"projects":["projects/PROJECT_ID_1","projects/PROJECT_ID_2"]},"filter":{"changeTypes":["CREATE"]},"pubsubTopic":"projects/PROJECT_ID_PUBSUB/topics/TOPIC_ID"}

For instructions on how to create and manage metadata change feeds, seeReceive notifications with metadata change feeds.

Notification message format

When a metadata change triggers a notification, Dataplex Universal Catalogpublishes a message to the specified Pub/Sub topic. The changeevent detail is captured in aPub/Submessage. The message consists ofattributes for filtering, and a data payload with details of the change.

For more information about consuming these messages, seeConsume notificationmessages.

Attributes

The attributes enable filtering messages within the topic. You can filtermessages on subscription usingPub/Sub subscriptionfilters.

The following fields are provided in the attributes:

• timestamp: The timestamp of when the change happened.
• entry_name: The resource name of the entry, in the formatprojects/PROJECT_ID/locations/LOCATION/entryGroups/ENTRY_GROUP_ID/entries/ENTRY_ID.
• entry_fqn: Thefully qualified nameof the entry.
• feed_name: The resource name of the metadata change feed, in theformatprojects/PROJECT_ID/locations/LOCATION/metadataChangeFeeds/FEED_ID.
• entry_type: The resource name of the entry type, in the formatprojects/PROJECT_NUMBER/locations/LOCATION/entryTypes/ENTRY_TYPE_ID.For more information, seeEntry types.
• entry_change_type: The type of change:CREATED,UPDATED, orDELETED.

The following example shows the attributes for an entry creation event:

{"feed_name":"projects/PROJECT_ID/locations/LOCATION/metadataFeeds/FEED_ID","entry_change_type":"CREATE","timestamp":"2026-02-03T23:12:03.054469Z","entry_type":"projects/PROJECT_NUMBER/locations/global/entryTypes/ENTRY_TYPE_ID"}

Data payload

The data payload of the Pub/Sub message is a JSON string thatcontains details about the change.

The following is an example of a data payload:

{"entryName":"projects/PROJECT_ID/locations/LOCATION/entryGroups/ENTRY_GROUP_ID/entries/ENTRY_ID","full_qualified_name":"bigquery:PROJECT_ID.DATASET_ID.TABLE_ID","updatedAspects":["projects/PROJECT_NUMBER/locations/global/aspectTypes/updated-aspect-type"],"createdAspects":["projects/PROJECT_NUMBER/locations/global/aspectTypes/created-aspect-type"],"deletedAspects":["projects/PROJECT_NUMBER/locations/global/aspectTypes/deleted-aspect-type"]}

VPC Service Controls considerations

Metadata change feeds are VPC Service Controls (VPC-SC) compliant.

• If a metadata change feed is organization-scoped, only projects within themetadata change feed's VPC Service Controls perimeter generatenotifications.

• If a metadata change feed is project- or entry-group-scoped, all specifiedprojects or entry groups must reside in the same VPC Service Controlsperimeter as the metadata change feed; otherwise, metadata change feedcreation fails.

Quotas and limitations

For information about quotas related to metadata change feeds, seeQuotas.

Review the following limitations for metadata change feeds:

• Delivery: metadata change feeds deliver notifications on an "at leastonce" basis. You must handle potential duplicate messages in yoursubscriber.

• Ordering: Dataplex Universal Catalog doesn't guarantee the order of messagedelivery.

• Latency: metadata change feed notifications are near real-time.

• Activation delay: a newly created or updated metadata change feedconfiguration can take up to 10 minutes to become active due to caching inthe backend.

• Payload: the initial notification message contains only changesignatures. For example, entry name, entry type, change type, and alist of changed aspect types or keys, and doesn't contain the actualchanged data (the aspect payload). You must call theDataplex Universal Catalog API(GetEntry)to retrieve the current state of the entry or aspect if needed.

• Generic storage aspects: metadata change feeds doesn't provide support for generic storageaspects.

Pricing

There are no direct charges for Dataplex Universal Catalog metadata change feeds.However, you incur costs for the resources used, such as Pub/Submessage delivery, storage, and data egress. SeePub/Subpricing.

What's next

• Learn how toReceive metadata notifications with metadata changefeeds.
• Troubleshoot metadata change feeds.