This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

This document describes the architecture and key concepts ofdata products in Dataplex Universal Catalog.

A data product is a logical, curated collection of data assets, formallypackaged to ensure it's discoverable, trusted, and accessible. The keycapabilities of a data product include the following:

• Organize catalog assets into a logical unit that solves a specific businessproblem and enables faster time to insights.
• Distribute with context that includes a description, documentation, and aspects.
• Establish trust with contracts that enable data producers to provideassurance to data consumers.
• Provide self-service workflow for data consumers to evaluate data productsand get access to data.

Key concepts

This section describes the key concepts and terminologies related to dataproducts.

Data product

A curated, logical grouping of data assets, formally packaged to be discoverable,trusted, and accessible for solving specific business problems.

Asset

A pointer to a physical data resource, such as a BigQuery dataset,table, or view. A data product contains of one or more assets.

Access group

Google groups are configured by data product owners and used by data productconsumers to request access. Asset permissions are assigned to these accessgroups.

Access groups simplify permission management for your data product. Theyact as user-friendly aliases (likeReader orAnalyst) for underlyingIAM groups. This lets data product owners assignpermissions at a high level and helps consumers request the correct levelof access.

Data product owner or data producer

The individual or team responsible for the creation and management of dataproducts. This includes managing quality, access, and documentation.

Data product consumer

The individual, team, or AI agent that consumes data products to generateinsights.

Contract

An agreement between the data product owner and its consumers. Thisagreement sets clear expectations by defining specific terms for how the datawill be provided and used, such as its refresh schedule and quality standards.

Example use case

Consider a data scientist analyzing an ecommerce business. Their goal is tofind the average order value (AOV) by traffic source and see if there's acorrelation between user age and order size. To do this, they need to combinedata from multiple tables, such asorder_details,user_traffic, anduser_demographic.

In a conventional setup, this process creates friction. To generate insights, thedata scientist must first discover the correct tables within the organization'svast data landscape, then contact each data owner, justify their access request,and wait for approval.

With data products, data owners can streamline this experience by packagingthe relevant assets into a single product named "Ecommerce Business Data". Thispackage includes the following:

• Assets

  • BigQuery tablesorder_details anduser_traffic(containing historical order data and traffic sources)
  • BigQuery viewuser_demographics (providing user detailswith PII excluded)

• Access groups

  • PredefinedReader andWriter groups to streamline access requests

• Contract

  • A contract defining the data refresh frequency (for example, weekly at8:00 AM PST)

• Context

  • Documentation with sample queries and other details
  • Additional metadata to depict data sensitivity

Data scientists can now discover this data product as a single logical unit.This lets them confidently generate insights to answer questions like,'What is the average order value for each traffic source?'—ultimatelyrevealing which sources generate the highest value customers.

Data product user flow

The data product lifecycle in Dataplex Universal Catalog involves two key userjourneys: one for the data product owner (or producer) who creates and managesthe data, and one for the data product consumer who discovers and uses it.

Data product owner journey

This journey focuses on packaging, securing, and governing the data products toensure it's trusted and accessible.

• Create: Define the data product and include assets. This involves the followingactions:

  • Configure the unique name, project, region, and description.
  • Add assets such as BigQuery tables, datasets, or views.
  • Configure access groups (for example,Analyst orReader) and map them tounderlying Google groups to simplify permission management.
  • Assign the necessary IAM roles to these access groups for thespecific assets.
  • Add a contract (a system aspect) to formally communicate the agreed-upondata refresh cadence, frequency, and threshold.

  For more information, seeCreate data products.

• Manage: Update the data product and ensure discoverability. This involvesthe following actions:

  • Update basic details, assets, permissions, and supplementary aspects(metadata), and rich text documentation.
  • Grant access to consumers to discover and request access to data products.

  For more information, seeManage data products.

Data product consumer journey

This journey focuses on quickly finding trusted data and gaining the necessarypermissions to use it.

• Discover: Find relevant, trusted data for a specific business problem. Thisinvolves the following actions:

  • Use theDataplex Universal Catalog Search withkeywords or natural language to find the packaged data product.
  • Review the data product's overview, assets, contract, and other aspects todetermine its fitness for use.

  For more information, seeSearch for data products.

• Request access: Ask the data product owner for permission to access the data.

  For more information, seeRequest access to data products.

• Use: Access the underlying assets to generate insights. This involves thefollowing action:

  • Upon approval, you can access the product and its assets. For example, ifthe asset is a BigQuery table, you can navigate to theBigQuery studio and query the data directly.

  For more information, seeConsume data products.

Assets supported

A data product can be composed of one or more data assets. In prview, thefollowing data assets are supported:

• BigQuery datasets
• BigQuery tables
• BigQuery views

Limitations

• Data products and their underlying assets must reside in the sameGoogle Cloud location.
• A data product can contain a maximum of 10 assets.
• You can create a maximum of 50 data products per project.
• Request approval workflow integration isn't available in preview. However,data product consumers can request access by triggering email notifications todata product owners.

What's next

• Learn how tocreate a data product.
• Learn more aboutmanaging data products.
• Learn how tosearch for data products.
• Learn how torequest access for data products.