This page explains how you can search for and manage your Spannerresources by using Data Catalog.

Data Catalogis a fully managed, scalable metadata management service withinDataplex Universal Catalog. It automatically catalogs the following metadata aboutSpanner instances, databases, tables, columns, and views:

• Name and fully-qualified name
• Location (region)
• Creation date and last modification date
• Schema (for tables and views)
• Description

Spannermetadata is automatically synced to Data Catalog at regularintervals, typically every few hours. You can use Data Catalog todiscover and understand your Spanner metadata. UseData Catalog to aid with the following activities:

• Analysis, including dependencies and suitability for a use case
• Change management
• Data movement (pipelines)
• Schema evolution

With Data Catalog, you can curate metadata by attaching tags toSpanner metadata entries. Each tag can have multiple metadatafields, and can be based on a predefined or custom tag template.

For example, you could attach the following tag to a column that contains asocial security number, which is personal identifiable information (PII):

pii:truepii_type:SSN

When youmove an instance that uses tags, the tags aren't automaticallymoved to the destination instance. Instead, you need to export tags from thesource instance before moving the instance, and import the tags into thedestination instance. For more information, seeExport and import tags.

To learn more about Data Catalog seeWhat isData Catalog.

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

• Select a project: Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
• Create a project: To create a project, you need the Project Creator role (roles/resourcemanager.projectCreator), which contains theresourcemanager.projects.create permission.Learn how to grant roles.

Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

Go to project selector

Verify that billing is enabled for your Google Cloud project.

1. Enable the Data Catalog API for the project.

   Enable the Data Catalog API

2. Check permissions.

   You need certain Identity and Access Management (IAM) roles and permissions to search for and attach tags to Spanner assets. For details, seeRequired IAM roles and permissions for Data Catalog.

Create tag templates

Tag temples are reusable structures that you can use to rapidly create new tags.Templates help you avoid duplication of work and incomplete tags. Create as manytag templates as you need.

To learn more, seeTags and tagtemplates.

Attach tags to Spanner assets

Attaching tags to Spanner assets lets you do the following:

• Add business metadata to the assets.
• Search for assets by business metadata and other custom metadata.

To learn more, seeTags and tagtemplates.

Search for Spanner assets

Use the Dataplex Universal Catalog search page in the Google Cloud consoleto search for Spanner assets.

1. Go to the Dataplex Universal Catalog search page.

   Go to Search

2. In theFilters panel, underSystems, selectSpanner.

   Dataplex Universal Catalog displays all known Spanner assets.

3. Optional: Do one of the following to narrow down the search:

   • Use the faceted search on theSearch page. Select checkboxes underData types,Projects, andTags.

   • In the search field, append a search parameter aftersystem=cloud_spanner. Separate parameters with spaces.

   For example, to view only databases, enter the following text in the searchfield, and then pressEnter.

   system=cloud_spannertype=database

   Note: To search for Spanner instances, usetype=service.To search for an instance configuration, useinstance_config=configuration-name.

   You can also use parentheses and the logical operatorsand andor for complex expressions. To learn more about the expressions that you can use in the search field, seeData Catalog search syntax.

4. In the results table, click the name of an asset to view the metadata forthat asset.

5. Optional: Do any of the following:

   • ClickADD OVERVIEW to add a rich text description of the asset.
   • ClickATTACH TAGS to add a tag to the asset.
   • For a table, click theSCHEMA tab to view the table columns.
   • For an instance (SERVICE), to view member databases, click theENTRY LIST tab, and then clickVIEW CHILD ENTRIES IN SEARCH. (If theENTRY LIST tab doesn't appear, then the instance has no databases.)

Example workflow - Drill down from instance to columns

In this example workflow, you start by searching for a Spannerinstance, then view a member database, then view a table in that database, andthen view the columns in the table.

1. Go to the Dataplex Universal Catalog search page.

   Go to Search

2. In theFilters panel, underSystems, selectSpanner.

3. To view all Spanner instances in Data Catalog,either select theService checkbox underData types, or enter thefollowing text in the search field and pressEnter.

   system=cloud_spannertype=service

4. Select an instance name.

5. On theSpanner service details page, click theENTRYLIST tab, and then clickVIEW CHILD ENTRIES IN SEARCH.

   Dataplex Universal Catalog displays the databases in the instance.

   Note: If there is noENTRY LIST tab, return to theSearch page andchoose a different instance.

6. On theSpanner database details page, click theENTRYLIST tab, and then clickVIEW CHILD ENTRIES IN SEARCH.

   Dataplex Universal Catalog displays the tables in the database.

7. Select a table name, and then on theSpanner table details page, clickSCHEMA to see the table columns.

8. Optional: To add a tag to a column, click the plus sign underColumntags.

When youmove a Spanner instance,the moving process deletes the instance tags that you created inData Catalog. To preserve your tags, you need to do the following:

• Query the tags associated with the instance.
• Copy the details for the tags.
• Create the tags on the moved instance.

Spanner sync data every 6 hours. Any metadata changes made onSpanner assets like instances, databases, tables, views orcolumns could take approximately 6 hours to propagate to Data Catalog.

Export tags from the source instance configuration

To list the tags for an instance (entry or entry group), use the Google Cloud CLIgcloud data-catalog tags list command as follows:

curl \'https://datacatalog.googleapis.com/v1/projects/PROJECT/locations/LOCATION/entryGroups/ENTRY_GROUP/tags?key=API_KEY' \  --header 'Authorization: BearerACCESS_TOKEN' \  --header 'Accept: application/json' \  --compressed

Replace the following:

• PROJECT: Project that contains the tags.
• LOCATION: Location for the tags.
• API_KEY: A unique string that lets you access an API.
• ACCESS_TOKEN: The access token that your application uses toauthenticate to the service.

Import tags into the destination configuration

Before you complete this procedure, do the following:

• Move the instance.
• Update Data Catalog with metadata.

To copy over a tag, create the tags on the moved instance using thegcloud data-catalog tags createcommand as follows:

curl --request POST \'https://datacatalog.googleapis.com/v1/entries:lookup?fullyQualifiedName=FQN&location=LOCATION&project=PROJECT&key=API_KEY' \  --header 'Authorization: BearerACCESS_TOKEN' \  --header 'Accept: application/json' \  --header 'Content-Type: application/json' \  --data '{"column":"myColumnName","name":"myTagName","template":"myTemplateName","fields":{"myDoubleField":{"doubleValue":0}}}' \  --compressed

Replace the following:

• FQN: Fully qualified name (FQN) of the resource. FQNs take twoforms:

  For non-regionalized resources:{SYSTEM}:{PROJECT}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  For regionalized resources:{SYSTEM}:{PROJECT}.{LOCATION_ID}.{PATH_TO_RESOURCE_SEPARATED_WITH_DOTS}

  Example for a DPMS table:

  dataproc_metastore:{PROJECT_ID}.{LOCATION_ID}.{INSTANCE_ID}.{DATABASE_ID}.{TABLE_ID}

• LOCATION: Location where the lookup is performed.

• PROJECT: Project where the lookup is performed.

• API_KEY: A unique string that lets you access an API.

• ACCESS_TOKEN: The access token that your application uses toauthenticate to the service.

Required IAM roles and permissions for Data Catalog

The following table shows the required IAM roles and permissionsfor the various Data Catalog operations.

For more information, seePredefined roles.

What's next

• What is Data Catalog
• Roles to search Google Cloud resources
• About fine-grained access control