By default, Vertex AI Search encrypts customer content at rest. Vertex AI Search handles encryption for you without any additional actions on your part. This option is calledGoogle default encryption.

If you want to control your encryption keys, then you can use customer-managed encryption keys (CMEKs) inCloud KMS with CMEK-integrated services including Vertex AI Search. Using Cloud KMS keys gives you control over their protection level, location, rotation schedule, usage and access permissions, and cryptographic boundaries. Using Cloud KMS also letsyoutrack key usage, view audit logs, andcontrol key lifecycles. Instead of Google owning and managing the symmetrickey encryption keys (KEKs) that protect your data, you control and manage these keys in Cloud KMS.

After you set up your resources with CMEKs, the experience of accessing your Vertex AI Search resources is similar to using Google default encryption. For more information about your encryption options, seeCustomer-managed encryption keys (CMEK).

Note: Data stores in US or EU regions require data encryption. The defaultencryption is with Google-owned and Google-managed encryption keys, but alternatively, youuse use your own keys as described here.

Limitations of Cloud KMS in Vertex AI Search

The following limitations apply to CMEK (Cloud KMS) keys inVertex AI Search:

• Keys that are already applied to a data store can't be changed, although keyversions can be rotated.

• You must use US or EU multi-region data stores and apps (not global ones).For more information about multi-regions and data residency, including limitsassociated with using non-global locations, seelocations.

• If you need to register more than one key for a project, contact your Googleaccount team to request a quota increase for CMEK configurations,providing a justification for why you need more than one key.

• Using external key manager (EKM) with CMEKis in GA with allowlist. To use EKM with CMEK, contact your Googleaccount team.

  The following limitations apply to EKM or HSM with CMEK:

  • Your EKM and HSM quota for encrypt and decrypt calls should have at least1,000 QPM of headroom. For how to check your quotas, seeCheck yourCloud KMS quotas.

  • If using EKM, the key must be reachable for more than 90% of any timewindow of longer than 30 seconds. If the key isn't reachable for thisamount of time, it can negatively impact indexing and search freshness.

  • If there are billing issues, persistent out-of-quota issues, or persistentunreachability issues for more than 12 hours, the service automaticallyturns down the CmekConfig associated with the EKM or HSM key.

• Data stores created before a key is registered to the project can't beprotected by the key.

• You can'ttune search modelsfor data stores that are protected by keys.

• For apps with multiple data stores, if one data store uses aCMEK configuration, all other data stores must also use the same CMEKconfiguration.

• For Vertex AI Search, Enterprise edition is required. Forinformation about Enterprise edition, seeAbout advanced features.

• First-party connectorsare not CMEK-compliant except for the "import-once" and "periodic" datastores for BigQuery and Cloud Storage.

• Healthcare search data stores and third-party connectors are CMEK-compliant.For general information about healthcare data stores,seeCreate a healthcare search data store. Forinformation about making third-party connectors CMEK-compliant, seeAboutsingle-region keys for third-party connectors.

• Keys don't apply to the following RAG APIs because these APIs don't storecontent at rest:check grounding,grounded generation, andranking.

• You can't use Terraform to configure CMEK for Vertex AI Search.

About single-region keys for third-party connectors

If you use third-party connectors and want to use your own keys to protect theconnected data, then you need to create three supplemental single-region keys inaddition to the multi-region key. The commands for creating keys are given inthe following procedure,Register your Cloud KMSkey.

The single keys need to be created for the following regions:

Multi-region	Single-regions
eu	europe-west1	europe-west4	europe-north1
us	us-east1	us-central1	us-west1

Before you begin

Make sure you satisfy the following prerequisites:

• Create a multi-region symmetric Cloud KMS key. SeeCreate a key ring andCreate a key in the Cloud KMS documentation.

  • Set the rotation period toNever (Manual rotation).

  • ForLocation, selectMulti-region, andselecteurope orus from the drop-down.

• The CryptoKey Encrypter/Decrypter IAM role(roles/cloudkms.cryptoKeyEncrypterDecrypter) on the key has beengranted to the Discovery Engine service agent.The service agent account has an email address that uses the followingformat:service-PROJECT_NUMBER@gcp-sa-discoveryengine.iam.gserviceaccount.com.For general instructions on how to add a role to aservice agent, seeGrant or revoke a singlerole.

• The CryptoKey Encrypter/Decrypter IAM role(roles/cloudkms.cryptoKeyEncrypterDecrypter) on the key has been granted tothe Cloud Storage service agent. If this role is not granted, data importfor CMEK-protected data stores will fail because Discovery Engine isnot able to make the CMEK-protected, temporary bucket and directory that isrequired for importing.

• Don't create any data stores or apps that you want managed by your key untilafter you have completed thekey registrationinstructions on this page.

• Enterprise edition features are turned on for the app. SeeTurn Enterprise edition on or off.

Register your Cloud KMS key

To encrypt data using CMEK, you must register your multi-regionkey. Optionally, if your data needs single-region keys,for example, when using third-party connectors, you need to register yoursingle-region keys.

Before you begin

Make sure of the following:

• The region isn't already protected by a key. The procedure below fails if akey is already registered for the region through the REST command.To determine if there is an active key in Vertex AI Search for alocation, seeView Cloud KMS keys.

• You have theDiscovery Engine Admin(roles/discoveryengine.admin) role.

Procedure

It can take several hours for the ingested data to show up in search results.

Note: After you create the data stores, verify that the data storesin the eu or us locations are protected by the keys. For more information, seeverify that a data store is protected by a Cloud KMSkey.

View Cloud KMS keys

To view a registered key for Vertex AI Search, do one of the following:

• If you have the CmekConfig resource name, call theGetCmekConfigmethod:

  curl-XGET\-H"Authorization: Bearer$(gcloudauthprint-access-token)"\"https://LOCATION-discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/cmekConfigs/CMEK_CONFIG_ID"

  Replace the following:

  • LOCATION: the multi-region of your data store:us oreu.
  • PROJECT_ID: the ID of your project that contains the data.
  • CMEK_CONFIG_ID: the ID of the CmekConfigresource. If you registered your key using the console, the ID isdefault_cmek_config.

  An example curl call and response looks like this:

  $ curl -X GET-H "Authorization: Bearer $(gcloud auth print-access-token)""https://us-discoveryengine.googleapis.com/v1/projects/my-ai-app-project-123/locations/us/cmekConfigs/default_cmek_config"
  {  "name":  "projects/my-ai-app-project-123/locations/us/cmekConfigs/default_cmek_config",  "kmsKey": "projects/key-project-456/locations/us/keyRings/my-key-ring/cryptoKeys/my-key"  "state": "ACTIVE"  "isDefault": true}

• If you don't have the CmekConfig resource name, call theListCmekConfigsmethod:

  curl-XGET\-H"Authorization: Bearer$(gcloudauthprint-access-token)"\"https://LOCATION-discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/cmekConfigs"

  Replace the following:

  • LOCATION: the multi-region of your data store:us oreu.
  • PROJECT_ID: the ID of your project that containsthe data.

  An example curl call and response looks like this:

  $ curl -X GET-H "Authorization: Bearer $(gcloud auth print-access-token)""https://us-discoveryengine.googleapis.com/v1/projects/my-ai-app-project-123/locations/us/cmekConfigs"
  {  "cmek_configs": [    {      "name":      "projects/my-ai-app-project-123/locations/us/cmekConfigs/default_cmek_config",      "kmsKey": "projects/key-project-456/locations/us/keyRings/my-key-ring/cryptoKeys/my-key"      "state": "ACTIVE"      "isDefault": true    }  ]}

Unregister your Cloud KMS key

To unregister your key from Vertex AI Search, follow thesesteps:

1. Call theDeleteCmekConfig method with the CmekConfig resource name thatyou want to unregister.

   curl-XDELETE\-H"Authorization: Bearer$(gcloudauthprint-access-token)"\"https://LOCATION-discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/cmekConfigs/CMEK_CONFIG_ID"

   Replace the following:

   • LOCATION: the multi-region of your data store:us oreu.
   • PROJECT_ID: the ID of your project thatcontains the data store.
   • CMEK_CONFIG_ID: the ID of the CmekConfigresource. If you registered your key using the console, the ID isdefault_cmek_config.

   An example curl call and response looks like this:

   $ curl -X DELETE-H "Authorization: Bearer $(gcloud auth print-access-token)""https://us-discoveryengine.googleapis.com/v1/projects/my-ai-app-project-123/locations/us/cmekConfigs/default_cmek_config" { "name": "projects/my-ai-app-project-123/locations/us/operations/delete-cmek-config-56789", "metadata": {  "@type": "type.googleapis.com/google.cloud.discoveryengine.v1.DeleteCmekConfigMetadata" }}

2. Optional: Record thename value returned by themethod and follow the instructions inGet detailsabout a long-running operation tosee when the operation is complete.

   It typically takes a few minutes to delete a key.

Note: A CmekConfig resource can only be deleted if no Vertex AI Searchresources are being protected by that CmekConfig resource.

Verify that a data store is protected by a key

Data stores that are created after your key is registered are protectedby the key. If you want to confirm that a particular data store is protected byyour key, follow these steps:

1. Run the following curl command on the data store:

   curl-XGET\-H"Authorization: Bearer$(gcloudauthprint-access-token)"\-H"Content-Type: application/json"\-H"x-goog-user-project:PROJECT_ID"\"https://LOCATION-discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/collections/default_collection/dataStores/DATA_STORE_ID"

   Replace the following:

   • LOCATION: the multi-region of your data store:us oreu.
   • PROJECT_ID: the ID of your project that contains the datastore.
   • DATA_STORE_ID: the ID of the data store.

   An example curl call looks like this:

   curl -X GET-H "Authorization: Bearer $(gcloud auth print-access-token)"-H "Content-Type: application/json"-H "x-goog-user-project: my-ai-app-project-123""https://us-discoveryengine.googleapis.com/v1/projects/my-ai-app-project-123/locations/us/collections/default_collection/dataStores/my-data-store-1"

2. Review the output from the command: If thecmekConfig field is in theoutput and thekmsKey field shows the key that you registered, then thedata store is protected by the key.

   An example response looks like this:

   { "name": "projects/969795412903/locations/us/collections/default_collection/dataStores/my-data-store-1", "displayName": "my-data-store-1", "industryVertical": "GENERIC", "createTime": "2023-09-05T21:20:21.520552Z", "solutionTypes": [   "SOLUTION_TYPE_SEARCH" ], "defaultSchemaId": "default_schema", "cmekConfig": {   "name": "projects/969795412903/locations/us/collections/default_collection/dataStores/my-data-store-1/cmekConfigs/default_cmek_config",   "kmsKey": "projects/my-ai-app-project-123/locations/us/keyRings/my-key-ring/cryptoKeys/my-key" }}

Other data protected by the Cloud KMS key

In addition to data in the data stores, your keys can protect other typesof app-owned core information held by Vertex AI Search, such as thesession data generated duringsearch withfollow-ups. This kind ofcore information is CMEK-protected if the data stores associated with the app areCMEK-protected.

Although, you can't run a specific command to verify that sessions areprotected, if you run theVerify that a data store is protected by akey command and see the key in the cmekConfig resource, then knowthat the session data is protected.

Rotate Cloud KMS keys

Preview

This product or feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA products and features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

When you rotate keys, you are creating a new version of the key and settingthe new version as the primary version. Leave the original version of the keyenabled for a while before disabling it. This gives any long-runningoperations that might be using the older key time to complete.

The following procedure outlines the steps to rotate keys for aVertex AI Search data store. For general information aboutrotating keys, seeKey rotation in theCloud KMS guide.

Important: Do not rotate keys on data stores associated with recommendationsapps or with any apps that need analytics, and don't rotate the single-regionkeys used for third-party connectors. SeeLimitations of Cloud KMS inVertex AI Search.

1. Reregister your key. Do this by repeating step 1 ofRegister yourCloud KMS key.

   Note: Reregistering is much faster than registering. You won't need tomonitor reregistering as a long-running operation.

2. See the instructions in theManage keys section of theCloud KMS guide to do the following:

   1. Create a new key version, enable it, and make it primary.

      After the new key is made primary, documents in the data store getre-encrypted using the new key, and any subsequent documents added to thedata store are encrypted with the new key.

   2. Leave the older key version enabled.

   3. After a week or so, disable the older key version and make sure thateverything is working as before.

   4. At some later date, when you are certain that no problems were caused bydisabling the older key version, you can destroy the older key version.

If a Cloud KMS key is disabled or revoked

If a key isdisabled or permissions for the key arerevoked, the data store stops ingesting data and stops serving data within 15minutes.However, re-enabling a key or restoring permissions takes a long time. It cantake up to 24 hours before the data store can resume serving data.

Therefore, don't disable a key unless necessary. Disabling and enabling a keyon a data store is a time-consuming operation. For example, repeatedly switchinga key between disabled and enabled means it will take a long time for thedata store to reach a protected state. Disabling a key andre-enabling it immediately afterward could result in days of downtimebecause the key is first disabled from the data store and subsequentlyre-enabled.

Troubleshoot: KMS key projects not found error

Symptom: When you try to create a data store using a KMS key, you receive anerror similar to the following:

KMS key projects/[...] not found for location: [...] and project number: [...]

Issue: This error message can be misleading. It doesn't necessarily meanthat the key doesn't exist. Instead, it might indicate that the KMS key hasn'tbeen registered for use with Vertex AI Search in the specified location.

Solution: To resolve this issue, register your key by following the steps inRegister your Cloud KMS key. After theregistration is complete, you can create data stores protected by that key.