Configure encryption with customer-managed encryption keys

Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page describes how to useCustomer Managed Encryption Keys (CMEK) to protectCloud Composer environments.

About Customer Managed Encryption Keys (CMEK)

By default, Cloud Composer encrypts customer content at rest. Cloud Composer handles encryption for you without any additional actions on your part. This option is calledGoogle default encryption.

If you want to control your encryption keys, then you can use customer-managed encryption keys (CMEKs) inCloud KMS with CMEK-integrated services including Cloud Composer. Using Cloud KMS keys gives you control over their protection level, location, rotation schedule, usage and access permissions, and cryptographic boundaries. Using Cloud KMS also letsyou view audit logs and control key lifecycles. Instead of Google owning and managing the symmetrickey encryption keys (KEKs) that protect your data, you control and manage these keys in Cloud KMS.

After you set up your resources with CMEKs, the experience of accessing your Cloud Composer resources is similar to using Google default encryption. For more information about your encryption options, seeCustomer-managed encryption keys (CMEK).

Data protected with CMEK encryption

Cloud Composer protects the following data with CMEK encryption:

  • Contents and schema of the Airflow database
  • Environment and Airflow task logs in Cloud Logging
  • Contents of the environment's bucket
  • Secrets stored in the environment's cluster
  • Persistent disks used by the task queue
  • Container images of environment components stored in Artifact Registryrepositories

For specific details about how the data is encrypted, see:

Data not protected with CMEK encryption

Cloud Monitoring does not support CMEK encryption. The name of yourenvironment and names of DAGs are stored in the Monitoringdatabase in the encrypted form using Google-owned and Google-managed encryption keys.

Cloud Composer stores the following information protected withGoogle-owned and Google-managed encryption keys, not customer-managed keys:

  • Environment name
  • Airflow configuration overrides

  • Environment variables

  • Labels

  • The names of some parameters stored by Cloud Composer can includea sub-string of environment's name.

Warning: If you use acustom environment's bucket, Cloud Composer doesnot alter the encryption settings of the bucket that you specify and thus doesnot encrypt it with a CMEK key (even if the project's policiesrequire encryption). Specifically, the custom bucket's encryption is notchanged if your environment uses CMEK encryption. Your custom bucket followspolicies of the project that it belongs to. You canmanually encrypt the environment's bucket with CMEK keys beforeyou create an environment.

Rotation of CMEK keys for Cloud Composer

After you configure encryption in your environment using CMEK keys, youmight also want to consider rotating these keys on a regular basis as describedinKMS documentation.

When you rotate a CMEK key, data encrypted with previous key versions isnot automatically re-encrypted with the new key version.For more information, seeRe-encrypting data.

Specifically, this applies to:

  • Objects stored in the environment's bucket.
  • Data stored in the Airflow database.
  • Allother data objects encrypted with CMEK in theCloud Composer environment.
  • Container images of environment components stored inArtifact Registry repositories.

Use a customer-managed encryption key for your environment

Warning: If you create an environment with a customer-managed encryption keyand later disable or delete the key, the environment becomesunusable. Itisnot possible to recover an environment if its encryption key is deletedor disabled.

Before you begin

Step 1. Create a customer-managed encryption key

Follow the steps outlined inCreating symmetric encryption keys to create a key in theregion where your environment is located.

Step 2. Grant roles to service agents

Important: If your project has aDomain restricted sharingpolicy that excludes addresses from the google.com domain. You must turn off(or expand) this policy to grant the required roles to service agents.After you grant the roles, you can revert the policy changes. This policy doesnot affect the creation of environments.

Console

Skip this step. You grant permissions toservice agents when youspecify a key for your environment.

gcloud

The followingservice agents must havetheCloud KMS CryptoKey Encrypter/Decrypter role on the key that youuse for your environment.

ReplacePROJECT_NUMBER with yourproject's number.

Service agent nameService account emailAPI service name
Cloud Composer Service Agentservice-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.comcomposer.googleapis.com
Cloud Storage Service Agentservice-PROJECT_NUMBER@gs-project-accounts.iam.gserviceaccount.comGrant encrypt/decrypt permissions withgcloud storage service-agent --authorize-cmek

  1. (If required) If some of these service accounts are not present in yourproject, it means that an identity for this service is not yet created.This can happen, for example, if you did not yet create anyCloud Composer environments in your project.

    To add these service accounts, create identities for the listed serviceswith the following command:

    gcloudbetaservicesidentitycreate\--service=API_SERVICE_NAME

    ReplaceAPI_SERVICE_NAME with theAPI service name of a servicethat does not have a service account in your project.

    Example:

    gcloudbetaservicesidentitycreate\--service=composer.googleapis.com

  2. Grant permissions to service agents:

    1. Grant the role to Cloud Composer Service Agent:

      gcloudkmskeysadd-iam-policy-bindingKEY_NAME\--locationKEY_LOCATION\--keyringKEY_RING_NAME\--member=serviceAccount:service-PROJECT_NUMBER@cloudcomposer-accounts.iam.gserviceaccount.com\--role=roles/cloudkms.cryptoKeyEncrypterDecrypter\--projectKEY_PROJECT_ID

    2. Grant encrypt/decrypt permissions toCloud Storage Service Agent.You can skip this step if you useacustom environment's bucket.

      gcloudstorageservice-agent\--authorize-cmek=projects/KEY_PROJECT_ID/locations/KEY_LOCATION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME

    Replace:

    • PROJECT_ID with yourproject's ID.
    • KEY_PROJECT_ID with the ID of the project that stores yourcustomer-managed key. If you use a key from another project, thisvalue is different from your project's ID. If you use a key from thesame project, this value is your project's ID.
    • PROJECT_NUMBER with yourproject's number.
    • KEY_LOCATION with your customer-managed key location. This locationmust be the same as your environment's location.
    • KEY_NAME with your customer-managed key name.
    • KEY_RING_NAME with the keyring that stores your customer-managed key.

    To get these values, you can rungcloud projects describe,gcloud kms keyrings list, andgcloud kms keys describe commands.

Step 3. Create an environment with CMEK

After youcreate a customer-managed encryption key, you can useit to create Cloud Composer environments.

Console

When youcreate an environment:

  1. Expand theAdvanced configuration section. In theData encryption section, selectCloud Key Management Service key.

  2. In theSelect a Cloud Key Management Service key drop-down list, select your key.

  3. If additional setup is required, a message appears to inform you. Inthis case:

    1. ClickOpen wizard.

    2. In thePrepare CMEK key for usage in Cloud Composerdialog, view the list ofservice agentsthat must have theCloud KMS CryptoKey Encrypter/Decrypter roleon the key.

    3. To grant the required roles and permissions, clickGrant.

    Important: If your project has aDomain restricted sharingpolicy that excludes addresses from the google.com domain. You must turn off(or expand) this policy to grant the required roles to service agents.After you grant the roles, you can revert the policy changes. This policy doesnot affect the creation of environments.

gcloud

The--kms-key argument specifies a customer-managed enryption key for yourenvironment.

For more information about creating environments, seeCreate environments. For example, you might want to specifyother parameters for your environment.

gcloudcomposerenvironmentscreateENVIRONMENT_NAME\--locationLOCATION\--image-versionIMAGE_VERSION\--kms-keyprojects/KEY_PROJECT_ID/locations/KEY_LOCATION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME

Replace:

  • ENVIRONMENT_NAME with the name of the environment.
  • IMAGE_VERSION with the name of the Cloud Composer image.
  • KEY_PROJECT_ID with the ID of the project where the key is located.If you use a key from another project, this value is different from yourproject's ID. If you use a key from the same project, this value is yourproject's ID.
  • LOCATION with the region where the environment is located.
  • KEY_LOCATION with the location of your customer-managed key. Thislocation must be the same as your environment's location.
  • KEY_NAME with your customer-managed key name.
  • KEY_RING_NAME with the keyring that stores your customer-managed key.

Example:

gcloudcomposerenvironmentscreateexample-environment\--locationus-central1\--image-versioncomposer-3-airflow-2.10.5-build.23\--kms-keyprojects/example-project/locations/us-central1/keyRings/example-key-ring/cryptoKeys/example-key

View the environment's encryption configuration

You can view encryption configuration for an existing environment:

Console

  1. In Google Cloud console, go to theEnvironments page.

    Go to Environments

  2. In the list of environments, click the name of your environment.TheEnvironment details page opens.

  3. Go to theEnvironment configuration tab.

  4. Details about encryption are listed in theData encryption key item.

gcloud

Run the followinggcloud command to view the encryption configuration

gcloudcomposerenvironmentsdescribe\ENVIRONMENT_NAME\--locationLOCATION\--format="value(config.encryptionConfig)"

Replace:

  • ENVIRONMENT_NAME with the name of the environment.
  • LOCATION with the region where the environment is located.

Example:

gcloudcomposerenvironmentsdescribe\example-environment\--locationus-central1\--format="value(config.encryptionConfig)"

Use CMEK for Cloud Composer logs

Cloud Logging supports encrypting Log storage with CMEK keys. We recommendto use the standard CMEK procedure for encrypting logs with CMEK keys.

To encrypt logs with CMEK keys, follow the instructions outlined inManage the keys that protect Logging storage data.

Deprecated: The following instructions about redirectingCloud Composer logs to a CMEK-encrypted Cloud Storagebucket are deprecated. If you use this approach in your project, considerswitching to encrypting Log storage with CMEK keys instead.

Redirect Cloud Composer logs to a CMEK-encrypted Cloud Storage bucket

If you expect your logs to contain sensitive data, you might want toredirect Cloud Composer logs to a CMEK-encryptedCloud Storage bucket by usingLog Router. Doing so prevents yourlogs from being sent to Monitoring.

If you require support from Cloud Customer Care, you might need to grant Googlesupport engineers access to the Cloud Composer logs stored inCloud Storage.

Note:Audit logs cannot be excluded. They are alwayssent to the default storage.

gcloud

  1. Create a new Cloud Storage bucket to store the logs.

    gcloudstoragebucketscreategs://BUCKET_NAME--location=LOCATION

    Replace:

    • LOCATION with the region where the environment is located.
    • BUCKET_NAME with the name for the bucket.

    Example:

    gcloudstoragebucketscreategs://composer-logs-us-central1-example-environment--location=us-central1

  2. Encrypt the bucket with your CMEK key.

    gcloudstoragebucketsupdategs://BUCKET_NAME\--default-encryption-key=projects/KEY_PROJECT_ID/locations/KEY_LOCATION/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME

    Replace:

    • KEY_PROJECT_ID with the ID of the project where the key is located.If you use a key from another project, this value is different fromyour project's ID. If you use a key from the same project, this valueis your project's ID.
    • KEY_LOCATION with the location of your customer-managed key. Thislocation must be the same as your environment's location.
    • KEY_RING_NAME with the keyring that stores your customer-managed key.
    • KEY_NAME with your customer-managed key name.
    • BUCKET_NAME with the name of the bucket.

    Example:

    gcloudstoragebucketsupdategs://composer-logs-us-central1-example-environment\--default-encryption-key=projects/example-project/locations/us-central1/keyRings/example-key-ring/cryptoKeys/example-key

  3. Create a new log sink.

    gcloudloggingsinkscreate\composer-log-sink-ENVIRONMENT_NAME\storage.googleapis.com/BUCKET_NAME\--log-filter"resource.type=cloud_composer_environment AND resource.labels.environment_name=ENVIRONMENT_NAME AND resource.labels.location=LOCATION"

    Replace:

    • ENVIRONMENT_NAME with the name of the environment.
    • LOCATION with the region where the environment is located.
    • BUCKET_NAME with the name of the bucket.

    Example:

    gcloudloggingsinkscreate\composer-log-sink-example-environment\storage.googleapis.com/composer-logs-us-central1-example-environment\--log-filter"resource.type=cloud_composer_environment AND resource.labels.environment_name=example-environment AND resource.labels.location=us-central1"

  4. Grant theStorage Object Creator role to the service account for thisbucket. The service account is shown in the result of the previouscommand).

    gcloudprojectsadd-iam-policy-binding\PROJECT_ID\--member="serviceAccount:LOGGING_SERVICE_AGENT"\--role="roles/storage.objectCreator"\--condition=None

    Replace:

    • PROJECT_ID with theProject ID.
    • LOGGING_SERVICE_AGENT with the Logging serviceagent account for this bucket. The name of this account is obtainedon the previous step.

    Example:

    gcloudprojectsadd-iam-policy-binding\example-project\--member="serviceAccount:example-sa@gcp-sa-logging.iam.gserviceaccount.com"\--role="roles/storage.objectCreator"\--condition=None

  5. Exclude the logs for your new environment fromMonitoring.

    gcloudbetaloggingsinksupdate_Default\--add-exclusionname=ENVIRONMENT_NAME-exclusion,filter="resource.type=cloud_composer_environment AND resource.labels.environment_name=ENVIRONMENT_NAME AND resource.labels.location=LOCATION"

    Replace:

    • ENVIRONMENT_NAME with the name of the environment.
    • LOCATION with the region where the environment is located.

    Example:

    gcloudbetaloggingsinksupdate_Default\--add-exclusionname=example-environment-exclusion,filter="resource.type=cloud_composer_environment AND resource.labels.environment_name=example-environment AND resource.labels.location=us-central1"

  6. Add organization-level CMEK encryption to the Log Router.

    gcloudloggingcmek-settingsdescribe\--organization=ORGANIZATION_ID
    gcloudkmskeysadd-iam-policy-binding\--project=KEY_PROJECT_ID\--memberLOGGING_SERVICE_AGENT\--roleroles/cloudkms.cryptoKeyEncrypterDecrypter\--location=KEY_LOCATION\--keyring=KEY_RING_NAME\KEY_NAME
    gcloudloggingcmek-settingsupdate\--organization=ORGANIZATION_ID\--kms-project=KEY_PROJECT_ID\--kms-keyring=KEY_RING_NAME\--kms-location=KEY_LOCATION\--kms-key-name=KEY_NAME

    Replace:

    • ORGANIZATION_ID with yourorganization ID.
    • KEY_PROJECT_ID with the ID of the project where the key is located.If you use a key from another project, this value is different fromyour project's ID. If you use a key from the same project, this valueis your project's ID.
    • KEY_RING_NAME with the keyring that stores your customer-managed key.
    • KEY_LOCATION with the location of your customer-managed key. Thislocation must be the same as your environment's location.
    • KEY_NAME with your customer-managed key name.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.