This page explains how to encrypt sensitive data in Google-provided templateparameters, by using encryption keys with Cloud KMS.

Overview

Some Google-provided templates enable you to encrypt sensitive data in thetemplate parameters, such as usernames, passwords, JDBC connection strings, andAPI keys. When supported, these templates include a parameter to specify theCloud KMS encryption key, such as:

• KMSEncryptionKey
• tokenKMSEncryptionKey
• apiKeyKMSEncryptionKey

To use Cloud KMS encryption keys with these templates, perform thefollowing steps:

1. Grant the Dataflowworker service account theCloud KMS CryptoKey Decrypter role.
2. Create an encryption key.
3. Use the key to encrypt the data.
4. Base64-encode the encrypted data.
5. When you run the template, specify the encryption key and use the encryptedparameter values.

If you specify an encryption key, you must encrypt all of the parameters thatsupport encryption. To understand which parameters can be encrypted, see thedocumentation for the specific template.

Example

The following example uses theMySQL to BigQuerytemplate.

1. Create a key ring.

   gcloudkmskeyringscreate"KEY_RING_NAME"\--location"global"

2. Create an encryption key.

   gcloudkmskeyscreate"KEY_NAME"\--location"global"\--keyring"KEY_RING_NAME"\--purpose"encryption"

3. Encrypt and base64-encode the username, password, and JDBC connection string.

   exportUSER_NAME=`echo-n"USER_NAME"\|gcloudkmsencrypt--key=quickstart--keyring=test--location=global--plaintext-file=---ciphertext-file=-\|base64-w0`exportPASSWORD=`echo-n"PASSWORD"\|gcloudkmsencrypt--key=quickstart--keyring=test--location=global--plaintext-file=---ciphertext-file=-\|base64-w0`exportCONNECTION_STRING=`echo-n"CONNECTION_STRING"\|gcloudkmsencrypt--key=quickstart--keyring=test--location=global--plaintext-file=---ciphertext-file=-\|base64-w0`

4. Run the template.

   gclouddataflowflex-templaterunmysql-job\--project=PROJECT_ID\--region=us-central1\--template-file-gcs-location=gs://dataflow-templates-us-central1/latest/flex/MySQL_to_BigQuery\--parameters\connectionURL="$CONNECTION_STRING",\query="SOURCE_SQL_QUERY",\outputTable=PROJECT_ID:DATASET.TABLE_NAME,\bigQueryLoadingTemporaryDirectory=CLOUD_STORAGE_PATH,\username="$USER_NAME",\password="$PASSWORD",\KMSEncryptionKey=projects/PROJECT_ID/locations/global/keyRings/KEY_RING_NAME/cryptoKeys/KEY_NAME

Troubleshooting

This section contains troubleshooting information for encrypting templateparameters.

Permission denied

When you run the job, you see aPERMISSION_DENIED error in the job logs,similar to the following:

PERMISSION_DENIED: Permission cloudkms.cryptoKeyVersions.useToDecrypt denied onresourceRESOURCE_PATH (or it may not exist)

To decrypt the data, the Dataflow worker service account needsthecloudkms.cryptoKeyVersions.useToDecrypt permission for the encryptionkey. Make sure the worker service account has theCloud KMS CryptoKey Decrypter role. For more information, seeDataflow security and permissions.

What's next

• Learn more aboutCloud Key Management Service.
• See the list ofGoogle-provided templates.