This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of theService Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see thelaunch stage descriptions.

For information about access to this release, see theaccess request page.

This document describes how to set up automatic provisioning and lifecyclemanagement of managed workload identities for Compute Engine. Youconfigure CA pools to issue certificates usingCertificate Authority Service (CA),which is a highly-available, scalable Google Cloud service that simplifies andautomates the deployment, management, and security of CA services. Each VMis provisioned with X.509 credentials from the configured CA pool. Thesecredentials can then be used to establish mTLS connections.

With managed workload identities for Compute Engine, you can implement mutuallyauthenticated and encrypted communications between any two Compute Engine VMs.Workload applications running on the configured VMs can use the X.509credentials for per-VMmTLS.These mTLS certificates are automatically rotated and managed for you byCertificate Authority Service.

Before you begin

• Request access to the managed workload identity preview.

  Warning: Do not proceed until you have received confirmation that your project has been added to the allowlist.

• Configure the Google Cloud CLI.

  Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

  gcloudinit

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.Note: You can run the gcloud CLI in the Google Cloud console without installing the Google Cloud CLI. To run the gcloud CLI in the Google Cloud console,use Cloud Shell.

• Configure Google Cloud CLI to use the allowlisted project for billing and quota.

    gcloud config set billing/quota_projectPROJECT_ID

  ReplacePROJECT_ID with the ID of the project that was added to the allowlist for the managed workload identity preview.

• Review theManaged workload identities overview documentation.

Enable the Compute Engine API:

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role (roles/serviceusage.serviceUsageAdmin), which contains theserviceusage.services.enable permission.Learn how to grant roles.

gcloudservicesenablecompute.googleapis.com

Required roles

To get the permissions that you need to create VMs that use managed workload identity certificatesfor authentication to other workloads, ask your administrator to grant you the following IAM roles on the project:

• Compute Instance Admin (v1) (roles/compute.instanceAdmin.v1)
• Service Account User (roles/iam.serviceAccountUser)

For more information about granting roles, seeManage access to projects, folders, and organizations.

You might also be able to get the required permissions throughcustom roles or otherpredefined roles.

Overview

To use managed workload identities for your applications, you must perform thefollowing tasks:

1. Security Administrator:

   • Create managed workload identitiesin a workload identity pool.
   • Define the workload attestation policy.
   • Configure Certificate Authority Service to issue certificates for managed workload identities.
   • Authorize managed workload identities to request certificates from the CA pool.
   • Define the trust and certificate issuance config.

2. Compute Administrator:

   • Get the configuration file to upload the partner metadata.
   • Enable managed workload identities for workloads running in Compute Engine:
     • Forindividuals VMs
     • Formanaged-instance groups (MIGs).
   • Access workload credentials on a Linux VM.

Configure managed workload identities in Identity and Access Management

• Follow the instructions inConfigure managed workload identities authentication.

  These instructions detail how to complete the following:

  • Create a workload identity pool.
  • Create namespaces in the workload identity pool. You use the namespaces tocreate administrative boundaries for your managed workload identities, forexample, a namespace for each of the applications owned by yourorganization.
  • Create a managed workload identity in a namespace in the workload identitypool. For example, you might create a namespace for an application andcreate managed identities within that namespace for the microservices thatsupport that application.
  • Create a service account. Compute Engine VMs can be authorized toreceive a managed workload identity based on the Google Cloud serviceaccount attached to the VM.
  • Create a workload attestation policy that allows your workload to be issuedcredentials for the managed workload identity. To be issues credentialsfor the managed workload identity, the workload must be within a specifiedproject and have the service account attached.
  • Configure Certificate Authority Service to issue certificates for managed workloadidentities:
    • Configure the root CA pool
    • Configure the subordinate CAs
    • Authorize managed workload identities to request certificatesfrom the CA pool

Get the configuration file to upload the partner metadata

Your security administrator creates a JSON file that contains the following:

• The configuration for the workload identity
• Thecertificate issuance config
• Thetrust config

This file should be namedCONFIGS.json. You use this file when creating aninstance template for MIGs or when creating an individual VM.

{"wc.compute.googleapis.com":{"entries":{"certificate-issuance-config":{"primary_certificate_authority_config":{"certificate_authority_config":{"ca_pool":"projects/PROJECT_ID/locations/SUBORDINATE_CA_POOL_REGION/caPools/SUBORDINATE_CA_POOL_ID"}},"key_algorithm":"rsa-2048"},"trust-config":{"POOL_ID.global.PROJECT_NUMBER.workload.id.goog":{"trust_anchors":[{"ca_pool":"projects/PROJECT_ID/locations/SUBORDINATE_CA_POOL_REGION/caPools/SUBORDINATE_CA_POOL_ID"}]}}}},"iam.googleapis.com":{"entries":{"workload-identity":"spiffe://POOL_ID.global.PROJECT_NUMBER.workload.id.goog/ns/NAMESPACE_ID/sa/MANAGED_IDENTITY_ID"}}}

Amanaged instance group (MIG)is a group of virtual machine (VM) instancesthat you treat as a single entity. Each VM in a MIG is created using aninstance template. To enable the VMs in the MIG to use managed workload identities,you specify the configuration in the instance template.

Create an instance template

Create aninstance template with themanaged workload identities feature enabled. Then use this template to create amanaged instance group (MIG).

gcloud beta compute instance-templates createINSTANCE_TEMPLATE_NAME \    --service-accountSERVICE_ACCOUNT_NAME@PROJECT_ID.iam.gserviceaccount.com \    --metadata enable-workload-certificate=true \    --partner-metadata-from-fileCONFIGS.json

For more information, seeCreate instance templates.

Create a managed instance group from the template

Create amanaged instance groupthat uses an instance template that enables the managed workload identities.For details on how to create the instance template, seeCreate an instance template.

gcloud compute instance-groups managed createINSTANCE_GROUP_NAME \    --size=SIZE \    --template=INSTANCE_TEMPLATE_NAME \    --zone=ZONE

For detailed information about creating MIGs, seeBasic scenarios for creating managed instance groups (MIGs)

Enable managed workload identities for individual VMs

You can enable managed workload identities for a VM either when creating the VMor by updating the partner metadata for an existing VM.

Create VMs with managed workload identities enabled

When creating a VM, to enable the managed workload identities feature for theVM, you must do the following:

• Specify a service account for the VM to use
• Set theenable-workload-certificate metadata attribute totrue

• Specify the certificate issuance config and trust configinformation aspartner metadata.

Enable managed workload identities on existing VMs

To enable managed workload identities for an existing VM, update the VM toconfigure the following:

• If the VM doesn't already have an attached service account, then create and attach a service account to the VM.
• Set theenable-workload-certificate metadata attribute totrue.
• Specify the certificate issuance config and trust config information aspartner metadata.

• Restart the VM.

Access workload credentials on a Linux VM

After successfully configuring workload to workload authentication using mTLS,you can access the issued credentials on your VM.

There are two ways to access the Compute Engine managed workload identitycredentials and the associated trust bundle:

• The file system on the VM
• The Compute Engine metadata server

Access the workload credentials and trust bundle using the file system on the VM

This method puts the X.509 credentials and the trust bundle at a specific pathinside the VM's file system. Applications can directly read the credentialsand the trust bundle from the file system. For examples of how to retrieve thecredentials, see the following examples on GitHub:

• Go
• Java
• C++

The VM must run the Compute Engine guest agent version 20231103.01 or later.Use the following command to check the version of the Compute Engine guestagent on your VM:

gcloud beta compute instances get-serial-port-outputINSTANCE_NAME \   --zone=ZONE | grep "GCE Agent Started"

If the version of the guest agent is less than 20231103.01, you can updateit by following the instructions inUpdating the guest environment.

To make the workload credentials and trust bundle available in the file systemof a VM, complete the following steps:

1. Install or update theCompute Engine guest agentto version 20231103.01 or later. The guest agent does the following:

   • Automatically retrieves the credentials and the trust bundle from theCompute Engine metadata server.
   • Ensures atomic writes to the file system while updating the X.509certificate and the corresponding private key.
   • Automatically refreshes the credentials and the trust bundle, for example,when the mTLS certificates are rotated.

2. After you install or update the Compute Engine guest agent on the guest OS,the workload refresh job creates the directory/var/run/secrets/workload-spiffe-credentials and sets the directorypermissions to0755 (rwxr-xr-x).

   The directory contains the following files created with0644 (rw-r--r--)permissions:

   • private_key.pem: a PEM-formatted private key
   • certificates.pem: a bundle of PEM-formatted X.509 certificates thatcan be presented to other VMs as the client certificate chain, or usedas a server certificate chain.

   • ca_certificates.pem: a bundle of PEM-formatted X.509 certificates touse as trust anchors when validating the certificates of peers.

     Note:ca_certificates.pem contains certificates for the local SPIFFEtrust domain for the VM, which is:

     spiffe://POOL_ID.global.PROJECT_NUMBER.workload.id.goog

   • config_status: a log file containing error messages.

3. Applications can read the certificates, private key and the trust bundle fromthe file system directly to establish mTLS connections.

   Note: Your application code must verify that the private key and certificatefile are consistent before using them to establish mTLS connection. Amismatch between the private key and certificate may happen if theCompute Engine guest agent refreshes the credentials from MDS in the timebetween the application reading theprivate_key.pem file and thecertificates.pem file or the other way around. For example, applicationswritten in Golang can use thetls.LoadX509KeyPair methodto check for consistency between the public and private key pair.

Access the workload credentials and trust bundle using the metadata server

An application running on a Compute Engine VM can directlyquery the metadata server endpointsand retrieve the credentials and the trust bundle. Theapplication is responsible for periodically checking the metadata serverendpoints for new credentials and updates to the trust bundle.

The Compute Engine metadata server exposes three HTTP endpoints to enablethe use of the managed workload identities feature by applications runninginside the VM.

• gce-workload-certificates/config-status: An endpoint containing any errorsin the configuration values provided through the VM metadata.
• gce-workload-certificates/workload-identities: An endpoint of identitiesmanaged by the Compute Engine control plane. This endpoint contains theX.509 certificate and the private key for the VM's trust domain.
• gce-workload-certificates/trust-anchors: An endpoint containing a set oftrusted certificates for peer X.509 certificate chain validation.

To learn more about querying the metadata for a VM instance, seeAbout VM metadata.

To access the workload credentials and trust bundle using the metadata server,your application should do the following:

1. Query thegce-workload-certificates/config-status endpoint. Ensure thatthe HTTP response code is200 and that the response doesn't contain anypartnerMetadataConfigsErrors errors. If such errors exist, update theappropriate configuration with valid values by following the steps discussedinUpdate certificate issuance and trust config.

   To check the value, you can run the following command on the VM:

   curl"http://metadata.google.internal/computeMetadata/v1/instance/gce-workload-certificates/config-status"-H"Metadata-Flavor: Google"

   Theconfig-status endpoint returns a JSON response with followingstructure:

   {    "partnerMetadataConfigsErrors": {        "errors": {  // A map of errors keyed by attribute name.            "ATTRIBUTE_NAME" : "ERROR_DETAILS",            ...        }    }}

2. Query thegce-workload-certificates/workload-identities endpoint. Ensurethat the HTTP response code is200. The endpoint returns a JSON responsewith following structure:

   { "workloadCredentials": {  // Credentials for the VM's trust domains   "spiffe://POOL_ID.global.PROJECT_NUMBER.workload.id.goog/ns/NAMESPACE_ID/sa/MANAGED_IDENTITY_ID": {      "certificatePem" : "X.509 certificate or certificate chain",      "privateKeyPem" : "Private for X.509 leaf certificate"   } }}

   Extract thecertificatePem and theprivateKeyPem. It is critical thatboth values are read from the same response to avoid mismatch between theprivate and public key in case the managed workload identities were refreshed bythe Compute Engine infrastructure.

3. Query thegce-workload-certificates/trust-anchors endpoint. Ensure thatthe HTTP response code is200. The response will only contain the trustanchors for the SPIFFE trust domain, if specified. Otherwise, the queryreturns an error. Thetrust-anchors endpoint returns a JSON response withfollowing structure:

   {    "trustAnchors": {  // Trust bundle for the VM's trust domains        "POOL_ID.global.PROJECT_NUMBER.workload.id.goog": {            "trustAnchorsPem" : "Trust bundle containing the X.509            roots certificates"        }    }}

   The contents oftrustAnchorsPem contains the trust bundle that can thenbe used to authenticate peer X.509 credentials when establishing a mTLSconnection.

Refreshing the credentials and trust bundle

The Compute Engine control plane automatically rotates the managed workloadidentity credentials and the trust anchors periodically.

If your applications use the file system to access the workload credentials andtrust bundle, the Compute Engine guest agent automatically refreshes thecredentials and the trust bundle, for example, when the mTLS certificates arerotated.

If your applications query the metadata server, then the applications running ona VM must periodically query the metadata server endpoints to get the latest setof managed workload identity credentials and the trust bundle. Failure to do socan break applications due to certificate expiration or changes to the trustbundle, which can cause mTLS connection establishment to fail. Googlerecommends that applications query the metadata server for the managedworkload identity credentials and the trust bundle every 5 minutes.

Update certificate issuance and trust config

You can modify the certificate issuance config and trust config for a VM thatuses managed workload identities.

Update the instance template for a managed instance group

To update the certificate issuance config and trust config values in aninstance template, you have to create a new template with the new values.Therefore, updating the certificate issuance config and trust config forexisting managed instance groups (MIGs) is not supported.

Update individual Compute Engine VMs

To update the certificate issuance config and trust config, update thecontents of theCONFIGS.json file and use thegcloud beta compute instances update commandto apply the updates:

gcloudbetacomputeinstancesupdateINSTANCE_NAME\--partner-metadata-from-fileFILENAME.json

Replace the following:

• INSTANCE_NAME: The name of the VM that you are updating theconfig values for
• FILENAME: The name of the modified config file, for exampleCONFIGS.json

Troubleshoot

To find methods for diagnosing and resolving common errors related toretrieving workload credentials, refer to theTroubleshoot workload to workload authenticationdocumentation.

What's next

• Learn more about the following concepts:
  • Authenticate to Compute Engine
  • Configure managed workload identity authentication
  • Basic scenarios for creating managed instance groups (MIGs)
  • Compute Engine guest agent