This document describes Workload Identity Federation for GKE, including how itworks, how enabling it affects your GKE clusters, and how togrant roles to Kubernetes entities in Identity and Access Management policies. In most cases,Workload Identity Federation for GKE is the recommended way to help secure and manage how yourworkloads that run on GKE access Google Cloud services.

This document is for Security specialists and Operators who manageworkloads on GKE that require access to other Google Cloudservices. To learn more about common roles and example tasks that we referencein Google Cloud content, seeCommon GKE user roles and tasks.

Terminology

This page distinguishes betweenKubernetes service accounts andIdentity and Access Management (IAM) service accounts.

Kubernetes service accounts

Kubernetes resources that provide an identity for processesrunning in your GKE pods.

IAM service accounts

Google Cloud resources that allow applications to make authorized calls toGoogle Cloud APIs.

What is Workload Identity Federation for GKE?

Applications running on GKE might need access toGoogle Cloud APIs such as Compute Engine API, BigQuery Storage API, orMachine Learning APIs.

Workload Identity Federation for GKE lets you use IAM policies to grantKubernetes workloads in your GKE cluster access to specificGoogle Cloud APIs without needing manual configuration or less securemethods like service account key files. UsingWorkload Identity Federation for GKE lets you assign distinct, fine-grained identities andauthorization for each application in your cluster.

Workload Identity Federation for GKE replaces the need to useMetadata concealment.The sensitive metadata protected by metadata concealment is also protected byWorkload Identity Federation for GKE.

Workload Identity Federation for GKE is available through IAMWorkload Identity Federation, which provides identities for workloads that runin environments inside and outside Google Cloud. You can useIAM Workload Identity Federation to securely authenticate tosupported Google Cloud APIsfrom workloads running on, for example, AWS, Azure, and self-managed Kubernetes.In GKE,Google Cloud manages the workload identity pool and provider for you anddoesn't require an external identity provider.

How Workload Identity Federation for GKE works

When you enable Workload Identity Federation for GKE on a cluster, GKE does thefollowing:

• Creates a fixedworkload identity pool for the cluster's Google Cloudproject with the following format:

  PROJECT_ID.svc.id.goog

  The workload identity pool provides a naming format that allowsIAM to understand and trust Kubernetes credentials.GKE doesn't delete this workload identity pool even if youdelete all of the clusters in your project.

• Registers the GKE cluster as an identity provider in theworkload identity pool.

• Deploys theGKE metadata server, whichintercepts credential requests from workloads, on every node.

For a list of resources that support allow policies, seeResource types that accept allow policies.

Use conditions in IAM policies

You can also limit the scope of the access by settingconditions in yourallow policies. Conditions are an extensible method of specifying when an allowpolicy should apply. For example, you could use conditions to grant temporaryaccess to a workload on a specific Google Cloud resource, eliminating theneed to manage that access manually.

Conditions might also be useful if you set your allow policies at the project,folder, or organization level instead of on specific resources likeSecret Manager secrets or Cloud Storage buckets.

To add a condition to your allow policy, use the following resources:

• Manage conditional role bindings:Add, modify, or remove conditional role bindings.
• Configure temporary access: Useconditions to set expiring access to Google Cloud resources in allowpolicies.
• Tags and conditional access: Use conditionsto only apply allow policies when resources have specific tags.

The following example expressions are for common scenarios in which you might useconditions. For a list of available attributes in expressions, seeAttribute reference for IAM Conditions.

request.time< timestamp('TIMESTAMP')

resource.matchTag('TAG_KEY', 'TAG_VALUE')

Reference Kubernetes resources in IAM policies

In your IAM policy, you refer to a Kubernetes resource by usingan IAMprincipal identifier to select the resource. Thisidentifier has the following syntax:

PREFIX://iam.googleapis.com/projects/1234567890/locations/global/workloadIdentityPools/example-project.svc.id.goog/SELECTOR

In this example, consider the following fields:

• PREFIX: must beprincipal orprincipalSet dependingon the resource that you select.principal is for a specific resource, likea single ServiceAccount.principalSet is for multiple resources that belong tothe specified resource, like all Pods in a specific cluster.
• SELECTOR: a string that selects a principal type. Forexample,kubernetes.serviceaccount.uid/SERVICEACCOUNT_UIDselects a ServiceAccount by its UID.

The following table shows the supported principal types in GKE:

principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/subject/ns/NAMESPACE/sa/SERVICEACCOUNT

principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/kubernetes.serviceaccount.uid/SERVICEACCOUNT_UID

principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/namespace/NAMESPACE

principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/PROJECT_ID.svc.id.goog/kubernetes.cluster/https://container.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/clusters/CLUSTER_NAME

Credential flow

When a workload sends a request to access a Google Cloud API, for examplewhen using a Google Cloud client library, the following authenticationsteps occur:

1. Application default credentials (ADC)requests a Google Cloud access token from the Compute Enginemetadata server that runs on the VM.
2. The GKE metadata server intercepts the token request and asksthe Kubernetes API server for a Kubernetes ServiceAccount token thatidentifies the requesting workload. This credential is a JSON web token(JWT) that's signed by the API server.
3. The GKE metadata server usesSecurity Token Service to exchange the JWT for ashort-livedfederated access tokenthat references the identity of the Kubernetes workload.

The federated access token that's returned by Security Token Service might havelimitations when trying to access some Google Cloud services, as describedinSupported products and limitations.If your selected Google Cloud service has limitations, you can optionallyconfigure service account impersonation. This method results in an access tokenfor an IAM service account that your workload can use to accessthe target service. For details, seelink Kubernetes ServiceAccounts to IAM.

The workload can then access any Google Cloud APIs that theIAM principal identifier of the workload can access.

Quota for the Exchange Token API in Security Token Service

The Exchange Token API in Security Token Service has aquota limitof 6,000 requests per minute. If you seeQUOTA_EXCEEDED errors, you canrequest an increase for theToken exchange requests per minute quota throughtheQuotas & System Limitspage.

This identity sameness also means that you must be able to trust every clusterin a specific workload identity pool. For example, if a new cluster, Cluster Cin the previous example was owned by an untrusted team, they could create abackend namespace and access Google Cloud APIs using theback-ksaServiceAccount, just like Cluster A and Cluster B.

To avoid untrusted access, place your clusters in separate projects to ensurethat they get different workload identity pools, or ensure that the namespacenames are distinct from each other to avoid a common principal identifier.

GKE metadata server

When you enable Workload Identity Federation for GKE for a cluster, every node in the clusterstores metadata on theGKE metadata server. TheGKE metadata server is a subset of theCompute Engine metadata server endpointsrequired for Kubernetes workloads.

The GKE metadata server runs as a DaemonSet, with one Pod onevery Linux node or a native Windows service on every Windows node in thecluster. The metadata server intercepts HTTP requests tohttp://metadata.google.internal(169.254.169.254:80). For example, theGET/computeMetadata/v1/instance/service-accounts/default/token request retrieves atoken for the IAM service account that the Pod is configured to impersonate.Traffic to the GKE metadata server never leaves the VM instancethat hosts the Pod.

Token lifetime

By default, the access token returned has a lifetime of 1 hour (3,600 seconds).To reduce client latency, the GKE metadata server caches theaccess tokens. In some situations, the cached token that the metadata serverreturns might be close to its expiry time.

Cloud Client Libraries have built-inlogic that, by default, checks whether the access token expires in the next 3minutes and 45 seconds. If the token is within the expiry period,GKE refreshes the token. Consecutive API calls can use therefreshed token.

If you use your own code to directly access Google Cloud APIs, implementsimilar logic to handle token expiry. Your code should do the following:

1. Check whether the access token expires after a period of 3 minutes and 45seconds. Theexp parameter in the token payload indicates the tokenexpiration timestamp.
2. If the token is set to expire in the next 3 minutes and 45 seconds, make atoken request.

The following tables describe the subset of Compute Engine metadata serverendpoints available with the GKE metadata server. For a full listof endpoints available in the Compute Engine metadata server, seeDefault VM metadata values.

Instance metadata

Instance metadata is stored under the following directory.

http://metadata.google.internal/computeMetadata/v1/instance/

Instance attributes

Instance attributes are stored under the following directory.

http://metadata.google.internal/computeMetadata/v1/instance/attributes/

The attributes listed in the table are the only supported attributes. If youattempt to access any unsupported attributes, thegke-metadata-server Pod inthekube-system namespace generates and logs a404 error.The error is similar to the following:

HTTP/404: generic::not_found: no child "", Reason: "NOT_FOUND", UserMessage: "Not Found"

If you are usingistio-proxy, you will encounter an error message like the following:

Error fetching GCP Metadata property gcp_gce_instance_template: metadata: GCE metadata "instance/attributes/UNSUPPORTED_ATTRIBUTE" not defined

Project metadata

Cluster project metadata is stored under the following directory.

http://metadata.google.internal/computeMetadata/v1/project/

Restrictions of Workload Identity Federation for GKE

• You can't change the name of the workload identity pool thatGKE creates for your Google Cloud project.

• If youlink Kubernetes ServiceAccounts to IAM service accountsto configure Workload Identity Federation for GKE, the GKE metadata serverreturns a value ofSERVICEACCOUNT_NAME.svc.id.googas the service account identifier. This identifier doesn't use the standardIAM principal identifier syntax, which might cause errors insome programmatic operations. To get the service account identifier as anIAM principal identifier, add theiam.gke.io/return-principal-id-as-email: "true" annotation to your KubernetesServiceAccount.

• When GKE enables theGKE metadata serveron a node pool, Pods can no longer access theCompute Engine metadata server.Instead, the GKE metadata server intercepts requests made fromthese pods to metadata endpoints, with the exception of Pods running on thehost network.

  Caution: Pods running on thehost network(hostNetwork: true) don't use Workload Identity Federation for GKE. GKEautomatically routes requests from these Pods to the Compute Engine metadataserver.

• When usingCloud Storage FUSE CSI driver with Standard GKE clusters with version1.33.3-gke.1226000 or later, the Pods running on the host network (hostNetwork: true) can authenticate using their own Kubernetes Service Account. For more information, seeConfigure access for Pods with host network.

• The GKE metadata server takes a few seconds to start acceptingrequests on a newly created Pod. Therefore, attempts toauthenticate using Workload Identity Federation for GKE within the first few seconds of a Pod's life mightfail. Retrying the call will resolve the problem. SeeTroubleshootingfor more details.

• GKE built-in logging and monitoring agents continue to use thenode's service account.

• Workload Identity Federation for GKE requiresmanual setup for Knative serving tocontinue releasing request metrics.

• Workload Identity Federation for GKE sets a limit of 500 concurrent connections to the GKEmetadata server for each node. Additional concurrent calls that exceed thislimit are placed in a wait queue for later processing. This queuing mechanismmight lead toHTTP/499 errors if the client timeout is reached before theGKE metadata server can process the request.

• The GKE metadata server uses memory resources proportional tothe total number of Kubernetes service accounts in your cluster. If yourcluster has more than 3000 Kubernetes service accounts, the kubelet mightterminate the metadata server Pods. For mitigations, refer toTroubleshooting.

• Workload Identity Federation for GKE operates within a VPC Service Controls perimeter, allowing access toresources within it. However,VPC Service Controlsdoes not enforce access control for cross-perimeter requests based on thesefederated identities. You can useservice account impersonation to accessresources in a different perimeter.

Alternatives to Workload Identity Federation for GKE

You can use one of the following alternatives to Workload Identity Federation for GKE to accessGoogle Cloud APIs from GKE. We recommend that you useWorkload Identity Federation for GKE because these alternatives require you to make certainsecurity compromises.

• Use theCompute Engine default service accountof your nodes. You can run node pools as anyIAM service accountin your project. If you don't specify a service account during node poolcreation, GKE uses theCompute Engine default service accountfor the project. The Compute Engine service account is shared by all workloadsdeployed on that node. This can result in over-provisioning of permissions, whichviolates the principle of least privilege and is inappropriate for multi-tenantclusters.

• Export service account keys and storethem asKubernetes Secretsthat you mount to your Pods as volumes.

  Caution: Service account keys are a security risk if not managed correctly. You should choose a more secure alternative to service account keyswhenever possible. If you must authenticate with a service account key, you are responsible for thesecurity of the private key and for other operations described by Best practices for managing service account keys.If you are prevented from creating a service account key, service account key creation mightbe disabled for your organization. For more information, see Managing secure-by-default organization resources.

  If you acquired the service account key from an external source, you must validate it before use.For more information, see Security requirements for externally sourced credentials.

What's next

• Learn how to enable and configure Workload Identity Federation for GKE.
• Learn about the Compute Engine metadata server.
• Learn about Workload Identity Federation in other environments.
• Provide Workload Identity Federation support for clusters in fleets by using fleet workload identity.