This page explains how network isolation and access controls work for yourGoogle Kubernetes Engine (GKE) cluster control plane and cluster nodes. This pagereplaces the page describing the concept of private clusters.

There are two aspects to consider when deciding how to configure network isolation:

• Control plane access which relates to who can access the cluster'scontrol plane.
• Cluster networking which relates to isolatingnodes.

This page shows you how to control and customize who can access the cluster'scontrol plane and cluster nodes (in a Standard cluster) or workloads(in an Autopilot cluster). This customization is relevant when you are decidingthe network configuration for your cluster. To go straight to defining yournetwork configuration, seeCustomize your network isolation.

Control plane access

In this section, you'll consider who can access your control plane.

Every GKE cluster has acontrol plane that handles Kubernetes APIrequests. The control plane runs on a virtual machine (VM) that is in aVPC network in a Google-managed project. A regional cluster hasmultiple replicas of the control plane, each of which runs on its own VM.

The control plane has two kinds of endpoints for cluster access:

• DNS-based endpoint
• IP-based endpoints

DNS-based endpoint

The DNS-based endpoint gives a unique and immutable DNS or fully qualified domain name (FQDN)for each cluster control plane. This DNS name can be used to access your controlplane for its entire lifecycle. The DNS name resolves to an endpoint that is accessible from any networkreachable by Google Cloud APIs, including on-premises or other cloud networks.Enabling the DNS-based endpoint eliminates the need for abastion host or proxy nodesto access the control plane from other VPC networks or externallocations.

To access the control plane endpoint, you need to configure IAMroles and policies, and authentication tokens. For more details on the exactpermissions required, seeCustomize your networkisolation.

In addition to IAM policies and tokens, you can also configurethe following access attributes:

• IP address or network-based controls withVPC Service Controls: to enhancesecurity for your GKE cluster control plane,VPC Service Controls adds another layer of access security. It usescontext-aware access based on attributes like network origin.

  VPC Service Controls doesn't directly support clusters with public IPaddresses for their control plane. However, GKE clusters,including both private and public clusters, gain protection fromVPC Service Controls when you access them using the DNS-based endpoint.

  You configure VPC Service Controls to protect your GKEcluster's DNS-based endpoint by includingcontainer.googleapis.com andkubernetesmetadata.googleapis.com in your service perimeter's restrictedservices list. Adding these APIs to your service perimeter will enableVPC-SC forall GKE API operations. This configuration helps ensure that yourdefined security perimeters govern access to the control plane.

  By using both IAM policies and VPC Service Controls to secureaccess to the DNS-based endpoint, you create a multi-layer security modelfor your cluster control plane. VPC Service Controls integrates withsupported Google Cloud services. It aligns your cluster's securityconfiguration with the data you host in other Google Cloud services.

• Private Service Connect or Cloud NAT: to access theDNS-based endpoint from clients that don't have public internet access. Bydefault, the DNS-based endpoint is accessible through Google CloudAPIs that are available on the public internet. To learn more, seeDNS-basedendpointin the Customize your network isolation page.

• Kubernetes authentication credentials: to authenticate to the DNS-basedendpoint by usingKubernetes ServiceAccount bearer tokensorX.509 client certificates.These authentication methods are disabled by default in GKEclusters. You can enable these methods when you configure the DNS-basedendpoint for a cluster.

  Best practice:

  ServiceAccount tokens and client certificates can add complexity andmanagement overhead to your security configuration. Unless your use caserequires one of these authentication methods,use IAM credentials to authenticate to your control plane.

IP-based endpoints

Optionally, you can also configure access to the control plane using IP-basedendpoints.

Terminology related to clusters and IP addresses

• Google Cloud external IP addresses:

  • External IP addresses assigned to any VM used by any customer hosted onGoogle Cloud. Google Cloud owns these IP addresses. To learn more,seeWhere can I find Compute Engine IP ranges?

  • External IP addresses used by Google Cloud products such asCloud Run or Cloud Run functions. Any client hostedon Google Cloud can instantiate these IP addresses. Google Cloud owns theseIP addresses.

• Google-reserved IP addresses: External IP addresses forGKE cluster management purposes. These IP addresses includeGKE managed processes and other production Google services.Google owns these IP addresses.

• GKE cluster IP address ranges: Internal IP addressesassigned to the cluster that GKE uses for the cluster'snodes, Pods, and Services.

• Internal IP addresses: IP addresses from your cluster'sVPC network. These IP addresses can include your cluster IPaddress, on-premises networks, theRFC 1918ranges, or the privatelyused public IP (PUPI) addresses that include non-RFC 1918 ranges.

• External endpoint: The external IP address that GKE assigns to thecontrol plane.

• Internal endpoint: The internal IP address that GKE assigns to thecontrol plane.

• VPC network: A virtual network in which you createsubnets with IP address ranges specifically for the cluster's nodes andPods.

When using IP-based endpoints, you have two options:

• Expose the control plane on both the external and internal endpoints.This means that the control plane's external endpoint is accessible from anexternal IP address, and the internal endpoint is accessible from yourcluster's VPC network. Nodes will communicate with the controlplane on the internal endpoint only.

• Expose the control plane on the internal endpoint only. This means thatclients on the internet cannot connect to the cluster and the control plane isaccessible from any IP address from your cluster's VPC network.Nodes will communicate with the control plane on the internal endpoint only.

  This is the most secure option when using IP-based endpoints as it prevents all internet accessto the control plane. This is a good choice if youhave workloads that—for example—require controlled access due to data privacyand security regulations.

In both of the preceding options, you can restrict which IP addresses reach theendpoints by configuringauthorized networks. If you use IP-based endpoints,then we strongly recommend that you add at least one authorized network.Authorized networks grant control plane access to a specific set of trusted IPv4addresses, and provide protection and additional security benefits for yourGKE cluster.

How authorized networks work

Authorized networks provide an IP-based firewall that controls access to theGKE control plane. Access to the control plane depends on thesource IP addresses. When you enable authorized networks, you configure the IPaddresses for which you want to allow access to the GKE cluster controlplane endpoint as a CIDR block list.

The following table shows:

• The preset IP addresses that can always access the GKE controlplane regardless of whether you enable authorized networks.
• The configurable IP addresses that can access the control plane when you allowlist them by enabling authorized networks.

• If youexpand a subnetthat is used by a cluster with authorized networks, you must update theauthorized network configuration to include the expanded IP address range.
• If you have clients connecting from networks with dynamic IP addresses, suchas employees on home networks, you must update the list of authorized networksfrequently to maintain access to the cluster.
• Disabling access to the control plane's external endpoint prevents you frominteracting with your cluster remotely. If you need to remotely access thecluster, you must use abastion host whichforwards client traffic to the cluster. In contrast, using a DNS-basedendpoint only requires setting up IAM permissions.
• IP-based endpoints don't directly integrate with VPC Service Controls.VPC Service Controls operate at the service perimeter level to control dataaccess and movement within Google Cloud. We recommend using both a DNS-basedendpoint with VPC Service Controls for robust security defense.
• You can specify up to 100 authorized IP address ranges (external and internalIP addresses inclusive).

Cluster networking access

This section discusses isolatingnodes within a Kubernetes cluster.

Enable private nodes

Prevent external clients from accessing nodes by provisioning those nodes onlywith internal IP addresses, making the nodes private. Workloads running on nodeswithout an external IP address cannot reach the internet unless NAT is enabledon the cluster's network. You can change these settings at any time.

You can enable private nodes at an individual cluster level or at the node pool(for Standard) or workload (for Autopilot) level. Enablingprivate nodes at the node pool or workload level overrides any nodeconfiguration at the cluster level.

If you update a public node pool to private mode, workloads that require accessoutside the cluster network might fail in the following scenarios:

• Clusters in a Shared VPC network where Private Google Access is notenabled. Manuallyenable Private Google Access to ensure GKEdownloads the assigned node image. For clusters that aren't in aShared VPC network, GKE automatically enablesPrivate Google Access.

• Workloads that require access to the internet where Cloud NAT is notenabled or a custom NAT solution is not defined. To allow egress traffic tothe internet,enable Cloud NAT or a custom NATsolution.

Whether or not you enable private nodes, the control plane still communicateswith all nodes through internal IP addresses only.

Benefits of network isolation

The following are the benefits of network isolation:

• Flexibility:

  • Clusters have unified and flexible configuration. Clusters with or withoutexternal endpoints all share the same architecture and support the samefunctionality. You can secure access based on controlsand best practices that meet your needs. All communication between the nodesin your cluster and the control plane use an internal IP address.
  • You can change the control plane access and cluster node configurationsettings at any time without having to re-create the cluster.
  • You can choose to enable the external endpoint of the control plane if youneed to manage your cluster from any location with internet access, or fromnetworks or devices that aren't directly connected with yourVPC. Or you can disable the external endpoint for enhancedsecurity if you need to maintain network isolation for sensitive workloads.In either case, you can use authorized networks to limit access to trustedIP ranges.

• Security:

  • DNS-based endpoints with VPC Service Controls provide a multi-layer securitymodel that protects your cluster against unauthorized networks as well asunauthorized identities accessing the control plane. VPC Service Controlsintegrate with Cloud Audit Logs to monitor access to the control plane.
  • Private nodes, and the workloads running on these nodes, are not directlyaccessible from the public internet, significantly reducing the potential forexternal attacks on your cluster.
  • You can block control plane access fromGoogle Cloud external IP addresses or from external IP addresses to fully isolate thecluster control plane and reduce exposure to potential security threats.

• Compliance: If you work in an industry with strict regulations for dataaccess and storage, private nodes help with compliance by ensuring thatsensitive data remains within your private network.

• Control: Private nodes give you granular control over how traffic flows inand out of your cluster. You can configure firewall rules and network policiesto allow only authorized communication. If you operate across multi-cloudenvironments, private nodes can help you establish secure and controlledcommunication between different environments.

• Cost: By enabling private nodes, you can reduce costs for nodes that don'trequire an external IP address to access public services on the internet.

What's next

• Follow the instructions inConfiguring network isolation to experiment with network isolation.
• Learn more aboutPrivate Service Connect.
• Read an overview ofcluster architecture.
• Learn aboutnetworking best practices.