This page provides a general overview of VPC-native clusters inGoogle Kubernetes Engine (GKE).

This page is for Cloud architects and Networking specialists whodesign and architect the network for their organization. To learn more aboutcommon roles and example tasks that we reference in Google Cloud content, seeCommon GKE user roles and tasks.

Overview

In GKE, clusters can be distinguished according to the way theyroute traffic from one Pod to another Pod.

A cluster that usesalias IP address ranges is called aVPC-native cluster. A cluster that usescustom static routesin a VPCnetwork is called aroutes-based cluster.

Benefits of VPC-native clusters

VPC-native clusters have several benefits:

• PodIP addresses are natively routable within the cluster's VPCnetwork and other VPC networks connected to it byVPC Network Peering.

• Pod IP addresses are reserved in the VPC networkbefore thePods are created in your cluster. This prevents conflict with otherresources in the VPC network and lets you better plan IPaddress allocations.

• Pod IP address ranges don't depend on custom static routes. They don'tconsume thesystem-generated and custom static routesquota. Instead, automatically-generatedsubnetroutes handle routing forVPC-native clusters.

• You can createfirewall rules that apply to just PodIP address ranges instead of any IP address on the cluster's nodes.

• Pod IP address ranges, and subnet secondary IP address ranges in general,are accessible from on-premises networks connected with Cloud VPNor Cloud Interconnect usingCloud Routers.

• Some features, such asnetwork endpoint groups(NEGs), only work with VPC-nativeclusters.

Default cluster network mode

VPC-native is the default network mode for all clusters inGKE versions 1.21.0-gke.1500 and later. For earlier versions, thedefault cluster network mode depends on how you create the cluster.

The following table lists the default cluster network mode forGKE cluster versions and cluster creation methods.

You can also create a routes-based cluster by specifying the--no-enable-ip-alias flag when you create the cluster.

IP address ranges for VPC-native clusters

When you create a VPC-native cluster, you specify a subnet in aVPC network. The cluster uses the followingsubnet IP addressranges:

IPv4 address allocation

VPC-native clusters allocate IPv4 addresses for nodes, Pods, andServices as follows.

• Node IPv4 addresses: The cluster utilizes the subnet's primary IPv4address range to assign IP addresses to all nodes.

• Pod IPv4 addresses: The cluster utilizes one or more subnet secondaryIPv4 address ranges for all Pod IPv4 addresses. This page focuses on the useof a single subnet secondary IPv4 address range. Forinformation about using multiple ranges, seeAddingPod IPv4 address ranges.

• Service IPv4 addresses: Two options are available:

  • Service IPv4 addresses from34.118.224.0/20: GKEAutopilot clusters running version 1.27 and laterand GKE Standard clusters running version 1.29and later use the34.118.224.0/20 IPv4 address range for Services.Google Cloud uses the34.118.224.0/20 address range for privatepurposes and doesn't publish any routes on the public internet for thisrange. You can't use this range for external IPv4 addresses of yourresources.

  • Service IPv4 addresses from a subnet secondary IPv4 address range:For all Service (ClusterIP) addresses, the cluster utilizes a subnetsecondary IPv4 address range that is different from the range(s) used byPods.

IPv6 address allocation (dual-stack networking)

• Node and Pod IPv6 addresses: in clusters enabled for dual-stacknetworking, the node's IPv6 address and all Pod IPv6 addresses originatefrom the node's designated/96 IPv6 address range. The node IP addressitself is the first/128 (single IPv6 address) within this range. For moreinformation, seedual-stack networking.

• Service IPv6 addresses: GKE clusters use the2600:2D00:0:4::0:0/64 IPv6 address range for Services. Google Clouduses the2600:2D00:0:4::0:0/64 address range for private purposes anddoesn't publish any routes on the public internet for this range. You can'tuse this range for external IPv6 addresses of your resources.

The following table provides a summary of IP address ranges for nodes, Pods, andServices:

Internal IPv4 addresses

The IPv4 addresses that you use for your VPC-native cluster's subnetsmust come from a valid subnet range. The valid ranges include private IPv4addresses, such asRFC 1918, and privatelyused public IP addresses. SeeValid ranges andRestricted ranges in the VPCdocumentation for more information about valid subnet IPv4 ranges.

For important information about using private addresses that aren't RFC1918 addresses, seeUsing non-RFC 1918 private IPv4 addressranges.

For important information about using privately used public IPv4 addressranges, seeEnable privately used public IP addressranges.

Subnet secondary IPv4 range assignment methods

You can assign Pod IP address ranges and Service (ClusterIP) address ranges toa VPC-native cluster. These IP address ranges can be managed byGKE or user-managed.

You must understand the following key terms to understand the secondary rangeassignment methods.

Assignment: assigning IP address ranges refers to the process of allocatinga specific subnet range to a VPC-native cluster. This establishesa pool of IP addresses that the components can use within the cluster, such asPods and Services.

Management: managing the IP address range refers to the ongoing (creation,update, deletion, reading) CRUD operations at the cluster, node pool, or Podlevel, related to the assigned subnet ranges and resource allocation within yourVPC-native cluster.

GKE-managed secondary ranges (default)

For GKE Autopilot clusters running version 1.27 andlater and GKE Standard clusters running versions 1.29and later, GKE assigns IP addresses for Services from aGKE-managed range by default:34.118.224.0/20. This eliminatesthe need for you to specify your own IP address range for Services. Thefollowing considerations apply:

• You can optionally specify custom ranges for Services by using the--services-ipv4-cidr flag.
• If you specify only a rangesize using the--services-ipv4-cidr flag(for example,/22), GKE still uses theGKE-managed range to obtain the sub-range of addresses.
• If you specify only a size for the Pod IP address range, for example--cluster-ipv4-cidr=/17,GKE automatically selects an available secondary IP address range from your VPC network.
• GKE doesn't create a separate secondary IP address range forServices when the GKE-managed range is used.

User-managed

For full control over IP address allocation, you can manually manage yourVPC-native cluster's subnets.

You cancreate the subnet's secondary IP addressranges, then create acluster that uses those ranges. During cluster creation, specify the subnetrange name for Pods and Services. If you manually create the secondary ranges,you must manage them yourself.

The smallest IP address range you can create without usingdiscontiguousmulti-Pod CIDR is /28, but thatrange would only let you create 1 node with a maximum of 8 Pods. You should usea range that is large enough for the maximum number of nodes that you need.

The minimum usable range also depends on themaximum number of Pods perNode.

Refer to the table inPod CIDR ranges in Standard clustersfor the minimum usable CIDR range for different values of maximum Pods per node.

If you exhaust your IP address range for Pods, you must do one of the following:

• Create a new cluster with a larger Pod address range.
• Re-create your node pools after decreasing the--max-pods-per-nodefor the node pools.
• Expand the secondary Pod IP address range usingdiscontiguous multi-PodCIDR.

Differences with routes-based clusters

The allocation scheme for Pod and Service (ClusterIP) addresses is differentthan the scheme used by a routes-based cluster. Instead of specifying a singleCIDR for Pods and Services together, you must choose or create two secondary IPaddress ranges in cluster's subnet: one for Pods and another for Services.

Shared VPC considerations

When creating a VPC-native cluster in aShared VPC environment, a project owner, editor,or Identity and Access Management (IAM) principal with the Network Admin role in theShared VPC host project must create the cluster's subnet and secondaryIP address ranges manually. A service project administrator who creates a cluster mustat least havesubnet-level permissionsto the subnet in the Shared VPC network's host project.

In a Shared VPC environment, secondary IP address ranges cannot bemanaged by GKE. A Network Admin in the Shared VPC hostproject must create the subnet and secondary IP address ranges before you cancreate the cluster. For an example showing how to set up aVPC-native cluster in a Shared VPC network, refer toSetting up clusters withShared VPC.

IP address range planning

Use the information in the following sections to help you calculate sizes forprimary and secondary IP address ranges of the subnet used by your cluster.

Subnet primary IPv4 address range

Consider the following when you plan the primary IPv4 address range of acluster's subnet:

• Every subnet must have a primary IP address range. This is the IP address rangethat GKE uses to allocate IP addresses for internal loadbalancers and nodes.
• You cannot shrink or change a subnet's primary IP address range after thesubnet has been created.
• After you create a subnet, you can expand its primary IP address range, butyou can't reduce it. If you expand a subnet that is used by a cluster withauthorized networks, you must add the expanded IP address range to thecontrol plane authorized networks list.Before expanding a range, be sure to review the limitations andrecommendations inExpand a primary IPv4range.
• The first two and last two IP addresses of a primary IP address range are reserved byGoogle Cloud.
• Clusters withPrivate Service Connect use the primary subnet rangeto provision the internal IP address assigned to the control plane endpoint. However, you can override this IP address provisioning with theprivate-endpoint-subnetwork flag. To learn more, seeCreate a cluster and select the control plane IP address range.

The following table shows the maximum number of nodes you can create given the sizeof the subnet's primary IPv4 address range and the cluster configuration:

• Scenario 1: maximum nodes in a cluster that uses the default subnet.
• Scenario 2: maximum nodes in a Private Service Connectcluster that doesn't use theprivate-endpoint-subnetwork flag.

Expand the primary IP address range

If you run out of IP addresses in the primary IP address range, you canexpandthe primary IP addressrange at any time, evenwhen Google Cloud resources, such as load balancers and network endpointgroups, use the subnet.

Before you expand the primary IP address range, consider the following:

• There must be no overlapping IP address ranges in the subnet.
• GKE uses the primary IP address range to allocate IPaddresses for internal load balancers and nodes.
• If the cluster usesauthorizednetworks, you must addthe expanded primary IP address range to the list of authorized networks.

Useful formulas

You can use the following formulas to:

• Calculate the maximum number of nodes,N, that a given netmaskcan support in clusters that use the default subnet. UseS forthe size of the netmask, whose valid range is between8 and29,inclusive.

  N = 2^{(32 -S)} - 4

• Calculate the size of the netmask,S, required to support amaximum ofN nodes:

  S = 32 - ⌈log₂(N + 4)⌉

  ⌈⌉ is theceiling (least integer)function,meaning round up to the next integer. The valid range for the size of thenetmask,S, is between8 and29, inclusive.

In Private Service Connect clusters that don't use theprivate-endpoint-subnetwork flag, you can use the preceding formulas, butreduce the value ofN by 1.

Cluster sizing considerations for secondary IP address range for Pods

To calculate the maximum number of Pods that your cluster can support,consider the size of the Pod subnet and the maximum number of Pods per node.The maximum number of Pods depends on the subnet mask and the maximum number ofPods per node. Additionally, remember that some IP addresses are reserved in theprimary range.

Key variables

• Q: the maximum number of Pods per node.
• DS: the selected CIDR block size for the Pod subnet (for example,/17).
• M: the netmask size for each node's Pod range.
• HM: the number of host bits for the node's Pod range netmask.
• HD: the number of host bits for the Pod subnet's CIDR block.
• MN: the maximum number of nodes that can be supported.
• MP: the maximum number of Pods that can be supported.
• S: the prefix length of the subnet's primary IP address range.
• N: the number of usable IP addresses in the subnet's primary IP address range.

Steps to calculate maximum Pods

1. Determine the maximum Pods per node (Q):

   • For Autopilot clusters, the value ofQ is fixed at 32.
   • For Standard clusters, you can configureQ.

2. Define the size of the CIDR block for the Pod subnet (DS):

   • Determine the selected CIDR block size for the Pod subnet (for example,/17).

3. Calculate the netmask size (M):

   M = 31 - ⌈log₂(Q)⌉

   ReplaceQ with the value of maximumPods per node. Use the ceiling function (⌈ ⌉) to round up to the nearestinteger.

4. Calculate the host bits for the Pod range (HM):

   HM = 32 - M

5. Calculate the host bits for the selected CIDR block (HD):

   HD = 32 - DS

6. Calculate the maximum number of nodes (MN):

   MN = 2<sup>(HD - HM)</sup>

   The result of this calculation is the maximum number ofnodes that the selected Pod subnet can support.

7. Calculate the maximum number of Pods (MP):

   MP = MN * Q

   The result of this calculation is the maximum number of Pods that thecluster can support.

8. Calculate the number of usable IP addresses in the primary range (N):noneN = 2<sup>(32-S)</sup> - 4WhereS is the prefix length of the primary CIDR range for the subnet.

   Note: The first two and last two IP addresses of a primary IP address rangeare reserved by Google Cloud.

Important notes

• All IP addresses in the secondary range are usable for Pods.
• These calculations provide thetheoretical maximums. Real-worldperformance can be affected by other factors.

Example

Suppose you are creating a GKE Autopilot cluster with the following:

• A Pod subnet CIDR block of/17 (DS = 17).
• A maximum of 32 Pods per node (Q = 32).

Calculate the maximum number of Pods:

1. M = 31 - ⌈log₂(32)⌉ = 26
2. HM = 32 - 26 = 6
3. HD = 32 - 17 = 15
4. MN = 2(15 - 6) = 512
5. MP = 512 * 32 = 16,384

This cluster can support a maximum of 512 nodes and 16,384 Pods.

You can add more IPv4 addresses for Pods byadding Pod IPv4 addressranges or byaddingsubnets to clusters.

Subnet secondary IP address range for Services

Carefully plan your secondary IP address range for Services. Because this isalso a subnet secondary IP address range, this range cannot be changed once thecluster is created.

If you usemulti-clusterservices, theServiceImport object uses IP addresses from the secondary IP address range forServices.

In GKE Autopilot clusters running version 1.27 and laterand GKE Standard clusters running versions 1.29 andlater, GKE assigns IP addresses for Services from aGKE-managed range by default:34.118.224.0/20. This reducesthe need for you to specify your own IP address range for Services. Thefollowing considerations apply:

• You can optionally specify custom ranges for Services by using the--services-ipv4-cidr flag or the--services-secondary-range-name flag.
• If you specify only a rangesize using the--services-ipv4-cidr flag(for example,/22), GKE still uses theGKE-managed range to obtain the sub-range of addresses.
• GKE doesn't create a separate secondary IP address range forServices when the Google-managed range is used. The GKE-managedrange doesn't use the secondary IP address range quota for your subnet.

The following table shows the maximum number of Services you can create in asingle cluster using the subnet, given the size of the subnet's secondary IPaddress range for Services.

Sharing IP address ranges across GKE clusters

You can share the primary range, secondary IP address range for Pods, andsecondary IP address range for Services between clusters in the same subnetwork.This behavior is available for both Standard and Autopilotclusters.

You might want to share IP address ranges if you have a centralized team that ismanaging the infrastructure for clusters. You can reduce overhead by creatingthree ranges, for Pods, Services and nodes, and reusing or sharing them,especially in a Shared VPC model. It can also make it easier for networkadministrators to manage IP addresses by not requiring them to create specificsubnets for each cluster.

Sharing the customized subnet range for the control plane

By default, GKE uses the primary subnet range to provision theinternal endpoint of the control plane. However, in clusters withPrivate Service Connect,you can configure GKE to provision the internal endpoint from adifferent subnet that you created. You can share this subnet among otherclusters, or across projects if you are using Shared VPC.

Sharing the primary IP address range for nodes

If you create more than one cluster in the subnet, the primary IP address rangefor nodes is shared by default.

Sharing the primary IP address for nodes has the following limitations:

• If you share the primary IP address range for nodes with two or moreVPC-native clusters, one cluster can use a large portion ofthe shared IP address range, leaving the other clusters without enough IPaddresses to expand.

Sharing the secondary IP address range for Pods

When you share the secondary range for Pods, each Pod still gets a unique IPaddress.

Sharing the secondary IP address range for Pods has the following limitations:

• If you share the secondary IP address range for Pods with two or moreVPC-native clusters, one cluster can use a large portion ofthe shared IP address range, leaving the other clusters without enough IPaddresses to expand.

• Among the different types of secondary ranges, both theGKE-managedandadditional Pod rangesare not shareable across clusters.

• To share a secondary IP address range, pass it on the command line with--cluster-secondary-range. You cannot use a shared secondary range whencreating clusters in the UI.

Sharing the secondary IP address range for Services

Two or more clusters can simultaneously use the same subnet secondary IPv4address range for Services when you useuser-managed secondaryranges.

To configure two or more clusters to share a common subnet secondary IPv4address range for Services, use the same subnet secondary IPv4 address rangewhen you create each cluster. There is no separate configuration flag requiredto share a common IPv4 address range for Services.

When sharing a common IPv4 address range for Services, each cluster uses theentire subnet secondary IPv4 address range for Services internally. The IPaddresses for Services are programmed within each cluster's node, but they arenot assigned to the network interface of any node. Service IP addresses are notroutable within the cluster's VPC network. Service IP addressesare only usable by client Pods within the same cluster as the Service.

When a Pod sends a packet to a Service IP address, the iptables or eBPFconfiguration on the node performs destination Network Address Translation(NAT), changing the destination IP address of the packet from the Service IPaddress to a serving Pod IP address. The packet is routed based on thedestination Pod IP address.

Sharing the secondary IP address range for Services provides the followingbenefits:

• Reduce the number of unique secondary IP address ranges for Services createdon a subnet
• Use fewer IP addresses

Sharing the secondary IP address range for Services has the followinglimitations:

• Sharing the secondary IP address range for Services is not supported withVPC scope Cloud DNS forGKE.

• You can't share ranges that match the following regular expression:

  ^gke-.*-services-[abcdef0-9]{8}

• To share a secondary IP address range for services, pass it on the commandline with--cluster-secondary-range. You cannot use a shared secondaryrange for services when creating clusters in the UI.

Node limiting ranges

The maximum number of Pods and Services for a given GKEcluster is limited by the size of the cluster's secondary ranges. The maximumnumber of nodes in the cluster is limited by the size of the cluster's subnet'sprimary IP address rangeand the cluster's Pod address range.

The following error message indicates that either the subnet's primary IPaddress range or the cluster's Pod IP address range (the subnet's secondary IPaddress range for Pods) has been exhausted:

Instance [node name] creation failed: IP space of [cluster subnet] isexhausted

You can add more IP addresses for nodes byexpanding the primarysubnet, or add new IPaddresses for Pods usingdiscontiguous multi-PodCIDR. For more information, seeNot enough free IP address space forPods.

IPv4/IPv6 dual-stack networking

With IPv4/IPv6 dual-stack networking, you can define how GKEallocates IP addresses (ipFamilies) to the following objects:

• For Pods and nodes, GKE allocates both IPv4 and IPv6addresses.
• For Services, GKE allocates either single-stack (IPv4 only orIPv6 only), or dual-stack addresses.

In GKE version 1.24 or later, you can enable dual-stacknetworking for new GKE clusters on standalone andShared VPC networks. You can also apply network policies with dual-stacknetworking enabled. If you see validation errors when enabling dual-stack networking onGKE clusters that have been upgraded from versions 1.24 toversions 1.25 or 1.26, contact the Google Cloudsupport team.

Benefits

Dual-stack networking provides the following benefits:

• Enables end-to-end IPv6 communication.
• Enables better performance compared to network address translation (NAT) orIP tunneling. This is achieved because there is no IPv6 to IPv4 translation.

Availability

Dual-stack networking with GKE has the following restrictions:

• Dual-stack networking is only available forVPC-nativeclusters clusters withGKE Dataplane V2 enabled.
• Dual-stack networking is only supported on subnets in custom modeVPCs. For more information, seeGoogle Cloud types ofVPC networks.
• Single-stack IPv6 addresses for Pods or nodes are not supported.
• Dual-stack clusters consume additional memory per node to support both IPv4and IPv6 compared to IPv4 only clusters. Consider this characteristic whensetting up large-scale clusters.
• Dual-stack clusters don't support Private Google Access over IPv6.
• GKE versions 1.26.0-gke.2200 and later supports IPv6 (AAAArecords) with Cloud DNS for cluster-internal operations andexternal DNS queries.
• Dual-stack Services are supported forClusterIP,NodePort, andLoadBalancer Services.
• IPv6 is not supported for Windows nodes.

Consider the preceding restrictions before creating a cluster with dual-stacknetworking. For more information, learnhow to create aVPC-native cluster with dual-stacknetworking.

Public and Private IPv6 address assignment

The following table provides a summary of public and private IPv6 addresses withdual-stack networking behavior and configurations:

To learn more, seehow to use a dual-stack network for aVPC-nativecluster.

Architecture

A cluster with IPv4/IPv6 dual-stack networking has the following rangesallocated:

• A /64 range to each subnet as a primary range.
• A /96 per-node range from the primary range to use as a primary node IPaddress range.

• A /112 per-node range from the primary node IP address range to use as a PodIP address range for that node. With dual-stack networking, Pods get theirIPv6 addresses from theoverall Pod IP address range (similar to nodes),and not from a secondary range for Pods reserved to IPv4 addresses.

  The overall Pod IP address range comprises non-overlapping rangesfrom the primary node IP range. Therefore, this Pod IP range isdiscontiguous.

• A /112 range to use for Services. This range comes from a /64 range from theGoogle private address range which has been reserved forGKE's secondary Services IP address range.

The following diagram shows how Google Cloud and GKEallocate IPv6 addresses:

In the diagram, the primary range of the VPC subnet is2600:1900:0:1::/64 and the reserved range for GKE Services is2600:2D00:0:4::0:0/64. Each node in the cluster has a /96 range for theprimary node IP address range and a /112 range for the Pod IP address range. Thecluster also has a /112 secondary Services IP address range.

You can create an IPv4/IPv6 dual-stack Service of typeClusterIPorNodePort.New GKE clusters running version 1.29 or later support dual-stackServices of typeLoadBalancer.

You can expose a Deployment with a Service of typeClusterIP,NodePort, orLoadBalancer. For each of these Service types, you can defineipFamilies andipFamilyPolicy fields as either IPv4, IPv6, or adual-stackService. For more information, see anexample of how to set up aDeployment.

What's next

• Learn more about VPC Network Peering.
• Learn how to create a VPC-nativecluster.
• Learn about GKE IP address utilizationinsights.