Proxy-only subnets for Envoy-based load balancers

This page describes how to work with proxy-only subnets used by Envoy-based loadbalancers. A proxy-only subnet provides a pool of IP addresses that are reservedexclusively for Envoy proxies used by Google Cloud load balancers. It cannot beused for any other purposes.

The proxies terminate incoming connections and then evaluate where eachrequest should go based on the URL map, the backend service's session affinity,the balancing mode of each backend instance group or NEG, and other factors.

  1. A client makes a connection to the IP address and port of the load balancer'sforwarding rule.

  2. Each proxy listens on the IP address and port specified by the correspondingload balancer's forwarding rule. One of the proxies receives and terminatesthe client's network connection.

  3. The proxy establishes a connection to the appropriate backend VM or endpointin a NEG, as determined by the load balancer's URL map and backend services.

Each of the load balancer's proxies is assigned an internal IP address. Packetssent from a proxy to a backend VM or endpoint has a source IP address from theproxy-only subnet.

The proxy-only subnet cannot be used for any other purpose. The IP address forthe load balancer's forwarding rule doesn't come from the proxy-only subnet.Also, the IP addresses of the backend VMs and endpoints don't come from theproxy-only subnet.

Supported load balancers and products

Envoy-based Cloud Load Balancing and Secure Web Proxy products requireproxy-only subnets:

Note:

If the client sends packets with the port explicitly specified, Envoy-basedload balancers don't remove the port number by default. For example, trafficwithexample.net:443 are first forwarded to an Envoy-based loadbalancing service and then routed to the backend by the URL map.

If there is no matching routing rule in the URL map, the traffic is routed tothe default backend. You can specify URL map routing rules to specify the portnumber by setting the hostname asexample.net:443 and path/*. For more information, seeURL Mapcomponents.

gRPC library for golang mandates addition of port at the end of the hostname,so if a request is initiated to the load balancer, ensure that we have amatching host rule in the load balancer configuration.

How proxy-only subnets fit into the load balancer's architecture

The following diagram shows the Google Cloud resources required fora regional internal Application Load Balancer.

Regional internal Application Load Balancer numbered components.
Regional internal Application Load Balancer numbered components (click to enlarge).

As shown in the diagrams, an Envoy-based load balancer deployment requiresat least two subnets:

  • The load balancer's backend VMs and backend endpoints use a single subnet whose primary IP address range is10.1.2.0/24 (in this example).This subnet isn't the proxy-only subnet. You can use multiple subnets for your backend VMs and endpoints if the subnets are in the same region as the load balancer. For internal Application Load Balancers, the load balancer's IP address associated with the forwarding rule can also be in this subnet (but does not need to be).
  • The proxy-only subnet is10.129.0.0/23 (in this example).

Planning the size of your proxy-only subnet

A proxy-only subnet must provide 64 or more IP addresses. This corresponds to aprefix length of/26 or shorter. We recommend that you start with a proxy-onlysubnet with a/23 prefix (512 proxy-only addresses) andchange thesize as your traffic needs change.

Note: You can use the non-RFC 1918 address range for the proxy-only subnet.This can be helpful if you have insufficient available addresses in yourRFC 1918 address range.

Proxies are allocated at the VPC level, not the load balancerlevel. You must create one proxy-only subnet in each region of aVPC network in which you useEnvoy-based loadbalancers. If you deploy multiple load balancers in the same regionand same VPC network, they share the same proxy-only subnet forload balancing. Envoy-based load balancers automatically scale the number ofproxies available to handle your traffic based on your traffic needs.

The number of proxies allocated to your load balancer is calculated based on themeasured capacity needed to handle your traffic over a 10-minute time period.During this time period, we look at the greater of:

  • The number of proxies needed to serve your traffic's bandwidth needs. Eachproxy instance can handle up to 18 MB per second. We monitor the totalbandwidth required and divide that total by the bandwidth that a proxyinstance can support.

  • The number of proxies needed to handle connections and requests. We countthe total of each of the following resources and divide each value by what aproxy instance can handle:

    • 600 (HTTP) or 150 (HTTPS) new connections per second
    • 3,000 active connections

    • 1,400 requests per second

      A proxy instance can handle 1,400 requests per second ifCloud Logging is disabled. If you enable Logging,your proxy instance can handle fewer requests per second. For example:logging 100% of requests decreases the proxy's request handling capacity to700 requests per second. You can configure Logging to samplea smaller percentage of traffic. This lets you meet your observability needswhile controlling your cost.

Each additional proxy incurs an additional hourly charge. To learn about howproxy-only subnets are billed, see theProxy instancecharge section in theCloud Load Balancingpricing documentation.

Envoy-based load balancers and Secure Web Proxy Envoy proxies

When configuring both an Envoy-based load balancer and Secure Web Proxy inthe same VPC, it is important to note the following:

  • Both the Envoy-based load balancer and Secure Web Proxy use IPaddresses from the same proxy-only subnet.

  • To accommodate IP address requirements for both the services, considerusing a larger proxy-only subnet, such as a/22 subnet. This helps ensuresufficient address space for both configurations.

  • We recommend that you monitor proxy capacity to track IP addressconsumption. This helps prevent exhaustion of the proxy-only subnet, whichcan disrupt services.

Note: Google Cloud doesn't warn you if your proxy-only subnet runs out ofIP addresses. However, you can configure Monitoring to monitorthe IP address usage of your proxy-only subnet. You candefine alertingpoliciesto set up an alert for theloadbalancing.googleapis.com/subnet/proxy_only/addresses metric.

Create a proxy-only subnet

You must create proxy-only subnets for Envoy-based load balancers regardless ofwhether your network is auto-mode or custom. Creating a proxy-only subnet isessentially the same procedure ascreating anysubnet, except with theaddition of some flags.

For a proxy-only subnet, the--purpose must be set to eitherREGIONAL_MANAGED_PROXY orGLOBAL_MANAGED_PROXY,depending on your loadbalancer.

You can't reuse an existing subnet as the proxy-only subnet; youmust create a new subnet in each region that has an Envoy-based load balancer.This is in part because thesubnets updatecommand doesn't allowmodifying a subnet's--purpose field.

Note: For regional internal Application Load Balancers, set the purpose of the subnet to--purpose=REGIONAL_MANAGED_PROXY. If you previously created a proxy-onlysubnet with--purpose=INTERNAL_HTTPS_LOAD_BALANCER, you will need tomigratethe subnet's purpose toREGIONAL_MANAGED_PROXY before youcan create other Envoy-based load balancers in the same region of theVPC network.

You must create a proxy-only subnet for use by the load balancers' proxies,before creating forwarding rules for your regional load balancers. If you try toconfigure a load balancer without first creating a proxy-only subnet for theregion, the load balancer creation process fails.

Console

  1. In the Google Cloud console, go to theVPC networks page.
    Go to the VPC networks page
  2. Click the name of the Shared VPC network that you want to add aproxy-only subnet to.
  3. ClickAdd subnet.
  4. Enter aName.
  5. Select aRegion.
  6. SetPurpose to one of the following:
    • For regional load balancers:Regional Managed Proxy
    • For cross-region load balancers:Cross-region Managed Proxy
    • Enter anIP address range.
    • ClickAdd.

gcloud

Thegcloud compute networks subnetscreatecommand creates a proxy-only subnet.

gcloud compute networks subnets createSUBNET_NAME \    --purpose=SUBNET_PURPOSE \    --role=ACTIVE \    --region=REGION \    --network=VPC_NETWORK_NAME \    --range=CIDR_RANGE

The fields are defined as follows:

  • SUBNET_NAME is the name of the proxy-only subnet.
  • SUBNET_PURPOSE is the purpose of the subnet. Set this toeitherREGIONAL_MANAGED_PROXY orGLOBAL_MANAGED_PROXY,depending onyour load balancer.
  • REGION is the region of the proxy-only subnet.
  • VPC_NETWORK_NAME is the name of the VPC networkthat contains the subnet.
  • CIDR_RANGE is the primary IP address range of the subnet. You must use asubnet mask no longer than26 so that at least 64 IP addresses areavailable for proxies in the region. The recommended subnet mask length is/23.

For a complete configuration example, refer toConfiguring the proxy-onlysubnet.

You must configure a firewall rule for your backends to accept connections fromthe proxy-only subnet. For a complete configuration example, including firewallrule setup, see the following:

Proxy availability

Sometimes Google Cloud regions don't have enough proxy capacity fora new load balancer. If this happens, the Google Cloud console provides aproxy availability warning message when you are creating your load balancer. Toresolve this issue, you can do one of the following:

  • Select a different region for your load balancer. This can be a practicaloption if you have backends in another region.
  • Select a VPC network that already has an allocatedproxy-only subnet.
  • Wait for the capacity issue to be resolved.

Change the size or address range of a proxy-only subnet

When the amount of traffic handled by your load balancer increases, you mightneed to increase the size of your proxy-only subnet to allow a larger number ofEnvoy proxies to power your load balancers.

You can't expand the primary IPv4 address range of a proxy-only subnet in thesame way that you would expand the primary IPv4 address range of a regularsubnet (with theexpand-ip-rangecommand). Instead, you mustreplace the proxy-only subnet with a new one. The replacement process works likethis:

  • Create a new proxy-only subnet, in the same region and VPCnetwork as the existing (original) proxy-only subnet. When creating this newproxy-only subnet, set itsrole toBACKUP. (For each proxy-only subnetpurpose, Google Cloud allows oneACTIVE and oneBACKUP proxy-onlysubnet to exist in a given region and VPC network.)

  • Adjust ingress allow firewall rules, applicable to your backends, so thatthey allow connections from the primary IPv4 address ranges of both theoriginal and new proxy-only subnets.

  • Set the role of the new proxy-only subnet toACTIVE and specify a drainingperiod to allow connections between your backends and the Envoy proxies inthe original proxy-only subnet to terminate. (Google Cloud automaticallysets the role of the original proxy-only subnet toBACKUP when you set therole of the new proxy-only subnet toACTIVE.)

  • Monitor the status of the original proxy-only subnet (see thegcloud tab for more information on monitoring). Once its statusisREADY, the subnet is no longer used, provided that its role isBACKUP.At this point, you can adjust ingress allow firewall rules to permitconnections from just the primary IPv4 address range of the new proxy-onlysubnet, and you can delete the original proxy-only subnet.

Console

  1. Create the new proxy-only subnet in the same region andVPC network, specifying a primary IPv4 address range thatmeets your needs. Set the role of the new proxy-only subnet to backup.

    1. In the Google Cloud console, go to theVPC networks page.
      Go to the VPC networks page
    2. Click the name of the Shared VPC network that you want to add aproxy-only subnet to.
    3. ClickAdd subnet.
    4. Enter aName.
    5. Select aRegion.
    6. SetPurpose to one of the following:
      • For regional load balancers:Regional Managed Proxy
      • For cross-region load balancers:Cross-region Managed Proxy
      • For theRole, selectBackup.
      • Enter anIP address range.
      • ClickAdd.

  2. Update ingress allow firewall rules that apply to your backend VMs orendpoints so that they include the primary IPv4 address ranges ofboththe original and new proxy-only subnets.

  3. Set the role of the new proxy-only subnet to active, and specify adraining timeout to allow connections between your backends and theoriginal proxy-only subnet to terminate. Google Cloud automaticallysets the role of the original proxy-only subnet to backup when you setthe role of the new proxy-only subnet to active.

    1. In the Google Cloud console, go to theVPC networks page.
      Go to the VPC networks page
    2. Click the name of the Shared VPC network that you want tomodify.
    3. UnderReserved proxy-only subnets for load balancing, locate thebackup subnet created in the previous step.
    4. ClickActivate.
    5. Specify an optionalDrain timeout.
    6. ClickActivate the subnet.

  4. After the connection draining timeout, or after you're confident thatconnections to your backend VMs or endpoints aren't coming fromproxies in the original proxy-only subnet, you can do the following:

    • Update ingress allow firewall rules that apply to your backend VMs orendpoints so that they include the primary IPv4 address range of justthe new proxy-only subnet.
    • Delete theoriginal proxy-only subnet.
Note: The gcloud steps provide more information about how tomonitor the draining status of the original proxy-only subnet.

gcloud

The following steps show how to swap an existing proxy-only subnet with anew proxy-only subnet. In all of the following steps:

  1. Create a new proxy-only subnet in the same region and VPCnetwork using thegcloud compute networks subnetscreate commandwith the--role=BACKUP flag.

    gcloud compute networks subnets createNEW_PROXY_ONLY_SUBNET_NAME \   --purpose=SUBNET_PURPOSE \   --role=BACKUP \   --region=REGION \   --network=VPC_NETWORK_NAME \   --range=NEW_PROXY_ONLY_SUBNET_RANGE

    Replace the following:

    • SUBNET_PURPOSE: EitherREGIONAL_MANAGED_PROXY orGLOBAL_MANAGED_PROXY depending on the load balancer that needs to usethe proxy-only subnet. For more information, seeSupported loadbalancers.

  2. Update ingress allow firewall rules that apply to your backend VMs orendpoints so that they include the primary IPv4 address ranges ofboththe original and new proxy-only subnets.

    gcloud compute firewall-rules updatePROXY_ONLY_SUBNET_FIREWALL_RULE \  --source-ranges=ORIGINAL_PROXY_ONLY_SUBNET_RANGE,NEW_PROXY_ONLY_SUBNET_RANGE

  3. Set the role of the new proxy-only subnet toACTIVE, and specify adraining timeout (--drain-timeout) to allow connections between yourbackends and the original proxy-only subnet to terminate.Google Cloud automatically sets the role of the original proxy-onlysubnet toBACKUP when you set the role of the new proxy-only subnet toACTIVE.

    To immediately break connections between your backends and Envoy proxiesin the original proxy-only subnet, set the--drain-timeout to0s.

    gcloud compute networks subnets updateNEW_PROXY_ONLY_SUBNET_NAME \   --region=REGION \   --role=ACTIVE \   --drain-timeout=CONNECTION_DRAINING_TIMEOUT

    Replace the following:

    • CONNECTION_DRAINING_TIMEOUT: the amount of time, inseconds, to let existing connections between your backends and Envoyproxies in the original proxy-only subnet terminate.

  4. Monitor the status of the original proxy-only subnet by using thegcloud compute networks subnetsdescribecommand.

    gcloud compute networks subnets describeORIGINAL_PROXY_ONLY_SUBNET_NAME \   --region=REGION

    Wait for draining to complete. During draining, the original proxy-onlysubnet's status isDRAINING. You might have to execute thedescribecommand a few times before the status of the original proxy-only subnetchanges toREADY.

  5. After the original proxy-only subnet isREADY with theBACKUP role,the subnet is no longer used. Update ingress allow firewall rules thatapply to your backend VMs or endpoints so that they include the primaryIPv4 address range of just the new proxy-only subnet.

    gcloud compute firewall-rules updatePROXY_ONLY_SUBNET_FIREWALL_RULE \  --source-ranges=NEW_PROXY_ONLY_SUBNET_RANGE

  6. Delete the original proxy-only subnet.

    gcloud compute networks subnets deleteORIGINAL_PROXY_ONLY_SUBNET_NAME \  --region=REGION

Migrate the purpose of a proxy-only subnet

If you previously created a proxy-only subnet with--purpose=INTERNAL_HTTPS_LOAD_BALANCER, you will need to migrate the subnet'spurpose toREGIONAL_MANAGED_PROXY before you can create other Envoy-based loadbalancers in the same region of the VPC network.

Console

If you're using the Google Cloud console to create the load balancer, youwill be prompted to migrate the purpose of a previously created proxy-onlysubnet from--purpose=INTERNAL_HTTPS_LOAD_BALANCER toREGIONAL_MANAGED_PROXY while creating the load balancer.

gcloud

To change the purpose of an existing proxy-only subnet from--purpose=INTERNAL_HTTPS_LOAD_BALANCER toREGIONAL_MANAGED_PROXY, use the following command:

gcloud compute networks subnets updatePROXY_ONLY_SUBNET \    --purpose=REGIONAL_MANAGED_PROXY \    --region=REGION

Migrating the purpose of a proxy-only subnet from--purpose=INTERNAL_HTTPS_LOAD_BALANCER toREGIONAL_MANAGED_PROXY does notcause any downtime. The change should take effect nearly instantly.

Delete a proxy-only subnet

Deleting a proxy-only subnet releases its primary IP address range so you canuse the range for another purpose. Google Cloud enforces the followingrules when it receives a request to delete a proxy-only subnet:

  • An active proxy-only subnet cannot be deleted if there is at least oneregional load balancer in the same region and VPCnetwork.

  • An active proxy-only subnet cannot be deleted if there is a backupproxy-only subnet in the same region and VPC network.

    If you try to delete an active proxy-only subnet before deleting the backup,the following error message appears: "Invalid resource usage: Cannot deleteACTIVE subnetwork because a BACKUP subnetwork exists."

Practically, these rules have the following effect:

  • If no regional load balancer is defined in a given region andVPC network, you can delete the proxy-only subnets in thatregion. If a backup proxy-only subnet exists, you must first delete it beforeyou can delete the active proxy-only subnet.

  • If you have at least one regional load balancer defined in a givenregion and VPC network, you cannot delete the active proxy-onlysubnet; however, you can promote a backup proxy-only subnet to the activerole, which automatically demotes the previously active proxy-only subnet tothe backup role. After connections are drained, you can delete thebackup (previously active) proxy-only subnet.

Refer todeleting subnets in theVPC network documentation for more information.

Limitations

The following constraints apply to proxy-only subnets:

  • You can't have both anINTERNAL_HTTPS_LOAD_BALANCER and aREGIONAL_MANAGED_PROXY subnet in the same network and region, in the sameway you can't have twoREGIONAL_MANAGED_PROXY proxies or twoINTERNAL_HTTPS_LOAD_BALANCER proxies.

  • You can create only one active and one backup proxy-only subnet in each regionin each VPC network.

  • You cannot create a backup proxy-only subnet unless you have already createdan active proxy-only subnet in that region and network.

  • You can change the role of a proxy-only subnet from backup to active byupdating the subnet. When you do that, Google Cloud automaticallychanges the previously active proxy-only subnet to backup. You cannotexplicitly set the role of a proxy-only subnet to backup by updating it.

  • During a proxy-only subnet's connection draining period(--drain-timeout), you cannot change the role of a proxy-only subnetfrom backup to active.

  • Proxy-only subnets don't supportVPC Flow Logs.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.