Configure failover for external passthrough Network Load Balancers

Note: This page describes how failover works for backend service-basedexternal passthrough Network Load Balancers. It does not apply to external passthrough Network Load Balancers with a targetpool backend.

This guide uses an example to teach you how to configure failoverfor an external passthrough Network Load Balancer with a backend service. Before followingthis guide, familiarize yourself with the following:

Permissions

To follow this guide, you need to create instances and modify anetwork in a project. You should be either a projectowner or editor, or you shouldhave all of the followingCompute Engine IAM roles:

TaskRequired Role
Create load balancer componentsNetwork Admin
Add and remove firewall rulesSecurity Admin
Create instancesCompute Instance Admin

For more information, see the following guides:

Setup

This guide shows you how to configure and test an external passthrough Network Load Balancerthat uses failover. The steps in this section describe how to configurethe following:

  1. Backend VMs:
    • One primary backend in an unmanaged instance group in zoneus-west1-a
    • One failover backend in an unmanaged instance group in zoneus-west1-c
  2. Firewall rules that allow incoming connections to backend VMs
  3. One client VM to test connections and observe failover behavior
  4. The following external passthrough Network Load Balancer components:
    • A health check for the backend service
    • A backend service in theus-west1 region to manageconnection distribution among the backend VMs
    • A forwarding rule and IP address for thefrontend of the load balancer

The architecture for this example looks like this:

Failover with an external passthrough Network Load Balancer.
Failover with an external passthrough Network Load Balancer (click to enlarge).
Note: Unmanaged instance groups are used for both the primary and failoverbackends in this example. For more information, seesupportedbackends.

Creating backend VMs and instance groups

In this step, you'll create the backend VMs and unmanaged instance groups:

  • The instance groupig-a inus-west1-a is a primarybackend with two VMs:
    • vm-a1
    • vm-a2
  • The instance groupig-c inus-west1-c is a failoverbackend with two VMs:
    • vm-c1
    • vm-c2

The primary and failover backends are placed in separate zones forinstructional clarity and to handle failover in case one zone goes down.

Each primary and backup VM is configured to run an Apache web server on TCPport 80. By default, Apache is configured to bind to any IP address. Networkload balancers deliver packets by preserving the destination IP.

Ensure that server software running on your primary and backup VMs is listeningon the IP address of the load balancer's forwarding rule. The destination IPaddress of a packet delivered to a backend VM by anexternal passthrough Network Load Balancer is the IP address of the forwarding rule.

For instructional simplicity, all primary and backup VMs run Debian GNU/Linux 9.

Console

Create backend VMs

  1. In the Google Cloud console, go to theVM instances page.Go to VM instances
  2. Repeat the following steps to create four VMs, using the following nameand zone combinations.
    • Name:vm-a1, zone:us-west1-a
    • Name:vm-a2, zone:us-west1-a
    • Name:vm-c1, zone:us-west1-c
    • Name:vm-c2, zone:us-west1-c
  3. ClickCreate instance.
  4. Set theName as indicated in step 2.
  5. For theRegion, chooseus-west1, and choose aZone asindicated in step 2.
  6. In theBoot disk section, ensure that the selected image isDebian GNU/Linux 12 (bookworm). ClickChoose to change the image ifnecessary.
  7. ClickAdvanced options.
  8. ClickNetworking and configure the following field:
    1. ForNetwork tags, enternetwork-lb.

  9. ClickManagement. Enter the following script into theStartup script field. The script contents are identical forall four VMs:

    #! /bin/bashapt-get updateapt-get install apache2 -ya2ensite default-ssla2enmod sslvm_hostname="$(curl -H "Metadata-Flavor:Google" \http://metadata.google.internal/computeMetadata/v1/instance/name)"echo "Page served from: $vm_hostname" | \tee /var/www/html/index.htmlsystemctl restart apache2

  10. ClickCreate.

Create instance groups

  1. In the Google Cloud console, go to theInstance groups page.

    Go to Instance groups

  2. Repeat the following steps to create two unmanaged instance groups eachwith two VMs in them, using these combinations.

    • Instance group:ig-a, zone:us-west1-a, VMs:vm-a1 andvm-a2
    • Instance group:ig-c, zone:us-west1-c, VMs:vm-c1 andvm-c2

  3. ClickCreate instance group.

  4. ClickNew unmanaged instance group.

  5. SetName as indicated in step 2.

  6. In theLocation section, chooseus-west1 for theRegion, andthen choose aZone as indicated in step 2.

  7. ForNetwork, enterdefault.

  8. In theVM instances section, add the VMs as indicated in step 2.

  9. ClickCreate.

gcloud

  1. Create four VMs by running the following command four times, usingthese four combinations forVM-NAME andZONE. The script contentsare identical for all four VMs.

    • VM-NAME ofvm-a1 andZONE ofus-west1-a
    • VM-NAME ofvm-a2 andZONE ofus-west1-a
    • VM-NAME ofvm-c1 andZONE ofus-west1-c
    • VM-NAME ofvm-c2 andZONE ofus-west1-c
    gcloud compute instances createVM-NAME \    --zone=ZONE \    --image-family=debian-12 \    --image-project=debian-cloud \    --tags=network-lb \    --metadata=startup-script='#! /bin/bash      apt-get update      apt-get install apache2 -y      a2ensite default-ssl      a2enmod ssl      vm_hostname="$(curl -H "Metadata-Flavor:Google" \      http://metadata.google.internal/computeMetadata/v1/instance/name)"      echo "Page served from: $vm_hostname" | \      tee /var/www/html/index.html      systemctl restart apache2'

  2. Create the two unmanaged instance groups in each zone:

    gcloud compute instance-groups unmanaged create ig-a \    --zone us-west1-agcloud compute instance-groups unmanaged create ig-c \    --zone us-west1-c

  3. Add the VMs to the appropriate instance groups:

    gcloud compute instance-groups unmanaged add-instances ig-a \    --zone us-west1-a \    --instances vm-a1,vm-a2gcloud compute instance-groups unmanaged add-instances ig-c \    --zone us-west1-c \    --instances vm-c1,vm-c2

Configuring firewall rules

Create a firewall rule that allows external traffic to reach the backendinstances.

Console

  1. In the Google Cloud console, go to theFirewall policies page.

    Go to Firewall policies

  2. ClickCreate firewall rule to create the rule to allow trafficfrom any source. Use the following values:

    1. Name:allow-network-lb.
    2. Network:default
    3. Priority:1000
    4. Direction of traffic: Ingress
    5. Action on match: Allow.
    6. Targets: Specified target tags
    7. Target tags:network-lb
    8. Source filter:IPv4 ranges
    9. Source IPv4 ranges:0.0.0.0/0, which allows traffic fromany source. This allows both external traffic and health checkprobes to reach the backend instances.
    10. Protocols and ports: ChooseSpecified protocols and ports.Select theTCP checkbox and enter80.

  3. ClickCreate.

  4. ClickCreate firewall rule again to create the rule to allowincoming SSH connections to the instances. Use the following values:

    1. Name:allow-ssh
    2. Network:default
    3. Priority:1000
    4. Direction of traffic: Ingress
    5. Action on match: Allow
    6. Targets: Specified target tags
    7. Target tags:network-lb
    8. Source filter:IPv4 ranges
    9. Source IPv4 ranges:0.0.0.0/0
    10. Protocols and ports: ChooseSpecified protocols and ports.Select theTCP checkbox and enter22.

  5. ClickCreate.

gcloud

gcloud compute firewall-rules create allow-network-lb \    --target-tags network-lb \    --allow tcp:80
gcloud compute firewall-rules create allow-ssh \    --target-tags network-lb \    --allow tcp:22

Configuring load balancer components

These steps configure the following external passthrough Network Load Balancer components:

  • Health check. This example uses an HTTP health check that checks for anHTTP200 OK status code.

  • Backend service. Because the example passes HTTP traffic through theload balancer, the configuration specifies TCP, not UDP. Toillustrate failover, this backend service has a failover ratio of0.75.

  • Forwarding rule. This example creates a single forwarding rule.

Console

Start your configuration

  1. In the Google Cloud console, go to theLoad balancing page.

    Go to Load balancing

  2. ClickCreate load balancer.
  3. ForType of load balancer, selectNetwork Load Balancer (TCP/UDP/SSL) and clickNext.
  4. ForProxy or passthrough, selectPassthrough load balancer and clickNext.
  5. ForPublic facing or internal, selectPublic facing (external) and clickNext.
  6. ClickConfigure.

Backend configuration

  1. On theCreate external passthrough Network Load Balancer page, intheLoad Balancer name field, entertcp-network-lb.
  2. In theRegion list, selectus-west1.
  3. ClickBackend configuration. The load balancerName you enteredpreviously appears, but is not modifiable.
  4. ClickBackend configuration and do the following:
    1. In theHealth check list, clickCreate a health check, andthen enter the following information:
      • Name:hc-http-80
      • Protocol:TCP
      • Port:80
    2. ClickCreate.
    3. In theBackends section, in theInstance group list,selectig-a.
    4. ClickAdd backend and repeat this step to addig-c.
    5. Select theUse this instance group as a failover group for backupcheckbox.
    6. ClickSave and continue.
    7. ClickAdvanced configurations.
    8. ForFailover Ratio, enter0.75.
    9. Verify that there is a blue check mark next toBackendconfiguration before continuing.

Frontend configuration

  1. ClickFrontend configuration.
  2. Enter aName ofnetwork-lb-forwarding-rule.
  3. UnderIP, click the drop-down menu and selectCreate IP address.
    1. On theReserve a new static IP address screen, assign aName ofnetwork-lb-ip.
    2. ClickReserve.
  4. ChooseSingle, and enter80 for thePort number.

  5. Click theDone button.

    A blue circle with a checkmark to the left ofFrontend configuration indicates a successful set-up.

Review the configuration

  1. Click theReview and finalize button to check all of your configurationsettings for the load balancer.

  2. If the settings are correct, clickCreate. It takes a few minutes forthe load balancer to be created.

    On the load balancing screen, under the Backend column for your newload balancer, you should see a green checkmark showing that the newload balancer is healthy.

gcloud

  1. Create a static external IP address for your load balancer.

    gcloud compute addresses create network-lb-ip \    --region us-west1

  2. Create a new HTTP health check to test TCP connectivity to the VMson 80.

    gcloud compute health-checks create http hc-http-80 \    --region us-west1 \    --port 80

  3. Create the backend service for HTTP traffic:

    gcloud compute backend-services create network-lb-backend-service \    --protocol tcp \    --region us-west1 \    --health-checks hc-http-80 \    --health-checks-region us-west1 \    --failover-ratio 0.75

  4. Add the primary backend to the backend service:

    gcloud compute backend-services add-backend network-lb-backend-service \    --region us-west1 \    --instance-group ig-a \    --instance-group-zone us-west1-a

  5. Add the failover backend to the backend service:

    gcloud compute backend-services add-backend network-lb-backend-service \    --region us-west1 \    --instance-group ig-c \    --instance-group-zone us-west1-c \    --failover

  6. Create a forwarding rule for the backend service. Use the IP addressreserved in step 1 as the static external IP address of the loadbalancer.

    gcloud compute forwarding-rules create network-lb-forwarding-rule \    --region us-west1 \    --load-balancing-scheme external \    --address network-lb-ip\    --ports 80 \    --backend-service network-lb-backend-service

Testing

These tests show how to validate your load balancer configuration and learnabout its expected behavior.

Send traffic to the load balancer

This procedure sends external traffic to the load balancer. You'll use thisprocedure to complete the other tests.

  1. Connect to the client VM instance.

    gcloud compute forwarding-rules describe network-lb-forwarding-rule \    --region us-west1

  2. Make a web request to the load balancer usingcurl to contact its IPaddress.

    curl http://IP_ADDRESS

  3. Note the text returned by thecurl command. The name of the backend VMgenerating the response is displayed in that text; for example:Page servedfrom: vm-a1

Testing initial state

After you've configured the example load balancer, all four of the backend VMsshould be healthy:

  • the two primary VMs,vm-a1 andvm-a2
  • the two backup VMs,vm-c1 andvm-c2

Follow thetest procedure to send traffic to the load balancer.Repeat the second step a few times. The expected behavior is for traffic to beserved by the two primary VMs,vm-a1 andvm-a2, because both of them arehealthy. You should see each primary VM serve a response approximately half ofthe timebecause no session affinity has beenconfigured for this load balancer.

Testing failover

This test simulates the failure ofvm-a1 so you can observe failover behavior.

  1. Connect to thevm-a1 VM.

    gcloud compute ssh vm-a1 --zone us-west1-a

  2. Stop the Apache web server. After ten seconds, Google Cloudconsiders this VM to be unhealthy. (Thehc-http-80 health check that youcreated in the setup uses the default check interval of five seconds andunhealthy threshold of two consecutive failed probes.)

    sudo apachectl stop

  3. Follow thetest procedure to send traffic to theload balancer. Repeat the second step afew times. The expected behavior is for traffic to now be served by the twobackup VMs,vm-c1 andvm-c2. Because only one primary VM,vm-a2, ishealthy, the ratio of healthy primary VMs to total primary VMs is0.5.This number is less than the failover threshold of0.75, soGoogle Cloud reconfigured the load balancer's active pool to use thebackup VMs. You should see each backup VM serve a response approximatelyhalf of the timeas long as no session affinity has beenconfigured for this load balancer.

Testing failback

This test simulates failback by restarting the Apache server onvm-a1.

  1. Connect to thevm-a1 VM.

    gcloud compute ssh vm-a1 --zone us-west1-a

  2. Start the Apache web server and wait 10 seconds.

    sudo apachectl start

  3. Follow theclient test procedure. Repeat the second step afew times. The expected behavior is for traffic to be served by the twoprimary VMs,vm-a1 andvm-a2. With both primary VMs being healthy, theratio of healthy primary VMs to total primary VMs is1.0, greater than thefailover threshold of0.75, so Google Cloud configured the activepool to use the primary VMs again.

Adding more backend VMs

This section extends the example configuration by adding more primary andbackup VMs to the load balancer. It does so by creating two more backendinstance groups to demonstrate that you can distribute primary and backup VMsamong multiple zones in the same region:

  • A third instance group,ig-d inus-west1-c, serves as a primarybackend with two VMs:
    • vm-d1
    • vm-d2
  • A fourth instance group,ig-b inus-west1-a, serves as a failoverbackend with two VMs:
    • vm-b1
    • vm-b2

The modified architecture for this example looks like this:

Multi-zone external passthrough Network Load Balancer failover.
Multi-zone external passthrough Network Load Balancer failover (click to enlarge).

Create additional VMs and instance groups

Follow these steps to create the additional primary and backup VMs and theircorresponding unmanaged instance groups.

Console

Create backend VMs

  1. In the Google Cloud console, go to theVM instances page.

    Go to VM instances

  2. Repeat the following steps to create four VMs, using the following nameand zone combinations.

    • Name:vm-b1, zone:us-west1-a
    • Name:vm-b2, zone:us-west1-a
    • Name:vm-d1, zone:us-west1-c
    • Name:vm-d2, zone:us-west1-c

  3. ClickCreate instance.

  4. Set theName as indicated in step 2.

  5. For theRegion, chooseus-west1, and choose aZone asindicated in step 2.

  6. In theBoot disk section, ensure that the selected image isDebian GNU/Linux 9 Stretch. ClickChoose to change the image ifnecessary.

  7. ClickAdvanced options.

  8. ClickNetworking and configure the following field:

    1. ForNetwork tags, enternetwork-lb.

  9. ClickManagement. Enter the following script into theStartup script field. The script contents are identical forall four VMs:

    #! /bin/bashapt-get updateapt-get install apache2 -ya2ensite default-ssla2enmod sslvm_hostname="$(curl -H "Metadata-Flavor:Google" \http://metadata.google.internal/computeMetadata/v1/instance/name)"echo "Page served from: $vm_hostname" | \tee /var/www/html/index.htmlsystemctl restart apache2

  10. ClickCreate.

Create instance groups

  1. In the Google Cloud console, go to theInstance groups page.

    Go to Instance groups

  2. Repeat the following steps to create two unmanaged instance groups eachwith two VMs in their one, using these combinations.

    • Instance group:ig-b, zone:us-west1-a, VMs:vm-b1 andvm-b2
    • Instance group:ig-d, zone:us-west1-c, VMs:vm-d1 andvm-d2

  3. ClickCreate instance group.

  4. ClickNew unmanaged instance group.

  5. SetName as indicated in step 2.

  6. In theLocation section, chooseus-west1 for theRegion, andthen choose aZone as indicated in step 2.

  7. ForNetwork, enterdefault.

  8. In theVM instances section, add the VMs as indicated in step 2.

  9. ClickCreate.

gcloud

  1. Create four VMs by running the following command four times, usingthese four combinations forVM-NAME andZONE. The script contentsare identical for all four VMs.

    • VM-NAME ofvm-b1 andZONE ofus-west1-a
    • VM-NAME ofvm-b2 andZONE ofus-west1-a
    • VM-NAME ofvm-d1 andZONE ofus-west1-c
    • VM-NAME ofvm-d2 andZONE ofus-west1-c
    gcloud compute instances createVM-NAME \    --zone=ZONE \    --image-family=debian-12 \    --image-project=debian-cloud \    --tags=network-lb \    --metadata=startup-script='#! /bin/bash      apt-get update      apt-get install apache2 -y      a2ensite default-ssl      a2enmod ssl      vm_hostname="$(curl -H "Metadata-Flavor:Google" \      http://metadata.google.internal/computeMetadata/v1/instance/name)"      echo "Page served from: $vm_hostname" | \      tee /var/www/html/index.html      systemctl restart apache2'

  2. Create the two unmanaged instance groups in each zone:

    gcloud compute instance-groups unmanaged create ig-b \    --zone us-west1-agcloud compute instance-groups unmanaged create ig-d \    --zone us-west1-c

  3. Add the VMs to the appropriate instance groups:

    gcloud compute instance-groups unmanaged add-instances ig-b \    --zone us-west1-a \    --instances vm-b1,vm-b2gcloud compute instance-groups unmanaged add-instances ig-d \    --zone us-west1-c \    --instances vm-d1,vm-d2

Adding a primary backend

You can use this procedure as a template for how to add an unmanaged instancegroup to an existing external passthrough Network Load Balancer's backend service as aprimary backend. For the example configuration, this procedure shows you how toadd instance groupig-d as a primary backend to thenetwork-lb load balancer.

Console

Edit the load balancer configuration to add a primary backend.

  1. In the Google Cloud console, go to theLoad balancing page.

    Go to Load balancing

  2. Click on the load balancer you want to modify.

  3. ClickEdit.

  4. ClickBackend configuration and make the following changes:

    1. UnderBackends, clickAdd Backend.
    2. From the dropdown, select the instance group to be added as aprimary backend. In this case,ig-d.
    3. ClickDone.
    4. Verify that there is a blue check mark next toBackendconfiguration before continuing.

Review the configuration

  1. Click theReview and finalize button and confirm that the newprimary backend is visible underBackend.

  2. If the settings are correct, clickCreate. It takes a few minutes forthe load balancer to be created.

    On the load balancing screen, under the Backend column for your newload balancer, you should see a green checkmark showing that the newload balancer is healthy.

gcloud

Use the followinggcloud command to add a primary backendto an existing external passthrough Network Load Balancer's backend service.

gcloud compute backend-services add-backendBACKEND_SERVICE_NAME \   --instance-groupINSTANCE_GROUP_NAME \   --instance-group-zoneINSTANCE_GROUP_ZONE \   --regionREGION

where:

  • BACKEND_SERVICE_NAME is the name of the load balancer's backendservice. For the example, usenetwork-lb-backend-service.
  • INSTANCE_GROUP_NAME is the name of the instance group to add as aprimary backend. For the example, useig-d.
  • INSTANCE_GROUP_ZONE is the zone where the instance group is defined.For the example, useus-west1-c.
  • REGION is the region of the load balancer. For the example, useus-west1.

Adding a failover backend

You can use this procedure as a template for how to add an unmanaged instancegroup to an existing external passthrough Network Load Balancer's backend service as afailover backend. For the example configuration, this procedure shows you how toadd instance groupig-b as a failover backend to thenetwork-lb load balancer.

Console

Edit the load balancer configuration to add a primary backend.

  1. In the Google Cloud console, go to theLoad balancing page.

    Go to Load balancing

  2. Click on the load balancer you want to modify.

  3. ClickEdit.

  4. ClickBackend configuration and make the following changes:

    1. UnderBackends, clickAdd Backend.
    2. From the dropdown, select the instance group to be added as afailover backend. In this case,ig-b.
    3. Select theUse this instance group as a failover group for backupcheckbox.
    4. ClickDone.
    5. Verify that there is a blue check mark next toBackendconfiguration before continuing.

Review the configuration

  1. Click theReview and finalize button and confirm that the newprimary backend is visible underBackend.

  2. If the settings are correct, clickCreate. It takes a few minutes forthe load balancer to be created.

    On the load balancing screen, under the Backend column for your newload balancer, you should see a green checkmark showing that the newload balancer is healthy.

gcloud

Use the followinggcloud command to add a failover backendto an existing external passthrough Network Load Balancer's backend service.

gcloud compute backend-services add-backendBACKEND_SERVICE_NAME \   --instance-groupINSTANCE_GROUP_NAME \   --instance-group-zoneINSTANCE_GROUP_ZONE \   --regionREGION \   --failover

where:

  • BACKEND_SERVICE_NAME is the name of the load balancer's backendservice. For the example, usenetwork-lb-backend-service.
  • INSTANCE_GROUP_NAME is the name of the instance group to add as afailover backend. For the example, useig-b.
  • INSTANCE_GROUP_ZONE is the zone where the instance group is defined.For the example, useus-west1-a.
  • REGION is the region of the load balancer. For the example, useus-west1.

Converting a primary or failover backend

You can use convert a primary backend to a failover backend, or vice versa,without having to remove the instance group from the external passthrough Network Load Balancer'sbackend service.

gcloud

Use the followinggcloud command to convert an existing primary backend toa failover backend:

gcloud compute backend-services update-backendBACKEND_SERVICE_NAME \   --instance-groupINSTANCE_GROUP_NAME \   --instance-group-zoneINSTANCE_GROUP_ZONE \   --regionREGION \   --failover

Use the followinggcloud command to convert an existing failover backend toa primary backend:

gcloud compute backend-services update-backendBACKEND_SERVICE_NAME \   --instance-groupINSTANCE_GROUP_NAME \   --instance-group-zoneINSTANCE_GROUP_ZONE \   --regionREGION \   --no-failover

where:

  • BACKEND_SERVICE_NAME is the name of the load balancer's backendservice.
  • INSTANCE_GROUP_NAME is the name of the instance group.
  • INSTANCE_GROUP_ZONE is the zone where the instance group is defined.
  • REGION is the region of the load balancer.

Configuring failover policies

This section describes how to manage afailoverpolicy for an external passthrough Network Load Balancer's backendservice. A failover policy consists of the:

  • Failover ratio
  • Dropping traffic when all backend VMs are unhealthy
  • Connection draining on failover

For more information on theparameters of a failover policy, see:

Defining a failover policy

The following instructions describe how to define the failover policy for anexisting external passthrough Network Load Balancer.

gcloud

To define a failover policy using the gcloud CLI, update theload balancer's backend service:

gcloud compute backend-services updateBACKEND_SERVICE_NAME \   --regionREGION \   --failover-ratioFAILOVER_RATIO \   --connection-drain-on-failover \   --connection-draining-timeout=CONNECTION_DRAINING_TIMEOUT \   --drop-traffic-if-unhealthy

where:

  • BACKEND_SERVICE_NAME is the name of the load balancer's backendservice. For the example, usenetwork-lb.
  • REGION is the region of the load balancer. For the example, useus-west1.
  • FAILOVER_RATIO is the failover ratio. Possible values are between0.0 and1.0, inclusive. For the example, use0.75.
  • CONNECTION_DRAINING_TIMEOUT allows TCP connections to persist, even onVMs that are no longer in the active pool, for up to the duration of theconnection draining timeout.

Viewing a failover policy

The following instructions describe how to view the existing failover policy foran external passthrough Network Load Balancer.

gcloud

To list the failover policy settings using the gcloud CLI,use the following command. Undefined settings in a failover policy use thedefault failover policy values.

gcloud compute backend-services describeBACKEND_SERVICE_NAME \   --regionREGION \   --format="get(failoverPolicy)"

where:

  • BACKEND_SERVICE_NAME is the name of the load balancer's backendservice. For the example, usenetwork-lb-backend-service.
  • REGION is the region of the load balancer. For the example, useus-west1.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.