This page explains concepts about how internal passthrough Network Load Balancers distribute traffic.

Backend selection and connection tracking

Backend selection and connection tracking work together to balance multipleconnections across different backends and to route all packets for eachconnection to the same backend. This is accomplished with a two-part strategy.First, a backend is selected using consistent hashing. Then, this selection isrecorded in a connection tracking table.

The following steps capture the backend selection and connection trackingprocess.

1.Check for a connection tracking table entry to use a previously selected backend

For anexisting connection, the load balancer uses the connectiontracking table to identify the previously selected backend for that connection.

The load balancer attempts to match each load-balanced packet with an entryin its connection tracking table using the following process:

• If the packet is a TCP packet with theSYN flag:

  • If the load balancer's connection tracking mode isPER_CONNECTION,continue to theIdentify eligible backendsstep. InPER_CONNECTION tracking mode, a TCP packet with theSYNflag always represents a new connection, regardless of theconfigured session affinity.

  • If the load balancer's connection tracking mode isPER_SESSIONand the session affinity is eitherNONE orCLIENT_IP_PORT_PROTO,continue to theIdentify eligible backendsstep. InPER_SESSIONtracking mode, a TCP packet with theSYN flag represents a newconnection only when using one of the 5-tuple session affinityoptions (NONE orCLIENT_IP_PORT_PROTO).

• For all other packets, the load balancer checks if the packet matches anexisting connection tracking table entry. The connection tuple (a set ofpacket characteristics) used to compare the packet to existing connectiontracking table entries depends on the connection tracking mode and sessionaffinity you configured. For information about which connection tupleis used for connection tracking, see thetable in theConnection trackingmode section.

  • If the packet matches a connection tracking table entry, the loadbalancer sends the packet to the previously selected backend.

  • If the packet doesn't match a connection tracking table entry,continue to theIdentify eligible backendsstep.

  For information about how long a connection tracking table entrypersists and under what conditions it persists,see theCreate a connection tracking table entry step.

2.Select an eligible backend for a new connection

For anew connection, the load balancer uses theconsistent hashingalgorithm to select a backend from among the eligible backends.

The following steps outline the process to select an eligible backend for a newconnection and then record that connection in a connection tracking table.

2.1Identify eligible backends

This step models which backends are candidates to receive new connections,taking into consideration health andfailoverpolicyconfiguration:

No failover policy

The set of eligible backends depends only on health checks:

• When at least one backend is healthy, eligible backends are all healthybackends.

• When all backends are unhealthy, eligible backends areall backends.

Failover policy configured

The set of eligible backends depends on health checks and failover policyconfiguration:

• When at least one backend is healthy, eligible backends are defined asfollows, using the first condition that is true from this ordered list:

  • If there are no healthy primary backends, eligible backends areallhealthy failover backends.
  • If there are no healthy failover backends, eligible backends areallhealthy primary backends.
  • If the failover ratio is set to0.0 (the default value), eligiblebackends areall healthy primary backends.
  • If the ratio of the number of healthy primary backends compared to thetotal number of primary backends is greater than or equal to theconfigured failover ratio, eligible backends areall healthy primarybackends.
  • The eligible backends areall healthy failover backends.

• When there are no healthy backends, eligible backends are defined asfollows:

  • If the load balancer's failover policy is configured to drop newconnections when no backends are healthy, the set of eligible backends isempty. The load balancer drops the packets for new connections.
  • If the load balancer's failover policy isnot configured to drop newconnections when no backends are healthy, health checks aren't relevant.Eligible backends areall primary backends.

2.2Adjust eligible backends for zonal affinity

This step is skipped if any of the following is true:

• Zonal affinity is disabled
• The client isincompatible with zonalaffinity
• Azonal matchwith the zone of a compatible client doesn't happen

If zonal affinity is enabled, a client is compatible with zonal affinity, and azonal match happens, new connections from the client are routed to an adjustedset of eligible backends. For more information, see the following:

• HowZONAL_AFFINITY_STAY_WITHIN_ZONEworks

• HowZONAL_AFFINITY_SPILL_CROSS_ZONE and spillover ratiowork

2.3Select an eligible backend

The load balancer usesconsistent hashingto select an eligible backend. The load balancer maintains hashesof eligible backends, mapped to a unit circle. When processing a packetfor a connection that's not in the connection tracking table, the loadbalancer computes a hash of the packet characteristics and maps that hashto the same unit circle, selecting an eligible backend on the circle'scircumference. The set of packet characteristics used to calculate thepacket hash is defined by thesession affinity setting.

• If a session affinity isn't explicitly configured, theNONE sessionaffinity is the default.

• The load balancer assigns new connections to eligible backends in a waythat is as consistent as possible even if the number of eligible backendschanges. The following benefits of consistent hashing show how the loadbalancer selects eligible backends for possible new connections that donot have connection tracking table entries:

  • The load balancer selects the same backend for all possible new connectionsthat have identical packet characteristics, as defined by session affinity,if the set of eligible backends doesnot change.

  • When a new eligible backend is added, approximately1/N possible newconnections map to the newly added backend. In this situation,N isthe count of eligible backendsafter the new backend is added.

  • When an eligible backend is removed, approximately1/N possible newconnections map to one of theN-1 remaining backends. In this situation,N is the count of backendsbefore the eligible backend is removed.

2.4Create a connection tracking table entry

After selecting a backend, theload balancer creates a connection tracking table entry. The connectiontracking table entry maps packet characteristics to the selected backend.The packet header fields used for this mapping depend on the connectiontracking mode and session affinity you configured.

The load balancer removes connection tracking table entries according to thefollowing rules:

• A connection tracking table entry is removed after the connection has beenidle. Unless you configure a custom idle timeout, the load balancer uses adefault idle timeout of 600 seconds. For more information, seeIdletimeout.

• Connection tracking table entries arenot removed when a TCP connectionis closed with aFIN orRST packet. Anynew TCP connection alwayscarries theSYN flag and is processed as described in theCheck for aconnection tracking table entry step.

• If a failover policy is configured and the connection draining on failoverand failback setting is disabled, the load balancer removes all entries inthe connection tracking table when the eligible backends switch from primaryto failover backends (failover), or switch from failover to primary backends(failback). For more information, seeConnection draining on failover andfailback.

• Entries in the connection tracking table can be removed if a backend becomesunhealthy. This behavior depends on the connection tracking mode, theprotocol, and the connection persistence on unhealthy backends setting. Formore information, seeConnection persistence on unhealthybackends.

• Entries in the connection tracking table are removed after the connectiondraining timeout that occurs following an event like deleting a backend VMor removing a backend VM from an instance group or NEG. For moreinformation, seeEnable connectiondraining.

Session affinity

Session affinity controls the distribution of new connections from clients tothe load balancer's backends.Internal passthrough Network Load Balancers use session affinity toselect a backend from a set of eligible backends as described in theIdentifyeligible backends andSelect an eligible backend steps in theBackend selection and connectiontracking section. You configure session affinity onthe backend service, not on each backend instance group or NEG.

Internal passthrough Network Load Balancers support the following session affinity settings. Eachsession affinity setting usesconsistent hashing to select an eligible backend.The session affinity setting determines which fields from the IP header andLayer 4 headers are used to calculate the hash.

To learn how the different session affinity settings affect the backendselection and connection tracking methods, see thetable in theConnection trackingmode section.

Selecting an eligible backend depends on calculating a hashof packet characteristics. Except for theCLIENT_IP_NO_DESTINATION sessionaffinity (1-tuple hash), the hash depends, in part, on thepacket destinationIP address.

The load balancer selects the same backend for all possible new connections thathave identical packet characteristics, as defined by session affinity, if theset of eligible backends does not change. If you need the same backend VM toprocess all packets from a client, based solely on the source IP address,regardless of destination IP addresses, use theCLIENT_IP_NO_DESTINATIONsession affinity.

Connection tracking policy

This section describes the settings that control the connection trackingbehavior of internal passthrough Network Load Balancers. A connection tracking policy includes the followingsettings:

• Connection tracking mode
• Connection persistence on unhealthy backends
• Idle timeout

Connection tracking mode

The load balancer's connection tracking table maps connection tuples topreviously selected backends in a hash table. The set of packet characteristicsthat compose each connection tuple is determined by the connection trackingmode and session affinity.

Internal passthrough Network Load Balancers track connections for all protocols that they support.

The connection tracking mode refers to the granularity of the each connectiontuple in the load balancer's connection tracking table. The connection tuplecan be 5-tuple or 3-tuple (PER_CONNECTION mode), or it can match the sessionaffinity setting (PER_SESSION mode).

• PER_CONNECTION. This is the default connectiontracking mode. This connection tracking mode uses a 5-tuplehash or a 3-tuple hash. Non-fragmented packets that include port informationsuch as TCP packets and non-fragmented UDP packets are tracked with 5-tuplehashes. All other packets are tracked with 3-tuple hashes.

• PER_SESSION. This connection tracking mode uses a hash consisting of thesame packet characteristics used by the session affinity hash. Depending on thechosen session affinity,PER_SESSION can result in connections that morefrequently match an existing connection tracking table entry, reducing thefrequency that a backend needs to be selected by the session affinity hash.

The following table summarizes how connection tracking mode and session affinitywork together to route all packets for each connection to the same backend.

To learn how to change the connection tracking mode, seeConfigure aconnection trackingpolicy.

Connection persistence on unhealthy backends

The connection persistence settings control whether an existing connectionpersists on a selected backend VM or endpoint after that backend becomesunhealthy, as long as the backend remains in the load balancer's configuredbackend group (in an instance group or a NEG).

The following connection persistence options are available:

• DEFAULT_FOR_PROTOCOL (default)
• NEVER_PERSIST
• ALWAYS_PERSIST

The following table summarizes connection persistence options and howconnections persist for different protocols, session affinity options, andtracking modes.

To learn how to change connection persistence behavior, seeConfigure aconnection trackingpolicy.

Idle timeout

A connection tracking table entry is removed after the connection has been idlefor a certain period of time. Unless you configure a custom idle timeout, theload balancer uses a default idle timeout value of 600 seconds (10 minutes).

The following table shows the minimum and maximum configurable idle timeoutvalues for different combinations of session affinity and connection trackingmode settings.

To learn how to change the idle timeout value, seeConfigure aconnection trackingpolicy.

Connections for single-client deployments

When testing connections to the IP address of an internal passthrough Network Load Balancer that only hasone client, you should keep the following in mind:

• If the client VM is not a VM being load balanced—that is, it is notalso a backend VM, new connections are delivered to the load balancer'shealthy backend VMs. However, because allsession affinityoptions rely on at least the client system's IP address,connections from the same client might be distributed to the same backend VMmore frequently than you might expect.

  Practically, this means that you cannot accurately monitor trafficdistribution through an internal passthrough Network Load Balancer by connecting to it from a singleclient. The number of clients needed to monitor traffic distribution variesdepending on the load balancer type, the type of traffic, and the number ofhealthy backends.

• If the client VM is also a backend VM of the load balancer, connectionssent to the IP address of the load balancer's forwarding rule are alwaysanswered by the same backend VM (which is also the client VM). This happensregardless of whether the backend VM is healthy. It happens forall trafficsent to the load balancer's IP address, not just traffic on the protocol andports specified in the load balancer's internal forwarding rule.

  For more information, seesending requests from load-balanced VMs.

Connection draining

Connection draining provides a configurable amount of additional time forestablished connections to persist in the load balancer's connectiontracking table when one of the following actions takes place:

• A virtual machine (VM) instance is removed from a backend instance group(this includes abandoning an instance in a backend managed instance group)
• A VM is stopped or deleted (this includes automatic actions like rollingupdates or scaling down a backend managed instance group)
• An endpoint is removed from a backend network endpoint group (NEG)

By default, connection draining for the aforementioned actions is disabled.For information about how connection draining is triggered and how toenable connection draining, seeEnabling connectiondraining.

UDP fragmentation

• UDP packets might become fragmented before reaching a Google CloudVPC network.
• Google Cloud VPC networks forward UDP fragments as theyarrive (without waiting for all fragments to arrive).
• Non-Google Cloud networks and on-premises network equipment mightforward UDP fragments as they arrive, delay fragmented UDP packets untilall fragments have arrived, or discard fragmented UDP packets. For details,see the documentation for the network provider or network equipment.

If you expect fragmented UDP packets and need to route them to the same backends,use the following forwarding rule and backend service configuration parameters:

• Forwarding rule configuration: Use only oneUDPforwarding rule per load-balanced IP address, and configure the forwardingrule to accept traffic on all ports. This ensures that all fragments arrive atthe same forwarding rule. Even though the fragmented packets (other than thefirst fragment) lack a destination port, configuring the forwarding rule toprocess traffic for all ports also configures it to receive UDP fragments thathave no port information. To configure all ports, either use theGoogle Cloud CLI to set--ports=ALL or use the APIto setallPorts toTrue.

• Backend service configuration: Set the backend service'ssessionaffinity toCLIENT_IP (2-tuple hash) orCLIENT_IP_PROTO (3-tuple hash) so that the same backend is selected for UDPpackets that include port information and UDP fragments (other than the firstfragment) that lack port information. Set the backend service'sconnectiontracking mode toPER_SESSION so that the connectiontracking table entries are built by using the same 2-tuple or 3-tuple hashes.

Failover

An internal passthrough Network Load Balancer lets you designate some backends as failoverbackends. These backends are only used when the number of healthy VMs in theprimary backend instance groups has fallen below a configurable threshold.By default, if all primary and failover VMs are unhealthy, as a last resortGoogle Cloud distributes new connections only among all the primary VMs.

When you add a backend to an internal passthrough Network Load Balancer's backend service,by default that backend is a primary backend. You can designate a backendto be a failover backend when you add it to the load balancer's backend service,or by editing the backend service later.

For more information about how failover is used for backend selection andconnection tracking, see theIdentify eligible backendsandCreate a connection tracking table entry stepsin theBackend selection and connection tracking section.

For more information about how failover works, seeFailover for internal passthrough Network Load Balancers.

What's next

• To configure and test an internal passthrough Network Load Balancer that uses failover, seeConfigure failover for internal passthrough Network Load Balancers.
• To configure and test an internal passthrough Network Load Balancer, seeSet up an internal passthrough Network Load Balancer.