Network bandwidth

Google Cloud accounts for bandwidth per compute instance, notper virtual network interface (vNIC) or IP address. An instance'smachine type defines its maximum possibleegress rate; however, you can only achieve that maximum possible egress rate inspecific situations.

This page outlines the network bandwidth limits, which are useful when planningyour deployments. It categorizes bandwidth using two dimensions:

  • Egress or ingress: As used on this page, egress and ingress are alwaysfrom the perspective of a Google Cloud instance:
    • Packets sentfrom a Google Cloud instance compose itsegress(outbound) traffic.
    • Packets sentto a Google Cloud instance compose itsingress(inbound) traffic.
  • How the packet is routed: A packet can be routed from a sending instanceor to a receiving instance using routes whose next hops arewithin aVPC network or routes outside of a VPC network.

All of the information on this page is applicable to Compute Enginecompute instances, as well as products that depend on Compute Engineinstances. For example, a Google Kubernetes Engine node is a Compute Engineinstance.

Note: For network bandwidth information for Accelerator-optimized machineseries, seeNetworking and GPU machines.

Configurations that impact network bandwidth

Neitheradditional virtual network interfaces (vNICs)nor additional IP addresses per vNIC increase ingress or egress bandwidth for acompute instance. For example, a C3 VM with 22 vCPUs is limited to23 Gbps total egress bandwidth. If you configure the C3 VM with twovNICs, the VM is still limited to 23 Gbps total egress bandwidth,not 23 Gbps bandwidth per vNIC.

The following sections describe how other compute instance configurations canimpact network bandwidth.

Use per VM Tier_1 networking performance

To get the highest possible ingress and egress bandwidth,configure Tier_1 networkingfor your compute instance.

Dynamic Network Interfaces

Dynamic Network Interfaces use thebandwidth of their parent vNIC. There is no traffic isolation within a parentvNIC. Network traffic from a Dynamic NIC can starve the otherDynamic NICs associated with the same parent vNIC. To avoid thisconflict, you can use Linux traffic control (TC) to craft application-specifictraffic shaping policies. These policies help to implement either fairness orcertain types of priority. For prioritization, you map traffic (for example forDynamic NICs) to a traffic class, and then map that traffic classto a quality of service. For an example of this approach, seeTraffic shaping with Red Hat.

Dynamic NICs aren't supported for Compute Engineinstances that run a Windows OS.

HPC computing with Cloud RDMA

Bandwidth is required for all three stages of a high performance computing (HPC)job: load, compute, and store. During the load and store phases, applicationscommonly use TCP for communication, and remote direct memory address (RDMA)for the compute stages. H4D instances using GVNIC for the TCP traffic provideup to 200 Gbps network bandwidth. If you also configure an H4D instanceto use Cloud RDMA, then the network bandwidth is shared among theconfigured network interfaces.

Network bandwidth allocation between Cloud RDMA traffic and TCP trafficis done dynamically. Instead of limiting both Cloud RDMA and TCPtraffic to 100 Gbps bandwidth each, the GVNIC network interface can use allof the available bandwidth when the Cloud RDMA network interface is notbeing used. Similarly, the Cloud RDMA network interface can use all ofthe available bandwidth when the GVNIC network interface is not being used. Whenboth network interface types are in use, Cloud RDMA traffic haspriority over TCP traffic because it is more performance sensitive.

Bandwidth summary

The following table illustrates the maximum possible bandwidth based on whethera packet is sent from (egress) or received by (ingress) a compute instance andthe packet routing method.

Egress bandwidth limits

Routing within
a VPC network
  • Primarily defined by aper-instance maximum egress bandwidth based on the sending instance's machine type and whether Tier_1 networking is enabled.

    • N2, N2D, C2, C2D, M3, and C4A VMs with Tier_1 networking support egress bandwidth limits up to 100 Gbps.
    • H3 VMs support VM-to-VM egress bandwidth limits up to 200 Gbps.
    • H4D instances support VM-to-VM egress bandwidth of up to 200 Gbps for Cloud RDMA and gVNIC combined.
    • X4, M4, A2, and G2 instances support egress bandwidth limits up to 100 Gbps.
    • G4 instances support egress bandwidth limits up to 400 Gbps.
    • A4X instances support egress bandwidth limits up to 2,000 Gbps.
    • A4 and A3 instances support egress bandwidth limits up to 3,600 Gbps.
    • C4, C4D, C3, C3D, and Z3 instances support up to 200 Gbps egress bandwidth limits with Tier_1 networking.
  • For other factors, definitions, and scenarios, seeEgress to destinations routable within a VPC network.
Routing outside
a VPC network
  • Primarily defined by a per-instance maximum egress bandwidth based on the sending instance's machine type and whether Tier_1 networking is enabled. Except for H3 VMs, a sending instance's maximum possible egress to a destination outside of its VPC network cannot exceed the following:

    • 7 Gbps total when Tier_1 networking isn't enabled
    • 25 Gbps total when Tier_1 networking is enabled
    • 3 Gbps per flow
  • For other factors, definitions, and caveats, seeEgress to destinations outside of a VPC network.

Ingress bandwidth limits

Routing within
a VPC network
  • Generally, ingress rates are similar to the egress rates for a machine type.
  • To get the highest possible ingress bandwidth,enable Tier_1 networking.
  • The size of your compute instance, the capacity of the server NIC, the traffic coming into other guest VMs running on the same host hardware, your guest OS network configuration, and the number of disk reads performed by your instance can all impact the ingress rate.
  • Google Cloud doesn't impose any additional limitations on ingress rates within a VPC network.
  • For other factors, definitions, and scenarios, seeIngress to destinations routable within a VPC network.
Routing outside
a VPC network
  • Google Cloud protects each compute instance by limiting ingress traffic routed outside a VPC network. The limit is the first of the following rates encountered:

    • 1,800,000 pps (packets per second)
    • 30 Gbps
  • For a machine series that supports multiple physical NICs, such as A3, the limit is the first of the following rates encountered:

    • 1,800,000 pps (packets per second) per physical NIC
    • 30 Gbps per physical NIC
  • For other factors, definitions, and scenarios, seeIngress to destinations outside of a VPC network.

Egress bandwidth

Google Cloud limits outbound (egress) bandwidth using per-instance maximumegress rates. These rates are based the machine type of the compute instancethat is sending the packet and whether the packet's destination is accessibleusing routes within a VPC network or routes outside of aVPC network. Outbound bandwidth includes packets emitted by allof the instance's NICs and data transferred to all Hyperdiskand Persistent Disk volumes connected to the instance.

Per-instance maximum egress bandwidth

Per-instance maximum egress bandwidth is generally 2 Gbps per vCPU, butthere are some differences and exceptions, depending on the machine series.The following table shows the range of maximum limits for egress bandwidth fortraffic routed within a VPC network.

The following table summarizes the maximum egress bandwidth for each machineseries. You can find the per-instance maximum egress bandwidth for every machinetype listed on its specific machine family page (using the links for eachmachine series in the table).

Maximum per-instance egress limit
Machine seriesStandardTier_1 networking
C4 andC4D100 Gbps200 Gbps
C4A50 Gbps100 Gbps
C3 andC3D100 Gbps200 Gbps
C2 andC2D32 Gbps100 Gbps
E216 GbpsN/A
E2 shared-core2 GbpsN/A
H4D andH3200 GbpsN/A
M4100 GbpsN/A
M332 Gbps100 Gbps
M232 Gbps on Intel Cascade Lake or later CPU
16 Gbps on other CPU platforms
N/A
M132 GbpsN/A
N4,N4A (Preview), andN4D50 GbpsN/A
N2 andN2D32 Gbps100 Gbps
N1 (excluding VMs with 1 vCPU)32 Gbps on Intel Skylake and later CPU
16 Gbps on earlier CPU platforms
N/A
N1 machine types with 1 vCPU2 GbpsN/A
N1 shared-core (f1-micro and g1-small)1 GbpsN/A
T2A andT2D32 GbpsN/A
X4100 GbpsN/A
Z3100 Gbps200 Gbps

For network bandwidth information for Accelerator-optimized machine series, seeNetworking and GPU machines.

Per-instance maximum egress bandwidth is not a guarantee. The actual egressbandwidth can be lowered according to factors such as the followingnon-exhaustive list:

  • Using VirtIO instead ofgVNIC withcompute instances that support both
  • Packet size
  • Protocol overhead
  • The number of flows
  • Ethernet driver settings of the compute instance's guest OS, such aschecksum offload and TCP segmentation offload (TSO)
  • Network congestion
  • In a situation where Persistent Disk I/Os compete with other network egresstraffic, 60% of the maximum network bandwidth is given to Persistent Diskwrites, leaving 40% for other network egress traffic. SeeFactors that affect disk performancefor more details.

To get the largest possible per-instance maximum egress bandwidth:

  • Enableper VM Tier_1 networking performancewith larger machine types.
  • Use the largest VPC networkmaximum transmission unit (MTU) supported by your networktopology. Larger MTUs can reduce packet-header overhead and increase payloaddata throughput.
  • Use the latest gVNIC driver version.
  • Use third generation or later machine series that useTitaniumto offload network processing from the host CPU.

Egress to destinations routable within a VPC network

From the perspective of a sending instance and for destination IP addressesaccessible by means of routeswithin a VPC network,Google Cloud limits outbound traffic using these rules:

  • Per-VM maximum egress bandwidth: The per-instance maximum egress bandwidthdescribed in thePer-instance maximum egress bandwidthsection.
  • Per-project inter-regional egress bandwidth: If a sending instance and aninternal destination or its next hop are in different regions,Google Cloud enforces a quota based on the region and project of thesending instance and the region of the internal destination or next hop. Formore information about this quota, seeInter-region network egress bandwidth(Mbps) from Compute instances in theVPC quotas and limits documentation.
  • Cloud VPN and Cloud Interconnect limits: When sendingtraffic from an instance to an internal IP address destination routable by anext hop Cloud VPN tunnel or Cloud Interconnect VLANattachment, egress bandwidth is limited by:

Destinations routable within a VPC network include all of thefollowing destinations, each of which is accessible from the perspective of thesending instance by a route whose next hop isnot the default internetgateway:

  • Regional internal IPv4 addresses insubnet primary IPv4 and subnet secondary IPv4 address ranges,including private IPv4 address ranges and privately used public IPv4 addressranges, used by these destination resources:
    • The primary internal IPv4 address of a receiving instance's networkinterface (vNIC). (When a sending instance connects to another instance'svNICexternal IPv4 address, packets are routed using a next hop defaultinternet gateway, soEgress to destinations outside of a VPC networkapplies instead.)
    • An internal IPv4 address in an alias IP range of a receiving instance'svNIC.
    • An internal IPv4 address of an internal forwarding rule for either protocolforwarding or for an internal passthrough Network Load Balancer.
  • Global internal IPv4 addresses for these destination resources:
  • Internal IPv6 subnet address ranges used bythese destination resources:
    • An IPv6 address from the/96 IPv6 address range assigned to a dual-stackor IPv6-only receiving instance's vNIC.
    • An IPv6 address from the/96 IPv6 address range of an internal forwardingrule for either protocol forwarding or for an internal passthrough Network Load Balancer.
  • External IPv6 subnet address ranges used bythese destination resourceswhen packets are routed using subnet routesor peering subnet routes within the VPC network or by customroutes within the VPC network that do not use the defaultinternet gateway next hop:
    • An IPv6 address from the/96 IPv6 address range assigned to a dual-stackor IPv6-only receiving instance's vNIC.
    • An IPv6 address from the/96 IPv6 address range of an external forwardingrule for either protocol forwarding or for an external passthrough Network Load Balancer.
  • Other destinations accessible using the following VPC networkroutes:

The following list ranks traffic from sending instances to internaldestinations, from highest possible bandwidth to lowest:

Egress to destinations outside of a VPC network

From the perspective of a sending instance and for destination IP addressesoutside of a VPC network, Google Cloud limits outboundtraffic to whichever of the following rates is reached first:

Destinations outside of a VPC network include all of thefollowing destinations, each of which is accessible by a route in the sendinginstance's VPC network whose next hopis the defaultinternet gateway:

  • Global external IPv4 and IPv6 addresses for external proxy Network Load Balancers andexternal Application Load Balancers
  • Regional external IPv4 addresses for Google Cloud resources, includingVM vNIC external IPv4 addresses, external IPv4 addresses for external protocolforwarding, external passthrough Network Load Balancers, and response packets to Cloud NATgateways.
  • Regional external IPv6 addresses in dual-stack or IPv6-only subnets withexternal IPv6 address ranges used byexternal IPv6 addresses of dual-stack or IPv6-only instances, externalprotocol forwarding, and external passthrough Network Load Balancers. The subnet must be located in aseparate, non-peered VPC network. The destination IPv6 addressrange must be accessible using a route in the sending instance'sVPC network whose next hopis the default internet gateway.If a dual-stack or IPv6-only subnet with an external IPv6 address range islocated in the same VPC network or in a peeredVPC network, seeEgress to destinations routable within a VPC networkinstead.
  • Other external destinations accessible using a static route in the sendinginstance's VPC network provided that the next hop for therouteis the default internet gateway.

For details about which Google Cloud resources use what types ofexternal IP addresses, seeExternal IP addresses.

Ingress bandwidth

Google Cloud handles inbound (ingress) bandwidth depending on how theincoming packet is routed to a receiving compute instance.

Ingress to destinations routable within a VPC network

A receiving instance can handle as many incoming packets as its machine type,operating system, and other network conditions permit. Google Cloud doesnot implement any purposeful bandwidth restriction on incoming packets deliveredto an instance if the incoming packet is delivered using routeswithin aVPC network:

  • Subnet routes in the receiving instance's VPC network
  • Peering subnet routes in a peered VPC network
  • Routes in another network whose next hops are Cloud VPN tunnels,Cloud Interconnect (VLAN) attachments, or Router applianceinstances located in the receiving instance's VPC network

Destinations for packets that are routed within a VPC networkinclude:

  • The primary internal IPv4 address of the receiving instance's networkinterface (NIC). Primary internal IPv4 addresses are regional internalIPv4 addresses that come from asubnet's primary IPv4 address range.
  • An internal IPv4 address from an alias IP range of the receiving instance'sNIC. Alias IP ranges can come from either a subnet's primary IPv4 addressrange or one of its secondary IPv4 address ranges.
  • An IPv6 address from the/96 IPv6 address range assigned to a dual-stackor IPv6-only receiving instance's NIC. Compute instance IPv6 ranges can comefrom these subnet IPv6 ranges:
  • An internal IPv4 address of a forwarding rule used by internal protocolforwarding to the receiving instance or internal passthrough Network Load Balancer where the receivinginstance is a backend of the load balancer. Internal forwarding rule IPv4addresses come from a subnet's primary IPv4 address range.
  • An internal IPv6 address from the/96 IPv6 range of a forwarding rule usedby internal protocol forwarding to the receiving instance or internal passthrough Network Load Balancerwhere the receiving instance is a backend of the load balancer. Internalforwarding rule IPv6 addresses come from a subnet's internal IPv6 addressrange.
  • An external IPv6 address from the/96 IPv6 range of a forwarding rule usedby external protocol forwarding to the receiving instance or external passthrough Network Load Balancer.The receiving instance is a backend of the load balancerwhen theincoming packet is routed within the VPC network using one ofthe routes listed previously in this section. External forwarding ruleIPv6 addresses come from a subnet's external IPv6 address range.
  • An IP address within the destination range of a custom static route that usesthe receiving instance as a next hop instance (next-hop-instance ornext-hop-address).
  • An IP address within the destination range of a custom static route using aninternal passthrough Network Load Balancer (next-hop-ilb) next hop, if the receiving instance is abackend for that load balancer.

Ingress to destinations outside of a VPC network

Google Cloud implements the following bandwidth limits for incomingpackets delivered to a receiving instance using routesoutside aVPC network. When load balancing is involved, the bandwidthlimits are applied individually to each receiving instance.

For machine series that don't support multiple physical NICs, the applicableinbound bandwidth restriction applies collectively to all virtual networkinterfaces (vNICs). The limit is the first of the following rates encountered:

  • 1,800,000 packets per second
  • 30 Gbps

For machine series that support multiple physical NICs, such as A3, theapplicable inbound bandwidth restriction applies individually to each physicalNIC. The limit is the first of the following rates encountered:

  • 1,800,000 packets per second per physical NIC
  • 30 Gbps per physical NIC
Note: Depending on the machine type of your compute instance and other factors,actual inbound traffic might be less than 30 Gbps or 1,800,000 packetsper second. Bandwidth from the internet is not covered by any SLA and issubject to network conditions.

Destinations for packets that are routed using routes outside of aVPC network include:

  • An external IPv4 address assigned in a one-to-one NAT access configuration onone of the receiving instance's network interfaces (NICs).
  • An external IPv6 address from the/96 IPv6 address range assigned to a vNICof a dual-stack or IPv6-only receiving instancewhen the incoming packet isrouted using a route outside of the receiving instance's VPCnetwork.
  • An external IPv4 address of a forwarding rule used by external protocolforwarding to the receiving instance or external passthrough Network Load Balancer where the receivinginstance is a backend of the load balancer.
  • An external IPv6 address from the/96 IPv6 range of a forwarding rule usedby external protocol forwarding to the receiving instance or external passthrough Network Load Balancer.The receiving instance must be a backend of the load balancerwhen theincoming packet is routed using a route outside of a VPCnetwork.
  • Established inbound responses processed by Cloud NAT.

Use jumbo frames to maximize network bandwidth

To receive and sendjumbo frames,configure the VPC network used by your compute instances; set themaximum transmission unit (MTU) to a larger value, up to 8896.

Higher MTU values increase the packet size and reduce the packet-headeroverhead, which increases payload data throughput.

You can use jumbo frames with the gVNIC driver version 1.3 or later onVM instances, or with the IDPF driver on bare metal instances.Not all Google Cloud public images include these drivers. Formore information about operating system support for jumbo frames, see theNetworking features tab on theOperating system detailspage.

Note: The image versions that indicate full support for jumbo frames includethe updated gVNIC driver, even if the guest OS shows the gve driver version as1.0.0.

If you are using an OS image that doesn't have full support for jumbo frames,you can manually install gVNIC driver version v1.3.0 or later. Googlerecommends installing the gVNIC driver version markedLatest to benefit fromadditional features and bug fixes. You can download the gVNIC drivers fromGitHub.

To manually update the gVNIC driver version in your guest OS, seeUse on non-supported operating systems.

Jumbo frames and GPU machines

For GPU machine types, use the recommended MTU settings for Jumbo frames. Formore information, seeRecommended MTU settings for Jumbo frames.

Receive and transmit queues

Each NIC or vNIC for a compute instance is assigned a number of receive andtransmit queues for processing packets from the network.

  • Receive Queue (RX): Queue to receive packets. When the NIC receives a packetfrom the network, the NIC selects the descriptor for an incoming packet fromthe queue, processes it and hands the packet to the guest OS over a packetqueue attached to a vCPU core using an interrupt. If the RX queueis full and there is no buffer available to place a packet, then thepacket is dropped. This can typically happen if an application isover-utilizing a vCPU core that is also attached to the selected packet queue.
  • Transmit Queue (TX): Queue to transmit packets. When the guest OS sendsa packet, a descriptor is allocated and placed in the TX queue. The NIC thenprocesses the descriptor and transmits the packet.

Default queue allocation

Unless you explicitlyassign queue counts for NICs, youcan model the algorithm Google Cloud uses to assign a fixed number of RXand TX queues per NIC in this way:

Bare metal instances
For bare metal instances, there is only one NIC, so the maximum queue count is 16.
VM instances that use the gVNIC network interface

For C4 instances, to improve performance, the following configurationsuse a fixed number of queues:

  • For Linux instances with 2 vCPUs, the queue count is 1.
  • For Linux instances with 4 vCPUs, the queue count is 2.

For the other machine series, the queue count depends on whether themachine series usesTitanium or not.

  • Forthird generation(excluding M3) and later instances that use Titanium:

    Divide the number of vCPUs by the number of vNICs (num_vcpus/num_vnics)and discard any remainder.

  • For first and second generation VMs that don't use Titanium:

    Divide the number of vCPUs by the number of vNICs, and then divide theresult by 2 (num_vcpus/num_vnics/2). Discard any remainder.

To finish the default queue count calculation:

  1. If the calculated number is less than 1, assign each vNIC one queueinstead.

  2. Determine if the calculated number is greater than the maximumnumber of queues per vNIC, which is16. If the calculated numberis greater than16, ignore the calculated number,and assign each vNIC 16 queues instead.

VM instances using the VirtIO network interface or a custom driver

Divide the number of vCPUs by the number of vNICs, and discard any remainder —[number of vCPUs/number of vNICs].

  1. If the calculated number is less than 1, assign each vNIC one queueinstead.

  2. Determine if the calculated number is greater than the maximumnumber of queues per vNIC, which is32. If the calculated numberis greater than32, ignore the calculated number, and assigneach vNIC 32 queues instead.

H4D instances with Cloud RDMA

For H4D instances that use Cloud RDMA, each physical host runs a single compute instance. Thus the instance gets all the available queue pairs. H4D instances have 16 queues for the gVNIC network interface and 16 queues for the IRDMA network interface.

Examples

The following examples show how to calculate the default number of queues fora VM instance:

  • If a VM instance uses VirtIO and has 16 vCPUs and 4 vNICs, the calculatednumber is[16/4] = 4. Google Cloud assigns each vNIC four queues.

  • If a VM instance uses gVNIC and has 128 vCPUs and two vNICs, the calculatednumber is[128/2/2] = 32. Google Cloud assigns each vNIC themaximum number of queues per vNIC possible. Google Cloudassigns16 queues per vNIC.

On Linux systems, you can useethtool to configure a vNIC with fewer queuesthan the number of queues Google Cloud assigns per vNIC.

Queue counts when using Dynamic Network Interface

If you use Dynamic Network Interfaces with your network interfaces, the queue countsdon't change. A Dynamic NIC doesn't have its own receive andtransmit queues; it uses the same queues as the parent vNIC.

Custom queue allocation for VM instances

Instead of thedefault queue allocation, you can assign acustom queue count (total of both RX and TX) to each vNIC when you create a newcompute instance by using the Compute Engine API.

The number of custom queues you specify must adhere to the following rules:

  • The minimum queue count you can assign per vNIC is one.

  • The maximum queue count you can assign to each vNIC of a VM instance is thelower of the vCPU count or the per vNIC maximum queue count, based on thedriver type:

    • UsingvirtIOor a custom driver, the maximum queue count is32.
    • UsinggVNIC, the maximum queuecount is16, except for the following, where the maximum queuecount is 32:
      • A2 or G2 instances
      • TPU instances
      • C2, C2D, N2, or N2D instances with Tier_1 networking enabled
    • For the followingConfidential VMconfigurations, the maximum queue count is8:

      • AMD SEV on C2D and N2D machine types
      • AMD SEV-SNP on N2D machine types
  • If you assign custom queue counts toall NICs of the compute instance,the sum of your queue count assignments must be less than or equal tothe number of vCPUs assigned to the instance.

You can oversubscribe the custom queue count for your vNICs. In other words, youcan have a sum of the queue counts assigned to all NICs for your VMinstance that is greater than the number of vCPUs for your instance. Tooversubscribe the custom queue count, the VM instance must satisfy thefollowing conditions:

  • Use gVNIC as the vNIC type for all NICs configured for the instance.
  • Uses a machine type that supports Tier_1 networking.
  • Has Tier_1 networking enabled.
  • Specified a custom queue count for all NICs configured for the instance.

With queue oversubscription, the maximum queue count for the VM instanceis 16 times the number of NICs. So, if you have 6 NICs configured for aninstance with 30 vCPUs, you can configure a maximum of (16 * 6), or 96 customqueues for your instance.

Examples

  • If a VM instance has 8 vCPUs and 3 vNICs, the maximum queue count forthe instance is the number of vCPUS, or 8. You can assign 1 queue tonic0,4 queues tonic1, and 3 queues tonic2. In this example, youcan't subsequently assign 4 queues tonic2 while keeping the other twovNIC queue assignments because the sum of assigned queues cannot exceed thenumber of vCPUs.

  • If you have a N2 VM with 96 vCPUs and 2 vNICs, you can assign both vNICs upto 32 queues each when using the virtIO driver, or up to 16 queues eachwhen using the gVNIC driver. If you enable Tier_1 networking for theN2 VM, then you can assign up to 32 queues to each vNIC. In this example,the sum of assigned queues is always less than or equal to the number ofvCPUs.

It's also possible to assign a custom queue count foronly some NICs, lettingGoogle Cloud assign queues to the remaining NICs. The number of queues youcan assign per vNIC is still subject to rules mentioned previously. You canmodel the feasibility of your configuration, and, if your configuration ispossible, the number of queues that Google Cloud assigns to the remainingvNICs with this process:

  1. Calculate the sum of queues for the vNICs using custom queue assignment. Foran example VM with 20 vCPUs and 6 vNICs, suppose you assignnic0 5queues,nic1 6 queues,nic2 4 queues, and let Google Cloudassign queues fornic3,nic4, andnic5. In this example, the sum ofcustom-assigned queues is5+6+4 = 15.

  2. Subtract the sum of custom-assigned queues from the number of vCPUs. If thedifference is less than the number of remaining vNICs for whichGoogle Cloud must assign queues, Google Cloud returns an errorbecause each vNIC must have at least one queue.

    Continuing the example with a VM that has 20 vCPUs and a sum of15custom-assigned queues, Google Cloud has20-15 = 5 queues left toassign to the remaining vNICs (nic3,nic4,nic5).

  3. Divide the difference from the previous step by the number of remainingvNICs and discard any remainder —⌊(number of vCPUs - sum of assignedqueues)/(number of remaining vNICs)⌋. This calculation always results in awhole number (not a fraction) that is at least equal to one because of theconstraint explained in the previous step. Google Cloud assigns eachremaining vNIC a queue count matching the calculated number as long as thecalculated number is not greater than the maximum number of queues per vNIC.The maximum number of queues per vNIC depends on the driver type:

  • Using virtIO or a custom driver, if the calculated number of queues foreach remaining vNIC is greater than32, Google Cloud assignseach remaining vNIC32 queues.
  • Using gVNIC, if the calculated number of queues for each remaining vNICis greater than the limit of16 or32 (depending on theVM configuration), Google Cloud assigns eachremaining vNIC16 queues.

Configure custom queue counts

To create a compute instance that uses a custom queue count for one or moreNICs or vNICs, complete the following steps.

In the following code examples, the VM is created with the network interfacetype set toGVNIC and per VM Tier_1 networking performance enabled. You can use these codeexamples to specify the maximum queue counts and queue oversubscription that isavailable for the supported machine types.

gcloud

  1. If you don't already have aVPC networkwith a subnet for each vNIC interface you plan to configure, create them.
  2. Use thegcloud compute instances create commandto create the compute instance. Repeat the--network-interface flag foreach vNIC that you want to configure for the instance, and includethequeue-count option.
    gcloud compute instances createINSTANCE_NAME \        --zone=ZONE \        --machine-type=MACHINE_TYPE \        --network-performance-configs=total-egress-bandwidth-tier=TIER_1  \        --network-interface=network=NETWORK_NAME_1,subnet=SUBNET_1,nic-type=GVNIC,queue-count=QUEUE_SIZE_1 \        --network-interface=network=NETWORK_NAME_2,subnet=SUBNET_2,nic-type=GVNIC,queue-count=QUEUE_SIZE_2

Replace the following:

  • INSTANCE_NAME: aname for the newcompute instance
  • ZONE: the zone to create the instance in
  • MACHINE_TYPE: themachine type of theinstance. To oversubscribe the queue count, the machine type youspecify must support gVNIC and Tier_1 networking.
  • NETWORK_NAME: the name of the network createdpreviously
  • SUBNET_*: the name of one of the subnets createdpreviously
  • QUEUE_SIZE: the number of queues for the vNIC,subject to the rules discussed inCustom queue allocation.

Terraform

  1. If you don't already have aVPC networkwith a subnet for each vNIC interface you plan to configure, create them.
  2. Create a compute instance with specific queue counts for vNICs using thegoogle_compute_instance resource.Repeat the--network-interface parameter for each vNICyou want to configure for the compute instance, and includethequeue-count parameter.

    # Queue oversubscription instanceresource "google_compute_instance" "VM_NAME" {project      = "PROJECT_ID"boot_disk {  auto_delete = true  device_name = "DEVICE_NAME"  initialize_params {     image="IMAGE_NAME"     size =DISK_SIZE     type = "DISK_TYPE"  }}machine_type = "MACHINE_TYPE"name         = "VM_NAME"zone = "ZONE"network_performance_config {    total_egress_bandwidth_tier = "TIER_1"}network_interface {    nic_type = "GVNIC"    queue_count =QUEUE_COUNT_1    subnetwork_project = "PROJECT_ID"    subnetwork = "SUBNET_1" }network_interface {    nic_type = "GVNIC"    queue_count =QUEUE_COUNT_2    subnetwork_project = "PROJECT_ID"    subnetwork = "SUBNET_2"}network_interface {    nic_type = "GVNIC"    queue_count =QUEUE_COUNT_3    subnetwork_project = "PROJECT_ID"    subnetwork = "SUBNET_3""}network_interface {    nic_type = "GVNIC"    queue_count =QUEUE_COUNT_4    subnetwork_project = "PROJECT_ID"    subnetwork = "SUBNET_4""}}

Replace the following:

  • VM_NAME: aname for the newcompute instance
  • PROJECT_ID: ID of the project to createthe instance in. Unless you are using a Shared VPC network, theproject you specify must be the same one in which all the subnetsand networks were created in.
  • DEVICE_NAME: The name to associate with the bootdisk in the guest OS
  • IMAGE_NAME: the name of an image,for example,"projects/debian-cloud/global/images/debian-11-bullseye-v20231010".
  • DISK_SIZE: the size of the boot disk, in GiB
  • DISK_TYPE: the type of disk to use for the bootdisk, for example,pd-standard
  • MACHINE_TYPE: themachine type of theinstance. To oversubscribe the queue count, the machine type youspecify must support gVNIC and Tier_1 networking.
  • ZONE: the zone to create the instance in
  • QUEUE_COUNT: the number of queues for the vNIC,subject to the rules discussed inCustom queue allocation.
  • SUBNET_*: the name of the subnet that the networkinterface connects to

REST

  1. If you don't already have aVPC networkwith a subnet for each vNIC interface you plan to configure, create them.
  2. Create a compute instance with specific queue counts for NICs using theinstances.insert method.Repeat thenetworkInterfaces property to configure multiple networkinterfaces.

    POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances{"name": "VM_NAME","machineType": "machineTypes/MACHINE_TYPE","networkPerformanceConfig": {    "totalEgressBandwidthTier": TIER_1},"networkInterfaces": [    {      "nicType": gVNIC,      "subnetwork":"regions/region/subnetworks/SUBNET_1",      "queueCount": "QUEUE_COUNT_1"    } ],"networkInterfaces": [    {      "nicType": gVNIC,      "subnetwork":"regions/region/subnetworks/SUBNET_2",      "queueCount": "QUEUE_COUNT_2"    } ],}

    Replace the following:

    • PROJECT_ID: ID of the project to createthe compute instance in
    • ZONE: zone to create the compute instance in
    • VM_NAME:name of thenew compute instance
    • MACHINE_TYPE: machine type,predefined orcustom,for the new compute instance. To oversubscribe the queue count, themachine type must support gVNIC and Tier_1 networking.
    • SUBNET_*: the name of the subnet that thenetwork interface connects to
    • QUEUE_COUNT: Number of queues for the vNIC,subject to the rules discussed inCustom queue allocation.

Queue allocations and changing the machine type

Compute instances are created with adefault queue allocation,or you can assign acustom queue count toeach virtual network interface card (vNIC) when you create a new computeinstance by using the Compute Engine API. The default or custom vNIC queue assignmentsare only set when creating a compute instance. If your instance has vNICs thatuse default queue counts, you canchange its machine type.If the machine type that you are changing to has a different number of vCPUs,the default queue counts for your instance are recalculated based on the newmachine type.

If your VM has vNICs which use custom, non-default queue counts,then you can change the machine type by using the Google Cloud CLI orCompute Engine API toupdate the instance properties.The conversion succeeds if the resulting VM supports the samequeue count per vNIC as the original instance. For VMs that usethe VirtIO-Net interface and have a custom queue count that is higher than 16per vNIC, you can't change the machine type to a third generation or latermachine type, because they use only gVNIC. Instead, you can migrate your VMto a third generation or later machine type by following theinstructions inMove your workload to a new compute instance.

What's next

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-12-15 UTC.