Network bandwidth Stay organized with collections Save and categorize content based on your preferences.
Google Cloud accounts for bandwidth per compute instance, notper virtual network interface (vNIC) or IP address. An instance'smachine type defines its maximum possibleegress rate; however, you can only achieve that maximum possible egress rate inspecific situations.
This page outlines the network bandwidth limits, which are useful when planningyour deployments. It categorizes bandwidth using two dimensions:
- Egress or ingress: As used on this page, egress and ingress are alwaysfrom the perspective of a Google Cloud instance:
- Packets sentfrom a Google Cloud instance compose itsegress(outbound) traffic.
- Packets sentto a Google Cloud instance compose itsingress(inbound) traffic.
- How the packet is routed: A packet can be routed from a sending instanceor to a receiving instance using routes whose next hops arewithin aVPC network or routes outside of a VPC network.
All of the information on this page is applicable to Compute Enginecompute instances, as well as products that depend on Compute Engineinstances. For example, a Google Kubernetes Engine node is a Compute Engineinstance.
Note: For network bandwidth information for Accelerator-optimized machineseries, seeNetworking and GPU machines.Configurations that impact network bandwidth
Neitheradditional virtual network interfaces (vNICs)nor additional IP addresses per vNIC increase ingress or egress bandwidth for acompute instance. For example, a C3 VM with 22 vCPUs is limited to23 Gbps total egress bandwidth. If you configure the C3 VM with twovNICs, the VM is still limited to 23 Gbps total egress bandwidth,not 23 Gbps bandwidth per vNIC.
The following sections describe how other compute instance configurations canimpact network bandwidth.
Use per VM Tier_1 networking performance
To get the highest possible ingress and egress bandwidth,configure Tier_1 networkingfor your compute instance.
Dynamic Network Interfaces
Dynamic Network Interfaces use thebandwidth of their parent vNIC. There is no traffic isolation within a parentvNIC. Network traffic from a Dynamic NIC can starve the otherDynamic NICs associated with the same parent vNIC. To avoid thisconflict, you can use Linux traffic control (TC) to craft application-specifictraffic shaping policies. These policies help to implement either fairness orcertain types of priority. For prioritization, you map traffic (for example forDynamic NICs) to a traffic class, and then map that traffic classto a quality of service. For an example of this approach, seeTraffic shaping with Red Hat.
Dynamic NICs aren't supported for Compute Engineinstances that run a Windows OS.
HPC computing with Cloud RDMA
Bandwidth is required for all three stages of a high performance computing (HPC)job: load, compute, and store. During the load and store phases, applicationscommonly use TCP for communication, and remote direct memory address (RDMA)for the compute stages. H4D instances using GVNIC for the TCP traffic provideup to 200 Gbps network bandwidth. If you also configure an H4D instanceto use Cloud RDMA, then the network bandwidth is shared among theconfigured network interfaces.
Network bandwidth allocation between Cloud RDMA traffic and TCP trafficis done dynamically. Instead of limiting both Cloud RDMA and TCPtraffic to 100 Gbps bandwidth each, the GVNIC network interface can use allof the available bandwidth when the Cloud RDMA network interface is notbeing used. Similarly, the Cloud RDMA network interface can use all ofthe available bandwidth when the GVNIC network interface is not being used. Whenboth network interface types are in use, Cloud RDMA traffic haspriority over TCP traffic because it is more performance sensitive.
Bandwidth summary
The following table illustrates the maximum possible bandwidth based on whethera packet is sent from (egress) or received by (ingress) a compute instance andthe packet routing method.
Egress bandwidth limits
| Routing within a VPC network |
|
|---|---|
| Routing outside a VPC network |
|
Ingress bandwidth limits
| Routing within a VPC network |
|
|---|---|
| Routing outside a VPC network |
|
Egress bandwidth
Google Cloud limits outbound (egress) bandwidth using per-instance maximumegress rates. These rates are based the machine type of the compute instancethat is sending the packet and whether the packet's destination is accessibleusing routes within a VPC network or routes outside of aVPC network. Outbound bandwidth includes packets emitted by allof the instance's NICs and data transferred to all Hyperdiskand Persistent Disk volumes connected to the instance.
Per-instance maximum egress bandwidth
Per-instance maximum egress bandwidth is generally 2 Gbps per vCPU, butthere are some differences and exceptions, depending on the machine series.The following table shows the range of maximum limits for egress bandwidth fortraffic routed within a VPC network.
The following table summarizes the maximum egress bandwidth for each machineseries. You can find the per-instance maximum egress bandwidth for every machinetype listed on its specific machine family page (using the links for eachmachine series in the table).
| Maximum per-instance egress limit | ||
|---|---|---|
| Machine series | Standard | Tier_1 networking |
| C4 andC4D | 100 Gbps | 200 Gbps |
| C4A | 50 Gbps | 100 Gbps |
| C3 andC3D | 100 Gbps | 200 Gbps |
| C2 andC2D | 32 Gbps | 100 Gbps |
| E2 | 16 Gbps | N/A |
| E2 shared-core | 2 Gbps | N/A |
| H4D andH3 | 200 Gbps | N/A |
| M4 | 100 Gbps | N/A |
| M3 | 32 Gbps | 100 Gbps |
| M2 | 32 Gbps on Intel Cascade Lake or later CPU 16 Gbps on other CPU platforms | N/A |
| M1 | 32 Gbps | N/A |
| N4,N4A (Preview), andN4D | 50 Gbps | N/A |
| N2 andN2D | 32 Gbps | 100 Gbps |
| N1 (excluding VMs with 1 vCPU) | 32 Gbps on Intel Skylake and later CPU 16 Gbps on earlier CPU platforms | N/A |
| N1 machine types with 1 vCPU | 2 Gbps | N/A |
| N1 shared-core (f1-micro and g1-small) | 1 Gbps | N/A |
| T2A andT2D | 32 Gbps | N/A |
| X4 | 100 Gbps | N/A |
| Z3 | 100 Gbps | 200 Gbps |
For network bandwidth information for Accelerator-optimized machine series, seeNetworking and GPU machines.
Per-instance maximum egress bandwidth is not a guarantee. The actual egressbandwidth can be lowered according to factors such as the followingnon-exhaustive list:
- Using VirtIO instead ofgVNIC withcompute instances that support both
- Packet size
- Protocol overhead
- The number of flows
- Ethernet driver settings of the compute instance's guest OS, such aschecksum offload and TCP segmentation offload (TSO)
- Network congestion
- In a situation where Persistent Disk I/Os compete with other network egresstraffic, 60% of the maximum network bandwidth is given to Persistent Diskwrites, leaving 40% for other network egress traffic. SeeFactors that affect disk performancefor more details.
To get the largest possible per-instance maximum egress bandwidth:
- Enableper VM Tier_1 networking performancewith larger machine types.
- Use the largest VPC networkmaximum transmission unit (MTU) supported by your networktopology. Larger MTUs can reduce packet-header overhead and increase payloaddata throughput.
- Use the latest gVNIC driver version.
- Use third generation or later machine series that useTitaniumto offload network processing from the host CPU.
Egress to destinations routable within a VPC network
From the perspective of a sending instance and for destination IP addressesaccessible by means of routeswithin a VPC network,Google Cloud limits outbound traffic using these rules:
- Per-VM maximum egress bandwidth: The per-instance maximum egress bandwidthdescribed in thePer-instance maximum egress bandwidthsection.
- Per-project inter-regional egress bandwidth: If a sending instance and aninternal destination or its next hop are in different regions,Google Cloud enforces a quota based on the region and project of thesending instance and the region of the internal destination or next hop. Formore information about this quota, seeInter-region network egress bandwidth(Mbps) from Compute instances in theVPC quotas and limits documentation.
- Cloud VPN and Cloud Interconnect limits: When sendingtraffic from an instance to an internal IP address destination routable by anext hop Cloud VPN tunnel or Cloud Interconnect VLANattachment, egress bandwidth is limited by:
- Maximum packet rate and bandwidth per Cloud VPN tunnel
- Maximum packet rate and bandwidth per VLAN attachment
- To fully use the bandwidth of multiple next hop Cloud VPNtunnels or Cloud Interconnect VLAN attachments using ECMProuting, you must use multiple TCP connections (unique 5-tuples).
Destinations routable within a VPC network include all of thefollowing destinations, each of which is accessible from the perspective of thesending instance by a route whose next hop isnot the default internetgateway:
- Regional internal IPv4 addresses insubnet primary IPv4 and subnet secondary IPv4 address ranges,including private IPv4 address ranges and privately used public IPv4 addressranges, used by these destination resources:
- The primary internal IPv4 address of a receiving instance's networkinterface (vNIC). (When a sending instance connects to another instance'svNICexternal IPv4 address, packets are routed using a next hop defaultinternet gateway, soEgress to destinations outside of a VPC networkapplies instead.)
- An internal IPv4 address in an alias IP range of a receiving instance'svNIC.
- An internal IPv4 address of an internal forwarding rule for either protocolforwarding or for an internal passthrough Network Load Balancer.
- Global internal IPv4 addresses for these destination resources:
- Internal IPv6 subnet address ranges used bythese destination resources:
- An IPv6 address from the
/96IPv6 address range assigned to a dual-stackor IPv6-only receiving instance's vNIC. - An IPv6 address from the
/96IPv6 address range of an internal forwardingrule for either protocol forwarding or for an internal passthrough Network Load Balancer.
- An IPv6 address from the
- External IPv6 subnet address ranges used bythese destination resourceswhen packets are routed using subnet routesor peering subnet routes within the VPC network or by customroutes within the VPC network that do not use the defaultinternet gateway next hop:
- An IPv6 address from the
/96IPv6 address range assigned to a dual-stackor IPv6-only receiving instance's vNIC. - An IPv6 address from the
/96IPv6 address range of an external forwardingrule for either protocol forwarding or for an external passthrough Network Load Balancer.
- An IPv6 address from the
- Other destinations accessible using the following VPC networkroutes:
- Dynamic routes
- Static routesexcept those that use adefault internet gateway next hop
- Peering custom routes
The following list ranks traffic from sending instances to internaldestinations, from highest possible bandwidth to lowest:
- Between compute instances in the same zone
- Between compute instances in different zones of the same region
- Between compute instances in different regions
- From a compute instance to Google Cloud APIs and services usingPrivate Google Access oraccessing Google APIs from an instance's external IP address.This includesPrivate Service Connect endpoints for Google APIs.
Egress to destinations outside of a VPC network
From the perspective of a sending instance and for destination IP addressesoutside of a VPC network, Google Cloud limits outboundtraffic to whichever of the following rates is reached first:
Per-instance egress bandwidth: The maximum bandwidth for all connectionsfrom a compute instance to destinations outside of a VPCnetwork is the smaller of thePer-instance maximum egress bandwidth and one of theserates:
- 25 Gbps, if Tier_1 networking is enabled
- 7 Gbps, if Tier_1 networking isn't enabled
- 1 Gbps for H4D and H3 instances
- 7 Gbps per physical NIC for machine series that support multiplephysical NICs, such as A3.
For example, even though an
c3-standard-44instance has a per-VMmaximum egress bandwidth of 32 Gbps, the per-VM egressbandwidth from ac3-standard-44VM to external destinations iseither 25 Gbps or 7 Gbps, depending on whetherTier_1 networking is enabled.Per-flow maximum egress rate: The maximum bandwidth foreach unique5-tuple connection, from a compute instance to a destination outside of aVPC network is 3 Gbps, except on H4D and H3, where itis 1 Gbps.
Per-project internet egress bandwidth: The maximum bandwidth for allconnections from compute instances ineach region of a project todestinations outside of a VPC network is defined by theproject'sInternet egress bandwidthquotas.
Destinations outside of a VPC network include all of thefollowing destinations, each of which is accessible by a route in the sendinginstance's VPC network whose next hopis the defaultinternet gateway:
- Global external IPv4 and IPv6 addresses for external proxy Network Load Balancers andexternal Application Load Balancers
- Regional external IPv4 addresses for Google Cloud resources, includingVM vNIC external IPv4 addresses, external IPv4 addresses for external protocolforwarding, external passthrough Network Load Balancers, and response packets to Cloud NATgateways.
- Regional external IPv6 addresses in dual-stack or IPv6-only subnets withexternal IPv6 address ranges used byexternal IPv6 addresses of dual-stack or IPv6-only instances, externalprotocol forwarding, and external passthrough Network Load Balancers. The subnet must be located in aseparate, non-peered VPC network. The destination IPv6 addressrange must be accessible using a route in the sending instance'sVPC network whose next hopis the default internet gateway.If a dual-stack or IPv6-only subnet with an external IPv6 address range islocated in the same VPC network or in a peeredVPC network, seeEgress to destinations routable within a VPC networkinstead.
- Other external destinations accessible using a static route in the sendinginstance's VPC network provided that the next hop for therouteis the default internet gateway.
For details about which Google Cloud resources use what types ofexternal IP addresses, seeExternal IP addresses.
Ingress bandwidth
Google Cloud handles inbound (ingress) bandwidth depending on how theincoming packet is routed to a receiving compute instance.
Ingress to destinations routable within a VPC network
A receiving instance can handle as many incoming packets as its machine type,operating system, and other network conditions permit. Google Cloud doesnot implement any purposeful bandwidth restriction on incoming packets deliveredto an instance if the incoming packet is delivered using routeswithin aVPC network:
- Subnet routes in the receiving instance's VPC network
- Peering subnet routes in a peered VPC network
- Routes in another network whose next hops are Cloud VPN tunnels,Cloud Interconnect (VLAN) attachments, or Router applianceinstances located in the receiving instance's VPC network
Destinations for packets that are routed within a VPC networkinclude:
- The primary internal IPv4 address of the receiving instance's networkinterface (NIC). Primary internal IPv4 addresses are regional internalIPv4 addresses that come from asubnet's primary IPv4 address range.
- An internal IPv4 address from an alias IP range of the receiving instance'sNIC. Alias IP ranges can come from either a subnet's primary IPv4 addressrange or one of its secondary IPv4 address ranges.
- An IPv6 address from the
/96IPv6 address range assigned to a dual-stackor IPv6-only receiving instance's NIC. Compute instance IPv6 ranges can comefrom these subnet IPv6 ranges:- Aninternal IPv6 address range.
- Anexternal IPv6 address rangewhenthe incoming packet is routed internally to the receiving instance usingone of the VPC network routes listed previously in thissection.
- An internal IPv4 address of a forwarding rule used by internal protocolforwarding to the receiving instance or internal passthrough Network Load Balancer where the receivinginstance is a backend of the load balancer. Internal forwarding rule IPv4addresses come from a subnet's primary IPv4 address range.
- An internal IPv6 address from the
/96IPv6 range of a forwarding rule usedby internal protocol forwarding to the receiving instance or internal passthrough Network Load Balancerwhere the receiving instance is a backend of the load balancer. Internalforwarding rule IPv6 addresses come from a subnet's internal IPv6 addressrange. - An external IPv6 address from the
/96IPv6 range of a forwarding rule usedby external protocol forwarding to the receiving instance or external passthrough Network Load Balancer.The receiving instance is a backend of the load balancerwhen theincoming packet is routed within the VPC network using one ofthe routes listed previously in this section. External forwarding ruleIPv6 addresses come from a subnet's external IPv6 address range. - An IP address within the destination range of a custom static route that usesthe receiving instance as a next hop instance (
next-hop-instanceornext-hop-address). - An IP address within the destination range of a custom static route using aninternal passthrough Network Load Balancer (
next-hop-ilb) next hop, if the receiving instance is abackend for that load balancer.
Ingress to destinations outside of a VPC network
Google Cloud implements the following bandwidth limits for incomingpackets delivered to a receiving instance using routesoutside aVPC network. When load balancing is involved, the bandwidthlimits are applied individually to each receiving instance.
For machine series that don't support multiple physical NICs, the applicableinbound bandwidth restriction applies collectively to all virtual networkinterfaces (vNICs). The limit is the first of the following rates encountered:
- 1,800,000 packets per second
- 30 Gbps
For machine series that support multiple physical NICs, such as A3, theapplicable inbound bandwidth restriction applies individually to each physicalNIC. The limit is the first of the following rates encountered:
- 1,800,000 packets per second per physical NIC
- 30 Gbps per physical NIC
Destinations for packets that are routed using routes outside of aVPC network include:
- An external IPv4 address assigned in a one-to-one NAT access configuration onone of the receiving instance's network interfaces (NICs).
- An external IPv6 address from the
/96IPv6 address range assigned to a vNICof a dual-stack or IPv6-only receiving instancewhen the incoming packet isrouted using a route outside of the receiving instance's VPCnetwork. - An external IPv4 address of a forwarding rule used by external protocolforwarding to the receiving instance or external passthrough Network Load Balancer where the receivinginstance is a backend of the load balancer.
- An external IPv6 address from the
/96IPv6 range of a forwarding rule usedby external protocol forwarding to the receiving instance or external passthrough Network Load Balancer.The receiving instance must be a backend of the load balancerwhen theincoming packet is routed using a route outside of a VPCnetwork. - Established inbound responses processed by Cloud NAT.
Use jumbo frames to maximize network bandwidth
To receive and sendjumbo frames,configure the VPC network used by your compute instances; set themaximum transmission unit (MTU) to a larger value, up to 8896.
Higher MTU values increase the packet size and reduce the packet-headeroverhead, which increases payload data throughput.
You can use jumbo frames with the gVNIC driver version 1.3 or later onVM instances, or with the IDPF driver on bare metal instances.Not all Google Cloud public images include these drivers. Formore information about operating system support for jumbo frames, see theNetworking features tab on theOperating system detailspage.
Note: The image versions that indicate full support for jumbo frames includethe updated gVNIC driver, even if the guest OS shows the gve driver version as1.0.0.If you are using an OS image that doesn't have full support for jumbo frames,you can manually install gVNIC driver version v1.3.0 or later. Googlerecommends installing the gVNIC driver version markedLatest to benefit fromadditional features and bug fixes. You can download the gVNIC drivers fromGitHub.
To manually update the gVNIC driver version in your guest OS, seeUse on non-supported operating systems.
Jumbo frames and GPU machines
For GPU machine types, use the recommended MTU settings for Jumbo frames. Formore information, seeRecommended MTU settings for Jumbo frames.
Receive and transmit queues
Each NIC or vNIC for a compute instance is assigned a number of receive andtransmit queues for processing packets from the network.
- Receive Queue (RX): Queue to receive packets. When the NIC receives a packetfrom the network, the NIC selects the descriptor for an incoming packet fromthe queue, processes it and hands the packet to the guest OS over a packetqueue attached to a vCPU core using an interrupt. If the RX queueis full and there is no buffer available to place a packet, then thepacket is dropped. This can typically happen if an application isover-utilizing a vCPU core that is also attached to the selected packet queue.
- Transmit Queue (TX): Queue to transmit packets. When the guest OS sendsa packet, a descriptor is allocated and placed in the TX queue. The NIC thenprocesses the descriptor and transmits the packet.
Default queue allocation
Unless you explicitlyassign queue counts for NICs, youcan model the algorithm Google Cloud uses to assign a fixed number of RXand TX queues per NIC in this way:
- Bare metal instances
- For bare metal instances, there is only one NIC, so the maximum queue count is 16.
- VM instances that use the gVNIC network interface
For C4 instances, to improve performance, the following configurationsuse a fixed number of queues:
- For Linux instances with 2 vCPUs, the queue count is 1.
- For Linux instances with 4 vCPUs, the queue count is 2.
For the other machine series, the queue count depends on whether themachine series usesTitanium or not.
Forthird generation(excluding M3) and later instances that use Titanium:
Divide the number of vCPUs by the number of vNICs (
num_vcpus/num_vnics)and discard any remainder.For first and second generation VMs that don't use Titanium:
Divide the number of vCPUs by the number of vNICs, and then divide theresult by 2 (
num_vcpus/num_vnics/2). Discard any remainder.
To finish the default queue count calculation:
If the calculated number is less than 1, assign each vNIC one queueinstead.
Determine if the calculated number is greater than the maximumnumber of queues per vNIC, which is
16. If the calculated numberis greater than16, ignore the calculated number,and assign each vNIC 16 queues instead.
- VM instances using the VirtIO network interface or a custom driver
Divide the number of vCPUs by the number of vNICs, and discard any remainder —
[number of vCPUs/number of vNICs].If the calculated number is less than 1, assign each vNIC one queueinstead.
Determine if the calculated number is greater than the maximumnumber of queues per vNIC, which is
32. If the calculated numberis greater than32, ignore the calculated number, and assigneach vNIC 32 queues instead.
- H4D instances with Cloud RDMA
For H4D instances that use Cloud RDMA, each physical host runs a single compute instance. Thus the instance gets all the available queue pairs. H4D instances have 16 queues for the gVNIC network interface and 16 queues for the IRDMA network interface.
Examples
The following examples show how to calculate the default number of queues fora VM instance:
If a VM instance uses VirtIO and has 16 vCPUs and 4 vNICs, the calculatednumber is
[16/4] = 4. Google Cloud assigns each vNIC four queues.If a VM instance uses gVNIC and has 128 vCPUs and two vNICs, the calculatednumber is
[128/2/2] = 32. Google Cloud assigns each vNIC themaximum number of queues per vNIC possible. Google Cloudassigns16queues per vNIC.
On Linux systems, you can useethtool to configure a vNIC with fewer queuesthan the number of queues Google Cloud assigns per vNIC.
Queue counts when using Dynamic Network Interface
If you use Dynamic Network Interfaces with your network interfaces, the queue countsdon't change. A Dynamic NIC doesn't have its own receive andtransmit queues; it uses the same queues as the parent vNIC.
Custom queue allocation for VM instances
Instead of thedefault queue allocation, you can assign acustom queue count (total of both RX and TX) to each vNIC when you create a newcompute instance by using the Compute Engine API.
The number of custom queues you specify must adhere to the following rules:
The minimum queue count you can assign per vNIC is one.
The maximum queue count you can assign to each vNIC of a VM instance is thelower of the vCPU count or the per vNIC maximum queue count, based on thedriver type:
- UsingvirtIOor a custom driver, the maximum queue count is
32. - UsinggVNIC, the maximum queuecount is
16, except for the following, where the maximum queuecount is 32:- A2 or G2 instances
- TPU instances
- C2, C2D, N2, or N2D instances with Tier_1 networking enabled
For the followingConfidential VMconfigurations, the maximum queue count is
8:- AMD SEV on C2D and N2D machine types
- AMD SEV-SNP on N2D machine types
- UsingvirtIOor a custom driver, the maximum queue count is
If you assign custom queue counts toall NICs of the compute instance,the sum of your queue count assignments must be less than or equal tothe number of vCPUs assigned to the instance.
You can oversubscribe the custom queue count for your vNICs. In other words, youcan have a sum of the queue counts assigned to all NICs for your VMinstance that is greater than the number of vCPUs for your instance. Tooversubscribe the custom queue count, the VM instance must satisfy thefollowing conditions:
- Use gVNIC as the vNIC type for all NICs configured for the instance.
- Uses a machine type that supports Tier_1 networking.
- Has Tier_1 networking enabled.
- Specified a custom queue count for all NICs configured for the instance.
With queue oversubscription, the maximum queue count for the VM instanceis 16 times the number of NICs. So, if you have 6 NICs configured for aninstance with 30 vCPUs, you can configure a maximum of (16 * 6), or 96 customqueues for your instance.
Examples
If a VM instance has 8 vCPUs and 3 vNICs, the maximum queue count forthe instance is the number of vCPUS, or 8. You can assign 1 queue to
nic0,4 queues tonic1, and 3 queues tonic2. In this example, youcan't subsequently assign 4 queues tonic2while keeping the other twovNIC queue assignments because the sum of assigned queues cannot exceed thenumber of vCPUs.If you have a N2 VM with 96 vCPUs and 2 vNICs, you can assign both vNICs upto 32 queues each when using the virtIO driver, or up to 16 queues eachwhen using the gVNIC driver. If you enable Tier_1 networking for theN2 VM, then you can assign up to 32 queues to each vNIC. In this example,the sum of assigned queues is always less than or equal to the number ofvCPUs.
It's also possible to assign a custom queue count foronly some NICs, lettingGoogle Cloud assign queues to the remaining NICs. The number of queues youcan assign per vNIC is still subject to rules mentioned previously. You canmodel the feasibility of your configuration, and, if your configuration ispossible, the number of queues that Google Cloud assigns to the remainingvNICs with this process:
Calculate the sum of queues for the vNICs using custom queue assignment. Foran example VM with 20 vCPUs and 6 vNICs, suppose you assign
nic05queues,nic16 queues,nic24 queues, and let Google Cloudassign queues fornic3,nic4, andnic5. In this example, the sum ofcustom-assigned queues is5+6+4 = 15.Subtract the sum of custom-assigned queues from the number of vCPUs. If thedifference is less than the number of remaining vNICs for whichGoogle Cloud must assign queues, Google Cloud returns an errorbecause each vNIC must have at least one queue.
Continuing the example with a VM that has 20 vCPUs and a sum of
15custom-assigned queues, Google Cloud has20-15 = 5queues left toassign to the remaining vNICs (nic3,nic4,nic5).Divide the difference from the previous step by the number of remainingvNICs and discard any remainder —
⌊(number of vCPUs - sum of assignedqueues)/(number of remaining vNICs)⌋. This calculation always results in awhole number (not a fraction) that is at least equal to one because of theconstraint explained in the previous step. Google Cloud assigns eachremaining vNIC a queue count matching the calculated number as long as thecalculated number is not greater than the maximum number of queues per vNIC.The maximum number of queues per vNIC depends on the driver type:
- Using virtIO or a custom driver, if the calculated number of queues foreach remaining vNIC is greater than
32, Google Cloud assignseach remaining vNIC32queues. - Using gVNIC, if the calculated number of queues for each remaining vNICis greater than the limit of
16or32(depending on theVM configuration), Google Cloud assigns eachremaining vNIC16queues.
Configure custom queue counts
To create a compute instance that uses a custom queue count for one or moreNICs or vNICs, complete the following steps.
In the following code examples, the VM is created with the network interfacetype set toGVNIC and per VM Tier_1 networking performance enabled. You can use these codeexamples to specify the maximum queue counts and queue oversubscription that isavailable for the supported machine types.
gcloud
- If you don't already have aVPC networkwith a subnet for each vNIC interface you plan to configure, create them.
- Use the
gcloud compute instances createcommandto create the compute instance. Repeat the--network-interfaceflag foreach vNIC that you want to configure for the instance, and includethequeue-countoption.
gcloud compute instances createINSTANCE_NAME \ --zone=ZONE \ --machine-type=MACHINE_TYPE \ --network-performance-configs=total-egress-bandwidth-tier=TIER_1 \ --network-interface=network=NETWORK_NAME_1,subnet=SUBNET_1,nic-type=GVNIC,queue-count=QUEUE_SIZE_1 \ --network-interface=network=NETWORK_NAME_2,subnet=SUBNET_2,nic-type=GVNIC,queue-count=QUEUE_SIZE_2
Replace the following:
INSTANCE_NAME: aname for the newcompute instanceZONE: the zone to create the instance inMACHINE_TYPE: themachine type of theinstance. To oversubscribe the queue count, the machine type youspecify must support gVNIC and Tier_1 networking.NETWORK_NAME: the name of the network createdpreviouslySUBNET_*: the name of one of the subnets createdpreviouslyQUEUE_SIZE: the number of queues for the vNIC,subject to the rules discussed inCustom queue allocation.
Terraform
- If you don't already have aVPC networkwith a subnet for each vNIC interface you plan to configure, create them.
Create a compute instance with specific queue counts for vNICs using the
google_compute_instanceresource.Repeat the--network-interfaceparameter for each vNICyou want to configure for the compute instance, and includethequeue-countparameter.# Queue oversubscription instanceresource "google_compute_instance" "VM_NAME" {project = "PROJECT_ID"boot_disk { auto_delete = true device_name = "DEVICE_NAME" initialize_params { image="IMAGE_NAME" size =DISK_SIZE type = "DISK_TYPE" }}machine_type = "MACHINE_TYPE"name = "VM_NAME"zone = "ZONE"network_performance_config { total_egress_bandwidth_tier = "TIER_1"}network_interface { nic_type = "GVNIC" queue_count =QUEUE_COUNT_1 subnetwork_project = "PROJECT_ID" subnetwork = "SUBNET_1" }network_interface { nic_type = "GVNIC" queue_count =QUEUE_COUNT_2 subnetwork_project = "PROJECT_ID" subnetwork = "SUBNET_2"}network_interface { nic_type = "GVNIC" queue_count =QUEUE_COUNT_3 subnetwork_project = "PROJECT_ID" subnetwork = "SUBNET_3""}network_interface { nic_type = "GVNIC" queue_count =QUEUE_COUNT_4 subnetwork_project = "PROJECT_ID" subnetwork = "SUBNET_4""}}
Replace the following:
VM_NAME: aname for the newcompute instancePROJECT_ID: ID of the project to createthe instance in. Unless you are using a Shared VPC network, theproject you specify must be the same one in which all the subnetsand networks were created in.DEVICE_NAME: The name to associate with the bootdisk in the guest OSIMAGE_NAME: the name of an image,for example,"projects/debian-cloud/global/images/debian-11-bullseye-v20231010".DISK_SIZE: the size of the boot disk, in GiBDISK_TYPE: the type of disk to use for the bootdisk, for example,pd-standardMACHINE_TYPE: themachine type of theinstance. To oversubscribe the queue count, the machine type youspecify must support gVNIC and Tier_1 networking.ZONE: the zone to create the instance inQUEUE_COUNT: the number of queues for the vNIC,subject to the rules discussed inCustom queue allocation.SUBNET_*: the name of the subnet that the networkinterface connects to
REST
- If you don't already have aVPC networkwith a subnet for each vNIC interface you plan to configure, create them.
Create a compute instance with specific queue counts for NICs using the
instances.insertmethod.Repeat thenetworkInterfacesproperty to configure multiple networkinterfaces.POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances{"name": "VM_NAME","machineType": "machineTypes/MACHINE_TYPE","networkPerformanceConfig": { "totalEgressBandwidthTier": TIER_1},"networkInterfaces": [ { "nicType": gVNIC, "subnetwork":"regions/region/subnetworks/SUBNET_1", "queueCount": "QUEUE_COUNT_1" } ],"networkInterfaces": [ { "nicType": gVNIC, "subnetwork":"regions/region/subnetworks/SUBNET_2", "queueCount": "QUEUE_COUNT_2" } ],}Replace the following:
PROJECT_ID: ID of the project to createthe compute instance inZONE: zone to create the compute instance inVM_NAME:name of thenew compute instanceMACHINE_TYPE: machine type,predefined orcustom,for the new compute instance. To oversubscribe the queue count, themachine type must support gVNIC and Tier_1 networking.SUBNET_*: the name of the subnet that thenetwork interface connects toQUEUE_COUNT: Number of queues for the vNIC,subject to the rules discussed inCustom queue allocation.
Queue allocations and changing the machine type
Compute instances are created with adefault queue allocation,or you can assign acustom queue count toeach virtual network interface card (vNIC) when you create a new computeinstance by using the Compute Engine API. The default or custom vNIC queue assignmentsare only set when creating a compute instance. If your instance has vNICs thatuse default queue counts, you canchange its machine type.If the machine type that you are changing to has a different number of vCPUs,the default queue counts for your instance are recalculated based on the newmachine type.
If your VM has vNICs which use custom, non-default queue counts,then you can change the machine type by using the Google Cloud CLI orCompute Engine API toupdate the instance properties.The conversion succeeds if the resulting VM supports the samequeue count per vNIC as the original instance. For VMs that usethe VirtIO-Net interface and have a custom queue count that is higher than 16per vNIC, you can't change the machine type to a third generation or latermachine type, because they use only gVNIC. Instead, you can migrate your VMto a third generation or later machine type by following theinstructions inMove your workload to a new compute instance.
What's next
- Learn more aboutmachine types.
- Learn more aboutVirtual machine instances.
- Create and start a VM instance.
- Configure per VM Tier_1 networking performancefor a compute instance.
- Complete the quickstart tutorialCreate a Linux VM instance in Compute Engine.
- Complete the quickstart tutorialCreate a Windows Server VM instance in Compute Engine.
Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2025-12-15 UTC.