CPU overcommit onsole-tenant nodeslets you schedule instances that can share their spare CPU cycles with eachother. This lets you overprovision sole-tenant node resources and schedule moreVM CPUs on a sole-tenant node than are normally available. CPU overcommit isespecially valuable for workloads that are underutilized but might experiencerelatively uncorrelated bursts.

CPU overcommit can help you reduce per-VM costs by spreading the cost of a sole-tenant node across more VMs. It can also reduce per-VM licensing costs whenusing per-socket or per-core licenses.

VMs with overcommitted CPUs can utilize otherwise unused CPU resources in thefollowing ways:

• If a sole-tenant node isn't full, overcommitted VMs can utilize unallocatedcores.

• If another VM on a sole-tenant node isn't utilizing all of its CPUresources—for example, because the CPU is idle—an overcommitted VMcan use those CPU resources.

Overcommit level

You can specify the value for the minimum number of CPUs that are allocated to aVM when you create a VM or after you stop a VM. The overcommit level representsthe minimum number of underlying CPU threads that are guaranteed to be availablefor a VM. If the VM has more vCPUs than underlying threads available, the VM'svCPUs share the underlying compute resources and run with degraded performance.

You can set this value for each VM, which lets you provision VMs with differentratios of CPU overcommit on a single sole-tenant node. Lower values reducecapacity requirements at the potential expense of performance if correlatedbursts occur. Determining an optimal value for the minimum number of CPUsrequires an understanding of your workload utilization and iterativemodification of the value.

When setting this value, keep in mind the following:

• If you don't set the value for the minimum number of CPUs, or you set thevalue for the minimum number of CPUs equal to the number of CPUs on the VM'smachine type, the VM's allowable overcommit ratio is 1.0. With an overcommitratio of 1.0, all of the CPUs are accessible only to this VM, and there areno CPU resources available to be overcommitted to other VMs.

• The minimum number of CPUs can't be greater than the number of CPUs specifiedby the VM's machine type.

• The sum of the values for the minimum number of CPUs for all of the VMs on asole-tenant node can't exceed the CPU capacity of thatsole-tenant node type,which on then1-node-96-624 node type is 96.

The value for the number of CPUs specified by the VM's machine type is a staticvalue, and represents the number of CPUs that a VM can burst up to from theminimum number if those CPUs are available. If you require a number of CPUsdifferent from those provided bypredefined machine types, you can use acustom machine type.

Considerations

Before configuring the CPU overcommit levels for VMs, consider the criticalityof your workload. Less critical workloads, such as development and testworkloads, can potentially tolerate higher overcommit levels. More criticalworkloads, such as a production payments system, might not tolerate as muchovercommit or any at all.

Also consider the utilization of your workload. Workloads with high CPUutilization are not good candidates for CPU overcommit because they won'thave spare utilization cycles for other overcommitted VMs to utilize.Additionally, workloads with low average CPU utilization, but low utilizationpeak, might benefit from different sizes of machine types.

Using CPU overcommit benefits uncorrelated bursty workloads that have high peakutilization and low average utilization because these workloads are more likelyto have available CPU resources to share across VMs when some VMs need to bursttheir utilization. If all of the VMs on a host burst at one time, the host won'thave sufficient resources for your VMs.

Workload limitations

CPU overcommit is best suited for workloads without stringent performancerequirements—for example, development and test workloads, and virtualdesktop infrastructures.

High levels of CPU overcommit might not be appropriate for workloads that aresensitive to performance.

For workloads with average and peak utilization that is consistently low,Google recommendsrightsizing. That is, instead of overcommitting CPUs, werecommend modifying the size of the VM instance to match the resourcerequirements of that workload.

If your instances are too highly overcommitted,move them to anothersole-tenant node.

Machine type limitations

You can only overcommit CPUs on the following:

• N1 machine series VMsthat are provisioned on node groups that are based on then1-node-96-624node type

• N2 machine series VMsthat are provisioned on node groups that are based on the followingnodetypes:

  • n2-node-80-640
  • n2-node-128-864

• N2D machine seriesVMs that are provisioned on node groups that are based on the followingnode types:

  • n2d-node-224-896
  • n2d-node-224-1792

Overcommit level limitations

You can only configure the minimum CPU on each sole-tenant node to half of theVM's CPUs, allowing for a maximum sole-tenant node overcommit ratio of 2.0.

VM scheduling limitations

Sole-tenant node groups based on sole-tenant node templates that aren'tconfigured for CPU overcommit don't allow for provisioning of VMs with CPUovercommit enabled. That is, you can't schedule a VM with a specified minimumnumber of CPUs on a sole-tenant node group that is not configured for CPUovercommit.

Quota

CPU quota is based on the number of vCPUs of thesole-tenant nodetype, not the potentialmaximum of vCPUs available for overcommitting.

Costs

Sole-tenant nodes that have CPU overcommit selected on their node template arecharged an additional 25%. This charge is in addition to the10% premium forrunning VMs on sole-tenant nodes. The CPUovercommit premium is fixed, regardless of the CPU overcommit level and how manyVMs are scheduled on the sole-tenant node.

Sole-tenant nodes offercommitted usediscounts.Sustained use discounts are availablefor the sole-tenancy premium and the CPU overcommit premium.

To estimate the cost of running VMs on sole-tenant nodes, see thePricingCalculator.

Configure sole-tenant VMs for overcommitting

To configure sole-tenant VMs to have CPU resources available for overcommitting,do the following:

1. Create a sole-tenant node template that has CPU overcommit enabled. You mustenable CPU overcommit while creating the node template. You can't enable CPUovercommit after creating a node template.

2. Create a sole-tenant node group based on the sole-tenant node template thathas CPU overcommit enabled.

3. Create a VM and do the following:

   1. Choose a machine type for the VM. The number of CPUs on the machine typerepresents the maximum number of CPUs that the VM can burst up to fromthe minimum number of CPUs if the minimum number of CPUs is less than thenumber of CPUs specified by the machine type.

      You can choose a different machine type for each VM on a sole-tenantnode, provided you don't exceed the CPU and memory capacity of thesole-tenant node.

   2. Specify the minimum number of CPUs to allocate to that single VM, or useamanaged instance groupto create multiple VMs that have the same CPU overcommit level.

Before you begin

• Create a sole-tenant node template and specify--cpu-overcommit-type=enabled.
• Create a sole-tenant node group based on the sole-tenant node template with CPU overcommit enabled.
• If you haven't already, set upauthentication. Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

  Select the tab for how you plan to use the samples on this page:

  Console

  When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

  gcloud

  1. Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

     gcloudinit

     If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

     Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.
  2. Set a default region and zone.

  REST

  To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

  Install the Google Cloud CLI. After installation,initialize the Google Cloud CLI by running the following command:

  gcloudinit

  If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity.

  Note: If you installed the gcloud CLI previously, make sure you have the latest version by runninggcloud components update.

  For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Set the CPU overcommit level

The following procedures show you how to create a sole-tenant VM with CPUresources available for overcommitting. If you need to modify the CPU overcommitlevel of a VM that is running, you must first stop the VM.

gcloudcomputeinstancescreateVM_NAME\--machine-type=MACHINE_TYPE\--min-node-cpu=MIN_VCPUS\--node-group=GROUP_NAME

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances{  "machineType": "zones/MACHINE_TYPE_ZONE/machineTypes/MACHINE_TYPE",  "name": "VM_NAME",  "scheduling": {    "minNodeCpus":MIN_VCPUS,    "nodeAffinities": [      {        "key": "compute.googleapis.com/node-group-name",        "operator": "IN",        "values": [          "GROUP_NAME"        ]      }    ]  },  ...}

Update the CPU overcommit level

The following procedures show you how to update the CPU overcommit level of asole-tenant VM.

Disable CPU overcommitment for sole-tenant VMs

The following procedures show you how to disable the CPU overcommitment of asole-tenant VM.

gcloudcomputeinstancesset-schedulingVM_NAME\--clear-min-node-cpu

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances/VM_NAME/setScheduling{  "minNodeCpus":null}

View CPU usage

To check the CPU usage of sole-tenant VMs in a sole-tenant node group, do thefollowing:

1. In the Google Cloud console, go to theSole-tenant nodes page.

   Go to Sole-tenant nodes

2. ClickNode groups.

3. Click the sole-tenant node group containing the sole-tenant node that hasthe VM with overcommitted CPUs.

4. Click the sole-tenant node that has the VM with overcommitted CPUs.

5. Under the name of the sole-tenant node, view theCPU usage,CPUovercommit type, and theMin CPU usage.

   • CPU usage shows the total of the maximum number of CPUs for all ofthe VMs on this sole-tenant node divided by the number of CPUs specifiedby the sole-tenant node type.

     The number of CPUs on the node available for overcommitting is thenumerator minus the denominator, and the overcommit level is the quotientof the numerator and the denominator.

   • Min CPU usage shows the sum of the minimum number of CPUsallocated for all of the VMs on a sole-tenant node divided by the numberof CPUs specified by the node type.

Optimize CPU overcommit levels

To help optimize tuning of your CPU overcommit levels, Compute Engineprovides theScheduler Wait Time metric. TheScheduler Wait Time metricindicates the aggregate wait time for all vCPUs on the VM and helps youdetermine the impact of CPU overcommit on the VM's performance.

Workload sensitivity varies, but a general rule is to use 20 milliseconds ofscheduler wait time accrued per second (20 msps) as the maximum wait timefor each vCPU. For example, if a VM is set to 8 vCPUs, then a rule-of-thumbthreshold is 160 msps, which results in an acceptable averageSchedulerWait Time of 20 msps per vCPU. The performance requirements of yourworkload will ultimately dictate acceptable thresholds.

1. In the Google Cloud console, go to theMonitoring page.

   Go to Monitoring

2. ClickMetrics explorer.

3. In theResource type field, enterVM Instance.

4. In theMetric field, enterScheduler Wait Time.

5. Optionally, set up alerting to trigger alerts for VM wait time thresholdsby clickingAlerting.

What's next

• Learn more about sole-tenant nodes

• Learn how to bring your own licenses