This document describes Compute Engine support for nested virtualization.Nested virtualization lets you run virtual machine (VM) instances inside ofother VMs so you can create your own virtualization environments. To supportnested virtualization, Compute Engine addsIntel Virtualization Technology (VT-x)instructions to VMs, so when you create a VM, the hypervisor that is already onthat VM can run additional VMs.

Compute Engine VMs run on a physical host that hasGoogle'ssecurity-hardened, KVM-based hypervisor.With nested virtualization, the physical host and its hypervisor are the level 0(L0) environment. The L0 environment can host multiple level 1 (L1) VMs. On eachL1 VM is another hypervisor, which is used to install the level 2 (L2) VMs.Figure 1 shows the relationship between the physical host, the L1 VMs, and theL2 VMs:

Use cases

Scenarios where you might consider using nested virtualization include thefollowing:

• You have VMs that you can't run on Compute Engine: For example, youmight have a disaster recovery solution for an on-premises workload that isrunning on VMs that fail over to Compute Engine VMs. Runningnested virtualization might save you time that you would use to portyour VMs to Compute Engine.

• You have a software-validation framework that you use to test and validate newversions of a software package on numerous versions of different OSes: Usingnested virtualization lets you avoid converting and managing a library ofCompute Engine images.

Performance considerations

Even with hardware-assisted nested virtualization, nested VMs might experience a10% or greater decrease in performance for workloads that are CPU-bound andpossibly greater than a 10% decrease for workloads that are input/output bound.

Restrictions

L1 VMs have the following restrictions:

• The only hypervisor supported in an L1 VM is Linux KVM; Microsoft Hyper-Visn't supported.

• You can't use the following VMs:

  • E2 VMs

  • Memory-optimized VMs

  • VMs powered byAMD and Arm processors

  • H4D VMs

  For information about the processors supported in each zone, seeAvailable regions and zones.

L2 VMs have the following restrictions:

• For licensed operating systems, you mustbring your own licenses.

Using nested virtualization

To use nested virtualization, complete the following steps:

1. Check whether the nested virtualization constraint isdisabled

2. Create an L1 VM that has nested virtualizationenabled

3. Create a nested L2VM

If you run into any issues while creating a VM that has nested virtualizationenabled or creating nested VMs, seeTroubleshoot nested virtualization.