Create nested VMs

Linux

Nested virtualization is allowed by default, so unless somebody modifies theconstraint for nested virtualization, you do not need to make any changes beforeyou create nested VMs in an organization, folder, or project. If your projectdoesn't belong to an organization, nested virtualization is allowed by defaultand you can't change the constraint. For information about how to modify theconstraint that determines whether you can create nested VMs, seeManage the nested virtualization constraint.

This document describes how to create various types of level 2 (L2) virtualmachine (VM) instances. Before creating a nested VM, you mustcreate an L1 VM that has nested virtualization enabled.For a description of L1 and L2 VMs, see theNested virtualization overview.

After creating an L1 VM that has nested virtualization enabled, you can do anyof the following:

Create an L2 VM with external network access
Create an L2 VM with a private network bridge to the L1 VM
Create an L2 VM with network access from outside the L1 VM

Before you begin

If you haven't already, then set up authentication.Authentication is the process by which your identity is verified for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
gcloud
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
For more information, seeAuthenticate for using REST in the Google Cloud authentication documentation.

Creating an L2 VM with external network access

Create an L2 VM with external network access by using the following procedure.This procedure usesqemu-system-x86_64 to start the L2 VM. If you are usinganother procedure to create an L2 VM and are experiencing trouble, reproduce theissue using this procedure before contactingSupport.

Create an L1 VM that has nested virtualization enabled.
Use thegcloud compute ssh command toconnect to the VM:
```
gcloud compute sshVM_NAME
```
ReplaceVM_NAME with the name of the VM to connectto.

Install the latestqemu-kvm package:

sudo apt update && sudo apt install qemu-kvm -y

Download a QEMU-compatible OS image to use for the L2 VM.
Use the following command to start the L2 VM. When prompted, log in withuser: root,password: root.
```
sudo qemu-system-x86_64 -enable-kvm -hdaIMAGE_NAME -m 512 -curses
```
ReplaceIMAGE_NAME with the name of theQEMU-compatible OS image to use for the L2 VM.
Test that your L2 VM has external access:
```
user@nested-vm:~$ host google.com
```

Creating an L2 VM with a private network bridge to the L1 VM

Create an L2 VM with a private network bridge to the previously created L1 VM byusing the following procedure. For information about changing the defaultmaximum transmission unit (MTU) for your VPC network, see themaximum transmission unit overview.

Create an L1 VM that has nested virtualization enabled.
Use thegcloud compute ssh command toconnect to the VM:
```
gcloud compute sshVM_NAME
```
ReplaceVM_NAME with the name of the VM to connectto.

Install the packages necessary to create the private bridge:

sudo apt update && sudo apt install uml-utilities qemu-kvm bridge-utils virtinst libvirt-daemon-system libvirt-clients -y

Start the default network that comes with thelibvirt package:
```
sudo virsh net-start default
```
Run the following command to check that you have thevirbr0 bridge:
```
ip addr
```

The output is similar to the following:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host       valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc pfifo_fast state UP group default qlen 1000    link/ether 42:01:0a:80:00:15 brd ff:ff:ff:ff:ff:ff    inet 10.128.0.21/32 brd 10.128.0.21 scope global eth0       valid_lft forever preferred_lft forever    inet6 fe80::4001:aff:fe80:15/64 scope link       valid_lft forever preferred_lft forever3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 52:54:00:8c:a6:a1 brd ff:ff:ff:ff:ff:ff    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0       valid_lft forever preferred_lft forever4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000    link/ether 52:54:00:8c:a6:a1 brd ff:ff:ff:ff:ff:ff

Create atap interface to go from the L1 VM to the L2 VM:
```
sudo tunctl -t tap0sudo ifconfig tap0 up
```
Bond thetap interface to the private bridge:
```
sudo brctl addif virbr0 tap0
```
Run the following command to verify the setup of the bridge network:
```
sudo brctl show
```

The output is similar to the following:

bridge name     bridge id               STP enabled     interfacesvirbr0          8000.5254008ca6a1       yes             tap0                                                        virbr0-nic

Download a QEMU-compatible OS image to use for the L2 VM.
Runscreen, and press Enter at the welcome prompt:
```
screen
```
Use the following command to start the L2 VM. When prompted, login in withuser: root,password: root.
```
sudo qemu-system-x86_64 -enable-kvm -hdaIMAGE_NAME -m 512 -net nic -net tap,ifname=tap0,script=no -curses
```
ReplaceIMAGE_NAME with the name of theQEMU-compatible OS image to use for the L2 VM.
On the L2 VM, runip addr show to confirm that the VM has an address in thevirbr0 space—for example,192.168.122.89:
```
user@nested-vm:~$ip addr
```
Start a placeholder web server on port8000:
```
user@nested-vm:~$python -m http.server
```
Detach from thescreen session withCtrl+A,Ctrl+D.
Test that your L1 VM can ping the L2 VM, replacing the following IP addresswith the IP address of the L2 VM:
```
curl 192.168.122.89:8000
```

The output is similar to the following:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><html><title>Directory listing for /</title><body><h2>Directory listing for /</h2><hr><ol><li><a href=".aptitude/">.aptitude/</a><li><a href=".bashrc">.bashrc</a><li><a href=".profile">.profile</a></ol><hr></body></html>

Creating an L2 VM with network access from outside the L1 VM

Permissions required for this task

To perform this task, you must have the followingpermissions:

compute.instances.create permissions on the project, folder, or organization

You can set up an L2 VM with an alias IP so that VMs outside the L1 VM canaccess the L2 VM. Use the following procedure to create an L2 VM with networkaccess by way of an alias IP from outside the previously created L1 VM. Forinformation about creating alias IP addresses, seeConfigure alias IP ranges.

The following procedure assumes a previously created subnet calledsubnet1. Ifyou already have a subnet with a different name, replacesubnet1 with the nameof your subnet, or create a new subnet namedsubnet1.

Create an L1 VM with nested virtualization enabled and include an alias IPrange and support for HTTP/HTTPS traffic:

gcloud

gcloud compute instances createVM_NAME --enable-nested-virtualization \    --tags http-server,https-server --can-ip-forward \    --min-cpu-platform "Intel Haswell" \    --network-interface subnet=subnet1,aliases=/30

ReplaceVM_NAME with the name for the L1 VM.

REST

POST https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/ZONE/instances{  ...  "name":VM_NAME,  "tags": {    "items": [      http-server,https-server    ],  },  "canIpForward": true,  "networkInterfaces": [    {      "subnetwork": "subnet1",      "aliasIpRanges": [        {          "ipCidrRange": "/30"        }      ],    }  ],  "minCpuPlatform": "Intel Haswell",  "advancedMachineFeatures": {    "enableNestedVirtualization": true  },  ...}

Replace the following:

PROJECT_ID: the project ID
ZONE: the zone to create the VM in
VM_NAME: the name of the VM

Use thegcloud compute ssh command toconnect to the VM. If you have trouble connecting to the VM, try resettingthe VM or modifying the firewall rules.
```
gcloud compute sshVM_NAME
```
ReplaceVM_NAME with the name of the VM to connectto.

Update the VM and install the necessary packages:

sudo apt update && sudo apt install uml-utilities qemu-kvm bridge-utils virtinst libvirt-daemon-system libvirt-clients -y

Start the default network that comes with thelibvirt package:
```
sudo virsh net-start default
```
Run the following command to check that you have thevirbr0 bridge:
```
user@nested-vm:~$ip addr
```

Verify output similar to the following:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    inet 127.0.0.1/8 scope host lo       valid_lft forever preferred_lft forever    inet6 ::1/128 scope host       valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc pfifo_fast state UP group default qlen 1000    link/ether 42:01:0a:80:00:15 brd ff:ff:ff:ff:ff:ff    inet 10.128.0.21/32 brd 10.128.0.21 scope global eth0       valid_lft forever preferred_lft forever    inet6 fe80::4001:aff:fe80:15/64 scope link       valid_lft forever preferred_lft forever3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000    link/ether 52:54:00:8c:a6:a1 brd ff:ff:ff:ff:ff:ff    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0       valid_lft forever preferred_lft forever4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000    link/ether 52:54:00:8c:a6:a1 brd ff:ff:ff:ff:ff:ff

Create atap interface to go from the L1 VM to the L2 VM:
```
sudo tunctl -t tap0sudo ifconfig tap0 up
```
Bond thetap interface to the private bridge:
```
sudo brctl addif virbr0 tap0
```
Run the following command to verify the setup of the bridge network:
```
sudo brctl show
```

Verify output similar to the following:

bridge name     bridge id               STP enabled     interfacesvirbr0          8000.5254008ca6a1       yes             tap0                                                        virbr0-nic

Download a QEMU-compatible OS image to use for the L2 VM.
Runscreen, and press Enter at the welcome prompt:
```
screen
```
Use the following command to start the nested VM. When prompted, login inwithuser: root,password: root.
```
sudo qemu-system-x86_64 -enable-kvm -hdaIMAGE_NAME -m 512 -net nic -net tap,ifname=tap0,script=no -curses
```
ReplaceIMAGE_NAME with the name of theQEMU-compatible OS image to use for the L2 VM.
On the L2 VM, runip addr to confirm that the L2 VM has an address in thevirbr0 space, such as192.168.122.89:
```
user@nested-vm:~$ip addr
```
Start a placeholder web server on port8000:
```
user@nested-vm:~$python -m http.server
```
Detach from thescreen session withCtrl+A,Ctrl+D.
Test that your L1 VM can ping the L2 VM, replace the IP address below withthe IP address of the L2 VM:
```
curl 192.168.122.89:8000
```

Verify that the response from the L2 VM is similar to the following:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><html><title>Directory listing for /</title><body><h2>Directory listing for /</h2><hr><ol><li><a href=".aptitude/">.aptitude/</a><li><a href=".bashrc">.bashrc</a><li><a href=".profile">.profile</a></ol><hr></body></html>

On the L1 VM, set upiptables to allow forwarding from the L1 VM to the L2VM. For the L2 OS image used in these instructions, you must flush the IPtables:
```
sudo iptables -F
```
Determine the L1 VM's alias IP:
```
ip route show table local
```

Verify that the output is similar to the following. For this example, thereare two IP addresses associated with the L2 VM'seth0 ethernet device. Thefirst,10.128.0.2, is the L2 VM's primary IP address, which is returned bysudo ifconfig -a. The second,10.128.0.13, is the L2 VM's alias IPaddress.

local 10.128.0.2 dev eth0 proto kernel scope host src 10.128.0.2broadcast 10.128.0.2 dev eth0 proto kernel scope link src 10.128.0.2local 10.128.0.13/30 dev eth0 proto 66 scope hostbroadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1broadcast 192.168.122.0 dev virbr0 proto kernel scope link src192.168.122.1 linkdownlocal 192.168.122.1 dev virbr0 proto kernel scope host src 192.168.122.1broadcast 192.168.122.255 dev virbr0 proto kernel scope link src192.168.122.1 linkdown

Run the following commands to forward traffic from the10.128.0.13 examplealias IP to the192.168.122.89 example IP for the L2 VM:

echo 1 | sudo tee /proc/sys/net/ipv4/ip_forwardsudo iptables -t nat -A PREROUTING -d 10.128.0.13 -j DNAT --to-destination 192.168.122.89sudo iptables -t nat -A POSTROUTING -s 192.168.122.89 -j MASQUERADEsudo iptables -A INPUT -p udp -j ACCEPTsudo iptables -A FORWARD -p tcp -j ACCEPTsudo iptables -A OUTPUT -p tcp -j ACCEPTsudo iptables -A OUTPUT -p udp -j ACCEPT

For information about troubleshootingiptables, seeiptables not forwarding traffic.

Verify L2 VM access from outside the L1 VM by logging onto another VM that ison the same network as the L1 VM, and making acurl request to the aliasIP, replacing the IP address below with the L2 VM's alias IP:
```
user@another-vm:~$curl 10.128.0.13:8000
```

Verify that thecurl response is similar to the following:

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><html><title>Directory listing for /</title><body><h2>Directory listing for /</h2><hr><ol><li><a href=".aptitude/">.aptitude/</a><li><a href=".bashrc">.bashrc</a><li><a href=".profile">.profile</a></ol><hr></body></html>

What's next

Troubleshoot nested virtualization

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-07-09 UTC.

Movatterモバイル変換

Create nested VMs Stay organized with collections Save and categorize content based on your preferences.

Before you begin

gcloud

REST

Creating an L2 VM with external network access

Creating an L2 VM with a private network bridge to the L1 VM

Creating an L2 VM with network access from outside the L1 VM

Permissions required for this task

gcloud

REST

What's next

Create nested VMs