Table of Contents
A virtual machine is a virtual computer (the "guest") running insideanother computer (the "host"). Virtual machines are useful for testing,running different operating systems, isolating parts of a system, andmore.
nvmm(4) (NetBSD Virtual Machine Monitor) is NetBSD's nativehypervisor.In regular usage, it's used as an "accelerator" for the QEMUvirtual machine software. It will make virtual machines on yourNetBSD host run faster by taking advantage of CPU virtualizationextensions.Currently, a CPU that supports AMD SVM or Intel VMX is required, butmore backends for other architectures may be added in the future.QEMU can also be used without an accelerator, with significantly reducedperformance.
Other hypervisors supported by NetBSD includeIntel HAXM(also used with QEMU), andXen,which has quite a different design.
When running modern operating systems as VM guests,you will generally want to use para-virtualized I/O, rather than havingQEMU emulate real hardware devices.On NetBSD, this is supported with thevirtio(4) drivers.
Many computers (especially laptops) have hardwarevirtualization capabilities disabled by default. You may need to enablethe necessary features from the firmware at boot.
Before loading the NVMM module, make sure the modulesin/stand are correct and up-to-date for the versionof the NetBSD kernel you are using.
The NetBSD Virtual Machine Monitor isn't active by default.It must be activated by loading thenvmm module withmodload(8):
#modload nvmm
Verify NVMM is loaded withmodstat(8):
#modstat | grep nvmmnvmm misc filesys - 0 - -
You can load the module automatically at boot time by adding thisline to/etc/modules.conf:
nvmm
Loading NVMM at boot time will also allow the system to run with asecmodel_securelevel(9) of 1, which prevents loading modulesafter boot.However, since NVMM blocks things like suspend, you may wishto unload it:
#modunload nvmm
By default the/dev/nvmm device is owned by theroot user and groupnvmm.You probably want to run virtual machines as a non-root user, e.g.for security reasons.
You can add users to the groupnvmmto allow them to run virtual machines. E.g. withuseradd(8).
Alternatively you can set the owner of thedevice/dev/nvmm to the user that should be allowedto run virtual machines:
#chown nia /dev/nvmm
You can see NVMM's current status withnvmmctl(8):
$nvmmctl identifynvmm: Kernel API version 2nvmm: State size 1008nvmm: Max machines 128nvmm: Max VCPUs per machine 256nvmm: Max RAM per machine 128Gnvmm: Arch Mach conf 0nvmm: Arch VCPU conf 0x3<CPUID,TPR>nvmm: Guest FPU states 0x3<x87,SSE>
QEMU is a CPU emulator and virtual machine that can useNVMM as an accelerator. It isn't included with NetBSD by default.However, it is available in pkgsrcasemulators/qemu, and can beinstalled withpkgin:
#pkgin install qemu
This command starts a VM in an X11 window with NVMM acceleration,the same CPU type as the host machine, two CPU cores,and one gigabyte of memory:
$qemu-system-x86_64 -accel nvmm \ -cpu max -smp cpus=2 -m 1G \ -display sdl,gl=on \ -cdrom NetBSD-9.1-amd64.iso
The guest system will be much slower without accelerationas every CPU instruction will have to be emulated.
You should also be able to see the virtual machine runningwithnvmmctl(8):
$nvmmctl listMachine ID VCPUs RAM Owner PID Creation Time---------- ----- ---- --------- ------------------------0 2 147M 10982 Sat May 8 10:09:59 2021
Generally, you will want to create a virtual drive to contain yourvirtual machine on the host. We’ll want to create aqcow2 imagebecause it provides better performance and is more versatile than araw image:
$qemu-img create -f qcow2 netbsd.qcow2 16G
A VirtIO block device provides the best performance.Add the following arguments toqemu-system-x86_64to use it:
-drive file=netbsd.qcow2,if=none,id=hd0 \-device virtio-blk-pci,drive=hd0Older operating systems may not have VirtIO drivers, in whichcase you can use a normal emulated disk:
-hda netbsd.qcow2Operating systems require a good source of randomness for systemsecurity, cryptography, and so on. In a VM, this is ideallyprovided by the host machine, which has greater access to theunderlying hardware. You can easily attach a VirtIO random numbergenerator device with the following arguments to QEMU:
-object rng-random,filename=/dev/urandom,id=viornd0 \-device virtio-rng-pci,rng=viornd0This requires no extra configuration on the host machine.
Entropy is generally required for secure communications.For more information on entropy, refer toentropy(7).
The simplest way to set up networking with QEMU is so-called"user networking". This will mean raw socket operations likeping(8) won’t work, but normal TCP/IP protocolslike HTTP/FTP/etc will work. Another way is with bridgednetworking, seeSection 30.3, “Configuring bridged networking on a NetBSD host”.
The most performant device type isvirtio-net-pci:
-netdev user,id=vioif0 -device virtio-net-pci,netdev=vioif0To use older guest operating systems that don’t support VirtIO,Intel Gigabit Ethernet is a good choice:
-netdev user,id=wm0 -device e1000,netdev=wm0Or an AMD PCnet card, for very old guest operating systems:
-netdev user,id=pcn0 -device pcnet,netdev=pcn0On a NetBSD host, the following QEMU arguments may be usedto enable audio:
-audiodev oss,id=oss,out.dev=/dev/audio,in.dev=/dev/audio \-device ac97,audiodev=ossac97 is the classic standardized sound driverfor x86 systems.
You may wish to change the/dev/audioXdevice being used, seeChapter 10,Audio.
You may need to adjust things further to get smooth playback,seeSection 30.4.3, “Smooth audio playback and latency in VMs”.
These arguments will create an X11 window with OpenGL enabled(for smooth scaling if the window is resized), using a VMware-compatibleVGA device, and an USB mouse:
-display sdl,gl=on -vga vmware \-usb -device usb-mouse,bus=usb-bus.0There is a VMware video driver included with X11 on NetBSD, sothe display will automatically configure whenstartx(1)runs and can be adjusted withxrandr(1).
A VNC display will allow remote access from a VNC client likenet/tigervnc, useful when running QEMUwith--daemonize on a server:
-display vnc=unix:/home/nia/.qemu-myvm-vnc -vga vmware-usb -device usb-mouse,bus=usb-bus.0A simpler option is acurses display,preferable for systems that don’tneed more than text output in a terminal:
-display cursesFor more information on configuring X11, seeChapter 9,The X Window System.
For more information on securely configuring VNC, seeQEMU’s online documentation on VNC.
While QEMU user networking is easy to use and doesn't require rootprivileges, it's generally slower than bridged networking using atap(4)device, and doesn't allow the use of diagnostic tools likeping(8)inside the guest.
To configure bridged networking on a NetBSD host, you must firstmake note of your host machine’s primary network interace.Find the one with an address assigned and a route to the outsideworld withifconfig(8).
In this example, the host machine’s primary interface iswm0. All of these commands run on the host machine.
Create a virtualtap(4) interface:
#ifconfig tap0 create#ifconfig tap0 descr "NetBSD VM" up
Create abridge(4) connecting the actual interfaceand the virtual interface:
#ifconfig bridge0 create#ifconfig bridge0 descr "LAN VM bridge" up#brconfig bridge0 add tap0 add wm0
Configure NetBSD to do this all at boot time by editing/etc/ifconfig.tap0:
createdescr "NetBSD VM" up! ifconfig bridge0 create! ifconfig bridge0 descr "LAN VM bridge" up! brconfig bridge0 add tap0 add wm0
You can now pass the arguments to QEMU to run withbridged networking:
-netdev tap,id=tap0,ifname=tap0,script=no -device virtio-net-pci,netdev=tap0For more information on NetBSD network configuration,seeChapter 24,Setting up TCP/IP on NetBSD in practice.
AVOID UNCLEAN SHUTDOWNS!This means pressing Ctrl+C or killing the virtual machine.In QEMU, the disks will rarely be synced, and data loss willalmost certainly occur.
You may wish to add thelog,noatimemount options in/etc/fstab next torw to speed upfsck(8).You can also enable thesync option,but this will significantly decrease performance.
Always shut down NetBSD safely using theshutdown(8)command and make backups.
QEMU's networking will sometimes configure an invalidIPv6 route on IPv4-only configurations, meaning programs like theNetBSD packaging tools will prefer IPv6 and spend a long time timing outbefore succeeding.
Work around this by editing/etc/rc.confto prefer IPv4 addresses:
ip6addrctl=YESip6addrctl_policy="ipv4_prefer"
Virtual machines cannot generally provide the same smooth playback atlow latency that real hardware provides.For smooth playback, you may need to increase NetBSD's audio latencyinside the VM:
$sysctl -w hw.audio0.blk_ms=100
To set this automatically automatically at boot time, add it to/etc/sysctl.conf.
You can test audio output in the VM. Ensure thataudiocfg(1) plays a continuous beep for each channel:
$audiocfg test 0
On physical hardware where the display resolutionis already set properly by the kernel, doing this will disablegraphical acceleration.
If you want to increase the size of the x86 console, enter the following at the NetBSD boot prompt:
>vesa 1024x768x32This setting can be made permanent in/boot.cfg.