emulated computer which sees them as if they were real devices.
A guest operating system running in the emulated computer accesses these
-devices, and runs as it were running on real hardware. For instance you can pass
-an iso image as a parameter to Qemu, and the OS running in the emulated computer
-will see a real CDROM inserted in a CD drive.
+devices, and runs as if it were running on real hardware. For instance, you can pass
+an ISO image as a parameter to Qemu, and the OS running in the emulated computer
+will see a real CD-ROM inserted into a CD drive.
Qemu can emulate a great variety of hardware from ARM to Sparc, but {pve} is
only concerned with 32 and 64 bits PC clone emulation, since it represents the
NOTE: You may sometimes encounter the term _KVM_ (Kernel-based Virtual Machine).
It means that Qemu is running with the support of the virtualization processor
-extensions, via the Linux kvm module. In the context of {pve} _Qemu_ and
-_KVM_ can be used interchangeably as Qemu in {pve} will always try to load the kvm
+extensions, via the Linux KVM module. In the context of {pve} _Qemu_ and
+_KVM_ can be used interchangeably, as Qemu in {pve} will always try to load the KVM
module.
Qemu inside {pve} runs as a root process, since this is required to access block
--------------------------------------------
The PC hardware emulated by Qemu includes a mainboard, network controllers,
-scsi, ide and sata controllers, serial ports (the complete list can be seen in
+SCSI, IDE and SATA controllers, serial ports (the complete list can be seen in
the `kvm(1)` man page) all of them emulated in software. All these devices
are the exact software equivalent of existing hardware devices, and if the OS
running in the guest has the proper drivers it will use the devices as if it
as measured with `bonnie++(8)`. Using the virtio network interface can deliver
up to three times the throughput of an emulated Intel E1000 network card, as
measured with `iperf(1)`. footnote:[See this benchmark on the KVM wiki
-http://www.linux-kvm.org/page/Using_VirtIO_NIC]
+https://www.linux-kvm.org/page/Using_VirtIO_NIC]
[[qm_virtual_machines_settings]]
{pve} allows to boot VMs with different firmware and machine types, namely
xref:qm_bios_and_uefi[SeaBIOS and OVMF]. In most cases you want to switch from
-the default SeabBIOS to OVMF only if you plan to use
+the default SeaBIOS to OVMF only if you plan to use
xref:qm_pci_passthrough[PCIe pass through]. A VMs 'Machine Type' defines the
hardware layout of the VM's virtual motherboard. You can choose between the
default https://en.wikipedia.org/wiki/Intel_440FX[Intel 440FX] or the
format does not support thin provisioning or snapshots by itself, requiring
cooperation from the storage layer for these tasks. It may, however, be up to
10% faster than the *QEMU image format*. footnote:[See this benchmark for details
- http://events.linuxfoundation.org/sites/events/files/slides/CloudOpen2013_Khoa_Huynh_v3.pdf]
+ https://events.static.linuxfound.org/sites/events/files/slides/CloudOpen2013_Khoa_Huynh_v3.pdf]
* the *VMware image format* only makes sense if you intend to import/export the
disk image to other hypervisors.
in that case it makes sense to set the number of sockets to what the license
allows you.
-Increasing the number of virtual cpus (cores and sockets) will usually provide a
+Increasing the number of virtual CPUs (cores and sockets) will usually provide a
performance improvement though that is heavily dependent on the use of the VM.
-Multithreaded applications will of course benefit from a large number of
-virtual cpus, as for each virtual cpu you add, Qemu will create a new thread of
+Multi-threaded applications will of course benefit from a large number of
+virtual CPUs, as for each virtual cpu you add, Qemu will create a new thread of
execution on the host system. If you're not sure about the workload of your VM,
it is usually a safe bet to set the number of *Total cores* to 2.
is greater than the number of cores on the server (e.g., 4 VMs with each 4
cores on a machine with only 8 cores). In that case the host system will
balance the Qemu execution threads between your server cores, just like if you
-were running a standard multithreaded application. However, {pve} will prevent
+were running a standard multi-threaded application. However, {pve} will prevent
you from starting VMs with more virtual CPU cores than physically available, as
this will only bring the performance down due to the cost of context switches.
^^^^^^^^^^^^^
Modern operating systems introduced the capability to hot-plug and, to a
-certain extent, hot-unplug CPUs in a running systems. Virtualisation allows us
+certain extent, hot-unplug CPUs in a running system. Virtualization allows us
to avoid a lot of the (physical) problems real hardware can cause in such
scenarios.
Still, this is a rather new and complicated feature, so its use should be
open-source, x86 BIOS implementation. SeaBIOS is a good choice for most
standard setups.
-There are, however, some scenarios in which a BIOS is not a good firmware
-to boot from, e.g. if you want to do VGA passthrough. footnote:[Alex Williamson has a very good blog entry about this.
-http://vfio.blogspot.co.at/2014/08/primary-graphics-assignment-without-vga.html]
-In such cases, you should rather use *OVMF*, which is an open-source UEFI implementation. footnote:[See the OVMF Project http://www.tianocore.org/ovmf/]
+Some operating systems (such as Windows 11) may require use of an UEFI
+compatible implementation instead. In such cases, you must rather use *OVMF*,
+which is an open-source UEFI implementation. footnote:[See the OVMF Project https://github.com/tianocore/tianocore.github.io/wiki/OVMF]
+
+There are other scenarios in which a BIOS is not a good firmware to boot from,
+e.g. if you want to do VGA passthrough. footnote:[Alex Williamson has a very
+good blog entry about this https://vfio.blogspot.co.at/2014/08/primary-graphics-assignment-without-vga.html]
If you want to use OVMF, there are several things to consider:
You can create such a disk with the following command:
- qm set <vmid> -efidisk0 <storage>:1,format=<format>
+----
+# qm set <vmid> -efidisk0 <storage>:1,format=<format>,efitype=4m,pre-enrolled-keys=1
+----
Where *<storage>* is the storage where you want to have the disk, and
*<format>* is a format which the storage supports. Alternatively, you can
create such a disk through the web interface with 'Add' -> 'EFI Disk' in the
hardware section of a VM.
+The *efitype* option specifies which version of the OVMF firmware should be
+used. For new VMs, this should always be '4m', as it supports Secure Boot and
+has more space allocated to support future development (this is the default in
+the GUI).
+
+*pre-enroll-keys* specifies if the efidisk should come pre-loaded with
+distribution-specific and Microsoft Standard Secure Boot keys. It also enables
+Secure Boot by default (though it can still be disabled in the OVMF menu within
+the VM).
+
+NOTE: If you want to start using Secure Boot in an existing VM (that still uses
+a '2m' efidisk), you need to recreate the efidisk. To do so, delete the old one
+(`qm set <vmid> -delete efidisk0`) and add a new one as described above. This
+will reset any custom configurations you have made in the OVMF menu!
+
When using OVMF with a virtual display (without VGA passthrough),
-you need to set the client resolution in the OVMF menu(which you can reach
+you need to set the client resolution in the OVMF menu (which you can reach
with a press of the ESC button during boot), or you have to choose
SPICE as the display type.
+[[qm_tpm]]
+Trusted Platform Module (TPM)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A *Trusted Platform Module* is a device which stores secret data - such as
+encryption keys - securely and provides tamper-resistance functions for
+validating system boot.
+
+Certain operating systems (e.g. Windows 11) require such a device to be attached
+to a machine (be it physical or virtual).
+
+A TPM is added by specifying a *tpmstate* volume. This works similar to an
+efidisk, in that it cannot be changed (only removed) once created. You can add
+one via the following command:
+
+----
+# qm set <vmid> -tpmstate0 <storage>:1,version=<version>
+----
+
+Where *<storage>* is the storage you want to put the state on, and *<version>*
+is either 'v1.2' or 'v2.0'. You can also add one via the web interface, by
+choosing 'Add' -> 'TPM State' in the hardware section of a VM.
+
+The 'v2.0' TPM spec is newer and better supported, so unless you have a specific
+implementation that requires a 'v1.2' TPM, it should be preferred.
+
+NOTE: Compared to a physical TPM, an emulated one does *not* provide any real
+security benefits. The point of a TPM is that the data on it cannot be modified
+easily, except via commands specified as part of the TPM spec. Since with an
+emulated device the data storage happens on a regular volume, it can potentially
+be edited by anyone with access to it.
+
[[qm_ivshmem]]
Inter-VM shared memory
~~~~~~~~~~~~~~~~~~~~~~
To add such a device, you can use `qm`:
- qm set <vmid> -ivshmem size=32,name=foo
+----
+# qm set <vmid> -ivshmem size=32,name=foo
+----
Where the size is in MiB. The file will be located under
`/dev/shm/pve-shm-$name` (the default name is the vmid).
boot' from the 'Options' Tab of your VM in the web interface, or set it with
the following command:
- qm set <vmid> -onboot 1
+----
+# qm set <vmid> -onboot 1
+----
.Start and Shutdown Order
be enforced between virtual machines running on the same host, not
cluster-wide.
+If you require a delay between the host boot and the booting of the first VM,
+see the section on xref:first_guest_boot_delay[Proxmox VE Node Management].
+
[[qm_qemu_agent]]
Qemu Guest Agent
If you have a cluster, you can migrate your VM to another host with
- qm migrate <vmid> <target>
+----
+# qm migrate <vmid> <target>
+----
There are generally two mechanisms for this
* The hosts have a working (and reliable) network connection.
* The target host must have the same or higher versions of the
{pve} packages. (It *might* work the other way, but this is never guaranteed)
+* The hosts have CPUs from the same vendor. (It *might* work otherwise, but this
+ is never guaranteed)
Offline Migration
~~~~~~~~~~~~~~~~~
[thumbnail="screenshot/gui-qemu-full-clone.png"]
VM installation is usually done using an installation media (CD-ROM)
-from the operation system vendor. Depending on the OS, this can be a
+from the operating system vendor. Depending on the OS, this can be a
time consuming task one might want to avoid.
An easy way to deploy many VMs of the same type is to copy an existing
e.g.:
----
- qm set VMID -vmgenid 1
- qm set VMID -vmgenid 00000000-0000-0000-0000-000000000000
+# qm set VMID -vmgenid 1
+# qm set VMID -vmgenid 00000000-0000-0000-0000-000000000000
----
NOTE: The initial addition of a 'vmgenid' device to an existing VM, may result
configuration with:
----
- qm set VMID -delete vmgenid
+# qm set VMID -delete vmgenid
----
The most prominent use case for 'vmgenid' are newer Microsoft Windows
VM name as read from the OVF manifest, and import the disks to the +local-lvm+
storage. You have to configure the network manually.
- qm importovf 999 WinDev1709Eval.ovf local-lvm
+----
+# qm importovf 999 WinDev1709Eval.ovf local-lvm
+----
The VM is ready to be started.
You can now create a new target VM for this image.
- qm create 600 --net0 virtio,bridge=vmbr0 --name vm600 --serial0 socket \
+----
+# qm create 600 --net0 virtio,bridge=vmbr0 --name vm600 --serial0 socket \
--bootdisk scsi0 --scsihw virtio-scsi-pci --ostype l26
+----
Add the disk image as +unused0+ to the VM, using the storage +pvedir+:
- qm importdisk 600 vm600.raw pvedir
+----
+# qm importdisk 600 vm600.raw pvedir
+----
Finally attach the unused disk to the SCSI controller of the VM:
- qm set 600 --scsi0 pvedir:600/vm-600-disk-1.raw
+----
+# qm set 600 --scsi0 pvedir:600/vm-600-disk-1.raw
+----
The VM is ready to be started.
You can add a hook script to VMs with the config property `hookscript`.
- qm set 100 -hookscript local:snippets/hookscript.pl
+----
+# qm set 100 --hookscript local:snippets/hookscript.pl
+----
It will be called during various phases of the guests lifetime.
For an example and documentation see the example script under
You can suspend a VM to disk with the GUI option `Hibernate` or with
- qm suspend ID --todisk
+----
+# qm suspend ID --todisk
+----
That means that the current content of the memory will be saved onto disk
and the VM gets stopped. On the next start, the memory content will be
Using an iso file uploaded on the 'local' storage, create a VM
with a 4 GB IDE disk on the 'local-lvm' storage
- qm create 300 -ide0 local-lvm:4 -net0 e1000 -cdrom local:iso/proxmox-mailgateway_2.1.iso
+----
+# qm create 300 -ide0 local-lvm:4 -net0 e1000 -cdrom local:iso/proxmox-mailgateway_2.1.iso
+----
Start the new VM
- qm start 300
+----
+# qm start 300
+----
Send a shutdown request, then wait until the VM is stopped.
- qm shutdown 300 && qm wait 300
+----
+# qm shutdown 300 && qm wait 300
+----
Same as above, but only wait for 40 seconds.
- qm shutdown 300 && qm wait 300 -timeout 40
+----
+# qm shutdown 300 && qm wait 300 -timeout 40
+----
Destroying a VM always removes it from Access Control Lists and it always
removes the firewall configuration of the VM. You have to activate
'--purge', if you want to additionally remove the VM from replication jobs,
backup jobs and HA resource configurations.
- qm destroy 300 --purge
+----
+# qm destroy 300 --purge
+----
+Move a disk image to a different storage.
+
+----
+# qm move-disk 300 scsi0 other-storage
+----
+
+Reassign a disk image to a different VM. This will remove the disk `scsi1` from
+the source VM and attaches it as `scsi3` to the target VM. In the background
+the disk image is being renamed so that the name matches the new owner.
+
+----
+# qm move-disk 300 scsi1 --target-vmid 400 --target-disk scsi3
+----
[[qm_configuration]]
prevent incompatible concurrent actions on the affected VMs. Sometimes
you need to remove such a lock manually (e.g., after a power failure).
- qm unlock <vmid>
+----
+# qm unlock <vmid>
+----
CAUTION: Only do that if you are sure the action which set the lock is
no longer running.
projects / pve-docs.git / blobdiff