[thumbnail="screenshot/gui-create-vm-os.png"]
-When creating a VM, setting the proper Operating System(OS) allows {pve} to
-optimize some low level parameters. For instance Windows OS expect the BIOS
-clock to use the local time, while Unix based OS expect the BIOS clock to have
-the UTC time.
+When creating a virtual machine (VM), setting the proper Operating System(OS)
+allows {pve} to optimize some low level parameters. For instance Windows OS
+expect the BIOS clock to use the local time, while Unix based OS expect the
+BIOS clock to have the UTC time.
+[[qm_system_settings]]
+System Settings
+~~~~~~~~~~~~~~~
+
+On VM creation you can change some basic system components of the new VM. You
+can specify which xref:qm_display[display type] you want to use.
+[thumbnail="screenshot/gui-create-vm-system.png"]
+Additionally, the xref:qm_hard_disk[SCSI controller] can be changed.
+If you plan to install the QEMU Guest Agent, or if your selected ISO image
+already ships and installs it automatically, you may want to tick the 'Qemu
+Agent' box, which lets {pve} know that it can use its features to show some
+more information, and complete some actions (for example, shutdown or
+snapshots) more intelligently.
+
+{pve} allows to boot VMs with different firmware and machine types, namely
+xref:qm_bios_and_uefi[SeaBIOS and OVMF]. In most cases you want to switch from
+the default SeabBIOS to OVMF only if you plan to use
+xref:qm_pci_passthrough[PCIe pass through]. A VMs 'Machine Type' defines the
+hardware layout of the VM's virtual motherboard. You can choose between the
+default https://en.wikipedia.org/wiki/Intel_440FX[Intel 440FX] or the
+https://ark.intel.com/content/www/us/en/ark/products/31918/intel-82q35-graphics-and-memory-controller.html[Q35]
+chipset, which also provides a virtual PCIe bus, and thus may be desired if
+one want's to pass through PCIe hardware.
[[qm_hard_disk]]
Hard Disk
~~~~~~~~~
+[[qm_hard_disk_bus]]
+Bus/Controller
+^^^^^^^^^^^^^^
Qemu can emulate a number of storage controllers:
* the *IDE* controller, has a design which goes back to the 1984 PC/AT disk
VirtIO SCSI Controller, in terms of features.
[thumbnail="screenshot/gui-create-vm-hard-disk.png"]
+
+[[qm_hard_disk_formats]]
+Image Format
+^^^^^^^^^^^^
On each controller you attach a number of emulated hard disks, which are backed
by a file or a block device residing in the configured storage. The choice of
a storage type will determine the format of the hard disk image. Storages which
* the *VMware image format* only makes sense if you intend to import/export the
disk image to other hypervisors.
+[[qm_hard_disk_cache]]
+Cache Mode
+^^^^^^^^^^
Setting the *Cache* mode of the hard drive will impact how the host system will
notify the guest systems of block write completions. The *No cache* default
means that the guest system will be notified that a write is complete when each
`zfspool`, so adding a disk image to other storages when the VM has replication
configured requires to skip replication for this disk image.
+[[qm_hard_disk_discard]]
+Trim/Discard
+^^^^^^^^^^^^
If your storage supports _thin provisioning_ (see the storage chapter in the
-{pve} guide), and your VM has a *SCSI* controller you can activate the *Discard*
-option on the hard disks connected to that controller. With *Discard* enabled,
-when the filesystem of a VM marks blocks as unused after removing files, the
-emulated SCSI controller will relay this information to the storage, which will
-then shrink the disk image accordingly.
-
-.IO Thread
+{pve} guide), you can activate the *Discard* option on a drive. With *Discard*
+set and a _TRIM_-enabled guest OS footnote:[TRIM, UNMAP, and discard
+https://en.wikipedia.org/wiki/Trim_%28computing%29], when the VM's filesystem
+marks blocks as unused after deleting files, the controller will relay this
+information to the storage, which will then shrink the disk image accordingly.
+For the guest to be able to issue _TRIM_ commands, you must enable the *Discard*
+option on the drive. Some guest operating systems may also require the
+*SSD Emulation* flag to be set. Note that *Discard* on *VirtIO Block* drives is
+only supported on guests using Linux Kernel 5.0 or higher.
+
+If you would like a drive to be presented to the guest as a solid-state drive
+rather than a rotational hard disk, you can set the *SSD emulation* option on
+that drive. There is no requirement that the underlying storage actually be
+backed by SSDs; this feature can be used with physical media of any type.
+Note that *SSD emulation* is not supported on *VirtIO Block* drives.
+
+
+[[qm_hard_disk_iothread]]
+IO Thread
+^^^^^^^^^
The option *IO Thread* can only be used when using a disk with the
*VirtIO* controller, or with the *SCSI* controller, when the emulated controller
type is *VirtIO SCSI single*.
With this enabled, Qemu creates one I/O thread per storage controller,
-instead of a single thread for all I/O, so it increases performance when
-multiple disks are used and each disk has its own storage controller.
-Note that backups do not currently work with *IO Thread* enabled.
+instead of a single thread for all I/O, so it can increase performance when
+multiple isks are used and each disk has its own storage controller.
[[qm_cpu]]
cores on a machine with only 8 cores). In that case the host system will
balance the Qemu execution threads between your server cores, just like if you
were running a standard multithreaded application. However, {pve} will prevent
-you from assigning more virtual CPU cores than physically available, as this will
-only bring the performance down due to the cost of context switches.
+you from starting VMs with more virtual CPU cores than physically available, as
+this will only bring the performance down due to the cost of context switches.
[[qm_cpu_resource_limits]]
Resource Limits
Meltdown / Spectre related CPU flags
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-There are two CPU flags related to the Meltdown and Spectre vulnerabilities
+There are several CPU flags related to the Meltdown and Spectre vulnerabilities
footnote:[Meltdown Attack https://meltdownattack.com/] which need to be set
manually unless the selected CPU type of your VM already enables them by default.
-The first, called 'pcid', helps to reduce the performance impact of the Meltdown
-mitigation called 'Kernel Page-Table Isolation (KPTI)', which effectively hides
-the Kernel memory from the user space. Without PCID, KPTI is quite an expensive
-mechanism footnote:[PCID is now a critical performance/security feature on x86
-https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU].
-
-The second CPU flag is called 'spec-ctrl', which allows an operating system to
-selectively disable or restrict speculative execution in order to limit the
-ability of attackers to exploit the Spectre vulnerability.
-
-There are two requirements that need to be fulfilled in order to use these two
+There are two requirements that need to be fulfilled in order to use these
CPU flags:
* The host CPU(s) must support the feature and propagate it to the guest's virtual CPU(s)
* The guest operating system must be updated to a version which mitigates the
attacks and is able to utilize the CPU feature
-In order to use 'spec-ctrl', your CPU or system vendor also needs to provide a
+Otherwise you need to set the desired CPU flag of the virtual CPU, either by
+editing the CPU options in the WebUI, or by setting the 'flags' property of the
+'cpu' option in the VM configuration file.
+
+For Spectre v1,v2,v4 fixes, your CPU or system vendor also needs to provide a
so-called ``microcode update'' footnote:[You can use `intel-microcode' /
`amd-microcode' from Debian non-free if your vendor does not provide such an
update. Note that not all affected CPUs can be updated to support spec-ctrl.]
for your CPU.
-To check if the {pve} host supports PCID, execute the following command as root:
+
+To check if the {pve} host is vulnerable, execute the following command as root:
----
-# grep ' pcid ' /proc/cpuinfo
+for f in /sys/devices/system/cpu/vulnerabilities/*; do echo "${f##*/} -" $(cat "$f"); done
----
-If this does not return empty your host's CPU has support for 'pcid'.
+A community script is also available to detect is the host is still vulnerable.
+footnote:[spectre-meltdown-checker https://meltdown.ovh/]
-To check if the {pve} host supports spec-ctrl, execute the following command as root:
+Intel processors
+^^^^^^^^^^^^^^^^
+* 'pcid'
++
+This reduces the performance impact of the Meltdown (CVE-2017-5754) mitigation
+called 'Kernel Page-Table Isolation (KPTI)', which effectively hides
+the Kernel memory from the user space. Without PCID, KPTI is quite an expensive
+mechanism footnote:[PCID is now a critical performance/security feature on x86
+https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU].
++
+To check if the {pve} host supports PCID, execute the following command as root:
++
----
-# grep ' spec_ctrl ' /proc/cpuinfo
+# grep ' pcid ' /proc/cpuinfo
----
++
+If this does not return empty your host's CPU has support for 'pcid'.
-If this does not return empty your host's CPU has support for 'spec-ctrl'.
+* 'spec-ctrl'
++
+Required to enable the Spectre v1 (CVE-2017-5753) and Spectre v2 (CVE-2017-5715) fix,
+in cases where retpolines are not sufficient.
+Included by default in Intel CPU models with -IBRS suffix.
+Must be explicitly turned on for Intel CPU models without -IBRS suffix.
+Requires an updated host CPU microcode (intel-microcode >= 20180425).
++
+* 'ssbd'
++
+Required to enable the Spectre V4 (CVE-2018-3639) fix. Not included by default in any Intel CPU model.
+Must be explicitly turned on for all Intel CPU models.
+Requires an updated host CPU microcode(intel-microcode >= 20180703).
-If you use `host' or another CPU type which enables the desired flags by
-default, and you updated your guest OS to make use of the associated CPU
-features, you're already set.
-Otherwise you need to set the desired CPU flag of the virtual CPU, either by
-editing the CPU options in the WebUI, or by setting the 'flags' property of the
-'cpu' option in the VM configuration file.
+AMD processors
+^^^^^^^^^^^^^^
+
+* 'ibpb'
++
+Required to enable the Spectre v1 (CVE-2017-5753) and Spectre v2 (CVE-2017-5715) fix,
+in cases where retpolines are not sufficient.
+Included by default in AMD CPU models with -IBPB suffix.
+Must be explicitly turned on for AMD CPU models without -IBPB suffix.
+Requires the host CPU microcode to support this feature before it can be used for guest CPUs.
+
+
+
+* 'virt-ssbd'
++
+Required to enable the Spectre v4 (CVE-2018-3639) fix.
+Not included by default in any AMD CPU model.
+Must be explicitly turned on for all AMD CPU models.
+This should be provided to guests, even if amd-ssbd is also provided, for maximum guest compatibility.
+Note that this must be explicitly enabled when when using the "host" cpu model,
+because this is a virtual feature which does not exist in the physical CPUs.
+
+
+* 'amd-ssbd'
++
+Required to enable the Spectre v4 (CVE-2018-3639) fix.
+Not included by default in any AMD CPU model. Must be explicitly turned on for all AMD CPU models.
+This provides higher performance than virt-ssbd, therefore a host supporting this should always expose this to guests if possible.
+virt-ssbd should none the less also be exposed for maximum guest compatibility as some kernels only know about virt-ssbd.
+
+
+* 'amd-no-ssb'
++
+Recommended to indicate the host is not vulnerable to Spectre V4 (CVE-2018-3639).
+Not included by default in any AMD CPU model.
+Future hardware generations of CPU will not be vulnerable to CVE-2018-3639,
+and thus the guest should be told not to enable its mitigations, by exposing amd-no-ssb.
+This is mutually exclusive with virt-ssbd and amd-ssbd.
+
NUMA
^^^^
This option is also required to hot-plug cores or RAM in a VM.
If the NUMA option is used, it is recommended to set the number of sockets to
-the number of sockets of the host system.
+the number of nodes of the host system.
vCPU hot-plug
^^^^^^^^^^^^^
process a great number of incoming connections, such as when the VM is running
as a router, reverse proxy or a busy HTTP server doing long polling.
+[[qm_display]]
+Display
+~~~~~~~
+
+QEMU can virtualize a few types of VGA hardware. Some examples are:
+
+* *std*, the default, emulates a card with Bochs VBE extensions.
+* *cirrus*, this was once the default, it emulates a very old hardware module
+with all its problems. This display type should only be used if really
+necessary footnote:[https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-considered-harmful/
+qemu: using cirrus considered harmful], e.g., if using Windows XP or earlier
+* *vmware*, is a VMWare SVGA-II compatible adapter.
+* *qxl*, is the QXL paravirtualized graphics card. Selecting this also
+enables https://www.spice-space.org/[SPICE] (a remote viewer protocol) for the
+VM.
+
+You can edit the amount of memory given to the virtual GPU, by setting
+the 'memory' option. This can enable higher resolutions inside the VM,
+especially with SPICE/QXL.
+
+As the memory is reserved by display device, selecting Multi-Monitor mode
+for SPICE (e.g., `qxl2` for dual monitors) has some implications:
+
+* Windows needs a device for each monitor, so if your 'ostype' is some
+version of Windows, {pve} gives the VM an extra device per monitor.
+Each device gets the specified amount of memory.
+
+* Linux VMs, can always enable more virtual monitors, but selecting
+a Multi-Monitor mode multiplies the memory given to the device with
+the number of monitors.
+
+Selecting `serialX` as display 'type' disables the VGA output, and redirects
+the Web Console to the selected serial port. A configured display 'memory'
+setting will be ignored in that case.
[[qm_usb_passthrough]]
USB Passthrough
~~~~~~~~~~~~~
In order to properly emulate a computer, QEMU needs to use a firmware.
-By default QEMU uses *SeaBIOS* for this, which is an open-source, x86 BIOS
-implementation. SeaBIOS is a good choice for most standard setups.
+Which, on common PCs often known as BIOS or (U)EFI, is executed as one of the
+first steps when booting a VM. It is responsible for doing basic hardware
+initialization and for providing an interface to the firmware and hardware for
+the operating system. By default QEMU uses *SeaBIOS* for this, which is an
+open-source, x86 BIOS implementation. SeaBIOS is a good choice for most
+standard setups.
There are, however, some scenarios in which a BIOS is not a good firmware
to boot from, e.g. if you want to do VGA passthrough. footnote:[Alex Williamson has a very good blog entry about this.
with a press of the ESC button during boot), or you have to choose
SPICE as the display type.
+[[qm_ivshmem]]
+Inter-VM shared memory
+~~~~~~~~~~~~~~~~~~~~~~
+
+You can add an Inter-VM shared memory device (`ivshmem`), which allows one to
+share memory between the host and a guest, or also between multiple guests.
+
+To add such a device, you can use `qm`:
+
+ qm set <vmid> -ivshmem size=32,name=foo
+
+Where the size is in MiB. The file will be located under
+`/dev/shm/pve-shm-$name` (the default name is the vmid).
+
+NOTE: Currently the device will get deleted as soon as any VM using it got
+shutdown or stopped. Open connections will still persist, but new connections
+to the exact same device cannot be made anymore.
+
+A use case for such a device is the Looking Glass
+footnote:[Looking Glass: https://looking-glass.hostfission.com/] project,
+which enables high performance, low-latency display mirroring between
+host and guest.
+
+[[qm_audio_device]]
+Audio Device
+~~~~~~~~~~~~
+
+To add an audio device run the following command:
+
+----
+qm set <vmid> -audio0 device=<device>
+----
+
+Supported audio devices are:
+
+* `ich9-intel-hda`: Intel HD Audio Controller, emulates ICH9
+* `intel-hda`: Intel HD Audio Controller, emulates ICH6
+* `AC97`: Audio Codec '97, useful for older operating systems like Windows XP
+
+NOTE: The audio device works only in combination with SPICE. Remote protocols
+like Microsoft's RDP have options to play sound. To use the physical audio
+device of the host use device passthrough (see
+xref:qm_pci_passthrough[PCI Passthrough] and
+xref:qm_usb_passthrough[USB Passthrough]).
+
[[qm_startup_and_shutdown]]
Automatic Start and Shutdown of Virtual Machines
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
be enforced between virtual machines running on the same host, not
cluster-wide.
+[[qm_spice_enhancements]]
+SPICE Enhancements
+~~~~~~~~~~~~~~~~~~
+
+SPICE Enhancements are optional features that can improve the remote viewer
+experience.
+
+To enable them via the GUI go to the *Options* panel of the virtual machine. Run
+the following command to enable them via the CLI:
+
+----
+qm set <vmid> -spice_enhancements foldersharing=1,videostreaming=all
+----
+
+NOTE: To use these features the <<qm_display,*Display*>> of the virtual machine
+must be set to SPICE (qxl).
+
+Folder Sharing
+^^^^^^^^^^^^^^
+
+Share a local folder with the guest. The `spice-webdavd` daemon needs to be
+installed in the guest. It makes the shared folder available through a local
+WebDAV server located at http://localhost:9843.
+
+For Windows guests the installer for the 'Spice WebDAV daemon' can be downloaded
+from the
+https://www.spice-space.org/download.html#windows-binaries[official SPICE website].
+
+Most Linux distributions have a package called `spice-webdavd` that can be
+installed.
+
+To share a folder in Virt-Viewer (Remote Viewer) go to 'File -> Preferences'.
+Select the folder to share and then enable the checkbox.
+
+NOTE: Folder sharing currently only works in the Linux version of Virt-Viewer.
+
+CAUTION: Experimental! Currently this feature does not work reliably.
+
+Video Streaming
+^^^^^^^^^^^^^^^
+
+Fast refreshing areas are encoded into a video stream. Two options exist:
+
+* *all*: Any fast refreshing area will be encoded into a video stream.
+* *filter*: Additional filters are used to decide if video streaming should be
+ used (currently only small window surfaces are skipped).
+
+A general recommendation if video streaming should be enabled and which option
+to choose from cannot be given. Your mileage may vary depending on the specific
+circumstances.
+
+Troubleshooting
+^^^^^^^^^^^^^^^
+
+.Shared folder does not show up
+
+Make sure the WebDAV service is enabled and running in the guest. On Windows it
+is called 'Spice webdav proxy'. In Linux the name is 'spice-webdavd' but can be
+different depending on the distribution.
+
+If the service is running, check the WebDAV server by opening
+http://localhost:9843 in a browser in the guest.
+
+It can help to restart the SPICE session.
[[qm_migration]]
Migration
disk image *Format* if the storage driver supports several formats.
+
-NOTE: A full clone need to read and copy all VM image data. This is
+NOTE: A full clone needs to read and copy all VM image data. This is
usually much slower than creating a linked clone.
+
Linked Clone::
-Modern storage drivers supports a way to generate fast linked
+Modern storage drivers support a way to generate fast linked
clones. Such a clone is a writable copy whose initial contents are the
same as the original data. Creating a linked clone is nearly
instantaneous, and initially consumes no additional space.
templates can later be used to create linked clones efficiently.
+
-NOTE: You cannot delete the original template while linked clones
-exists.
+NOTE: You cannot delete an original template while linked clones
+exist.
+
It is not possible to change the *Target storage* for linked clones,
different node. The only restriction is that the VM is on shared
storage, and that storage is also available on the target node.
-To avoid resource conflicts, all network interface MAC addresses gets
+To avoid resource conflicts, all network interface MAC addresses get
randomized, and we generate a new 'UUID' for the VM BIOS (smbios1)
setting.
the disk images. If you want to change the template, create a linked
clone and modify that.
+VM Generation ID
+----------------
+
+{pve} supports Virtual Machine Generation ID ('vmgenid') footnote:[Official
+'vmgenid' Specification
+https://docs.microsoft.com/en-us/windows/desktop/hyperv_v2/virtual-machine-generation-identifier]
+for virtual machines.
+This can be used by the guest operating system to detect any event resulting
+in a time shift event, for example, restoring a backup or a snapshot rollback.
+
+When creating new VMs, a 'vmgenid' will be automatically generated and saved
+in its configuration file.
+
+To create and add a 'vmgenid' to an already existing VM one can pass the
+special value `1' to let {pve} autogenerate one or manually set the 'UUID'
+footnote:[Online GUID generator http://guid.one/] by using it as value,
+e.g.:
+
+----
+ qm set VMID -vmgenid 1
+ qm set VMID -vmgenid 00000000-0000-0000-0000-000000000000
+----
+
+NOTE: The initial addition of a 'vmgenid' device to an existing VM, may result
+in the same effects as a change on snapshot rollback, backup restore, etc., has
+as the VM can interpret this as generation change.
+
+In the rare case the 'vmgenid' mechanism is not wanted one can pass `0' for
+its value on VM creation, or retroactively delete the property in the
+configuration with:
+
+----
+ qm set VMID -delete vmgenid
+----
+
+The most prominent use case for 'vmgenid' are newer Microsoft Windows
+operating systems, which use it to avoid problems in time sensitive or
+replicate services (e.g., databases, domain controller
+footnote:[https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/get-started/virtual-dc/virtualized-domain-controller-architecture])
+on snapshot rollback, backup restore or a whole VM clone operation.
+
Importing Virtual Machines and disk images
------------------------------------------
Microsoft provides
https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/[Virtual Machines downloads]
- to get started with Windows development.We are going to use one of these
+ to get started with Windows development.We are going to use one of these
to demonstrate the OVF import feature.
Download the Virtual Machine zip
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-After getting informed about the user agreement, choose the _Windows 10
+After getting informed about the user agreement, choose the _Windows 10
Enterprise (Evaluation - Build)_ for the VMware platform, and download the zip.
Extract the disk image from the zip
Adding an external disk image to a Virtual Machine
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-You can also add an existing disk image to a VM, either coming from a
+You can also add an existing disk image to a VM, either coming from a
foreign hypervisor, or one that you created yourself.
Suppose you created a Debian/Ubuntu disk image with the 'vmdebootstrap' tool:
include::qm-cloud-init.adoc[]
endif::wiki[]
+ifndef::wiki[]
+include::qm-pci-passthrough.adoc[]
+endif::wiki[]
+
+Hookscripts
+-----------
+
+You can add a hook script to VMs with the config property `hookscript`.
+
+ qm set 100 -hookscript local:snippets/hookscript.pl
+It will be called during various phases of the guests lifetime.
+For an example and documentation see the example script under
+`/usr/share/pve-docs/examples/guest-example-hookscript.pl`.
Managing Virtual Machines with `qm`
------------------------------------
projects / pve-docs.git / blobdiff