the `kvm(1)` man page) all of them emulated in software. All these devices
are the exact software equivalent of existing hardware devices, and if the OS
running in the guest has the proper drivers it will use the devices as if it
-were running on real hardware. This allows QEMU to runs _unmodified_ operating
+were running on real hardware. This allows QEMU to run _unmodified_ operating
systems.
This however has a performance cost, as running in software what was meant to
controller, a paravirtualized network card, a paravirtualized serial port,
a paravirtualized SCSI controller, etc ...
-It is highly recommended to use the virtio devices whenever you can, as they
-provide a big performance improvement. Using the virtio generic disk controller
-versus an emulated IDE controller will double the sequential write throughput,
-as measured with `bonnie++(8)`. Using the virtio network interface can deliver
-up to three times the throughput of an emulated Intel E1000 network card, as
-measured with `iperf(1)`. footnote:[See this benchmark on the KVM wiki
-https://www.linux-kvm.org/page/Using_VirtIO_NIC]
+TIP: It is *highly recommended* to use the virtio devices whenever you can, as
+they provide a big performance improvement and are generally better maintained.
+Using the virtio generic disk controller versus an emulated IDE controller will
+double the sequential write throughput, as measured with `bonnie++(8)`. Using
+the virtio network interface can deliver up to three times the throughput of an
+emulated Intel E1000 network card, as measured with `iperf(1)`. footnote:[See
+this benchmark on the KVM wiki https://www.linux-kvm.org/page/Using_VirtIO_NIC]
[[qm_virtual_machines_settings]]
^^^^^^^^^^^^^^
QEMU can emulate a number of storage controllers:
+TIP: It is highly recommended to use the *VirtIO SCSI* or *VirtIO Block*
+controller for performance reasons and because they are better maintained.
+
* the *IDE* controller, has a design which goes back to the 1984 PC/AT disk
controller. Even if this controller has been superseded by recent designs,
each and every OS you can think of has support for it, making it a great choice
hardware, and can connect up to 14 storage devices. {pve} emulates by default a
LSI 53C895A controller.
+
-A SCSI controller of type _VirtIO SCSI_ is the recommended setting if you aim for
-performance and is automatically selected for newly created Linux VMs since
-{pve} 4.3. Linux distributions have support for this controller since 2012, and
-FreeBSD since 2014. For Windows OSes, you need to provide an extra iso
-containing the drivers during the installation.
+A SCSI controller of type _VirtIO SCSI single_ and enabling the
+xref:qm_hard_disk_iothread[IO Thread] setting for the attached disks is
+recommended if you aim for performance. This is the default for newly created
+Linux VMs since {pve} 7.3. Each disk will have its own _VirtIO SCSI_ controller,
+and QEMU will handle the disks IO in a dedicated thread. Linux distributions
+have support for this controller since 2012, and FreeBSD since 2014. For Windows
+OSes, you need to provide an extra ISO containing the drivers during the
+installation.
// https://pve.proxmox.com/wiki/Paravirtualized_Block_Drivers_for_Windows#During_windows_installation.
-If you aim at maximum performance, you can select a SCSI controller of type
-_VirtIO SCSI single_ which will allow you to select the *IO Thread* option.
-When selecting _VirtIO SCSI single_ QEMU will create a new controller for
-each disk, instead of adding all disks to the same controller.
* The *VirtIO Block* controller, often just called VirtIO or virtio-blk,
is an older type of paravirtualized controller. It has been superseded by the
[[qm_hard_disk_iothread]]
IO Thread
^^^^^^^^^
-The option *IO Thread* can only be used when using a disk with the
-*VirtIO* controller, or with the *SCSI* controller, when the emulated controller
- type is *VirtIO SCSI single*.
-With this enabled, QEMU creates one I/O thread per storage controller,
-rather than a single thread for all I/O. This can increase performance when
-multiple disks are used and each disk has its own storage controller.
-
+The option *IO Thread* can only be used when using a disk with the *VirtIO*
+controller, or with the *SCSI* controller, when the emulated controller type is
+*VirtIO SCSI single*. With *IO Thread* enabled, QEMU creates one I/O thread per
+storage controller rather than handling all I/O in the main event loop or vCPU
+threads. One benefit is better work distribution and utilization of the
+underlying storage. Another benefit is reduced latency (hangs) in the guest for
+very I/O-intensive host workloads, since neither the main thread nor a vCPU
+thread can be blocked by disk I/O.
[[qm_cpu]]
CPU
to your VM, you can passthrough a USB device from where your SPICE client is,
directly to the VM (for example an input device or hardware dongle).
+It is also possible to map devices on a cluster level, so that they can be
+properly used with HA and hardware changes are detected and non root users
+can configure them. See xref:resource_mapping[Resource Mapping]
+for details on that.
[[qm_bios_and_uefi]]
BIOS and UEFI
standard setups.
Some operating systems (such as Windows 11) may require use of an UEFI
-compatible implementation instead. In such cases, you must rather use *OVMF*,
+compatible implementation. In such cases, you must use *OVMF* instead,
which is an open-source UEFI implementation. footnote:[See the OVMF Project https://github.com/tianocore/tianocore.github.io/wiki/OVMF]
There are other scenarios in which the SeaBIOS may not be the ideal firmware to
https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso[Fedora
VirtIO driver ISO].
+[[qm_qga_enable]]
Enable Guest Agent Communication
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
*Options* panel. A fresh start of the VM is necessary for the changes to take
effect.
+[[qm_qga_auto_trim]]
+Automatic TRIM Using QGA
+^^^^^^^^^^^^^^^^^^^^^^^^
+
It is possible to enable the 'Run guest-trim' option. With this enabled,
{pve} will issue a trim command to the guest after the following
operations that have the potential to write out zeros to the storage:
On a thin provisioned storage, this can help to free up unused space.
+NOTE: There is a caveat with ext4 on Linux, because it uses an in-memory
+optimization to avoid issuing duplicate TRIM requests. Since the guest doesn't
+know about the change in the underlying storage, only the first guest-trim will
+run as expected. Subsequent ones, until the next reboot, will only consider
+parts of the filesystem that changed since then.
+
+[[qm_qga_fsfreeze]]
+Filesystem Freeze & Thaw on Backup
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+By default, guest filesystems are synced via the 'fs-freeze' QEMU Guest Agent
+Command when a backup is performed, to provide consistency.
+
+On Windows guests, some applications might handle consistent backups themselves
+by hooking into the Windows VSS (Volume Shadow Copy Service) layer, a
+'fs-freeze' then might interfere with that. For example, it has been observed
+that calling 'fs-freeze' with some SQL Servers triggers VSS to call the SQL
+Writer VSS module in a mode that breaks the SQL Server backup chain for
+differential backups.
+
+For such setups you can configure {pve} to not issue a freeze-and-thaw cycle on
+backup by setting the `freeze-fs-on-backup` QGA option to `0`. This can also be
+done via the GUI with the 'Freeze/thaw guest filesystems on backup for
+consistency' option.
+
+IMPORTANT: Disabling this option can potentially lead to backups with inconsistent
+filesystems and should therefore only be disabled if you know what you are
+doing.
+
Troubleshooting
^^^^^^^^^^^^^^^
3. The first non-shared storage from any VM disk.
4. The storage `local` as a fallback.
+[[resource_mapping]]
+Resource Mapping
+~~~~~~~~~~~~~~~~
+
+When using or referencing local resources (e.g. address of a pci device), using
+the raw address or id is sometimes problematic, for example:
+
+* when using HA, a different device with the same id or path may exist on the
+ target node, and if one is not careful when assigning such guests to HA
+ groups, the wrong device could be used, breaking configurations.
+
+* changing hardware can change ids and paths, so one would have to check all
+ assigned devices and see if the path or id is still correct.
+
+To handle this better, one can define cluster wide resource mappings, such that
+a resource has a cluster unique, user selected identifier which can correspond
+to different devices on different hosts. With this, HA won't start a guest with
+a wrong device, and hardware changes can be detected.
+
+Creating such a mapping can be done with the {pve} web GUI under `Datacenter`
+in the relevant tab in the `Resource Mappings` category, or on the cli with
+
+----
+# pvesh create /cluster/mapping/<type> <options>
+----
+
+Where `<type>` is the hardware type (currently either `pci` or `usb`) and
+`<options>` are the device mappings and other configuration parameters.
+
+Note that the options must include a map property with all identifying
+properties of that hardware, so that it's possible to verify the hardware did
+not change and the correct device is passed through.
+
+For example to add a PCI device as `device1` with the path `0000:01:00.0` that
+has the device id `0001` and the vendor id `0002` on the node `node1`, and
+`0000:02:00.0` on `node2` you can add it with:
+
+----
+# pvesh create /cluster/mapping/pci --id device1 \
+ --map node=node1,path=0000:01:00.0,id=0002:0001 \
+ --map node=node2,path=0000:02:00.0,id=0002:0001
+----
+
+You must repeat the `map` parameter for each node where that device should have
+a mapping (note that you can currently only map one USB device per node per
+mapping).
+
+Using the GUI makes this much easier, as the correct properties are
+automatically picked up and sent to the API.
+
+It's also possible for PCI devices to provide multiple devices per node with
+multiple map properties for the nodes. If such a device is assigned to a guest,
+the first free one will be used when the guest is started. The order of the
+paths given is also the order in which they are tried, so arbitrary allocation
+policies can be implemented.
+
+This is useful for devices with SR-IOV, since some times it is not important
+which exact virtual function is passed through.
+
+You can assign such a device to a guest either with the GUI or with
+
+----
+# qm set ID -hostpci0 <name>
+----
+
+for PCI devices, or
+
+----
+# qm set <vmid> -usb0 <name>
+----
+
+for USB devices.
+
+Where `<vmid>` is the guests id and `<name>` is the chosen name for the created
+mapping. All usual options for passing through the devices are allowed, such as
+`mdev`.
+
+To create mappings `Mapping.Modify` on `/mapping/<type>/<name>` is necessary
+(where `<type>` is the device type and `<name>` is the name of the mapping).
+
+To use these mappings, `Mapping.Use` on `/mapping/<type>/<name>` is necessary
+(in addition to the normal guest privileges to edit the configuration).
+
Managing Virtual Machines with `qm`
------------------------------------
projects / pve-docs.git / blobdiff