X-Git-Url: https://git.proxmox.com/?p=pve-docs.git;a=blobdiff_plain;f=qm.adoc;h=b84de9ea9f8751ed69afd57aa7e9958dde167da9;hp=0d453c8bc5c94356b84f768b06e6c32802b3d74e;hb=43530f6fe44c20926717a95e02aa19400ad2409c;hpb=6e4c46c4cb08bcd49c4384c2b0817923b1d63585 diff --git a/qm.adoc b/qm.adoc index 0d453c8..b84de9e 100644 --- a/qm.adoc +++ b/qm.adoc @@ -117,16 +117,42 @@ OS Settings [thumbnail="screenshot/gui-create-vm-os.png"] -When creating a VM, setting the proper Operating System(OS) allows {pve} to -optimize some low level parameters. For instance Windows OS expect the BIOS -clock to use the local time, while Unix based OS expect the BIOS clock to have -the UTC time. +When creating a virtual machine (VM), setting the proper Operating System(OS) +allows {pve} to optimize some low level parameters. For instance Windows OS +expect the BIOS clock to use the local time, while Unix based OS expect the +BIOS clock to have the UTC time. +[[qm_system_settings]] +System Settings +~~~~~~~~~~~~~~~ + +On VM creation you can change some basic system components of the new VM. You +can specify which xref:qm_display[display type] you want to use. +[thumbnail="screenshot/gui-create-vm-system.png"] +Additionally, the xref:qm_hard_disk[SCSI controller] can be changed. +If you plan to install the QEMU Guest Agent, or if your selected ISO image +already ships and installs it automatically, you may want to tick the 'Qemu +Agent' box, which lets {pve} know that it can use its features to show some +more information, and complete some actions (for example, shutdown or +snapshots) more intelligently. + +{pve} allows to boot VMs with different firmware and machine types, namely +xref:qm_bios_and_uefi[SeaBIOS and OVMF]. In most cases you want to switch from +the default SeabBIOS to OVMF only if you plan to use +xref:qm_pci_passthrough[PCIe pass through]. A VMs 'Machine Type' defines the +hardware layout of the VM's virtual motherboard. You can choose between the +default https://en.wikipedia.org/wiki/Intel_440FX[Intel 440FX] or the +https://ark.intel.com/content/www/us/en/ark/products/31918/intel-82q35-graphics-and-memory-controller.html[Q35] +chipset, which also provides a virtual PCIe bus, and thus may be desired if +one wants to pass through PCIe hardware. [[qm_hard_disk]] Hard Disk ~~~~~~~~~ +[[qm_hard_disk_bus]] +Bus/Controller +^^^^^^^^^^^^^^ Qemu can emulate a number of storage controllers: * the *IDE* controller, has a design which goes back to the 1984 PC/AT disk @@ -159,6 +185,10 @@ is an older type of paravirtualized controller. It has been superseded by the VirtIO SCSI Controller, in terms of features. [thumbnail="screenshot/gui-create-vm-hard-disk.png"] + +[[qm_hard_disk_formats]] +Image Format +^^^^^^^^^^^^ On each controller you attach a number of emulated hard disks, which are backed by a file or a block device residing in the configured storage. The choice of a storage type will determine the format of the hard disk image. Storages which @@ -173,10 +203,13 @@ either the *raw disk image format* or the *QEMU image format*. format does not support thin provisioning or snapshots by itself, requiring cooperation from the storage layer for these tasks. It may, however, be up to 10% faster than the *QEMU image format*. footnote:[See this benchmark for details - http://events.linuxfoundation.org/sites/events/files/slides/CloudOpen2013_Khoa_Huynh_v3.pdf] + https://events.static.linuxfound.org/sites/events/files/slides/CloudOpen2013_Khoa_Huynh_v3.pdf] * the *VMware image format* only makes sense if you intend to import/export the disk image to other hypervisors. +[[qm_hard_disk_cache]] +Cache Mode +^^^^^^^^^^ Setting the *Cache* mode of the hard drive will impact how the host system will notify the guest systems of block write completions. The *No cache* default means that the guest system will be notified that a write is complete when each @@ -192,26 +225,36 @@ As of {pve} 5.0, replication requires the disk images to be on a storage of type `zfspool`, so adding a disk image to other storages when the VM has replication configured requires to skip replication for this disk image. +[[qm_hard_disk_discard]] +Trim/Discard +^^^^^^^^^^^^ If your storage supports _thin provisioning_ (see the storage chapter in the -{pve} guide), and your VM has a *SCSI* controller you can activate the *Discard* -option on the hard disks connected to that controller. With *Discard* enabled, -when the filesystem of a VM marks blocks as unused after removing files, the -emulated SCSI controller will relay this information to the storage, which will -then shrink the disk image accordingly. +{pve} guide), you can activate the *Discard* option on a drive. With *Discard* +set and a _TRIM_-enabled guest OS footnote:[TRIM, UNMAP, and discard +https://en.wikipedia.org/wiki/Trim_%28computing%29], when the VM's filesystem +marks blocks as unused after deleting files, the controller will relay this +information to the storage, which will then shrink the disk image accordingly. +For the guest to be able to issue _TRIM_ commands, you must enable the *Discard* +option on the drive. Some guest operating systems may also require the +*SSD Emulation* flag to be set. Note that *Discard* on *VirtIO Block* drives is +only supported on guests using Linux Kernel 5.0 or higher. If you would like a drive to be presented to the guest as a solid-state drive rather than a rotational hard disk, you can set the *SSD emulation* option on that drive. There is no requirement that the underlying storage actually be backed by SSDs; this feature can be used with physical media of any type. +Note that *SSD emulation* is not supported on *VirtIO Block* drives. + -.IO Thread +[[qm_hard_disk_iothread]] +IO Thread +^^^^^^^^^ The option *IO Thread* can only be used when using a disk with the *VirtIO* controller, or with the *SCSI* controller, when the emulated controller type is *VirtIO SCSI single*. With this enabled, Qemu creates one I/O thread per storage controller, -instead of a single thread for all I/O, so it increases performance when +rather than a single thread for all I/O. This can increase performance when multiple disks are used and each disk has its own storage controller. -Note that backups do not currently work with *IO Thread* enabled. [[qm_cpu]] @@ -240,8 +283,8 @@ is greater than the number of cores on the server (e.g., 4 VMs with each 4 cores on a machine with only 8 cores). In that case the host system will balance the Qemu execution threads between your server cores, just like if you were running a standard multithreaded application. However, {pve} will prevent -you from assigning more virtual CPU cores than physically available, as this will -only bring the performance down due to the cost of context switches. +you from starting VMs with more virtual CPU cores than physically available, as +this will only bring the performance down due to the cost of context switches. [[qm_cpu_resource_limits]] Resource Limits @@ -309,6 +352,17 @@ the kvm64 default. If you don’t care about live migration or have a homogeneou cluster where all nodes have the same CPU, set the CPU type to host, as in theory this will give your guests maximum performance. +Custom CPU Types +^^^^^^^^^^^^^^^^ + +You can specify custom CPU types with a configurable set of features. These are +maintained in the configuration file `/etc/pve/virtual-guest/cpu-models.conf` by +an administrator. See `man cpu-models.conf` for format details. + +Specified custom types can be selected by any user with the `Sys.Audit` +privilege on `/nodes`. When configuring a custom CPU type for a VM via the CLI +or API, the name needs to be prefixed with 'custom-'. + Meltdown / Spectre related CPU flags ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -432,7 +486,7 @@ will allow proper distribution of the VM resources on the host system. This option is also required to hot-plug cores or RAM in a VM. If the NUMA option is used, it is recommended to set the number of sockets to -the number of sockets of the host system. +the number of nodes of the host system. vCPU hot-plug ^^^^^^^^^^^^^ @@ -608,7 +662,8 @@ necessary footnote:[https://www.kraxel.org/blog/2014/10/qemu-using-cirrus-consid qemu: using cirrus considered harmful], e.g., if using Windows XP or earlier * *vmware*, is a VMWare SVGA-II compatible adapter. * *qxl*, is the QXL paravirtualized graphics card. Selecting this also -enables SPICE for the VM. +enables https://www.spice-space.org/[SPICE] (a remote viewer protocol) for the +VM. You can edit the amount of memory given to the virtual GPU, by setting the 'memory' option. This can enable higher resolutions inside the VM, @@ -671,8 +726,12 @@ BIOS and UEFI ~~~~~~~~~~~~~ In order to properly emulate a computer, QEMU needs to use a firmware. -By default QEMU uses *SeaBIOS* for this, which is an open-source, x86 BIOS -implementation. SeaBIOS is a good choice for most standard setups. +Which, on common PCs often known as BIOS or (U)EFI, is executed as one of the +first steps when booting a VM. It is responsible for doing basic hardware +initialization and for providing an interface to the firmware and hardware for +the operating system. By default QEMU uses *SeaBIOS* for this, which is an +open-source, x86 BIOS implementation. SeaBIOS is a good choice for most +standard setups. There are, however, some scenarios in which a BIOS is not a good firmware to boot from, e.g. if you want to do VGA passthrough. footnote:[Alex Williamson has a very good blog entry about this. @@ -698,6 +757,128 @@ you need to set the client resolution in the OVMF menu(which you can reach with a press of the ESC button during boot), or you have to choose SPICE as the display type. +[[qm_ivshmem]] +Inter-VM shared memory +~~~~~~~~~~~~~~~~~~~~~~ + +You can add an Inter-VM shared memory device (`ivshmem`), which allows one to +share memory between the host and a guest, or also between multiple guests. + +To add such a device, you can use `qm`: + + qm set -ivshmem size=32,name=foo + +Where the size is in MiB. The file will be located under +`/dev/shm/pve-shm-$name` (the default name is the vmid). + +NOTE: Currently the device will get deleted as soon as any VM using it got +shutdown or stopped. Open connections will still persist, but new connections +to the exact same device cannot be made anymore. + +A use case for such a device is the Looking Glass +footnote:[Looking Glass: https://looking-glass.io/] project, which enables high +performance, low-latency display mirroring between host and guest. + +[[qm_audio_device]] +Audio Device +~~~~~~~~~~~~ + +To add an audio device run the following command: + +---- +qm set -audio0 device= +---- + +Supported audio devices are: + +* `ich9-intel-hda`: Intel HD Audio Controller, emulates ICH9 +* `intel-hda`: Intel HD Audio Controller, emulates ICH6 +* `AC97`: Audio Codec '97, useful for older operating systems like Windows XP + +There are two backends available: + +* 'spice' +* 'none' + +The 'spice' backend can be used in combination with xref:qm_display[SPICE] while +the 'none' backend can be useful if an audio device is needed in the VM for some +software to work. To use the physical audio device of the host use device +passthrough (see xref:qm_pci_passthrough[PCI Passthrough] and +xref:qm_usb_passthrough[USB Passthrough]). Remote protocols like Microsoft’s RDP +have options to play sound. + + +[[qm_virtio_rng]] +VirtIO RNG +~~~~~~~~~~ + +A RNG (Random Number Generator) is a device providing entropy ('randomness') to +a system. A virtual hardware-RNG can be used to provide such entropy from the +host system to a guest VM. This helps to avoid entropy starvation problems in +the guest (a situation where not enough entropy is available and the system may +slow down or run into problems), especially during the guests boot process. + +To add a VirtIO-based emulated RNG, run the following command: + +---- +qm set -rng0 source=[,max_bytes=X,period=Y] +---- + +`source` specifies where entropy is read from on the host and has to be one of +the following: + +* `/dev/urandom`: Non-blocking kernel entropy pool (preferred) +* `/dev/random`: Blocking kernel pool (not recommended, can lead to entropy + starvation on the host system) +* `/dev/hwrng`: To pass through a hardware RNG attached to the host (if multiple + are available, the one selected in + `/sys/devices/virtual/misc/hw_random/rng_current` will be used) + +A limit can be specified via the `max_bytes` and `period` parameters, they are +read as `max_bytes` per `period` in milliseconds. However, it does not represent +a linear relationship: 1024B/1000ms would mean that up to 1 KiB of data becomes +available on a 1 second timer, not that 1 KiB is streamed to the guest over the +course of one second. Reducing the `period` can thus be used to inject entropy +into the guest at a faster rate. + +By default, the limit is set to 1024 bytes per 1000 ms (1 KiB/s). It is +recommended to always use a limiter to avoid guests using too many host +resources. If desired, a value of '0' for `max_bytes` can be used to disable +all limits. + +[[qm_bootorder]] +Device Boot Order +~~~~~~~~~~~~~~~~~ + +QEMU can tell the guest which devices it should boot from, and in which order. +This can be specified in the config via the `boot` property, e.g.: + +---- +boot: order=scsi0;net0;hostpci0 +---- + +[thumbnail="screenshot/gui-qemu-edit-bootorder.png"] + +This way, the guest would first attempt to boot from the disk `scsi0`, if that +fails, it would go on to attempt network boot from `net0`, and in case that +fails too, finally attempt to boot from a passed through PCIe device (seen as +disk in case of NVMe, otherwise tries to launch into an option ROM). + +On the GUI you can use a drag-and-drop editor to specify the boot order, and use +the checkbox to enable or disable certain devices for booting altogether. + +NOTE: If your guest uses multiple disks to boot the OS or load the bootloader, +all of them must be marked as 'bootable' (that is, they must have the checkbox +enabled or appear in the list in the config) for the guest to be able to boot. +This is because recent SeaBIOS and OVMF versions only initialize disks if they +are marked 'bootable'. + +In any case, even devices not appearing in the list or having the checkmark +disabled will still be available to the guest, once it's operating system has +booted and initialized them. The 'bootable' flag only affects the guest BIOS and +bootloader. + + [[qm_startup_and_shutdown]] Automatic Start and Shutdown of Virtual Machines ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -743,6 +924,127 @@ be enforced between virtual machines running on the same host, not cluster-wide. +[[qm_qemu_agent]] +Qemu Guest Agent +~~~~~~~~~~~~~~~~ + +The Qemu Guest Agent is a service which runs inside the VM, providing a +communication channel between the host and the guest. It is used to exchange +information and allows the host to issue commands to the guest. + +For example, the IP addresses in the VM summary panel are fetched via the guest +agent. + +Or when starting a backup, the guest is told via the guest agent to sync +outstanding writes via the 'fs-freeze' and 'fs-thaw' commands. + +For the guest agent to work properly the following steps must be taken: + +* install the agent in the guest and make sure it is running +* enable the communication via the agent in {pve} + +Install Guest Agent +^^^^^^^^^^^^^^^^^^^ + +For most Linux distributions, the guest agent is available. The package is +usually named `qemu-guest-agent`. + +For Windows, it can be installed from the +https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso[Fedora +VirtIO driver ISO]. + +Enable Guest Agent Communication +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Communication from {pve} with the guest agent can be enabled in the VM's +*Options* panel. A fresh start of the VM is necessary for the changes to take +effect. + +It is possible to enable the 'Run guest-trim' option. With this enabled, +{pve} will issue a trim command to the guest after the following +operations that have the potential to write out zeros to the storage: + +* moving a disk to another storage +* live migrating a VM to another node with local storage + +On a thin provisioned storage, this can help to free up unused space. + +Troubleshooting +^^^^^^^^^^^^^^^ + +.VM does not shut down + +Make sure the guest agent is installed and running. + +Once the guest agent is enabled, {pve} will send power commands like +'shutdown' via the guest agent. If the guest agent is not running, commands +cannot get executed properly and the shutdown command will run into a timeout. + +[[qm_spice_enhancements]] +SPICE Enhancements +~~~~~~~~~~~~~~~~~~ + +SPICE Enhancements are optional features that can improve the remote viewer +experience. + +To enable them via the GUI go to the *Options* panel of the virtual machine. Run +the following command to enable them via the CLI: + +---- +qm set -spice_enhancements foldersharing=1,videostreaming=all +---- + +NOTE: To use these features the <> of the virtual machine +must be set to SPICE (qxl). + +Folder Sharing +^^^^^^^^^^^^^^ + +Share a local folder with the guest. The `spice-webdavd` daemon needs to be +installed in the guest. It makes the shared folder available through a local +WebDAV server located at http://localhost:9843. + +For Windows guests the installer for the 'Spice WebDAV daemon' can be downloaded +from the +https://www.spice-space.org/download.html#windows-binaries[official SPICE website]. + +Most Linux distributions have a package called `spice-webdavd` that can be +installed. + +To share a folder in Virt-Viewer (Remote Viewer) go to 'File -> Preferences'. +Select the folder to share and then enable the checkbox. + +NOTE: Folder sharing currently only works in the Linux version of Virt-Viewer. + +CAUTION: Experimental! Currently this feature does not work reliably. + +Video Streaming +^^^^^^^^^^^^^^^ + +Fast refreshing areas are encoded into a video stream. Two options exist: + +* *all*: Any fast refreshing area will be encoded into a video stream. +* *filter*: Additional filters are used to decide if video streaming should be + used (currently only small window surfaces are skipped). + +A general recommendation if video streaming should be enabled and which option +to choose from cannot be given. Your mileage may vary depending on the specific +circumstances. + +Troubleshooting +^^^^^^^^^^^^^^^ + +.Shared folder does not show up + +Make sure the WebDAV service is enabled and running in the guest. On Windows it +is called 'Spice webdav proxy'. In Linux the name is 'spice-webdavd' but can be +different depending on the distribution. + +If the service is running, check the WebDAV server by opening +http://localhost:9843 in a browser in the guest. + +It can help to restart the SPICE session. + [[qm_migration]] Migration --------- @@ -824,7 +1126,7 @@ migrate a VM to a totally different storage. You can also change the disk image *Format* if the storage driver supports several formats. + -NOTE: A full clone need to read and copy all VM image data. This is +NOTE: A full clone needs to read and copy all VM image data. This is usually much slower than creating a linked clone. + @@ -835,7 +1137,7 @@ never includes any additional snapshots from the original VM. Linked Clone:: -Modern storage drivers supports a way to generate fast linked +Modern storage drivers support a way to generate fast linked clones. Such a clone is a writable copy whose initial contents are the same as the original data. Creating a linked clone is nearly instantaneous, and initially consumes no additional space. @@ -852,8 +1154,8 @@ can convert any VM into a read-only <>). Such templates can later be used to create linked clones efficiently. + -NOTE: You cannot delete the original template while linked clones -exists. +NOTE: You cannot delete an original template while linked clones +exist. + It is not possible to change the *Target storage* for linked clones, @@ -864,7 +1166,7 @@ The *Target node* option allows you to create the new VM on a different node. The only restriction is that the VM is on shared storage, and that storage is also available on the target node. -To avoid resource conflicts, all network interface MAC addresses gets +To avoid resource conflicts, all network interface MAC addresses get randomized, and we generate a new 'UUID' for the VM BIOS (smbios1) setting. @@ -1025,6 +1327,38 @@ ifndef::wiki[] include::qm-pci-passthrough.adoc[] endif::wiki[] +Hookscripts +----------- + +You can add a hook script to VMs with the config property `hookscript`. + + qm set 100 -hookscript local:snippets/hookscript.pl + +It will be called during various phases of the guests lifetime. +For an example and documentation see the example script under +`/usr/share/pve-docs/examples/guest-example-hookscript.pl`. + +[[qm_hibernate]] +Hibernation +----------- + +You can suspend a VM to disk with the GUI option `Hibernate` or with + + qm suspend ID --todisk + +That means that the current content of the memory will be saved onto disk +and the VM gets stopped. On the next start, the memory content will be +loaded and the VM can continue where it was left off. + +[[qm_vmstatestorage]] +.State storage selection +If no target storage for the memory is given, it will be automatically +chosen, the first of: + +1. The storage `vmstatestorage` from the VM config. +2. The first shared storage from any VM disk. +3. The first non-shared storage from any VM disk. +4. The storage `local` as a fallback. Managing Virtual Machines with `qm` ------------------------------------ @@ -1055,6 +1389,14 @@ Same as above, but only wait for 40 seconds. qm shutdown 300 && qm wait 300 -timeout 40 +Destroying a VM always removes it from Access Control Lists and it always +removes the firewall configuration of the VM. You have to activate +'--purge', if you want to additionally remove the VM from replication jobs, +backup jobs and HA resource configurations. + + qm destroy 300 --purge + + [[qm_configuration]] Configuration @@ -1070,12 +1412,12 @@ unique cluster wide. .Example VM Configuration ---- +boot: order=virtio0;net0 cores: 1 sockets: 1 memory: 512 name: webmail ostype: l26 -bootdisk: virtio0 net0: e1000=EE:D2:28:5F:B6:3E,bridge=vmbr0 virtio0: local:vm-100-disk-1,size=32G ---- @@ -1135,6 +1477,10 @@ There are a few snapshot related properties like `parent` and relationship between snapshots. `snaptime` is the snapshot creation time stamp (Unix epoch). +You can optionally save the memory of a running VM with the option `vmstate`. +For details about how the target storage gets chosen for the VM state, see +xref:qm_vmstatestorage[State storage selection] in the chapter +xref:qm_hibernate[Hibernation]. [[qm_options]] Options