[[qm_pci_passthrough]]
PCI(e) Passthrough
------------------
+ifdef::wiki[]
+:pve-toplevel:
+endif::wiki[]
PCI(e) passthrough is a mechanism to give a virtual machine control over
a PCI device from the host. This can have some advantages over using
hardware, but even then, many modern system can support this.
Please refer to your hardware vendor to check if they support this feature
-under Linux for your specific setup
+under Linux for your specific setup.
Configuration
some configuration to enable PCI(e) passthrough.
-IOMMU
-+++++
+.IOMMU
-The IOMMU has to be activated on the kernel commandline. The easiest way is to
-enable trough grub. Edit `'/etc/default/grub'' and add the following to th
-'GRUB_CMDLINE_LINUX_DEFAULT' variable:
+The IOMMU has to be activated on the
+xref:sysboot_edit_kernel_cmdline[kernel commandline].
+
+The command line parameters are:
* for Intel CPUs:
+
amd_iommu=on
----
-To bring this change in effect, make sure you run:
-
-----
-# update-grub
-----
-Kernel Modules
-++++++++++++++
+.Kernel Modules
You have to make sure the following modules are loaded. This can be achieved by
adding them to `'/etc/modules''
# update-initramfs -u -k all
----
-Finish Configuration
-++++++++++++++++++++
+.Finish Configuration
Finally reboot to bring the changes into effect and check that it is indeed
enabled.
----
-# dmesg -e DMAR -e IOMMU -e AMD-Vi
+# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
----
should display that `IOMMU`, `Directed I/O` or `Interrupt Remapping` is
Please be aware that this option can make your system unstable.
====
+GPU Passthrough Notes
+^^^^^^^^^^^^^^^^^^^^^
+
+It is not possible to display the frame buffer of the GPU via NoVNC or SPICE on
+the {pve} web interface.
+
+When passing through a whole GPU or a vGPU and graphic output is wanted, one
+has to either physically connect a monitor to the card, or configure a remote
+desktop software (for example, VNC or RDP) inside the guest.
+
+If you want to use the GPU as a hardware accelerator, for example, for
+programs using OpenCL or CUDA, this is not required.
+
Host Device Passthrough
~~~~~~~~~~~~~~~~~~~~~~~
Host Configuration
^^^^^^^^^^^^^^^^^^
-In this case, the host cannot use the card. There are two methods to achieve
+In this case, the host must not use the card. There are two methods to achieve
this:
* pass the device IDs to the options of the 'vfio-pci' modules by adding
the vendor and device IDs obtained by:
+
----
-# lcpci -nn
+# lspci -nn
----
* blacklist the driver completely on the host, ensuring that it is free to bind
xref:qm_pci_passthrough_update_initramfs[update the `initramfs`] again and
reboot after that.
+.Verify Configuration
+
+To check if your changes were successful, you can use
+
+----
+# lspci -nnk
+----
+
+and check your device entry. If it says
+
+----
+Kernel driver in use: vfio-pci
+----
+
+or the 'in use' line is missing entirely, the device is ready to be used for
+passthrough.
+
[[qm_pci_passthrough_vm_config]]
VM Configuration
^^^^^^^^^^^^^^^^
# qm set VMID -hostpci0 00:02.0
----
-If your device has multiple functions, you can pass them through all together
-with the shortened syntax ``00:02`'
+If your device has multiple functions (e.g., ``00:02.0`' and ``00:02.1`' ),
+you can pass them through all together with the shortened syntax ``00:02`'
There are some options to which may be necessary, depending on the device
and guest OS:
* *romfile=<path>*, is an optional path to a ROM file for the device to use.
This is a relative path under */usr/share/kvm/*.
-Example
-+++++++
+.Example
An example of PCIe passthrough with a GPU set to primary:
----
+
To make this change persistent you can use the `sysfsutils` Debian package.
-After installation configure it via */etc/sysfs.conf* or a `FILE.conf' inf
+After installation configure it via */etc/sysfs.conf* or a `FILE.conf' in
*/etc/sysfs.d/*.
VM Configuration
to enable this feature in the BIOS/EFI first, or to use a specific PCI(e) port
for it to work. In doubt, consult the manual of the platform or contact its
vendor.
+
+Mediated Devices (vGPU, GVT-g)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Mediated devices are another method to reuse features and performance from
+physical hardware for virtualized hardware. These are found most common in
+virtualized GPU setups such as Intels GVT-g and Nvidias vGPUs used in their
+GRID technology.
+
+With this, a physical Card is able to create virtual cards, similar to SR-IOV.
+The difference is that mediated devices do not appear as PCI(e) devices in the
+host, and are such only suited for using in virtual machines.
+
+
+Host Configuration
+^^^^^^^^^^^^^^^^^^
+
+In general your card's driver must support that feature, otherwise it will
+not work. So please refer to your vendor for compatible drivers and how to
+configure them.
+
+Intels drivers for GVT-g are integrated in the Kernel and should work
+with 5th, 6th and 7th generation Intel Core Processors, as well as E3 v4, E3
+v5 and E3 v6 Xeon Processors.
+
+To enable it for Intel Graphics, you have to make sure to load the module
+'kvmgt' (for example via `/etc/modules`) and to enable it on the
+xref:sysboot_edit_kernel_cmdline[Kernel commandline] and add the following parameter:
+
+----
+ i915.enable_gvt=1
+----
+
+After that remember to
+xref:qm_pci_passthrough_update_initramfs[update the `initramfs`],
+and reboot your host.
+
+VM Configuration
+^^^^^^^^^^^^^^^^
+
+To use a mediated device, simply specify the `mdev` property on a `hostpciX`
+VM configuration option.
+
+You can get the supported devices via the 'sysfs'. For example, to list the
+supported types for the device '0000:00:02.0' you would simply execute:
+
+----
+# ls /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types
+----
+
+Each entry is a directory which contains the following important files:
+
+* 'available_instances' contains the amount of still available instances of
+this type, each 'mdev' use in a VM reduces this.
+* 'description' contains a short description about the capabilities of the type
+* 'create' is the endpoint to create such a device, {pve} does this
+automatically for you, if a 'hostpciX' option with `mdev` is configured.
+
+Example configuration with an `Intel GVT-g vGPU` (`Intel Skylake 6700k`):
+
+----
+# qm set VMID -hostpci0 00:02.0,mdev=i915-GVTg_V5_4
+----
+
+With this set, {pve} automatically creates such a device on VM start, and
+cleans it up again when the VM stops.
+
+ifdef::wiki[]
+
+See Also
+~~~~~~~~
+
+* link:/wiki/Pci_passthrough[PCI Passthrough Examples]
+
+endif::wiki[]