1. Introduction
================
-The doc proposes best practices on how to use PCI Express/PCI device
-in PCI Express based machines and explains the reasoning behind them.
+The doc proposes best practices on how to use PCI Express (PCIe) / PCI
+devices in PCI Express based machines and explains the reasoning behind
+them.
+
+Note that the PCIe features are available only when using the 'q35'
+machine type on x86 architecture and the 'virt' machine type on AArch64.
+Other machine types do not use PCIe at this time.
The following presentations accompany this document:
(1) Q35 overview.
- http://wiki.qemu.org/images/4/4e/Q35.pdf
+ https://wiki.qemu.org/images/4/4e/Q35.pdf
(2) A comparison between PCI and PCI Express technologies.
- http://wiki.qemu.org/images/f/f6/PCIvsPCIe.pdf
+ https://wiki.qemu.org/images/f/f6/PCIvsPCIe.pdf
Note: The usage examples are not intended to replace the full
documentation, please use QEMU help to retrieve all options.
strangely when PCI Express devices are integrated
with the Root Complex.
- (2) PCI Express Root Ports (ioh3420), for starting exclusively PCI Express
- hierarchies.
+ (2) PCI Express Root Ports (pcie-root-port), for starting exclusively
+ PCI Express hierarchies.
- (3) DMI-PCI Bridges (i82801b11-bridge), for starting legacy PCI
+ (3) PCI Express to PCI Bridge (pcie-pci-bridge), for starting legacy PCI
hierarchies.
(4) Extra Root Complexes (pxb-pcie), if multiple PCI Express Root Buses
pcie.0 bus
----------------------------------------------------------------------------
| | | |
- ----------- ------------------ ------------------ --------------
- | PCI Dev | | PCIe Root Port | | DMI-PCI Bridge | | pxb-pcie |
- ----------- ------------------ ------------------ --------------
+ ----------- ------------------ ------------------- --------------
+ | PCI Dev | | PCIe Root Port | | PCIe-PCI Bridge | | pxb-pcie |
+ ----------- ------------------ ------------------- --------------
2.1.1 To plug a device into pcie.0 as a Root Complex Integrated Endpoint use:
-device <dev>[,bus=pcie.0]
2.1.2 To expose a new PCI Express Root Bus use:
-device pxb-pcie,id=pcie.1,bus_nr=x[,numa_node=y][,addr=z]
- Only PCI Express Root Ports and DMI-PCI bridges can be connected
- to the pcie.1 bus:
- -device ioh3420,id=root_port1[,bus=pcie.1][,chassis=x][,slot=y][,addr=z] \
- -device i82801b11-bridge,id=dmi_pci_bridge1,bus=pcie.1
+ PCI Express Root Ports and PCI Express to PCI bridges can be
+ connected to the pcie.1 bus:
+ -device pcie-root-port,id=root_port1[,bus=pcie.1][,chassis=x][,slot=y][,addr=z] \
+ -device pcie-pci-bridge,id=pcie_pci_bridge1,bus=pcie.1
2.2 PCI Express only hierarchy
------------
2.2.1 Plugging a PCI Express device into a PCI Express Root Port:
- -device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z] \
+ -device pcie-root-port,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z] \
-device <dev>,bus=root_port1
2.2.2 Using multi-function PCI Express Root Ports:
- -device ioh3420,id=root_port1,multifunction=on,chassis=x,addr=z.0[,slot=y][,bus=pcie.0] \
- -device ioh3420,id=root_port2,chassis=x1,addr=z.1[,slot=y1][,bus=pcie.0] \
- -device ioh3420,id=root_port3,chassis=x2,addr=z.2[,slot=y2][,bus=pcie.0] \
+ -device pcie-root-port,id=root_port1,multifunction=on,chassis=x,addr=z.0[,slot=y][,bus=pcie.0] \
+ -device pcie-root-port,id=root_port2,chassis=x1,addr=z.1[,slot=y1][,bus=pcie.0] \
+ -device pcie-root-port,id=root_port3,chassis=x2,addr=z.2[,slot=y2][,bus=pcie.0] \
2.2.3 Plugging a PCI Express device into a Switch:
- -device ioh3420,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z] \
+ -device pcie-root-port,id=root_port1,chassis=x,slot=y[,bus=pcie.0][,addr=z] \
-device x3130-upstream,id=upstream_port1,bus=root_port1[,addr=x] \
-device xio3130-downstream,id=downstream_port1,bus=upstream_port1,chassis=x1,slot=y1[,addr=z1]] \
-device <dev>,bus=downstream_port1
Legacy PCI devices can be plugged into pcie.0 as Integrated Endpoints,
but, as mentioned in section 5, doing so means the legacy PCI
device in question will be incapable of hot-unplugging.
-Besides that use DMI-PCI Bridges (i82801b11-bridge) in combination
-with PCI-PCI Bridges (pci-bridge) to start PCI hierarchies.
+Besides that use PCI Express to PCI Bridges (pcie-pci-bridge) in
+combination with PCI-PCI Bridges (pci-bridge) to start PCI hierarchies.
-Prefer flat hierarchies. For most scenarios a single DMI-PCI Bridge
+Prefer flat hierarchies. For most scenarios a single PCI Express to PCI Bridge
(having 32 slots) and several PCI-PCI Bridges attached to it
(each supporting also 32 slots) will support hundreds of legacy devices.
-The recommendation is to populate one PCI-PCI Bridge under the DMI-PCI Bridge
-until is full and then plug a new PCI-PCI Bridge...
+The recommendation is to populate one PCI-PCI Bridge under the
+PCI Express to PCI Bridge until is full and then plug a new PCI-PCI Bridge...
pcie.0 bus
----------------------------------------------
| |
- ----------- ------------------
- | PCI Dev | | DMI-PCI BRIDGE |
- ---------- ------------------
+ ----------- -------------------
+ | PCI Dev | | PCIe-PCI Bridge |
+ ----------- -------------------
| |
------------------ ------------------
- | PCI-PCI Bridge | | PCI-PCI Bridge | ...
+ | PCI-PCI Bridge | | PCI-PCI Bridge |
------------------ ------------------
| |
----------- -----------
2.3.1 To plug a PCI device into pcie.0 as an Integrated Endpoint use:
-device <dev>[,bus=pcie.0]
2.3.2 Plugging a PCI device into a PCI-PCI Bridge:
- -device i82801b11-bridge,id=dmi_pci_bridge1[,bus=pcie.0] \
- -device pci-bridge,id=pci_bridge1,bus=dmi_pci_bridge1[,chassis_nr=x][,addr=y] \
+ -device pcie-pci-bridge,id=pcie_pci_bridge1[,bus=pcie.0] \
+ -device pci-bridge,id=pci_bridge1,bus=pcie_pci_bridge1[,chassis_nr=x][,addr=y] \
-device <dev>,bus=pci_bridge1[,addr=x]
Note that 'addr' cannot be 0 unless shpc=off parameter is passed to
- the PCI Bridge.
+ the PCI Bridge/PCI Express to PCI Bridge.
3. IO space issues
===================
cannot be hot-plugged/hot-unplugged:
(1) PCI Express Integrated Endpoints
(2) PCI Express Root Ports
- (3) DMI-PCI Bridges
+ (3) PCI Express to PCI Bridges
(4) pxb-pcie
Be aware that PCI Express Downstream Ports can't be hot-plugged into
an existing PCI Express Upstream Port.
-PCI devices can be hot-plugged into PCI-PCI Bridges. The PCI hot-plug is ACPI
-based and can work side by side with the PCI Express native hot-plug.
+PCI devices can be hot-plugged into PCI Express to PCI and PCI-PCI Bridges.
+The PCI hot-plug into PCI-PCI bridge is ACPI based, whereas hot-plug into
+PCI Express to PCI bridges is SHPC-based. They both can work side by side with
+the PCI Express native hot-plug.
PCI Express devices can be natively hot-plugged/hot-unplugged into/from
PCI Express Root Ports (and PCI Express Downstream Ports).
5.1 Planning for hot-plug:
(1) PCI hierarchy
Leave enough PCI-PCI Bridge slots empty or add one
- or more empty PCI-PCI Bridges to the DMI-PCI Bridge.
+ or more empty PCI-PCI Bridges to the PCI Express to PCI Bridge.
For each such PCI-PCI Bridge the Guest Firmware is expected to reserve
4K IO space and 2M MMIO range to be used for all devices behind it.
+ Appropriate PCI capability is designed, see pcie_pci_bridge.txt.
Because of the hard IO limit of around 10 PCI Bridges (~ 40K space)
per system don't use more than 9 PCI-PCI Bridges, leaving 4K for the
projects / mirror_qemu.git / blobdiff