]>
Commit | Line | Data |
---|---|---|
25034616 KO |
1 | PCI SR/IOV EMULATION SUPPORT |
2 | ============================ | |
3 | ||
4 | Description | |
5 | =========== | |
6 | SR/IOV (Single Root I/O Virtualization) is an optional extended capability | |
7 | of a PCI Express device. It allows a single physical function (PF) to appear as multiple | |
8 | virtual functions (VFs) for the main purpose of eliminating software | |
9 | overhead in I/O from virtual machines. | |
10 | ||
9b765724 | 11 | QEMU now implements the basic common functionality to enable an emulated device |
2a3f8b33 | 12 | to support SR/IOV. |
25034616 KO |
13 | |
14 | Implementation | |
15 | ============== | |
16 | Implementing emulation of an SR/IOV capable device typically consists of | |
17 | implementing support for two types of device classes; the "normal" physical device | |
9b765724 | 18 | (PF) and the virtual device (VF). From QEMU's perspective, the VFs are just |
25034616 KO |
19 | like other devices, except that some of their properties are derived from |
20 | the PF. | |
21 | ||
22 | A virtual function is different from a physical function in that the BAR | |
23 | space for all VFs are defined by the BAR registers in the PFs SR/IOV | |
24 | capability. All VFs have the same BARs and BAR sizes. | |
25 | ||
26 | Accesses to these virtual BARs then is computed as | |
27 | ||
28 | <VF BAR start> + <VF number> * <BAR sz> + <offset> | |
29 | ||
30 | From our emulation perspective this means that there is a separate call for | |
31 | setting up a BAR for a VF. | |
32 | ||
33 | 1) To enable SR/IOV support in the PF, it must be a PCI Express device so | |
34 | you would need to add a PCI Express capability in the normal PCI | |
35 | capability list. You might also want to add an ARI (Alternative | |
36 | Routing-ID Interpretation) capability to indicate that your device | |
37 | supports functions beyond it's "own" function space (0-7), | |
38 | which is necessary to support more than 7 functions, or | |
39 | if functions extends beyond offset 7 because they are placed at an | |
40 | offset > 1 or have stride > 1. | |
41 | ||
42 | ... | |
43 | #include "hw/pci/pcie.h" | |
44 | #include "hw/pci/pcie_sriov.h" | |
45 | ||
46 | pci_your_pf_dev_realize( ... ) | |
47 | { | |
48 | ... | |
49 | int ret = pcie_endpoint_cap_init(d, 0x70); | |
50 | ... | |
445416e3 | 51 | pcie_ari_init(d, 0x100); |
25034616 KO |
52 | ... |
53 | ||
54 | /* Add and initialize the SR/IOV capability */ | |
55 | pcie_sriov_pf_init(d, 0x200, "your_virtual_dev", | |
56 | vf_devid, initial_vfs, total_vfs, | |
57 | fun_offset, stride); | |
58 | ||
59 | /* Set up individual VF BARs (parameters as for normal BARs) */ | |
60 | pcie_sriov_pf_init_vf_bar( ... ) | |
61 | ... | |
62 | } | |
63 | ||
64 | For cleanup, you simply call: | |
65 | ||
66 | pcie_sriov_pf_exit(device); | |
67 | ||
68 | which will delete all the virtual functions and associated resources. | |
69 | ||
70 | 2) Similarly in the implementation of the virtual function, you need to | |
71 | make it a PCI Express device and add a similar set of capabilities | |
72 | except for the SR/IOV capability. Then you need to set up the VF BARs as | |
73 | subregions of the PFs SR/IOV VF BARs by calling | |
74 | pcie_sriov_vf_register_bar() instead of the normal pci_register_bar() call: | |
75 | ||
76 | pci_your_vf_dev_realize( ... ) | |
77 | { | |
78 | ... | |
79 | int ret = pcie_endpoint_cap_init(d, 0x60); | |
80 | ... | |
445416e3 | 81 | pcie_ari_init(d, 0x100); |
25034616 KO |
82 | ... |
83 | memory_region_init(mr, ... ) | |
84 | pcie_sriov_vf_register_bar(d, bar_nr, mr); | |
85 | ... | |
86 | } | |
87 | ||
88 | Testing on Linux guest | |
89 | ====================== | |
90 | The easiest is if your device driver supports sysfs based SR/IOV | |
91 | enabling. Support for this was added in kernel v.3.8, so not all drivers | |
92 | support it yet. | |
93 | ||
94 | To enable 4 VFs for a device at 01:00.0: | |
95 | ||
96 | modprobe yourdriver | |
97 | echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs | |
98 | ||
99 | You should now see 4 VFs with lspci. | |
100 | To turn SR/IOV off again - the standard requires you to turn it off before you can enable | |
101 | another VF count, and the emulation enforces this: | |
102 | ||
103 | echo 0 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs | |
104 | ||
105 | Older drivers typically provide a max_vfs module parameter | |
106 | to enable it at load time: | |
107 | ||
108 | modprobe yourdriver max_vfs=4 | |
109 | ||
110 | To disable the VFs again then, you simply have to unload the driver: | |
111 | ||
112 | rmmod yourdriver |