]>
Commit | Line | Data |
---|---|---|
11fdf7f2 TL |
1 | .. SPDX-License-Identifier: BSD-3-Clause |
2 | Copyright(c) 2010-2016 Intel Corporation. | |
7c673cae FG |
3 | |
4 | Vhost Sample Application | |
5 | ======================== | |
6 | ||
7 | The vhost sample application demonstrates integration of the Data Plane | |
8 | Development Kit (DPDK) with the Linux* KVM hypervisor by implementing the | |
9 | vhost-net offload API. The sample application performs simple packet | |
10 | switching between virtual machines based on Media Access Control (MAC) | |
11 | address or Virtual Local Area Network (VLAN) tag. The splitting of Ethernet | |
12 | traffic from an external switch is performed in hardware by the Virtual | |
13 | Machine Device Queues (VMDQ) and Data Center Bridging (DCB) features of | |
14 | the Intel® 82599 10 Gigabit Ethernet Controller. | |
15 | ||
16 | Testing steps | |
17 | ------------- | |
18 | ||
19 | This section shows the steps how to test a typical PVP case with this | |
20 | vhost-switch sample, whereas packets are received from the physical NIC | |
21 | port first and enqueued to the VM's Rx queue. Through the guest testpmd's | |
22 | default forwarding mode (io forward), those packets will be put into | |
23 | the Tx queue. The vhost-switch example, in turn, gets the packets and | |
24 | puts back to the same physical NIC port. | |
25 | ||
26 | Build | |
27 | ~~~~~ | |
28 | ||
11fdf7f2 | 29 | To compile the sample application see :doc:`compiling`. |
7c673cae | 30 | |
11fdf7f2 | 31 | The application is located in the ``vhost`` sub-directory. |
7c673cae | 32 | |
11fdf7f2 TL |
33 | .. note:: |
34 | In this example, you need build DPDK both on the host and inside guest. | |
7c673cae FG |
35 | |
36 | Start the vswitch example | |
37 | ~~~~~~~~~~~~~~~~~~~~~~~~~ | |
38 | ||
39 | .. code-block:: console | |
40 | ||
11fdf7f2 | 41 | ./vhost-switch -l 0-3 -n 4 --socket-mem 1024 \ |
7c673cae FG |
42 | -- --socket-file /tmp/sock0 --client \ |
43 | ... | |
44 | ||
45 | Check the `Parameters`_ section for the explanations on what do those | |
46 | parameters mean. | |
47 | ||
48 | .. _vhost_app_run_vm: | |
49 | ||
50 | Start the VM | |
51 | ~~~~~~~~~~~~ | |
52 | ||
53 | .. code-block:: console | |
54 | ||
55 | qemu-system-x86_64 -machine accel=kvm -cpu host \ | |
56 | -m $mem -object memory-backend-file,id=mem,size=$mem,mem-path=/dev/hugepages,share=on \ | |
57 | -mem-prealloc -numa node,memdev=mem \ | |
58 | \ | |
59 | -chardev socket,id=char1,path=/tmp/sock0,server \ | |
60 | -netdev type=vhost-user,id=hostnet1,chardev=char1 \ | |
61 | -device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:00:00:14 \ | |
62 | ... | |
63 | ||
64 | .. note:: | |
65 | For basic vhost-user support, QEMU 2.2 (or above) is required. For | |
66 | some specific features, a higher version might be need. Such as | |
67 | QEMU 2.7 (or above) for the reconnect feature. | |
68 | ||
69 | .. _vhost_app_run_dpdk_inside_guest: | |
70 | ||
71 | Run testpmd inside guest | |
72 | ~~~~~~~~~~~~~~~~~~~~~~~~ | |
73 | ||
74 | Make sure you have DPDK built inside the guest. Also make sure the | |
75 | corresponding virtio-net PCI device is bond to a uio driver, which | |
76 | could be done by: | |
77 | ||
78 | .. code-block:: console | |
79 | ||
80 | modprobe uio_pci_generic | |
9f95a23c | 81 | $RTE_SDK/usertools/dpdk-devbind.py -b uio_pci_generic 0000:00:04.0 |
7c673cae FG |
82 | |
83 | Then start testpmd for packet forwarding testing. | |
84 | ||
85 | .. code-block:: console | |
86 | ||
11fdf7f2 | 87 | ./x86_64-native-gcc/app/testpmd -l 0-1 -- -i |
7c673cae FG |
88 | > start tx_first |
89 | ||
90 | Inject packets | |
91 | -------------- | |
92 | ||
93 | While a virtio-net is connected to vhost-switch, a VLAN tag starts with | |
94 | 1000 is assigned to it. So make sure configure your packet generator | |
95 | with the right MAC and VLAN tag, you should be able to see following | |
96 | log from the vhost-switch console. It means you get it work:: | |
97 | ||
98 | VHOST_DATA: (0) mac 52:54:00:00:00:14 and vlan 1000 registered | |
99 | ||
100 | ||
101 | .. _vhost_app_parameters: | |
102 | ||
103 | Parameters | |
104 | ---------- | |
105 | ||
106 | **--socket-file path** | |
107 | Specifies the vhost-user socket file path. | |
108 | ||
109 | **--client** | |
110 | DPDK vhost-user will act as the client mode when such option is given. | |
111 | In the client mode, QEMU will create the socket file. Otherwise, DPDK | |
112 | will create it. Put simply, it's the server to create the socket file. | |
113 | ||
114 | ||
115 | **--vm2vm mode** | |
116 | The vm2vm parameter sets the mode of packet switching between guests in | |
117 | the host. | |
118 | ||
9f95a23c | 119 | - 0 disables vm2vm, implying that VM's packets will always go to the NIC port. |
7c673cae FG |
120 | - 1 means a normal mac lookup packet routing. |
121 | - 2 means hardware mode packet forwarding between guests, it allows packets | |
122 | go to the NIC port, hardware L2 switch will determine which guest the | |
123 | packet should forward to or need send to external, which bases on the | |
124 | packet destination MAC address and VLAN tag. | |
125 | ||
126 | **--mergeable 0|1** | |
127 | Set 0/1 to disable/enable the mergeable Rx feature. It's disabled by default. | |
128 | ||
129 | **--stats interval** | |
130 | The stats parameter controls the printing of virtio-net device statistics. | |
131 | The parameter specifies an interval (in unit of seconds) to print statistics, | |
132 | with an interval of 0 seconds disabling statistics. | |
133 | ||
134 | **--rx-retry 0|1** | |
135 | The rx-retry option enables/disables enqueue retries when the guests Rx queue | |
136 | is full. This feature resolves a packet loss that is observed at high data | |
137 | rates, by allowing it to delay and retry in the receive path. This option is | |
138 | enabled by default. | |
139 | ||
140 | **--rx-retry-num num** | |
141 | The rx-retry-num option specifies the number of retries on an Rx burst, it | |
142 | takes effect only when rx retry is enabled. The default value is 4. | |
143 | ||
144 | **--rx-retry-delay msec** | |
145 | The rx-retry-delay option specifies the timeout (in micro seconds) between | |
146 | retries on an RX burst, it takes effect only when rx retry is enabled. The | |
147 | default value is 15. | |
148 | ||
149 | **--dequeue-zero-copy** | |
11fdf7f2 | 150 | Dequeue zero copy will be enabled when this option is given. it is worth to |
9f95a23c | 151 | note that if NIC is bound to driver with iommu enabled, dequeue zero copy |
11fdf7f2 TL |
152 | cannot work at VM2NIC mode (vm2vm=0) due to currently we don't setup iommu |
153 | dma mapping for guest memory. | |
7c673cae FG |
154 | |
155 | **--vlan-strip 0|1** | |
156 | VLAN strip option is removed, because different NICs have different behaviors | |
157 | when disabling VLAN strip. Such feature, which heavily depends on hardware, | |
158 | should be removed from this example to reduce confusion. Now, VLAN strip is | |
159 | enabled and cannot be disabled. | |
160 | ||
11fdf7f2 TL |
161 | **--builtin-net-driver** |
162 | A very simple vhost-user net driver which demonstrates how to use the generic | |
163 | vhost APIs will be used when this option is given. It is disabled by default. | |
164 | ||
7c673cae FG |
165 | Common Issues |
166 | ------------- | |
167 | ||
168 | * QEMU fails to allocate memory on hugetlbfs, with an error like the | |
169 | following:: | |
170 | ||
171 | file_ram_alloc: can't mmap RAM pages: Cannot allocate memory | |
172 | ||
173 | When running QEMU the above error indicates that it has failed to allocate | |
174 | memory for the Virtual Machine on the hugetlbfs. This is typically due to | |
175 | insufficient hugepages being free to support the allocation request. The | |
176 | number of free hugepages can be checked as follows: | |
177 | ||
178 | .. code-block:: console | |
179 | ||
180 | cat /sys/kernel/mm/hugepages/hugepages-<pagesize>/nr_hugepages | |
181 | ||
182 | The command above indicates how many hugepages are free to support QEMU's | |
183 | allocation request. | |
184 | ||
11fdf7f2 | 185 | * Failed to build DPDK in VM |
7c673cae | 186 | |
11fdf7f2 | 187 | Make sure "-cpu host" QEMU option is given. |
7c673cae | 188 | |
11fdf7f2 | 189 | * Device start fails if NIC's max queues > the default number of 128 |
7c673cae | 190 | |
11fdf7f2 TL |
191 | mbuf pool size is dependent on the MAX_QUEUES configuration, if NIC's |
192 | max queue number is larger than 128, device start will fail due to | |
193 | insufficient mbuf. | |
7c673cae | 194 | |
11fdf7f2 TL |
195 | Change the default number to make it work as below, just set the number |
196 | according to the NIC's property. :: | |
7c673cae | 197 | |
11fdf7f2 TL |
198 | make EXTRA_CFLAGS="-DMAX_QUEUES=320" |
199 | ||
200 | * Option "builtin-net-driver" is incompatible with QEMU | |
201 | ||
202 | QEMU vhost net device start will fail if protocol feature is not negotiated. | |
203 | DPDK virtio-user pmd can be the replacement of QEMU. |