]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | .. BSD LICENSE |
2 | Copyright(c) 2016 Intel Corporation. All rights reserved. | |
3 | All rights reserved. | |
4 | ||
5 | Redistribution and use in source and binary forms, with or without | |
6 | modification, are permitted provided that the following conditions | |
7 | are met: | |
8 | ||
9 | * Redistributions of source code must retain the above copyright | |
10 | notice, this list of conditions and the following disclaimer. | |
11 | * Redistributions in binary form must reproduce the above copyright | |
12 | notice, this list of conditions and the following disclaimer in | |
13 | the documentation and/or other materials provided with the | |
14 | distribution. | |
15 | * Neither the name of Intel Corporation nor the names of its | |
16 | contributors may be used to endorse or promote products derived | |
17 | from this software without specific prior written permission. | |
18 | ||
19 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | |
20 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | |
21 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | |
22 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | |
23 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | |
24 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | |
25 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | |
26 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | |
27 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
28 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | |
29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
30 | ||
31 | I40E Poll Mode Driver | |
32 | ====================== | |
33 | ||
34 | The I40E PMD (librte_pmd_i40e) provides poll mode driver support | |
35 | for the Intel X710/XL710/X722 10/40 Gbps family of adapters. | |
36 | ||
37 | ||
38 | Features | |
39 | -------- | |
40 | ||
41 | Features of the I40E PMD are: | |
42 | ||
43 | - Multiple queues for TX and RX | |
44 | - Receiver Side Scaling (RSS) | |
45 | - MAC/VLAN filtering | |
46 | - Packet type information | |
47 | - Flow director | |
48 | - Cloud filter | |
49 | - Checksum offload | |
50 | - VLAN/QinQ stripping and inserting | |
51 | - TSO offload | |
52 | - Promiscuous mode | |
53 | - Multicast mode | |
54 | - Port hardware statistics | |
55 | - Jumbo frames | |
56 | - Link state information | |
57 | - Link flow control | |
58 | - Mirror on port, VLAN and VSI | |
59 | - Interrupt mode for RX | |
60 | - Scattered and gather for TX and RX | |
61 | - Vector Poll mode driver | |
62 | - DCB | |
63 | - VMDQ | |
64 | - SR-IOV VF | |
65 | - Hot plug | |
66 | - IEEE1588/802.1AS timestamping | |
67 | ||
68 | ||
69 | Prerequisites | |
70 | ------------- | |
71 | ||
72 | - Identifying your adapter using `Intel Support | |
73 | <http://www.intel.com/support>`_ and get the latest NVM/FW images. | |
74 | ||
75 | - Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to setup the basic DPDK environment. | |
76 | ||
77 | - To get better performance on Intel platforms, please follow the "How to get best performance with NICs on Intel platforms" | |
78 | section of the :ref:`Getting Started Guide for Linux <linux_gsg>`. | |
79 | ||
80 | ||
81 | Pre-Installation Configuration | |
82 | ------------------------------ | |
83 | ||
84 | Config File Options | |
85 | ~~~~~~~~~~~~~~~~~~~ | |
86 | ||
87 | The following options can be modified in the ``config`` file. | |
88 | Please note that enabling debugging options may affect system performance. | |
89 | ||
90 | - ``CONFIG_RTE_LIBRTE_I40E_PMD`` (default ``y``) | |
91 | ||
92 | Toggle compilation of the ``librte_pmd_i40e`` driver. | |
93 | ||
94 | - ``CONFIG_RTE_LIBRTE_I40E_DEBUG_*`` (default ``n``) | |
95 | ||
96 | Toggle display of generic debugging messages. | |
97 | ||
98 | - ``CONFIG_RTE_LIBRTE_I40E_RX_ALLOW_BULK_ALLOC`` (default ``y``) | |
99 | ||
100 | Toggle bulk allocation for RX. | |
101 | ||
102 | - ``CONFIG_RTE_LIBRTE_I40E_INC_VECTOR`` (default ``n``) | |
103 | ||
104 | Toggle the use of Vector PMD instead of normal RX/TX path. | |
105 | To enable vPMD for RX, bulk allocation for Rx must be allowed. | |
106 | ||
107 | - ``CONFIG_RTE_LIBRTE_I40E_RX_OLFLAGS_ENABLE`` (default ``y``) | |
108 | ||
109 | Toggle to enable RX ``olflags``. | |
110 | This is only meaningful when Vector PMD is used. | |
111 | ||
112 | - ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC`` (default ``n``) | |
113 | ||
114 | Toggle to use a 16-byte RX descriptor, by default the RX descriptor is 32 byte. | |
115 | ||
116 | - ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF`` (default ``64``) | |
117 | ||
118 | Number of queues reserved for PF. | |
119 | ||
120 | - ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF`` (default ``4``) | |
121 | ||
122 | Number of queues reserved for each SR-IOV VF. | |
123 | ||
124 | - ``CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM`` (default ``4``) | |
125 | ||
126 | Number of queues reserved for each VMDQ Pool. | |
127 | ||
128 | - ``CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL`` (default ``-1``) | |
129 | ||
130 | Interrupt Throttling interval. | |
131 | ||
132 | ||
133 | Driver Compilation | |
134 | ~~~~~~~~~~~~~~~~~~ | |
135 | ||
136 | To compile the I40E PMD see :ref:`Getting Started Guide for Linux <linux_gsg>` or | |
137 | :ref:`Getting Started Guide for FreeBSD <freebsd_gsg>` depending on your platform. | |
138 | ||
139 | ||
140 | Linux | |
141 | ----- | |
142 | ||
143 | ||
144 | Running testpmd | |
145 | ~~~~~~~~~~~~~~~ | |
146 | ||
147 | This section demonstrates how to launch ``testpmd`` with Intel XL710/X710 | |
148 | devices managed by ``librte_pmd_i40e`` in the Linux operating system. | |
149 | ||
150 | #. Load ``igb_uio`` or ``vfio-pci`` driver: | |
151 | ||
152 | .. code-block:: console | |
153 | ||
154 | modprobe uio | |
155 | insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko | |
156 | ||
157 | or | |
158 | ||
159 | .. code-block:: console | |
160 | ||
161 | modprobe vfio-pci | |
162 | ||
163 | #. Bind the XL710/X710 adapters to ``igb_uio`` or ``vfio-pci`` loaded in the previous step: | |
164 | ||
165 | .. code-block:: console | |
166 | ||
167 | ./tools/dpdk-devbind.py --bind igb_uio 0000:83:00.0 | |
168 | ||
169 | Or setup VFIO permissions for regular users and then bind to ``vfio-pci``: | |
170 | ||
171 | .. code-block:: console | |
172 | ||
173 | ./tools/dpdk-devbind.py --bind vfio-pci 0000:83:00.0 | |
174 | ||
175 | #. Start ``testpmd`` with basic parameters: | |
176 | ||
177 | .. code-block:: console | |
178 | ||
179 | ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 83:00.0 -- -i | |
180 | ||
181 | Example output: | |
182 | ||
183 | .. code-block:: console | |
184 | ||
185 | ... | |
186 | EAL: PCI device 0000:83:00.0 on NUMA socket 1 | |
187 | EAL: probe driver: 8086:1572 rte_i40e_pmd | |
188 | EAL: PCI memory mapped at 0x7f7f80000000 | |
189 | EAL: PCI memory mapped at 0x7f7f80800000 | |
190 | PMD: eth_i40e_dev_init(): FW 5.0 API 1.5 NVM 05.00.02 eetrack 8000208a | |
191 | Interactive-mode selected | |
192 | Configuring Port 0 (socket 0) | |
193 | ... | |
194 | ||
195 | PMD: i40e_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are | |
196 | satisfied.Rx Burst Bulk Alloc function will be used on port=0, queue=0. | |
197 | ||
198 | ... | |
199 | Port 0: 68:05:CA:26:85:84 | |
200 | Checking link statuses... | |
201 | Port 0 Link Up - speed 10000 Mbps - full-duplex | |
202 | Done | |
203 | ||
204 | testpmd> | |
205 | ||
206 | ||
207 | SR-IOV: Prerequisites and sample Application Notes | |
208 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
209 | ||
210 | #. Load the kernel module: | |
211 | ||
212 | .. code-block:: console | |
213 | ||
214 | modprobe i40e | |
215 | ||
216 | Check the output in dmesg: | |
217 | ||
218 | .. code-block:: console | |
219 | ||
220 | i40e 0000:83:00.1 ens802f0: renamed from eth0 | |
221 | ||
222 | #. Bring up the PF ports: | |
223 | ||
224 | .. code-block:: console | |
225 | ||
226 | ifconfig ens802f0 up | |
227 | ||
228 | #. Create VF device(s): | |
229 | ||
230 | Echo the number of VFs to be created into the ``sriov_numvfs`` sysfs entry | |
231 | of the parent PF. | |
232 | ||
233 | Example: | |
234 | ||
235 | .. code-block:: console | |
236 | ||
237 | echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs | |
238 | ||
239 | ||
240 | #. Assign VF MAC address: | |
241 | ||
242 | Assign MAC address to the VF using iproute2 utility. The syntax is: | |
243 | ||
244 | .. code-block:: console | |
245 | ||
246 | ip link set <PF netdev id> vf <VF id> mac <macaddr> | |
247 | ||
248 | Example: | |
249 | ||
250 | .. code-block:: console | |
251 | ||
252 | ip link set ens802f0 vf 0 mac a0:b0:c0:d0:e0:f0 | |
253 | ||
254 | #. Assign VF to VM, and bring up the VM. | |
255 | Please see the documentation for the *I40E/IXGBE/IGB Virtual Function Driver*. | |
256 | ||
257 | ||
258 | Sample Application Notes | |
259 | ------------------------ | |
260 | ||
261 | Vlan filter | |
262 | ~~~~~~~~~~~ | |
263 | ||
264 | Vlan filter only works when Promiscuous mode is off. | |
265 | ||
266 | To start ``testpmd``, and add vlan 10 to port 0: | |
267 | ||
268 | .. code-block:: console | |
269 | ||
270 | ./app/testpmd -c ffff -n 4 -- -i --forward-mode=mac | |
271 | ... | |
272 | ||
273 | testpmd> set promisc 0 off | |
274 | testpmd> rx_vlan add 10 0 | |
275 | ||
276 | ||
277 | Flow Director | |
278 | ~~~~~~~~~~~~~ | |
279 | ||
280 | The Flow Director works in receive mode to identify specific flows or sets of flows and route them to specific queues. | |
281 | The Flow Director filters can match the different fields for different type of packet: flow type, specific input set per flow type and the flexible payload. | |
282 | ||
283 | The default input set of each flow type is:: | |
284 | ||
285 | ipv4-other : src_ip_address, dst_ip_address | |
286 | ipv4-frag : src_ip_address, dst_ip_address | |
287 | ipv4-tcp : src_ip_address, dst_ip_address, src_port, dst_port | |
288 | ipv4-udp : src_ip_address, dst_ip_address, src_port, dst_port | |
289 | ipv4-sctp : src_ip_address, dst_ip_address, src_port, dst_port, | |
290 | verification_tag | |
291 | ipv6-other : src_ip_address, dst_ip_address | |
292 | ipv6-frag : src_ip_address, dst_ip_address | |
293 | ipv6-tcp : src_ip_address, dst_ip_address, src_port, dst_port | |
294 | ipv6-udp : src_ip_address, dst_ip_address, src_port, dst_port | |
295 | ipv6-sctp : src_ip_address, dst_ip_address, src_port, dst_port, | |
296 | verification_tag | |
297 | l2_payload : ether_type | |
298 | ||
299 | The flex payload is selected from offset 0 to 15 of packet's payload by default, while it is masked out from matching. | |
300 | ||
301 | Start ``testpmd`` with ``--disable-rss`` and ``--pkt-filter-mode=perfect``: | |
302 | ||
303 | .. code-block:: console | |
304 | ||
305 | ./app/testpmd -c ffff -n 4 -- -i --disable-rss --pkt-filter-mode=perfect \ | |
306 | --rxq=8 --txq=8 --nb-cores=8 --nb-ports=1 | |
307 | ||
308 | Add a rule to direct ``ipv4-udp`` packet whose ``dst_ip=2.2.2.5, src_ip=2.2.2.3, src_port=32, dst_port=32`` to queue 1: | |
309 | ||
310 | .. code-block:: console | |
311 | ||
312 | testpmd> flow_director_filter 0 mode IP add flow ipv4-udp \ | |
313 | src 2.2.2.3 32 dst 2.2.2.5 32 vlan 0 flexbytes () \ | |
314 | fwd pf queue 1 fd_id 1 | |
315 | ||
316 | Check the flow director status: | |
317 | ||
318 | .. code-block:: console | |
319 | ||
320 | testpmd> show port fdir 0 | |
321 | ||
322 | ######################## FDIR infos for port 0 #################### | |
323 | MODE: PERFECT | |
324 | SUPPORTED FLOW TYPE: ipv4-frag ipv4-tcp ipv4-udp ipv4-sctp ipv4-other | |
325 | ipv6-frag ipv6-tcp ipv6-udp ipv6-sctp ipv6-other | |
326 | l2_payload | |
327 | FLEX PAYLOAD INFO: | |
328 | max_len: 16 payload_limit: 480 | |
329 | payload_unit: 2 payload_seg: 3 | |
330 | bitmask_unit: 2 bitmask_num: 2 | |
331 | MASK: | |
332 | vlan_tci: 0x0000, | |
333 | src_ipv4: 0x00000000, | |
334 | dst_ipv4: 0x00000000, | |
335 | src_port: 0x0000, | |
336 | dst_port: 0x0000 | |
337 | src_ipv6: 0x00000000,0x00000000,0x00000000,0x00000000, | |
338 | dst_ipv6: 0x00000000,0x00000000,0x00000000,0x00000000 | |
339 | FLEX PAYLOAD SRC OFFSET: | |
340 | L2_PAYLOAD: 0 1 2 3 4 5 6 ... | |
341 | L3_PAYLOAD: 0 1 2 3 4 5 6 ... | |
342 | L4_PAYLOAD: 0 1 2 3 4 5 6 ... | |
343 | FLEX MASK CFG: | |
344 | ipv4-udp: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
345 | ipv4-tcp: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
346 | ipv4-sctp: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
347 | ipv4-other: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
348 | ipv4-frag: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
349 | ipv6-udp: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
350 | ipv6-tcp: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
351 | ipv6-sctp: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
352 | ipv6-other: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
353 | ipv6-frag: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
354 | l2_payload: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | |
355 | guarant_count: 1 best_count: 0 | |
356 | guarant_space: 512 best_space: 7168 | |
357 | collision: 0 free: 0 | |
358 | maxhash: 0 maxlen: 0 | |
359 | add: 0 remove: 0 | |
360 | f_add: 0 f_remove: 0 | |
361 | ||
362 | ||
363 | Delete all flow director rules on a port: | |
364 | ||
365 | .. code-block:: console | |
366 | ||
367 | testpmd> flush_flow_director 0 | |
368 | ||
369 | Floating VEB | |
370 | ~~~~~~~~~~~~~ | |
371 | ||
372 | The Intel® Ethernet Controller X710 and XL710 Family support a feature called | |
373 | "Floating VEB". | |
374 | ||
375 | A Virtual Ethernet Bridge (VEB) is an IEEE Edge Virtual Bridging (EVB) term | |
376 | for functionality that allows local switching between virtual endpoints within | |
377 | a physical endpoint and also with an external bridge/network. | |
378 | ||
379 | A "Floating" VEB doesn't have an uplink connection to the outside world so all | |
380 | switching is done internally and remains within the host. As such, this | |
381 | feature provides security benefits. | |
382 | ||
383 | In addition, a Floating VEB overcomes a limitation of normal VEBs where they | |
384 | cannot forward packets when the physical link is down. Floating VEBs don't need | |
385 | to connect to the NIC port so they can still forward traffic from VF to VF | |
386 | even when the physical link is down. | |
387 | ||
388 | Therefore, with this feature enabled VFs can be limited to communicating with | |
389 | each other but not an outside network, and they can do so even when there is | |
390 | no physical uplink on the associated NIC port. | |
391 | ||
392 | To enable this feature, the user should pass a ``devargs`` parameter to the | |
393 | EAL, for example:: | |
394 | ||
395 | -w 84:00.0,enable_floating_veb=1 | |
396 | ||
397 | In this configuration the PMD will use the floating VEB feature for all the | |
398 | VFs created by this PF device. | |
399 | ||
400 | Alternatively, the user can specify which VFs need to connect to this floating | |
401 | VEB using the ``floating_veb_list`` argument:: | |
402 | ||
403 | -w 84:00.0,enable_floating_veb=1,floating_veb_list=1;3-4 | |
404 | ||
405 | In this example ``VF1``, ``VF3`` and ``VF4`` connect to the floating VEB, | |
406 | while other VFs connect to the normal VEB. | |
407 | ||
408 | The current implementation only supports one floating VEB and one regular | |
409 | VEB. VFs can connect to a floating VEB or a regular VEB according to the | |
410 | configuration passed on the EAL command line. | |
411 | ||
412 | The floating VEB functionality requires a NIC firmware version of 5.0 | |
413 | or greater. | |
414 | ||
415 | ||
416 | Limitations or Known issues | |
417 | --------------------------- | |
418 | ||
419 | MPLS packet classification on X710/XL710 | |
420 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
421 | ||
422 | For firmware versions prior to 5.0, MPLS packets are not recognized by the NIC. | |
423 | The L2 Payload flow type in flow director can be used to classify MPLS packet | |
424 | by using a command in testpmd like: | |
425 | ||
426 | testpmd> flow_director_filter 0 mode IP add flow l2_payload ether \ | |
427 | 0x8847 flexbytes () fwd pf queue <N> fd_id <M> | |
428 | ||
429 | With the NIC firmware version 5.0 or greater, some limited MPLS support | |
430 | is added: Native MPLS (MPLS in Ethernet) skip is implemented, while no | |
431 | new packet type, no classification or offload are possible. With this change, | |
432 | L2 Payload flow type in flow director cannot be used to classify MPLS packet | |
433 | as with previous firmware versions. Meanwhile, the Ethertype filter can be | |
434 | used to classify MPLS packet by using a command in testpmd like: | |
435 | ||
436 | testpmd> ethertype_filter 0 add mac_ignr 00:00:00:00:00:00 ethertype \ | |
437 | 0x8847 fwd queue <M> | |
438 | ||
439 | 16 Byte Descriptor cannot be used on DPDK VF | |
440 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
441 | ||
442 | If the Linux i40e kernel driver is used as host driver, while DPDK i40e PMD | |
443 | is used as the VF driver, DPDK cannot choose 16 byte receive descriptor. That | |
444 | is to say, user should keep ``CONFIG_RTE_LIBRTE_I40E_16BYTE_RX_DESC=n`` in | |
445 | config file. | |
446 | ||
447 | Link down with i40e kernel driver after DPDK application exist | |
448 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
449 | ||
450 | After DPDK application quit, and the device is bound back to Linux i40e | |
451 | kernel driver, the link cannot be up after ``ifconfig <dev> up``. | |
452 | To work around this issue, ``ethtool -s <dev> autoneg on`` should be | |
453 | set first and then the link can be brought up through ``ifconfig <dev> up``. | |
454 | ||
455 | NOTE: requires Linux kernel i40e driver version >= 1.4.X | |
456 | ||
457 | Receive packets with Ethertype 0x88A8 | |
458 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
459 | ||
460 | Due to the FW limitation, PF can receive packets with Ethertype 0x88A8 | |
461 | only when floating VEB is disabled. |