]>
Commit | Line | Data |
---|---|---|
11fdf7f2 TL |
1 | .. SPDX-License-Identifier: BSD-3-Clause |
2 | Copyright(c) 2010-2014 Intel Corporation. | |
3 | ||
4 | Kernel NIC Interface Sample Application | |
5 | ======================================= | |
6 | ||
7 | The Kernel NIC Interface (KNI) is a DPDK control plane solution that | |
8 | allows userspace applications to exchange packets with the kernel networking stack. | |
9 | To accomplish this, DPDK userspace applications use an IOCTL call | |
10 | to request the creation of a KNI virtual device in the Linux* kernel. | |
11 | The IOCTL call provides interface information and the DPDK's physical address space, | |
12 | which is re-mapped into the kernel address space by the KNI kernel loadable module | |
13 | that saves the information to a virtual device context. | |
14 | The DPDK creates FIFO queues for packet ingress and egress | |
15 | to the kernel module for each device allocated. | |
16 | ||
17 | The KNI kernel loadable module is a standard net driver, | |
18 | which upon receiving the IOCTL call access the DPDK's FIFO queue to | |
19 | receive/transmit packets from/to the DPDK userspace application. | |
20 | The FIFO queues contain pointers to data packets in the DPDK. This: | |
21 | ||
22 | * Provides a faster mechanism to interface with the kernel net stack and eliminates system calls | |
23 | ||
24 | * Facilitates the DPDK using standard Linux* userspace net tools (tcpdump, ftp, and so on) | |
25 | ||
26 | * Eliminate the copy_to_user and copy_from_user operations on packets. | |
27 | ||
28 | The Kernel NIC Interface sample application is a simple example that demonstrates the use | |
29 | of the DPDK to create a path for packets to go through the Linux* kernel. | |
30 | This is done by creating one or more kernel net devices for each of the DPDK ports. | |
31 | The application allows the use of standard Linux tools (ethtool, ifconfig, tcpdump) with the DPDK ports and | |
32 | also the exchange of packets between the DPDK application and the Linux* kernel. | |
33 | ||
34 | Overview | |
35 | -------- | |
36 | ||
37 | The Kernel NIC Interface sample application uses two threads in user space for each physical NIC port being used, | |
38 | and allocates one or more KNI device for each physical NIC port with kernel module's support. | |
39 | For a physical NIC port, one thread reads from the port and writes to KNI devices, | |
40 | and another thread reads from KNI devices and writes the data unmodified to the physical NIC port. | |
41 | It is recommended to configure one KNI device for each physical NIC port. | |
42 | If configured with more than one KNI devices for a physical NIC port, | |
43 | it is just for performance testing, or it can work together with VMDq support in future. | |
44 | ||
45 | The packet flow through the Kernel NIC Interface application is as shown in the following figure. | |
46 | ||
47 | .. _figure_kernel_nic: | |
48 | ||
49 | .. figure:: img/kernel_nic.* | |
50 | ||
51 | Kernel NIC Application Packet Flow | |
52 | ||
53 | Compiling the Application | |
54 | ------------------------- | |
55 | ||
56 | To compile the sample application see :doc:`compiling`. | |
57 | ||
58 | The application is located in the ``kni`` sub-directory. | |
59 | ||
60 | .. note:: | |
61 | ||
62 | This application is intended as a linuxapp only. | |
63 | ||
64 | Loading the Kernel Module | |
65 | ------------------------- | |
66 | ||
67 | Loading the KNI kernel module without any parameter is the typical way a DPDK application | |
68 | gets packets into and out of the kernel net stack. | |
69 | This way, only one kernel thread is created for all KNI devices for packet receiving in kernel side: | |
70 | ||
71 | .. code-block:: console | |
72 | ||
73 | #insmod rte_kni.ko | |
74 | ||
75 | Pinning the kernel thread to a specific core can be done using a taskset command such as following: | |
76 | ||
77 | .. code-block:: console | |
78 | ||
79 | #taskset -p 100000 `pgrep --fl kni_thread | awk '{print $1}'` | |
80 | ||
81 | This command line tries to pin the specific kni_thread on the 20th lcore (lcore numbering starts at 0), | |
82 | which means it needs to check if that lcore is available on the board. | |
83 | This command must be sent after the application has been launched, as insmod does not start the kni thread. | |
84 | ||
85 | For optimum performance, | |
86 | the lcore in the mask must be selected to be on the same socket as the lcores used in the KNI application. | |
87 | ||
88 | To provide flexibility of performance, the kernel module of the KNI, | |
89 | located in the kmod sub-directory of the DPDK target directory, | |
90 | can be loaded with parameter of kthread_mode as follows: | |
91 | ||
92 | * #insmod rte_kni.ko kthread_mode=single | |
93 | ||
94 | This mode will create only one kernel thread for all KNI devices for packet receiving in kernel side. | |
95 | By default, it is in this single kernel thread mode. | |
96 | It can set core affinity for this kernel thread by using Linux command taskset. | |
97 | ||
98 | * #insmod rte_kni.ko kthread_mode =multiple | |
99 | ||
100 | This mode will create a kernel thread for each KNI device for packet receiving in kernel side. | |
101 | The core affinity of each kernel thread is set when creating the KNI device. | |
102 | The lcore ID for each kernel thread is provided in the command line of launching the application. | |
103 | Multiple kernel thread mode can provide scalable higher performance. | |
104 | ||
105 | To measure the throughput in a loopback mode, the kernel module of the KNI, | |
106 | located in the kmod sub-directory of the DPDK target directory, | |
107 | can be loaded with parameters as follows: | |
108 | ||
109 | * #insmod rte_kni.ko lo_mode=lo_mode_fifo | |
110 | ||
111 | This loopback mode will involve ring enqueue/dequeue operations in kernel space. | |
112 | ||
113 | * #insmod rte_kni.ko lo_mode=lo_mode_fifo_skb | |
114 | ||
115 | This loopback mode will involve ring enqueue/dequeue operations and sk buffer copies in kernel space. | |
116 | ||
117 | Running the Application | |
118 | ----------------------- | |
119 | ||
120 | The application requires a number of command line options: | |
121 | ||
122 | .. code-block:: console | |
123 | ||
124 | kni [EAL options] -- -P -p PORTMASK --config="(port,lcore_rx,lcore_tx[,lcore_kthread,...])[,port,lcore_rx,lcore_tx[,lcore_kthread,...]]" | |
125 | ||
126 | Where: | |
127 | ||
128 | * -P: Set all ports to promiscuous mode so that packets are accepted regardless of the packet's Ethernet MAC destination address. | |
129 | Without this option, only packets with the Ethernet MAC destination address set to the Ethernet address of the port are accepted. | |
130 | ||
131 | * -p PORTMASK: Hexadecimal bitmask of ports to configure. | |
132 | ||
133 | * --config="(port,lcore_rx, lcore_tx[,lcore_kthread, ...]) [, port,lcore_rx, lcore_tx[,lcore_kthread, ...]]": | |
134 | Determines which lcores of RX, TX, kernel thread are mapped to which ports. | |
135 | ||
136 | Refer to *DPDK Getting Started Guide* for general information on running applications and the Environment Abstraction Layer (EAL) options. | |
137 | ||
138 | The -c coremask or -l corelist parameter of the EAL options should include the lcores indicated by the lcore_rx and lcore_tx, | |
139 | but does not need to include lcores indicated by lcore_kthread as they are used to pin the kernel thread on. | |
140 | The -p PORTMASK parameter should include the ports indicated by the port in --config, neither more nor less. | |
141 | ||
142 | The lcore_kthread in --config can be configured none, one or more lcore IDs. | |
143 | In multiple kernel thread mode, if configured none, a KNI device will be allocated for each port, | |
144 | while no specific lcore affinity will be set for its kernel thread. | |
145 | If configured one or more lcore IDs, one or more KNI devices will be allocated for each port, | |
146 | while specific lcore affinity will be set for its kernel thread. | |
147 | In single kernel thread mode, if configured none, a KNI device will be allocated for each port. | |
148 | If configured one or more lcore IDs, | |
149 | one or more KNI devices will be allocated for each port while | |
150 | no lcore affinity will be set as there is only one kernel thread for all KNI devices. | |
151 | ||
152 | For example, to run the application with two ports served by six lcores, one lcore of RX, one lcore of TX, | |
153 | and one lcore of kernel thread for each port: | |
154 | ||
155 | .. code-block:: console | |
156 | ||
157 | ./build/kni -l 4-7 -n 4 -- -P -p 0x3 --config="(0,4,6,8),(1,5,7,9)" | |
158 | ||
159 | KNI Operations | |
160 | -------------- | |
161 | ||
162 | Once the KNI application is started, one can use different Linux* commands to manage the net interfaces. | |
163 | If more than one KNI devices configured for a physical port, | |
164 | only the first KNI device will be paired to the physical device. | |
165 | Operations on other KNI devices will not affect the physical port handled in user space application. | |
166 | ||
167 | Assigning an IP address: | |
168 | ||
169 | .. code-block:: console | |
170 | ||
171 | #ifconfig vEth0_0 192.168.0.1 | |
172 | ||
173 | Displaying the NIC registers: | |
174 | ||
175 | .. code-block:: console | |
176 | ||
177 | #ethtool -d vEth0_0 | |
178 | ||
179 | Dumping the network traffic: | |
180 | ||
181 | .. code-block:: console | |
182 | ||
183 | #tcpdump -i vEth0_0 | |
184 | ||
185 | Change the MAC address: | |
186 | ||
187 | .. code-block:: console | |
188 | ||
189 | #ifconfig vEth0_0 hw ether 0C:01:02:03:04:08 | |
190 | ||
191 | When the DPDK userspace application is closed, all the KNI devices are deleted from Linux*. | |
192 | ||
193 | Explanation | |
194 | ----------- | |
195 | ||
196 | The following sections provide some explanation of code. | |
197 | ||
198 | Initialization | |
199 | ~~~~~~~~~~~~~~ | |
200 | ||
201 | Setup of mbuf pool, driver and queues is similar to the setup done in the :doc:`l2_forward_real_virtual`.. | |
202 | In addition, one or more kernel NIC interfaces are allocated for each | |
203 | of the configured ports according to the command line parameters. | |
204 | ||
205 | The code for allocating the kernel NIC interfaces for a specific port is | |
206 | in the function ``kni_alloc``. | |
207 | ||
208 | The other step in the initialization process that is unique to this sample application | |
209 | is the association of each port with lcores for RX, TX and kernel threads. | |
210 | ||
211 | * One lcore to read from the port and write to the associated one or more KNI devices | |
212 | ||
213 | * Another lcore to read from one or more KNI devices and write to the port | |
214 | ||
215 | * Other lcores for pinning the kernel threads on one by one | |
216 | ||
217 | This is done by using the ``kni_port_params_array[]`` array, which is indexed by the port ID. | |
218 | The code is in the function ``parse_config``. | |
219 | ||
220 | Packet Forwarding | |
221 | ~~~~~~~~~~~~~~~~~ | |
222 | ||
223 | After the initialization steps are completed, the main_loop() function is run on each lcore. | |
224 | This function first checks the lcore_id against the user provided lcore_rx and lcore_tx | |
225 | to see if this lcore is reading from or writing to kernel NIC interfaces. | |
226 | ||
227 | For the case that reads from a NIC port and writes to the kernel NIC interfaces (``kni_ingress``), | |
228 | the packet reception is the same as in L2 Forwarding sample application | |
229 | (see :ref:`l2_fwd_app_rx_tx_packets`). | |
230 | The packet transmission is done by sending mbufs into the kernel NIC interfaces by rte_kni_tx_burst(). | |
231 | The KNI library automatically frees the mbufs after the kernel successfully copied the mbufs. | |
232 | ||
233 | For the other case that reads from kernel NIC interfaces | |
234 | and writes to a physical NIC port (``kni_egress``), | |
235 | packets are retrieved by reading mbufs from kernel NIC interfaces by ``rte_kni_rx_burst()``. | |
236 | The packet transmission is the same as in the L2 Forwarding sample application | |
237 | (see :ref:`l2_fwd_app_rx_tx_packets`). | |
238 | ||
239 | Callbacks for Kernel Requests | |
240 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
241 | ||
242 | To execute specific PMD operations in user space requested by some Linux* commands, | |
243 | callbacks must be implemented and filled in the struct rte_kni_ops structure. | |
244 | Currently, setting a new MTU, change in MAC address, configuring promiscusous mode and | |
245 | configuring the network interface(up/down) re supported. | |
246 | Default implementation for following is available in rte_kni library. | |
247 | Application may choose to not implement following callbacks: | |
248 | ||
249 | - ``config_mac_address`` | |
250 | - ``config_promiscusity`` |