]>
Commit | Line | Data |
---|---|---|
11fdf7f2 TL |
1 | .. SPDX-License-Identifier: BSD-3-Clause |
2 | Copyright(c) 2010-2014 Intel Corporation. | |
7c673cae FG |
3 | |
4 | VMDQ and DCB Forwarding Sample Application | |
5 | ========================================== | |
6 | ||
7 | The VMDQ and DCB Forwarding sample application is a simple example of packet processing using the DPDK. | |
8 | The application performs L2 forwarding using VMDQ and DCB to divide the incoming traffic into queues. | |
9 | The traffic splitting is performed in hardware by the VMDQ and DCB features of the Intel® 82599 and X710/XL710 Ethernet Controllers. | |
10 | ||
11 | Overview | |
12 | -------- | |
13 | ||
14 | This sample application can be used as a starting point for developing a new application that is based on the DPDK and | |
15 | uses VMDQ and DCB for traffic partitioning. | |
16 | ||
17 | The VMDQ and DCB filters work on MAC and VLAN traffic to divide the traffic into input queues on the basis of the Destination MAC | |
18 | address, VLAN ID and VLAN user priority fields. | |
19 | VMDQ filters split the traffic into 16 or 32 groups based on the Destination MAC and VLAN ID. | |
20 | Then, DCB places each packet into one of queues within that group, based upon the VLAN user priority field. | |
21 | ||
22 | All traffic is read from a single incoming port (port 0) and output on port 1, without any processing being performed. | |
23 | With Intel® 82599 NIC, for example, the traffic is split into 128 queues on input, where each thread of the application reads from | |
24 | multiple queues. When run with 8 threads, that is, with the -c FF option, each thread receives and forwards packets from 16 queues. | |
25 | ||
26 | As supplied, the sample application configures the VMDQ feature to have 32 pools with 4 queues each as indicated in :numref:`figure_vmdq_dcb_example`. | |
27 | The Intel® 82599 10 Gigabit Ethernet Controller NIC also supports the splitting of traffic into 16 pools of 8 queues. While the | |
28 | Intel® X710 or XL710 Ethernet Controller NICs support many configurations of VMDQ pools of 4 or 8 queues each. For simplicity, only 16 | |
29 | or 32 pools is supported in this sample. And queues numbers for each VMDQ pool can be changed by setting CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VM | |
30 | in config/common_* file. | |
31 | The nb-pools, nb-tcs and enable-rss parameters can be passed on the command line, after the EAL parameters: | |
32 | ||
33 | .. code-block:: console | |
34 | ||
35 | ./build/vmdq_dcb [EAL options] -- -p PORTMASK --nb-pools NP --nb-tcs TC --enable-rss | |
36 | ||
37 | where, NP can be 16 or 32, TC can be 4 or 8, rss is disabled by default. | |
38 | ||
39 | .. _figure_vmdq_dcb_example: | |
40 | ||
41 | .. figure:: img/vmdq_dcb_example.* | |
42 | ||
43 | Packet Flow Through the VMDQ and DCB Sample Application | |
44 | ||
45 | ||
46 | In Linux* user space, the application can display statistics with the number of packets received on each queue. | |
47 | To have the application display the statistics, send a SIGHUP signal to the running application process. | |
48 | ||
49 | The VMDQ and DCB Forwarding sample application is in many ways simpler than the L2 Forwarding application | |
50 | (see :doc:`l2_forward_real_virtual`) | |
51 | as it performs unidirectional L2 forwarding of packets from one port to a second port. | |
52 | No command-line options are taken by this application apart from the standard EAL command-line options. | |
53 | ||
54 | .. note:: | |
55 | ||
56 | Since VMD queues are being used for VMM, this application works correctly | |
57 | when VTd is disabled in the BIOS or Linux* kernel (intel_iommu=off). | |
58 | ||
59 | Compiling the Application | |
60 | ------------------------- | |
61 | ||
7c673cae | 62 | |
7c673cae | 63 | |
11fdf7f2 | 64 | To compile the sample application see :doc:`compiling`. |
7c673cae | 65 | |
11fdf7f2 | 66 | The application is located in the ``vmdq_dcb`` sub-directory. |
7c673cae FG |
67 | |
68 | Running the Application | |
69 | ----------------------- | |
70 | ||
9f95a23c | 71 | To run the example in a linux environment: |
7c673cae FG |
72 | |
73 | .. code-block:: console | |
74 | ||
11fdf7f2 | 75 | user@target:~$ ./build/vmdq_dcb -l 0-3 -n 4 -- -p 0x3 --nb-pools 32 --nb-tcs 4 |
7c673cae FG |
76 | |
77 | Refer to the *DPDK Getting Started Guide* for general information on running applications and | |
78 | the Environment Abstraction Layer (EAL) options. | |
79 | ||
80 | Explanation | |
81 | ----------- | |
82 | ||
83 | The following sections provide some explanation of the code. | |
84 | ||
85 | Initialization | |
86 | ~~~~~~~~~~~~~~ | |
87 | ||
88 | The EAL, driver and PCI configuration is performed largely as in the L2 Forwarding sample application, | |
89 | as is the creation of the mbuf pool. | |
90 | See :doc:`l2_forward_real_virtual`. | |
91 | Where this example application differs is in the configuration of the NIC port for RX. | |
92 | ||
93 | The VMDQ and DCB hardware feature is configured at port initialization time by setting the appropriate values in the | |
94 | rte_eth_conf structure passed to the rte_eth_dev_configure() API. | |
95 | Initially in the application, | |
96 | a default structure is provided for VMDQ and DCB configuration to be filled in later by the application. | |
97 | ||
98 | .. code-block:: c | |
99 | ||
100 | /* empty vmdq+dcb configuration structure. Filled in programmatically */ | |
101 | static const struct rte_eth_conf vmdq_dcb_conf_default = { | |
102 | .rxmode = { | |
103 | .mq_mode = ETH_MQ_RX_VMDQ_DCB, | |
104 | .split_hdr_size = 0, | |
7c673cae FG |
105 | }, |
106 | .txmode = { | |
107 | .mq_mode = ETH_MQ_TX_VMDQ_DCB, | |
108 | }, | |
109 | /* | |
110 | * should be overridden separately in code with | |
111 | * appropriate values | |
112 | */ | |
113 | .rx_adv_conf = { | |
114 | .vmdq_dcb_conf = { | |
115 | .nb_queue_pools = ETH_32_POOLS, | |
116 | .enable_default_pool = 0, | |
117 | .default_pool = 0, | |
118 | .nb_pool_maps = 0, | |
119 | .pool_map = {{0, 0},}, | |
120 | .dcb_tc = {0}, | |
121 | }, | |
122 | .dcb_rx_conf = { | |
123 | .nb_tcs = ETH_4_TCS, | |
124 | /** Traffic class each UP mapped to. */ | |
125 | .dcb_tc = {0}, | |
126 | }, | |
127 | .vmdq_rx_conf = { | |
128 | .nb_queue_pools = ETH_32_POOLS, | |
129 | .enable_default_pool = 0, | |
130 | .default_pool = 0, | |
131 | .nb_pool_maps = 0, | |
132 | .pool_map = {{0, 0},}, | |
133 | }, | |
134 | }, | |
135 | .tx_adv_conf = { | |
136 | .vmdq_dcb_tx_conf = { | |
137 | .nb_queue_pools = ETH_32_POOLS, | |
138 | .dcb_tc = {0}, | |
139 | }, | |
140 | }, | |
141 | }; | |
142 | ||
143 | The get_eth_conf() function fills in an rte_eth_conf structure with the appropriate values, | |
144 | based on the global vlan_tags array, | |
145 | and dividing up the possible user priority values equally among the individual queues | |
146 | (also referred to as traffic classes) within each pool. With Intel® 82599 NIC, | |
147 | if the number of pools is 32, then the user priority fields are allocated 2 to a queue. | |
148 | If 16 pools are used, then each of the 8 user priority fields is allocated to its own queue within the pool. | |
149 | With Intel® X710/XL710 NICs, if number of tcs is 4, and number of queues in pool is 8, | |
150 | then the user priority fields are allocated 2 to one tc, and a tc has 2 queues mapping to it, then | |
151 | RSS will determine the destination queue in 2. | |
152 | For the VLAN IDs, each one can be allocated to possibly multiple pools of queues, | |
153 | so the pools parameter in the rte_eth_vmdq_dcb_conf structure is specified as a bitmask value. | |
154 | For destination MAC, each VMDQ pool will be assigned with a MAC address. In this sample, each VMDQ pool | |
155 | is assigned to the MAC like 52:54:00:12:<port_id>:<pool_id>, that is, | |
156 | the MAC of VMDQ pool 2 on port 1 is 52:54:00:12:01:02. | |
157 | ||
158 | .. code-block:: c | |
159 | ||
160 | const uint16_t vlan_tags[] = { | |
161 | 0, 1, 2, 3, 4, 5, 6, 7, | |
162 | 8, 9, 10, 11, 12, 13, 14, 15, | |
163 | 16, 17, 18, 19, 20, 21, 22, 23, | |
164 | 24, 25, 26, 27, 28, 29, 30, 31 | |
165 | }; | |
166 | ||
167 | /* pool mac addr template, pool mac addr is like: 52 54 00 12 port# pool# */ | |
168 | static struct ether_addr pool_addr_template = { | |
169 | .addr_bytes = {0x52, 0x54, 0x00, 0x12, 0x00, 0x00} | |
170 | }; | |
171 | ||
172 | /* Builds up the correct configuration for vmdq+dcb based on the vlan tags array | |
173 | * given above, and the number of traffic classes available for use. */ | |
174 | static inline int | |
175 | get_eth_conf(struct rte_eth_conf *eth_conf) | |
176 | { | |
177 | struct rte_eth_vmdq_dcb_conf conf; | |
178 | struct rte_eth_vmdq_rx_conf vmdq_conf; | |
179 | struct rte_eth_dcb_rx_conf dcb_conf; | |
180 | struct rte_eth_vmdq_dcb_tx_conf tx_conf; | |
181 | uint8_t i; | |
182 | ||
183 | conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; | |
184 | vmdq_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; | |
185 | tx_conf.nb_queue_pools = (enum rte_eth_nb_pools)num_pools; | |
186 | conf.nb_pool_maps = num_pools; | |
187 | vmdq_conf.nb_pool_maps = num_pools; | |
188 | conf.enable_default_pool = 0; | |
189 | vmdq_conf.enable_default_pool = 0; | |
190 | conf.default_pool = 0; /* set explicit value, even if not used */ | |
191 | vmdq_conf.default_pool = 0; | |
192 | ||
193 | for (i = 0; i < conf.nb_pool_maps; i++) { | |
194 | conf.pool_map[i].vlan_id = vlan_tags[i]; | |
195 | vmdq_conf.pool_map[i].vlan_id = vlan_tags[i]; | |
196 | conf.pool_map[i].pools = 1UL << i ; | |
197 | vmdq_conf.pool_map[i].pools = 1UL << i; | |
198 | } | |
199 | for (i = 0; i < ETH_DCB_NUM_USER_PRIORITIES; i++){ | |
200 | conf.dcb_tc[i] = i % num_tcs; | |
201 | dcb_conf.dcb_tc[i] = i % num_tcs; | |
202 | tx_conf.dcb_tc[i] = i % num_tcs; | |
203 | } | |
204 | dcb_conf.nb_tcs = (enum rte_eth_nb_tcs)num_tcs; | |
205 | (void)(rte_memcpy(eth_conf, &vmdq_dcb_conf_default, sizeof(*eth_conf))); | |
206 | (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_dcb_conf, &conf, | |
207 | sizeof(conf))); | |
208 | (void)(rte_memcpy(ð_conf->rx_adv_conf.dcb_rx_conf, &dcb_conf, | |
209 | sizeof(dcb_conf))); | |
210 | (void)(rte_memcpy(ð_conf->rx_adv_conf.vmdq_rx_conf, &vmdq_conf, | |
211 | sizeof(vmdq_conf))); | |
212 | (void)(rte_memcpy(ð_conf->tx_adv_conf.vmdq_dcb_tx_conf, &tx_conf, | |
213 | sizeof(tx_conf))); | |
214 | if (rss_enable) { | |
215 | eth_conf->rxmode.mq_mode= ETH_MQ_RX_VMDQ_DCB_RSS; | |
216 | eth_conf->rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IP | | |
217 | ETH_RSS_UDP | | |
218 | ETH_RSS_TCP | | |
219 | ETH_RSS_SCTP; | |
220 | } | |
221 | return 0; | |
222 | } | |
223 | ||
224 | ...... | |
225 | ||
226 | /* Set mac for each pool.*/ | |
227 | for (q = 0; q < num_pools; q++) { | |
228 | struct ether_addr mac; | |
229 | mac = pool_addr_template; | |
230 | mac.addr_bytes[4] = port; | |
231 | mac.addr_bytes[5] = q; | |
232 | printf("Port %u vmdq pool %u set mac %02x:%02x:%02x:%02x:%02x:%02x\n", | |
233 | port, q, | |
234 | mac.addr_bytes[0], mac.addr_bytes[1], | |
235 | mac.addr_bytes[2], mac.addr_bytes[3], | |
236 | mac.addr_bytes[4], mac.addr_bytes[5]); | |
237 | retval = rte_eth_dev_mac_addr_add(port, &mac, | |
238 | q + vmdq_pool_base); | |
239 | if (retval) { | |
240 | printf("mac addr add failed at pool %d\n", q); | |
241 | return retval; | |
242 | } | |
243 | } | |
244 | ||
245 | Once the network port has been initialized using the correct VMDQ and DCB values, | |
246 | the initialization of the port's RX and TX hardware rings is performed similarly to that | |
247 | in the L2 Forwarding sample application. | |
248 | See :doc:`l2_forward_real_virtual` for more information. | |
249 | ||
250 | Statistics Display | |
251 | ~~~~~~~~~~~~~~~~~~ | |
252 | ||
9f95a23c | 253 | When run in a linux environment, |
7c673cae FG |
254 | the VMDQ and DCB Forwarding sample application can display statistics showing the number of packets read from each RX queue. |
255 | This is provided by way of a signal handler for the SIGHUP signal, | |
256 | which simply prints to standard output the packet counts in grid form. | |
257 | Each row of the output is a single pool with the columns being the queue number within that pool. | |
258 | ||
259 | To generate the statistics output, use the following command: | |
260 | ||
261 | .. code-block:: console | |
262 | ||
263 | user@host$ sudo killall -HUP vmdq_dcb_app | |
264 | ||
265 | Please note that the statistics output will appear on the terminal where the vmdq_dcb_app is running, | |
266 | rather than the terminal from which the HUP signal was sent. |