]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | .. BSD LICENSE |
2 | Copyright(c) 2015-2016 Intel Corporation. All rights reserved. | |
3 | All rights reserved. | |
4 | ||
5 | Redistribution and use in source and binary forms, with or without | |
6 | modification, are permitted provided that the following conditions | |
7 | are met: | |
8 | ||
9 | * Redistributions of source code must retain the above copyright | |
10 | notice, this list of conditions and the following disclaimer. | |
11 | * Redistributions in binary form must reproduce the above copyright | |
12 | notice, this list of conditions and the following disclaimer in | |
13 | the documentation and/or other materials provided with the | |
14 | distribution. | |
15 | * Neither the name of Intel Corporation nor the names of its | |
16 | contributors may be used to endorse or promote products derived | |
17 | from this software without specific prior written permission. | |
18 | ||
19 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | |
20 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | |
21 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | |
22 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | |
23 | OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | |
24 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | |
25 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | |
26 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | |
27 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | |
28 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | |
29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | |
30 | ||
31 | Internet Protocol (IP) Pipeline Application | |
32 | =========================================== | |
33 | ||
34 | Application overview | |
35 | -------------------- | |
36 | ||
37 | The *Internet Protocol (IP) Pipeline* application is intended to be a vehicle for rapid development of packet processing | |
38 | applications running on multi-core CPUs. | |
39 | ||
40 | The application provides a library of reusable functional blocks called pipelines. | |
41 | These pipelines can be seen as prefabricated blocks that can be instantiated and inter-connected through packet queues | |
42 | to create complete applications (super-pipelines). | |
43 | ||
44 | Pipelines are created and inter-connected through the application configuration file. | |
45 | By using different configuration files, different applications are effectively created, therefore this application | |
46 | can be seen as an application generator. | |
47 | The configuration of each pipeline can be updated at run-time through the application Command Line Interface (CLI). | |
48 | ||
49 | Main application components are: | |
50 | ||
51 | **A Library of reusable pipelines** | |
52 | ||
53 | * Each pipeline represents a functional block, e.g. flow classification, firewall, routing, master, etc. | |
54 | ||
55 | * Each pipeline type can be instantiated several times in the same application, which each instance configured | |
56 | separately and mapped to a single CPU core. | |
57 | Each CPU core can run one or several pipeline instances, which can be of same or different type. | |
58 | ||
59 | * Pipeline instances are inter-connected through packet queues (for packet processing) and message queues | |
60 | (for run-time configuration). | |
61 | ||
62 | * Pipelines are implemented using DPDK Packet Framework. | |
63 | ||
64 | * More pipeline types can always be built and added to the existing pipeline types. | |
65 | ||
66 | **The Configuration file** | |
67 | ||
68 | * The configuration file defines the application structure. | |
69 | By using different configuration files, different applications are created. | |
70 | ||
71 | * All the application resources are created and configured through the application configuration file: | |
72 | pipeline instances, buffer pools, links (i.e. network interfaces), hardware device RX/TX queues, | |
73 | software queues, traffic manager devices, EAL startup arguments, etc. | |
74 | ||
75 | * The configuration file syntax is “define by reference”, meaning that resources are defined as they are referenced. | |
76 | First time a resource name is detected, it is registered with default parameters. | |
77 | Optionally, the resource parameters can be further refined through a configuration file section dedicated to | |
78 | that resource. | |
79 | ||
80 | * Command Line Interface (CLI) | |
81 | ||
82 | **Global CLI commands: link configuration, etc.** | |
83 | ||
84 | * Common pipeline CLI commands: ping (keep-alive), statistics, etc. | |
85 | ||
86 | * Pipeline type specific CLI commands: used to configure instances of specific pipeline type. | |
87 | These commands are registered with the application when the pipeline type is registered. | |
88 | For example, the commands for routing pipeline instances include: route add, route delete, route list, etc. | |
89 | ||
90 | * CLI commands can be grouped into scripts that can be invoked at initialization and at runtime. | |
91 | ||
92 | ||
93 | Design goals | |
94 | ------------ | |
95 | ||
96 | ||
97 | Rapid development | |
98 | ~~~~~~~~~~~~~~~~~ | |
99 | ||
100 | This application enables rapid development through quick connectivity of standard components called pipelines. | |
101 | These components are built using DPDK Packet Framework and encapsulate packet processing features at different levels: | |
102 | ports, tables, actions, pipelines and complete applications. | |
103 | ||
104 | Pipeline instances are instantiated, configured and inter-connected through low complexity configuration files loaded | |
105 | during application initialization. | |
106 | Each pipeline instance is mapped to a single CPU core, with each CPU core able to run one or multiple pipeline | |
107 | instances of same or different types. By loading a different configuration file, a different application is | |
108 | effectively started. | |
109 | ||
110 | ||
111 | Flexibility | |
112 | ~~~~~~~~~~~ | |
113 | ||
114 | Each packet processing application is typically represented as a chain of functional stages which is often called | |
115 | the functional pipeline of the application. | |
116 | These stages are mapped to CPU cores to create chains of CPU cores (pipeline model), clusters of CPU cores | |
117 | (run-to-completion model) or chains of clusters of CPU cores (hybrid model). | |
118 | ||
119 | This application allows all the above programming models. | |
120 | By applying changes to the configuration file, the application provides the flexibility to reshuffle its | |
121 | building blocks in different ways until the configuration providing the best performance is identified. | |
122 | ||
123 | ||
124 | Move pipelines around | |
125 | ^^^^^^^^^^^^^^^^^^^^^ | |
126 | ||
127 | The mapping of pipeline instances to CPU cores can be reshuffled through the configuration file. | |
128 | One or several pipeline instances can be mapped to the same CPU core. | |
129 | ||
130 | .. _figure_ip_pipelines_1: | |
131 | ||
132 | .. figure:: img/ip_pipelines_1.* | |
133 | ||
134 | Example of moving pipeline instances across different CPU cores | |
135 | ||
136 | ||
137 | Move tables around | |
138 | ^^^^^^^^^^^^^^^^^^ | |
139 | ||
140 | There is some degree of flexibility for moving tables from one pipeline instance to another. | |
141 | Based on the configuration arguments passed to each pipeline instance in the configuration file, specific tables | |
142 | can be enabled or disabled. | |
143 | This way, a specific table can be “moved” from pipeline instance A to pipeline instance B by simply disabling its | |
144 | associated functionality for pipeline instance A while enabling it for pipeline instance B. | |
145 | ||
146 | Due to requirement to have simple syntax for the configuration file, moving tables across different pipeline | |
147 | instances is not as flexible as the mapping of pipeline instances to CPU cores, or mapping actions to pipeline tables. | |
148 | Complete flexibility in moving tables from one pipeline to another could be achieved through a complex pipeline | |
149 | description language that would detail the structural elements of the pipeline (ports, tables and actions) and | |
150 | their connectivity, resulting in complex syntax for the configuration file, which is not acceptable. | |
151 | Good configuration file readability through simple syntax is preferred. | |
152 | ||
153 | *Example*: the IP routing pipeline can run the routing function only (with ARP function run by a different | |
154 | pipeline instance), or it can run both the routing and ARP functions as part of the same pipeline instance. | |
155 | ||
156 | ||
157 | .. _figure_ip_pipelines_2: | |
158 | ||
159 | .. figure:: img/ip_pipelines_2.* | |
160 | ||
161 | Example of moving tables across different pipeline instances | |
162 | ||
163 | ||
164 | Move actions around | |
165 | ^^^^^^^^^^^^^^^^^^^ | |
166 | ||
167 | When it makes sense, packet processing actions can be moved from one pipeline instance to another. | |
168 | Based on the configuration arguments passed to each pipeline instance in the configuration file, specific actions | |
169 | can be enabled or disabled. | |
170 | This way, a specific action can be "moved" from pipeline instance A to pipeline instance B by simply disabling its | |
171 | associated functionality for pipeline instance A while enabling it for pipeline instance B. | |
172 | ||
173 | *Example*: The flow actions of accounting, traffic metering, application identification, NAT, etc can be run as part | |
174 | of the flow classification pipeline instance or split across several flow actions pipeline instances, depending on | |
175 | the number of flow instances and their compute requirements. | |
176 | ||
177 | ||
178 | .. _figure_ip_pipelines_3: | |
179 | ||
180 | .. figure:: img/ip_pipelines_3.* | |
181 | ||
182 | Example of moving actions across different tables and pipeline instances | |
183 | ||
184 | ||
185 | Performance | |
186 | ~~~~~~~~~~~ | |
187 | ||
188 | Performance of the application is the highest priority requirement. | |
189 | Flexibility is not provided at the expense of performance. | |
190 | ||
191 | The purpose of flexibility is to provide an incremental development methodology that allows monitoring the | |
192 | performance evolution: | |
193 | ||
194 | * Apply incremental changes in the configuration (e.g. mapping on pipeline instances to CPU cores) | |
195 | in order to identify the configuration providing the best performance for a given application; | |
196 | ||
197 | * Add more processing incrementally (e.g. by enabling more actions for specific pipeline instances) until | |
198 | the application is feature complete while checking the performance impact at each step. | |
199 | ||
200 | ||
201 | Debug capabilities | |
202 | ~~~~~~~~~~~~~~~~~~ | |
203 | ||
204 | The application provides a significant set of debug capabilities: | |
205 | ||
206 | * Command Line Interface (CLI) support for statistics polling: pipeline instance ping (keep-alive checks), | |
207 | pipeline instance statistics per input port/output port/table, link statistics, etc; | |
208 | ||
209 | * Logging: Turn on/off application log messages based on priority level; | |
210 | ||
211 | Running the application | |
212 | ----------------------- | |
213 | ||
214 | The application startup command line is:: | |
215 | ||
216 | ip_pipeline [-f CONFIG_FILE] [-s SCRIPT_FILE] -p PORT_MASK [-l LOG_LEVEL] | |
217 | ||
218 | The application startup arguments are: | |
219 | ||
220 | ``-f CONFIG_FILE`` | |
221 | ||
222 | * Optional: Yes | |
223 | ||
224 | * Default: ``./config/ip_pipeline.cfg`` | |
225 | ||
226 | * Argument: Path to the configuration file to be loaded by the application. | |
227 | Please refer to the :ref:`ip_pipeline_configuration_file` for details on how to write the configuration file. | |
228 | ||
229 | ``-s SCRIPT_FILE`` | |
230 | ||
231 | * Optional: Yes | |
232 | ||
233 | * Default: Not present | |
234 | ||
235 | * Argument: Path to the CLI script file to be run by the master pipeline at application startup. | |
236 | No CLI script file will be run at startup of this argument is not present. | |
237 | ||
238 | ``-p PORT_MASK`` | |
239 | ||
240 | * Optional: No | |
241 | ||
242 | * Default: N/A | |
243 | ||
244 | * Argument: Hexadecimal mask of NIC port IDs to be used by the application. | |
245 | First port enabled in this mask will be referenced as LINK0 as part of the application configuration file, | |
246 | next port as LINK1, etc. | |
247 | ||
248 | ``-l LOG_LEVEL`` | |
249 | ||
250 | * Optional: Yes | |
251 | ||
252 | * Default: 1 (High priority) | |
253 | ||
254 | * Argument: Log level to determine which application messages are to be printed to standard output. | |
255 | Available log levels are: 0 (None), 1 (High priority), 2 (Low priority). | |
256 | Only application messages whose priority is higher than or equal to the application log level will be printed. | |
257 | ||
258 | ||
259 | Application stages | |
260 | ------------------ | |
261 | ||
262 | ||
263 | Configuration | |
264 | ~~~~~~~~~~~~~ | |
265 | ||
266 | During this stage, the application configuration file is parsed and its content is loaded into the application data | |
267 | structures. | |
268 | In case of any configuration file parse error, an error message is displayed and the application is terminated. | |
269 | Please refer to the :ref:`ip_pipeline_configuration_file` for a description of the application configuration file format. | |
270 | ||
271 | ||
272 | Configuration checking | |
273 | ~~~~~~~~~~~~~~~~~~~~~~ | |
274 | ||
275 | In the absence of any parse errors, the loaded content of application data structures is checked for overall consistency. | |
276 | In case of any configuration check error, an error message is displayed and the application is terminated. | |
277 | ||
278 | ||
279 | Initialization | |
280 | ~~~~~~~~~~~~~~ | |
281 | ||
282 | During this stage, the application resources are initialized and the handles to access them are saved into the | |
283 | application data structures. | |
284 | In case of any initialization error, an error message is displayed and the application is terminated. | |
285 | ||
286 | The typical resources to be initialized are: pipeline instances, buffer pools, links (i.e. network interfaces), | |
287 | hardware device RX/TX queues, software queues, traffic management devices, etc. | |
288 | ||
289 | ||
290 | .. _ip_pipeline_runtime: | |
291 | ||
292 | Run-time | |
293 | ~~~~~~~~ | |
294 | ||
295 | Each CPU core runs the pipeline instances assigned to it in time sharing mode and in round robin order: | |
296 | ||
297 | 1. *Packet processing task*: The pipeline run-time code is typically a packet *processing* task built on top of | |
298 | DPDK Packet Framework rte_pipeline library, which reads bursts of packets from the pipeline input ports, | |
299 | performs table lookups and executes the identified actions for all tables in the pipeline, with packet | |
300 | eventually written to pipeline output ports or dropped. | |
301 | ||
302 | 2. *Message handling task*: Each CPU core will also periodically execute the *message handling* code of each | |
303 | of the pipelines mapped to it. | |
304 | The pipeline message handling code is processing the messages that are pending in the pipeline input message | |
305 | queues, which are typically sent by the master CPU core for the on-the-fly pipeline configuration: check | |
306 | that pipeline is still alive (ping), add/delete entries in the pipeline tables, get statistics, etc. | |
307 | The frequency of executing the message handling code is usually much smaller than the frequency of executing | |
308 | the packet processing work. | |
309 | ||
310 | Please refer to the :ref:`ip_pipeline_pipeline_section` for more details about the application pipeline module encapsulation. | |
311 | ||
312 | .. _ip_pipeline_configuration_file: | |
313 | ||
314 | Configuration file syntax | |
315 | ------------------------- | |
316 | ||
317 | Syntax overview | |
318 | ~~~~~~~~~~~~~~~ | |
319 | ||
320 | The syntax of the configuration file is designed to be simple, which favors readability. | |
321 | The configuration file is parsed using the DPDK library librte_cfgfile, which supports simple | |
322 | `INI file format <http://en.wikipedia.org/wiki/INI_file>`__ for configuration files. | |
323 | ||
324 | As result, the configuration file is split into several sections, with each section containing one or more entries. | |
325 | The scope of each entry is its section, and each entry specifies a variable that is assigned a specific value. | |
326 | Any text after the ``;`` character is considered a comment and is therefore ignored. | |
327 | ||
328 | The following are application specific: number of sections, name of each section, number of entries of each section, | |
329 | name of the variables used for each section entry, the value format (e.g. signed/unsigned integer, string, etc) | |
330 | and range of each section entry variable. | |
331 | ||
332 | Generic example of configuration file section: | |
333 | ||
334 | .. code-block:: ini | |
335 | ||
336 | [<section_name>] | |
337 | ||
338 | <variable_name_1> = <value_1> | |
339 | ||
340 | ; ... | |
341 | ||
342 | <variable_name_N> = <value_N> | |
343 | ||
344 | ||
345 | Application resources present in the configuration file | |
346 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
347 | ||
348 | .. _table_ip_pipelines_resource_name: | |
349 | ||
350 | .. tabularcolumns:: |p{4cm}|p{6cm}|p{6cm}| | |
351 | ||
352 | .. table:: Application resource names in the configuration file | |
353 | ||
354 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
355 | | Resource type | Format | Examples | | |
356 | +============================+=============================+=================================================+ | |
357 | | Pipeline | ``PIPELINE<ID>`` | ``PIPELINE0``, ``PIPELINE1`` | | |
358 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
359 | | Mempool | ``MEMPOOL<ID>`` | ``MEMPOOL0``, ``MEMPOOL1`` | | |
360 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
361 | | Link (network interface) | ``LINK<ID>`` | ``LINK0``, ``LINK1`` | | |
362 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
363 | | Link RX queue | ``RXQ<LINK_ID>.<QUEUE_ID>`` | ``RXQ0.0``, ``RXQ1.5`` | | |
364 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
365 | | Link TX queue | ``TXQ<LINK_ID>.<QUEUE_ID>`` | ``TXQ0.0``, ``TXQ1.5`` | | |
366 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
367 | | Software queue | ``SWQ<ID>`` | ``SWQ0``, ``SWQ1`` | | |
368 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
369 | | Traffic Manager | ``TM<LINK_ID>`` | ``TM0``, ``TM1`` | | |
370 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
371 | | KNI (kernel NIC interface) | ``KNI<LINK_ID>`` | ``KNI0``, ``KNI1`` | | |
372 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
373 | | Source | ``SOURCE<ID>`` | ``SOURCE0``, ``SOURCE1`` | | |
374 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
375 | | Sink | ``SINK<ID>`` | ``SINK0``, ``SINK1`` | | |
376 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
377 | | Message queue | ``MSGQ<ID>`` | ``MSGQ0``, ``MSGQ1``, | | |
378 | | | ``MSGQ-REQ-PIPELINE<ID>`` | ``MSGQ-REQ-PIPELINE2``, ``MSGQ-RSP-PIPELINE2,`` | | |
379 | | | ``MSGQ-RSP-PIPELINE<ID>`` | ``MSGQ-REQ-CORE-s0c1``, ``MSGQ-RSP-CORE-s0c1`` | | |
380 | | | ``MSGQ-REQ-CORE-<CORE_ID>`` | | | |
381 | | | ``MSGQ-RSP-CORE-<CORE_ID>`` | | | |
382 | +----------------------------+-----------------------------+-------------------------------------------------+ | |
383 | ||
384 | ``LINK`` instances are created implicitly based on the ``PORT_MASK`` application startup argument. | |
385 | ``LINK0`` is the first port enabled in the ``PORT_MASK``, port 1 is the next one, etc. | |
386 | The LINK ID is different than the DPDK PMD-level NIC port ID, which is the actual position in the bitmask mentioned above. | |
387 | For example, if bit 5 is the first bit set in the bitmask, then ``LINK0`` is having the PMD ID of 5. | |
388 | This mechanism creates a contiguous LINK ID space and isolates the configuration file against changes in the board | |
389 | PCIe slots where NICs are plugged in. | |
390 | ||
391 | ``RXQ``, ``TXQ``, ``TM`` and ``KNI`` instances have the LINK ID as part of their name. | |
392 | For example, ``RXQ2.1``, ``TXQ2.1`` and ``TM2`` are all associated with ``LINK2``. | |
393 | ||
394 | ||
395 | Rules to parse the configuration file | |
396 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
397 | ||
398 | The main rules used to parse the configuration file are: | |
399 | ||
400 | 1. Application resource name determines the type of resource based on the name prefix. | |
401 | ||
402 | *Example*: all software queues need to start with ``SWQ`` prefix, so ``SWQ0`` and ``SWQ5`` are valid software | |
403 | queue names. | |
404 | ||
405 | 2. An application resource is defined by creating a configuration file section with its name. | |
406 | The configuration file section allows fine tuning on any of the resource parameters. | |
407 | Some resource parameters are mandatory, in which case it is required to have them specified as part of the | |
408 | section, while some others are optional, in which case they get assigned their default value when not present. | |
409 | ||
410 | *Example*: section ``SWQ0`` defines a software queue named SWQ0, whose parameters are detailed as part of this section. | |
411 | ||
412 | 3. An application resource can also be defined by referencing it. | |
413 | Referencing a resource takes place by simply using its name as part of the value assigned to a variable in any | |
414 | configuration file section. | |
415 | In this case, the resource is registered with all its parameters having their default values. | |
416 | Optionally, a section with the resource name can be added to the configuration file to fine tune some or all | |
417 | of the resource parameters. | |
418 | ||
419 | *Example*: in section ``PIPELINE3``, variable ``pktq_in`` includes ``SWQ5`` as part of its list, which results | |
420 | in defining a software queue named ``SWQ5``; when there is no ``SWQ5`` section present in the configuration file, | |
421 | ``SWQ5`` gets registered with default parameters. | |
422 | ||
423 | ||
424 | .. _ip_pipeline_pipeline_section: | |
425 | ||
426 | PIPELINE section | |
427 | ~~~~~~~~~~~~~~~~ | |
428 | ||
429 | .. _table_ip_pipelines_pipeline_section_1: | |
430 | ||
431 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| | |
432 | ||
433 | .. table:: Configuration file PIPELINE section (1/2) | |
434 | ||
435 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
436 | | Section | Description | Optional | Range | Default value | | |
437 | +===============+===========================================================+===============+========================+================+ | |
438 | | type | Pipeline type. Defines the functionality to be | NO | See "List | N/A | | |
439 | | | executed. | | of pipeline types" | | | |
440 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
441 | | core | CPU core to run the current pipeline. | YES | See "CPU Core | CPU socket 0, | | |
442 | | | | | notation" | core 0, | | |
443 | | | | | | hyper-thread 0 | | |
444 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
445 | | pktq_in | Packet queues to serve as input ports for the | YES | List of input | Empty list | | |
446 | | | current pipeline instance. The acceptable packet | | packet queue IDs | | | |
447 | | | queue types are: ``RXQ``, ``SWQ``, ``TM`` and ``SOURCE``. | | | | | |
448 | | | First device in this list is used as pipeline input port | | | | | |
449 | | | 0, second as pipeline input port 1, etc. | | | | | |
450 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
451 | | pktq_out | Packet queues to serve as output ports for the | YES | List of output | Empty list | | |
452 | | | current pipeline instance. The acceptable packet | | packet queue IDs. | | | |
453 | | | queue types are: ``TXQ``, ``SWQ``, ``TM`` and ``SINK``. | | | | | |
454 | | | First device in this list is used as pipeline output | | | | | |
455 | | | port 0, second as pipeline output port 1, etc. | | | | | |
456 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
457 | ||
458 | .. _table_ip_pipelines_pipeline_section_2: | |
459 | ||
460 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| | |
461 | ||
462 | .. table:: Configuration file PIPELINE section (2/2) | |
463 | ||
464 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
465 | | Section | Description | Optional | Range | Default value | | |
466 | +===============+===========================================================+===============+========================+================+ | |
467 | | msgq_in | Input message queues. These queues contain | YES | List of message | Empty list | | |
468 | | | request messages that need to be handled by the | | queue IDs | | | |
469 | | | current pipeline instance. The type and format of | | | | | |
470 | | | request messages is defined by the pipeline type. | | | | | |
471 | | | For each pipeline instance, there is an input | | | | | |
472 | | | message queue defined implicitly, whose name is: | | | | | |
473 | | | ``MSGQ-REQ-<PIPELINE_ID>``. This message queue | | | | | |
474 | | | should not be mentioned as part of msgq_in list. | | | | | |
475 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
476 | | msgq_out | Output message queues. These queues are used by | YES | List of message | Empty list | | |
477 | | | the current pipeline instance to write response | | queue IDs | | | |
478 | | | messages as result of request messages being | | | | | |
479 | | | handled. The type and format of response | | | | | |
480 | | | messages is defined by the pipeline type. | | | | | |
481 | | | For each pipeline instance, there is an output | | | | | |
482 | | | message queue defined implicitly, whose name is: | | | | | |
483 | | | ``MSGQ-RSP-<PIPELINE_ID>``. This message queue | | | | | |
484 | | | should not be mentioned as part of msgq_out list. | | | | | |
485 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
486 | | timer_period | Time period, measured in milliseconds, | YES | milliseconds | 1 ms | | |
487 | | | for handling the input message queues. | | | | | |
488 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
489 | | <any other> | Arguments to be passed to the current pipeline | Depends on | Depends on | Depends on | | |
490 | | | instance. Format of the arguments, their type, | pipeline type | pipeline type | pipeline type | | |
491 | | | whether each argument is optional or mandatory | | | | | |
492 | | | and its default value (when optional) are defined | | | | | |
493 | | | by the pipeline type. | | | | | |
494 | | | The value of the arguments is applicable to the | | | | | |
495 | | | current pipeline instance only. | | | | | |
496 | +---------------+-----------------------------------------------------------+---------------+------------------------+----------------+ | |
497 | ||
498 | ||
499 | CPU core notation | |
500 | ^^^^^^^^^^^^^^^^^ | |
501 | ||
502 | The CPU Core notation is:: | |
503 | ||
504 | <CPU core> ::= [s|S<CPU socket ID>][c|C]<CPU core ID>[h|H] | |
505 | ||
506 | For example:: | |
507 | ||
508 | CPU socket 0, core 0, hyper-thread 0: 0, c0, s0c0 | |
509 | ||
510 | CPU socket 0, core 0, hyper-thread 1: 0h, c0h, s0c0h | |
511 | ||
512 | CPU socket 3, core 9, hyper-thread 1: s3c9h | |
513 | ||
514 | ||
515 | MEMPOOL section | |
516 | ~~~~~~~~~~~~~~~ | |
517 | ||
518 | .. _table_ip_pipelines_mempool_section: | |
519 | ||
520 | .. tabularcolumns:: |p{2.5cm}|p{6cm}|p{1.5cm}|p{1.5cm}|p{3cm}| | |
521 | ||
522 | .. table:: Configuration file MEMPOOL section | |
523 | ||
524 | +---------------+-----------------------------------------------+----------+----------+---------------------------+ | |
525 | | Section | Description | Optional | Type | Default value | | |
526 | +===============+===============================================+==========+==========+===========================+ | |
527 | | buffer_size | Buffer size (in bytes) for the current | YES | uint32_t | 2048 | | |
528 | | | buffer pool. | | | + sizeof(struct rte_mbuf) | | |
529 | | | | | | + HEADROOM | | |
530 | +---------------+-----------------------------------------------+----------+----------+---------------------------+ | |
531 | | pool_size | Number of buffers in the current buffer pool. | YES | uint32_t | 32K | | |
532 | +---------------+-----------------------------------------------+----------+----------+---------------------------+ | |
533 | | cache_size | Per CPU thread cache size (in number of | YES | uint32_t | 256 | | |
534 | | | buffers) for the current buffer pool. | | | | | |
535 | +---------------+-----------------------------------------------+----------+----------+---------------------------+ | |
536 | | cpu | CPU socket ID where to allocate memory for | YES | uint32_t | 0 | | |
537 | | | the current buffer pool. | | | | | |
538 | +---------------+-----------------------------------------------+----------+----------+---------------------------+ | |
539 | ||
540 | ||
541 | LINK section | |
542 | ~~~~~~~~~~~~ | |
543 | ||
544 | .. _table_ip_pipelines_link_section: | |
545 | ||
546 | .. tabularcolumns:: |p{3cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{2cm}| | |
547 | ||
548 | .. table:: Configuration file LINK section | |
549 | ||
550 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
551 | | Section entry | Description | Optional | Type | Default value | | |
552 | +=================+==============================================+==========+==========+===================+ | |
553 | | arp_q | NIC RX queue where ARP packets should | YES | 0 .. 127 | 0 (default queue) | | |
554 | | | be filtered. | | | | | |
555 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
556 | | tcp_syn_local_q | NIC RX queue where TCP packets with SYN | YES | 0 .. 127 | 0 (default queue) | | |
557 | | | flag should be filtered. | | | | | |
558 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
559 | | ip_local_q | NIC RX queue where IP packets with local | YES | 0 .. 127 | 0 (default queue) | | |
560 | | | destination should be filtered. | | | | | |
561 | | | When TCP, UDP and SCTP local queues are | | | | | |
562 | | | defined, they take higher priority than this | | | | | |
563 | | | queue. | | | | | |
564 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
565 | | tcp_local_q | NIC RX queue where TCP packets with local | YES | 0 .. 127 | 0 (default queue) | | |
566 | | | destination should be filtered. | | | | | |
567 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
568 | | udp_local_q | NIC RX queue where TCP packets with local | YES | 0 .. 127 | 0 (default queue) | | |
569 | | | destination should be filtered. | | | | | |
570 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
571 | | sctp_local_q | NIC RX queue where TCP packets with local | YES | 0 .. 127 | 0 (default queue) | | |
572 | | | destination should be filtered. | | | | | |
573 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
574 | | promisc | Indicates whether current link should be | YES | YES/NO | YES | | |
575 | | | started in promiscuous mode. | | | | | |
576 | +-----------------+----------------------------------------------+----------+----------+-------------------+ | |
577 | ||
578 | ||
579 | RXQ section | |
580 | ~~~~~~~~~~~ | |
581 | ||
582 | .. _table_ip_pipelines_rxq_section: | |
583 | ||
584 | .. tabularcolumns:: |p{3cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{2cm}| | |
585 | ||
586 | .. table:: Configuration file RXQ section | |
587 | ||
588 | +---------------+--------------------------------------------+----------+----------+---------------+ | |
589 | | Section | Description | Optional | Type | Default value | | |
590 | +===============+============================================+==========+==========+===============+ | |
591 | | mempool | Mempool to use for buffer allocation for | YES | uint32_t | MEMPOOL0 | | |
592 | | | current NIC RX queue. The mempool ID has | | | | | |
593 | | | to be associated with a valid instance | | | | | |
594 | | | defined in the mempool entry of the global | | | | | |
595 | | | section. | | | | | |
596 | +---------------+--------------------------------------------+----------+----------+---------------+ | |
597 | | Size | NIC RX queue size (number of descriptors) | YES | uint32_t | 128 | | |
598 | +---------------+--------------------------------------------+----------+----------+---------------+ | |
599 | | burst | Read burst size (number of descriptors) | YES | uint32_t | 32 | | |
600 | +---------------+--------------------------------------------+----------+----------+---------------+ | |
601 | ||
602 | ||
603 | TXQ section | |
604 | ~~~~~~~~~~~ | |
605 | ||
606 | .. _table_ip_pipelines_txq_section: | |
607 | ||
608 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{2cm}|p{1.5cm}| | |
609 | ||
610 | .. table:: Configuration file TXQ section | |
611 | ||
612 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
613 | | Section | Description | Optional | Type | Default value | | |
614 | +===============+==============================================+==========+==================+===============+ | |
615 | | size | NIC TX queue size (number of descriptors) | YES | uint32_t | 512 | | |
616 | | | | | power of 2 | | | |
617 | | | | | > 0 | | | |
618 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
619 | | burst | Write burst size (number of descriptors) | YES | uint32_t | 32 | | |
620 | | | | | power of 2 | | | |
621 | | | | | 0 < burst < size | | | |
622 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
623 | | dropless | When dropless is set to NO, packets can be | YES | YES/NO | NO | | |
624 | | | dropped if not enough free slots are | | | | | |
625 | | | currently available in the queue, so the | | | | | |
626 | | | write operation to the queue is non- | | | | | |
627 | | | blocking. | | | | | |
628 | | | When dropless is set to YES, packets cannot | | | | | |
629 | | | be dropped if not enough free slots are | | | | | |
630 | | | currently available in the queue, so the | | | | | |
631 | | | write operation to the queue is blocking, as | | | | | |
632 | | | the write operation is retried until enough | | | | | |
633 | | | free slots become available and all the | | | | | |
634 | | | packets are successfully written to the | | | | | |
635 | | | queue. | | | | | |
636 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
637 | | n_retries | Number of retries. Valid only when dropless | YES | uint32_t | 0 | | |
638 | | | is set to YES. When set to 0, it indicates | | | | | |
639 | | | unlimited number of retries. | | | | | |
640 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
641 | ||
642 | ||
643 | SWQ section | |
644 | ~~~~~~~~~~~ | |
645 | ||
646 | .. _table_ip_pipelines_swq_section: | |
647 | ||
648 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| | |
649 | ||
650 | .. table:: Configuration file SWQ section | |
651 | ||
652 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
653 | | Section | Description | Optional | Type | Default value | | |
654 | +===============+==============================================+==========+==================+===============+ | |
655 | | size | Queue size (number of packets) | YES | uint32_t | 256 | | |
656 | | | | | power of 2 | | | |
657 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
658 | | burst_read | Read burst size (number of packets) | YES | uint32_t | 32 | | |
659 | | | | | power of 2 | | | |
660 | | | | | 0 < burst < size | | | |
661 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
662 | | burst_write | Write burst size (number of packets) | YES | uint32_t | 32 | | |
663 | | | | | power of 2 | | | |
664 | | | | | 0 < burst < size | | | |
665 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
666 | | dropless | When dropless is set to NO, packets can be | YES | YES/NO | NO | | |
667 | | | dropped if not enough free slots are | | | | | |
668 | | | currently available in the queue, so the | | | | | |
669 | | | write operation to the queue is non- | | | | | |
670 | | | blocking. | | | | | |
671 | | | When dropless is set to YES, packets cannot | | | | | |
672 | | | be dropped if not enough free slots are | | | | | |
673 | | | currently available in the queue, so the | | | | | |
674 | | | write operation to the queue is blocking, as | | | | | |
675 | | | the write operation is retried until enough | | | | | |
676 | | | free slots become available and all the | | | | | |
677 | | | packets are successfully written to the | | | | | |
678 | | | queue. | | | | | |
679 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
680 | | n_retries | Number of retries. Valid only when dropless | YES | uint32_t | 0 | | |
681 | | | is set to YES. When set to 0, it indicates | | | | | |
682 | | | unlimited number of retries. | | | | | |
683 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
684 | | cpu | CPU socket ID where to allocate memory | YES | uint32_t | 0 | | |
685 | | | for this SWQ. | | | | | |
686 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
687 | ||
688 | ||
689 | TM section | |
690 | ~~~~~~~~~~ | |
691 | ||
692 | .. _table_ip_pipelines_tm_section: | |
693 | ||
694 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| | |
695 | ||
696 | .. table:: Configuration file TM section | |
697 | ||
698 | +---------------+---------------------------------------------+----------+----------+---------------+ | |
699 | | Section | Description | Optional | Type | Default value | | |
700 | +===============+=============================================+==========+==========+===============+ | |
701 | | Cfg | File name to parse for the TM configuration | YES | string | tm_profile | | |
702 | | | to be applied. The syntax of this file is | | | | | |
703 | | | described in the examples/qos_sched DPDK | | | | | |
704 | | | application documentation. | | | | | |
705 | +---------------+---------------------------------------------+----------+----------+---------------+ | |
706 | | burst_read | Read burst size (number of packets) | YES | uint32_t | 64 | | |
707 | +---------------+---------------------------------------------+----------+----------+---------------+ | |
708 | | burst_write | Write burst size (number of packets) | YES | uint32_t | 32 | | |
709 | +---------------+---------------------------------------------+----------+----------+---------------+ | |
710 | ||
711 | ||
712 | KNI section | |
713 | ~~~~~~~~~~~ | |
714 | ||
715 | .. _table_ip_pipelines_kni_section: | |
716 | ||
717 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| | |
718 | ||
719 | .. table:: Configuration file KNI section | |
720 | ||
721 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
722 | | Section | Description | Optional | Type | Default value | | |
723 | +===============+==============================================+==========+==================+===============+ | |
724 | | core | CPU core to run the KNI kernel thread. | YES | See "CPU Core | Not set | | |
725 | | | When core config is set, the KNI kernel | | notation" | | | |
726 | | | thread will be bound to the particular core. | | | | | |
727 | | | When core config is not set, the KNI kernel | | | | | |
728 | | | thread will be scheduled by the OS. | | | | | |
729 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
730 | | mempool | Mempool to use for buffer allocation for | YES | uint32_t | MEMPOOL0 | | |
731 | | | current KNI port. The mempool ID has | | | | | |
732 | | | to be associated with a valid instance | | | | | |
733 | | | defined in the mempool entry of the global | | | | | |
734 | | | section. | | | | | |
735 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
736 | | burst_read | Read burst size (number of packets) | YES | uint32_t | 32 | | |
737 | | | | | power of 2 | | | |
738 | | | | | 0 < burst < size | | | |
739 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
740 | | burst_write | Write burst size (number of packets) | YES | uint32_t | 32 | | |
741 | | | | | power of 2 | | | |
742 | | | | | 0 < burst < size | | | |
743 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
744 | | dropless | When dropless is set to NO, packets can be | YES | YES/NO | NO | | |
745 | | | dropped if not enough free slots are | | | | | |
746 | | | currently available in the queue, so the | | | | | |
747 | | | write operation to the queue is non- | | | | | |
748 | | | blocking. | | | | | |
749 | | | When dropless is set to YES, packets cannot | | | | | |
750 | | | be dropped if not enough free slots are | | | | | |
751 | | | currently available in the queue, so the | | | | | |
752 | | | write operation to the queue is blocking, as | | | | | |
753 | | | the write operation is retried until enough | | | | | |
754 | | | free slots become available and all the | | | | | |
755 | | | packets are successfully written to the | | | | | |
756 | | | queue. | | | | | |
757 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
758 | | n_retries | Number of retries. Valid only when dropless | YES | uint64_t | 0 | | |
759 | | | is set to YES. When set to 0, it indicates | | | | | |
760 | | | unlimited number of retries. | | | | | |
761 | +---------------+----------------------------------------------+----------+------------------+---------------+ | |
762 | ||
763 | ||
764 | SOURCE section | |
765 | ~~~~~~~~~~~~~~ | |
766 | ||
767 | .. _table_ip_pipelines_source_section: | |
768 | ||
769 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{2cm}| | |
770 | ||
771 | .. table:: Configuration file SOURCE section | |
772 | ||
773 | +---------------+---------------------------------------+----------+----------+---------------+ | |
774 | | Section | Description | Optional | Type | Default value | | |
775 | +===============+=======================================+==========+==========+===============+ | |
776 | | Mempool | Mempool to use for buffer allocation. | YES | uint32_t | MEMPOOL0 | | |
777 | +---------------+---------------------------------------+----------+----------+---------------+ | |
778 | | Burst | Read burst size (number of packets) | | uint32_t | 32 | | |
779 | +---------------+---------------------------------------+----------+----------+---------------+ | |
780 | ||
781 | ||
782 | SINK section | |
783 | ~~~~~~~~~~~~ | |
784 | ||
785 | Currently, there are no parameters to be passed to a sink device, so | |
786 | SINK section is not allowed. | |
787 | ||
788 | MSGQ section | |
789 | ~~~~~~~~~~~~ | |
790 | ||
791 | .. _table_ip_pipelines_msgq_section: | |
792 | ||
793 | .. tabularcolumns:: |p{2.5cm}|p{7cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}| | |
794 | ||
795 | .. table:: Configuration file MSGQ section | |
796 | ||
797 | +---------+--------------------------------------------+----------+------------+---------------+ | |
798 | | Section | Description | Optional | Type | Default value | | |
799 | +=========+============================================+==========+============+===============+ | |
800 | | size | Queue size (number of packets) | YES | uint32_t | 64 | | |
801 | | | | | != 0 | | | |
802 | | | | | power of 2 | | | |
803 | +---------+--------------------------------------------+----------+------------+---------------+ | |
804 | | cpu | CPU socket ID where to allocate memory for | YES | uint32_t | 0 | | |
805 | | | the current queue. | | | | | |
806 | +---------+--------------------------------------------+----------+------------+---------------+ | |
807 | ||
808 | ||
809 | EAL section | |
810 | ~~~~~~~~~~~ | |
811 | ||
812 | The application generates the EAL parameters rather than reading them from the command line. | |
813 | ||
814 | The CPU core mask parameter is generated based on the core entry of all PIPELINE sections. | |
815 | All the other EAL parameters can be set from this section of the application configuration file. | |
816 | ||
817 | ||
818 | Library of pipeline types | |
819 | ------------------------- | |
820 | ||
821 | Pipeline module | |
822 | ~~~~~~~~~~~~~~~ | |
823 | ||
824 | A pipeline is a self-contained module that implements a packet processing function and is typically implemented on | |
825 | top of the DPDK Packet Framework *librte_pipeline* library. | |
826 | The application provides a run-time mechanism to register different pipeline types. | |
827 | ||
828 | Depending on the required configuration, each registered pipeline type (pipeline class) is instantiated one or | |
829 | several times, with each pipeline instance (pipeline object) assigned to one of the available CPU cores. | |
830 | Each CPU core can run one or more pipeline instances, which might be of same or different types. | |
831 | For more information of the CPU core threading model, please refer to the :ref:`ip_pipeline_runtime` section. | |
832 | ||
833 | ||
834 | Pipeline type | |
835 | ^^^^^^^^^^^^^ | |
836 | ||
837 | Each pipeline type is made up of a back-end and a front-end. The back-end represents the packet processing engine | |
838 | of the pipeline, typically implemented using the DPDK Packet Framework libraries, which reads packets from the | |
839 | input packet queues, handles them and eventually writes them to the output packet queues or drops them. | |
840 | The front-end represents the run-time configuration interface of the pipeline, which is exposed as CLI commands. | |
841 | The front-end communicates with the back-end through message queues. | |
842 | ||
843 | .. _table_ip_pipelines_back_end: | |
844 | ||
845 | .. tabularcolumns:: |p{1cm}|p{2cm}|p{12cm}| | |
846 | ||
847 | .. table:: Pipeline back-end | |
848 | ||
849 | +------------+------------------+--------------------------------------------------------------------+ | |
850 | | Field name | Field type | Description | | |
851 | +============+==================+====================================================================+ | |
852 | | f_init | Function pointer | Function to initialize the back-end of the current pipeline | | |
853 | | | | instance. Typical work implemented by this function for the | | |
854 | | | | current pipeline instance: | | |
855 | | | | Memory allocation; | | |
856 | | | | Parse the pipeline type specific arguments; | | |
857 | | | | Initialize the pipeline input ports, output ports and tables, | | |
858 | | | | interconnect input ports to tables; | | |
859 | | | | Set the message handlers. | | |
860 | +------------+------------------+--------------------------------------------------------------------+ | |
861 | | f_free | Function pointer | Function to free the resources allocated by the back-end of the | | |
862 | | | | current pipeline instance. | | |
863 | +------------+------------------+--------------------------------------------------------------------+ | |
864 | | f_run | Function pointer | Set to NULL for pipelines implemented using the DPDK library | | |
865 | | | | librte_pipeline (typical case), and to non-NULL otherwise. This | | |
866 | | | | mechanism is made available to support quick integration of | | |
867 | | | | legacy code. | | |
868 | | | | This function is expected to provide the packet processing | | |
869 | | | | related code to be called as part of the CPU thread dispatch | | |
870 | | | | loop, so this function is not allowed to contain an infinite loop. | | |
871 | +------------+------------------+--------------------------------------------------------------------+ | |
872 | | f_timer | Function pointer | Function to read the pipeline input message queues, handle | | |
873 | | | | the request messages, create response messages and write | | |
874 | | | | the response queues. The format of request and response | | |
875 | | | | messages is defined by each pipeline type, with the exception | | |
876 | | | | of some requests which are mandatory for all pipelines (e.g. | | |
877 | | | | ping, statistics). | | |
878 | +------------+------------------+--------------------------------------------------------------------+ | |
879 | | f_track | Function pointer | See section Tracking pipeline output port to physical link | | |
880 | +------------+------------------+--------------------------------------------------------------------+ | |
881 | ||
882 | ||
883 | .. _table_ip_pipelines_front_end: | |
884 | ||
885 | .. tabularcolumns:: |p{1cm}|p{2cm}|p{12cm}| | |
886 | ||
887 | .. table:: Pipeline front-end | |
888 | ||
889 | +------------+-----------------------+-------------------------------------------------------------------+ | |
890 | | Field name | Field type | Description | | |
891 | +============+=======================+===================================================================+ | |
892 | | f_init | Function pointer | Function to initialize the front-end of the current pipeline | | |
893 | | | | instance. | | |
894 | +------------+-----------------------+-------------------------------------------------------------------+ | |
895 | | f_free | Function pointer | Function to free the resources allocated by the front-end of | | |
896 | | | | the current pipeline instance. | | |
897 | +------------+-----------------------+-------------------------------------------------------------------+ | |
898 | | cmds | Array of CLI commands | Array of CLI commands to be registered to the application CLI | | |
899 | | | | for the current pipeline type. Even though the CLI is executed | | |
900 | | | | by a different pipeline (typically, this is the master pipeline), | | |
901 | | | | from modularity perspective is more efficient to keep the | | |
902 | | | | message client side (part of the front-end) together with the | | |
903 | | | | message server side (part of the back-end). | | |
904 | +------------+-----------------------+-------------------------------------------------------------------+ | |
905 | ||
906 | ||
907 | Tracking pipeline output port to physical link | |
908 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
909 | ||
910 | Each pipeline instance is a standalone block that does not have visibility into the other pipeline instances or | |
911 | the application-level pipeline inter-connectivity. | |
912 | In some cases, it is useful for a pipeline instance to get application level information related to pipeline | |
913 | connectivity, such as to identify the output link (e.g. physical NIC port) where one of its output ports connected, | |
914 | either directly or indirectly by traversing other pipeline instances. | |
915 | ||
916 | Tracking can be successful or unsuccessful. | |
917 | Typically, tracking for a specific pipeline instance is successful when each one of its input ports can be mapped | |
918 | to a single output port, meaning that all packets read from the current input port can only go out on a single | |
919 | output port. | |
920 | Depending on the pipeline type, some exceptions may be allowed: a small portion of the packets, considered exception | |
921 | packets, are sent out on an output port that is pre-configured for this purpose. | |
922 | ||
923 | For pass-through pipeline type, the tracking is always successful. | |
924 | For pipeline types as flow classification, firewall or routing, the tracking is only successful when the number of | |
925 | output ports for the current pipeline instance is 1. | |
926 | ||
927 | This feature is used by the IP routing pipeline for adding/removing implicit routes every time a link is brought | |
928 | up/down. | |
929 | ||
930 | ||
931 | Table copies | |
932 | ^^^^^^^^^^^^ | |
933 | ||
934 | Fast table copy: pipeline table used by pipeline for the packet processing task, updated through messages, table | |
935 | data structures are optimized for lookup operation. | |
936 | ||
937 | Slow table copy: used by the configuration layer, typically updated through CLI commands, kept in sync with the fast | |
938 | copy (its update triggers the fast copy update). | |
939 | Required for executing advanced table queries without impacting the packet processing task, therefore the slow copy | |
940 | is typically organized using different criteria than the fast copy. | |
941 | ||
942 | Examples: | |
943 | ||
944 | * Flow classification: Search through current set of flows (e.g. list all flows with a specific source IP address); | |
945 | ||
946 | * Firewall: List rules in descending order of priority; | |
947 | ||
948 | * Routing table: List routes sorted by prefix depth and their type (local, remote, default); | |
949 | ||
950 | * ARP: List entries sorted per output interface. | |
951 | ||
952 | ||
953 | Packet meta-data | |
954 | ^^^^^^^^^^^^^^^^ | |
955 | ||
956 | Packet meta-data field offsets provided as argument to pipeline instances are essentially defining the data structure | |
957 | for the packet meta-data used by the current application use-case. | |
958 | It is very useful to put it in the configuration file as a comment in order to facilitate the readability of the | |
959 | configuration file. | |
960 | ||
961 | The reason to use field offsets for defining the data structure for the packet meta-data is due to the C language | |
962 | limitation of not being able to define data structures at run-time. | |
963 | Feature to consider: have the configuration file parser automatically generate and print the data structure defining | |
964 | the packet meta-data for the current application use-case. | |
965 | ||
966 | Packet meta-data typically contains: | |
967 | ||
968 | 1. Pure meta-data: intermediate data per packet that is computed internally, passed between different tables of | |
969 | the same pipeline instance (e.g. lookup key for the ARP table is obtained from the routing table), or between | |
970 | different pipeline instances (e.g. flow ID, traffic metering color, etc); | |
971 | ||
972 | 2. Packet fields: typically, packet header fields that are read directly from the packet, or read from the packet | |
973 | and saved (duplicated) as a working copy at a different location within the packet meta-data (e.g. Diffserv | |
974 | 5-tuple, IP destination address, etc). | |
975 | ||
976 | Several strategies are used to design the packet meta-data, as described in the next subsections. | |
977 | ||
978 | ||
979 | Store packet meta-data in a different cache line as the packet headers | |
980 | """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" | |
981 | ||
982 | This approach is able to support protocols with variable header length, like MPLS, where the offset of IP header | |
983 | from the start of the packet (and, implicitly, the offset of the IP header in the packet buffer) is not fixed. | |
984 | Since the pipelines typically require the specification of a fixed offset to the packet fields (e.g. Diffserv | |
985 | 5-tuple, used by the flow classification pipeline, or the IP destination address, used by the IP routing pipeline), | |
986 | the workaround is to have the packet RX pipeline copy these fields at fixed offsets within the packet meta-data. | |
987 | ||
988 | As this approach duplicates some of the packet fields, it requires accessing more cache lines per packet for filling | |
989 | in selected packet meta-data fields (on RX), as well as flushing selected packet meta-data fields into the | |
990 | packet (on TX). | |
991 | ||
992 | Example: | |
993 | ||
994 | .. code-block:: ini | |
995 | ||
996 | ||
997 | ; struct app_pkt_metadata { | |
998 | ; uint32_t ip_da; | |
999 | ; uint32_t hash; | |
1000 | ; uint32_t flow_id; | |
1001 | ; uint32_t color; | |
1002 | ; } __attribute__((__packed__)); | |
1003 | ; | |
1004 | ||
1005 | [PIPELINE1] | |
1006 | ; Packet meta-data offsets | |
1007 | ip_da_offset = 0; Used by: routing | |
1008 | hash_offset = 4; Used by: RX, flow classification | |
1009 | flow_id_offset = 8; Used by: flow classification, flow actions | |
1010 | color_offset = 12; Used by: flow actions, routing | |
1011 | ||
1012 | ||
1013 | Overlay the packet meta-data in the same cache line with the packet headers | |
1014 | """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" | |
1015 | ||
1016 | This approach is minimizing the number of cache line accessed per packet by storing the packet metadata in the | |
1017 | same cache line with the packet headers. | |
1018 | To enable this strategy, either some headroom is reserved for meta-data at the beginning of the packet headers | |
1019 | cache line (e.g. if 16 bytes are needed for meta-data, then the packet headroom can be set to 128+16 bytes, so | |
1020 | that NIC writes the first byte of the packet at offset 16 from the start of the first packet cache line), | |
1021 | or meta-data is reusing the space of some packet headers that are discarded from the packet (e.g. input Ethernet | |
1022 | header). | |
1023 | ||
1024 | Example: | |
1025 | ||
1026 | .. code-block:: ini | |
1027 | ||
1028 | ; struct app_pkt_metadata { | |
1029 | ; uint8_t headroom[RTE_PKTMBUF_HEADROOM]; /* 128 bytes (default) */ | |
1030 | ; union { | |
1031 | ; struct { | |
1032 | ; struct ether_hdr ether; /* 14 bytes */ | |
1033 | ; struct qinq_hdr qinq; /* 8 bytes */ | |
1034 | ; }; | |
1035 | ; struct { | |
1036 | ; uint32_t hash; | |
1037 | ; uint32_t flow_id; | |
1038 | ; uint32_t color; | |
1039 | ; }; | |
1040 | ; }; | |
1041 | ; struct ipv4_hdr ip; /* 20 bytes */ | |
1042 | ; } __attribute__((__packed__)); | |
1043 | ; | |
1044 | [PIPELINE2] | |
1045 | ; Packet meta-data offsets | |
1046 | qinq_offset = 142; Used by: RX, flow classification | |
1047 | ip_da_offset = 166; Used by: routing | |
1048 | hash_offset = 128; Used by: RX, flow classification | |
1049 | flow_id_offset = 132; Used by: flow classification, flow actions | |
1050 | color_offset = 136; Used by: flow actions, routing | |
1051 | ||
1052 | ||
1053 | List of pipeline types | |
1054 | ~~~~~~~~~~~~~~~~~~~~~~ | |
1055 | ||
1056 | .. _table_ip_pipelines_types: | |
1057 | ||
1058 | .. tabularcolumns:: |p{3cm}|p{5cm}|p{4cm}|p{4cm}| | |
1059 | ||
1060 | .. table:: List of pipeline types provided with the application | |
1061 | ||
1062 | +-----------------------+-----------------------------+-----------------------+------------------------------------------+ | |
1063 | | Name | Table(s) | Actions | Messages | | |
1064 | +=======================+=============================+=======================+==========================================+ | |
1065 | | Pass-through | Passthrough | 1. Pkt metadata build | 1. Ping | | |
1066 | | | | 2. Flow hash | 2. Stats | | |
1067 | | Note: depending on | | 3. Pkt checks | | | |
1068 | | port type, can be | | 4. Load balancing | | | |
1069 | | used for RX, TX, IP | | | | | |
1070 | | fragmentation, IP | | | | | |
1071 | | reassembly or Traffic | | | | | |
1072 | | Management | | | | | |
1073 | +-----------------------+-----------------------------+-----------------------+------------------------------------------+ | |
1074 | | Flow classification | Exact match | 1. Flow ID | 1. Ping | | |
1075 | | | | | | | |
1076 | | | * Key = byte array | 2. Flow stats | 2. Stats | | |
1077 | | | (source: pkt metadata) | 3. Metering | 3. Flow stats | | |
1078 | | | * Data = action dependent | 4. Network Address | 4. Action stats | | |
1079 | | | | 5. Translation (NAT) | 5. Flow add/ update/ delete | | |
1080 | | | | | 6. Default flow add/ update/ delete | | |
1081 | | | | | 7. Action update | | |
1082 | +-----------------------+-----------------------------+-----------------------+------------------------------------------+ | |
1083 | | Flow actions | Array | 1. Flow stats | 1. Ping | | |
1084 | | | | | | | |
1085 | | | * Key = Flow ID | 2. Metering | 2. Stats | | |
1086 | | | (source: pkt metadata) | 3. Network Address | 3. Action stats | | |
1087 | | | * Data = action dependent | 4. Translation (NAT) | 4. Action update | | |
1088 | +-----------------------+-----------------------------+-----------------------+------------------------------------------+ | |
1089 | | Firewall | ACL | 1. Allow/Drop | 1. Ping | | |
1090 | | | | | | | |
1091 | | | * Key = n-tuple | | 2. Stats | | |
1092 | | | (source: pkt headers) | | 3. Rule add/ update/ delete | | |
1093 | | | * Data = none | | 4. Default rule add/ update/ delete | | |
1094 | +-----------------------+-----------------------------+-----------------------+------------------------------------------+ | |
1095 | | IP routing | LPM (IPv4 or IPv6, | 1. TTL decrement and | 1. Ping | | |
1096 | | | depending on pipeline type) | 2. IPv4 checksum | 2. Stats | | |
1097 | | | | | | | |
1098 | | | * Key = IP destination | 3. update | 3. Route add/ update/ delete | | |
1099 | | | (source: pkt metadata) | 4. Header | 4. Default route add/ update/ delete | | |
1100 | | | * Data = Dependent on | 5. encapsulation | 5. ARP entry add/ update/ delete | | |
1101 | | | actions and next hop | 6. (based on next hop | 6. Default ARP entry add/ update/ delete | | |
1102 | | | type | 7. type) | | | |
1103 | | | | | | | |
1104 | | | Hash table (for ARP, only | | | | |
1105 | | | | | | | |
1106 | | | when ARP is enabled) | | | | |
1107 | | | | | | | |
1108 | | | * Key = (Port ID, | | | | |
1109 | | | next hop IP address) | | | | |
1110 | | | (source: pkt meta-data) | | | | |
1111 | | | * Data: MAC address | | | | |
1112 | +-----------------------+-----------------------------+-----------------------+------------------------------------------+ | |
1113 | ||
1114 | ||
1115 | ||
1116 | Command Line Interface (CLI) | |
1117 | ---------------------------- | |
1118 | ||
1119 | Global CLI commands | |
1120 | ~~~~~~~~~~~~~~~~~~~ | |
1121 | ||
1122 | .. _table_ip_pipelines_cli_commands: | |
1123 | ||
1124 | .. tabularcolumns:: |p{3cm}|p{6cm}|p{6cm}| | |
1125 | ||
1126 | .. table:: Global CLI commands | |
1127 | ||
1128 | +---------+---------------------------------------+--------------------------------------------+ | |
1129 | | Command | Description | Syntax | | |
1130 | +=========+=======================================+============================================+ | |
1131 | | run | Run CLI commands script file. | run <file> | | |
1132 | | | | <file> = path to file with CLI commands to | | |
1133 | | | | execute | | |
1134 | +---------+---------------------------------------+--------------------------------------------+ | |
1135 | | quit | Gracefully terminate the application. | quit | | |
1136 | +---------+---------------------------------------+--------------------------------------------+ | |
1137 | ||
1138 | ||
1139 | CLI commands for link configuration | |
1140 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
1141 | ||
1142 | .. _table_ip_pipelines_runtime_config: | |
1143 | ||
1144 | .. tabularcolumns:: |p{3cm}|p{6cm}|p{6cm}| | |
1145 | ||
1146 | .. table:: List of run-time configuration commands for link configuration | |
1147 | ||
1148 | +-------------+--------------------+--------------------------------------------+ | |
1149 | | Command | Description | Syntax | | |
1150 | +=============+====================+============================================+ | |
1151 | | link config | Link configuration | link <link ID> config <IP address> <depth> | | |
1152 | +-------------+--------------------+--------------------------------------------+ | |
1153 | | link up | Link up | link <link ID> up | | |
1154 | +-------------+--------------------+--------------------------------------------+ | |
1155 | | link down | Link down | link <link ID> down | | |
1156 | +-------------+--------------------+--------------------------------------------+ | |
1157 | | link ls | Link list | link ls | | |
1158 | +-------------+--------------------+--------------------------------------------+ | |
1159 | ||
1160 | ||
1161 | CLI commands common for all pipeline types | |
1162 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
1163 | ||
1164 | .. _table_ip_pipelines_mandatory: | |
1165 | ||
1166 | .. tabularcolumns:: |p{3cm}|p{6cm}|p{6cm}| | |
1167 | ||
1168 | .. table:: CLI commands mandatory for all pipelines | |
1169 | ||
1170 | +--------------------+------------------------------------------------------+----------------------------------------------+ | |
1171 | | Command | Description | Syntax | | |
1172 | +====================+======================================================+==============================================+ | |
1173 | | ping | Check whether specific pipeline instance is alive. | p <pipeline ID> ping | | |
1174 | | | The master pipeline sends a ping request | | | |
1175 | | | message to given pipeline instance and waits for | | | |
1176 | | | a response message back. | | | |
1177 | | | Timeout message is displayed when the response | | | |
1178 | | | message is not received before the timer | | | |
1179 | | | expires. | | | |
1180 | +--------------------+------------------------------------------------------+----------------------------------------------+ | |
1181 | | stats | Display statistics for specific pipeline input port, | p <pipeline ID> stats port in <port in ID> | | |
1182 | | | output port or table. | p <pipeline ID> stats port out <port out ID> | | |
1183 | | | | p <pipeline ID> stats table <table ID> | | |
1184 | +--------------------+------------------------------------------------------+----------------------------------------------+ | |
1185 | | input port enable | Enable given input port for specific pipeline | p <pipeline ID> port in <port ID> enable | | |
1186 | | | instance. | | | |
1187 | +--------------------+------------------------------------------------------+----------------------------------------------+ | |
1188 | | input port disable | Disable given input port for specific pipeline | p <pipeline ID> port in <port ID> disable | | |
1189 | | | instance. | | | |
1190 | +--------------------+------------------------------------------------------+----------------------------------------------+ | |
1191 | ||
1192 | Pipeline type specific CLI commands | |
1193 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
1194 | ||
1195 | The pipeline specific CLI commands are part of the pipeline type front-end. |