ceph/src/spdk/dpdk/doc/guides/nics/softnic.rst

   1 ..  SPDX-License-Identifier: BSD-3-Clause
   2     Copyright(c) 2018 Intel Corporation.
   3
   4 Soft NIC Poll Mode Driver
   5 =========================
   6
   7 The Soft NIC allows building custom NIC pipelines in software. The Soft NIC pipeline
   8 is DIY and reconfigurable through ``firmware`` (DPDK Packet Framework script).
   9
  10 The Soft NIC leverages the DPDK Packet Framework libraries (librte_port,
  11 librte_table and librte_pipeline) to make it modular, flexible and extensible
  12 with new functionality. Please refer to DPDK Programmer's Guide, Chapter
  13 ``Packet Framework`` and DPDK Sample Application User Guide,
  14 Chapter ``IP Pipeline Application`` for more details.
  15
  16 The Soft NIC is configured through the standard DPDK ethdev API (ethdev, flow,
  17 QoS, security). The internal framework is not externally visible.
  18
  19 Key benefits:
  20  - Can be used to augment missing features to HW NICs.
  21  - Allows consumption of advanced DPDK features without application redesign.
  22  - Allows out-of-the-box performance boost of DPDK consumers applications simply by
  23    instantiating this type of Ethernet device.
  24
  25 Flow
  26 ----
  27 * ``Device creation``: Each Soft NIC instance is a virtual device.
  28
  29 * ``Device start``: The Soft NIC firmware script is executed every time the device
  30   is started. The firmware script typically creates several internal objects,
  31   such as: memory pools, SW queues, traffic manager, action profiles, pipelines,
  32   etc.
  33
  34 * ``Device stop``: All the internal objects that were previously created by the
  35   firmware script during device start are now destroyed.
  36
  37 * ``Device run``: Each Soft NIC device needs one or several CPU cores to run.
  38   The firmware script maps each internal pipeline to a CPU core. Multiple
  39   pipelines can be mapped to the same CPU core. In order for a given pipeline
  40   assigned to CPU core X to run, the application needs to periodically call on
  41   CPU core X the `rte_pmd_softnic_run()` function for the current Soft NIC
  42   device.
  43
  44 * ``Application run``: The application reads packets from the Soft NIC device RX
  45   queues and writes packets to the Soft NIC device TX queues.
  46
  47 Supported Operating Systems
  48 ---------------------------
  49
  50 Any Linux distribution fulfilling the conditions described in ``System Requirements``
  51 section of :ref:`the DPDK documentation <linux_gsg>` or refer to *DPDK
  52 Release Notes*.
  53
  54 Build options
  55 -------------
  56
  57 The default PMD configuration available in the common_linux configuration file:
  58
  59 CONFIG_RTE_LIBRTE_PMD_SOFTNIC=y
  60
  61 Once the DPDK is built, all the DPDK applications include support for the
  62 Soft NIC PMD.
  63
  64 Soft NIC PMD arguments
  65 ----------------------
  66
  67 The user can specify below arguments in EAL ``--vdev`` options to create the
  68 Soft NIC device instance:
  69
  70         --vdev "net_softnic0,firmware=firmware.cli,conn_port=8086"
  71
  72 #.  ``firmware``: path to the firmware script used for Soft NIC configuration.
  73     The example "firmware" script is provided at `drivers/net/softnic/`.
  74     (Optional: No, Default = NA)
  75
  76 #.  ``conn_port``: tcp connection port (non-zero value) used by remote client
  77     (for examples- telnet, netcat, etc.) to connect and configure Soft NIC device in run-time.
  78     (Optional: yes, Default value: 0, no connection with external client)
  79
  80 #.  ``cpu_id``: numa node id. (Optional: yes, Default value: 0)
  81
  82 #.  ``tm_n_queues``: number of traffic manager's scheduler queues. The traffic manager
  83     is based on DPDK *librte_sched* library. (Optional: yes, Default value: 65,536 queues)
  84
  85 #.  ``tm_qsize0``: size of scheduler queue 0 per traffic class of the pipes/subscribers.
  86     (Optional: yes, Default: 64)
  87
  88 #.  ``tm_qsize1``: size of scheduler queue 1 per traffic class of the pipes/subscribers.
  89     (Optional: yes, Default: 64)
  90
  91 #.  ``tm_qsize2``: size of scheduler queue 2 per traffic class of the pipes/subscribers.
  92     (Optional: yes, Default: 64)
  93
  94 #.  ``tm_qsize3``: size of scheduler queue 3 per traffic class of the pipes/subscribers.
  95     (Optional: yes, Default: 64)
  96
  97
  98 Soft NIC testing
  99 ----------------
 100
 101 * Run testpmd application in Soft NIC forwarding mode with loopback feature
 102   enabled on Soft NIC port:
 103
 104     .. code-block:: console
 105
 106          ./testpmd -c 0x3 --vdev 'net_softnic0,firmware=<script path>/firmware.cli,cpu_id=0,conn_port=8086' -- -i
 107               --forward-mode=softnic --portmask=0x2
 108
 109     .. code-block:: console
 110
 111         ...
 112         Interactive-mode selected
 113         Set softnic packet forwarding mode
 114         ...
 115         Configuring Port 0 (socket 0)
 116         Port 0: 90:E2:BA:37:9D:DC
 117         Configuring Port 1 (socket 0)
 118
 119         ; SPDX-License-Identifier: BSD-3-Clause
 120         ; Copyright(c) 2018 Intel Corporation
 121
 122         link LINK dev 0000:02:00.0
 123
 124         pipeline RX period 10 offset_port_id 0
 125         pipeline RX port in bsz 32 link LINK rxq 0
 126         pipeline RX port out bsz 32 swq RXQ0
 127         pipeline RX table match stub
 128         pipeline RX port in 0 table 0
 129
 130         pipeline TX period 10 offset_port_id 0
 131         pipeline TX port in bsz 32 swq TXQ0
 132         pipeline TX port out bsz 32 link LINK txq 0
 133         pipeline TX table match stub
 134         pipeline TX port in 0 table 0
 135
 136         thread 1 pipeline RX enable
 137         thread 1 pipeline TX enable
 138         Port 1: 00:00:00:00:00:00
 139         Checking link statuses...
 140         Done
 141         testpmd>
 142
 143 * Start forwarding
 144
 145     .. code-block:: console
 146
 147          testpmd> start
 148          softnic packet forwarding - ports=1 - cores=1 - streams=1 - NUMA support enabled, MP over anonymous pages disabled
 149          Logical Core 1 (socket 0) forwards packets on 1 streams:
 150          RX P=2/Q=0 (socket 0) -> TX P=2/Q=0 (socket 0) peer=02:00:00:00:00:02
 151
 152          softnic packet forwarding packets/burst=32
 153          nb forwarding cores=1 - nb forwarding ports=1
 154          port 0: RX queue number: 1 Tx queue number: 1
 155          Rx offloads=0x1000 Tx offloads=0x0
 156          RX queue: 0
 157          RX desc=512 - RX free threshold=32
 158          RX threshold registers: pthresh=8 hthresh=8  wthresh=0
 159          RX Offloads=0x0
 160          TX queue: 0
 161          TX desc=512 - TX free threshold=32
 162          TX threshold registers: pthresh=32 hthresh=0  wthresh=0
 163          TX offloads=0x0 - TX RS bit threshold=32
 164          port 1: RX queue number: 1 Tx queue number: 1
 165          Rx offloads=0x0 Tx offloads=0x0
 166          RX queue: 0
 167          RX desc=0 - RX free threshold=0
 168          RX threshold registers: pthresh=0 hthresh=0  wthresh=0
 169          RX Offloads=0x0
 170          TX queue: 0
 171          TX desc=0 - TX free threshold=0
 172          TX threshold registers: pthresh=0 hthresh=0  wthresh=0
 173          TX offloads=0x0 - TX RS bit threshold=0
 174
 175 * Start remote client (e.g. telnet) to communicate with the softnic device:
 176
 177     .. code-block:: console
 178
 179         $ telnet 127.0.0.1 8086
 180         Trying 127.0.0.1...
 181         Connected to 127.0.0.1.
 182         Escape character is '^]'.
 183
 184         Welcome to Soft NIC!
 185
 186         softnic>
 187
 188 * Add/update Soft NIC pipeline table match-action entries from telnet client:
 189
 190     .. code-block:: console
 191
 192         softnic> pipeline RX table 0 rule add match default action fwd port 0
 193         softnic> pipeline TX table 0 rule add match default action fwd port 0
 194
 195 Soft NIC Firmware
 196 -----------------
 197
 198 The Soft NIC firmware, for example- `softnic/firmware.cli`, consists of following CLI commands
 199 for creating and managing software based NIC pipelines. For more details, please refer to CLI
 200 command description provided in `softnic/rte_eth_softnic_cli.c`.
 201
 202 * Physical port for packets send/receive:
 203
 204     .. code-block:: console
 205
 206         link LINK dev 0000:02:00.0
 207
 208 * Pipeline create:
 209
 210     .. code-block:: console
 211
 212         pipeline RX period 10 offset_port_id 0           (Soft NIC rx-path pipeline)
 213         pipeline TX period 10 offset_port_id 0           (Soft NIC tx-path pipeline)
 214
 215 * Pipeline input/output port create
 216
 217     .. code-block:: console
 218
 219         pipeline RX port in bsz 32 link LINK rxq 0      (Soft NIC rx pipeline input port)
 220         pipeline RX port out bsz 32 swq RXQ0            (Soft NIC rx pipeline output port)
 221         pipeline TX port in bsz 32 swq TXQ0             (Soft NIC tx pipeline input port)
 222         pipeline TX port out bsz 32 link LINK txq 0     (Soft NIC tx pipeline output port)
 223
 224 * Pipeline table create
 225
 226     .. code-block:: console
 227
 228         pipeline RX table match stub             (Soft NIC rx pipeline match-action table)
 229         pipeline TX table match stub             (Soft NIC tx pipeline match-action table)
 230
 231 * Pipeline input port connection with table
 232
 233     .. code-block:: console
 234
 235         pipeline RX port in 0 table 0          (Soft NIC rx pipeline input port 0 connection with table 0)
 236         pipeline TX port in 0 table 0          (Soft NIC tx pipeline input port 0 connection with table 0)
 237
 238 * Pipeline table match-action rules add
 239
 240     .. code-block:: console
 241
 242         pipeline RX table 0 rule add match default action fwd port 0        (Soft NIC rx pipeline table 0 rule)
 243         pipeline TX table 0 rule add match default action fwd port 0        (Soft NIC tx pipeline table 0 rule)
 244
 245 * Enable pipeline on CPU thread
 246
 247     .. code-block:: console
 248
 249         thread 1 pipeline RX enable        (Soft NIC rx pipeline enable on cpu thread id 1)
 250         thread 1 pipeline TX enable        (Soft NIC tx pipeline enable on cpu thread id 1)
 251
 252 QoS API Support:
 253 ----------------
 254
 255 SoftNIC PMD implements ethdev traffic management APIs ``rte_tm.h`` that
 256 allow building and committing traffic manager hierarchy, configuring hierarchy
 257 nodes of the Quality of Service (QoS) scheduler supported by DPDK librte_sched
 258 library. Furthermore, APIs for run-time update to the traffic manager hierarchy
 259 are supported by PMD.
 260
 261 SoftNIC PMD also implements ethdev traffic metering and policing APIs
 262 ``rte_mtr.h`` that enables metering and marking of the packets with the
 263 appropriate color (green, yellow or red), according to the traffic metering
 264 algorithm. For the meter output color, policer actions like
 265 `keep the packet color same`, `change the packet color` or `drop the packet`
 266 can be configured.
 267
 268 .. Note::
 269
 270     The SoftNIC does not support the meter object shared by several flows,
 271     thus only supports creating meter object private to the flow. Once meter
 272     object is successfully created, it can be linked to the specific flow by
 273     specifying the ``meter`` flow action in the flow rule.
 274
 275 Flow API support:
 276 -----------------
 277
 278 The SoftNIC PMD implements ethdev flow APIs ``rte_flow.h`` that allow validating
 279 flow rules, adding flow rules to the SoftNIC pipeline as table rules, deleting
 280 and querying the flow rules. The PMD provides new cli command for creating the
 281 flow group and their mapping to the SoftNIC pipeline and table. This cli should
 282 be configured as part of firmware file.
 283
 284     .. code-block:: console
 285
 286         flowapi map group <group_id> ingress | egress pipeline <pipeline_name> \
 287             table <table_id>
 288
 289 From the flow attributes of the flow, PMD uses the group id to get the mapped
 290 pipeline and table. PMD supports number of flow actions such as
 291 ``JMP, QUEUE, RSS, DROP, COUNT, METER, VXLAN`` etc.
 292
 293 .. Note::
 294
 295     The flow must have one terminating actions i.e.
 296     ``JMP or RSS or QUEUE or DROP``. For the count and drop actions the
 297     underlying PMD doesn't support the functionality yet. So it is not
 298     recommended for use.
 299
 300 The flow API can be tested with the help of testpmd application. The SoftNIC
 301 firmware specifies CLI commands for port configuration, pipeline creation,
 302 action profile creation and table creation. Once application gets initialized,
 303 the flow rules can be added through the testpmd CLI.
 304 The PMD will translate the flow rules to the SoftNIC pipeline tables rules.
 305
 306 Example:
 307 ~~~~~~~~
 308 Example demonstrates the flow queue action using the SoftNIC firmware and testpmd
 309 commands.
 310
 311 * Prepare SoftNIC firmware
 312
 313     .. code-block:: console
 314
 315         link LINK0 dev 0000:83:00.0
 316         link LINK1 dev 0000:81:00.0
 317         pipeline RX period 10 offset_port_id 0
 318         pipeline RX port in bsz 32 link LINK0 rxq 0
 319         pipeline RX port in bsz 32 link LINK1 rxq 0
 320         pipeline RX port out bsz 32 swq RXQ0
 321         pipeline RX port out bsz 32 swq RXQ1
 322         table action profile AP0 ipv4 offset 278 fwd
 323         pipeline RX table match hash ext key 16 mask
 324             00FF0000FFFFFFFFFFFFFFFFFFFFFFFF \
 325             offset 278 buckets 16K size 65K action AP0
 326         pipeline RX port in 0 table 0
 327         pipeline RX port in 1 table 0
 328         flowapi map group 0 ingress pipeline RX table 0
 329         pipeline TX period 10 offset_port_id 0
 330         pipeline TX port in bsz 32 swq TXQ0
 331         pipeline TX port in bsz 32 swq TXQ1
 332         pipeline TX port out bsz 32 link LINK0 txq 0
 333         pipeline TX port out bsz 32 link LINK1 txq 0
 334         pipeline TX table match hash ext key 16 mask
 335             00FF0000FFFFFFFFFFFFFFFFFFFFFFFF \
 336             offset 278 buckets 16K size 65K action AP0
 337         pipeline TX port in 0 table 0
 338         pipeline TX port in 1 table 0
 339         pipeline TX table 0 rule add match hash ipv4_5tuple
 340             1.10.11.12 2.20.21.22 100 200 6 action fwd port 0
 341         pipeline TX table 0 rule add match hash ipv4_5tuple
 342             1.10.11.13 2.20.21.23 100 200 6 action fwd port 1
 343         thread 25 pipeline RX enable
 344         thread 25 pipeline TX enable
 345
 346 * Run testpmd:
 347
 348     .. code-block:: console
 349
 350         ./x86_64-native-linux-gcc/app/testpmd -l 23-25  -n 4 \
 351                                     --vdev 'net_softnic0, \
 352                                     firmware=./drivers/net/softnic/ \
 353                                         firmware.cli, \
 354                                     cpu_id=1,conn_port=8086' -- \
 355                                     -i --forward-mode=softnic --rxq=2, \
 356                                     --txq=2, --disable-rss --portmask=0x4
 357
 358 * Configure flow rules on softnic:
 359
 360     .. code-block:: console
 361
 362         flow create 2 group 0 ingress pattern eth / ipv4 proto mask 255 src \
 363             mask 255.255.255.255 dst mask  255.255.255.255 src spec
 364             1.10.11.12 dst spec 2.20.21.22 proto spec 6 / tcp src mask 65535 \
 365             dst mask 65535 src spec 100 dst spec 200 / end actions queue \
 366             index 0 / end
 367         flow create 2 group 0 ingress pattern eth / ipv4 proto mask 255 src \
 368             mask 255.255.255.255 dst mask  255.255.255.255 src spec 1.10.11.13 \
 369             dst spec 2.20.21.23 proto spec 6 / tcp src mask 65535 dst mask \
 370             65535 src spec 100 dst spec 200 / end actions queue index 1 / end