ceph/src/dpdk/doc/guides/nics/qede.rst

   1 ..  BSD LICENSE
   2     Copyright (c) 2016 QLogic Corporation
   3     All rights reserved.
   4
   5     Redistribution and use in source and binary forms, with or without
   6     modification, are permitted provided that the following conditions
   7     are met:
   8
   9     * Redistributions of source code must retain the above copyright
  10     notice, this list of conditions and the following disclaimer.
  11     * Redistributions in binary form must reproduce the above copyright
  12     notice, this list of conditions and the following disclaimer in
  13     the documentation and/or other materials provided with the
  14     distribution.
  15     * Neither the name of QLogic Corporation nor the names of its
  16     contributors may be used to endorse or promote products derived
  17     from this software without specific prior written permission.
  18
  19     THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  20     "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
  21     LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
  22     A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
  23     OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
  24     SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
  25     LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
  26     DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
  27     THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  28     (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
  29     OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  30
  31 QEDE Poll Mode Driver
  32 ======================
  33
  34 The QEDE poll mode driver library (**librte_pmd_qede**) implements support
  35 for **QLogic FastLinQ QL4xxxx 25G/40G/100G CNA** family of adapters as well
  36 as their virtual functions (VF) in SR-IOV context. It is supported on
  37 several standard Linux distros like RHEL7.x, SLES12.x and Ubuntu.
  38 It is compile-tested under FreeBSD OS.
  39
  40 More information can be found at `QLogic Corporation's Website
  41 <http://www.qlogic.com>`_.
  42
  43 Supported Features
  44 ------------------
  45
  46 - Unicast/Multicast filtering
  47 - Promiscuous mode
  48 - Allmulti mode
  49 - Port hardware statistics
  50 - Jumbo frames
  51 - VLAN offload - Filtering and stripping
  52 - Stateless checksum offloads (IPv4/TCP/UDP)
  53 - Multiple Rx/Tx queues
  54 - RSS (with RETA/hash table/key)
  55 - TSS
  56 - Multiple MAC address
  57 - Default pause flow control
  58 - SR-IOV VF
  59 - MTU change
  60 - Multiprocess aware
  61 - Scatter-Gather
  62
  63 Non-supported Features
  64 ----------------------
  65
  66 - SR-IOV PF
  67 - Tunneling offloads
  68 - LRO/TSO
  69 - NPAR
  70
  71 Supported QLogic Adapters
  72 -------------------------
  73
  74 - QLogic FastLinQ QL4xxxx 10G/25G/40G/100G CNAs.
  75
  76 Prerequisites
  77 -------------
  78
  79 - Requires firmware version **8.10.x.** and management firmware
  80   version **8.10.x or higher**. Firmware may be available
  81   inbox in certain newer Linux distros under the standard directory
  82   ``E.g. /lib/firmware/qed/qed_init_values-8.10.9.0.bin``
  83
  84 - If the required firmware files are not available then visit
  85   `QLogic Driver Download Center <http://driverdownloads.qlogic.com>`_.
  86
  87 Performance note
  88 ~~~~~~~~~~~~~~~~
  89
  90 - For better performance, it is recommended to use 4K or higher RX/TX rings.
  91
  92 Config File Options
  93 ~~~~~~~~~~~~~~~~~~~
  94
  95 The following options can be modified in the ``.config`` file. Please note that
  96 enabling debugging options may affect system performance.
  97
  98 - ``CONFIG_RTE_LIBRTE_QEDE_PMD`` (default **y**)
  99
 100   Toggle compilation of QEDE PMD driver.
 101
 102 - ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_INFO`` (default **n**)
 103
 104   Toggle display of generic debugging messages.
 105
 106 - ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_DRIVER`` (default **n**)
 107
 108   Toggle display of ecore related messages.
 109
 110 - ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_TX`` (default **n**)
 111
 112   Toggle display of transmit fast path run-time messages.
 113
 114 - ``CONFIG_RTE_LIBRTE_QEDE_DEBUG_RX`` (default **n**)
 115
 116   Toggle display of receive fast path run-time messages.
 117
 118 - ``CONFIG_RTE_LIBRTE_QEDE_FW`` (default **""**)
 119
 120   Gives absolute path of firmware file.
 121   ``Eg: "/lib/firmware/qed/qed_init_values_zipped-8.10.9.0.bin"``
 122   Empty string indicates driver will pick up the firmware file
 123   from the default location.
 124
 125 Driver Compilation
 126 ~~~~~~~~~~~~~~~~~~
 127
 128 To compile QEDE PMD for Linux x86_64 gcc target, run the following ``make``
 129 command::
 130
 131    cd <DPDK-source-directory>
 132    make config T=x86_64-native-linuxapp-gcc install
 133
 134 To compile QEDE PMD for Linux x86_64 clang target, run the following ``make``
 135 command::
 136
 137    cd <DPDK-source-directory>
 138    make config T=x86_64-native-linuxapp-clang install
 139
 140 To compile QEDE PMD for FreeBSD x86_64 clang target, run the following ``gmake``
 141 command::
 142
 143    cd <DPDK-source-directory>
 144    gmake config T=x86_64-native-bsdapp-clang install
 145
 146 To compile QEDE PMD for FreeBSD x86_64 gcc target, run the following ``gmake``
 147 command::
 148
 149    cd <DPDK-source-directory>
 150    gmake config T=x86_64-native-bsdapp-gcc install -Wl,-rpath=\
 151                                         /usr/local/lib/gcc49 CC=gcc49
 152
 153
 154 Sample Application Notes
 155 ~~~~~~~~~~~~~~~~~~~~~~~~
 156
 157 This section demonstrates how to launch ``testpmd`` with QLogic 4xxxx
 158 devices managed by ``librte_pmd_qede`` in Linux operating system.
 159
 160 #. Request huge pages:
 161
 162    .. code-block:: console
 163
 164       echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages/ \
 165                                                                 nr_hugepages
 166
 167 #. Load ``igb_uio`` driver:
 168
 169    .. code-block:: console
 170
 171       insmod ./x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
 172
 173 #. Bind the QLogic 4xxxx adapters to ``igb_uio`` loaded in the
 174    previous step:
 175
 176    .. code-block:: console
 177
 178       ./tools/dpdk-devbind.py --bind igb_uio 0000:84:00.0 0000:84:00.1 \
 179                                               0000:84:00.2 0000:84:00.3
 180
 181 #. Start ``testpmd`` with basic parameters:
 182    (Enable QEDE_DEBUG_INFO=y to view informational messages)
 183
 184    .. code-block:: console
 185
 186       testpmd -c 0xff1 -n 4 -- -i --nb-cores=8 --portmask=0xf --rxd=4096 \
 187       --txd=4096 --txfreet=4068 --enable-rx-cksum --rxq=4 --txq=4 \
 188       --rss-ip --rss-udp
 189
 190       [...]
 191
 192     EAL: PCI device 0000:84:00.0 on NUMA socket 1
 193     EAL:   probe driver: 1077:1634 rte_qede_pmd
 194     EAL:   Not managed by a supported kernel driver, skipped
 195     EAL: PCI device 0000:84:00.1 on NUMA socket 1
 196     EAL:   probe driver: 1077:1634 rte_qede_pmd
 197     EAL:   Not managed by a supported kernel driver, skipped
 198     EAL: PCI device 0000:88:00.0 on NUMA socket 1
 199     EAL:   probe driver: 1077:1656 rte_qede_pmd
 200     EAL:   PCI memory mapped at 0x7f738b200000
 201     EAL:   PCI memory mapped at 0x7f738b280000
 202     EAL:   PCI memory mapped at 0x7f738b300000
 203     PMD: Chip details : BB1
 204     PMD: Driver version : QEDE PMD 8.7.9.0_1.0.0
 205     PMD: Firmware version : 8.7.7.0
 206     PMD: Management firmware version : 8.7.8.0
 207     PMD: Firmware file : /lib/firmware/qed/qed_init_values_zipped-8.7.7.0.bin
 208     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_common_dev_init:macaddr \
 209                                                         00:0e:1e:d2:09:9c
 210       [...]
 211     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 0 num_desc 4096 \
 212                                                 tx_free_thresh 4068 socket 0
 213     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 1 num_desc 4096 \
 214                                                 tx_free_thresh 4068 socket 0
 215     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 2 num_desc 4096 \
 216                                                  tx_free_thresh 4068 socket 0
 217     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_tx_queue_setup:txq 3 num_desc 4096 \
 218                                                  tx_free_thresh 4068 socket 0
 219     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 0 num_desc 4096 \
 220                                                 rx_buf_size=2148 socket 0
 221     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 1 num_desc 4096 \
 222                                                 rx_buf_size=2148 socket 0
 223     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 2 num_desc 4096 \
 224                                                 rx_buf_size=2148 socket 0
 225     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_rx_queue_setup:rxq 3 num_desc 4096 \
 226                                                 rx_buf_size=2148 socket 0
 227     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_dev_start:port 0
 228     [QEDE PMD: (84:00.0:dpdk-port-0)]qede_dev_start:link status: down
 229       [...]
 230     Checking link statuses...
 231     Port 0 Link Up - speed 25000 Mbps - full-duplex
 232     Port 1 Link Up - speed 25000 Mbps - full-duplex
 233     Port 2 Link Up - speed 25000 Mbps - full-duplex
 234     Port 3 Link Up - speed 25000 Mbps - full-duplex
 235     Done
 236     testpmd>
 237
 238
 239 SR-IOV: Prerequisites and Sample Application Notes
 240 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 241
 242 This section provides instructions to configure SR-IOV with Linux OS.
 243
 244 **Note**: librte_pmd_qede will be used to bind to SR-IOV VF device and Linux native kernel driver (QEDE) will function as SR-IOV PF driver. Requires PF driver to be 8.10.x.x or higher.
 245
 246 #. Verify SR-IOV and ARI capability is enabled on the adapter using ``lspci``:
 247
 248    .. code-block:: console
 249
 250       lspci -s <slot> -vvv
 251
 252    Example output:
 253
 254    .. code-block:: console
 255
 256       [...]
 257       Capabilities: [1b8 v1] Alternative Routing-ID Interpretation (ARI)
 258       [...]
 259       Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV)
 260       [...]
 261       Kernel driver in use: igb_uio
 262
 263 #. Load the kernel module:
 264
 265    .. code-block:: console
 266
 267       modprobe qede
 268
 269    Example output:
 270
 271    .. code-block:: console
 272
 273       systemd-udevd[4848]: renamed network interface eth0 to ens5f0
 274       systemd-udevd[4848]: renamed network interface eth1 to ens5f1
 275
 276 #. Bring up the PF ports:
 277
 278    .. code-block:: console
 279
 280       ifconfig ens5f0 up
 281       ifconfig ens5f1 up
 282
 283 #. Create VF device(s):
 284
 285    Echo the number of VFs to be created into ``"sriov_numvfs"`` sysfs entry
 286    of the parent PF.
 287
 288    Example output:
 289
 290    .. code-block:: console
 291
 292       echo 2 > /sys/devices/pci0000:00/0000:00:03.0/0000:81:00.0/sriov_numvfs
 293
 294
 295 #. Assign VF MAC address:
 296
 297    Assign MAC address to the VF using iproute2 utility. The syntax is::
 298
 299       ip link set <PF iface> vf <VF id> mac <macaddr>
 300
 301    Example output:
 302
 303    .. code-block:: console
 304
 305       ip link set ens5f0 vf 0 mac 52:54:00:2f:9d:e8
 306
 307
 308 #. PCI Passthrough:
 309
 310    The VF devices may be passed through to the guest VM using ``virt-manager`` or
 311    ``virsh``. QEDE PMD should be used to bind the VF devices in the guest VM
 312    using the instructions outlined in the Application notes above.