INSTALL.DPDK.md

   1 OVS DPDK INSTALL GUIDE
   2 ================================
   3
   4 ## Contents
   5
   6 1. [Overview](#overview)
   7 2. [Building and Installation](#build)
   8 3. [Setup OVS DPDK datapath](#ovssetup)
   9 4. [DPDK in the VM](#builddpdk)
  10 5. [OVS Testcases](#ovstc)
  11 6. [Limitations ](#ovslimits)
  12
  13 ## <a name="overview"></a> 1. Overview
  14
  15 Open vSwitch can use DPDK lib to operate entirely in userspace.
  16 This file provides information on installation and use of Open vSwitch
  17 using DPDK datapath.  This version of Open vSwitch should be built manually
  18 with `configure` and `make`.
  19
  20 The DPDK support of Open vSwitch is considered 'experimental'.
  21
  22 ### Prerequisites
  23
  24 * Required: DPDK 16.07
  25 * Hardware: [DPDK Supported NICs] when physical ports in use
  26
  27 ## <a name="build"></a> 2. Building and Installation
  28
  29 ### 2.1 Configure & build the Linux kernel
  30
  31 On Linux Distros running kernel version >= 3.0, kernel rebuild is not required
  32 and only grub cmdline needs to be updated for enabling IOMMU [VFIO support - 3.2].
  33 For older kernels, check if kernel is built with  UIO, HUGETLBFS, PROC_PAGE_MONITOR,
  34 HPET, HPET_MMAP support.
  35
  36 Detailed system requirements can be found at [DPDK requirements] and also refer to
  37 advanced install guide [INSTALL.DPDK-ADVANCED.md]
  38
  39 ### 2.2 Install DPDK
  40   1. [Download DPDK] and extract the file, for example in to /usr/src
  41      and set DPDK_DIR
  42
  43      ```
  44      cd /usr/src/
  45      wget http://dpdk.org/browse/dpdk/snapshot/dpdk-16.07.zip
  46      unzip dpdk-16.07.zip
  47
  48      export DPDK_DIR=/usr/src/dpdk-16.07
  49      cd $DPDK_DIR
  50      ```
  51
  52   2. Configure and Install DPDK
  53
  54      Build and install the DPDK library.
  55
  56      ```
  57      export DPDK_TARGET=x86_64-native-linuxapp-gcc
  58      export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET
  59      make install T=$DPDK_TARGET DESTDIR=install
  60      ```
  61
  62      Note: For IVSHMEM, Set `export DPDK_TARGET=x86_64-ivshmem-linuxapp-gcc`
  63
  64 ### 2.3 Install OVS
  65   OVS can be installed using different methods. For OVS to use DPDK datapath,
  66   it has to be configured with DPDK support and is done by './configure --with-dpdk'.
  67   This section focus on generic recipe that suits most cases and for distribution
  68   specific instructions, refer [INSTALL.Fedora.md], [INSTALL.RHEL.md] and
  69   [INSTALL.Debian.md].
  70
  71   The OVS sources can be downloaded in different ways and skip this section
  72   if already having the correct sources. Otherwise download the correct version using
  73   one of the below suggested methods and follow the documentation of that specific
  74   version.
  75
  76   - OVS stable releases can be downloaded in compressed format from [Download OVS]
  77
  78      ```
  79      cd /usr/src
  80      wget http://openvswitch.org/releases/openvswitch-<version>.tar.gz
  81      tar -zxvf openvswitch-<version>.tar.gz
  82      export OVS_DIR=/usr/src/openvswitch-<version>
  83      ```
  84
  85   - OVS current development can be clone using 'git' tool
  86
  87      ```
  88      cd /usr/src/
  89      git clone https://github.com/openvswitch/ovs.git
  90      export OVS_DIR=/usr/src/ovs
  91      ```
  92
  93   - Install OVS dependencies
  94
  95      GNU make, GCC 4.x (or) Clang 3.4, libnuma (Mandatory)
  96      libssl, libcap-ng, Python 2.7 (Optional)
  97      More information can be found at [Build Requirements]
  98
  99   - Configure, Install OVS
 100
 101      ```
 102      cd $OVS_DIR
 103      ./boot.sh
 104      ./configure --with-dpdk=$DPDK_BUILD
 105      make install
 106      ```
 107
 108      Note: Passing DPDK_BUILD can be skipped if DPDK library is installed in
 109      standard locations i.e `./configure --with-dpdk` should suffice.
 110
 111   Additional information can be found in [INSTALL.md].
 112
 113 ## <a name="ovssetup"></a> 3. Setup OVS with DPDK datapath
 114
 115 ### 3.1 Setup Hugepages
 116
 117   Allocate and mount 2M Huge pages:
 118
 119   - For persistent allocation of huge pages, write to hugepages.conf file
 120     in /etc/sysctl.d
 121
 122     `echo 'vm.nr_hugepages=2048' > /etc/sysctl.d/hugepages.conf`
 123
 124   - For run-time allocation of huge pages
 125
 126     `sysctl -w vm.nr_hugepages=N` where N = No. of 2M huge pages allocated
 127
 128   - To verify hugepage configuration
 129
 130     `grep HugePages_ /proc/meminfo`
 131
 132   - Mount hugepages
 133
 134     `mount -t hugetlbfs none /dev/hugepages`
 135
 136     Note: Mount hugepages if not already mounted by default.
 137
 138 ### 3.2 Setup DPDK devices using VFIO
 139
 140   - Supported with kernel version >= 3.6
 141   - VFIO needs support from BIOS and kernel.
 142   - BIOS changes:
 143
 144     Enable VT-d, can be verified from `dmesg | grep -e DMAR -e IOMMU` output
 145
 146   - GRUB bootline:
 147
 148     Add `iommu=pt intel_iommu=on`, can be verified from `cat /proc/cmdline` output
 149
 150   - Load modules and bind the NIC to VFIO driver
 151
 152     ```
 153     modprobe vfio-pci
 154     sudo /usr/bin/chmod a+x /dev/vfio
 155     sudo /usr/bin/chmod 0666 /dev/vfio/*
 156     $DPDK_DIR/tools/dpdk-devbind.py --bind=vfio-pci eth1
 157     $DPDK_DIR/tools/dpdk-devbind.py --status
 158     ```
 159
 160   Note: If running kernels < 3.6 UIO drivers to be used,
 161   please check [DPDK in the VM], DPDK devices using UIO section for the steps.
 162
 163 ### 3.3 Setup OVS
 164
 165   1. DB creation (One time step)
 166
 167      ```
 168      mkdir -p /usr/local/etc/openvswitch
 169      mkdir -p /usr/local/var/run/openvswitch
 170      rm /usr/local/etc/openvswitch/conf.db
 171      ovsdb-tool create /usr/local/etc/openvswitch/conf.db  \
 172             /usr/local/share/openvswitch/vswitch.ovsschema
 173      ```
 174
 175   2. Start ovsdb-server
 176
 177      No SSL support
 178
 179      ```
 180      ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
 181          --remote=db:Open_vSwitch,Open_vSwitch,manager_options \
 182          --pidfile --detach
 183      ```
 184
 185      SSL support
 186
 187      ```
 188      ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
 189          --remote=db:Open_vSwitch,Open_vSwitch,manager_options \
 190          --private-key=db:Open_vSwitch,SSL,private_key \
 191          --certificate=Open_vSwitch,SSL,certificate \
 192          --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach
 193      ```
 194
 195   3. Initialize DB (One time step)
 196
 197      ```
 198      ovs-vsctl --no-wait init
 199      ```
 200
 201   4. Start vswitchd
 202
 203      DPDK configuration arguments can be passed to vswitchd via Open_vSwitch
 204      'other_config' column. The important configuration options are listed below.
 205      Defaults will be provided for all values not explicitly set. Refer
 206      ovs-vswitchd.conf.db(5) for additional information on configuration options.
 207
 208      * dpdk-init
 209      Specifies whether OVS should initialize and support DPDK ports. This is
 210      a boolean, and defaults to false.
 211
 212      * dpdk-lcore-mask
 213      Specifies the CPU cores on which dpdk lcore threads should be spawned and
 214      expects hex string (eg '0x123').
 215
 216      * dpdk-socket-mem
 217      Comma separated list of memory to pre-allocate from hugepages on specific
 218      sockets.
 219
 220      * dpdk-hugepage-dir
 221      Directory where hugetlbfs is mounted
 222
 223      * vhost-sock-dir
 224      Option to set the path to the vhost_user unix socket files.
 225
 226      NOTE: Changing any of these options requires restarting the ovs-vswitchd
 227      application.
 228
 229      Open vSwitch can be started as normal. DPDK will be initialized as long
 230      as the dpdk-init option has been set to 'true'.
 231
 232      ```
 233      export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
 234      ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
 235      ovs-vswitchd unix:$DB_SOCK --pidfile --detach
 236      ```
 237
 238      If allocated more than one GB hugepage (as for IVSHMEM), set amount and
 239      use NUMA node 0 memory. For details on using ivshmem with DPDK, refer to
 240      [OVS Testcases].
 241
 242      ```
 243      ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="1024,0"
 244      ovs-vswitchd unix:$DB_SOCK --pidfile --detach
 245      ```
 246
 247      To better scale the work loads across cores, Multiple pmd threads can be
 248      created and pinned to CPU cores by explicity specifying pmd-cpu-mask.
 249      eg: To spawn 2 pmd threads and pin them to cores 1, 2
 250
 251      ```
 252      ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=6
 253      ```
 254
 255   5. Create bridge & add DPDK devices
 256
 257      create a bridge with datapath_type "netdev" in the configuration database
 258
 259      `ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev`
 260
 261      Now you can add DPDK devices. OVS expects DPDK device names to start with
 262      "dpdk" and end with a portid. vswitchd should print (in the log file) the
 263      number of dpdk devices found.
 264
 265      ```
 266      ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
 267      ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk
 268      ```
 269
 270      After the DPDK ports get added to switch, a polling thread continuously polls
 271      DPDK devices and consumes 100% of the core as can be checked from 'top' and 'ps' cmds.
 272
 273      ```
 274      top -H
 275      ps -eLo pid,psr,comm | grep pmd
 276      ```
 277
 278      Note: creating bonds of DPDK interfaces is slightly different to creating
 279      bonds of system interfaces.  For DPDK, the interface type must be explicitly
 280      set, for example:
 281
 282      ```
 283      ovs-vsctl add-bond br0 dpdkbond dpdk0 dpdk1 -- set Interface dpdk0 type=dpdk -- set Interface dpdk1 type=dpdk
 284      ```
 285
 286   6. PMD thread statistics
 287
 288      ```
 289      # Check current stats
 290        ovs-appctl dpif-netdev/pmd-stats-show
 291
 292      # Clear previous stats
 293        ovs-appctl dpif-netdev/pmd-stats-clear
 294      ```
 295
 296   7. Port/rxq assigment to PMD threads
 297
 298      ```
 299      # Show port/rxq assignment
 300        ovs-appctl dpif-netdev/pmd-rxq-show
 301      ```
 302
 303      To change default rxq assignment to pmd threads rxqs may be manually
 304      pinned to desired cores using:
 305
 306      ```
 307      ovs-vsctl set Interface <iface> \
 308                other_config:pmd-rxq-affinity=<rxq-affinity-list>
 309      ```
 310      where:
 311
 312      ```
 313      <rxq-affinity-list> ::= NULL | <non-empty-list>
 314      <non-empty-list> ::= <affinity-pair> |
 315                           <affinity-pair> , <non-empty-list>
 316      <affinity-pair> ::= <queue-id> : <core-id>
 317      ```
 318
 319      Example:
 320
 321      ```
 322      ovs-vsctl set interface dpdk0 options:n_rxq=4 \
 323                other_config:pmd-rxq-affinity="0:3,1:7,3:8"
 324
 325      Queue #0 pinned to core 3;
 326      Queue #1 pinned to core 7;
 327      Queue #2 not pinned.
 328      Queue #3 pinned to core 8;
 329      ```
 330
 331      After that PMD threads on cores where RX queues was pinned will become
 332      `isolated`. This means that this thread will poll only pinned RX queues.
 333
 334      WARNING: If there are no `non-isolated` PMD threads, `non-pinned` RX queues
 335      will not be polled. Also, if provided `core_id` is not available (ex. this
 336      `core_id` not in `pmd-cpu-mask`), RX queue will not be polled by any
 337      PMD thread.
 338
 339      Isolation of PMD threads also can be checked using
 340      `ovs-appctl dpif-netdev/pmd-rxq-show` command.
 341
 342   8. Stop vswitchd & Delete bridge
 343
 344      ```
 345      ovs-appctl -t ovs-vswitchd exit
 346      ovs-appctl -t ovsdb-server exit
 347      ovs-vsctl del-br br0
 348      ```
 349
 350 ## <a name="builddpdk"></a> 4. DPDK in the VM
 351
 352 DPDK 'testpmd' application can be run in the Guest VM for high speed
 353 packet forwarding between vhostuser ports. DPDK and testpmd application
 354 has to be compiled on the guest VM. Below are the steps for setting up
 355 the testpmd application in the VM. More information on the vhostuser ports
 356 can be found in [Vhost Walkthrough].
 357
 358   * Instantiate the Guest
 359
 360   ```
 361   Qemu version >= 2.2.0
 362
 363   export VM_NAME=Centos-vm
 364   export GUEST_MEM=3072M
 365   export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2
 366   export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch
 367
 368   qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm -m $GUEST_MEM -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 -drive file=$QCOW2_IMAGE -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off --nographic -snapshot
 369   ```
 370
 371   * Download the DPDK Srcs to VM and build DPDK
 372
 373   ```
 374   cd /root/dpdk/
 375   wget http://dpdk.org/browse/dpdk/snapshot/dpdk-16.07.zip
 376   unzip dpdk-16.07.zip
 377   export DPDK_DIR=/root/dpdk/dpdk-16.07
 378   export DPDK_TARGET=x86_64-native-linuxapp-gcc
 379   export DPDK_BUILD=$DPDK_DIR/$DPDK_TARGET
 380   cd $DPDK_DIR
 381   make install T=$DPDK_TARGET DESTDIR=install
 382   ```
 383
 384   * Build the test-pmd application
 385
 386   ```
 387   cd app/test-pmd
 388   export RTE_SDK=$DPDK_DIR
 389   export RTE_TARGET=$DPDK_TARGET
 390   make
 391   ```
 392
 393   * Setup Huge pages and DPDK devices using UIO
 394
 395   ```
 396   sysctl vm.nr_hugepages=1024
 397   mkdir -p /dev/hugepages
 398   mount -t hugetlbfs hugetlbfs /dev/hugepages (only if not already mounted)
 399   modprobe uio
 400   insmod $DPDK_BUILD/kmod/igb_uio.ko
 401   $DPDK_DIR/tools/dpdk-devbind.py --status
 402   $DPDK_DIR/tools/dpdk-devbind.py -b igb_uio 00:03.0 00:04.0
 403   ```
 404
 405   vhost ports pci ids can be retrieved using `lspci | grep Ethernet` cmd.
 406
 407 ## <a name="ovstc"></a> 5. OVS Testcases
 408
 409   Below are few testcases and the list of steps to be followed.
 410
 411 ### 5.1 PHY-PHY
 412
 413   The steps (1-5) in 3.3 section will create & initialize DB, start vswitchd and also
 414   add DPDK devices to bridge 'br0'.
 415
 416   1. Add Test flows to forward packets betwen DPDK port 0 and port 1
 417
 418        ```
 419        # Clear current flows
 420        ovs-ofctl del-flows br0
 421
 422        # Add flows between port 1 (dpdk0) to port 2 (dpdk1)
 423        ovs-ofctl add-flow br0 in_port=1,action=output:2
 424        ovs-ofctl add-flow br0 in_port=2,action=output:1
 425        ```
 426
 427 ### 5.2 PHY-VM-PHY [VHOST LOOPBACK]
 428
 429   The steps (1-5) in 3.3 section will create & initialize DB, start vswitchd and also
 430   add DPDK devices to bridge 'br0'.
 431
 432   1. Add dpdkvhostuser ports to bridge 'br0'. More information on the dpdkvhostuser ports
 433      can be found in [Vhost Walkthrough].
 434
 435        ```
 436        ovs-vsctl add-port br0 dpdkvhostuser0 -- set Interface dpdkvhostuser0 type=dpdkvhostuser
 437        ovs-vsctl add-port br0 dpdkvhostuser1 -- set Interface dpdkvhostuser1 type=dpdkvhostuser
 438        ```
 439
 440   2. Add Test flows to forward packets betwen DPDK devices and VM ports
 441
 442        ```
 443        # Clear current flows
 444        ovs-ofctl del-flows br0
 445
 446        # Add flows
 447        ovs-ofctl add-flow br0 in_port=1,action=output:3
 448        ovs-ofctl add-flow br0 in_port=3,action=output:1
 449        ovs-ofctl add-flow br0 in_port=4,action=output:2
 450        ovs-ofctl add-flow br0 in_port=2,action=output:4
 451
 452        # Dump flows
 453        ovs-ofctl dump-flows br0
 454        ```
 455
 456   3. Instantiate Guest VM using Qemu cmdline
 457
 458        Guest Configuration
 459
 460        ```
 461        | configuration        | values | comments
 462        |----------------------|--------|-----------------
 463        | qemu version         | 2.2.0  |
 464        | qemu thread affinity | core 5 | taskset 0x20
 465        | memory               | 4GB    | -
 466        | cores                | 2      | -
 467        | Qcow2 image          | CentOS7| -
 468        | mrg_rxbuf            | off    | -
 469        ```
 470
 471        Instantiate Guest
 472
 473        ```
 474        export VM_NAME=vhost-vm
 475        export GUEST_MEM=3072M
 476        export QCOW2_IMAGE=/root/CentOS7_x86_64.qcow2
 477        export VHOST_SOCK_DIR=/usr/local/var/run/openvswitch
 478
 479        taskset 0x20 qemu-system-x86_64 -name $VM_NAME -cpu host -enable-kvm -m $GUEST_MEM -object memory-backend-file,id=mem,size=$GUEST_MEM,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc -smp sockets=1,cores=2 -drive file=$QCOW2_IMAGE -chardev socket,id=char0,path=$VHOST_SOCK_DIR/dpdkvhostuser0 -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1,mrg_rxbuf=off -chardev socket,id=char1,path=$VHOST_SOCK_DIR/dpdkvhostuser1 -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce -device virtio-net-pci,mac=00:00:00:00:00:02,netdev=mynet2,mrg_rxbuf=off --nographic -snapshot
 480        ```
 481
 482   4. Guest VM using libvirt
 483
 484      The below is a simple xml configuration of 'demovm' guest that can be instantiated
 485      using 'virsh'. The guest uses a pair of vhostuser port and boots with 4GB RAM and 2 cores.
 486      More information can be found in [Vhost Walkthrough].
 487
 488        ```
 489        <domain type='kvm'>
 490          <name>demovm</name>
 491          <uuid>4a9b3f53-fa2a-47f3-a757-dd87720d9d1d</uuid>
 492          <memory unit='KiB'>4194304</memory>
 493          <currentMemory unit='KiB'>4194304</currentMemory>
 494          <memoryBacking>
 495            <hugepages>
 496              <page size='2' unit='M' nodeset='0'/>
 497            </hugepages>
 498          </memoryBacking>
 499          <vcpu placement='static'>2</vcpu>
 500          <cputune>
 501            <shares>4096</shares>
 502            <vcpupin vcpu='0' cpuset='4'/>
 503            <vcpupin vcpu='1' cpuset='5'/>
 504            <emulatorpin cpuset='4,5'/>
 505          </cputune>
 506          <os>
 507            <type arch='x86_64' machine='pc'>hvm</type>
 508            <boot dev='hd'/>
 509          </os>
 510          <features>
 511            <acpi/>
 512            <apic/>
 513          </features>
 514          <cpu mode='host-model'>
 515            <model fallback='allow'/>
 516            <topology sockets='2' cores='1' threads='1'/>
 517            <numa>
 518              <cell id='0' cpus='0-1' memory='4194304' unit='KiB' memAccess='shared'/>
 519            </numa>
 520          </cpu>
 521          <on_poweroff>destroy</on_poweroff>
 522          <on_reboot>restart</on_reboot>
 523          <on_crash>destroy</on_crash>
 524          <devices>
 525            <emulator>/usr/bin/qemu-kvm</emulator>
 526            <disk type='file' device='disk'>
 527              <driver name='qemu' type='qcow2' cache='none'/>
 528              <source file='/root/CentOS7_x86_64.qcow2'/>
 529              <target dev='vda' bus='virtio'/>
 530            </disk>
 531            <disk type='dir' device='disk'>
 532              <driver name='qemu' type='fat'/>
 533              <source dir='/usr/src/dpdk-16.07'/>
 534              <target dev='vdb' bus='virtio'/>
 535              <readonly/>
 536            </disk>
 537            <interface type='vhostuser'>
 538              <mac address='00:00:00:00:00:01'/>
 539              <source type='unix' path='/usr/local/var/run/openvswitch/dpdkvhostuser0' mode='client'/>
 540               <model type='virtio'/>
 541              <driver queues='2'>
 542                <host mrg_rxbuf='off'/>
 543              </driver>
 544            </interface>
 545            <interface type='vhostuser'>
 546              <mac address='00:00:00:00:00:02'/>
 547              <source type='unix' path='/usr/local/var/run/openvswitch/dpdkvhostuser1' mode='client'/>
 548              <model type='virtio'/>
 549              <driver queues='2'>
 550                <host mrg_rxbuf='off'/>
 551              </driver>
 552            </interface>
 553            <serial type='pty'>
 554              <target port='0'/>
 555            </serial>
 556            <console type='pty'>
 557              <target type='serial' port='0'/>
 558            </console>
 559          </devices>
 560        </domain>
 561        ```
 562
 563   5. DPDK Packet forwarding in Guest VM
 564
 565      To accomplish this, DPDK and testpmd application have to be first compiled
 566      on the VM and the steps are listed in [DPDK in the VM].
 567
 568        * Run test-pmd application
 569
 570        ```
 571        cd $DPDK_DIR/app/test-pmd;
 572        ./testpmd -c 0x3 -n 4 --socket-mem 1024 -- --burst=64 -i --txqflags=0xf00 --disable-hw-vlan
 573        set fwd mac retry
 574        start
 575        ```
 576
 577        * Bind vNIC back to kernel once the test is completed.
 578
 579        ```
 580        $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:03.0
 581        $DPDK_DIR/tools/dpdk-devbind.py --bind=virtio-pci 0000:00:04.0
 582        ```
 583        Note: Appropriate PCI IDs to be passed in above example. The PCI IDs can be
 584        retrieved using '$DPDK_DIR/tools/dpdk-devbind.py --status' cmd.
 585
 586 ### 5.3 PHY-VM-PHY [IVSHMEM]
 587
 588   The steps for setup of IVSHMEM are covered in section 5.2(PVP - IVSHMEM)
 589   of [OVS Testcases] in ADVANCED install guide.
 590
 591 ## <a name="ovslimits"></a> 6. Limitations
 592
 593   - Supports MTU size 1500, MTU setting for DPDK netdevs will be in future OVS release.
 594   - Currently DPDK ports does not use HW offload functionality.
 595   - Network Interface Firmware requirements:
 596     Each release of DPDK is validated against a specific firmware version for
 597     a supported Network Interface. New firmware versions introduce bug fixes,
 598     performance improvements and new functionality that DPDK leverages. The
 599     validated firmware versions are available as part of the release notes for
 600     DPDK. It is recommended that users update Network Interface firmware to
 601     match what has been validated for the DPDK release.
 602
 603     For DPDK 16.07, the list of validated firmware versions can be found at:
 604
 605     http://dpdk.org/doc/guides/rel_notes/release_16.07.html
 606
 607
 608 Bug Reporting:
 609 --------------
 610
 611 Please report problems to bugs@openvswitch.org.
 612
 613
 614 [DPDK requirements]: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html
 615 [Download DPDK]: http://dpdk.org/browse/dpdk/refs/
 616 [Download OVS]: http://openvswitch.org/releases/
 617 [DPDK Supported NICs]: http://dpdk.org/doc/nics
 618 [Build Requirements]: https://github.com/openvswitch/ovs/blob/master/INSTALL.md#build-requirements
 619 [INSTALL.DPDK-ADVANCED.md]: INSTALL.DPDK-ADVANCED.md
 620 [OVS Testcases]: INSTALL.DPDK-ADVANCED.md#ovstc
 621 [Vhost Walkthrough]: INSTALL.DPDK-ADVANCED.md#vhost
 622 [DPDK in the VM]: INSTALL.DPDK.md#builddpdk
 623 [INSTALL.md]:INSTALL.md
 624 [INSTALL.Fedora.md]:INSTALL.Fedora.md
 625 [INSTALL.RHEL.md]:INSTALL.RHEL.md
 626 [INSTALL.Debian.md]:INSTALL.Debian.md