ovn-northd: Add logical flows to support native DNS

[mirror_ovs.git] / ovn / northd / ovn-northd.8.xml

<?xml version="1.0" encoding="utf-8"?>
<manpage program="ovn-northd" section="8" title="ovn-northd">
    <h1>Name</h1>
    <p>ovn-northd -- Open Virtual Network central control daemon</p>

    <h1>Synopsis</h1>
    <p><code>ovn-northd</code> [<var>options</var>]</p>

    <h1>Description</h1>
    <p>
      <code>ovn-northd</code> is a centralized daemon responsible for
      translating the high-level OVN configuration into logical
      configuration consumable by daemons such as
      <code>ovn-controller</code>.  It translates the logical network
      configuration in terms of conventional network concepts, taken
      from the OVN Northbound Database (see <code>ovn-nb</code>(5)),
      into logical datapath flows in the OVN Southbound Database (see
      <code>ovn-sb</code>(5)) below it.
    </p>

    <h1>Options</h1>
    <dl>
      <dt><code>--ovnnb-db=<var>database</var></code></dt>
      <dd>
        The OVSDB database containing the OVN Northbound Database.  If the
        <env>OVN_NB_DB</env> environment variable is set, its value is used
        as the default.  Otherwise, the default is
        <code>unix:@RUNDIR@/ovnnb_db.sock</code>.
      </dd>
      <dt><code>--ovnsb-db=<var>database</var></code></dt>
      <dd>
        The OVSDB database containing the OVN Southbound Database.  If the
        <env>OVN_SB_DB</env> environment variable is set, its value is used
        as the default.  Otherwise, the default is
        <code>unix:@RUNDIR@/ovnsb_db.sock</code>.
      </dd>
    </dl>
    <p>
      <var>database</var> in the above options must take one of the following
      forms:
    </p>
    <xi:include href="ovsdb/remote-active.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>
    <xi:include href="ovsdb/remote-passive.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>

    <h2>Daemon Options</h2>
    <xi:include href="lib/daemon.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>

    <h2>Logging Options</h2>
    <xi:include href="lib/vlog.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>

    <h2>PKI Options</h2>
    <p>
      PKI configuration is required in order to use SSL for the connections to
      the Northbound and Southbound databases.
    </p>
    <xi:include href="lib/ssl.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>

    <h2>Other Options</h2>

    <xi:include href="lib/common.xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>

    <h1>Runtime Management Commands</h1>
    <p>
      <code>ovs-appctl</code> can send commands to a running
      <code>ovn-northd</code> process.  The currently supported commands
      are described below.
      <dl>
      <dt><code>exit</code></dt>
      <dd>
        Causes <code>ovn-northd</code> to gracefully terminate.
      </dd>
      </dl>
    </p>

    <h1>Logical Flow Table Structure</h1>

    <p>
      One of the main purposes of <code>ovn-northd</code> is to populate the
      <code>Logical_Flow</code> table in the <code>OVN_Southbound</code>
      database.  This section describes how <code>ovn-northd</code> does this
      for switch and router logical datapaths.
    </p>

    <h2>Logical Switch Datapaths</h2>

    <h3>Ingress Table 0: Admission Control and Ingress Port Security - L2</h3>

    <p>
      Ingress table 0 contains these logical flows:
    </p>

    <ul>
      <li>
        Priority 100 flows to drop packets with VLAN tags or multicast Ethernet
        source addresses.
      </li>

      <li>
        Priority 50 flows that implement ingress port security for each enabled
        logical port.  For logical ports on which port security is enabled,
        these match the <code>inport</code> and the valid <code>eth.src</code>
        address(es) and advance only those packets to the next flow table.  For
        logical ports on which port security is not enabled, these advance all
        packets that match the <code>inport</code>.
      </li>
    </ul>

    <p>
      There are no flows for disabled logical ports because the default-drop
      behavior of logical flow tables causes packets that ingress from them to
      be dropped.
    </p>

    <h3>Ingress Table 1: Ingress Port Security - IP</h3>

    <p>
      Ingress table 1 contains these logical flows:
    </p>

    <ul>
      <li>
        <p>
          For each element in the port security set having one or more IPv4 or
          IPv6 addresses (or both),
        </p>

        <ul>
          <li>
            Priority 90 flow to allow IPv4 traffic if it has IPv4 addresses
            which match the <code>inport</code>, valid <code>eth.src</code>
            and valid <code>ip4.src</code> address(es).
          </li>

          <li>
            Priority 90 flow to allow IPv4 DHCP discovery traffic if it has a
            valid <code>eth.src</code>. This is necessary since DHCP discovery
            messages are sent from the unspecified IPv4 address (0.0.0.0) since
            the IPv4 address has not yet been assigned.
          </li>

          <li>
            Priority 90 flow to allow IPv6 traffic if it has IPv6 addresses
            which match the <code>inport</code>, valid <code>eth.src</code> and
            valid <code>ip6.src</code> address(es).
          </li>

          <li>
            Priority 90 flow to allow IPv6 DAD (Duplicate Address Detection)
            traffic if it has a valid <code>eth.src</code>. This is is
            necessary since DAD include requires joining an multicast group and
            sending neighbor solicitations for the newly assigned address. Since
            no address is yet assigned, these are sent from the unspecified
            IPv6 address (::).
          </li>

          <li>
            Priority 80 flow to drop IP (both IPv4 and IPv6) traffic which
            match the <code>inport</code> and valid <code>eth.src</code>.
          </li>
        </ul>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 2: Ingress Port Security - Neighbor discovery</h3>

    <p>
      Ingress table 2 contains these logical flows:
    </p>

    <ul>
      <li>
        <p>
          For each element in the port security set,
        </p>

        <ul>
          <li>
            Priority 90 flow to allow ARP traffic which match the
            <code>inport</code> and valid <code>eth.src</code> and
            <code>arp.sha</code>. If the element has one or more
            IPv4 addresses, then it also matches the valid
            <code>arp.spa</code>.
          </li>

          <li>
            Priority 90 flow to allow IPv6 Neighbor Solicitation and
            Advertisement traffic which match the <code>inport</code>,
            valid <code>eth.src</code> and
            <code>nd.sll</code>/<code>nd.tll</code>.
            If the element has one or more IPv6 addresses, then it also
            matches the valid <code>nd.target</code> address(es) for Neighbor
            Advertisement traffic.
          </li>

          <li>
            Priority 80 flow to drop ARP and IPv6 Neighbor Solicitation and
            Advertisement traffic which match the <code>inport</code> and
            valid <code>eth.src</code>.
          </li>
        </ul>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 3: <code>from-lport</code> Pre-ACLs</h3>

    <p>
      This table prepares flows for possible stateful ACL processing in
      ingress table <code>ACLs</code>.  It contains a priority-0 flow that
      simply moves traffic to the next table.  If stateful ACLs are used in the
      logical datapath, a priority-100 flow is added that sets a hint
      (with <code>reg0[0] = 1; next;</code>) for table
      <code>Pre-stateful</code> to send IP packets to the connection tracker
      before eventually advancing to ingress table <code>ACLs</code>.
    </p>

    <h3>Ingress Table 4: Pre-LB</h3>

    <p>
      This table prepares flows for possible stateful load balancing processing
      in ingress table <code>LB</code> and <code>Stateful</code>.  It contains
      a priority-0 flow that simply moves traffic to the next table.  If load
      balancing rules with virtual IP addresses (and ports) are configured in
      <code>OVN_Northbound</code> database for a logical switch datapath, a
      priority-100 flow is added for each configured virtual IP address
      <var>VIP</var> with a match <code>ip &amp;&amp; ip4.dst == <var>VIP</var>
      </code> that sets an action <code>reg0[0] = 1; next;</code> to act as a
      hint for table <code>Pre-stateful</code> to send IP packets to the
      connection tracker for packet de-fragmentation before eventually
      advancing to ingress table <code>LB</code>.
    </p>

    <h3>Ingress Table 5: Pre-stateful</h3>

    <p>
      This table prepares flows for all possible stateful processing
      in next tables.  It contains a priority-0 flow that simply moves
      traffic to the next table.  A priority-100 flow sends the packets to
      connection tracker based on a hint provided by the previous tables
      (with a match for <code>reg0[0] == 1</code>) by using the
      <code>ct_next;</code> action.
    </p>

    <h3>Ingress table 6: <code>from-lport</code> ACLs</h3>

    <p>
      Logical flows in this table closely reproduce those in the
      <code>ACL</code> table in the <code>OVN_Northbound</code> database
      for the <code>from-lport</code> direction. The <code>priority</code>
      values from the <code>ACL</code> table have a limited range and have
      1000 added to them to leave room for OVN default flows at both
      higher and lower priorities.
    </p>
    <ul>
      <li>
        <code>allow</code> ACLs translate into logical flows with
        the <code>next;</code> action.  If there are any stateful ACLs
        on this datapath, then <code>allow</code> ACLs translate to
        <code>ct_commit; next;</code> (which acts as a hint for the next tables
        to commit the connection to conntrack),
      </li>
      <li>
        <code>allow-related</code> ACLs translate into logical
        flows with the <code>ct_commit(ct_label=0/1); next;</code> actions
        for new connections and <code>reg0[1] = 1; next;</code> for existing
        connections.
      </li>
      <li>
        Other ACLs translate to <code>drop;</code> for new or untracked
        connections and <code>ct_commit(ct_label=1/1);</code> for known
        connections.  Setting <code>ct_label</code> marks a connection
        as one that was previously allowed, but should no longer be
        allowed due to a policy change.
      </li>
    </ul>

    <p>
      This table also contains a priority 0 flow with action
      <code>next;</code>, so that ACLs allow packets by default.  If the
      logical datapath has a statetful ACL, the following flows will
      also be added:
    </p>

    <ul>
      <li>
        A priority-1 flow that sets the hint to commit IP traffic to the
        connection tracker (with action <code>reg0[1] = 1; next;</code>).  This
        is needed for the default allow policy because, while the initiator's
        direction may not have any stateful rules, the server's may and then
        its return traffic would not be known and marked as invalid.
      </li>

      <li>
        A priority-65535 flow that allows any traffic in the reply
        direction for a connection that has been committed to the
        connection tracker (i.e., established flows), as long as
        the committed flow does not have <code>ct_label.blocked</code> set.
        We only handle traffic in the reply direction here because
        we want all packets going in the request direction to still
        go through the flows that implement the currently defined
        policy based on ACLs.  If a connection is no longer allowed by
        policy, <code>ct_label.blocked</code> will get set and packets in the
        reply direction will no longer be allowed, either.
      </li>

      <li>
        A priority-65535 flow that allows any traffic that is considered
        related to a committed flow in the connection tracker (e.g., an
        ICMP Port Unreachable from a non-listening UDP port), as long
        as the committed flow does not have <code>ct_label.blocked</code> set.
      </li>

      <li>
        A priority-65535 flow that drops all traffic marked by the
        connection tracker as invalid.
      </li>

      <li>
        A priority-65535 flow that drops all trafic in the reply direction
        with <code>ct_label.blocked</code> set meaning that the connection
        should no longer be allowed due to a policy change.  Packets
        in the request direction are skipped here to let a newly created
        ACL re-allow this connection.
      </li>
    </ul>

    <h3>Ingress Table 7: <code>from-lport</code> QoS marking</h3>

    <p>
      Logical flows in this table closely reproduce those in the
      <code>QoS</code> table in the <code>OVN_Northbound</code> database
      for the <code>from-lport</code> direction.
    </p>

    <ul>
      <li>
        For every qos_rules for every logical switch a flow will be added at
        priorities mentioned in the QoS table.
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 8: LB</h3>

    <p>
      It contains a priority-0 flow that simply moves traffic to the next
      table.  For established connections a priority 100 flow matches on
      <code>ct.est &amp;&amp; !ct.rel &amp;&amp; !ct.new &amp;&amp;
      !ct.inv</code> and sets an action <code>reg0[2] = 1; next;</code> to act
      as a hint for table <code>Stateful</code> to send packets through
      connection tracker to NAT the packets.  (The packet will automatically
      get DNATed to the same IP address as the first packet in that
      connection.)
    </p>

    <h3>Ingress Table 9: Stateful</h3>

    <ul>
      <li>
        For all the configured load balancing rules for a switch in
        <code>OVN_Northbound</code> database that includes a L4 port
        <var>PORT</var> of protocol <var>P</var> and IPv4 address
        <var>VIP</var>, a priority-120 flow that matches on
        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP
        </var>&amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst == <var>PORT
        </var></code> with an action of <code>ct_lb(<var>args</var>)</code>,
        where <var>args</var> contains comma separated IPv4 addresses (and
        optional port numbers) to load balance to.
      </li>
      <li>
        For all the configured load balancing rules for a switch in
        <code>OVN_Northbound</code> database that includes just an IP address
        <var>VIP</var> to match on, a priority-110 flow that matches on
        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP</var></code>
        with an action of <code>ct_lb(<var>args</var>)</code>, where
        <var>args</var> contains comma separated IPv4 addresses.
      </li>
      <li>
        A priority-100 flow commits packets to connection tracker using
        <code>ct_commit; next;</code> action based on a hint provided by
        the previous tables (with a match for <code>reg0[1] == 1</code>).
      </li>
      <li>
        A priority-100 flow sends the packets to connection tracker using
        <code>ct_lb;</code> as the action based on a hint provided by the
        previous tables (with a match for <code>reg0[2] == 1</code>).
      </li>
      <li>
        A priority-0 flow that simply moves traffic to the next table.
      </li>
    </ul>

    <h3>Ingress Table 10: ARP/ND responder</h3>

    <p>
      This table implements ARP/ND responder in a logical switch for known
      IPs.  The advantage of the ARP responder flow is to limit ARP
      broadcasts by locally responding to ARP requests without the need to
      send to other hypervisors.  One common case is when the inport is a
      logical port associated with a VIF and the broadcast is responded to
      on the local hypervisor rather than broadcast across the whole
      network and responded to by the destination VM.  This behavior is
      proxy ARP.
    </p>

    <p>
      ARP requests arrive from VMs from a logical switch inport of type
      default.  For this case, the logical switch proxy ARP rules can be
      for other VMs or logical router ports.  Logical switch proxy ARP
      rules may be programmed both for mac binding of IP addresses on
      other logical switch VIF ports (which are of the default logical
      switch port type, representing connectivity to VMs or containers),
      and for mac binding of IP addresses on logical switch router type
      ports, representing their logical router port peers.  In order to
      support proxy ARP for logical router ports, an IP address must be
      configured on the logical switch router type port, with the same
      value as the peer logical router port.  The configured MAC addresses
      must match as well.  When a VM sends an ARP request for a distributed
      logical router port and if the peer router type port of the attached
      logical switch does not have an IP address configured, the ARP request
      will be broadcast on the logical switch.  One of the copies of the ARP
      request will go through the logical switch router type port to the
      logical router datapath, where the logical router ARP responder will
      generate a reply.  The MAC binding of a distributed logical router,
      once learned by an associated VM, is used for all that VM's
      communication needing routing.  Hence, the action of a VM re-arping for
      the mac binding of the logical router port should be rare.
    </p>

    <p>
      Logical switch ARP responder proxy ARP rules can also be hit when
      receiving ARP requests externally on a L2 gateway port.  In this case,
      the hypervisor acting as an L2 gateway, responds to the ARP request on
      behalf of a destination VM.
    </p>

    <p>
      Note that ARP requests received from <code>localnet</code> or
      <code>vtep</code> logical inports can either go directly to VMs, in
      which case the VM responds or can hit an ARP responder for a logical
      router port if the packet is used to resolve a logical router port
      next hop address.  In either case, logical switch ARP responder rules
      will not be hit.  It contains these logical flows:
     </p>

    <ul>
      <li>
        Priority-100 flows to skip the ARP responder if inport is of type
        <code>localnet</code> or <code>vtep</code> and advances directly
        to the next table.  ARP requests sent to <code>localnet</code> or
        <code>vtep</code> ports can be received by multiple hypervisors.
        Now, because the same mac binding rules are downloaded to all
        hypervisors, each of the multiple hypervisors will respond.  This
        will confuse L2 learning on the source of the ARP requests.  ARP
        requests received on an inport of type <code>router</code> are not
        expected to hit any logical switch ARP responder flows.  However,
        no skip flows are installed for these packets, as there would be
        some additional flow cost for this and the value appears limited.
      </li>

      <li>
        <p>
          Priority-50 flows that match ARP requests to each known IP address
          <var>A</var> of every logical switch port, and respond with ARP
          replies directly with corresponding Ethernet address <var>E</var>:
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = <var>E</var>;
arp.tpa = arp.spa;
arp.spa = <var>A</var>;
outport = inport;
flags.loopback = 1;
output;
        </pre>

        <p>
          These flows are omitted for logical ports (other than router ports)
          that are down.
        </p>
      </li>

      <li>
        <p>
          Priority-50 flows that match IPv6 ND neighbor solicitations to
          each known IP address <var>A</var> (and <var>A</var>'s
          solicited node address) of every logical switch port, and
          respond with neighbor advertisements directly with
          corresponding Ethernet address <var>E</var>:
        </p>

        <pre>
nd_na {
    eth.src = <var>E</var>;
    ip6.src = <var>A</var>;
    nd.target = <var>A</var>;
    nd.tll = <var>E</var>;
    outport = inport;
    flags.loopback = 1;
    output;
};
        </pre>

        <p>
          These flows are omitted for logical ports (other than router ports)
          that are down.
        </p>
      </li>

      <li>
        <p>
          Priority-100 flows with match criteria like the ARP and ND flows
          above, except that they only match packets from the
          <code>inport</code> that owns the IP addresses in question, with
          action <code>next;</code>.  These flows prevent OVN from replying to,
          for example, an ARP request emitted by a VM for its own IP address.
          A VM only makes this kind of request to attempt to detect a duplicate
          IP address assignment, so sending a reply will prevent the VM from
          accepting the IP address that it owns.
        </p>

        <p>
          In place of <code>next;</code>, it would be reasonable to use
          <code>drop;</code> for the flows' actions.  If everything is working
          as it is configured, then this would produce equivalent results,
          since no host should reply to the request.  But ARPing for one's own
          IP address is intended to detect situations where the network is not
          working as configured, so dropping the request would frustrate that
          intent.
        </p>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and advances to
        the next table.
      </li>
    </ul>

    <h3>Ingress Table 11: DHCP option processing</h3>

    <p>
      This table adds the DHCPv4 options to a DHCPv4 packet from the
      logical ports configured with IPv4 address(es) and DHCPv4 options,
      and similarly for DHCPv6 options.
    </p>

    <ul>
      <li>
        <p>
          A priority-100 logical flow is added for these logical ports
          which matches the IPv4 packet with <code>udp.src</code> = 68 and
          <code>udp.dst</code> = 67 and applies the action
          <code>put_dhcp_opts</code> and advances the packet to the next table.
        </p>

        <pre>
reg0[3] = put_dhcp_opts(offer_ip = <var>ip</var>, <var>options</var>...);
next;
        </pre>

        <p>
          For DHCPDISCOVER and DHCPREQUEST, this transforms the packet into a
          DHCP reply, adds the DHCP offer IP <var>ip</var> and options to the
          packet, and stores 1 into reg0[3].  For other kinds of packets, it
          just stores 0 into reg0[3].  Either way, it continues to the next
          table.
        </p>

      </li>

      <li>
        <p>
          A priority-100 logical flow is added for these logical ports
          which matches the IPv6 packet with <code>udp.src</code> = 546 and
          <code>udp.dst</code> = 547 and applies the action
          <code>put_dhcpv6_opts</code> and advances the packet to the next
          table.
        </p>

        <pre>
reg0[3] = put_dhcpv6_opts(ia_addr = <var>ip</var>, <var>options</var>...);
next;
        </pre>

        <p>
          For DHCPv6 Solicit/Request/Confirm packets, this transforms the
          packet into a DHCPv6 Advertise/Reply, adds the DHCPv6 offer IP
          <var>ip</var> and options to the packet, and stores 1 into reg0[3].
          For other kinds of packets, it just stores 0 into reg0[3]. Either
          way, it continues to the next table.
        </p>
      </li>

      <li>
        A priority-0 flow that matches all packets to advances to table 11.
      </li>
    </ul>

    <h3>Ingress Table 12: DHCP responses</h3>

    <p>
      This table implements DHCP responder for the DHCP replies generated by
      the previous table.
    </p>

    <ul>
      <li>
        <p>
          A priority 100 logical flow is added for the logical ports configured
          with DHCPv4 options which matches IPv4 packets with <code>udp.src == 68
          &amp;&amp; udp.dst == 67 &amp;&amp; reg0[3] == 1</code> and
          responds back to the <code>inport</code> after applying these
          actions.  If <code>reg0[3]</code> is set to 1, it means that the
          action <code>put_dhcp_opts</code> was successful.
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
ip4.dst = <var>A</var>;
ip4.src = <var>S</var>;
udp.src = 67;
udp.dst = 68;
outport = <var>P</var>;
flags.loopback = 1;
output;
        </pre>

        <p>
          where <var>E</var> is the server MAC address and <var>S</var> is the
          server IPv4 address defined in the DHCPv4 options and <var>A</var> is
          the IPv4 address defined in the logical port's addresses column.
        </p>

        <p>
          (This terminates ingress packet processing; the packet does not go
           to the next ingress table.)
        </p>
      </li>

      <li>
        <p>
          A priority 100 logical flow is added for the logical ports configured
          with DHCPv6 options which matches IPv6 packets with <code>udp.src == 546
          &amp;&amp; udp.dst == 547 &amp;&amp; reg0[3] == 1</code> and
          responds back to the <code>inport</code> after applying these
          actions.  If <code>reg0[3]</code> is set to 1, it means that the
          action <code>put_dhcpv6_opts</code> was successful.
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
ip6.dst = <var>A</var>;
ip6.src = <var>S</var>;
udp.src = 547;
udp.dst = 546;
outport = <var>P</var>;
flags.loopback = 1;
output;
        </pre>

        <p>
          where <var>E</var> is the server MAC address and <var>S</var> is the
          server IPv6 LLA address  generated from the <code>server_id</code>
          defined in the DHCPv6 options and <var>A</var> is
          the IPv6 address defined in the logical port's addresses column.
        </p>

        <p>
          (This terminates packet processing; the packet does not go on the
          next ingress table.)
        </p>
      </li>

      <li>
        A priority-0 flow that matches all packets to advances to table 12.
      </li>
    </ul>

    <h3>Ingress Table 13 DNS Lookup</h3>

    <p>
      This table looks up and resolves the DNS names to the corresponding
      configured IP address(es).
    </p>

    <ul>
      <li>
        <p>
          A priority-100 logical flow for each logical switch datapath
          if it is configured with DNS records, which matches the IPv4 and IPv6
          packets with <code>udp.dst</code> = 53 and applies the action
          <code>dns_lookup</code> and advances the packet to the next table.
        </p>

        <pre>
reg0[4] = dns_lookup(); next;
        </pre>

        <p>
          For valid DNS packets, this transforms the packet into a DNS
          reply if the DNS name can be resolved, and stores 1 into reg0[4].
          For failed DNS resolution or other kinds of packets, it just stores
          0 into reg0[4]. Either way, it continues to the next table.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 14 DNS Responses</h3>

    <p>
      This table implements DNS responder for the DNS replies generated by
      the previous table.
    </p>

    <ul>
      <li>
        <p>
          A priority-100 logical flow for each logical switch datapath
          if it is configured with DNS records, which matches the IPv4 and IPv6
          packets with <code>udp.dst = 53 &amp;&amp; reg0[4] == 1</code>
          and responds back to the <code>inport</code> after applying these
          actions.  If <code>reg0[4]</code> is set to 1, it means that the
          action <code>dns_lookup</code> was successful.
        </p>

        <pre>
eth.dst &lt;-&gt; eth.src;
ip4.src &lt;-&gt; ip4.dst;
udp.dst = udp.src;
udp.src = 53;
outport = <var>P</var>;
flags.loopback = 1;
output;
        </pre>

        <p>
          (This terminates ingress packet processing; the packet does not go
           to the next ingress table.)
        </p>
      </li>
    </ul>

    <h3>Ingress Table 15 Destination Lookup</h3>

    <p>
      This table implements switching behavior.  It contains these logical
      flows:
    </p>

    <ul>
      <li>
        A priority-100 flow that outputs all packets with an Ethernet broadcast
        or multicast <code>eth.dst</code> to the <code>MC_FLOOD</code>
        multicast group, which <code>ovn-northd</code> populates with all
        enabled logical ports.
      </li>

      <li>
        <p>
          One priority-50 flow that matches each known Ethernet address against
          <code>eth.dst</code> and outputs the packet to the single associated
          output port.
        </p>

        <p>
          For the Ethernet address on a logical switch port of type
          <code>router</code>, when that logical switch port's
          <ref column="addresses" table="Logical_Switch_Port"
          db="OVN_Northbound"/> column is set to <code>router</code> and
          the connected logical router port specifies a
          <code>redirect-chassis</code>:
        </p>

        <ul>
          <li>
            The flow for the connected logical router port's Ethernet
            address is only programmed on the <code>redirect-chassis</code>.
          </li>

          <li>
            If the logical router has rules specified in
            <ref column="nat" table="Logical_Router" db="OVN_Northbound"/> with
            <ref column="external_mac" table="NAT" db="OVN_Northbound"/>, then
            those addresses are also used to populate the switch's destination
            lookup on the chassis where
            <ref column="logical_port" table="NAT" db="OVN_Northbound"/> is
            resident.
          </li>
        </ul>
      </li>

      <li>
        One priority-0 fallback flow that matches all packets and outputs them
        to the <code>MC_UNKNOWN</code> multicast group, which
        <code>ovn-northd</code> populates with all enabled logical ports that
        accept unknown destination packets.  As a small optimization, if no
        logical ports accept unknown destination packets,
        <code>ovn-northd</code> omits this multicast group and logical flow.
      </li>
    </ul>

    <h3>Egress Table 0: Pre-LB</h3>

    <p>
      This table is similar to ingress table <code>Pre-LB</code>.  It
      contains a priority-0 flow that simply moves traffic to the next table.
      If any load balancing rules exist for the datapath, a priority-100 flow
      is added with a match of <code>ip</code> and action of <code>reg0[0] = 1;
       next;</code> to act as a hint for table <code>Pre-stateful</code> to
      send IP packets to the connection tracker for packet de-fragmentation.
    </p>

    <h3>Egress Table 1: <code>to-lport</code> Pre-ACLs</h3>

    <p>
      This is similar to ingress table <code>Pre-ACLs</code> except for
     <code>to-lport</code> traffic.
    </p>

    <h3>Egress Table 2: Pre-stateful</h3>

    <p>
      This is similar to ingress table <code>Pre-stateful</code>.
    </p>

    <h3>Egress Table 3: LB</h3>
    <p>
      This is similar to ingress table <code>LB</code>.
    </p>

    <h3>Egress Table 4: <code>to-lport</code> ACLs</h3>

    <p>
      This is similar to ingress table <code>ACLs</code> except for
      <code>to-lport</code> ACLs.
    </p>

    <h3>Egress Table 5: <code>to-lport</code> QoS marking</h3>

    <p>
      This is similar to ingress table <code>QoS marking</code> except for
      <code>to-lport</code> qos rules.
    </p>

    <h3>Egress Table 6: Stateful</h3>

    <p>
      This is similar to ingress table <code>Stateful</code> except that
      there are no rules added for load balancing new connections.
    </p>

    <p>
      Also the following flows are added.
    </p>
    <ul>
      <li>
        A priority 34000 logical flow is added for each logical port which
        has DHCPv4 options defined to allow the DHCPv4 reply packet and which has
        DHCPv6 options defined to allow the DHCPv6 reply packet from the
        <code>Ingress Table 12: DHCP responses</code>.
      </li>

      <li>
        A priority 34000 logical flow is added for each logical switch datapath
        configured with DNS records with the match <code>udp.dst = 53</code>
        to allow the DNS reply packet from the
        <code>Ingress Table 14:DNS responses</code>.
      </li>
    </ul>

    <h3>Egress Table 7: Egress Port Security - IP</h3>

    <p>
      This is similar to the port security logic in table
      <code>Ingress Port Security - IP</code> except that <code>outport</code>,
      <code>eth.dst</code>, <code>ip4.dst</code> and <code>ip6.dst</code>
      are checked instead of <code>inport</code>, <code>eth.src</code>,
      <code>ip4.src</code> and <code>ip6.src</code>
    </p>

    <h3>Egress Table 8: Egress Port Security - L2</h3>

    <p>
      This is similar to the ingress port security logic in ingress table
      <code>Admission Control and Ingress Port Security - L2</code>,
      but with important differences.  Most obviously, <code>outport</code> and
      <code>eth.dst</code> are checked instead of <code>inport</code> and
      <code>eth.src</code>.  Second, packets directed to broadcast or multicast
      <code>eth.dst</code> are always accepted instead of being subject to the
      port security rules; this is implemented through a priority-100 flow that
      matches on <code>eth.mcast</code> with action <code>output;</code>.
      Finally, to ensure that even broadcast and multicast packets are not
      delivered to disabled logical ports, a priority-150 flow for each
      disabled logical <code>outport</code> overrides the priority-100 flow
      with a <code>drop;</code> action.
    </p>

    <h2>Logical Router Datapaths</h2>

    <p>
      Logical router datapaths will only exist for <ref table="Logical_Router"
      db="OVN_Northbound"/> rows in the <ref db="OVN_Northbound"/> database
      that do not have <ref column="enabled" table="Logical_Router"
      db="OVN_Northbound"/> set to <code>false</code>
    </p>

    <h3>Ingress Table 0: L2 Admission Control</h3>

    <p>
      This table drops packets that the router shouldn't see at all based on
      their Ethernet headers.  It contains the following flows:
    </p>

    <ul>
      <li>
        Priority-100 flows to drop packets with VLAN tags or multicast Ethernet
        source addresses.
      </li>

      <li>
        <p>
          For each enabled router port <var>P</var> with Ethernet address
          <var>E</var>, a priority-50 flow that matches <code>inport ==
          <var>P</var> &amp;&amp; (eth.mcast || eth.dst ==
          <var>E</var></code>), with action <code>next;</code>.
        </p>

        <p>
          For the gateway port on a distributed logical router (where
          one of the logical router ports specifies a
          <code>redirect-chassis</code>), the above flow matching
          <code>eth.dst == <var>E</var></code> is only programmed on
          the gateway port instance on the
          <code>redirect-chassis</code>.
        </p>
      </li>

      <li>
        <p>
          For each <code>dnat_and_snat</code> NAT rule on a distributed
          router that specifies an external Ethernet address <var>E</var>,
          a priority-50 flow that matches <code>inport == <var>GW</var>
          &amp;&amp; eth.dst == <var>E</var></code>, where <var>GW</var>
          is the logical router gateway port, with action
          <code>next;</code>.
        </p>

        <p>
          This flow is only programmed on the gateway port instance on
          the chassis where the <code>logical_port</code> specified in
          the NAT rule resides.
        </p>
      </li>
    </ul>

    <p>
      Other packets are implicitly dropped.
    </p>

    <h3>Ingress Table 1: IP Input</h3>

    <p>
      This table is the core of the logical router datapath functionality.  It
      contains the following flows to implement very basic IP host
      functionality.
    </p>

    <ul>
      <li>
        <p>
          L3 admission control: A priority-100 flow drops packets that match
          any of the following:
        </p>

        <ul>
          <li>
            <code>ip4.src[28..31] == 0xe</code> (multicast source)
          </li>
          <li>
            <code>ip4.src == 255.255.255.255</code> (broadcast source)
          </li>
          <li>
            <code>ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8</code>
            (localhost source or destination)
          </li>
          <li>
            <code>ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8</code> (zero
            network source or destination)
          </li>
          <li>
            <code>ip4.src</code> or <code>ip6.src</code> is any IP
            address owned by the router, unless the packet was recirculated
            due to egress loopback as indicated by
            <code>REGBIT_EGRESS_LOOPBACK</code>.
          </li>
          <li>
            <code>ip4.src</code> is the broadcast address of any IP network
            known to the router.
          </li>
        </ul>
      </li>

      <li>
        <p>
          ICMP echo reply.  These flows reply to ICMP echo requests received
          for the router's IP address.  Let <var>A</var> be an IP address
          owned by a router port.  Then, for each <var>A</var> that is
          an IPv4 address, a priority-90 flow matches on
          <code>ip4.dst == <var>A</var></code> and
          <code>icmp4.type == 8 &amp;&amp; icmp4.code == 0</code>
          (ICMP echo request).  For each <var>A</var> that is an IPv6
          address, a priority-90 flow matches on
          <code>ip6.dst == <var>A</var></code> and
          <code>icmp6.type == 128 &amp;&amp; icmp6.code == 0</code>
          (ICMPv6 echo request).  The port of the router that receives the
          echo request does not matter. Also, the <code>ip.ttl</code> of
          the echo request packet is not checked, so it complies with
          RFC 1812, section 4.2.2.9. Flows for ICMPv4 echo requests use the
          following actions:
        </p>

        <pre>
ip4.dst &lt;-&gt; ip4.src;
ip.ttl = 255;
icmp4.type = 0;
flags.loopback = 1;
next;
        </pre>

        <p>
          Flows for ICMPv6 echo requests use the following actions:
        </p>

        <pre>
ip6.dst &lt;-&gt; ip6.src;
ip.ttl = 255;
icmp6.type = 129;
flags.loopback = 1;
next;
        </pre>
      </li>

      <li>
        <p>
          Reply to ARP requests.
        </p>

        <p>
          These flows reply to ARP requests for the router's own IP address.
          For each router port <var>P</var> that owns IP address <var>A</var>
          and Ethernet address <var>E</var>, a priority-90 flow matches
          <code>inport == <var>P</var> &amp;&amp; arp.op == 1 &amp;&amp;
          arp.tpa == <var>A</var></code> (ARP request) with the following
          actions:
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = <var>E</var>;
arp.tpa = arp.spa;
arp.spa = <var>A</var>;
outport = <var>P</var>;
flags.loopback = 1;
output;
        </pre>

        <p>
          For the gateway port on a distributed logical router (where
          one of the logical router ports specifies a
          <code>redirect-chassis</code>), the above flows are only
          programmed on the gateway port instance on the
          <code>redirect-chassis</code>.  This behavior avoids generation
          of multiple ARP responses from different chassis, and allows
          upstream MAC learning to point to the
          <code>redirect-chassis</code>.
        </p>
      </li>

      <li>
        <p>
          These flows reply to ARP requests for the virtual IP addresses
          configured in the router for DNAT or load balancing.  For a
          configured DNAT IP address or a load balancer VIP <var>A</var>,
          for each router port <var>P</var> with Ethernet
          address <var>E</var>, a priority-90 flow matches
          <code>inport == <var>P</var> &amp;&amp; arp.op == 1 &amp;&amp;
          arp.tpa == <var>A</var></code> (ARP request)
          with the following actions:
        </p>

        <pre>
eth.dst = eth.src;
eth.src = <var>E</var>;
arp.op = 2; /* ARP reply. */
arp.tha = arp.sha;
arp.sha = <var>E</var>;
arp.tpa = arp.spa;
arp.spa = <var>A</var>;
outport = <var>P</var>;
flags.loopback = 1;
output;
        </pre>

        <p>
          For the gateway port on a distributed logical router with NAT
          (where one of the logical router ports specifies a
          <code>redirect-chassis</code>):
        </p>

        <ul>
          <li>
            If the corresponding NAT rule cannot be handled in a
            distributed manner, then this flow is only programmed on
            the gateway port instance on the
            <code>redirect-chassis</code>.  This behavior avoids
            generation of multiple ARP responses from different chassis,
            and allows upstream MAC learning to point to the
            <code>redirect-chassis</code>.
          </li>

          <li>
            <p>
              If the corresponding NAT rule can be handled in a distributed
              manner, then this flow is only programmed on the gateway port
              instance where the <code>logical_port</code> specified in the
              NAT rule resides.
            </p>

            <p>
              Some of the actions are different for this case, using the
              <code>external_mac</code> specified in the NAT rule rather
              than the gateway port's Ethernet address <var>E</var>:
            </p>

            <pre>
eth.src = <var>external_mac</var>;
arp.sha = <var>external_mac</var>;
            </pre>

            <p>
              This behavior avoids generation of multiple ARP responses
              from different chassis, and allows upstream MAC learning to
              point to the correct chassis.
            </p>
          </li>
        </ul>
      </li>

      <li>
        ARP reply handling.  This flow uses ARP replies to populate the
        logical router's ARP table.  A priority-90 flow with match <code>arp.op
        == 2</code> has actions <code>put_arp(inport, arp.spa,
        arp.sha);</code>.
      </li>

      <li>
        <p>
          Reply to IPv6 Neighbor Solicitations.  These flows reply to
          Neighbor Solicitation requests for the router's own IPv6
          address and populate the logical router's mac binding table.
          For each router port <var>P</var> that owns IPv6 address
          <var>A</var>, solicited node address <var>S</var>, and
          Ethernet address <var>E</var>, a priority-90 flow matches
          <code>inport == <var>P</var> &amp;&amp; nd_ns &amp;&amp;
          ip6.dst == {<var>A</var>, <var>E</var>} &amp;&amp; nd.target
          == <var>A</var></code> with the following actions:
        </p>

        <pre>
put_nd(inport, ip6.src, nd.sll);
nd_na {
    eth.src = <var>E</var>;
    ip6.src = <var>A</var>;
    nd.target = <var>A</var>;
    nd.tll = <var>E</var>;
    outport = inport;
    flags.loopback = 1;
    output;
};
        </pre>

        <p>
          For the gateway port on a distributed logical router (where
          one of the logical router ports specifies a
          <code>redirect-chassis</code>), the above flows replying to
          IPv6 Neighbor Solicitations are only programmed on the
          gateway port instance on the <code>redirect-chassis</code>.
          This behavior avoids generation of multiple replies from
          different chassis, and allows upstream MAC learning to point
          to the <code>redirect-chassis</code>.
        </p>
      </li>

      <li>
        IPv6 neighbor advertisement handling.  This flow uses neighbor
        advertisements to populate the logical router's mac binding
        table.  A priority-90 flow with match <code>nd_na</code>
        has actions <code>put_nd(inport, nd.target, nd.tll);</code>.
      </li>

      <li>
        IPv6 neighbor solicitation for non-hosted addresses handling.
        This flow uses neighbor solicitations to populate the logical
        router's mac binding table (ones that were directed at the
        logical router would have matched the priority-90 neighbor
        solicitation flow already).  A priority-80 flow with match
        <code>nd_ns</code> has actions
        <code>put_nd(inport, ip6.src, nd.sll);</code>.
      </li>

      <li>
        <p>
          UDP port unreachable.  Priority-80 flows generate ICMP port
          unreachable messages in reply to UDP datagrams directed to the
          router's IP address.  The logical router doesn't accept any UDP
          traffic so it always generates such a reply.
        </p>

        <p>
          These flows should not match IP fragments with nonzero offset.
        </p>

        <p>
          Details TBD.  Not yet implemented.
        </p>
      </li>

      <li>
        <p>
          TCP reset.  Priority-80 flows generate TCP reset messages in reply to
          TCP datagrams directed to the router's IP address.  The logical
          router doesn't accept any TCP traffic so it always generates such a
          reply.
        </p>

        <p>
          These flows should not match IP fragments with nonzero offset.
        </p>

        <p>
          Details TBD.  Not yet implemented.
        </p>
      </li>

      <li>
        <p>
          Protocol unreachable.  Priority-70 flows generate ICMP protocol
          unreachable messages in reply to packets directed to the router's IP
          address on IP protocols other than UDP, TCP, and ICMP.
        </p>

        <p>
          These flows should not match IP fragments with nonzero offset.
        </p>

        <p>
          Details TBD.  Not yet implemented.
        </p>
      </li>

      <li>
        Drop other IP traffic to this router.  These flows drop any other
        traffic destined to an IP address of this router that is not already
        handled by one of the flows above, which amounts to ICMP (other than
        echo requests) and fragments with nonzero offsets.  For each IP address
        <var>A</var> owned by the router, a priority-60 flow matches
        <code>ip4.dst == <var>A</var></code> and drops the traffic.  An
        exception is made and the above flow is not added if the router
        port's own IP address is used to SNAT packets passing through that
        router.
      </li>
    </ul>

    <p>
      The flows above handle all of the traffic that might be directed to the
      router itself.  The following flows (with lower priorities) handle the
      remaining traffic, potentially for forwarding:
    </p>

    <ul>
      <li>
        Drop Ethernet local broadcast.  A priority-50 flow with match
        <code>eth.bcast</code> drops traffic destined to the local Ethernet
        broadcast address.  By definition this traffic should not be forwarded.
      </li>

      <li>
        <p>
          ICMP time exceeded.  For each router port <var>P</var>, whose IP
          address is <var>A</var>, a priority-40 flow with match <code>inport
          == <var>P</var> &amp;&amp; ip.ttl == {0, 1} &amp;&amp;
          !ip.later_frag</code> matches packets whose TTL has expired, with the
          following actions to send an ICMP time exceeded reply:
        </p>

        <pre>
icmp4 {
    icmp4.type = 11; /* Time exceeded. */
    icmp4.code = 0;  /* TTL exceeded in transit. */
    ip4.dst = ip4.src;
    ip4.src = <var>A</var>;
    ip.ttl = 255;
    next;
};
        </pre>

        <p>
          Not yet implemented.
        </p>
      </li>

      <li>
        TTL discard.  A priority-30 flow with match <code>ip.ttl == {0,
        1}</code> and actions <code>drop;</code> drops other packets whose TTL
        has expired, that should not receive a ICMP error reply (i.e. fragments
        with nonzero offset).
      </li>

      <li>
        Next table.  A priority-0 flows match all packets that aren't already
        handled and uses actions <code>next;</code> to feed them to the next
        table.
      </li>
    </ul>

    <h3>Ingress Table 2: DEFRAG</h3>

    <p>
      This is to send packets to connection tracker for tracking and
      defragmentation.  It contains a priority-0 flow that simply moves traffic
      to the next table.  If load balancing rules with virtual IP addresses
      (and ports) are configured in <code>OVN_Northbound</code> database for a
      Gateway router, a priority-100 flow is added for each configured virtual
      IP address <var>VIP</var> with a match <code>ip &amp;&amp;
      ip4.dst == <var>VIP</var></code> that sets an action
      <code>ct_next;</code> to send IP packets to the connection tracker for
      packet de-fragmentation and tracking before sending it to the next table.
    </p>

    <h3>Ingress Table 3: UNSNAT</h3>

    <p>
      This is for already established connections' reverse traffic.
      i.e., SNAT has already been done in egress pipeline and now the
      packet has entered the ingress pipeline as part of a reply.  It is
      unSNATted here.
    </p>

    <p>Ingress Table 3: UNSNAT on Gateway Routers</p>

    <ul>
      <li>
        <p>
          If the Gateway router has been configured to force SNAT any
          previously DNATted packets to <var>B</var>, a priority-110 flow
          matches <code>ip &amp;&amp; ip4.dst == <var>B</var></code> with
          an action <code>ct_snat; next;</code>.
        </p>

        <p>
          If the Gateway router has been configured to force SNAT any
          previously load-balanced packets to <var>B</var>, a priority-100 flow
          matches <code>ip &amp;&amp; ip4.dst == <var>B</var></code> with
          an action <code>ct_snat; next;</code>.
        </p>

        <p>
          For each NAT configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-90 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>B</var></code> with an action
          <code>ct_snat; next;</code>.
        </p>

        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <p>Ingress Table 3: UNSNAT on Distributed Routers</p>

    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>B</var> &amp;&amp; inport == <var>GW</var></code>,
          where <var>GW</var> is the logical router gateway port, with an
          action <code>ct_snat; next;</code>.
        </p>

        <p>
          If the NAT rule cannot be handled in a distributed manner, then
          the priority-100 flow above is only programmed on the
          <code>redirect-chassis</code>.
        </p>

        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-50 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>B</var></code> with an action
          <code>REGBIT_NAT_REDIRECT = 1; next;</code>.  This flow is for
          east/west traffic to a NAT destination IPv4 address.  By
          setting the <code>REGBIT_NAT_REDIRECT</code> flag, in the
          ingress table <code>Gateway Redirect</code> this will trigger a
          redirect to the instance of the gateway port on the
          <code>redirect-chassis</code>.
        </p>

        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 4: DNAT</h3>

    <p>
      Packets enter the pipeline with destination IP address that needs to
      be DNATted from a virtual IP address to a real IP address.  Packets
      in the reverse direction needs to be unDNATed.
    </p>

    <p>Ingress Table 4: DNAT on Gateway Routers</p>

    <ul>
      <li>
        For all the configured load balancing rules for Gateway router in
        <code>OVN_Northbound</code> database that includes a L4 port
        <var>PORT</var> of protocol <var>P</var> and IPv4 address
        <var>VIP</var>, a priority-120 flow that matches on
        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP</var>
        &amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst == <var>PORT
        </var></code> with an action of <code>ct_lb(<var>args</var>)</code>,
        where <var>args</var> contains comma separated IPv4 addresses (and
        optional port numbers) to load balance to.  If the Gateway router
        is configured to force SNAT any load-balanced packets, the above
        action will be replaced by <code>flags.force_snat_for_lb = 1;
        ct_lb(<var>args</var>);</code>.
      </li>

      <li>
        For all the configured load balancing rules for Gateway router in
        <code>OVN_Northbound</code> database that includes a L4 port
        <var>PORT</var> of protocol <var>P</var> and IPv4 address
        <var>VIP</var>, a priority-120 flow that matches on
        <code>ct.est &amp;&amp; ip &amp;&amp; ip4.dst == <var>VIP</var>
        &amp;&amp; <var>P</var> &amp;&amp; <var>P</var>.dst == <var>PORT
        </var></code> with an action of <code>ct_dnat;</code>.
        If the Gateway router is configured to force SNAT any load-balanced
        packets, the above action will be replaced by
        <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
      </li>

      <li>
        For all the configured load balancing rules for Gateway router in
        <code>OVN_Northbound</code> database that includes just an IP address
        <var>VIP</var> to match on, a priority-110 flow that matches on
        <code>ct.new &amp;&amp; ip &amp;&amp; ip4.dst ==
        <var>VIP</var></code> with an action of
        <code>ct_lb(<var>args</var>)</code>, where <var>args</var> contains
        comma separated IPv4 addresses.  If the Gateway router
        is configured to force SNAT any load-balanced packets, the above
        action will be replaced by <code>flags.force_snat_for_lb = 1;
        ct_lb(<var>args</var>);</code>.
      </li>

      <li>
        For all the configured load balancing rules for Gateway router in
        <code>OVN_Northbound</code> database that includes just an IP address
        <var>VIP</var> to match on, a priority-110 flow that matches on
        <code>ct.est &amp;&amp; ip &amp;&amp; ip4.dst ==
        <var>VIP</var></code> with an action of <code>ct_dnat;</code>.
        If the Gateway router is configured to force SNAT any load-balanced
        packets, the above action will be replaced by
        <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
      </li>

      <li>
        For each configuration in the OVN Northbound database, that asks
        to change the destination IP address of a packet from <var>A</var> to
        <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
        ip4.dst == <var>A</var></code> with an action
        <code>flags.loopback = 1; ct_dnat(<var>B</var>);</code>.  If the
        Gateway router is configured to force SNAT any DNATed packet,
        the above action will be replaced by
        <code>flags.force_snat_for_dnat = 1; flags.loopback = 1;
        ct_dnat(<var>B</var>);</code>.
      </li>

      <li>
        For all IP packets of a Gateway router, a priority-50 flow with an
        action <code>flags.loopback = 1; ct_dnat;</code>.
      </li>

      <li>
        A priority-0 logical flow with match <code>1</code> has actions
        <code>next;</code>.
      </li>
    </ul>

    <p>Ingress Table 4: DNAT on Distributed Routers</p>

    <p>
      On distributed routers, the DNAT table only handles packets
      with destination IP address that needs to be DNATted from a
      virtual IP address to a real IP address.  The unDNAT processing
      in the reverse direction is handled in a separate table in the
      egress pipeline.
    </p>

    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the destination IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-100 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>B</var> &amp;&amp; inport == <var>GW</var></code>,
          where <var>GW</var> is the logical router gateway port, with an
          action <code>ct_dnat(<var>B</var>);</code>.
        </p>

        <p>
          If the NAT rule cannot be handled in a distributed manner, then
          the priority-100 flow above is only programmed on the
          <code>redirect-chassis</code>.
        </p>

        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the destination IP address of a packet from <var>A</var> to
          <var>B</var>, a priority-50 flow matches <code>ip &amp;&amp;
          ip4.dst == <var>B</var></code> with an action
          <code>REGBIT_NAT_REDIRECT = 1; next;</code>.  This flow is for
          east/west traffic to a NAT destination IPv4 address.  By
          setting the <code>REGBIT_NAT_REDIRECT</code> flag, in the
          ingress table <code>Gateway Redirect</code> this will trigger a
          redirect to the instance of the gateway port on the
          <code>redirect-chassis</code>.
        </p>

        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 5: IP Routing</h3>

    <p>
      A packet that arrives at this table is an IP packet that should be
      routed to the address in <code>ip4.dst</code> or
      <code>ip6.dst</code>.  This table implements IP routing, setting
      <code>reg0</code> (or <code>xxreg0</code> for IPv6) to the next-hop IP
      address (leaving <code>ip4.dst</code> or <code>ip6.dst</code>, the
      packet's final destination, unchanged) and advances to the next
      table for ARP resolution.  It also sets <code>reg1</code> (or
      <code>xxreg1</code>) to the IP address owned by the selected router
      port (ingress table <code>ARP Request</code> will generate an ARP
      request, if needed, with <code>reg0</code> as the target protocol
      address and <code>reg1</code> as the source protocol address).
    </p>

    <p>
      This table contains the following logical flows:
    </p>

    <ul>
      <li>
        <p>
          For distributed logical routers where one of the logical router
          ports specifies a <code>redirect-chassis</code>, a priority-300
          logical flow with match <code>REGBIT_NAT_REDIRECT == 1</code> has
          actions <code>ip.ttl--; next;</code>.  The <code>outport</code>
          will be set later in the Gateway Redirect table.
        </p>
      </li>

      <li>
        <p>
          IPv4 routing table.  For each route to IPv4 network <var>N</var> with
          netmask <var>M</var>, on router port <var>P</var> with IP address
          <var>A</var> and Ethernet
          address <var>E</var>, a logical flow with match <code>ip4.dst ==
          <var>N</var>/<var>M</var></code>, whose priority is the number of
          1-bits in <var>M</var>, has the following actions:
        </p>

        <pre>
ip.ttl--;
reg0 = <var>G</var>;
reg1 = <var>A</var>;
eth.src = <var>E</var>;
outport = <var>P</var>;
flags.loopback = 1;
next;
        </pre>

        <p>
          (Ingress table 1 already verified that <code>ip.ttl--;</code> will
          not yield a TTL exceeded error.)
        </p>

        <p>
          If the route has a gateway, <var>G</var> is the gateway IP address.
          Instead, if the route is from a configured static route, <var>G</var>
          is the next hop IP address.  Else it is <code>ip4.dst</code>.
        </p>
      </li>

      <li>
        <p>
          IPv6 routing table.  For each route to IPv6 network
          <var>N</var> with netmask <var>M</var>, on router port
          <var>P</var> with IP address <var>A</var> and Ethernet address
          <var>E</var>, a logical flow with match in CIDR notation
          <code>ip6.dst == <var>N</var>/<var>M</var></code>,
          whose priority is the integer value of <var>M</var>, has the
          following actions:
        </p>

        <pre>
ip.ttl--;
xxreg0 = <var>G</var>;
xxreg1 = <var>A</var>;
eth.src = <var>E</var>;
outport = <var>P</var>;
flags.loopback = 1;
next;
        </pre>

        <p>
          (Ingress table 1 already verified that <code>ip.ttl--;</code> will
          not yield a TTL exceeded error.)
        </p>

        <p>
          If the route has a gateway, <var>G</var> is the gateway IP address.
          Instead, if the route is from a configured static route, <var>G</var>
          is the next hop IP address.  Else it is <code>ip6.dst</code>.
        </p>

        <p>
          If the address <var>A</var> is in the link-local scope, the
          route will be limited to sending on the ingress port.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 6: ARP/ND Resolution</h3>

    <p>
      Any packet that reaches this table is an IP packet whose next-hop
      IPv4 address is in <code>reg0</code> or IPv6 address is in
      <code>xxreg0</code>.  (<code>ip4.dst</code> or
      <code>ip6.dst</code> contains the final destination.)  This table
      resolves the IP address in <code>reg0</code> (or
      <code>xxreg0</code>) into an output port in <code>outport</code>
      and an Ethernet address in <code>eth.dst</code>, using the
      following flows:
    </p>

    <ul>
      <li>
        <p>
          For distributed logical routers where one of the logical router
          ports specifies a <code>redirect-chassis</code>, a priority-200
          logical flow with match <code>REGBIT_NAT_REDIRECT == 1</code> has
          actions <code>eth.dst = <var>E</var>; next;</code>, where
          <var>E</var> is the ethernet address of the router's distributed
          gateway port.
        </p>
      </li>

      <li>
        <p>
          Static MAC bindings.  MAC bindings can be known statically based on
          data in the <code>OVN_Northbound</code> database.  For router ports
          connected to logical switches, MAC bindings can be known statically
          from the <code>addresses</code> column in the
          <code>Logical_Switch_Port</code> table.  For router ports
          connected to other logical routers, MAC bindings can be known
          statically from the <code>mac</code> and <code>networks</code>
          column in the <code>Logical_Router_Port</code> table.
        </p>

        <p>
          For each IPv4 address <var>A</var> whose host is known to have
          Ethernet address <var>E</var> on router port <var>P</var>, a
          priority-100 flow with match <code>outport === <var>P</var>
          &amp;&amp; reg0 == <var>A</var></code> has actions
          <code>eth.dst = <var>E</var>; next;</code>.
        </p>

        <p>
          For each IPv6 address <var>A</var> whose host is known to have
          Ethernet address <var>E</var> on router port <var>P</var>, a
          priority-100 flow with match <code>outport === <var>P</var>
          &amp;&amp; xxreg0 == <var>A</var></code> has actions
          <code>eth.dst = <var>E</var>; next;</code>.
        </p>

        <p>
          For each logical router port with an IPv4 address <var>A</var> and
          a mac address of <var>E</var> that is reachable via a different
          logical router port <var>P</var>, a priority-100 flow with
          match <code>outport === <var>P</var> &amp;&amp; reg0 ==
          <var>A</var></code> has actions <code>eth.dst = <var>E</var>;
          next;</code>.
        </p>

        <p>
          For each logical router port with an IPv6 address <var>A</var> and
          a mac address of <var>E</var> that is reachable via a different
          logical router port <var>P</var>, a priority-100 flow with
          match <code>outport === <var>P</var> &amp;&amp; xxreg0 ==
          <var>A</var></code> has actions <code>eth.dst = <var>E</var>;
          next;</code>.
        </p>
      </li>

      <li>
        <p>
          Dynamic MAC bindings.  These flows resolve MAC-to-IP bindings
          that have become known dynamically through ARP or neighbor
          discovery.  (The ingress table <code>ARP Request</code> will
          issue an ARP or neighbor solicitation request for cases where
          the binding is not yet known.)
        </p>

        <p>
          A priority-0 logical flow with match <code>ip4</code> has actions
          <code>get_arp(outport, reg0); next;</code>.
        </p>

        <p>
          A priority-0 logical flow with match <code>ip6</code> has actions
          <code>get_nd(outport, xxreg0); next;</code>.
        </p>
      </li>
    </ul>

    <h3>Ingress Table 7: Gateway Redirect</h3>

    <p>
      For distributed logical routers where one of the logical router
      ports specifies a <code>redirect-chassis</code>, this table redirects
      certain packets to the distributed gateway port instance on the
      <code>redirect-chassis</code>.  This table has the following flows:
    </p>

    <ul>
      <li>
        A priority-200 logical flow with match
        <code>REGBIT_NAT_REDIRECT == 1</code> has actions
        <code>outport = <var>CR</var>; next;</code>, where <var>CR</var>
        is the <code>chassisredirect</code> port representing the instance
        of the logical router distributed gateway port on the
        <code>redirect-chassis</code>.
      </li>

      <li>
        A priority-150 logical flow with match
        <code>outport == <var>GW</var> &amp;&amp;
        eth.dst == 00:00:00:00:00:00</code> has actions
        <code>outport = <var>CR</var>; next;</code>, where
        <var>GW</var> is the logical router distributed gateway
        port and <var>CR</var> is the <code>chassisredirect</code>
        port representing the instance of the logical router
        distributed gateway port on the
        <code>redirect-chassis</code>.
      </li>

      <li>
        For each NAT rule in the OVN Northbound database that can
        be handled in a distributed manner, a priority-100 logical
        flow with match <code>ip4.src == <var>B</var> &amp;&amp;
        outport == <var>GW</var></code>, where <var>GW</var> is
        the logical router distributed gateway port, with actions
        <code>next;</code>.
      </li>

      <li>
        A priority-50 logical flow with match
        <code>outport == <var>GW</var></code> has actions
        <code>outport = <var>CR</var>; next;</code>, where
        <var>GW</var> is the logical router distributed gateway
        port and <var>CR</var> is the <code>chassisredirect</code>
        port representing the instance of the logical router
        distributed gateway port on the
        <code>redirect-chassis</code>.
      </li>

      <li>
        A priority-0 logical flow with match <code>1</code> has actions
        <code>next;</code>.
      </li>
    </ul>

    <h3>Ingress Table 8: ARP Request</h3>

    <p>
      In the common case where the Ethernet destination has been resolved, this
      table outputs the packet.  Otherwise, it composes and sends an ARP
      request.  It holds the following flows:
    </p>

    <ul>
      <li>
        <p>
          Unknown MAC address.  A priority-100 flow with match <code>eth.dst ==
          00:00:00:00:00:00</code> has the following actions:
        </p>

        <pre>
arp {
    eth.dst = ff:ff:ff:ff:ff:ff;
    arp.spa = reg1;
    arp.tpa = reg0;
    arp.op = 1;  /* ARP request. */
    output;
};
        </pre>

        <p>
          (Ingress table <code>IP Routing</code> initialized <code>reg1</code>
          with the IP address owned by <code>outport</code> and
          <code>reg0</code> with the next-hop IP address)
        </p>

        <p>
          The IP packet that triggers the ARP request is dropped.
        </p>
      </li>

      <li>
        Known MAC address.  A priority-0 flow with match <code>1</code> has
        actions <code>output;</code>.
      </li>
    </ul>

    <h3>Egress Table 0: UNDNAT</h3>

    <p>
      This is for already established connections' reverse traffic.
      i.e., DNAT has already been done in ingress pipeline and now the
      packet has entered the egress pipeline as part of a reply.  For
      NAT on a distributed router, it is unDNATted here.  For Gateway
      routers, the unDNAT processing is carried out in the ingress DNAT
      table.
    </p>

    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database that asks
          to change the destination IP address of a packet from an IP
          address of <var>A</var> to <var>B</var>, a priority-100 flow
          matches <code>ip &amp;&amp; ip4.src == <var>B</var>
          &amp;&amp; outport == <var>GW</var></code>, where <var>GW</var>
          is the logical router gateway port, with an action
          <code>ct_dnat;</code>.
        </p>

        <p>
          If the NAT rule cannot be handled in a distributed manner, then
          the priority-100 flow above is only programmed on the
          <code>redirect-chassis</code>.
        </p>

        <p>
          If the NAT rule can be handled in a distributed manner, then
          there is an additional action
          <code>eth.src = <var>EA</var>;</code>, where <var>EA</var>
          is the ethernet address associated with the IP address
          <var>A</var> in the NAT rule.  This allows upstream MAC
          learning to point to the correct chassis.
        </p>
      </li>

      <li>
        A priority-0 logical flow with match <code>1</code> has actions
        <code>next;</code>.
      </li>
    </ul>

    <h3>Egress Table 1: SNAT</h3>

    <p>
      Packets that are configured to be SNATed get their source IP address
      changed based on the configuration in the OVN Northbound database.
    </p>

    <p>Egress Table 1: SNAT on Gateway Routers</p>

    <ul>
      <li>
        <p>
          If the Gateway router in the OVN Northbound database has been
          configured to force SNAT a packet (that has been previously DNATted)
          to <var>B</var>, a priority-100 flow matches
          <code>flags.force_snat_for_dnat == 1 &amp;&amp; ip</code> with an
          action <code>ct_snat(<var>B</var>);</code>.
        </p>
        <p>
          If the Gateway router in the OVN Northbound database has been
          configured to force SNAT a packet (that has been previously
          load-balanced) to <var>B</var>, a priority-100 flow matches
          <code>flags.force_snat_for_lb == 1 &amp;&amp; ip</code> with an
          action <code>ct_snat(<var>B</var>);</code>.
        </p>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from an IP address of
          <var>A</var> or to change the source IP address of a packet that
          belongs to network <var>A</var> to <var>B</var>, a flow matches
          <code>ip &amp;&amp; ip4.src == <var>A</var></code> with an action
          <code>ct_snat(<var>B</var>);</code>.  The priority of the flow
          is calculated based on the mask of <var>A</var>, with matches
          having larger masks getting higher priorities.
        </p>
        <p>
          A priority-0 logical flow with match <code>1</code> has actions
          <code>next;</code>.
        </p>
      </li>
    </ul>

    <p>Egress Table 1: SNAT on Distributed Routers</p>

    <ul>
      <li>
        <p>
          For each configuration in the OVN Northbound database, that asks
          to change the source IP address of a packet from an IP address of
          <var>A</var> or to change the source IP address of a packet that
          belongs to network <var>A</var> to <var>B</var>, a flow matches
          <code>ip &amp;&amp; ip4.src == <var>A</var> &amp;&amp;
          outport == <var>GW</var></code>, where <var>GW</var> is the
          logical router gateway port, with an action
          <code>ct_snat(<var>B</var>);</code>.  The priority of the flow
          is calculated based on the mask of <var>A</var>, with matches
          having larger masks getting higher priorities.
        </p>

        <p>
          If the NAT rule cannot be handled in a distributed manner, then
          the flow above is only programmed on the
          <code>redirect-chassis</code>.
        </p>

        <p>
          If the NAT rule can be handled in a distributed manner, then
          there is an additional action
          <code>eth.src = <var>EA</var>;</code>, where <var>EA</var>
          is the ethernet address associated with the IP address
          <var>A</var> in the NAT rule.  This allows upstream MAC
          learning to point to the correct chassis.
        </p>
      </li>

      <li>
        A priority-0 logical flow with match <code>1</code> has actions
        <code>next;</code>.
      </li>
    </ul>

    <h3>Egress Table 2: Egress Loopback</h3>

    <p>
      For distributed logical routers where one of the logical router
      ports specifies a <code>redirect-chassis</code>.
    </p>

    <p>
      Earlier in the ingress pipeline, some east-west traffic was
      redirected to the <code>chassisredirect</code> port, based on
      flows in the <code>UNSNAT</code> and <code>DNAT</code> ingress
      tables setting the <code>REGBIT_NAT_REDIRECT</code> flag, which
      then triggered a match to a flow in the
      <code>Gateway Redirect</code> ingress table.  The intention was
      not to actually send traffic out the distributed gateway port
      instance on the <code>redirect-chassis</code>.  This traffic was
      sent to the distributed gateway port instance in order for DNAT
      and/or SNAT processing to be applied.
    </p>

    <p>
      While UNDNAT and SNAT processing have already occurred by this
      point, this traffic needs to be forced through egress loopback on
      this distributed gateway port instance, in order for UNSNAT and
      DNAT processing to be applied, and also for IP routing and ARP
      resolution after all of the NAT processing, so that the packet can
      be forwarded to the destination.
    </p>

    <p>
      This table has the following flows:
    </p>

    <ul>
      <li>
        <p>
          For each NAT rule in the OVN Northbound database on a
          distributed router, a priority-100 logical flow with match
          <code>ip4.dst == <var>E</var> &amp;&amp;
          outport == <var>GW</var></code>, where <var>E</var> is the
          external IP address specified in the NAT rule, and <var>GW</var>
          is the logical router distributed gateway port, with the
          following actions:
        </p>

        <pre>
clone {
    ct_clear;
    inport = outport;
    outport = "";
    flags = 0;
    flags.loopback = 1;
    reg0 = 0;
    reg1 = 0;
    ...
    reg9 = 0;
    REGBIT_EGRESS_LOOPBACK = 1;
    next(pipeline=ingress, table=0);
};
        </pre>

        <p>
          <code>flags.loopback</code> is set since in_port is unchanged
          and the packet may return back to that port after NAT processing.
          <code>REGBIT_EGRESS_LOOPBACK</code> is set to indicate that
          egress loopback has occurred, in order to skip the source IP
          address check against the router address.
        </p>
      </li>

      <li>
        A priority-0 logical flow with match <code>1</code> has actions
        <code>next;</code>.
      </li>
    </ul>

    <h3>Egress Table 3: Delivery</h3>

    <p>
      Packets that reach this table are ready for delivery.  It contains
      priority-100 logical flows that match packets on each enabled logical
      router port, with action <code>output;</code>.
    </p>

</manpage>