in the reverse direction needs to be unDNATed.
</p>
- <p>Ingress Table 4: DNAT on Gateway Routers</p>
+ <p>Ingress Table 4: Load balancing DNAT rules</p>
+
+ <p>
+ Following load balancing DNAT flows are added for Gateway router or
+ Router with gateway port. These flows are programmed only on the
+ <code>redirect-chassis</code>.
+ </p>
<ul>
<li>
- For all the configured load balancing rules for Gateway router in
- <code>OVN_Northbound</code> database that includes a L4 port
- <var>PORT</var> of protocol <var>P</var> and IPv4 address
- <var>VIP</var>, a priority-120 flow that matches on
+ For all the configured load balancing rules for a Gateway router or
+ Router with gateway port in <code>OVN_Northbound</code> database that
+ includes a L4 port <var>PORT</var> of protocol <var>P</var> and IPv4
+ address <var>VIP</var>, a priority-120 flow that matches on
<code>ct.new && ip && ip4.dst == <var>VIP</var>
&& <var>P</var> && <var>P</var>.dst == <var>PORT
</var></code> with an action of <code>ct_lb(<var>args</var>)</code>,
where <var>args</var> contains comma separated IPv4 addresses (and
- optional port numbers) to load balance to. If the Gateway router
- is configured to force SNAT any load-balanced packets, the above
- action will be replaced by <code>flags.force_snat_for_lb = 1;
+ optional port numbers) to load balance to. If the router is configured
+ to force SNAT any load-balanced packets, the above action will be
+ replaced by <code>flags.force_snat_for_lb = 1;
ct_lb(<var>args</var>);</code>.
</li>
<li>
- For all the configured load balancing rules for Gateway router in
+ For all the configured load balancing rules for a router in
<code>OVN_Northbound</code> database that includes a L4 port
<var>PORT</var> of protocol <var>P</var> and IPv4 address
<var>VIP</var>, a priority-120 flow that matches on
<code>ct.est && ip && ip4.dst == <var>VIP</var>
&& <var>P</var> && <var>P</var>.dst == <var>PORT
- </var></code> with an action of <code>ct_dnat;</code>.
- If the Gateway router is configured to force SNAT any load-balanced
- packets, the above action will be replaced by
- <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
+ </var></code> with an action of <code>ct_dnat;</code>. If the router is
+ configured to force SNAT any load-balanced packets, the above action
+ will be replaced by <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
</li>
<li>
- For all the configured load balancing rules for Gateway router in
+ For all the configured load balancing rules for a router in
<code>OVN_Northbound</code> database that includes just an IP address
<var>VIP</var> to match on, a priority-110 flow that matches on
<code>ct.new && ip && ip4.dst ==
<var>VIP</var></code> with an action of
<code>ct_lb(<var>args</var>)</code>, where <var>args</var> contains
- comma separated IPv4 addresses. If the Gateway router
- is configured to force SNAT any load-balanced packets, the above
- action will be replaced by <code>flags.force_snat_for_lb = 1;
- ct_lb(<var>args</var>);</code>.
+ comma separated IPv4 addresses. If the router is configured to force
+ SNAT any load-balanced packets, the above action will be replaced by
+ <code>flags.force_snat_for_lb = 1; ct_lb(<var>args</var>);</code>.
</li>
<li>
- For all the configured load balancing rules for Gateway router in
+ For all the configured load balancing rules for a router in
<code>OVN_Northbound</code> database that includes just an IP address
<var>VIP</var> to match on, a priority-110 flow that matches on
<code>ct.est && ip && ip4.dst ==
<var>VIP</var></code> with an action of <code>ct_dnat;</code>.
- If the Gateway router is configured to force SNAT any load-balanced
+ If the router is configured to force SNAT any load-balanced
packets, the above action will be replaced by
<code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
</li>
+ </ul>
+ <p>Ingress Table 4: DNAT on Gateway Routers</p>
+
+ <ul>
<li>
For each configuration in the OVN Northbound database, that asks
to change the destination IP address of a packet from <var>A</var> to
</p>
<ul>
+ <li>
+ <p>
+ For all the configured load balancing rules for a router with gateway
+ port in <code>OVN_Northbound</code> database that includes an IPv4
+ address <code>VIP</code>, for every backend IPv4 address <var>B</var>
+ defined for the <code>VIP</code> a priority-120 flow is programmed on
+ <code>redirect-chassis</code> that matches
+ <code>ip && ip4.src == <var>B</var> &&
+ outport == <var>GW</var></code>, where <var>GW</var> is the logical
+ router gateway port with an action <code>ct_dnat;</code>. If the
+ backend IPv4 address <var>B</var> is also configured with L4 port
+ <var>PORT</var> of protocol <var>P</var>, then the
+ match also includes <code>P.src</code> == <var>PORT</var>.
+ </p>
+
+ <p>
+ If the router is configured to force SNAT any load-balanced packets,
+ above action will be replaced by
+ <code>flags.force_snat_for_lb = 1; ct_dnat;</code>.
+ </p>
+ </li>
+
<li>
<p>
For each configuration in the OVN Northbound database that asks
static void
add_router_lb_flow(struct hmap *lflows, struct ovn_datapath *od,
struct ds *match, struct ds *actions, int priority,
- const char *lb_force_snat_ip)
+ const char *lb_force_snat_ip, char *backend_ips,
+ bool is_udp)
{
/* A match and actions for new connections. */
char *new_match = xasprintf("ct.new && %s", ds_cstr(match));
free(new_match);
free(est_match);
+
+ if (!od->l3dgw_port || !od->l3redirect_port || !backend_ips) {
+ return;
+ }
+
+ /* Add logical flows to UNDNAT the load balanced reverse traffic in
+ * the router egress pipleine stage - S_ROUTER_OUT_UNDNAT if the logical
+ * router has a gateway router port associated.
+ */
+ struct ds undnat_match = DS_EMPTY_INITIALIZER;
+ ds_put_cstr(&undnat_match, "ip4 && (");
+ char *start, *next, *ip_str;
+ start = next = xstrdup(backend_ips);
+ ip_str = strsep(&next, ",");
+ bool backend_ips_found = false;
+ while (ip_str && ip_str[0]) {
+ char *ip_address = NULL;
+ uint16_t port = 0;
+ ip_address_and_port_from_lb_key(ip_str, &ip_address, &port);
+ if (!ip_address) {
+ break;
+ }
+
+ ds_put_format(&undnat_match, "(ip4.src == %s", ip_address);
+ free(ip_address);
+ if (port) {
+ ds_put_format(&undnat_match, " && %s.src == %d) || ",
+ is_udp ? "udp" : "tcp", port);
+ } else {
+ ds_put_cstr(&undnat_match, ") || ");
+ }
+ ip_str = strsep(&next, ",");
+ backend_ips_found = true;
+ }
+
+ free(start);
+ if (!backend_ips_found) {
+ ds_destroy(&undnat_match);
+ return;
+ }
+ ds_chomp(&undnat_match, ' ');
+ ds_chomp(&undnat_match, '|');
+ ds_chomp(&undnat_match, '|');
+ ds_chomp(&undnat_match, ' ');
+ ds_put_format(&undnat_match, ") && outport == %s && "
+ "is_chassis_resident(%s)", od->l3dgw_port->json_key,
+ od->l3redirect_port->json_key);
+ if (lb_force_snat_ip) {
+ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 120,
+ ds_cstr(&undnat_match),
+ "flags.force_snat_for_lb = 1; ct_dnat;");
+ } else {
+ ovn_lflow_add(lflows, od, S_ROUTER_OUT_UNDNAT, 120,
+ ds_cstr(&undnat_match), "ct_dnat;");
+ }
+
+ ds_destroy(&undnat_match);
}
static void
}
/* Load balancing and packet defrag are only valid on
- * Gateway routers. */
- if (!smap_get(&od->nbr->options, "chassis")) {
+ * Gateway routers or router with gateway port. */
+ if (!smap_get(&od->nbr->options, "chassis") && !od->l3dgw_port) {
continue;
}
ip_address);
free(ip_address);
+ int prio = 110;
+ bool is_udp = lb->protocol && !strcmp(lb->protocol, "udp") ?
+ true : false;
if (port) {
- if (lb->protocol && !strcmp(lb->protocol, "udp")) {
+ if (is_udp) {
ds_put_format(&match, " && udp && udp.dst == %d",
port);
} else {
ds_put_format(&match, " && tcp && tcp.dst == %d",
port);
}
- add_router_lb_flow(lflows, od, &match, &actions, 120,
- lb_force_snat_ip);
- } else {
- add_router_lb_flow(lflows, od, &match, &actions, 110,
- lb_force_snat_ip);
+ prio = 120;
+ }
+
+ if (od->l3redirect_port) {
+ ds_put_format(&match, " && is_chassis_resident(%s)",
+ od->l3redirect_port->json_key);
}
+ add_router_lb_flow(lflows, od, &match, &actions, prio,
+ lb_force_snat_ip, node->value, is_udp);
}
}
/connection dropped.*/d"])
AT_CLEANUP
+AT_SETUP([ovn -- load balancing in router with gateway router port])
+AT_KEYWORDS([ovnlb])
+
+CHECK_CONNTRACK()
+CHECK_CONNTRACK_NAT()
+ovn_start
+OVS_TRAFFIC_VSWITCHD_START()
+ADD_BR([br-int])
+
+# Set external-ids in br-int needed for ovn-controller
+ovs-vsctl \
+ -- set Open_vSwitch . external-ids:system-id=hv1 \
+ -- set Open_vSwitch . external-ids:ovn-remote=unix:$ovs_base/ovn-sb/ovn-sb.sock \
+ -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \
+ -- set Open_vSwitch . external-ids:ovn-encap-ip=169.0.0.1 \
+ -- set bridge br-int fail-mode=secure other-config:disable-in-band=true
+
+# Start ovn-controller
+start_daemon ovn-controller
+
+# Logical network:
+# One LR R1 with switches foo (192.168.1.0/24), bar (192.168.2.0/24),
+# and alice (172.16.1.0/24) connected to it. The port between R1 and
+# alice is the router gateway port where the R1 LB rules are applied.
+#
+# foo -- R1 -- bar
+# |
+# alice ----
+
+ovn-nbctl lr-add R1
+
+ovn-nbctl ls-add foo
+ovn-nbctl ls-add bar
+ovn-nbctl ls-add alice
+
+ovn-nbctl lrp-add R1 foo 00:00:01:01:02:03 192.168.1.1/24
+ovn-nbctl lrp-add R1 bar 00:00:01:01:02:04 192.168.2.1/24
+ovn-nbctl lrp-add R1 alice 00:00:02:01:02:03 172.16.1.1/24 \
+ -- set Logical_Router_Port alice options:redirect-chassis=hv1
+
+# Connect foo to R1
+ovn-nbctl lsp-add foo rp-foo -- set Logical_Switch_Port rp-foo \
+ type=router options:router-port=foo \
+ -- lsp-set-addresses rp-foo router
+
+# Connect bar to R1
+ovn-nbctl lsp-add bar rp-bar -- set Logical_Switch_Port rp-bar \
+ type=router options:router-port=bar \
+ -- lsp-set-addresses rp-bar router
+
+# Connect alice to R1
+ovn-nbctl lsp-add alice rp-alice -- set Logical_Switch_Port rp-alice \
+ type=router options:router-port=alice \
+ -- lsp-set-addresses rp-alice router
+
+# Logical port 'foo1' in switch 'foo'.
+ADD_NAMESPACES(foo1)
+ADD_VETH(foo1, foo1, br-int, "192.168.1.2/24", "f0:00:00:01:02:03", \
+ "192.168.1.1")
+ovn-nbctl lsp-add foo foo1 \
+-- lsp-set-addresses foo1 "f0:00:00:01:02:03 192.168.1.2"
+
+# Logical port 'foo2' in switch 'foo'.
+ADD_NAMESPACES(foo2)
+ADD_VETH(foo2, foo2, br-int, "192.168.1.3/24", "f0:00:00:01:02:06", \
+ "192.168.1.1")
+ovn-nbctl lsp-add foo foo2 \
+-- lsp-set-addresses foo2 "f0:00:00:01:02:06 192.168.1.3"
+
+# Logical port 'bar1' in switch 'bar'.
+ADD_NAMESPACES(bar1)
+ADD_VETH(bar1, bar1, br-int, "192.168.2.2/24", "f0:00:00:01:02:04", \
+ "192.168.2.1")
+ovn-nbctl lsp-add bar bar1 \
+-- lsp-set-addresses bar1 "f0:00:00:01:02:04 192.168.2.2"
+
+# Logical port 'alice1' in switch 'alice'.
+ADD_NAMESPACES(alice1)
+ADD_VETH(alice1, alice1, br-int, "172.16.1.2/24", "f0:00:00:01:02:05", \
+ "172.16.1.1")
+ovn-nbctl lsp-add alice alice1 \
+-- lsp-set-addresses alice1 "f0:00:00:01:02:05 172.16.1.2"
+
+# Config OVN load-balancer with a VIP.
+uuid=`ovn-nbctl create load_balancer vips:172.16.1.10="192.168.1.2,192.168.2.2"`
+ovn-nbctl set logical_router R1 load_balancer=$uuid
+
+# Config OVN load-balancer with another VIP (this time with ports).
+ovn-nbctl set load_balancer $uuid vips:'"172.16.1.11:8000"'='"192.168.1.2:80,192.168.2.2:80"'
+
+# Wait for ovn-controller to catch up.
+ovn-nbctl --wait=hv sync
+OVS_WAIT_UNTIL([ovs-ofctl -O OpenFlow13 dump-groups br-int | \
+grep 'nat(dst=192.168.2.2:80)'])
+
+# Start webservers in 'foo1', 'bar1'.
+OVS_START_L7([foo1], [http])
+OVS_START_L7([bar1], [http])
+
+dnl Should work with the virtual IP address through NAT
+for i in `seq 1 20`; do
+ echo Request $i
+ NS_CHECK_EXEC([alice1], [wget 172.16.1.10 -t 5 -T 1 --retry-connrefused -v -o wget$i.log])
+done
+
+dnl Each server should have at least one connection.
+AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.10) |
+sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
+tcp,orig=(src=172.16.1.2,dst=172.16.1.10,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
+tcp,orig=(src=172.16.1.2,dst=172.16.1.10,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
+])
+
+dnl Test load-balancing that includes L4 ports in NAT.
+for i in `seq 1 20`; do
+ echo Request $i
+ NS_CHECK_EXEC([alice1], [wget 172.16.1.11:8000 -t 5 -T 1 --retry-connrefused -v -o wget$i.log])
+done
+
+dnl Each server should have at least one connection.
+AT_CHECK([ovs-appctl dpctl/dump-conntrack | FORMAT_CT(172.16.1.11) |
+sed -e 's/zone=[[0-9]]*/zone=<cleared>/'], [0], [dnl
+tcp,orig=(src=172.16.1.2,dst=172.16.1.11,sport=<cleared>,dport=<cleared>),reply=(src=192.168.1.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
+tcp,orig=(src=172.16.1.2,dst=172.16.1.11,sport=<cleared>,dport=<cleared>),reply=(src=192.168.2.2,dst=172.16.1.2,sport=<cleared>,dport=<cleared>),zone=<cleared>,protoinfo=(state=<cleared>)
+])
+
+OVS_APP_EXIT_AND_WAIT([ovn-controller])
+
+as ovn-sb
+OVS_APP_EXIT_AND_WAIT([ovsdb-server])
+
+as ovn-nb
+OVS_APP_EXIT_AND_WAIT([ovsdb-server])
+
+as northd
+OVS_APP_EXIT_AND_WAIT([ovn-northd])
+
+as
+OVS_TRAFFIC_VSWITCHD_STOP(["/failed to query port patch-.*/d
+/connection dropped.*/d"])
+AT_CLEANUP
+
AT_SETUP([ovn -- DNAT and SNAT on distributed router - N/S])
AT_KEYWORDS([ovnnat])
projects / ovs.git / commitdiff