ovn: Add a case of policy based routing.

ovn: Add a case of policy based routing.

OVN currently supports multiple gateway routers (residing on
different chassis) connected to the same logical topology.

When external traffic enters the logical topology, they can enter
from any gateway routers and reach its eventual destination. This
is achieved with proper static routes configured on the gateway
routers.

But when traffic is initiated in the logical space by a logical
port, we do not have a good way to distribute that traffic across
multiple gateway routers.

This commit introduces one particular way to do it. Based on the
source IP address or source IP network of the packet, we can now
jump to a specific gateway router.

This is very useful for a specific use case of Kubernetes.
When traffic is initiated inside a container heading to outside world,
we want to be able to send such traffic outside the gateway router
residing in the same host as that of the container. Since each
host gets a specific subnet, we can use source IP address based
policy routing to decide on the gateway router.

Rationale for using the same routing table for both source and
destination IP address based routing:

Some hardware network vendors support policy routing in a different table
on arbitrary "match".  And when a packet enters, if there is a match
in policy based routing table, the default routing table is not
consulted at all.  In case of OVN, we mainly want policy based routing
for north-south traffic. We want east-west traffic to flow as-is. Creating
a separate table for policy based routing complicates the configuration
quite a bit. For e.g., if we have a source IP network based rule added,
to decide a particular gateway router as a next hop, we should add rules at
a higher priority for all the connected routes to make sure that east-west
traffic is not effected in the policy based routing table itself.

Signed-off-by: Gurucharan Shetty <guru@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>