Flavio Leitner [Mon, 26 Oct 2020 19:03:19 +0000 (16:03 -0300)]
flow: Support extra padding length.
Although not required, padding can be optionally added until
the packet length is MTU bytes. A packet with extra padding
currently fails sanity checks.
Ben Pfaff [Wed, 11 Nov 2020 23:22:44 +0000 (15:22 -0800)]
db-ctl-base: Add {in} and {not-in} set relational operators.
I would have found these useful for the OVN tests. The {in} operator
is the same as {<=}, but it's still useful to have the alternate syntax
because most of the time we think of set inclusion separately from
set subsets. The {not-in} operator is different from any existing
operator though.
Signed-off-by: Ben Pfaff <blp@ovn.org> Acked-by: Ilya Maximets <i.maximets@ovn.org>
Ben Pfaff [Thu, 21 Jan 2021 22:33:18 +0000 (14:33 -0800)]
pcap-file: Fix calculation of TCP payload length in tcp_reader_run().
The calculation in tcp_reader_run() failed to account for L2 padding.
This fixes the problem, by moving the existing function
tcp_payload_length() from a conntrack private header file into
dp-packet.h and renaming it to suit the dp_packet style.
Signed-off-by: Ben Pfaff <blp@ovn.org> Acked-by: Ilya Maximets <i.maximets@ovn.org>
Timothy Redaelli [Fri, 22 Jan 2021 14:50:07 +0000 (15:50 +0100)]
rhel: Update '--with-dpdk' argument for DPDK 20.11.
With DPDK 20.11, meson and pkgconfig are used instead of the old
Makefile-based system and so --with-dpdk option is changed to only
accept shared or static instead of the directory.
This commit uses --with-dpdk=shared since Fedora and RHEL ship shared
libraries of DPDK.
Fixes: 252e1e576443 ("dpdk: Update to use DPDK v20.11.") Signed-off-by: Timothy Redaelli <tredaelli@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Mon, 1 Feb 2021 11:41:35 +0000 (12:41 +0100)]
github: Don't fail the job if 'apt update' failed.
Some repositories that are enabled in GHA are not stable and lead
to 'apt update' failures:
E: The repository
'https://apt.postgresql.org/pub/repos/apt bionic-pgdg Release'
no longer has a Release file.
This causes the job failure.
In most cases we don't really need any packages from these failed
repositories, so we could try to continue the job.
Previously this kind of failures happened on older branches with
ubuntu 16.04 base image, so we have this workaround already there.
Now it started to fail on bionic images, so fixing there too.
Sunil Pai G [Thu, 28 Jan 2021 10:32:24 +0000 (10:32 +0000)]
acinclude: Remove default library for DPDK.
The default DPDK library used before this patch in case pkg-config
fails to find libdpdk is only valid for make based DPDK builds.
Hence remove them.
As a consequence, now this error message [1] is thrown when pkg-config
cannot find libdpdk instead of proceeding to check for a faulty
pkg-config and reporting incorrect error message [2].
Also, update the documentation to export PKG_CONFIG_PATH since on some
systems, the default install path for DPDK libraries is not present in
the default search path of pkg-config.
Ex: for Fedora 32 default pkg-config search path:
/usr/lib64/pkgconfig:/usr/share/pkgconfig
while by default Meson installs DPDK libraries at:
/usr/local/lib64/pkgconfig
[1] Package libdpdk was not found in the pkg-config search path.
Perhaps you should add the directory containing `libdpdk.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libdpdk', required by 'virtual:world', not found
[2] checking for DPDK... no
checking for faulty pkg-config version... yes
configure: error: Please upgrade pkg-config
Also, update the build documentation for AVX512 optimization.
Fixes: 252e1e576443 ("dpdk: Update to use DPDK v20.11.") Reported-by: Kevin Traynor <ktraynor@redhat.com> Acked-by: Kevin Traynor <ktraynor@redhat.com> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Fri, 29 Jan 2021 12:44:36 +0000 (13:44 +0100)]
ovsdb-doc: Add build dependency on dirs.py.
ovsdb-doc includes python code that requires dirs.py to exist.
This change fixes broken 'make manpage-check' target:
# make manpage-check
Traceback (most recent call last):
File "./ovsdb/ovsdb-doc", line 25, in <module>
import ovs.db.schema
File "/root/ovs/python/ovs/db/schema.py", line 19, in <module>
import ovs.db.types
File "/root/ovs/python/ovs/db/types.py", line 18, in <module>
import ovs.db.data
File "/root/ovs/python/ovs/db/data.py", line 22, in <module>
import ovs.jsonrpc
File "/root/ovs/python/ovs/jsonrpc.py", line 21, in <module>
import ovs.poller
File "/root/ovs/python/ovs/poller.py", line 23, in <module>
import ovs.vlog
File "/root/ovs/python/ovs/vlog.py", line 25, in <module>
import ovs.dirs
ModuleNotFoundError: No module named 'ovs.dirs'
Fixes: 943c4a325045 ("python: set ovs.dirs variables with build system values") Acked-by: Mark Gray <mark.d.gray@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Flavio Leitner [Wed, 28 Oct 2020 19:47:52 +0000 (16:47 -0300)]
ovs-ctl: Prepend OVS binary directories to PATH.
The inclusion of LSB functions in ovs-lib resets $PATH to
system's default. Then ovs-ctl appends missing directories
including the OVS default ones $sbindir and $bindir.
The problem is that the wrong binaries can be used if they
are available in the system's default locations because of
the PATH wrong order. The same issue happens if one changes
$OVS_BINDIR or $OVS_SBINDIR variables.
The solution is to prepend those directories if they are not
already in PATH.
Reported-by: Mark Gray <mark.d.gray@redhat.com> Signed-off-by: Flavio Leitner <fbl@sysclose.org> Acked-by: Mark Gray <mark.d.gray@redhat.com> Acked-by: Paolo Valerio <pvalerio@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Mao YingMing [Tue, 12 Jan 2021 05:22:57 +0000 (13:22 +0800)]
dpctl: Fix dpctl process command parameter error.
Fix the following error:
$ ovs-appctl dpctl/dump-conntrack -m -s system@ovs-system zone=0
"dpctl/dump-conntrack" command takes at most 2 arguments
ovs-appctl: ovs-vswitchd: server returned an error
$ ovs-appctl dpctl/dump-flows -m --names system@ovs-system filter=in_port\(2\) type=ovs
"dpctl/dump-flows" command takes at most 4 arguments
ovs-appctl: ovs-vswitchd: server returned an error
Signed-off-by: Mao YingMing <maoyingming@baidu.com> Acked-by: Mark Gray <mark.d.gray@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ben Pfaff [Sat, 21 Nov 2020 19:47:01 +0000 (11:47 -0800)]
ovsdb-idl: Break into two layers.
This change breaks the IDL into two layers: the IDL proper, whose
interface to its client is unchanged, and a low-level library called
the OVSDB "client synchronization" (CS) library. There are two
reasons for this change. First, the IDL is big and complicated and
I think that this change factors out some of that complication into
a simpler lower layer. Second, the OVN northd implementation based
on DDlog can benefit from the client synchronization library even
though it would actually be made increasingly complicated by the IDL.
Ben Pfaff [Fri, 18 Dec 2020 00:01:48 +0000 (16:01 -0800)]
test-ovsdb: Log steps in idl test.
Until now, "test-ovsdb idl" has printed the steps that it goes through
to stdout. This commit also makes it log the same information. This
makes it easier to match up the steps with the rest of the log (in
particular with the jsonrpc logging).
Signed-off-by: Ben Pfaff <blp@ovn.org> Acked-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Fri, 18 Dec 2020 13:34:55 +0000 (14:34 +0100)]
odp-util: Fix abort while formatting nsh actions.
OVS should not exit if it cannot format NSH actions for the user.
It should just report the error like the other formatting functions do.
Credit to OSS-Fuzz.
Reported-at: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21509 Fixes: 1fc11c5948cf ("Generic encap and decap support for NSH") Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Acked-by: Mark Gray <mark.d.gray@redhat.com>
Greg Rose [Tue, 12 Jan 2021 19:15:30 +0000 (11:15 -0800)]
acinclude: Warn when --with-linux parameter is supplied.
Building the Linux kernel module from the Open vSwitch source tree
is deprecated. Emit a warning when the '--with-linux' configure
parameter is supplied to remind the user.
Ilya Maximets [Fri, 11 Dec 2020 20:54:47 +0000 (21:54 +0100)]
ovsdb: Use column diffs for ovsdb and raft log entries.
Currently, ovsdb-server stores complete value for the column in a database
file and in a raft log in case this column changed. This means that
transaction that adds, for example, one new acl to a port group creates
a log entry with all UUIDs of all existing acls + one new. Same for
ports in logical switches and routers and more other columns with sets
in Northbound DB.
There could be thousands of acls in one port group or thousands of ports
in a single logical switch. And the typical use case is to add one new
if we're starting a new service/VM/container or adding one new node in a
kubernetes or OpenStack cluster. This generates huge amount of traffic
within ovsdb raft cluster, grows overall memory consumption and hurts
performance since all these UUIDs are parsed and formatted to/from json
several times and stored on disks. And more values we have in a set -
more space a single log entry will occupy and more time it will take to
process by ovsdb-server cluster members.
Simple test:
1. Start OVN sandbox with clustered DBs:
# make sandbox SANDBOXFLAGS='--nbdb-model=clustered --sbdb-model=clustered'
2. Run a script that creates one port group and adds 4000 acls into it:
# cat ../memory-test.sh
pg_name=my_port_group
export OVN_NB_DAEMON=$(ovn-nbctl --pidfile --detach --log-file -vsocket_util:off)
ovn-nbctl pg-add $pg_name
for i in $(seq 1 4000); do
echo "Iteration: $i"
ovn-nbctl --log acl-add $pg_name from-lport $i udp drop
done
ovn-nbctl acl-del $pg_name
ovn-nbctl pg-del $pg_name
ovs-appctl -t $(pwd)/sandbox/nb1 memory/show
ovn-appctl -t ovn-nbctl exit
---
4. Check the current memory consumption of ovsdb-server processes and
space occupied by database files:
# ls sandbox/[ns]b*.db -alh
# ps -eo vsz,rss,comm,cmd | egrep '=[ns]b[123].pid'
Test results with current ovsdb log format:
On-disk Nb DB size : ~369 MB
RSS of Nb ovsdb-servers: ~2.7 GB
Time to finish the test: ~2m
In order to mitigate memory consumption issues and reduce computational
load on ovsdb-servers let's store diff between old and new values
instead. This will make size of each log entry that adds single acl to
port group (or port to logical switch or anything else like that) very
small and independent from the number of already existing acls (ports,
etc.).
Added a new marker '_is_diff' into a file transaction to specify that
this transaction contains diffs instead of replacements for the existing
data.
One side effect is that this change will actually increase the size of
file transaction that removes more than a half of entries from the set,
because diff will be larger than the resulted new value. However, such
operations are rare.
Test results with change applied:
On-disk Nb DB size : ~2.7 MB ---> reduced by 99%
RSS of Nb ovsdb-servers: ~580 MB ---> reduced by 78%
Time to finish the test: ~1m27s ---> reduced by 27%
After this change new ovsdb-server is still able to read old databases,
but old ovsdb-server will not be able to read new ones.
Since new servers could join ovsdb cluster dynamically it's hard to
implement any runtime mechanism to handle cases where different
versions of ovsdb-server joins the cluster. However we still need to
handle cluster upgrades. For this case added special command line
argument to disable new functionality. Documentation updated with the
recommended way to upgrade the ovsdb cluster.
Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Eli Britstein [Mon, 28 Dec 2020 10:19:03 +0000 (10:19 +0000)]
netdev-offload-dpdk: Implement flow flush.
Remove all the rules for the specified netdev.
Signed-off-by: Eli Britstein <elibr@nvidia.com> Reviewed-by: Gaetan Rivet <gaetanr@nvidia.com> Acked-by: Emma Finn <emma.finn@intel.com> Tested-by: Emma Finn <emma.finn@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Eli Britstein [Mon, 28 Dec 2020 10:19:02 +0000 (10:19 +0000)]
netdev-offload-dpdk: Refactor disassociate and flow destroy.
Refactor disassociation to be removed from flow destroy, and to use
already found object instead of re-searching it.
Signed-off-by: Eli Britstein <elibr@nvidia.com> Reviewed-by: Gaetan Rivet <gaetanr@nvidia.com> Acked-by: Emma Finn <emma.finn@intel.com> Tested-by: Emma Finn <emma.finn@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Eli Britstein [Mon, 28 Dec 2020 10:19:01 +0000 (10:19 +0000)]
netdev-offload-dpdk: Keep netdev in offload object.
Keep the netdev of the offload rule as a field in the offload object as
a pre-step towards support flushing of the offload rules.
Signed-off-by: Eli Britstein <elibr@nvidia.com> Reviewed-by: Gaetan Rivet <gaetanr@nvidia.com> Acked-by: Emma Finn <emma.finn@intel.com> Tested-by: Emma Finn <emma.finn@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Eli Britstein [Mon, 28 Dec 2020 10:19:00 +0000 (10:19 +0000)]
dpif-netdev: Flush offload rules upon port deletion.
When a port is deleted, flow deletion requests are posted, and the netdev
is removed from offload netdevs map. Following flow deletion handling may
be done after the netdev has already been removed from the offload
netdevs map, so the HW rule is not removed and the data object is not
freed (memory leak). Flush offload rules upon port deletion, and disable
pending handling of offloads to fix it.
Signed-off-by: Eli Britstein <elibr@nvidia.com> Reviewed-by: Gaetan Rivet <gaetanr@nvidia.com> Acked-by: Emma Finn <emma.finn@intel.com> Tested-by: Emma Finn <emma.finn@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
dpif-netdev: Add parameters to configure PMD auto load balance.
Two important parts of how PMD auto load balance operates are how
loaded a core needs to be and how much improvement is estimated
before a PMD auto load balance can trigger.
Previously they were hardcoded to 95% loaded and 25% variance
improvement.
These default values may not be suitable for all use cases and
we may want to use a more (or less) aggressive rebalance, either
on the pmd load threshold or on the minimum variance improvement
threshold.
The defaults are not changed, but "pmd-auto-lb-load-threshold" and
"pmd-auto-lb-improvement-threshold" parameters are added to override
the defaults.
$ ovs-vsctl set open_vswitch . other_config:pmd-auto-lb-load-threshold="70"
$ ovs-vsctl set open_vswitch . other_config:pmd-auto-lb-improvement-threshold="20"
Signed-off-by: Christophe Fontaine <cfontain@redhat.com> Co-Authored-by: Kevin Traynor <ktraynor@redhat.com> Signed-off-by: Kevin Traynor <ktraynor@redhat.com> Acked-by: David Marchand <david.marchand@redhat.com> Acked-by: Ian Stokes <ian.stokes@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Kevin Traynor [Mon, 21 Dec 2020 13:10:30 +0000 (13:10 +0000)]
dpif-netdev: Add log for PMD auto load balance interval parameter.
Previously if the parameter for the PMD auto load balance minimum
interval was changed at runtime, it was not logged unless the
PMD auto load balance feature was also changed to enabled.
Log the parameter anytime it changes, and use minutes when it is
logged as that is the user input format.
Fixes: 5bf84282482a ("Adding support for PMD auto load balancing") Signed-off-by: Kevin Traynor <ktraynor@redhat.com> Acked-by: David Marchand <david.marchand@redhat.com> Acked-by: Ian Stokes <ian.stokes@intel.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Fri, 8 Jan 2021 11:46:56 +0000 (12:46 +0100)]
acinclude: Strip out -mno-avx512f provided by DPDK.
DPDK forces '-mno-avx512f' flag for the application if the toolchain
used to build DPDK had broken AVX512 support. But OVS could be built
with a completely different or fixed toolchain with correct avx512
support. In this case OVS will detect that toolchain is good and will
try to build AVX512-optimized classifier. However, '-mno-avx512f'
flag will be passed from the DPDK side breaking the build:
In file included from /gcc/x86_64-linux-gnu/8/include/immintrin.h:55,
from /gcc/x86_64-linux-gnu/8/include/x86intrin.h:48,
from /dpdk/../x86_64-linux-gnu/dpdk/rte_vect.h:28,
from /dpdk/../x86_64-linux-gnu/dpdk/rte_memcpy.h:17,
from /dpdk/rte_mempool.h:51,
from /dpdk/rte_mbuf.h:38,
from ../lib/dp-packet.h:25,
from ../lib/dpif.h:380,
from ../lib/dpif-netdev.h:23,
from ../lib/dpif-netdev-lookup-avx512-gather.c:22:
/usr/lib/gcc/x86_64-linux-gnu/8/include/avx512bwintrin.h:413:1: error:
inlining failed in call to always_inline '_mm512_sad_epu8':
target specific option mismatch
_mm512_sad_epu8 (__m512i __A, __m512i __B)
Fix that by stripping out `-mno-avx512f` as we already do for '-march'.
This will allow the OVS to decide if the AVX512 can be used.
Reordering of CFLAGS (i.e. adding DPDK flags before OVS ones) is not an
option since autotools might reorder them back later and it's very
unpredictable.
Reported-at: https://github.com/openvswitch/ovs-issues/issues/201 Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Acked-by: Harry van Haaren <harry.van.haaren@intel.com> Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Emma Finn [Mon, 7 Dec 2020 16:27:15 +0000 (16:27 +0000)]
Revert "netdev-offload-dpdk: Fix for broken ethernet matching HWOL for XL710NIC."
Removing temporary patch - 023f257 (netdev-offload-dpdk: Fix for broken
ethernet matching HWOL for XL710NIC).
Ethernet pattern is now being set correctly withtin the i40e PMD.
Signed-off-by: Emma Finn <emma.finn@intel.com> Signed-off-by: Ian Stokes <ian.stokes@intel.com>
A packet that contains multiple instances of certain TLVs will cause
lldpd to continually allocate memory and leak the old memory. As an
example, multiple instances of system name TLV will cause old values
to be dropped by the decoding routine.
Flavio Leitner [Mon, 28 Dec 2020 22:02:55 +0000 (19:02 -0300)]
Documentation: Simplify the website main page.
The initial website page is difficult to read because of
the large amount of links from different parts of the whole
documentation. Most of all those links come from their
index page referenced in the section 'Contents' on the side.
Another issue is that because the page is static, new links
might not get included.
This patch simplifies the main page by highlighting the project
level documentation. The static part is reduced to the main
level index pages.
All the links are available by clicking on 'Full Table of
Contents' at the end of Documentation section.
Signed-off-by: Flavio Leitner <fbl@sysclose.org> Signed-off-by: Ben Pfaff <blp@ovn.org>
Ben Pfaff [Tue, 22 Dec 2020 00:21:00 +0000 (16:21 -0800)]
reconnect: Add Python implementation of received_attempt(), and test.
This follows up on commit 4241d652e465 ("jsonrpc: Avoid disconnecting
prematurely due to long poll intervals."), which implemented the same
thing in C.
Mark Gray [Tue, 5 Jan 2021 22:53:40 +0000 (17:53 -0500)]
ovs-monitor-ipsec: Allow exit of ipsec daemon maintaining state.
When 'ovs-monitor-ipsec' exits, it clears all persistent state (i.e.
active ipsec connections, /etc/ipsec.conf, certs/keys). In some
use-cases, we may want to exit and maintain state so that ipsec
connectivity is maintained. One example of this is during an
upgrade. This will require the caller to clear this persistent
state when appropriate (e.g. before 'ovs-monitor-ipsec') is restarted.
Mark Gray [Mon, 4 Jan 2021 08:45:18 +0000 (03:45 -0500)]
ovs-monitor-ipsec: Fix active connection regex.
Connections are added to IPsec using a connection name
that is determined from the OVS port name and the tunnel
type.
GRE connections take the form:
<iface>-<ver>
Other connections take the form:
<iface>-in-<ver>
<iface>-out-<ver>
The regex '|' operator parses strings left to right looking
for the first match that it can find. '.*' is also greedy. This
causes incorrect interface names to be parsed from active
connections as other tunnel types are parsed as type
GRE. This gives unexpected "is outdated" warnings and the
connection is torn down.
For example,
'ovn-424242-in-1' will produce an incorrect interface name of
'ovn-424242-in' instead of 'ovn-424242'.
There are a number of ways this could be resolved including
a cleverer regular expression, or re.findall(). However, this
approach was taken as it simplifies the code easing maintainability.
Fixes: 22c5eafb6efa ("ipsec: reintroduce IPsec support for tunneling")
Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1908789 Signed-off-by: Mark Gray <mark.d.gray@redhat.com> Acked-by: Eelco Chaudron <echaudro@redhat.com> Acked-by: Flavio Leitner <fbl@sysclose.org> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Mark Gray [Thu, 24 Dec 2020 12:57:01 +0000 (07:57 -0500)]
ovs-monitor-ipsec: Add support for tunnel 'local_ip'.
In the libreswan case, 'ovs-monitor-ipsec' sets
'left' to '%defaultroute' which will use the local address
of the default route interface as the source IP address. In
multihomed environments, this may not be correct if the user
wants to specify what the source IP address is. In OVS, this
can be set for tunnel ports using the 'local_ip' option. This
patch also uses that option to populate the 'ipsec.conf'
configuration. If the 'local_ip' option is not present, it
will default to the previous behaviour of using '%defaultroute'
Eelco Chaudron [Tue, 13 Oct 2020 08:36:48 +0000 (10:36 +0200)]
dpctl: Add add/mod/del-flows command.
When you would like to add, modify, or delete a lot of flows in the
datapath, for example when you want to measure performance, adding
one flow at the time won't scale. This as it takes a decent amount
of time to set up the datapath connection.
This new command is in-line with the same command available in
ovs-ofctl which allows the same thing, with the only difference that
we do not verify all lines before we start execution. This allows for
a continuous add/delete stream. For example with a command like this:
python3 -c 'while True:
for i in range(0, 1000):
print("add in_port(0),eth(),eth_type(0x800),ipv4(src=100.1.{}.{}) 1".format(int(i / 256), i % 256))
for i in range(0, 1000):
print("delete in_port(0),eth(),eth_type(0x800),ipv4(src=100.1.{}.{})".format(int(i / 256), i % 256))' \
| sudo utilities/ovs-dpctl add-flows -
Daniel Alvarez [Mon, 25 May 2020 15:28:21 +0000 (17:28 +0200)]
ovs-ctl: Don't overwrite external-id hostname.
ovs-ctl started to add the hostname as external-id [0] at some point.
However, this can be problematic as if it's already set by an external
entity it will get overwritten. In RHEL systems, systemd will invoke
ovs-ctl to start OVS and that will overwrite it to the hostname of the
machine.
For OVN this can have a big impact because if, for whatever reason the
hostname changes and the host gets restarted, ovn-controller won't
claim the ports back leaving the workloads unaccessible.
Also, it makes sense to not overwrite it as 1) it's an external_id,
so it will actually let external entities to configure it (unlike now),
and 2) it's optional. In the case that some systems were relying on
ovs-ctl to set the external-id for the first time (e.g onboarding
of a new hypervisor), this patch is not changing such behavior.
Martin Varghese [Thu, 17 Dec 2020 07:18:41 +0000 (12:48 +0530)]
tunnel: Bareudp Tunnel Support.
There are various L3 encapsulation standards using UDP being discussed to
leverage the UDP based load balancing capability of different networks.
MPLSoUDP (__ https://tools.ietf.org/html/rfc7510) is one among them.
The Bareudp tunnel provides a generic L3 encapsulation support for
tunnelling different L3 protocols like MPLS, IP, NSH etc. inside a UDP
tunnel.
An example to create bareudp device to tunnel MPLS traffic is
given
The bareudp device supports special handling for MPLS & IP as
they can have multiple ethertypes. MPLS procotcol can have ethertypes
ETH_P_MPLS_UC (unicast) & ETH_P_MPLS_MC (multicast). IP protocol can have
ethertypes ETH_P_IP (v4) & ETH_P_IPV6 (v6).
The bareudp device to tunnel L3 traffic with multiple ethertypes
(MPLS & IP) can be created by passing the L3 protocol name as string in
the field payload_type. An example to create bareudp device to tunnel
MPLS unicast & multicast traffic is given below.::
Jianbo Liu [Thu, 17 Dec 2020 02:47:32 +0000 (02:47 +0000)]
dpif-netlink: Fix issues of the offloaded flows counter.
The n_offloaded_flows counter is saved in dpif, and this is the first
one when ofproto is created. When flow operation is done by ovs-appctl
commands, such as, dpctl/add-flow, a new dpif is opened, and the
n_offloaded_flows in it can't be used. So, instead of using counter,
the number of offloaded flows is queried from each netdev, then sum
them up. To achieve this, a new API is added in netdev_flow_api to get
how many flows assigned to a netdev.
In order to get better performance, this number is calculated directly
from tc_to_ufid hmap for netdev-offload-tc, because flow dumping by tc
takes much time if there are many flows offloaded.
Fixes: af0618470507 ("dpif-netlink: Count the number of offloaded rules") Signed-off-by: Jianbo Liu <jianbol@nvidia.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Update tutorial for newer versions of Faucet and Open vSwitch.
Newer versions of Faucet use a dynamic OpenFlow pipeline based on what
features are enabled in the configuration file. Update log output, flow
table dumps and explanations to be consistent with newer Faucet versions.
Remove mentions of bugs that we have since fixed in Faucet since the
tutorial was originally written.
Adds documentation on changes to Open vSwitch commands to recommend
using a version that is compatible with the features of the tutorial.
Ilya Maximets [Thu, 17 Dec 2020 17:22:12 +0000 (18:22 +0100)]
ovsdb-tool: Fix datum leak in the show-log command.
Fixes: 4e92542cefb7 ("ovsdb-tool: Make "show-log" convert raw JSON to easier-to-read syntax.") Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Acked-by: Dumitru Ceara <dceara@redhat.com>
Lorenzo Bianconi [Wed, 25 Nov 2020 10:12:59 +0000 (11:12 +0100)]
raft: Add some debugging information to cluster/status command.
Introduce the following info useful for cluster debugging to
cluster/status command:
- time elapsed from last start/complete election
- election trigger (e.g. timeout)
- number of disconnections
- time elapsed from last raft messaged received
Acked-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Eelco Chaudron [Mon, 12 Oct 2020 11:38:38 +0000 (07:38 -0400)]
conntrack: add generic IP protocol support
Currently, userspace conntrack only tracks TCP, UDP, and ICMP, and all
other IP protocols are discarded, and the +inv state is returned. This
is not in line with the kernel conntrack. Where if no L4 information can
be extracted it's treated as generic L3. The change below mimics the
behavior of the kernel.
ofproto-dpif-xlate: Stop forwarding MLD reports to group ports.
According with rfc4541 section 2.1.1, a snooping switch
should forward membership reports only to ports with
routers attached.The current code violates the RFC
forwarding membership reports to group ports as well.
The same issue doesn't exist with IPv4.
Ben Pfaff [Thu, 19 Nov 2020 00:39:46 +0000 (16:39 -0800)]
jsonrpc: Avoid disconnecting prematurely due to long poll intervals.
Open vSwitch has a few different jsonrpc-based protocols that depend on
jsonrpc_session to make sure that the connection is up and working.
In turn, jsonrpc_session uses the "reconnect" state machine to send
probes if nothing is received. This works fine in normal circumstances.
In unusual circumstances, though, it can happen that the program is
busy and doesn't even try to receive anything for a long time. Then the
timer can time out without a good reason; if it had tried to receive
something, it would have.
There's a solution that the clients of jsonrpc_session could adopt.
Instead of first calling jsonrpc_session_run(), which is what calls into
"reconnect" to deal with timing out, and then calling into
jsonrpc_session_recv(), which is what tries to receive something, they
could use the opposite order. That would make sure that the timeout
was always based on a recent attempt to receive something. Great.
The actual code in OVS that uses jsonrpc_session, though, tends to use
the opposite order, and there are enough users and this is a subtle
enough issue that it could get flipped back around even if we fixed it
now. So this commit takes a different approach. Instead of fixing
this in the users of jsonrpc_session, we fix it in the users of
reconnect: make them tell when they've tried to receive something (or
disable this particular feature).
This commit fixes the problem that way. It's kind of hard to reproduce
but I'm pretty sure that I've seen it a number of times in testing.
Signed-off-by: Ben Pfaff <blp@ovn.org> Acked-by: Ilya Maximets <i.maximets@ovn.org>
For credit all authors of the original commits to 'dpdk-latest' with the
above changes have been added as co-authors for this commit.
Signed-off-by: David Marchand <david.marchand@redhat.com> Co-authored-by: David Marchand <david.marchand@redhat.com> Signed-off-by: Sunil Pai G <sunil.pai.g@intel.com> Co-authored-by: Sunil Pai G <sunil.pai.g@intel.com> Signed-off-by: Eli Britstein <elibr@nvidia.com> Co-authored-by: Eli Britstein <elibr@nvidia.com> Tested-by: Harry van Haaren <harry.van.haaren@intel.com> Tested-by: Govindharajan, Hariprasad <hariprasad.govindharajan@intel.com> Tested-by: Tonghao Zhang <xiangxia.m.yue@gmail.com> Acked-by: Ilya Maximets <i.maximets@ovn.org> Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Dumitru Ceara [Fri, 4 Dec 2020 14:54:41 +0000 (15:54 +0100)]
ovsdb-idl: Fix expected condition seqno when changes are pending.
Commit 17f22fe46142 tried to address this but only covered some of the
cases.
The correct way to report the expected seqno is to take into account if
there already is a condition change that was requested to the server but
not acked yet. In that case, the new condition change request will be
sent only after the already requested one is acked. That is, expected
condition seqno when conditions are up to date is db->cond_seqno + 2 in
this case.
Fixes: 17f22fe46142 ("ovsdb-idl: Return correct seqno from ovsdb_idl_db_set_condition().") Suggested-by: Ilya Maximets <i.maximets@ovn.org> Signed-off-by: Dumitru Ceara <dceara@redhat.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Thu, 26 Nov 2020 00:43:57 +0000 (01:43 +0100)]
ovsdb-cluster.at: Fix infinite loop in torture tests.
For some reason, while running cluster torture tests in GitHub Actions
workflow, failure of 'echo' command doesn't fail the loop and subshell
never exits, but keeps infinitely printing errors after breaking from
the loop on the right side of the pipeline:
testsuite: line 8591: echo: write error: Broken pipe
Presumably, that is caused by some shell configuration option, but
I have no idea which one and I'm not able to reproduce locally with
shell configuration options provided in GitHub documentation.
Let's just add an explicit 'exit' on 'echo' failure. This will
guarantee exit from the loop and the subshell regardless of
configuration.
Fixes: 0f03ae3754ec ("ovsdb: Improve timing in cluster torture test.") Acked-by: Simon Horman <simon.horman@netronome.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
wenxu [Tue, 24 Nov 2020 03:01:09 +0000 (11:01 +0800)]
lib/tc: fix parse act pedit for tos rewrite
Check overlap between current pedit key, which is always 4 bytes
(range [off, off + 3]), and a map entry in flower_pedit_map
sf = ROUND_DOWN(mf, 4) (range [sf|mf, (mf + sz - 1)|ef]).
So for the tos the rewite the off + 3(3) is greater than mf,
and should less than ef(4) but not mf+sz(2).
Signed-off-by: wenxu <wenxu@ucloud.cn> Signed-off-by: Simon Horman <simon.horman@netronome.com>
Dumitru Ceara [Mon, 30 Nov 2020 16:41:41 +0000 (17:41 +0100)]
ovsdb-idl: Fix use-after-free when deleting orphaned rows.
It's possible that the IDL client processes multiple jsonrpc updates
in a single ovsdb_idl_run().
Considering the following updates processed in a single IDL run:
1. Update row R1 from table A while R1 is also referenced by row R2 from
table B:
- this adds R1 to table A's track_list.
2. Delete row R1 from table A while R1 is also referenced by row R2 from
table B:
- because row R2 still refers to row R1, this will create an orphan
R1.
- at this point R1 is still in table A's hmap.
When the IDL client calls ovsdb_idl_track_clear() after it has finished
processing the tracked changes, row R1 gets freed leaving a dangling
pointer in table A's hmap.
To fix this we don't free rows in ovsdb_idl_track_clear() if they are
orphan and still referenced by other rows, i.e., the row's 'dst_arcs'
list is not empty. Later, when all arc sources (e.g., R2) are
deleted, the orphan R1 will be cleaned up as well.
The only exception is when the whole contents of the IDL are flushed,
in ovsdb_idl_db_clear(), in which case it's safe to free all rows.
Reported-by: Ilya Maximets <i.maximets@ovn.org> Fixes: 932104f483ef ("ovsdb-idl: Add support for change tracking.") Signed-off-by: Dumitru Ceara <dceara@redhat.com> Acked-by: Han Zhou <hzhou@ovn.org> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Dumitru Ceara [Mon, 30 Nov 2020 16:41:29 +0000 (17:41 +0100)]
ovsdb-idl: Fix memleak when deleting orphan rows.
Pure IDL orphan rows, i.e., for which no "insert" operation was seen,
which are part of tables with change tracking enabled should also be
freed when the table track_list is flushed.
Reported-by: Ilya Maximets <i.maximets@ovn.org> Fixes: 72aeb243a52a ("ovsdb-idl: Tracking - preserve data for deleted rows.") Signed-off-by: Dumitru Ceara <dceara@redhat.com> Acked-by: Han Zhou <hzhou@ovn.org> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Dumitru Ceara [Mon, 30 Nov 2020 16:41:14 +0000 (17:41 +0100)]
ovsdb-idl: Fix memleak when reinserting tracked orphan rows.
Considering the following updates processed by an IDL client:
1. Delete row R1 from table A while R1 is also referenced by row R2 from
table B:
- because row R2 still refers to row R1, this will create an orphan
R1 but also sets row->tracked_old_datum to report to the IDL client
that the row has been deleted.
2. Insert row R1 to table A.
- because orphan R1 already existed in the IDL, it will be reused.
- R1 still has row->tracked_old_datum set (and may also be on the
table->track_list).
3. Delete row R2 from table B and row R1 from table A.
- row->tracked_old_datum is set again but the previous
tracked_old_datum was never freed.
IDL clients use the deleted old_datum values so when multiple delete
operations are received for a row, always track the first one as that
will match the contents of the row the IDL client knew about.
Running the newly added test case with valgrind, without the fix,
produces the following report:
==23113== 327 (240 direct, 87 indirect) bytes in 1 blocks are definitely lost in loss record 43 of 43
==23113== at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==23113== by 0x476761: xmalloc (util.c:138)
==23113== by 0x45D8B3: ovsdb_idl_insert_row (ovsdb-idl.c:3431)
==23113== by 0x45B7F9: ovsdb_idl_process_update2 (ovsdb-idl.c:2670)
==23113== by 0x45AFCF: ovsdb_idl_db_parse_update__ (ovsdb-idl.c:2479)
==23113== by 0x45B262: ovsdb_idl_db_parse_update (ovsdb-idl.c:2542)
==23113== by 0x45ABBE: ovsdb_idl_db_parse_update_rpc (ovsdb-idl.c:2358)
==23113== by 0x4576DD: ovsdb_idl_process_msg (ovsdb-idl.c:865)
==23113== by 0x457973: ovsdb_idl_run (ovsdb-idl.c:944)
==23113== by 0x40B7B9: do_idl (test-ovsdb.c:2523)
==23113== by 0x44425D: ovs_cmdl_run_command__ (command-line.c:247)
==23113== by 0x44430E: ovs_cmdl_run_command (command-line.c:278)
==23113== by 0x404BA6: main (test-ovsdb.c:76)
Fixes: 72aeb243a52a ("ovsdb-idl: Tracking - preserve data for deleted rows.") Signed-off-by: Dumitru Ceara <dceara@redhat.com> Acked-by: Han Zhou <hzhou@ovn.org> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
rcu: Add support for consolidated-RCU reader checking
This commit adds RCU-reader checks to list_for_each_entry_rcu() and
hlist_for_each_entry_rcu(). These checks are optional, and are indicated
by a lockdep expression passed to a new optional argument to these two
macros. If this optional lockdep expression is omitted, these two macros
act as before, checking for an RCU read-side critical section.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
[ paulmck: Update to eliminate return within macro and update comment. ] Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Backport portion of upstream commit for hlist_for_each_entry_rcu() macro
so that it can be used in following bug fix.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org> Signed-off-by: Greg Rose <gvrose8192@gmail.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Mon, 23 Nov 2020 08:37:47 +0000 (09:37 +0100)]
ovsdb-idl: Fix iteration over tracked rows with no actual data.
When idl removes orphan rows, those rows are inserted into the
'track_list'. This allows iterators such as *_FOR_EACH_TRACKED () to
return orphan rows that never had any data to the IDL user. In this
case, it is difficult for the user to understand whether it is a row
with no data (there was no "insert" / "modify" for this row) or it is
a row with zero data (columns were cleared by DB transaction).
The main problem with this condition is that rows without data will
have NULL pointers instead of references that should be there according
to the database schema. For example, ovn-controller might crash:
ERROR: AddressSanitizer: SEGV on unknown address 0x000000000100
(pc 0x00000055e9b2 bp 0x7ffef6180880 sp 0x7ffef6180860 T0)
The signal is caused by a READ memory access.
Hint: address points to the zero page.
#0 0x55e9b1 in handle_deleted_lport /controller/binding.c
#1 0x55e903 in handle_deleted_vif_lport /controller/binding.c:2072:5
#2 0x55e059 in binding_handle_port_binding_changes /controller/binding.c:2155:23
#3 0x5a6395 in runtime_data_sb_port_binding_handler /controller/ovn-controller.c:1454:10
#4 0x5e15b3 in engine_compute /lib/inc-proc-eng.c:306:18
#5 0x5e0faf in engine_run_node /lib/inc-proc-eng.c:352:14
#6 0x5e0e04 in engine_run /lib/inc-proc-eng.c:377:9
#7 0x5a03de in main /controller/ovn-controller.c
#8 0x7f4fd9c991a2 in __libc_start_main (/lib64/libc.so.6+0x271a2)
#9 0x483f0d in _start (/controller/ovn-controller+0x483f0d)
It doesn't make much sense to return non-real rows to the user, so it's
best to exclude them from iteration.
Test included. Without the fix, provided test will print empty orphan
rows that was never received by idl as tracked changes.
Fixes: 932104f483ef ("ovsdb-idl: Add support for change tracking.") Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Acked-by: Dumitru Ceara <dceara@redhat.com>
Ilya Maximets [Wed, 18 Nov 2020 21:18:58 +0000 (22:18 +0100)]
checkpatch: Add check for a whitespace after cast.
Coding style says: "Put a space between the ``()`` used in a cast and
the expression whose type is cast: ``(void *) 0``.".
This style rule is frequently overlooked. Let's check for it.
Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Acked-by: Ian Stokes <ian.stokes@intel.com>
Ilya Maximets [Mon, 23 Nov 2020 22:34:28 +0000 (23:34 +0100)]
github: Add GitHub Actions workflow.
This is an initial version of GitHub Actions support. It mostly
mimics our current Travis CI build matrix with slight differences.
The main issue is that we don't have ARM support here.
Minor difference that we can not install 32-bit versions of libunwind
and libunbound since those are not avaialble in repository.
Higher concurrency level allows to finish all tests less than in 20
minutes. Which is 3 times faster than in Travis.
.travis folder renamed to .ci to highlight that it used not only for
Travis CI. Travis CI support will be reduced to only test ARM builds
soon and will be completely removed when travis-ci.org will be turned
into read-only mode.
What happened to Travis CI:
https://mail.openvswitch.org/pipermail/ovs-dev/2020-November/377773.html
Acked-by: Simon Horman <simon.horman@netronome.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Ilya Maximets [Wed, 25 Nov 2020 12:35:48 +0000 (13:35 +0100)]
ci: Don't use 'native' machine for DPDK cache.
It's possible that actual HW where CI is running is slightly different
between jobs. That makes all unit tests to fail with cached DPDK
builds due to 'Illegal instruction' crashes. Changing machine
type to 'default' to generate binaries as generic as possible and avoid
this kind of issues.
Changing the name of a cache version file, so we will not use old
'native' builds that are currently in cache.
Fixes: 7654a3ed0b38 ("travis: Cache DPDK build.") Acked-by: Kevin Traynor <ktraynor@redhat.com> Acked-by: Simon Horman <simon.horman@netronome.com> Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
projects / mirror_ovs.git / log