Ben Pfaff [Wed, 26 May 2010 22:53:57 +0000 (15:53 -0700)]
debian: Attempt to keep debian/changelog up-to-date.
Invariably we forget to update the version number in debian/changelog as
we change OVS's own version number. This is embarrassing.
This commit introduces two different times to automatically update the
debian/changelog version number: whenever boot.sh runs and whenever
"make dist" runs. In the latter case, only the version number in the
distributed tarball is updated, but that seems OK.
Reported by Joan Cirer <joan@ev0.net> most recently, and by others over
the last year or so too.
Ben Pfaff [Wed, 26 May 2010 20:37:57 +0000 (13:37 -0700)]
ofproto: Avoid null pointer dereference if in_band_create() fails.
in_band_create() can fail if something goes wrong with the network device
that represents the local port. In that case update_in_band_remotes()
should not call in_band_set_remotes(), but it did anyway. This commit fixes
it.
Ben Pfaff [Wed, 26 May 2010 22:32:34 +0000 (15:32 -0700)]
datapath: Make datapath-protocol.h portable to non-Linux systems.
datapath-protocol.h is not a very clean interface. I originally intended
it to be solely a Linux-kernel specific interface. Over time it became
a general-purpose interface to dpifs. This is not a good situation,
because clearly the header is still Linux-specific.
In the long run, the correct solution is to separate the generic and
Linux-specific bits. This is not that patch. Instead, this patch modifies
datapath-protocol.h enough that it can be used on non-Linux hosts. In
particular I tested that it works OK with FreeBSD 8.0.
Ben Pfaff [Wed, 26 May 2010 17:09:56 +0000 (10:09 -0700)]
datapath: Make gre.h safe to include by itself.
gre.h is Linux-specific, and it uses Linux-specific types, so it has to
#include <linux/types.h>. We probably got away with it until now because
it was always included after some other header that had already included
that one.
Ben Pfaff [Wed, 26 May 2010 22:24:13 +0000 (15:24 -0700)]
tests: Speed up classifier test.
Many of the classifier tests take time exponential in the number of fields,
because the existing compare_classifiers() iterates over 2^n_fields
possibilities. This is very slow.
This commit fixes the problem by only testing a fixed number of random
possibilities instead of all of them. This makes it much, much faster.
Ben Pfaff [Wed, 26 May 2010 16:53:08 +0000 (09:53 -0700)]
tests: Avoid nonportable \x escapes in printf(1) invocation.
The \x escape is not part of POSIX, but it is a common extension. The
dash shell's built-in "printf" implementation does not include this
extension, which caused the testsuite to be generated incorrectly if it
is used as the default shell (as it is on newer versions of Debian and
Ubuntu).
This commit fixes the problem by using standard octal escapes instead.
Ben Pfaff [Wed, 12 May 2010 17:56:45 +0000 (10:56 -0700)]
poll-loop: Change poll_timer_wait() parameter from "int" to "long long".
Every so often I get concerned because OVS does most of its time arithmetic
in "long long int" but poll_timer_wait() takes an "int", so there is
potential for truncating a large value to a small value or a positive value
to a negative value. That would cause excessive wakeups and possibly 100%
CPU usage.
This commit therefore changes poll_timer_wait()'s parameter type from "int"
to "long long int". The file-scope 'timeout' variable remains type "int"
because that is the type of poll()'s timeout argument.
Factoring poll_timer_wait() into two functions is not necessary here but it
comes in handy in the following patch.
Ben Pfaff [Tue, 25 May 2010 22:42:44 +0000 (15:42 -0700)]
datapath: Fix ODP_PORT_GROUP_GET implementation.
The final argument to do_get_port_group() is supposed to be a user pointer
to the number of ports, to be updated with put_user(), but it was actually
a kernel pointer, so "ovs-dpctl dump-groups" and anything else that used
this ioctl would always fail with -EFAULT. This commit fixes it.
Bug introduced in commit 44e05eca "datapath: Prepare to support 32-bit
compatibility ioctls" for normal ioctls and for compat ioctls at their
introduction in commit 3fbd517acf"datapath: Add 32-bit compatibility
ioctls."
Jesse Gross [Wed, 19 May 2010 21:12:27 +0000 (14:12 -0700)]
netdev-linux: Quiet down ingress policing.
If we attempt to remove ingress policing and receive "invalid
argument" it means that policing isn't compiled into the kernel.
If it isn't compiled in then accept that policing has been
successfully removed.
Jesse Gross [Wed, 19 May 2010 00:30:42 +0000 (17:30 -0700)]
gre: Check whether IPv6 is compiled into the kernel.
Add guards to check whether IPv6 is supported by the kernel instead
of causing the module to fail to load. If IPv6 is not supported
these packets can still be encapsulated but they will not receive any
special treatment such as path MTU discovery.
Jesse Gross [Fri, 14 May 2010 20:54:16 +0000 (13:54 -0700)]
patch: Remove veth driver.
Now that we have a new patch implementation, remove the veth driver
and its userspace components. Then rename 'patchnew' to 'patch'.
The new implementation is a drop-in replacement for the old one.
Jesse Gross [Fri, 14 May 2010 20:49:43 +0000 (13:49 -0700)]
netdev: Add 'patch' userspace implementation.
Add a netdev to talk to the 'patch' vport in the kenerl. Since
there is currently a 'patch' implementation using the veth driver,
this one is temporarily called 'patchnew'.
Justin Pettit [Mon, 17 May 2010 21:13:32 +0000 (14:13 -0700)]
datapath: Fix build with backported dev_get_stats().
An upcoming backport adds dev_get_stats() to a pre-2.6.29 Linux kernel, so we
have to check for the presence of an dev_get_stats() definition instead of for
the particular kernel version.
Wei Yongjun [Mon, 17 May 2010 07:15:44 +0000 (15:15 +0800)]
tests: fix compile warning of tests/test-ovsdb.c
This patch fixed the following compile warning:
tests/test-ovsdb.c: In function 'do_evaluate_conditions':
tests/test-ovsdb.c:744: warning: format '%2d' expects type 'int', but argument 2 has type 'size_t'
tests/test-ovsdb.c: In function 'do_execute_mutations':
tests/test-ovsdb.c:850: warning: format '%2d' expects type 'int', but argument 2 has type 'size_t'
tests/test-ovsdb.c: In function 'do_query':
tests/test-ovsdb.c:972: warning: format '%2d' expects type 'int', but argument 2 has type 'size_t'
tests/test-ovsdb.c: In function 'do_query_distinct':
tests/test-ovsdb.c:1105: warning: format '%2d' expects type 'int', but argument 2 has type 'size_t'
tests/test-ovsdb.c: In function 'parse_uuids':
tests/test-ovsdb.c:1600: warning: format '%d' expects type 'int', but argument 2 has type 'size_t'
Wei Yongjun [Mon, 17 May 2010 07:14:55 +0000 (15:14 +0800)]
ofproto: fix compile warning of ofproto/ofproto.c
This patch fixed the following compile warning:
ofproto/ofproto.c: In function 'handle_role_request':
ofproto/ofproto.c:3543: warning: format '%zu' expects type 'size_t', but argument 5 has type 'int'
ofproto/ofproto.c: In function 'handle_vendor':
ofproto/ofproto.c:3593: warning: format '%zu' expects type 'size_t', but argument 5 has type 'int'
ofproto/ofproto.c:3602: warning: format '%zu' expects type 'size_t', but argument 5 has type 'int'
Wei Yongjun [Mon, 17 May 2010 07:08:17 +0000 (15:08 +0800)]
ovsdb-idl: fix compile warning of lib/ovsdb-idl.c
This patch fixed the following compile warning:
lib/ovsdb-idl.c: In function 'ovsdb_idl_txn_process_inc_reply':
lib/ovsdb-idl.c:1524: warning: format '%u' expects type 'unsigned int', but argument 5 has type 'size_t'
lib/ovsdb-idl.c:1538: warning: format '%ld' expects type 'long int', but argument 5 has type 'long long int'
lib/ovsdb-idl.c:1550: warning: format '%u' expects type 'unsigned int', but argument 5 has type 'size_t'
lib/ovsdb-idl.c: In function 'ovsdb_idl_txn_process_insert_reply':
lib/ovsdb-idl.c:1579: warning: format '%u' expects type 'unsigned int', but argument 5 has type 'size_t'
Ben Pfaff [Fri, 14 May 2010 22:44:39 +0000 (15:44 -0700)]
datapath: Fix build with backported skb_dst().
An upcoming backport adds skb_dst() to a pre-2.6.31 Linux kernel, so we
have to check for the presence of an skb_dst() definition instead of for
the particular kernel version.
Jesse Gross [Wed, 12 May 2010 20:45:49 +0000 (13:45 -0700)]
datapath: Hold rcu_read_lock where we claim to.
Many of the vport operations require that either RTNL lock or
rcu_read_lock be held. However, operations from userspace often
hold a different lock so grab rcu_read_lock as well.
Jesse Gross [Wed, 12 May 2010 19:40:45 +0000 (12:40 -0700)]
datapath: Don't expect bottom-halves to be disabled.
We currently document that BHs need to be disabled when handling
received packets. However, this isn't actually generally the
case (usually preemption is disabled but not BHs). Only one place
actually relies on BHs being disabled so fix that and update the
documentation of our expectations.
Jesse Gross [Wed, 12 May 2010 18:40:58 +0000 (11:40 -0700)]
datapath: Disable bottom-halves where necessary.
Places that update per-cpu stats without locking need to have bottom
halves disabled. Otherwise we can be running in process context and
in the middle of an operation and be interrupted by a softirq.
Jesse Gross [Wed, 12 May 2010 18:26:55 +0000 (11:26 -0700)]
datapath: Use spin_lock_bh() consistently.
We are never called in hardirq context - only process or softirq.
Therefore it is not necessary to disable interrupts with
spin_lock_irqsave(), so use spin_lock_bh() everywhere.
Ben Pfaff [Thu, 13 May 2010 23:08:14 +0000 (16:08 -0700)]
stream-ssl: Flush OpenSSL error queue after calling SSL_shutdown().
The OpenSSL manpage for SSL_get_error() says this:
In addition to ssl and ret, SSL_get_error() inspects the current
thread's OpenSSL error queue. Thus, SSL_get_error() must be used in
the same thread that performed the TLS/SSL I/O operation, and no other
OpenSSL function calls should appear in between. The current thread's
error queue must be empty before the TLS/SSL I/O operation is
attempted, or SSL_get_error() will not work reliably.
We weren't taking this advice literally enough, which meant that this
would happen:
1. Call SSL_shutdown() on one connection.
2. Call SSL_read() on another connection, returning 0 bytes. (This is
normal. It just means that no more data has arrived yet.)
3. Call SSL_get_error() for that second connection to check whether
the 0-byte return value was a real error. (This should return
SSL_ERROR_WANT_READ to indicate that more data is needed.)
4. Actually get some other error indicating that the SSL_shutdown()
call returned an error.
This commit fixes the problem by flushing the OpenSSL error queue after
calling SSL_shutdown().
Without this commit, starting an ovsdb-server with two active SSL remotes,
running two ovsdb-clients listening for connections from the ovsdb-server
remotes, then killing one of the ovsdb-clients (with e.g. Control+C), will
cause ovsdb-server to drop the other ovsdb-client connnection the next time
that SSL_read() is called on it. With this commit, this scenario works
correctly (e.g. ovsdb-server keeps the remaining connection up).
Ben Pfaff [Thu, 13 May 2010 22:25:27 +0000 (15:25 -0700)]
datapath: Add 32-bit compatibility ioctls.
When a 32-bit userspace program runs on a 64-bit kernel, data structures
that contain members whose sizes or alignments change from 32- to 64-bit
must be translated when they are passed to ioctls. This commit adds such
support for openvswitch_mod.
We should really reconsider some parts of the Open vSwitch ioctl interface
to avoid needing as much translation as we do.
The advantages of the double-underscore variants of copy_to_user(),
copy_from_user(), get_user(), and put_user() are pretty marginal, at best,
in the places where we are using them, and it's not always obvious that we
are making the right calls to access_ok() beforehand. So switch to the
safe variants without double underscores.
Ben Pfaff [Thu, 13 May 2010 20:21:33 +0000 (13:21 -0700)]
datapath: Prepare to support 32-bit compatibility ioctls.
This commit prepares the core of datapath.c and vport.c to reduce the
amount of new code duplication when the following commit adds support for
32-bit compatibility ioctls. It breaks a number of functions apart into
pairs of functions: one that copies data to and from userspace and another
that does the real work.
This change is a pure refactoring that should not change behavior.
Ben Pfaff [Mon, 10 May 2010 20:53:26 +0000 (13:53 -0700)]
datapath: Avoid possibility of negative 'n_flows' in struct odp_flowvec.
do_flowvec_ioctl() was checking for too-big 'n_flows' but not negative
'n_flows'. We could add that check too, but 'n_flows' should never be
negative so it's better to just use an unsigned type.
Ben Pfaff [Wed, 12 May 2010 17:02:23 +0000 (10:02 -0700)]
daemon: Throttle max respawning rate.
If a monitored daemon dies quickly at startup, the system can waste a lot
of CPU time continually restarting it. This commit prevents a given
daemon from restarting more than once every 10 seconds.
Ben Pfaff [Tue, 11 May 2010 17:56:10 +0000 (10:56 -0700)]
daemon: Allow monitored daemon to dump core no more than once.
If the monitored daemon dumps core frequently, then this can quickly
exhaust the host's disk space. This commit limits core dumps to at most
one per monitored session (typically, once per boot).
Ben Pfaff [Thu, 13 May 2010 16:43:33 +0000 (09:43 -0700)]
ofproto: Prefer "master" and "other" connections for snooping over "slave".
This makes "ovs-ofctl snoop" and anything else that connects to the
switch's "snoop" socket typically more useful in the presence of multiple
controllers, since the "master" connection is the one with the most
interesting traffic.
Ben Pfaff [Fri, 7 May 2010 16:29:02 +0000 (09:29 -0700)]
ofproto-sflow: Maintain table of ports even when clearing configuration.
When ofproto_sflow_set_options() fails, it calls ofproto_sflow_clear() to
deconfigure the ofproto_sflow object. But ofproto_sflow_clear() deletes
all of the object's record of datapath ports. That means that the next
call to ofproto_sflow_set_options(), if it succeeds, will believe that the
datapath has no ports.
This commit fixes the problem by only clearing ofproto_sflow's record of
datapath ports when it is destroyed, not just when a configuration error
occurs.
Ben Pfaff [Tue, 11 May 2010 19:44:58 +0000 (12:44 -0700)]
ovs-ofctl: Add "snoop" command.
The "snoop" command does roughly the same thing as "monitor", but it is
easier to use in the common case where one just wants to look at the
OpenFlow controller connection for a bridge. Instead of, for example,
ovs-ofctl monitor unix:/var/run/openvswitch/br0.snoop
one merely types
ovs-ofctl snoop br0
Ben Pfaff [Thu, 6 May 2010 00:08:28 +0000 (17:08 -0700)]
ofproto: Don't send async messages on transient connections by default.
ofproto supports listening for "transient connections" from clients such
as ovs-ofctl. These OpenFlow connections are not supposed to receive
asynchronous messages by default, unless they ask for them by setting an
nonzero packet-in send length. This feature got broken some time back.
This commit fixes it.
Ben Pfaff [Wed, 5 May 2010 17:31:44 +0000 (10:31 -0700)]
Diagnose attempts to connect the wrong protocol to a network port.
Sometimes, when a user asks me to help debug a problem, it turns out that
an SSL connection was being made on a TCP port, or vice versa, or that an
OpenFlow connection was being made on a JSON-RPC port, or vice versa, and
so on. This commit adds log messages that diagnose this kind of problem,
e.g. "tcp:127.0.0.1:6633: received JSON-RPC data on OpenFlow channel".
Jesse Gross [Mon, 10 May 2010 23:16:33 +0000 (16:16 -0700)]
vport: Record and free packets on unattached ports.
We throw away packets that are received on vports not attached to
a datapath but we are actually leaking them. This records that an
error took place and frees the skb.
Jesse Gross [Tue, 11 May 2010 01:23:18 +0000 (18:23 -0700)]
flow: Zero tun_id if wildcarded.
Normally match fields are zeroed if they are wildcarded in
normalize_match(). However, tun_id isn't part of struct ofp_match
so do it when we convert to a flow instead.
Jesse Gross [Mon, 10 May 2010 20:34:41 +0000 (13:34 -0700)]
bridge: Add iface to hash table before calling iface_is_internal().
When creating an interface we need to check whether it is internal.
However, the function iface_is_internal() does a lookup on the
interface name but we haven't added it to the hash table yet. This
adds the interface to the table early on in iface_create.
Ben Pfaff [Fri, 7 May 2010 21:31:04 +0000 (14:31 -0700)]
Add header for access to potentially unaligned data.
I had been under the impression that "memcpy" was a valid way to copy
unaligned data into an aligned location for access. But testing on SPARC
has shown that GCC doesn't always honor that intention. It seems that, if
GCC can see that there is a pointer of a type that requires alignment to
a given object, then it will access it directly regardless of whether
memcpy() is used to copy it.
This commit adds a new header with functions to access unaligned data. I
managed to come up with two techniques, one GCC-specific, one generic, that
do avoid the misaligned access in my test case. The GCC-specific technique
is the same one used by the Linux kernel (although no code has been
literally copied). The other one seemed obvious but possibly slow
depending on the compiler's ability to optimize.
Ben Pfaff [Fri, 7 May 2010 17:44:01 +0000 (10:44 -0700)]
ovsdb-client: Serialize columns in predictable order on "monitor" command.
The "monitor" command goes to some trouble to write its output in a
predictable order, so that test programs can reliably compare it against
expectations. This commit fixes up one part that was missing, which is
that the columns were not being ordered predictably (it depended on
hash order, which differs between big-endian and little-endian systems).
It also updates the test suite to expect the new order.
Ben Pfaff [Fri, 7 May 2010 17:41:06 +0000 (10:41 -0700)]
tests: Fix bug in "ovsdb-tool compact" test.
This test examines the OVSDB database log and checks that it contains what
it should for specified transactions. However, the database log ordering
differs between big-endian and little-endian architectures because it is
written out in hash order. We don't want to incur the expense of always
sorting the log as we write it out, so instead this commit fixes the
problem by sorting the log as it reads it, using the "test-json" program.
Ben Pfaff [Fri, 7 May 2010 17:38:13 +0000 (10:38 -0700)]
tests: Fix bug in "weak references" test.
One part of the "weak references" test inserts invalid all-zeros weak
references into two columns and expects to get an error message mentioning
one of them. Unfortunately the one that actually gets mentioned depends
on hash ordering and thus differs between big-endian and little-endian
machines. This commit fixes the problem by only putting an invalid
reference in a single column, instead of two of them.
Ben Pfaff [Fri, 7 May 2010 17:34:35 +0000 (10:34 -0700)]
tests: Fix bug in ovsdb-server test suite.
The formatting of OVSDB syntax errors differed between big-endian and
little-endian systems, which caused the "database multiplexing
implementation" test to fail on SPARC. This commit fixes the problem by
always outputting JSON in syntax errors in deterministic (sorted) order.
Ben Pfaff [Fri, 7 May 2010 00:04:11 +0000 (17:04 -0700)]
util: Fix GCC false-positive warning for CONTAINER_OF.
On sparc, GCC would issue the following warning for every use of
CONTAINER_OF:
warning: cast increases required alignment of target type
This is a false positive: assuming that the data structure that it is
applied to was allocated properly, the use of CONTAINER_OF to reach it is
valid. And if it was not allocated properly, then code that accesses it
other ways will have trouble too.
Enables checksum offloading, scatter/gather, and TSO on internal
devices. While these optimizations were not previously enabled on
internal ports we already could receive these types of packets from
Xen guests. This has the obvious performance benefits when these
packets can be passed directly to hardware.
There is also a more subtle benefit for GRE on Xen. GRE packets
pass through OVS twice - once before encapsulation and once after
encapsulation, moving through an internal device in the process.
If it is a SG packet (as is common on Xen), a copy was necessary
to linearize for the internal device. However, Xen uses the
memory allocator to track packets so when the original packet is
freed after the copy netback notifies the guest that the packet
has been sent, despite the fact that it is actually sitting in the
transmit queue. The guest then sends packets as fast as the CPU
can handle, overflowing the transmit queue. By enabling SG on
the internal device, we avoid the copy and keep the accounting
correct.
In certain circumstances this patch can decrease performance for
TCP. TCP has its own mechanism for tracking in-flight packets
and therefore does not benefit from the corrected socket accounting.
However, certain NICs do not like SG when it is not being used for
TSO (these packets can no longer be handled by TSO after GRE
encapsulation). These NICs presumably enable SG even though they
can't handle it well because TSO requires SG.
Jesse Gross [Thu, 6 May 2010 22:15:50 +0000 (15:15 -0700)]
datapath: Break out test for dev_disable_lro().
It seems that dev_disable_lro() and skb_warn_if_lro() are not always
defined at the same time, despite the fact that they are typically
used together. This independently tests for them.
Justin Pettit [Thu, 6 May 2010 21:05:25 +0000 (14:05 -0700)]
Fix issue with "strict" deletion of flows
OpenFlow provides the ability to delete flows that match a "strict"
description. This means that wildcards are not active, and thus will
only match a single flow that exactly matches the description. The code
that checks for a match is pretty dumb and still compares the values of
fields that are wildcarded. A recent change added a "tun_id" matching
field, but did not zero out the field when it was supposed to be
ignored, which broke the matching used by strict deletions. This sets
the field regardless of whether the field is wildcarded or not.
Ben Pfaff [Mon, 3 May 2010 22:38:31 +0000 (15:38 -0700)]
netdev-linux: Optimize removing policing from an interface.
It is very expensive to start a subprocess and, especially, to wait for it
to complete. This replaces the most common subprocess operation in
netdev_linux_set_policing() by a Netlink socket operation, which is much
faster.
Without this and the other netdev-linux commits, my 1000-interface test
case runs in 1 min 48 s. With them, it runs in 25 seconds.
Ben Pfaff [Mon, 3 May 2010 18:47:56 +0000 (11:47 -0700)]
bridge: Optimize trunk port common case.
Profiling with qprof showed that bitmap_set_multiple() and bitmap_equal()
were eating up quite a bit of CPU time during bridge reconfiguration (up
to about 10% of total runtime). This is completely avoidable in the common
case where a port trunks all VLANs, where we don't really need a bitmap at
all. This commit implements that optimization.
Ben Pfaff [Mon, 3 May 2010 20:42:39 +0000 (13:42 -0700)]
bridge: Optimize port_lookup() using a hash.
Before this commit and the preceding one, with 1000 interfaces strcmp()
took 36% and port_lookup() took 8% of total runtime when reconfiguring
bridges. With these two commits the percentage is reduced to 3% and 0%,
respectively.
Ben Pfaff [Wed, 5 May 2010 21:00:47 +0000 (14:00 -0700)]
bridge: Optimize iface_lookup() and port_lookup_iface() with a hash.
Before this commit and the following one, with 1000 interfaces strcmp()
took 36% and port_lookup() took 8% of total runtime when reconfiguring
bridges. With these two commits the percentage is reduced to 3% and 0%,
respectively.
Ben Pfaff [Mon, 3 May 2010 22:43:49 +0000 (15:43 -0700)]
ovs-vswitchd: Implement "exit" unixctl command.
This is useful for profiling, since common profilers do not print anything
until the process terminates, and only if the process terminates in the
ordinary way by calling exit().
Neil McKee [Wed, 5 May 2010 20:26:23 +0000 (13:26 -0700)]
sflow: Always add poller and sampler together.
he ofproto_sflow_add_poller() and ofproto_sflow_add_sampler() calls
should always be made together, either when a port is added
dynamically with ofproto_sflow_add_port() or when the sflow_agent is
first created in ofproto_sflow_set_options(). I was seeing odd
behavior where either the pollers or the samplers would never be
instantiated depending on the order that things happened. (It's OK to
add the same sampler or poller again, because the library routines
sfl_agent_addPoller() and sfl_agent_addSampler() will just return the
existing one if it is there. Perhaps we should add a comment to make
that clear?).
I changed the parameters to ofproto_sflow_add_sampler to make it work
the same way as ofproto_sflow_add_poller, where the options are
extracted from os->options within the function itself.