2 Licensed under the Apache License, Version 2.0 (the "License"); you may
3 not use this file except in compliance with the License. You may obtain
4 a copy of the License at
6 http://www.apache.org/licenses/LICENSE-2.0
8 Unless required by applicable law or agreed to in writing, software
9 distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
10 WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
11 License for the specific language governing permissions and limitations
14 Convention for heading levels in Open vSwitch documentation:
16 ======= Heading 0 (reserved for the title in a document)
22 Avoid deeper levels because they do not render well.
28 Q: What versions of OpenFlow does Open vSwitch support?
30 A: The following table lists the versions of OpenFlow supported by each
31 version of Open vSwitch:
33 =============== ===== ===== ===== ===== ===== ===== =====
34 Open vSwitch OF1.0 OF1.1 OF1.2 OF1.3 OF1.4 OF1.5 OF1.6
35 =============== ===== ===== ===== ===== ===== ===== =====
36 1.9 and earlier yes --- --- --- --- --- ---
37 1.10 yes --- (*) (*) --- --- ---
38 1.11 yes --- (*) (*) --- --- ---
39 2.0 yes (*) (*) (*) --- --- ---
40 2.1 yes (*) (*) (*) --- --- ---
41 2.2 yes (*) (*) (*) (%) (*) ---
42 2.3 yes yes yes yes (*) (*) ---
43 2.4 yes yes yes yes (*) (*) ---
44 2.5 yes yes yes yes (*) (*) (*)
45 =============== ===== ===== ===== ===== ===== ===== =====
47 (*) Supported, with one or more missing features.
48 (%) Experimental, unsafe implementation.
50 Open vSwitch 2.3 enables OpenFlow 1.0, 1.1, 1.2, and 1.3 by default in
51 ovs-vswitchd. In Open vSwitch 1.10 through 2.2, OpenFlow 1.1, 1.2, and 1.3
52 must be enabled manually in ovs-vswitchd.
54 Some versions of OpenFlow are supported with missing features and therefore
55 not enabled by default: OpenFlow 1.4 and 1.5, in Open vSwitch 2.3 and
56 later, as well as OpenFlow 1.6 in Open vSwitch 2.5 and later. Also, the
57 OpenFlow 1.6 specification is still under development and thus subject to
60 In any case, the user may override the default:
62 - To enable OpenFlow 1.0, 1.1, 1.2, and 1.3 on bridge br0::
64 $ ovs-vsctl set bridge br0 \
65 protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13
67 - To enable OpenFlow 1.0, 1.1, 1.2, 1.3, 1.4, and 1.5 on bridge br0::
69 $ ovs-vsctl set bridge br0 \
70 protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFlow15
72 - To enable only OpenFlow 1.0 on bridge br0::
74 $ ovs-vsctl set bridge br0 protocols=OpenFlow10
76 All current versions of ovs-ofctl enable only OpenFlow 1.0 by default. Use
77 the -O option to enable support for later versions of OpenFlow in
78 ovs-ofctl. For example::
80 $ ovs-ofctl -O OpenFlow13 dump-flows br0
82 (Open vSwitch 2.2 had an experimental implementation of OpenFlow 1.4 that
83 could cause crashes. We don't recommend enabling it.)
85 :doc:`/topics/openflow` tracks support for OpenFlow 1.1 and later features.
86 When support for OpenFlow 1.4 and 1.5 is solidly implemented, Open vSwitch
87 will enable those version by default.
89 Q: Does Open vSwitch support MPLS?
91 A: Before version 1.11, Open vSwitch did not support MPLS. That is, these
92 versions can match on MPLS Ethernet types, but they cannot match, push, or
93 pop MPLS labels, nor can they look past MPLS labels into the encapsulated
96 Open vSwitch versions 1.11, 2.0, and 2.1 have very minimal support for
97 MPLS. With the userspace datapath only, these versions can match, push, or
98 pop a single MPLS label, but they still cannot look past MPLS labels (even
99 after popping them) into the encapsulated packet. Kernel datapath support
100 is unchanged from earlier versions.
102 Open vSwitch version 2.3 can match, push, or pop a single MPLS label and
103 look past the MPLS label into the encapsulated packet. Both userspace and
104 kernel datapaths will be supported, but MPLS processing always happens in
105 userspace either way, so kernel datapath performance will be disappointing.
107 Open vSwitch version 2.4 can match, push, or pop up to 3 MPLS labels and
108 look past the MPLS label into the encapsulated packet. It will have kernel
109 support for MPLS, yielding improved performance.
111 Q: I'm getting "error type 45250 code 0". What's that?
113 A: This is a Open vSwitch extension to OpenFlow error codes. Open vSwitch
114 uses this extension when it must report an error to an OpenFlow controller
115 but no standard OpenFlow error code is suitable.
117 Open vSwitch logs the errors that it sends to controllers, so the easiest
118 thing to do is probably to look at the ovs-vswitchd log to find out what
121 If you want to dissect the extended error message yourself, the format is
122 documented in include/openflow/nicira-ext.h in the Open vSwitch source
123 distribution. The extended error codes are documented in
124 include/openvswitch/ofp-errors.h.
126 Q: Some of the traffic that I'd expect my OpenFlow controller to see doesn't
127 actually appear through the OpenFlow connection, even though I know that it's
130 A: By default, Open vSwitch assumes that OpenFlow controllers are connected
131 "in-band", that is, that the controllers are actually part of the network
132 that is being controlled. In in-band mode, Open vSwitch sets up special
133 "hidden" flows to make sure that traffic can make it back and forth between
134 OVS and the controllers. These hidden flows are higher priority than any
135 flows that can be set up through OpenFlow, and they are not visible through
136 normal OpenFlow flow table dumps.
138 Usually, the hidden flows are desirable and helpful, but occasionally they
139 can cause unexpected behavior. You can view the full OpenFlow flow table,
140 including hidden flows, on bridge br0 with the command::
142 $ ovs-appctl bridge/dump-flows br0
144 to help you debug. The hidden flows are those with priorities
145 greater than 65535 (the maximum priority that can be set with
148 The ``Documentation/topics/design`` doc describes the in-band model in
151 If your controllers are not actually in-band (e.g. they are on
152 localhost via 127.0.0.1, or on a separate network), then you should
153 configure your controllers in "out-of-band" mode. If you have one
154 controller on bridge br0, then you can configure out-of-band mode
157 $ ovs-vsctl set controller br0 connection-mode=out-of-band
159 Q: Some of the OpenFlow flows that my controller sets up don't seem to apply to
160 certain traffic, especially traffic between OVS and the controller itself.
164 Q: I configured all my controllers for out-of-band control mode but "ovs-appctl
165 bridge/dump-flows" still shows some hidden flows.
167 A: You probably have a remote manager configured (e.g. with "ovs-vsctl
168 set-manager"). By default, Open vSwitch assumes that managers need in-band
169 rules set up on every bridge. You can disable these rules on bridge br0
172 $ ovs-vsctl set bridge br0 other-config:disable-in-band=true
174 This actually disables in-band control entirely for the bridge, as if all
175 the bridge's controllers were configured for out-of-band control.
177 Q: My OpenFlow controller doesn't see the VLANs that I expect.
179 A: See answer under "VLANs", above.
181 Q: I ran ``ovs-ofctl add-flow br0 nw_dst=192.168.0.1,actions=drop`` but I got a
182 funny message like this::
184 ofp_util|INFO|normalization changed ofp_match, details:
185 ofp_util|INFO| pre: nw_dst=192.168.0.1
188 and when I ran ``ovs-ofctl dump-flows br0`` I saw that my nw_dst match had
189 disappeared, so that the flow ends up matching every packet.
191 A: The term "normalization" in the log message means that a flow cannot
192 match on an L3 field without saying what L3 protocol is in use. The
193 "ovs-ofctl" command above didn't specify an L3 protocol, so the L3 field
196 In this case, the L3 protocol could be IP or ARP. A correct command for
197 each possibility is, respectively::
199 $ ovs-ofctl add-flow br0 ip,nw_dst=192.168.0.1,actions=drop
203 $ ovs-ofctl add-flow br0 arp,nw_dst=192.168.0.1,actions=drop
205 Similarly, a flow cannot match on an L4 field without saying what L4
206 protocol is in use. For example, the flow match ``tp_src=1234`` is, by
207 itself, meaningless and will be ignored. Instead, to match TCP source port
208 1234, write ``tcp,tp_src=1234``, or to match UDP source port 1234, write
211 Q: How can I figure out the OpenFlow port number for a given port?
213 A: The ``OFPT_FEATURES_REQUEST`` message requests an OpenFlow switch to
214 respond with an ``OFPT_FEATURES_REPLY`` that, among other information,
215 includes a mapping between OpenFlow port names and numbers. From a command
216 prompt, ``ovs-ofctl show br0`` makes such a request and prints the response
219 The Interface table in the Open vSwitch database also maps OpenFlow port
220 names to numbers. To print the OpenFlow port number associated with
221 interface eth0, run::
223 $ ovs-vsctl get Interface eth0 ofport
225 You can print the entire mapping with::
227 $ ovs-vsctl -- --columns=name,ofport list Interface
229 but the output mixes together interfaces from all bridges in the database,
230 so it may be confusing if more than one bridge exists.
232 In the Open vSwitch database, ofport value ``-1`` means that the interface
233 could not be created due to an error. (The Open vSwitch log should
234 indicate the reason.) ofport value ``[]`` (the empty set) means that the
235 interface hasn't been created yet. The latter is normally an intermittent
236 condition (unless ovs-vswitchd is not running).
238 Q: I added some flows with my controller or with ovs-ofctl, but when I run
239 "ovs-dpctl dump-flows" I don't see them.
241 A: ovs-dpctl queries a kernel datapath, not an OpenFlow switch. It won't
242 display the information that you want. You want to use ``ovs-ofctl
243 dump-flows`` instead.
245 Q: It looks like each of the interfaces in my bonded port shows up as an
246 individual OpenFlow port. Is that right?
248 A: Yes, Open vSwitch makes individual bond interfaces visible as OpenFlow
249 ports, rather than the bond as a whole. The interfaces are treated
250 together as a bond for only a few purposes:
252 - Sending a packet to the OFPP_NORMAL port. (When an OpenFlow controller
253 is not configured, this happens implicitly to every packet.)
255 - Mirrors configured for output to a bonded port.
257 It would make a lot of sense for Open vSwitch to present a bond as a single
258 OpenFlow port. If you want to contribute an implementation of such a
259 feature, please bring it up on the Open vSwitch development mailing list at
262 Q: I have a sophisticated network setup involving Open vSwitch, VMs or multiple
263 hosts, and other components. The behavior isn't what I expect. Help!
265 A: To debug network behavior problems, trace the path of a packet,
266 hop-by-hop, from its origin in one host to a remote host. If that's
267 correct, then trace the path of the response packet back to the origin.
269 The open source tool called ``plotnetcfg`` can help to understand the
270 relationship between the networking devices on a single host.
272 Usually a simple ICMP echo request and reply (``ping``) packet is good
273 enough. Start by initiating an ongoing ``ping`` from the origin host to a
274 remote host. If you are tracking down a connectivity problem, the "ping"
275 will not display any successful output, but packets are still being sent.
276 (In this case the packets being sent are likely ARP rather than ICMP.)
278 Tools available for tracing include the following:
280 - ``tcpdump`` and ``wireshark`` for observing hops across network devices,
281 such as Open vSwitch internal devices and physical wires.
283 - ``ovs-appctl dpif/dump-flows <br>`` in Open vSwitch 1.10 and later or
284 ``ovs-dpctl dump-flows <br>`` in earlier versions. These tools allow one
285 to observe the actions being taken on packets in ongoing flows.
287 See ovs-vswitchd(8) for ``ovs-appctl dpif/dump-flows`` documentation,
288 ovs-dpctl(8) for ``ovs-dpctl dump-flows`` documentation, and "Why are
289 there so many different ways to dump flows?" above for some background.
291 - ``ovs-appctl ofproto/trace`` to observe the logic behind how ovs-vswitchd
292 treats packets. See ovs-vswitchd(8) for documentation. You can out more
293 details about a given flow that ``ovs-dpctl dump-flows`` displays, by
294 cutting and pasting a flow from the output into an ``ovs-appctl
295 ofproto/trace`` command.
297 - SPAN, RSPAN, and ERSPAN features of physical switches, to observe what
298 goes on at these physical hops.
300 Starting at the origin of a given packet, observe the packet at each hop in
301 turn. For example, in one plausible scenario, you might:
303 1. ``tcpdump`` the ``eth`` interface through which an ARP egresses a VM,
306 2. ``tcpdump`` the ``vif`` or ``tap`` interface through which the ARP
307 ingresses the host machine.
309 3. Use ``ovs-dpctl dump-flows`` to spot the ARP flow and observe the host
310 interface through which the ARP egresses the physical machine. You may
311 need to use ``ovs-dpctl show`` to interpret the port numbers. If the
312 output seems surprising, you can use ``ovs-appctl ofproto/trace`` to
313 observe details of how ovs-vswitchd determined the actions in the
314 ``ovs-dpctl dump-flows`` output.
316 4. ``tcpdump`` the ``eth`` interface through which the ARP egresses the
319 5. ``tcpdump`` the ``eth`` interface through which the ARP ingresses the
320 physical machine, at the remote host that receives the ARP.
322 6. Use ``ovs-dpctl dump-flows`` to spot the ARP flow on the remote host
323 remote host that receives the ARP and observe the VM ``vif`` or ``tap``
324 interface to which the flow is directed. Again, ``ovs-dpctl show`` and
325 ``ovs-appctl ofproto/trace`` might help.
327 7. ``tcpdump`` the ``vif`` or ``tap`` interface to which the ARP is
330 8. ``tcpdump`` the ``eth`` interface through which the ARP ingresses a VM,
333 It is likely that during one of these steps you will figure out the
334 problem. If not, then follow the ARP reply back to the origin, in reverse.
336 Q: How do I make a flow drop packets?
338 A: To drop a packet is to receive it without forwarding it. OpenFlow
339 explicitly specifies forwarding actions. Thus, a flow with an empty set of
340 actions does not forward packets anywhere, causing them to be dropped. You
341 can specify an empty set of actions with ``actions=`` on the ovs-ofctl
342 command line. For example::
344 $ ovs-ofctl add-flow br0 priority=65535,actions=
346 would cause every packet entering switch br0 to be dropped.
348 You can write "drop" explicitly if you like. The effect is the same.
349 Thus, the following command also causes every packet entering switch br0 to
352 $ ovs-ofctl add-flow br0 priority=65535,actions=drop
354 ``drop`` is not an action, either in OpenFlow or Open vSwitch. Rather, it
355 is only a way to say that there are no actions.
357 Q: I added a flow to send packets out the ingress port, like this::
359 $ ovs-ofctl add-flow br0 in_port=2,actions=2
361 but OVS drops the packets instead.
363 A: Yes, OpenFlow requires a switch to ignore attempts to send a packet out
364 its ingress port. The rationale is that dropping these packets makes it
365 harder to loop the network. Sometimes this behavior can even be
366 convenient, e.g. it is often the desired behavior in a flow that forwards a
367 packet to several ports ("floods" the packet).
369 Sometimes one really needs to send a packet out its ingress port
370 ("hairpin"). In this case, output to ``OFPP_IN_PORT``, which in ovs-ofctl
371 syntax is expressed as just ``in_port``, e.g.::
373 $ ovs-ofctl add-flow br0 in_port=2,actions=in_port
375 This also works in some circumstances where the flow doesn't match on the
376 input port. For example, if you know that your switch has five ports
377 numbered 2 through 6, then the following will send every received packet
378 out every port, even its ingress port::
380 $ ovs-ofctl add-flow br0 actions=2,3,4,5,6,in_port
384 $ ovs-ofctl add-flow br0 actions=all,in_port
386 Sometimes, in complicated flow tables with multiple levels of ``resubmit``
387 actions, a flow needs to output to a particular port that may or may not be
388 the ingress port. It's difficult to take advantage of ``OFPP_IN_PORT`` in
389 this situation. To help, Open vSwitch provides, as an OpenFlow extension,
390 the ability to modify the in_port field. Whatever value is currently in
391 the in_port field is the port to which outputs will be dropped, as well as
392 the destination for ``OFPP_IN_PORT``. This means that the following will
393 reliably output to port 2 or to ports 2 through 6, respectively::
395 $ ovs-ofctl add-flow br0 in_port=2,actions=load:0->NXM_OF_IN_PORT[],2
396 $ ovs-ofctl add-flow br0 actions=load:0->NXM_OF_IN_PORT[],2,3,4,5,6
398 If the input port is important, then one may save and restore it on the
401 $ ovs-ofctl add-flow br0 actions=push:NXM_OF_IN_PORT[],\
402 load:0->NXM_OF_IN_PORT[],\
406 Q: My bridge br0 has host 192.168.0.1 on port 1 and host 192.168.0.2 on port 2.
407 I set up flows to forward only traffic destined to the other host and drop
408 other traffic, like this::
410 priority=5,in_port=1,ip,nw_dst=192.168.0.2,actions=2
411 priority=5,in_port=2,ip,nw_dst=192.168.0.1,actions=1
412 priority=0,actions=drop
414 But it doesn't work--I don't get any connectivity when I do this. Why?
416 A: These flows drop the ARP packets that IP hosts use to establish IP
417 connectivity over Ethernet. To solve the problem, add flows to allow ARP
418 to pass between the hosts::
420 priority=5,in_port=1,arp,actions=2
421 priority=5,in_port=2,arp,actions=1
423 This issue can manifest other ways, too. The following flows that match on
424 Ethernet addresses instead of IP addresses will also drop ARP packets,
425 because ARP requests are broadcast instead of being directed to a specific
428 priority=5,in_port=1,dl_dst=54:00:00:00:00:02,actions=2
429 priority=5,in_port=2,dl_dst=54:00:00:00:00:01,actions=1
430 priority=0,actions=drop
432 The solution already described above will also work in this case. It may
433 be better to add flows to allow all multicast and broadcast traffic::
435 priority=5,in_port=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,actions=2
436 priority=5,in_port=2,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,actions=1
438 Q: My bridge disconnects from my controller on add-port/del-port.
440 A: Reconfiguring your bridge can change your bridge's datapath-id because
441 Open vSwitch generates datapath-id from the MAC address of one of its
442 ports. In that case, Open vSwitch disconnects from controllers because
443 there's no graceful way to notify controllers about the change of
446 To avoid the behaviour, you can configure datapath-id manually.::
448 $ ovs-vsctl set bridge br0 other-config:datapath-id=0123456789abcdef
450 Q: My controller complains that OVS is not buffering packets.
453 A: "Packet buffering" is an optional OpenFlow feature, and controllers
454 should detect how many "buffers" an OpenFlow switch implements. It was
455 recently noticed that OVS implementation of the buffering feature was not
456 compliant to OpenFlow specifications. Rather than fix it and risk
457 controller incompatibility, the buffering feature is removed as of OVS 2.7.
458 Controllers are already expected to work properly in cases where the switch
459 can not buffer packets, but sends full packets in "packet-in" messages
460 instead, so this change should not affect existing users. After the change
461 OVS always sends the ``buffer_id`` as ``0xffffffff`` in "packet-in"
462 messages and will send an error response if any other value of this field
463 is included in a "packet-out" or a "flow mod" sent by a controller.
465 Q: How does OVS divide flows among buckets in an OpenFlow "select" group?
467 A: In Open vSwitch 2.3 and earlier, Open vSwitch used the destination
468 Ethernet address to choose a bucket in a select group.
470 Open vSwitch 2.4 and later by default hashes the source and destination
471 Ethernet address, VLAN ID, Ethernet type, IPv4/v6 source and destination
472 address and protocol, and for TCP and SCTP only, the source and destination
473 ports. The hash is "symmetric", meaning that exchanging source and
474 destination addresses does not change the bucket selection.
476 Select groups in Open vSwitch 2.4 and later can be configured to use a
477 different hash function, using a Netronome extension to the OpenFlow 1.5+
478 group_mod message. For more information, see
479 Documentation/group-selection-method-property.txt in the Open vSwitch
480 source tree. (OpenFlow 1.5 support in Open vSwitch is still experimental.)
482 Q: I added a flow to accept packets on VLAN 123 and output them on VLAN 456,
485 $ ovs-ofctl add-flow br0 dl_vlan=123,actions=output:1,mod_vlan_vid:456
487 but the packets are actually being output in VLAN 123. Why?
489 A: OpenFlow actions are executed in the order specified. Thus, the actions
490 above first output the packet, then change its VLAN. Since the output
491 occurs before changing the VLAN, the change in VLAN will have no visible
494 To solve this and similar problems, order actions so that changes to
495 headers happen before output, e.g.::
497 $ ovs-ofctl add-flow br0 dl_vlan=123,actions=mod_vlan_vid:456,output:1
499 Q: The "learn" action can't learn the action I want, can you improve it?
501 A: By itself, the "learn" action can only put two kinds of actions into the
502 flows that it creates: "load" and "output" actions. If "learn" is used in
503 isolation, these are severe limits.
505 However, "learn" is not meant to be used in isolation. It is a primitive
506 meant to be used together with other Open vSwitch features to accomplish a
507 task. Its existing features are enough to accomplish most tasks.
509 Here is an outline of a typical pipeline structure that allows for
510 versatile behavior using "learn":
512 - Flows in table A contain a "learn" action, that populates flows in table
513 L, that use a "load" action to populate register R with information about
516 - Flows in table B contain two sequential resubmit actions: one to table L
517 and another one to table B+1.
519 - Flows in table B+1 match on register R and act differently depending on
520 what the flows in table L loaded into it.
522 This approach can be used to implement many "learn"-based features. For
525 - Resubmit to a table selected based on learned information, e.g. see:
526 https://mail.openvswitch.org/pipermail/ovs-discuss/2016-June/021694.html
528 - MAC learning in the middle of a pipeline, as described in
529 :doc:`/tutorials/ovs-advanced`
531 - TCP state based firewalling, by learning outgoing connections based on
532 SYN packets and matching them up with incoming packets.
534 - At least some of the features described in T. A. Hoff, "Extending Open
535 vSwitch to Facilitate Creation of Stateful SDN Applications".