]> git.proxmox.com Git - mirror_ovs.git/blob - FAQ.md
d283c1aeee53a0a7c088cd7a4f1dc077b2831b67
[mirror_ovs.git] / FAQ.md
1 Frequently Asked Questions
2 ==========================
3
4 Open vSwitch <http://openvswitch.org>
5
6 General
7 -------
8
9 ### Q: What is Open vSwitch?
10
11 A: Open vSwitch is a production quality open source software switch
12 designed to be used as a vswitch in virtualized server
13 environments. A vswitch forwards traffic between different VMs on
14 the same physical host and also forwards traffic between VMs and
15 the physical network. Open vSwitch supports standard management
16 interfaces (e.g. sFlow, NetFlow, IPFIX, RSPAN, CLI), and is open to
17 programmatic extension and control using OpenFlow and the OVSDB
18 management protocol.
19
20 Open vSwitch as designed to be compatible with modern switching
21 chipsets. This means that it can be ported to existing high-fanout
22 switches allowing the same flexible control of the physical
23 infrastructure as the virtual infrastructure. It also means that
24 Open vSwitch will be able to take advantage of on-NIC switching
25 chipsets as their functionality matures.
26
27 ### Q: What virtualization platforms can use Open vSwitch?
28
29 A: Open vSwitch can currently run on any Linux-based virtualization
30 platform (kernel 2.6.32 and newer), including: KVM, VirtualBox, Xen,
31 Xen Cloud Platform, XenServer. As of Linux 3.3 it is part of the
32 mainline kernel. The bulk of the code is written in platform-
33 independent C and is easily ported to other environments. We welcome
34 inquires about integrating Open vSwitch with other virtualization
35 platforms.
36
37 ### Q: How can I try Open vSwitch?
38
39 A: The Open vSwitch source code can be built on a Linux system. You can
40 build and experiment with Open vSwitch on any Linux machine.
41 Packages for various Linux distributions are available on many
42 platforms, including: Debian, Ubuntu, Fedora.
43
44 You may also download and run a virtualization platform that already
45 has Open vSwitch integrated. For example, download a recent ISO for
46 XenServer or Xen Cloud Platform. Be aware that the version
47 integrated with a particular platform may not be the most recent Open
48 vSwitch release.
49
50 ### Q: Does Open vSwitch only work on Linux?
51
52 A: No, Open vSwitch has been ported to a number of different operating
53 systems and hardware platforms. Most of the development work occurs
54 on Linux, but the code should be portable to any POSIX system. We've
55 seen Open vSwitch ported to a number of different platforms,
56 including FreeBSD, Windows, and even non-POSIX embedded systems.
57
58 By definition, the Open vSwitch Linux kernel module only works on
59 Linux and will provide the highest performance. However, a userspace
60 datapath is available that should be very portable.
61
62 ### Q: What's involved with porting Open vSwitch to a new platform or switching ASIC?
63
64 A: The [PORTING.md] document describes how one would go about
65 porting Open vSwitch to a new operating system or hardware platform.
66
67 ### Q: Why would I use Open vSwitch instead of the Linux bridge?
68
69 A: Open vSwitch is specially designed to make it easier to manage VM
70 network configuration and monitor state spread across many physical
71 hosts in dynamic virtualized environments. Please see
72 [WHY-OVS.md] for a more detailed description of how Open vSwitch
73 relates to the Linux Bridge.
74
75 ### Q: How is Open vSwitch related to distributed virtual switches like the VMware vNetwork distributed switch or the Cisco Nexus 1000V?
76
77 A: Distributed vswitch applications (e.g., VMware vNetwork distributed
78 switch, Cisco Nexus 1000V) provide a centralized way to configure and
79 monitor the network state of VMs that are spread across many physical
80 hosts. Open vSwitch is not a distributed vswitch itself, rather it
81 runs on each physical host and supports remote management in a way
82 that makes it easier for developers of virtualization/cloud
83 management platforms to offer distributed vswitch capabilities.
84
85 To aid in distribution, Open vSwitch provides two open protocols that
86 are specially designed for remote management in virtualized network
87 environments: OpenFlow, which exposes flow-based forwarding state,
88 and the OVSDB management protocol, which exposes switch port state.
89 In addition to the switch implementation itself, Open vSwitch
90 includes tools (ovs-ofctl, ovs-vsctl) that developers can script and
91 extend to provide distributed vswitch capabilities that are closely
92 integrated with their virtualization management platform.
93
94 ### Q: Why doesn't Open vSwitch support distribution?
95
96 A: Open vSwitch is intended to be a useful component for building
97 flexible network infrastructure. There are many different approaches
98 to distribution which balance trade-offs between simplicity,
99 scalability, hardware compatibility, convergence times, logical
100 forwarding model, etc. The goal of Open vSwitch is to be able to
101 support all as a primitive building block rather than choose a
102 particular point in the distributed design space.
103
104 ### Q: How can I contribute to the Open vSwitch Community?
105
106 A: You can start by joining the mailing lists and helping to answer
107 questions. You can also suggest improvements to documentation. If
108 you have a feature or bug you would like to work on, send a mail to
109 one of the mailing lists:
110
111 http://openvswitch.org/mlists/
112
113 ### Q: Why can I no longer connect to my OpenFlow controller or OVSDB
114 manager?
115
116 A: Starting in OVS 2.4, we switched the default ports to the
117 IANA-specified port numbers for OpenFlow (6633->6653) and OVSDB
118 (6632->6640). We recommend using these port numbers, but if you
119 cannot, all the programs allow overriding the default port. See the
120 appropriate man page.
121
122
123 Releases
124 --------
125
126 ### Q: What does it mean for an Open vSwitch release to be LTS (long-term support)?
127
128 A: All official releases have been through a comprehensive testing
129 process and are suitable for production use. Planned releases will
130 occur several times a year. If a significant bug is identified in an
131 LTS release, we will provide an updated release that includes the
132 fix. Releases that are not LTS may not be fixed and may just be
133 supplanted by the next major release. The current LTS release is
134 2.3.x.
135
136 ### Q: What Linux kernel versions does each Open vSwitch release work with?
137
138 A: The following table lists the Linux kernel versions against which the
139 given versions of the Open vSwitch kernel module will successfully
140 build. The Linux kernel versions are upstream kernel versions, so
141 Linux kernels modified from the upstream sources may not build in
142 some cases even if they are based on a supported version. This is
143 most notably true of Red Hat Enterprise Linux (RHEL) kernels, which
144 are extensively modified from upstream.
145
146 | Open vSwitch | Linux kernel
147 |:------------:|:-------------:
148 | 1.4.x | 2.6.18 to 3.2
149 | 1.5.x | 2.6.18 to 3.2
150 | 1.6.x | 2.6.18 to 3.2
151 | 1.7.x | 2.6.18 to 3.3
152 | 1.8.x | 2.6.18 to 3.4
153 | 1.9.x | 2.6.18 to 3.8
154 | 1.10.x | 2.6.18 to 3.8
155 | 1.11.x | 2.6.18 to 3.8
156 | 2.0.x | 2.6.32 to 3.10
157 | 2.1.x | 2.6.32 to 3.11
158 | 2.3.x | 2.6.32 to 3.14
159 | 2.4.x | 2.6.32 to 3.19
160
161 Open vSwitch userspace should also work with the Linux kernel module
162 built into Linux 3.3 and later.
163
164 Open vSwitch userspace is not sensitive to the Linux kernel version.
165 It should build against almost any kernel, certainly against 2.6.32
166 and later.
167
168 ### Q: I get an error like this when I configure Open vSwitch:
169
170 configure: error: Linux kernel in <dir> is version <x>, but
171 version newer than <y> is not supported (please refer to the
172 FAQ for advice)
173
174 What should I do?
175
176 A: You have the following options:
177
178 - Use the Linux kernel module supplied with the kernel that you are
179 using. (See also the following FAQ.)
180
181 - If there is a newer released version of Open vSwitch, consider
182 building that one, because it may support the kernel that you are
183 building against. (To find out, consult the table in the
184 previous FAQ.)
185
186 - The Open vSwitch "master" branch may support the kernel that you
187 are using, so consider building the kernel module from "master".
188
189 All versions of Open vSwitch userspace are compatible with all
190 versions of the Open vSwitch kernel module, so you do not have to
191 use the kernel module from one source along with the userspace
192 programs from the same source.
193
194 ### Q: What features are not available in the Open vSwitch kernel datapath that ships as part of the upstream Linux kernel?
195
196 A: The kernel module in upstream Linux does not include support for
197 LISP. Work is in progress to add support for LISP to the upstream
198 Linux version of the Open vSwitch kernel module. For now, if you
199 need this feature, use the kernel module from the Open vSwitch
200 distribution instead of the upstream Linux kernel module.
201
202 Certain features require kernel support to function or to have
203 reasonable performance. If the ovs-vswitchd log file indicates that
204 a feature is not supported, consider upgrading to a newer upstream
205 Linux release or using the kernel module paired with the userspace
206 distribution.
207
208 ### Q: Why do tunnels not work when using a kernel module other than the one packaged with Open vSwitch?
209
210 A: Support for tunnels was added to the upstream Linux kernel module
211 after the rest of Open vSwitch. As a result, some kernels may contain
212 support for Open vSwitch but not tunnels. The minimum kernel version
213 that supports each tunnel protocol is:
214
215 | Protocol | Linux Kernel
216 |:--------:|:-------------:
217 | GRE | 3.11
218 | VXLAN | 3.12
219 | Geneve | 3.18
220 | LISP | <not upstream>
221 | STT | <not upstream>
222
223 If you are using a version of the kernel that is older than the one
224 listed above, it is still possible to use that tunnel protocol. However,
225 you must compile and install the kernel module included with the Open
226 vSwitch distribution rather than the one on your machine. If problems
227 persist after doing this, check to make sure that the module that is
228 loaded is the one you expect.
229
230 ### Q: Why are UDP tunnel checksums not computed for VXLAN or Geneve?
231
232 A: Generating outer UDP checksums requires kernel support that was not
233 part of the initial implementation of these protocols. If using the
234 upstream Linux Open vSwitch module, you must use kernel 4.0 or
235 newer. The out-of-tree modules from Open vSwitch release 2.4 and later
236 support UDP checksums.
237
238 ### Q: What features are not available when using the userspace datapath?
239
240 A: Tunnel virtual ports are not supported, as described in the
241 previous answer. It is also not possible to use queue-related
242 actions. On Linux kernels before 2.6.39, maximum-sized VLAN packets
243 may not be transmitted.
244
245 ### Q: What Linux kernel versions does IPFIX flow monitoring work with?
246
247 A: IPFIX flow monitoring requires the Linux kernel module from Linux
248 3.10 or later, or the out-of-tree module from Open vSwitch version
249 1.10.90 or later.
250
251 ### Q: Should userspace or kernel be upgraded first to minimize downtime?
252
253 In general, the Open vSwitch userspace should be used with the
254 kernel version included in the same release or with the version
255 from upstream Linux. However, when upgrading between two releases
256 of Open vSwitch it is best to migrate userspace first to reduce
257 the possibility of incompatibilities.
258
259 ### Q: What happened to the bridge compatibility feature?
260
261 A: Bridge compatibility was a feature of Open vSwitch 1.9 and earlier.
262 When it was enabled, Open vSwitch imitated the interface of the
263 Linux kernel "bridge" module. This allowed users to drop Open
264 vSwitch into environments designed to use the Linux kernel bridge
265 module without adapting the environment to use Open vSwitch.
266
267 Open vSwitch 1.10 and later do not support bridge compatibility.
268 The feature was dropped because version 1.10 adopted a new internal
269 architecture that made bridge compatibility difficult to maintain.
270 Now that many environments use OVS directly, it would be rarely
271 useful in any case.
272
273 To use bridge compatibility, install OVS 1.9 or earlier, including
274 the accompanying kernel modules (both the main and bridge
275 compatibility modules), following the instructions that come with
276 the release. Be sure to start the ovs-brcompatd daemon.
277
278
279 Terminology
280 -----------
281
282 ### Q: I thought Open vSwitch was a virtual Ethernet switch, but the documentation keeps talking about bridges. What's a bridge?
283
284 A: In networking, the terms "bridge" and "switch" are synonyms. Open
285 vSwitch implements an Ethernet switch, which means that it is also
286 an Ethernet bridge.
287
288 ### Q: What's a VLAN?
289
290 A: See the "VLAN" section below.
291
292
293 Basic Configuration
294 -------------------
295
296 ### Q: How do I configure a port as an access port?
297
298 A: Add "tag=VLAN" to your "ovs-vsctl add-port" command. For example,
299 the following commands configure br0 with eth0 as a trunk port (the
300 default) and tap0 as an access port for VLAN 9:
301
302 ovs-vsctl add-br br0
303 ovs-vsctl add-port br0 eth0
304 ovs-vsctl add-port br0 tap0 tag=9
305
306 If you want to configure an already added port as an access port,
307 use "ovs-vsctl set", e.g.:
308
309 ovs-vsctl set port tap0 tag=9
310
311 ### Q: How do I configure a port as a SPAN port, that is, enable mirroring of all traffic to that port?
312
313 A: The following commands configure br0 with eth0 and tap0 as trunk
314 ports. All traffic coming in or going out on eth0 or tap0 is also
315 mirrored to tap1; any traffic arriving on tap1 is dropped:
316
317 ovs-vsctl add-br br0
318 ovs-vsctl add-port br0 eth0
319 ovs-vsctl add-port br0 tap0
320 ovs-vsctl add-port br0 tap1 \
321 -- --id=@p get port tap1 \
322 -- --id=@m create mirror name=m0 select-all=true output-port=@p \
323 -- set bridge br0 mirrors=@m
324
325 To later disable mirroring, run:
326
327 ovs-vsctl clear bridge br0 mirrors
328
329 ### Q: Does Open vSwitch support configuring a port in promiscuous mode?
330
331 A: Yes. How you configure it depends on what you mean by "promiscuous
332 mode":
333
334 - Conventionally, "promiscuous mode" is a feature of a network
335 interface card. Ordinarily, a NIC passes to the CPU only the
336 packets actually destined to its host machine. It discards
337 the rest to avoid wasting memory and CPU cycles. When
338 promiscuous mode is enabled, however, it passes every packet
339 to the CPU. On an old-style shared-media or hub-based
340 network, this allows the host to spy on all packets on the
341 network. But in the switched networks that are almost
342 everywhere these days, promiscuous mode doesn't have much
343 effect, because few packets not destined to a host are
344 delivered to the host's NIC.
345
346 This form of promiscuous mode is configured in the guest OS of
347 the VMs on your bridge, e.g. with "ifconfig".
348
349 - The VMware vSwitch uses a different definition of "promiscuous
350 mode". When you configure promiscuous mode on a VMware vNIC,
351 the vSwitch sends a copy of every packet received by the
352 vSwitch to that vNIC. That has a much bigger effect than just
353 enabling promiscuous mode in a guest OS. Rather than getting
354 a few stray packets for which the switch does not yet know the
355 correct destination, the vNIC gets every packet. The effect
356 is similar to replacing the vSwitch by a virtual hub.
357
358 This "promiscuous mode" is what switches normally call "port
359 mirroring" or "SPAN". For information on how to configure
360 SPAN, see "How do I configure a port as a SPAN port, that is,
361 enable mirroring of all traffic to that port?"
362
363 ### Q: How do I configure a VLAN as an RSPAN VLAN, that is, enable mirroring of all traffic to that VLAN?
364
365 A: The following commands configure br0 with eth0 as a trunk port and
366 tap0 as an access port for VLAN 10. All traffic coming in or going
367 out on tap0, as well as traffic coming in or going out on eth0 in
368 VLAN 10, is also mirrored to VLAN 15 on eth0. The original tag for
369 VLAN 10, in cases where one is present, is dropped as part of
370 mirroring:
371
372 ovs-vsctl add-br br0
373 ovs-vsctl add-port br0 eth0
374 ovs-vsctl add-port br0 tap0 tag=10
375 ovs-vsctl \
376 -- --id=@m create mirror name=m0 select-all=true select-vlan=10 \
377 output-vlan=15 \
378 -- set bridge br0 mirrors=@m
379
380 To later disable mirroring, run:
381
382 ovs-vsctl clear bridge br0 mirrors
383
384 Mirroring to a VLAN can disrupt a network that contains unmanaged
385 switches. See ovs-vswitchd.conf.db(5) for details. Mirroring to a
386 GRE tunnel has fewer caveats than mirroring to a VLAN and should
387 generally be preferred.
388
389 ### Q: Can I mirror more than one input VLAN to an RSPAN VLAN?
390
391 A: Yes, but mirroring to a VLAN strips the original VLAN tag in favor
392 of the specified output-vlan. This loss of information may make
393 the mirrored traffic too hard to interpret.
394
395 To mirror multiple VLANs, use the commands above, but specify a
396 comma-separated list of VLANs as the value for select-vlan. To
397 mirror every VLAN, use the commands above, but omit select-vlan and
398 its value entirely.
399
400 When a packet arrives on a VLAN that is used as a mirror output
401 VLAN, the mirror is disregarded. Instead, in standalone mode, OVS
402 floods the packet across all the ports for which the mirror output
403 VLAN is configured. (If an OpenFlow controller is in use, then it
404 can override this behavior through the flow table.) If OVS is used
405 as an intermediate switch, rather than an edge switch, this ensures
406 that the RSPAN traffic is distributed through the network.
407
408 Mirroring to a VLAN can disrupt a network that contains unmanaged
409 switches. See ovs-vswitchd.conf.db(5) for details. Mirroring to a
410 GRE tunnel has fewer caveats than mirroring to a VLAN and should
411 generally be preferred.
412
413 ### Q: How do I configure mirroring of all traffic to a GRE tunnel?
414
415 A: The following commands configure br0 with eth0 and tap0 as trunk
416 ports. All traffic coming in or going out on eth0 or tap0 is also
417 mirrored to gre0, a GRE tunnel to the remote host 192.168.1.10; any
418 traffic arriving on gre0 is dropped:
419
420 ovs-vsctl add-br br0
421 ovs-vsctl add-port br0 eth0
422 ovs-vsctl add-port br0 tap0
423 ovs-vsctl add-port br0 gre0 \
424 -- set interface gre0 type=gre options:remote_ip=192.168.1.10 \
425 -- --id=@p get port gre0 \
426 -- --id=@m create mirror name=m0 select-all=true output-port=@p \
427 -- set bridge br0 mirrors=@m
428
429 To later disable mirroring and destroy the GRE tunnel:
430
431 ovs-vsctl clear bridge br0 mirrors
432 ovs-vcstl del-port br0 gre0
433
434 ### Q: Does Open vSwitch support ERSPAN?
435
436 A: No. ERSPAN is an undocumented proprietary protocol. As an
437 alternative, Open vSwitch supports mirroring to a GRE tunnel (see
438 above).
439
440 ### Q: How do I connect two bridges?
441
442 A: First, why do you want to do this? Two connected bridges are not
443 much different from a single bridge, so you might as well just have
444 a single bridge with all your ports on it.
445
446 If you still want to connect two bridges, you can use a pair of
447 patch ports. The following example creates bridges br0 and br1,
448 adds eth0 and tap0 to br0, adds tap1 to br1, and then connects br0
449 and br1 with a pair of patch ports.
450
451 ovs-vsctl add-br br0
452 ovs-vsctl add-port br0 eth0
453 ovs-vsctl add-port br0 tap0
454 ovs-vsctl add-br br1
455 ovs-vsctl add-port br1 tap1
456 ovs-vsctl \
457 -- add-port br0 patch0 \
458 -- set interface patch0 type=patch options:peer=patch1 \
459 -- add-port br1 patch1 \
460 -- set interface patch1 type=patch options:peer=patch0
461
462 Bridges connected with patch ports are much like a single bridge.
463 For instance, if the example above also added eth1 to br1, and both
464 eth0 and eth1 happened to be connected to the same next-hop switch,
465 then you could loop your network just as you would if you added
466 eth0 and eth1 to the same bridge (see the "Configuration Problems"
467 section below for more information).
468
469 If you are using Open vSwitch 1.9 or an earlier version, then you
470 need to be using the kernel module bundled with Open vSwitch rather
471 than the one that is integrated into Linux 3.3 and later, because
472 Open vSwitch 1.9 and earlier versions need kernel support for patch
473 ports. This also means that in Open vSwitch 1.9 and earlier, patch
474 ports will not work with the userspace datapath, only with the
475 kernel module.
476
477 ### Q: How do I configure a bridge without an OpenFlow local port? (Local port in the sense of OFPP_LOCAL)
478
479 A: Open vSwitch does not support such a configuration.
480 Bridges always have their local ports.
481
482
483 Implementation Details
484 ----------------------
485
486 ### Q: I hear OVS has a couple of kinds of flows. Can you tell me about them?
487
488 A: Open vSwitch uses different kinds of flows for different purposes:
489
490 - OpenFlow flows are the most important kind of flow. OpenFlow
491 controllers use these flows to define a switch's policy.
492 OpenFlow flows support wildcards, priorities, and multiple
493 tables.
494
495 When in-band control is in use, Open vSwitch sets up a few
496 "hidden" flows, with priority higher than a controller or the
497 user can configure, that are not visible via OpenFlow. (See
498 the "Controller" section of the FAQ for more information
499 about hidden flows.)
500
501 - The Open vSwitch software switch implementation uses a second
502 kind of flow internally. These flows, called "datapath" or
503 "kernel" flows, do not support priorities and comprise only a
504 single table, which makes them suitable for caching. (Like
505 OpenFlow flows, datapath flows do support wildcarding, in Open
506 vSwitch 1.11 and later.) OpenFlow flows and datapath flows
507 also support different actions and number ports differently.
508
509 Datapath flows are an implementation detail that is subject to
510 change in future versions of Open vSwitch. Even with the
511 current version of Open vSwitch, hardware switch
512 implementations do not necessarily use this architecture.
513
514 Users and controllers directly control only the OpenFlow flow
515 table. Open vSwitch manages the datapath flow table itself, so
516 users should not normally be concerned with it.
517
518 ### Q: Why are there so many different ways to dump flows?
519
520 A: Open vSwitch has two kinds of flows (see the previous question), so
521 it has commands with different purposes for dumping each kind of
522 flow:
523
524 - `ovs-ofctl dump-flows <br>` dumps OpenFlow flows, excluding
525 hidden flows. This is the most commonly useful form of flow
526 dump. (Unlike the other commands, this should work with any
527 OpenFlow switch, not just Open vSwitch.)
528
529 - `ovs-appctl bridge/dump-flows <br>` dumps OpenFlow flows,
530 including hidden flows. This is occasionally useful for
531 troubleshooting suspected issues with in-band control.
532
533 - `ovs-dpctl dump-flows [dp]` dumps the datapath flow table
534 entries for a Linux kernel-based datapath. In Open vSwitch
535 1.10 and later, ovs-vswitchd merges multiple switches into a
536 single datapath, so it will show all the flows on all your
537 kernel-based switches. This command can occasionally be
538 useful for debugging.
539
540 - `ovs-appctl dpif/dump-flows <br>`, new in Open vSwitch 1.10,
541 dumps datapath flows for only the specified bridge, regardless
542 of the type.
543
544 ### Q: How does multicast snooping works with VLANs?
545
546 A: Open vSwitch maintains snooping tables for each VLAN.
547
548
549 Performance
550 -----------
551
552 ### Q: I just upgraded and I see a performance drop. Why?
553
554 A: The OVS kernel datapath may have been updated to a newer version than
555 the OVS userspace components. Sometimes new versions of OVS kernel
556 module add functionality that is backwards compatible with older
557 userspace components but may cause a drop in performance with them.
558 Especially, if a kernel module from OVS 2.1 or newer is paired with
559 OVS userspace 1.10 or older, there will be a performance drop for
560 TCP traffic.
561
562 Updating the OVS userspace components to the latest released
563 version should fix the performance degradation.
564
565 To get the best possible performance and functionality, it is
566 recommended to pair the same versions of the kernel module and OVS
567 userspace.
568
569
570 Configuration Problems
571 ----------------------
572
573 ### Q: I created a bridge and added my Ethernet port to it, using commands
574 like these:
575
576 ovs-vsctl add-br br0
577 ovs-vsctl add-port br0 eth0
578
579 and as soon as I ran the "add-port" command I lost all connectivity
580 through eth0. Help!
581
582 A: A physical Ethernet device that is part of an Open vSwitch bridge
583 should not have an IP address. If one does, then that IP address
584 will not be fully functional.
585
586 You can restore functionality by moving the IP address to an Open
587 vSwitch "internal" device, such as the network device named after
588 the bridge itself. For example, assuming that eth0's IP address is
589 192.168.128.5, you could run the commands below to fix up the
590 situation:
591
592 ifconfig eth0 0.0.0.0
593 ifconfig br0 192.168.128.5
594
595 (If your only connection to the machine running OVS is through the
596 IP address in question, then you would want to run all of these
597 commands on a single command line, or put them into a script.) If
598 there were any additional routes assigned to eth0, then you would
599 also want to use commands to adjust these routes to go through br0.
600
601 If you use DHCP to obtain an IP address, then you should kill the
602 DHCP client that was listening on the physical Ethernet interface
603 (e.g. eth0) and start one listening on the internal interface
604 (e.g. br0). You might still need to manually clear the IP address
605 from the physical interface (e.g. with "ifconfig eth0 0.0.0.0").
606
607 There is no compelling reason why Open vSwitch must work this way.
608 However, this is the way that the Linux kernel bridge module has
609 always worked, so it's a model that those accustomed to Linux
610 bridging are already used to. Also, the model that most people
611 expect is not implementable without kernel changes on all the
612 versions of Linux that Open vSwitch supports.
613
614 By the way, this issue is not specific to physical Ethernet
615 devices. It applies to all network devices except Open vSwitch
616 "internal" devices.
617
618 ### Q: I created a bridge and added a couple of Ethernet ports to it,
619 ### using commands like these:
620
621 ovs-vsctl add-br br0
622 ovs-vsctl add-port br0 eth0
623 ovs-vsctl add-port br0 eth1
624
625 and now my network seems to have melted: connectivity is unreliable
626 (even connectivity that doesn't go through Open vSwitch), all the
627 LEDs on my physical switches are blinking, wireshark shows
628 duplicated packets, and CPU usage is very high.
629
630 A: More than likely, you've looped your network. Probably, eth0 and
631 eth1 are connected to the same physical Ethernet switch. This
632 yields a scenario where OVS receives a broadcast packet on eth0 and
633 sends it out on eth1, then the physical switch connected to eth1
634 sends the packet back on eth0, and so on forever. More complicated
635 scenarios, involving a loop through multiple switches, are possible
636 too.
637
638 The solution depends on what you are trying to do:
639
640 - If you added eth0 and eth1 to get higher bandwidth or higher
641 reliability between OVS and your physical Ethernet switch,
642 use a bond. The following commands create br0 and then add
643 eth0 and eth1 as a bond:
644
645 ovs-vsctl add-br br0
646 ovs-vsctl add-bond br0 bond0 eth0 eth1
647
648 Bonds have tons of configuration options. Please read the
649 documentation on the Port table in ovs-vswitchd.conf.db(5)
650 for all the details.
651
652 - Perhaps you don't actually need eth0 and eth1 to be on the
653 same bridge. For example, if you simply want to be able to
654 connect each of them to virtual machines, then you can put
655 each of them on a bridge of its own:
656
657 ovs-vsctl add-br br0
658 ovs-vsctl add-port br0 eth0
659
660 ovs-vsctl add-br br1
661 ovs-vsctl add-port br1 eth1
662
663 and then connect VMs to br0 and br1. (A potential
664 disadvantage is that traffic cannot directly pass between br0
665 and br1. Instead, it will go out eth0 and come back in eth1,
666 or vice versa.)
667
668 - If you have a redundant or complex network topology and you
669 want to prevent loops, turn on spanning tree protocol (STP).
670 The following commands create br0, enable STP, and add eth0
671 and eth1 to the bridge. The order is important because you
672 don't want have to have a loop in your network even
673 transiently:
674
675 ovs-vsctl add-br br0
676 ovs-vsctl set bridge br0 stp_enable=true
677 ovs-vsctl add-port br0 eth0
678 ovs-vsctl add-port br0 eth1
679
680 The Open vSwitch implementation of STP is not well tested.
681 Please report any bugs you observe, but if you'd rather avoid
682 acting as a beta tester then another option might be your
683 best shot.
684
685 ### Q: I can't seem to use Open vSwitch in a wireless network.
686
687 A: Wireless base stations generally only allow packets with the source
688 MAC address of NIC that completed the initial handshake.
689 Therefore, without MAC rewriting, only a single device can
690 communicate over a single wireless link.
691
692 This isn't specific to Open vSwitch, it's enforced by the access
693 point, so the same problems will show up with the Linux bridge or
694 any other way to do bridging.
695
696 ### Q: I can't seem to add my PPP interface to an Open vSwitch bridge.
697
698 A: PPP most commonly carries IP packets, but Open vSwitch works only
699 with Ethernet frames. The correct way to interface PPP to an
700 Ethernet network is usually to use routing instead of switching.
701
702 ### Q: Is there any documentation on the database tables and fields?
703
704 A: Yes. ovs-vswitchd.conf.db(5) is a comprehensive reference.
705
706 ### Q: When I run ovs-dpctl I no longer see the bridges I created. Instead,
707 I only see a datapath called "ovs-system". How can I see datapath
708 information about a particular bridge?
709
710 A: In version 1.9.0, OVS switched to using a single datapath that is
711 shared by all bridges of that type. The "ovs-appctl dpif/*"
712 commands provide similar functionality that is scoped by the bridge.
713
714 ### Q: I created a GRE port using ovs-vsctl so why can't I send traffic or
715 see the port in the datapath?
716
717 A: On Linux kernels before 3.11, the OVS GRE module and Linux GRE module
718 cannot be loaded at the same time. It is likely that on your system the
719 Linux GRE module is already loaded and blocking OVS (to confirm, check
720 dmesg for errors regarding GRE registration). To fix this, unload all
721 GRE modules that appear in lsmod as well as the OVS kernel module. You
722 can then reload the OVS module following the directions in
723 [INSTALL.md], which will ensure that dependencies are satisfied.
724
725 ### Q: Open vSwitch does not seem to obey my packet filter rules.
726
727 A: It depends on mechanisms and configurations you want to use.
728
729 You cannot usefully use typical packet filters, like iptables, on
730 physical Ethernet ports that you add to an Open vSwitch bridge.
731 This is because Open vSwitch captures packets from the interface at
732 a layer lower below where typical packet-filter implementations
733 install their hooks. (This actually applies to any interface of
734 type "system" that you might add to an Open vSwitch bridge.)
735
736 You can usefully use typical packet filters on Open vSwitch
737 internal ports as they are mostly ordinary interfaces from the point
738 of view of packet filters.
739
740 For example, suppose you create a bridge br0 and add Ethernet port
741 eth0 to it. Then you can usefully add iptables rules to affect the
742 internal interface br0, but not the physical interface eth0. (br0
743 is also where you would add an IP address, as discussed elsewhere
744 in the FAQ.)
745
746 For simple filtering rules, it might be possible to achieve similar
747 results by installing appropriate OpenFlow flows instead.
748
749 If the use of a particular packet filter setup is essential, Open
750 vSwitch might not be the best choice for you. On Linux, you might
751 want to consider using the Linux Bridge. (This is the only choice if
752 you want to use ebtables rules.) On NetBSD, you might want to
753 consider using the bridge(4) with BRIDGE_IPF option.
754
755 ### Q: It seems that Open vSwitch does nothing when I removed a port and
756 then immediately put it back. For example, consider that p1 is
757 a port of type=internal:
758
759 ovs-vsctl del-port br0 p1 -- \
760 add-port br0 p1 -- \
761 set interface p1 type=internal
762
763 A: It's an expected behaviour.
764
765 If del-port and add-port happen in a single OVSDB transaction as
766 your example, Open vSwitch always "skips" the intermediate steps.
767 Even if they are done in multiple transactions, it's still allowed
768 for Open vSwitch to skip the intermediate steps and just implement
769 the overall effect. In both cases, your example would be turned
770 into a no-op.
771
772 If you want to make Open vSwitch actually destroy and then re-create
773 the port for some side effects like resetting kernel setting for the
774 corresponding interface, you need to separate operations into multiple
775 OVSDB transactions and ensure that at least the first one does not have
776 --no-wait. In the following example, the first ovs-vsctl will block
777 until Open vSwitch reloads the new configuration and removes the port:
778
779 ovs-vsctl del-port br0 p1
780 ovs-vsctl add-port br0 p1 -- \
781 set interface p1 type=internal
782
783 ### Q: I want to add thousands of ports to an Open vSwitch bridge, but
784 it takes too long (minutes or hours) to do it with ovs-vsctl. How
785 can I do it faster?
786
787 A: If you add them one at a time with ovs-vsctl, it can take a long
788 time to add thousands of ports to an Open vSwitch bridge. This is
789 because every invocation of ovs-vsctl first reads the current
790 configuration from OVSDB. As the number of ports grows, this
791 starts to take an appreciable amount of time, and when it is
792 repeated thousands of times the total time becomes significant.
793
794 The solution is to add the ports in one invocation of ovs-vsctl (or
795 a small number of them). For example, using bash:
796
797 ovs-vsctl add-br br0
798 cmds=; for i in {1..5000}; do cmds+=" -- add-port br0 p$i"; done
799 ovs-vsctl $cmds
800
801 takes seconds, not minutes or hours, in the OVS sandbox environment.
802
803 ### Q: I created a bridge named br0. My bridge shows up in "ovs-vsctl
804 show", but "ovs-ofctl show br0" just prints "br0 is not a bridge
805 or a socket".
806
807 A: Open vSwitch wasn't able to create the bridge. Check the
808 ovs-vswitchd log for details (Debian and Red Hat packaging for Open
809 vSwitch put it in /var/log/openvswitch/ovs-vswitchd.log).
810
811 In general, the Open vSwitch database reflects the desired
812 configuration state. ovs-vswitchd monitors the database and, when
813 it changes, reconfigures the system to reflect the new desired
814 state. This normally happens very quickly. Thus, a discrepancy
815 between the database and the actual state indicates that
816 ovs-vswitchd could not implement the configuration, and so one
817 should check the log to find out why. (Another possible cause is
818 that ovs-vswitchd is not running. This will make "ovs-vsctl"
819 commands hang, if they change the configuration, unless one
820 specifies "--no-wait".)
821
822 ### Q: I have a bridge br0. I added a new port vif1.0, and it shows
823 up in "ovs-vsctl show", but "ovs-vsctl list port" says that it has
824 OpenFlow port ("ofport") -1, and "ovs-ofctl show br0" doesn't show
825 vif1.0 at all.
826
827 A: Open vSwitch wasn't able to create the port. Check the
828 ovs-vswitchd log for details (Debian and Red Hat packaging for Open
829 vSwitch put it in /var/log/openvswitch/ovs-vswitchd.log). Please
830 see the previous question for more information.
831
832 You may want to upgrade to Open vSwitch 2.3 (or later), in which
833 ovs-vsctl will immediately report when there is an issue creating a
834 port.
835
836 ### Q: I created a tap device tap0, configured an IP address on it, and
837 added it to a bridge, like this:
838
839 tunctl -t tap0
840 ifconfig tap0 192.168.0.123
841 ovs-vsctl add-br br0
842 ovs-vsctl add-port br0 tap0
843
844 I expected that I could then use this IP address to contact other
845 hosts on the network, but it doesn't work. Why not?
846
847 A: The short answer is that this is a misuse of a "tap" device. Use
848 an "internal" device implemented by Open vSwitch, which works
849 differently and is designed for this use. To solve this problem
850 with an internal device, instead run:
851
852 ovs-vsctl add-br br0
853 ovs-vsctl add-port br0 int0 -- set Interface int0 type=internal
854 ifconfig int0 192.168.0.123
855
856 Even more simply, you can take advantage of the internal port that
857 every bridge has under the name of the bridge:
858
859 ovs-vsctl add-br br0
860 ifconfig br0 192.168.0.123
861
862 In more detail, a "tap" device is an interface between the Linux
863 (or *BSD) network stack and a user program that opens it as a
864 socket. When the "tap" device transmits a packet, it appears in
865 the socket opened by the userspace program. Conversely, when the
866 userspace program writes to the "tap" socket, the kernel TCP/IP
867 stack processes the packet as if it had been received by the "tap"
868 device.
869
870 Consider the configuration above. Given this configuration, if you
871 "ping" an IP address in the 192.168.0.x subnet, the Linux kernel
872 routing stack will transmit an ARP on the tap0 device. Open
873 vSwitch userspace treats "tap" devices just like any other network
874 device; that is, it doesn't open them as "tap" sockets. That means
875 that the ARP packet will simply get dropped.
876
877 You might wonder why the Open vSwitch kernel module doesn't
878 intercept the ARP packet and bridge it. After all, Open vSwitch
879 intercepts packets on other devices. The answer is that Open
880 vSwitch only intercepts *received* packets, but this is a packet
881 being transmitted. The same thing happens for all other types of
882 network devices, except for Open vSwitch "internal" ports. If you,
883 for example, add a physical Ethernet port to an OVS bridge,
884 configure an IP address on a physical Ethernet port, and then issue
885 a "ping" to an address in that subnet, the same thing happens: an
886 ARP gets transmitted on the physical Ethernet port and Open vSwitch
887 never sees it. (You should not do that, as documented at the
888 beginning of this section.)
889
890 It can make sense to add a "tap" device to an Open vSwitch bridge,
891 if some userspace program (other than Open vSwitch) has opened the
892 tap socket. This is the case, for example, if the "tap" device was
893 created by KVM (or QEMU) to simulate a virtual NIC. In such a
894 case, when OVS bridges a packet to the "tap" device, the kernel
895 forwards that packet to KVM in userspace, which passes it along to
896 the VM, and in the other direction, when the VM sends a packet, KVM
897 writes it to the "tap" socket, which causes OVS to receive it and
898 bridge it to the other OVS ports. Please note that in such a case
899 no IP address is configured on the "tap" device (there is normally
900 an IP address configured in the virtual NIC inside the VM, but this
901 is not visible to the host Linux kernel or to Open vSwitch).
902
903 There is one special case in which Open vSwitch does directly read
904 and write "tap" sockets. This is an implementation detail of the
905 Open vSwitch userspace switch, which implements its "internal"
906 ports as Linux (or *BSD) "tap" sockets. In such a userspace
907 switch, OVS receives packets sent on the "tap" device used to
908 implement an "internal" port by reading the associated "tap"
909 socket, and bridges them to the rest of the switch. In the other
910 direction, OVS transmits packets bridged to the "internal" port by
911 writing them to the "tap" socket, causing them to be processed by
912 the kernel TCP/IP stack as if they had been received on the "tap"
913 device. Users should not need to be concerned with this
914 implementation detail.
915
916 Open vSwitch has a network device type called "tap". This is
917 intended only for implementing "internal" ports in the OVS
918 userspace switch and should not be used otherwise. In particular,
919 users should not configure KVM "tap" devices as type "tap" (use
920 type "system", the default, instead).
921
922
923 Quality of Service (QoS)
924 ------------------------
925
926 ### Q: How do I configure Quality of Service (QoS)?
927
928 A: Suppose that you want to set up bridge br0 connected to physical
929 Ethernet port eth0 (a 1 Gbps device) and virtual machine interfaces
930 vif1.0 and vif2.0, and that you want to limit traffic from vif1.0
931 to eth0 to 10 Mbps and from vif2.0 to eth0 to 20 Mbps. Then, you
932 could configure the bridge this way:
933
934 ovs-vsctl -- \
935 add-br br0 -- \
936 add-port br0 eth0 -- \
937 add-port br0 vif1.0 -- set interface vif1.0 ofport_request=5 -- \
938 add-port br0 vif2.0 -- set interface vif2.0 ofport_request=6 -- \
939 set port eth0 qos=@newqos -- \
940 --id=@newqos create qos type=linux-htb \
941 other-config:max-rate=1000000000 \
942 queues:123=@vif10queue \
943 queues:234=@vif20queue -- \
944 --id=@vif10queue create queue other-config:max-rate=10000000 -- \
945 --id=@vif20queue create queue other-config:max-rate=20000000
946
947 At this point, bridge br0 is configured with the ports and eth0 is
948 configured with the queues that you need for QoS, but nothing is
949 actually directing packets from vif1.0 or vif2.0 to the queues that
950 we have set up for them. That means that all of the packets to
951 eth0 are going to the "default queue", which is not what we want.
952
953 We use OpenFlow to direct packets from vif1.0 and vif2.0 to the
954 queues reserved for them:
955
956 ovs-ofctl add-flow br0 in_port=5,actions=set_queue:123,normal
957 ovs-ofctl add-flow br0 in_port=6,actions=set_queue:234,normal
958
959 Each of the above flows matches on the input port, sets up the
960 appropriate queue (123 for vif1.0, 234 for vif2.0), and then
961 executes the "normal" action, which performs the same switching
962 that Open vSwitch would have done without any OpenFlow flows being
963 present. (We know that vif1.0 and vif2.0 have OpenFlow port
964 numbers 5 and 6, respectively, because we set their ofport_request
965 columns above. If we had not done that, then we would have needed
966 to find out their port numbers before setting up these flows.)
967
968 Now traffic going from vif1.0 or vif2.0 to eth0 should be
969 rate-limited.
970
971 By the way, if you delete the bridge created by the above commands,
972 with:
973
974 ovs-vsctl del-br br0
975
976 then that will leave one unreferenced QoS record and two
977 unreferenced Queue records in the Open vSwich database. One way to
978 clear them out, assuming you don't have other QoS or Queue records
979 that you want to keep, is:
980
981 ovs-vsctl -- --all destroy QoS -- --all destroy Queue
982
983 If you do want to keep some QoS or Queue records, or the Open
984 vSwitch you are using is older than version 1.8 (which added the
985 --all option), then you will have to destroy QoS and Queue records
986 individually.
987
988 ### Q: I configured Quality of Service (QoS) in my OpenFlow network by
989 adding records to the QoS and Queue table, but the results aren't
990 what I expect.
991
992 A: Did you install OpenFlow flows that use your queues? This is the
993 primary way to tell Open vSwitch which queues you want to use. If
994 you don't do this, then the default queue will be used, which will
995 probably not have the effect you want.
996
997 Refer to the previous question for an example.
998
999 ### Q: I'd like to take advantage of some QoS feature that Open vSwitch
1000 doesn't yet support. How do I do that?
1001
1002 A: Open vSwitch does not implement QoS itself. Instead, it can
1003 configure some, but not all, of the QoS features built into the
1004 Linux kernel. If you need some QoS feature that OVS cannot
1005 configure itself, then the first step is to figure out whether
1006 Linux QoS supports that feature. If it does, then you can submit a
1007 patch to support Open vSwitch configuration for that feature, or
1008 you can use "tc" directly to configure the feature in Linux. (If
1009 Linux QoS doesn't support the feature you want, then first you have
1010 to add that support to Linux.)
1011
1012 ### Q: I configured QoS, correctly, but my measurements show that it isn't
1013 working as well as I expect.
1014
1015 A: With the Linux kernel, the Open vSwitch implementation of QoS has
1016 two aspects:
1017
1018 - Open vSwitch configures a subset of Linux kernel QoS
1019 features, according to what is in OVSDB. It is possible that
1020 this code has bugs. If you believe that this is so, then you
1021 can configure the Linux traffic control (QoS) stack directly
1022 with the "tc" program. If you get better results that way,
1023 you can send a detailed bug report to bugs@openvswitch.org.
1024
1025 It is certain that Open vSwitch cannot configure every Linux
1026 kernel QoS feature. If you need some feature that OVS cannot
1027 configure, then you can also use "tc" directly (or add that
1028 feature to OVS).
1029
1030 - The Open vSwitch implementation of OpenFlow allows flows to
1031 be directed to particular queues. This is pretty simple and
1032 unlikely to have serious bugs at this point.
1033
1034 However, most problems with QoS on Linux are not bugs in Open
1035 vSwitch at all. They tend to be either configuration errors
1036 (please see the earlier questions in this section) or issues with
1037 the traffic control (QoS) stack in Linux. The Open vSwitch
1038 developers are not experts on Linux traffic control. We suggest
1039 that, if you believe you are encountering a problem with Linux
1040 traffic control, that you consult the tc manpages (e.g. tc(8),
1041 tc-htb(8), tc-hfsc(8)), web resources (e.g. http://lartc.org/), or
1042 mailing lists (e.g. http://vger.kernel.org/vger-lists.html#netdev).
1043
1044 ### Q: Does Open vSwitch support OpenFlow meters?
1045
1046 A: Since version 2.0, Open vSwitch has OpenFlow protocol support for
1047 OpenFlow meters. There is no implementation of meters in the Open
1048 vSwitch software switch (neither the kernel-based nor userspace
1049 switches).
1050
1051
1052 VLANs
1053 -----
1054
1055 ### Q: What's a VLAN?
1056
1057 A: At the simplest level, a VLAN (short for "virtual LAN") is a way to
1058 partition a single switch into multiple switches. Suppose, for
1059 example, that you have two groups of machines, group A and group B.
1060 You want the machines in group A to be able to talk to each other,
1061 and you want the machine in group B to be able to talk to each
1062 other, but you don't want the machines in group A to be able to
1063 talk to the machines in group B. You can do this with two
1064 switches, by plugging the machines in group A into one switch and
1065 the machines in group B into the other switch.
1066
1067 If you only have one switch, then you can use VLANs to do the same
1068 thing, by configuring the ports for machines in group A as VLAN
1069 "access ports" for one VLAN and the ports for group B as "access
1070 ports" for a different VLAN. The switch will only forward packets
1071 between ports that are assigned to the same VLAN, so this
1072 effectively subdivides your single switch into two independent
1073 switches, one for each group of machines.
1074
1075 So far we haven't said anything about VLAN headers. With access
1076 ports, like we've described so far, no VLAN header is present in
1077 the Ethernet frame. This means that the machines (or switches)
1078 connected to access ports need not be aware that VLANs are
1079 involved, just like in the case where we use two different physical
1080 switches.
1081
1082 Now suppose that you have a whole bunch of switches in your
1083 network, instead of just one, and that some machines in group A are
1084 connected directly to both switches 1 and 2. To allow these
1085 machines to talk to each other, you could add an access port for
1086 group A's VLAN to switch 1 and another to switch 2, and then
1087 connect an Ethernet cable between those ports. That works fine,
1088 but it doesn't scale well as the number of switches and the number
1089 of VLANs increases, because you use up a lot of valuable switch
1090 ports just connecting together your VLANs.
1091
1092 This is where VLAN headers come in. Instead of using one cable and
1093 two ports per VLAN to connect a pair of switches, we configure a
1094 port on each switch as a VLAN "trunk port". Packets sent and
1095 received on a trunk port carry a VLAN header that says what VLAN
1096 the packet belongs to, so that only two ports total are required to
1097 connect the switches, regardless of the number of VLANs in use.
1098 Normally, only switches (either physical or virtual) are connected
1099 to a trunk port, not individual hosts, because individual hosts
1100 don't expect to see a VLAN header in the traffic that they receive.
1101
1102 None of the above discussion says anything about particular VLAN
1103 numbers. This is because VLAN numbers are completely arbitrary.
1104 One must only ensure that a given VLAN is numbered consistently
1105 throughout a network and that different VLANs are given different
1106 numbers. (That said, VLAN 0 is usually synonymous with a packet
1107 that has no VLAN header, and VLAN 4095 is reserved.)
1108
1109 ### Q: VLANs don't work.
1110
1111 A: Many drivers in Linux kernels before version 3.3 had VLAN-related
1112 bugs. If you are having problems with VLANs that you suspect to be
1113 driver related, then you have several options:
1114
1115 - Upgrade to Linux 3.3 or later.
1116
1117 - Build and install a fixed version of the particular driver
1118 that is causing trouble, if one is available.
1119
1120 - Use a NIC whose driver does not have VLAN problems.
1121
1122 - Use "VLAN splinters", a feature in Open vSwitch 1.4 and later
1123 that works around bugs in kernel drivers. To enable VLAN
1124 splinters on interface eth0, use the command:
1125
1126 ovs-vsctl set interface eth0 other-config:enable-vlan-splinters=true
1127
1128 For VLAN splinters to be effective, Open vSwitch must know
1129 which VLANs are in use. See the "VLAN splinters" section in
1130 the Interface table in ovs-vswitchd.conf.db(5) for details on
1131 how Open vSwitch infers in-use VLANs.
1132
1133 VLAN splinters increase memory use and reduce performance, so
1134 use them only if needed.
1135
1136 - Apply the "vlan workaround" patch from the XenServer kernel
1137 patch queue, build Open vSwitch against this patched kernel,
1138 and then use ovs-vlan-bug-workaround(8) to enable the VLAN
1139 workaround for each interface whose driver is buggy.
1140
1141 (This is a nontrivial exercise, so this option is included
1142 only for completeness.)
1143
1144 It is not always easy to tell whether a Linux kernel driver has
1145 buggy VLAN support. The ovs-vlan-test(8) and ovs-test(8) utilities
1146 can help you test. See their manpages for details. Of the two
1147 utilities, ovs-test(8) is newer and more thorough, but
1148 ovs-vlan-test(8) may be easier to use.
1149
1150 ### Q: VLANs still don't work. I've tested the driver so I know that it's OK.
1151
1152 A: Do you have VLANs enabled on the physical switch that OVS is
1153 attached to? Make sure that the port is configured to trunk the
1154 VLAN or VLANs that you are using with OVS.
1155
1156 ### Q: Outgoing VLAN-tagged traffic goes through OVS to my physical switch
1157 and to its destination host, but OVS seems to drop incoming return
1158 traffic.
1159
1160 A: It's possible that you have the VLAN configured on your physical
1161 switch as the "native" VLAN. In this mode, the switch treats
1162 incoming packets either tagged with the native VLAN or untagged as
1163 part of the native VLAN. It may also send outgoing packets in the
1164 native VLAN without a VLAN tag.
1165
1166 If this is the case, you have two choices:
1167
1168 - Change the physical switch port configuration to tag packets
1169 it forwards to OVS with the native VLAN instead of forwarding
1170 them untagged.
1171
1172 - Change the OVS configuration for the physical port to a
1173 native VLAN mode. For example, the following sets up a
1174 bridge with port eth0 in "native-tagged" mode in VLAN 9:
1175
1176 ovs-vsctl add-br br0
1177 ovs-vsctl add-port br0 eth0 tag=9 vlan_mode=native-tagged
1178
1179 In this situation, "native-untagged" mode will probably work
1180 equally well. Refer to the documentation for the Port table
1181 in ovs-vswitchd.conf.db(5) for more information.
1182
1183 ### Q: I added a pair of VMs on different VLANs, like this:
1184
1185 ovs-vsctl add-br br0
1186 ovs-vsctl add-port br0 eth0
1187 ovs-vsctl add-port br0 tap0 tag=9
1188 ovs-vsctl add-port br0 tap1 tag=10
1189
1190 but the VMs can't access each other, the external network, or the
1191 Internet.
1192
1193 A: It is to be expected that the VMs can't access each other. VLANs
1194 are a means to partition a network. When you configured tap0 and
1195 tap1 as access ports for different VLANs, you indicated that they
1196 should be isolated from each other.
1197
1198 As for the external network and the Internet, it seems likely that
1199 the machines you are trying to access are not on VLAN 9 (or 10) and
1200 that the Internet is not available on VLAN 9 (or 10).
1201
1202 ### Q: I added a pair of VMs on the same VLAN, like this:
1203
1204 ovs-vsctl add-br br0
1205 ovs-vsctl add-port br0 eth0
1206 ovs-vsctl add-port br0 tap0 tag=9
1207 ovs-vsctl add-port br0 tap1 tag=9
1208
1209 The VMs can access each other, but not the external network or the
1210 Internet.
1211
1212 A: It seems likely that the machines you are trying to access in the
1213 external network are not on VLAN 9 and that the Internet is not
1214 available on VLAN 9. Also, ensure VLAN 9 is set up as an allowed
1215 trunk VLAN on the upstream switch port to which eth0 is connected.
1216
1217 ### Q: Can I configure an IP address on a VLAN?
1218
1219 A: Yes. Use an "internal port" configured as an access port. For
1220 example, the following configures IP address 192.168.0.7 on VLAN 9.
1221 That is, OVS will forward packets from eth0 to 192.168.0.7 only if
1222 they have an 802.1Q header with VLAN 9. Conversely, traffic
1223 forwarded from 192.168.0.7 to eth0 will be tagged with an 802.1Q
1224 header with VLAN 9:
1225
1226 ovs-vsctl add-br br0
1227 ovs-vsctl add-port br0 eth0
1228 ovs-vsctl add-port br0 vlan9 tag=9 -- set interface vlan9 type=internal
1229 ifconfig vlan9 192.168.0.7
1230
1231 See also the following question.
1232
1233 ### Q: I configured one IP address on VLAN 0 and another on VLAN 9, like
1234 this:
1235
1236 ovs-vsctl add-br br0
1237 ovs-vsctl add-port br0 eth0
1238 ifconfig br0 192.168.0.5
1239 ovs-vsctl add-port br0 vlan9 tag=9 -- set interface vlan9 type=internal
1240 ifconfig vlan9 192.168.0.9
1241
1242 but other hosts that are only on VLAN 0 can reach the IP address
1243 configured on VLAN 9. What's going on?
1244
1245 A: RFC 1122 section 3.3.4.2 "Multihoming Requirements" describes two
1246 approaches to IP address handling in Internet hosts:
1247
1248 - In the "Strong ES Model", where an ES is a host ("End
1249 System"), an IP address is primarily associated with a
1250 particular interface. The host discards packets that arrive
1251 on interface A if they are destined for an IP address that is
1252 configured on interface B. The host never sends packets from
1253 interface A using a source address configured on interface B.
1254
1255 - In the "Weak ES Model", an IP address is primarily associated
1256 with a host. The host accepts packets that arrive on any
1257 interface if they are destined for any of the host's IP
1258 addresses, even if the address is configured on some
1259 interface other than the one on which it arrived. The host
1260 does not restrict itself to sending packets from an IP
1261 address associated with the originating interface.
1262
1263 Linux uses the weak ES model. That means that when packets
1264 destined to the VLAN 9 IP address arrive on eth0 and are bridged to
1265 br0, the kernel IP stack accepts them there for the VLAN 9 IP
1266 address, even though they were not received on vlan9, the network
1267 device for vlan9.
1268
1269 To simulate the strong ES model on Linux, one may add iptables rule
1270 to filter packets based on source and destination address and
1271 adjust ARP configuration with sysctls.
1272
1273 BSD uses the strong ES model.
1274
1275 ### Q: My OpenFlow controller doesn't see the VLANs that I expect.
1276
1277 A: The configuration for VLANs in the Open vSwitch database (e.g. via
1278 ovs-vsctl) only affects traffic that goes through Open vSwitch's
1279 implementation of the OpenFlow "normal switching" action. By
1280 default, when Open vSwitch isn't connected to a controller and
1281 nothing has been manually configured in the flow table, all traffic
1282 goes through the "normal switching" action. But, if you set up
1283 OpenFlow flows on your own, through a controller or using ovs-ofctl
1284 or through other means, then you have to implement VLAN handling
1285 yourself.
1286
1287 You can use "normal switching" as a component of your OpenFlow
1288 actions, e.g. by putting "normal" into the lists of actions on
1289 ovs-ofctl or by outputting to OFPP_NORMAL from an OpenFlow
1290 controller. In situations where this is not suitable, you can
1291 implement VLAN handling yourself, e.g.:
1292
1293 - If a packet comes in on an access port, and the flow table
1294 needs to send it out on a trunk port, then the flow can add
1295 the appropriate VLAN tag with the "mod_vlan_vid" action.
1296
1297 - If a packet comes in on a trunk port, and the flow table
1298 needs to send it out on an access port, then the flow can
1299 strip the VLAN tag with the "strip_vlan" action.
1300
1301 ### Q: I configured ports on a bridge as access ports with different VLAN
1302 tags, like this:
1303
1304 ovs-vsctl add-br br0
1305 ovs-vsctl set-controller br0 tcp:192.168.0.10:6653
1306 ovs-vsctl add-port br0 eth0
1307 ovs-vsctl add-port br0 tap0 tag=9
1308 ovs-vsctl add-port br0 tap1 tag=10
1309
1310 but the VMs running behind tap0 and tap1 can still communicate,
1311 that is, they are not isolated from each other even though they are
1312 on different VLANs.
1313
1314 A: Do you have a controller configured on br0 (as the commands above
1315 do)? If so, then this is a variant on the previous question, "My
1316 OpenFlow controller doesn't see the VLANs that I expect," and you
1317 can refer to the answer there for more information.
1318
1319 ### Q: How MAC learning works with VLANs?
1320
1321 A: Open vSwitch implements Independent VLAN Learning (IVL) for
1322 OFPP_NORMAL action. I.e. it logically has separate learning tables
1323 for each VLANs.
1324
1325
1326 VXLANs
1327 -----
1328
1329 ### Q: What's a VXLAN?
1330
1331 A: VXLAN stands for Virtual eXtensible Local Area Network, and is a means
1332 to solve the scaling challenges of VLAN networks in a multi-tenant
1333 environment. VXLAN is an overlay network which transports an L2 network
1334 over an existing L3 network. For more information on VXLAN, please see
1335 RFC 7348:
1336
1337 http://tools.ietf.org/html/rfc7348
1338
1339 ### Q: How much of the VXLAN protocol does Open vSwitch currently support?
1340
1341 A: Open vSwitch currently supports the framing format for packets on the
1342 wire. There is currently no support for the multicast aspects of VXLAN.
1343 To get around the lack of multicast support, it is possible to
1344 pre-provision MAC to IP address mappings either manually or from a
1345 controller.
1346
1347 ### Q: What destination UDP port does the VXLAN implementation in Open vSwitch
1348 use?
1349
1350 A: By default, Open vSwitch will use the assigned IANA port for VXLAN, which
1351 is 4789. However, it is possible to configure the destination UDP port
1352 manually on a per-VXLAN tunnel basis. An example of this configuration is
1353 provided below.
1354
1355 ovs-vsctl add-br br0
1356 ovs-vsctl add-port br0 vxlan1 -- set interface vxlan1
1357 type=vxlan options:remote_ip=192.168.1.2 options:key=flow
1358 options:dst_port=8472
1359
1360
1361 Using OpenFlow (Manually or Via Controller)
1362 -------------------------------------------
1363
1364 ### Q: What versions of OpenFlow does Open vSwitch support?
1365
1366 A: The following table lists the versions of OpenFlow supported by
1367 each version of Open vSwitch:
1368
1369 Open vSwitch OF1.0 OF1.1 OF1.2 OF1.3 OF1.4 OF1.5
1370 ###============ ===== ===== ===== ===== ===== =====
1371 1.9 and earlier yes --- --- --- --- ---
1372 1.10 yes --- [*] [*] --- ---
1373 1.11 yes --- [*] [*] --- ---
1374 2.0 yes [*] [*] [*] --- ---
1375 2.1 yes [*] [*] [*] --- ---
1376 2.2 yes [*] [*] [*] [%] [*]
1377 2.3 yes yes yes yes [*] [*]
1378
1379 [*] Supported, with one or more missing features.
1380 [%] Experimental, unsafe implementation.
1381
1382 Open vSwitch 2.3 enables OpenFlow 1.0, 1.1, 1.2, and 1.3 by default
1383 in ovs-vswitchd. In Open vSwitch 1.10 through 2.2, OpenFlow 1.1,
1384 1.2, and 1.3 must be enabled manually in ovs-vswitchd. OpenFlow
1385 1.4 and 1.5 are also supported, with missing features, in Open
1386 vSwitch 2.3 and later, but not enabled by default. In any case,
1387 the user may override the default:
1388
1389 - To enable OpenFlow 1.0, 1.1, 1.2, and 1.3 on bridge br0:
1390
1391 ovs-vsctl set bridge br0 protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13
1392
1393 - To enable OpenFlow 1.0, 1.1, 1.2, 1.3, 1.4, and 1.5 on bridge br0:
1394
1395 ovs-vsctl set bridge br0 protocols=OpenFlow10,OpenFlow11,OpenFlow12,OpenFlow13,OpenFlow14,OpenFlow15
1396
1397 - To enable only OpenFlow 1.0 on bridge br0:
1398
1399 ovs-vsctl set bridge br0 protocols=OpenFlow10
1400
1401 All current versions of ovs-ofctl enable only OpenFlow 1.0 by
1402 default. Use the -O option to enable support for later versions of
1403 OpenFlow in ovs-ofctl. For example:
1404
1405 ovs-ofctl -O OpenFlow13 dump-flows br0
1406
1407 (Open vSwitch 2.2 had an experimental implementation of OpenFlow
1408 1.4 that could cause crashes. We don't recommend enabling it.)
1409
1410 [OPENFLOW-1.1+.md] in the Open vSwitch source tree tracks support for
1411 OpenFlow 1.1 and later features. When support for OpenFlow 1.4 and
1412 1.5 is solidly implemented, Open vSwitch will enable those version
1413 by default. Also, the OpenFlow 1.5 specification is still under
1414 development and thus subject to change.
1415
1416 ### Q: Does Open vSwitch support MPLS?
1417
1418 A: Before version 1.11, Open vSwitch did not support MPLS. That is,
1419 these versions can match on MPLS Ethernet types, but they cannot
1420 match, push, or pop MPLS labels, nor can they look past MPLS labels
1421 into the encapsulated packet.
1422
1423 Open vSwitch versions 1.11, 2.0, and 2.1 have very minimal support
1424 for MPLS. With the userspace datapath only, these versions can
1425 match, push, or pop a single MPLS label, but they still cannot look
1426 past MPLS labels (even after popping them) into the encapsulated
1427 packet. Kernel datapath support is unchanged from earlier
1428 versions.
1429
1430 Open vSwitch version 2.3 can match, push, or pop a single MPLS
1431 label and look past the MPLS label into the encapsulated packet.
1432 Both userspace and kernel datapaths will be supported, but MPLS
1433 processing always happens in userspace either way, so kernel
1434 datapath performance will be disappointing.
1435
1436 Open vSwitch version 2.4 can match, push, or pop up to 3 MPLS
1437 labels and look past the MPLS label into the encapsulated packet.
1438 It will have kernel support for MPLS, yielding improved
1439 performance.
1440
1441 ### Q: I'm getting "error type 45250 code 0". What's that?
1442
1443 A: This is a Open vSwitch extension to OpenFlow error codes. Open
1444 vSwitch uses this extension when it must report an error to an
1445 OpenFlow controller but no standard OpenFlow error code is
1446 suitable.
1447
1448 Open vSwitch logs the errors that it sends to controllers, so the
1449 easiest thing to do is probably to look at the ovs-vswitchd log to
1450 find out what the error was.
1451
1452 If you want to dissect the extended error message yourself, the
1453 format is documented in include/openflow/nicira-ext.h in the Open
1454 vSwitch source distribution. The extended error codes are
1455 documented in lib/ofp-errors.h.
1456
1457 Q1: Some of the traffic that I'd expect my OpenFlow controller to see
1458 doesn't actually appear through the OpenFlow connection, even
1459 though I know that it's going through.
1460 Q2: Some of the OpenFlow flows that my controller sets up don't seem
1461 to apply to certain traffic, especially traffic between OVS and
1462 the controller itself.
1463
1464 A: By default, Open vSwitch assumes that OpenFlow controllers are
1465 connected "in-band", that is, that the controllers are actually
1466 part of the network that is being controlled. In in-band mode,
1467 Open vSwitch sets up special "hidden" flows to make sure that
1468 traffic can make it back and forth between OVS and the controllers.
1469 These hidden flows are higher priority than any flows that can be
1470 set up through OpenFlow, and they are not visible through normal
1471 OpenFlow flow table dumps.
1472
1473 Usually, the hidden flows are desirable and helpful, but
1474 occasionally they can cause unexpected behavior. You can view the
1475 full OpenFlow flow table, including hidden flows, on bridge br0
1476 with the command:
1477
1478 ovs-appctl bridge/dump-flows br0
1479
1480 to help you debug. The hidden flows are those with priorities
1481 greater than 65535 (the maximum priority that can be set with
1482 OpenFlow).
1483
1484 The DESIGN file at the top level of the Open vSwitch source
1485 distribution describes the in-band model in detail.
1486
1487 If your controllers are not actually in-band (e.g. they are on
1488 localhost via 127.0.0.1, or on a separate network), then you should
1489 configure your controllers in "out-of-band" mode. If you have one
1490 controller on bridge br0, then you can configure out-of-band mode
1491 on it with:
1492
1493 ovs-vsctl set controller br0 connection-mode=out-of-band
1494
1495 ### Q: I configured all my controllers for out-of-band control mode but
1496 "ovs-appctl bridge/dump-flows" still shows some hidden flows.
1497
1498 A: You probably have a remote manager configured (e.g. with "ovs-vsctl
1499 set-manager"). By default, Open vSwitch assumes that managers need
1500 in-band rules set up on every bridge. You can disable these rules
1501 on bridge br0 with:
1502
1503 ovs-vsctl set bridge br0 other-config:disable-in-band=true
1504
1505 This actually disables in-band control entirely for the bridge, as
1506 if all the bridge's controllers were configured for out-of-band
1507 control.
1508
1509 ### Q: My OpenFlow controller doesn't see the VLANs that I expect.
1510
1511 A: See answer under "VLANs", above.
1512
1513 ### Q: I ran "ovs-ofctl add-flow br0 nw_dst=192.168.0.1,actions=drop"
1514 but I got a funny message like this:
1515
1516 ofp_util|INFO|normalization changed ofp_match, details:
1517 ofp_util|INFO| pre: nw_dst=192.168.0.1
1518 ofp_util|INFO|post:
1519
1520 and when I ran "ovs-ofctl dump-flows br0" I saw that my nw_dst
1521 match had disappeared, so that the flow ends up matching every
1522 packet.
1523
1524 A: The term "normalization" in the log message means that a flow
1525 cannot match on an L3 field without saying what L3 protocol is in
1526 use. The "ovs-ofctl" command above didn't specify an L3 protocol,
1527 so the L3 field match was dropped.
1528
1529 In this case, the L3 protocol could be IP or ARP. A correct
1530 command for each possibility is, respectively:
1531
1532 ovs-ofctl add-flow br0 ip,nw_dst=192.168.0.1,actions=drop
1533
1534 and
1535
1536 ovs-ofctl add-flow br0 arp,nw_dst=192.168.0.1,actions=drop
1537
1538 Similarly, a flow cannot match on an L4 field without saying what
1539 L4 protocol is in use. For example, the flow match "tp_src=1234"
1540 is, by itself, meaningless and will be ignored. Instead, to match
1541 TCP source port 1234, write "tcp,tp_src=1234", or to match UDP
1542 source port 1234, write "udp,tp_src=1234".
1543
1544 ### Q: How can I figure out the OpenFlow port number for a given port?
1545
1546 A: The OFPT_FEATURES_REQUEST message requests an OpenFlow switch to
1547 respond with an OFPT_FEATURES_REPLY that, among other information,
1548 includes a mapping between OpenFlow port names and numbers. From a
1549 command prompt, "ovs-ofctl show br0" makes such a request and
1550 prints the response for switch br0.
1551
1552 The Interface table in the Open vSwitch database also maps OpenFlow
1553 port names to numbers. To print the OpenFlow port number
1554 associated with interface eth0, run:
1555
1556 ovs-vsctl get Interface eth0 ofport
1557
1558 You can print the entire mapping with:
1559
1560 ovs-vsctl -- --columns=name,ofport list Interface
1561
1562 but the output mixes together interfaces from all bridges in the
1563 database, so it may be confusing if more than one bridge exists.
1564
1565 In the Open vSwitch database, ofport value -1 means that the
1566 interface could not be created due to an error. (The Open vSwitch
1567 log should indicate the reason.) ofport value [] (the empty set)
1568 means that the interface hasn't been created yet. The latter is
1569 normally an intermittent condition (unless ovs-vswitchd is not
1570 running).
1571
1572 ### Q: I added some flows with my controller or with ovs-ofctl, but when I
1573 run "ovs-dpctl dump-flows" I don't see them.
1574
1575 A: ovs-dpctl queries a kernel datapath, not an OpenFlow switch. It
1576 won't display the information that you want. You want to use
1577 "ovs-ofctl dump-flows" instead.
1578
1579 ### Q: It looks like each of the interfaces in my bonded port shows up
1580 as an individual OpenFlow port. Is that right?
1581
1582 A: Yes, Open vSwitch makes individual bond interfaces visible as
1583 OpenFlow ports, rather than the bond as a whole. The interfaces
1584 are treated together as a bond for only a few purposes:
1585
1586 - Sending a packet to the OFPP_NORMAL port. (When an OpenFlow
1587 controller is not configured, this happens implicitly to
1588 every packet.)
1589
1590 - Mirrors configured for output to a bonded port.
1591
1592 It would make a lot of sense for Open vSwitch to present a bond as
1593 a single OpenFlow port. If you want to contribute an
1594 implementation of such a feature, please bring it up on the Open
1595 vSwitch development mailing list at dev@openvswitch.org.
1596
1597 ### Q: I have a sophisticated network setup involving Open vSwitch, VMs or
1598 multiple hosts, and other components. The behavior isn't what I
1599 expect. Help!
1600
1601 A: To debug network behavior problems, trace the path of a packet,
1602 hop-by-hop, from its origin in one host to a remote host. If
1603 that's correct, then trace the path of the response packet back to
1604 the origin.
1605
1606 Usually a simple ICMP echo request and reply ("ping") packet is
1607 good enough. Start by initiating an ongoing "ping" from the origin
1608 host to a remote host. If you are tracking down a connectivity
1609 problem, the "ping" will not display any successful output, but
1610 packets are still being sent. (In this case the packets being sent
1611 are likely ARP rather than ICMP.)
1612
1613 Tools available for tracing include the following:
1614
1615 - "tcpdump" and "wireshark" for observing hops across network
1616 devices, such as Open vSwitch internal devices and physical
1617 wires.
1618
1619 - "ovs-appctl dpif/dump-flows <br>" in Open vSwitch 1.10 and
1620 later or "ovs-dpctl dump-flows <br>" in earlier versions.
1621 These tools allow one to observe the actions being taken on
1622 packets in ongoing flows.
1623
1624 See ovs-vswitchd(8) for "ovs-appctl dpif/dump-flows"
1625 documentation, ovs-dpctl(8) for "ovs-dpctl dump-flows"
1626 documentation, and "Why are there so many different ways to
1627 dump flows?" above for some background.
1628
1629 - "ovs-appctl ofproto/trace" to observe the logic behind how
1630 ovs-vswitchd treats packets. See ovs-vswitchd(8) for
1631 documentation. You can out more details about a given flow
1632 that "ovs-dpctl dump-flows" displays, by cutting and pasting
1633 a flow from the output into an "ovs-appctl ofproto/trace"
1634 command.
1635
1636 - SPAN, RSPAN, and ERSPAN features of physical switches, to
1637 observe what goes on at these physical hops.
1638
1639 Starting at the origin of a given packet, observe the packet at
1640 each hop in turn. For example, in one plausible scenario, you
1641 might:
1642
1643 1. "tcpdump" the "eth" interface through which an ARP egresses
1644 a VM, from inside the VM.
1645
1646 2. "tcpdump" the "vif" or "tap" interface through which the ARP
1647 ingresses the host machine.
1648
1649 3. Use "ovs-dpctl dump-flows" to spot the ARP flow and observe
1650 the host interface through which the ARP egresses the
1651 physical machine. You may need to use "ovs-dpctl show" to
1652 interpret the port numbers. If the output seems surprising,
1653 you can use "ovs-appctl ofproto/trace" to observe details of
1654 how ovs-vswitchd determined the actions in the "ovs-dpctl
1655 dump-flows" output.
1656
1657 4. "tcpdump" the "eth" interface through which the ARP egresses
1658 the physical machine.
1659
1660 5. "tcpdump" the "eth" interface through which the ARP
1661 ingresses the physical machine, at the remote host that
1662 receives the ARP.
1663
1664 6. Use "ovs-dpctl dump-flows" to spot the ARP flow on the
1665 remote host that receives the ARP and observe the VM "vif"
1666 or "tap" interface to which the flow is directed. Again,
1667 "ovs-dpctl show" and "ovs-appctl ofproto/trace" might help.
1668
1669 7. "tcpdump" the "vif" or "tap" interface to which the ARP is
1670 directed.
1671
1672 8. "tcpdump" the "eth" interface through which the ARP
1673 ingresses a VM, from inside the VM.
1674
1675 It is likely that during one of these steps you will figure out the
1676 problem. If not, then follow the ARP reply back to the origin, in
1677 reverse.
1678
1679 ### Q: How do I make a flow drop packets?
1680
1681 A: To drop a packet is to receive it without forwarding it. OpenFlow
1682 explicitly specifies forwarding actions. Thus, a flow with an
1683 empty set of actions does not forward packets anywhere, causing
1684 them to be dropped. You can specify an empty set of actions with
1685 "actions=" on the ovs-ofctl command line. For example:
1686
1687 ovs-ofctl add-flow br0 priority=65535,actions=
1688
1689 would cause every packet entering switch br0 to be dropped.
1690
1691 You can write "drop" explicitly if you like. The effect is the
1692 same. Thus, the following command also causes every packet
1693 entering switch br0 to be dropped:
1694
1695 ovs-ofctl add-flow br0 priority=65535,actions=drop
1696
1697 "drop" is not an action, either in OpenFlow or Open vSwitch.
1698 Rather, it is only a way to say that there are no actions.
1699
1700 ### Q: I added a flow to send packets out the ingress port, like this:
1701
1702 ovs-ofctl add-flow br0 in_port=2,actions=2
1703
1704 but OVS drops the packets instead.
1705
1706 A: Yes, OpenFlow requires a switch to ignore attempts to send a packet
1707 out its ingress port. The rationale is that dropping these packets
1708 makes it harder to loop the network. Sometimes this behavior can
1709 even be convenient, e.g. it is often the desired behavior in a flow
1710 that forwards a packet to several ports ("floods" the packet).
1711
1712 Sometimes one really needs to send a packet out its ingress port
1713 ("hairpin"). In this case, output to OFPP_IN_PORT, which in
1714 ovs-ofctl syntax is expressed as just "in_port", e.g.:
1715
1716 ovs-ofctl add-flow br0 in_port=2,actions=in_port
1717
1718 This also works in some circumstances where the flow doesn't match
1719 on the input port. For example, if you know that your switch has
1720 five ports numbered 2 through 6, then the following will send every
1721 received packet out every port, even its ingress port:
1722
1723 ovs-ofctl add-flow br0 actions=2,3,4,5,6,in_port
1724
1725 or, equivalently:
1726
1727 ovs-ofctl add-flow br0 actions=all,in_port
1728
1729 Sometimes, in complicated flow tables with multiple levels of
1730 "resubmit" actions, a flow needs to output to a particular port
1731 that may or may not be the ingress port. It's difficult to take
1732 advantage of OFPP_IN_PORT in this situation. To help, Open vSwitch
1733 provides, as an OpenFlow extension, the ability to modify the
1734 in_port field. Whatever value is currently in the in_port field is
1735 the port to which outputs will be dropped, as well as the
1736 destination for OFPP_IN_PORT. This means that the following will
1737 reliably output to port 2 or to ports 2 through 6, respectively:
1738
1739 ovs-ofctl add-flow br0 in_port=2,actions=load:0->NXM_OF_IN_PORT[],2
1740 ovs-ofctl add-flow br0 actions=load:0->NXM_OF_IN_PORT[],2,3,4,5,6
1741
1742 If the input port is important, then one may save and restore it on
1743 the stack:
1744
1745 ovs-ofctl add-flow br0 actions=push:NXM_OF_IN_PORT[],\
1746 load:0->NXM_OF_IN_PORT[],\
1747 2,3,4,5,6,\
1748 pop:NXM_OF_IN_PORT[]
1749
1750 ### Q: My bridge br0 has host 192.168.0.1 on port 1 and host 192.168.0.2
1751 on port 2. I set up flows to forward only traffic destined to the
1752 other host and drop other traffic, like this:
1753
1754 priority=5,in_port=1,ip,nw_dst=192.168.0.2,actions=2
1755 priority=5,in_port=2,ip,nw_dst=192.168.0.1,actions=1
1756 priority=0,actions=drop
1757
1758 But it doesn't work--I don't get any connectivity when I do this.
1759 Why?
1760
1761 A: These flows drop the ARP packets that IP hosts use to establish IP
1762 connectivity over Ethernet. To solve the problem, add flows to
1763 allow ARP to pass between the hosts:
1764
1765 priority=5,in_port=1,arp,actions=2
1766 priority=5,in_port=2,arp,actions=1
1767
1768 This issue can manifest other ways, too. The following flows that
1769 match on Ethernet addresses instead of IP addresses will also drop
1770 ARP packets, because ARP requests are broadcast instead of being
1771 directed to a specific host:
1772
1773 priority=5,in_port=1,dl_dst=54:00:00:00:00:02,actions=2
1774 priority=5,in_port=2,dl_dst=54:00:00:00:00:01,actions=1
1775 priority=0,actions=drop
1776
1777 The solution already described above will also work in this case.
1778 It may be better to add flows to allow all multicast and broadcast
1779 traffic:
1780
1781 priority=5,in_port=1,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,actions=2
1782 priority=5,in_port=2,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,actions=1
1783
1784 ### Q: My bridge disconnects from my controller on add-port/del-port.
1785
1786 A: Reconfiguring your bridge can change your bridge's datapath-id because
1787 Open vSwitch generates datapath-id from the MAC address of one of its ports.
1788 In that case, Open vSwitch disconnects from controllers because there's
1789 no graceful way to notify controllers about the change of datapath-id.
1790
1791 To avoid the behaviour, you can configure datapath-id manually.
1792
1793 ovs-vsctl set bridge br0 other-config:datapath-id=0123456789abcdef
1794
1795 ### Q: My controller is getting errors about "buffers". What's going on?
1796
1797 A: When a switch sends a packet to an OpenFlow controller using a
1798 "packet-in" message, it can also keep a copy of that packet in a
1799 "buffer", identified by a 32-bit integer "buffer_id". There are
1800 two advantages to buffering. First, when the controller wants to
1801 tell the switch to do something with the buffered packet (with a
1802 "packet-out" OpenFlow request), it does not need to send another
1803 copy of the packet back across the OpenFlow connection, which
1804 reduces the bandwidth cost of the connection and improves latency.
1805 This enables the second advantage: the switch can optionally send
1806 only the first part of the packet to the controller (assuming that
1807 the switch only needs to look at the first few bytes of the
1808 packet), further reducing bandwidth and improving latency.
1809
1810 However, buffering introduces some issues of its own. First, any
1811 switch has limited resources, so if the controller does not use a
1812 buffered packet, the switch has to decide how long to keep it
1813 buffered. When many packets are sent to a controller and buffered,
1814 Open vSwitch can discard buffered packets that the controller has
1815 not used after as little as 5 seconds. This means that
1816 controllers, if they make use of packet buffering, should use the
1817 buffered packets promptly. (This includes sending a "packet-out"
1818 with no actions if the controller does not want to do anything with
1819 a buffered packet, to clear the packet buffer and effectively
1820 "drop" its packet.)
1821
1822 Second, packet buffers are one-time-use, meaning that a controller
1823 cannot use a single packet buffer in two or more "packet-out"
1824 commands. Open vSwitch will respond with an error to the second
1825 and subsequent "packet-out"s in such a case.
1826
1827 Finally, a common error early in controller development is to try
1828 to use buffer_id 0 in a "packet-out" message as if 0 represented
1829 "no buffered packet". This is incorrect usage: the buffer_id with
1830 this meaning is actually 0xffffffff.
1831
1832 ovs-vswitchd(8) describes some details of Open vSwitch packet
1833 buffering that the OpenFlow specification requires implementations
1834 to document.
1835
1836
1837 Development
1838 -----------
1839
1840 ### Q: How do I implement a new OpenFlow message?
1841
1842 A: Add your new message to "enum ofpraw" and "enum ofptype" in
1843 lib/ofp-msgs.h, following the existing pattern. Then recompile and
1844 fix all of the new warnings, implementing new functionality for the
1845 new message as needed. (If you configure with --enable-Werror, as
1846 described in [INSTALL.md], then it is impossible to miss any warnings.)
1847
1848 If you need to add an OpenFlow vendor extension message for a
1849 vendor that doesn't yet have any extension messages, then you will
1850 also need to edit build-aux/extract-ofp-msgs.
1851
1852 ### Q: How do I add support for a new field or header?
1853
1854 A: Add new members for your field to "struct flow" in lib/flow.h, and
1855 add new enumerations for your new field to "enum mf_field_id" in
1856 lib/meta-flow.h, following the existing pattern. Also, add support
1857 to miniflow_extract() in lib/flow.c for extracting your new field
1858 from a packet into struct miniflow. Then recompile and fix all of
1859 the new warnings, implementing new functionality for the new field
1860 or header as needed. (If you configure with --enable-Werror, as
1861 described in [INSTALL.md], then it is impossible to miss any
1862 warnings.)
1863
1864 If you want kernel datapath support for your new field, you also
1865 need to modify the kernel module for the operating systems you are
1866 interested in. This isn't mandatory, since fields understood only
1867 by userspace work too (with a performance penalty), so it's
1868 reasonable to start development without it. If you implement
1869 kernel module support for Linux, then the Linux kernel "netdev"
1870 mailing list is the place to submit that support first; please read
1871 up on the Linux kernel development process separately. The Windows
1872 datapath kernel module support, on the other hand, is maintained
1873 within the OVS tree, so patches for that can go directly to
1874 ovs-dev.
1875
1876 ### Q: How do I add support for a new OpenFlow action?
1877
1878 A: Add your new action to "enum ofp_raw_action_type" in
1879 lib/ofp-actions.c, following the existing pattern. Then recompile
1880 and fix all of the new warnings, implementing new functionality for
1881 the new action as needed. (If you configure with --enable-Werror,
1882 as described in [INSTALL.md], then it is impossible to miss any
1883 warnings.)
1884
1885 If you need to add an OpenFlow vendor extension action for a vendor
1886 that doesn't yet have any extension actions, then you will also
1887 need to edit build-aux/extract-ofp-actions.
1888
1889
1890 Contact
1891 -------
1892
1893 bugs@openvswitch.org
1894 http://openvswitch.org/
1895
1896 [PORTING.md]:PORTING.md
1897 [WHY-OVS.md]:WHY-OVS.md
1898 [INSTALL.md]:INSTALL.md
1899 [OPENFLOW-1.1+.md]:OPENFLOW-1.1+.md