]>
Commit | Line | Data |
---|---|---|
2567fb84 SF |
1 | .. |
2 | Licensed under the Apache License, Version 2.0 (the "License"); you may | |
3 | not use this file except in compliance with the License. You may obtain | |
4 | a copy of the License at | |
5 | ||
6 | http://www.apache.org/licenses/LICENSE-2.0 | |
7 | ||
8 | Unless required by applicable law or agreed to in writing, software | |
9 | distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | |
10 | WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | |
11 | License for the specific language governing permissions and limitations | |
12 | under the License. | |
13 | ||
14 | Convention for heading levels in Open vSwitch documentation: | |
15 | ||
16 | ======= Heading 0 (reserved for the title in a document) | |
17 | ------- Heading 1 | |
18 | ~~~~~~~ Heading 2 | |
19 | +++++++ Heading 3 | |
20 | ''''''' Heading 4 | |
21 | ||
22 | Avoid deeper levels because they do not render well. | |
23 | ||
24 | ========================================= | |
25 | Integration Guide for Centralized Control | |
26 | ========================================= | |
27 | ||
28 | This document describes how to integrate Open vSwitch onto a new platform to | |
29 | expose the state of the switch and attached devices for centralized control. | |
30 | (If you are looking to port the switching components of Open vSwitch to a new | |
7c9afefd SF |
31 | platform, refer to :doc:`porting`) The focus of this guide is on hypervisors, |
32 | but many of the interfaces are useful for hardware switches, as well. The | |
33 | XenServer integration is the most mature implementation, so most of the | |
34 | examples are drawn from it. | |
2567fb84 SF |
35 | |
36 | The externally visible interface to this integration is platform-agnostic. We | |
37 | encourage anyone who integrates Open vSwitch to use the same interface, because | |
38 | keeping a uniform interface means that controllers require less customization | |
39 | for individual platforms (and perhaps no customization at all). | |
40 | ||
41 | Integration centers around the Open vSwitch database and mostly involves the | |
42 | ``external_ids`` columns in several of the tables. These columns are not | |
43 | interpreted by Open vSwitch itself. Instead, they provide information to a | |
44 | controller that permits it to associate a database record with a more | |
45 | meaningful entity. In contrast, the ``other_config`` column is used to | |
46 | configure behavior of the switch. The main job of the integrator, then, is to | |
47 | ensure that these values are correctly populated and maintained. | |
48 | ||
49 | An integrator sets the columns in the database by talking to the ovsdb-server | |
50 | daemon. A few of the columns can be set during startup by calling the ovs-ctl | |
51 | tool from inside the startup scripts. The ``xenserver/etc_init.d_openvswitch`` | |
52 | script provides examples of its use, and the ovs-ctl(8) manpage contains | |
53 | complete documentation. At runtime, ovs-vsctl can be be used to set columns in | |
54 | the database. The script ``xenserver/etc_xensource_scripts_vif`` contains | |
55 | examples of its use, and ovs-vsctl(8) manpage contains complete documentation. | |
56 | ||
57 | Python and C bindings to the database are provided if deeper integration with a | |
58 | program are needed. The XenServer ovs-xapi-sync daemon | |
59 | (``xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync``) provides an example | |
60 | of using the Python bindings. More information on the python bindings is | |
61 | available at ``python/ovs/db/idl.py``. Information on the C bindings is | |
62 | available at ``lib/ovsdb-idl.h``. | |
63 | ||
64 | The following diagram shows how integration scripts fit into the Open vSwitch | |
65 | architecture: | |
66 | ||
67 | :: | |
68 | ||
69 | Diagram | |
70 | ||
71 | +----------------------------------------+ | |
72 | | Controller Cluster + | |
73 | +----------------------------------------+ | |
74 | | | |
75 | | | |
76 | +----------------------------------------------------------+ | |
77 | | | | | |
78 | | +--------------+---------------+ | | |
79 | | | | | | |
80 | | +-------------------+ +------------------+ | | |
81 | | | ovsdb-server |-----------| ovs-vswitchd | | | |
82 | | +-------------------+ +------------------+ | | |
83 | | | | | | |
84 | | +---------------------+ | | | |
85 | | | Integration scripts | | | | |
86 | | | (ex: ovs-xapi-sync) | | | | |
87 | | +---------------------+ | | | |
88 | | | Userspace | | |
89 | |----------------------------------------------------------| | |
90 | | | Kernel | | |
91 | | | | | |
92 | | +---------------------+ | | |
93 | | | OVS Kernel Module | | | |
94 | | +---------------------+ | | |
95 | +----------------------------------------------------------+ | |
96 | ||
97 | A description of the most relevant fields for integration follows. By setting | |
98 | these values, controllers are able to understand the network and manage it more | |
99 | dynamically and precisely. For more details about the database and each | |
100 | individual column, please refer to the ovs-vswitchd.conf.db(5) manpage. | |
101 | ||
102 | ``Open_vSwitch`` table | |
103 | ---------------------- | |
104 | ||
105 | The ``Open_vSwitch`` table describes the switch as a whole. The | |
106 | ``system_type`` and ``system_version`` columns identify the platform to the | |
107 | controller. The ``external_ids:system-id`` key uniquely identifies the | |
108 | physical host. In XenServer, the system-id will likely be the same as the UUID | |
109 | returned by ``xe host-list``. This key allows controllers to distinguish | |
110 | between multiple hypervisors. | |
111 | ||
112 | Most of this configuration can be done with the ovs-ctl command at startup. | |
113 | For example: | |
114 | ||
115 | :: | |
116 | ||
117 | $ ovs-ctl --system-type="XenServer" --system-version="6.0.0-50762p" \ | |
118 | --system-id="${UUID}" "${other_options}" start | |
119 | ||
120 | Alternatively, the ovs-vsctl command may be used to set a particular value at | |
121 | runtime. For example: | |
122 | ||
123 | :: | |
124 | ||
125 | $ ovs-vsctl set open_vswitch . external-ids:system-id='"${UUID}"' | |
126 | ||
127 | The ``other_config:enable-statistics`` key may be set to ``true`` to have OVS | |
128 | populate the database with statistics (e.g., number of CPUs, memory, system | |
129 | load) for the controller's use. | |
130 | ||
131 | Bridge table | |
132 | ------------ | |
133 | ||
134 | The Bridge table describes individual bridges within an Open vSwitch instance. | |
135 | The ``external-ids:bridge-id`` key uniquely identifies a particular bridge. In | |
136 | XenServer, this will likely be the same as the UUID returned by ``xe | |
137 | network-list`` for that particular bridge. | |
138 | ||
139 | For example, to set the identifier for bridge "br0", the following command can | |
140 | be used: | |
141 | ||
142 | :: | |
143 | ||
144 | $ ovs-vsctl set Bridge br0 external-ids:bridge-id='"${UUID}"' | |
145 | ||
146 | The MAC address of the bridge may be manually configured by setting it with the | |
147 | ``other_config:hwaddr`` key. For example: | |
148 | ||
149 | :: | |
150 | ||
151 | $ ovs-vsctl set Bridge br0 other_config:hwaddr="12:34:56:78:90:ab" | |
152 | ||
153 | Interface table | |
154 | --------------- | |
155 | ||
156 | The Interface table describes an interface under the control of Open vSwitch. | |
157 | The ``external_ids`` column contains keys that are used to provide additional | |
158 | information about the interface: | |
159 | ||
160 | attached-mac | |
161 | ||
162 | This field contains the MAC address of the device attached to the interface. | |
163 | On a hypervisor, this is the MAC address of the interface as seen inside a | |
164 | VM. It does not necessarily correlate to the host-side MAC address. For | |
165 | example, on XenServer, the MAC address on a VIF in the hypervisor is always | |
166 | FE:FF:FF:FF:FF:FF, but inside the VM a normal MAC address is seen. | |
167 | ||
168 | iface-id | |
169 | ||
170 | This field uniquely identifies the interface. In hypervisors, this allows | |
171 | the controller to follow VM network interfaces as VMs migrate. A well-chosen | |
172 | identifier should also allow an administrator or a controller to associate | |
173 | the interface with the corresponding object in the VM management system. For | |
174 | example, the Open vSwitch integration with XenServer by default uses the | |
175 | XenServer assigned UUID for a VIF record as the iface-id. | |
176 | ||
177 | iface-status | |
178 | ||
179 | In a hypervisor, there are situations where there are multiple interface | |
180 | choices for a single virtual ethernet interface inside a VM. Valid values | |
181 | are "active" and "inactive". A complete description is available in the | |
182 | ovs-vswitchd.conf.db(5) manpage. | |
183 | ||
184 | vm-id | |
185 | ||
186 | This field uniquely identifies the VM to which this interface belongs. A | |
187 | single VM may have multiple interfaces attached to it. | |
188 | ||
189 | As in the previous tables, the ovs-vsctl command may be used to configure the | |
190 | values. For example, to set the ``iface-id`` on eth0, the following command | |
191 | can be used: | |
192 | ||
193 | :: | |
194 | ||
195 | $ ovs-vsctl set Interface eth0 external-ids:iface-id='"${UUID}"' | |
189565c9 BS |
196 | |
197 | ||
198 | HA for OVN DB servers using pacemaker | |
199 | ------------------------------------- | |
200 | ||
201 | The ovsdb servers can work in either active or backup mode. In backup mode, db | |
202 | server will be connected to an active server and replicate the active servers | |
203 | contents. At all times, the data can be transacted only from the active server. | |
204 | When the active server dies for some reason, entire OVN operations will be | |
205 | stalled. | |
206 | ||
7c9afefd | 207 | `Pacemaker <http://clusterlabs.org/pacemaker.html>`__ is a cluster resource |
189565c9 BS |
208 | manager which can manage a defined set of resource across a set of clustered |
209 | nodes. Pacemaker manages the resource with the help of the resource agents. | |
7c9afefd SF |
210 | One among the resource agent is `OCF |
211 | <http://www.linux-ha.org/wiki/OCF_Resource_Agents>`__ | |
189565c9 BS |
212 | |
213 | OCF is nothing but a shell script which accepts a set of actions and returns an | |
214 | appropriate status code. | |
215 | ||
216 | With the help of the OCF resource agent ovn/utilities/ovndb-servers.ocf, one | |
217 | can defined a resource for the pacemaker such that pacemaker will always | |
218 | maintain one running active server at any time. | |
219 | ||
7c9afefd SF |
220 | After creating a pacemaker cluster, use the following commands to create one |
221 | active and multiple backup servers for OVN databases:: | |
189565c9 | 222 | |
7c9afefd | 223 | $ pcs resource create ovndb_servers ocf:ovn:ovndb-servers \ |
189565c9 BS |
224 | master_ip=x.x.x.x \ |
225 | ovn_ctl=<path of the ovn-ctl script> \ | |
7d911540 GL |
226 | op monitor interval="10s" \ |
227 | op monitor role=Master interval="15s" | |
7c9afefd | 228 | $ pcs resource master ovndb_servers-master ovndb_servers \ |
189565c9 BS |
229 | meta notify="true" |
230 | ||
7c9afefd SF |
231 | The `master_ip` and `ovn_ctl` are the parameters that will be used by the OCF |
232 | script. `ovn_ctl` is optional, if not given, it assumes a default value of | |
189565c9 | 233 | /usr/share/openvswitch/scripts/ovn-ctl. `master_ip` is the IP address on which |
e51e9854 GL |
234 | the active database server is expected to be listening, the slave node uses it |
235 | to connect to the master node. You can add the optional parameters | |
236 | 'nb_master_port', 'nb_master_protocol', 'sb_master_port', 'sb_master_protocol' | |
237 | to set the protocol and port. | |
189565c9 | 238 | |
7c9afefd SF |
239 | Whenever the active server dies, pacemaker is responsible to promote one of the |
240 | backup servers to be active. Both ovn-controller and ovn-northd needs the | |
189565c9 BS |
241 | ip-address at which the active server is listening. With pacemaker changing the |
242 | node at which the active server is run, it is not efficient to instruct all the | |
2010abfc GL |
243 | ovn-controllers and the ovn-northd to listen to the latest active server's |
244 | ip-address. | |
189565c9 BS |
245 | |
246 | This problem can be solved by using a native ocf resource agent | |
7c9afefd SF |
247 | ``ocf:heartbeat:IPaddr2``. The IPAddr2 resource agent is just a resource with |
248 | an ip-address. When we colocate this resource with the active server, pacemaker | |
189565c9 BS |
249 | will enable the active server to be connected with a single ip-address all the |
250 | time. This is the ip-address that needs to be given as the parameter while | |
251 | creating the `ovndb_servers` resource. | |
252 | ||
253 | Use the following command to create the IPAddr2 resource and colocate it | |
7c9afefd | 254 | with the active server:: |
189565c9 | 255 | |
7c9afefd | 256 | $ pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=x.x.x.x \ |
189565c9 | 257 | op monitor interval=30s |
a38f5323 GL |
258 | $ pcs constraint order promote ovndb_servers-master then VirtualIP |
259 | $ pcs constraint colocation add VirtualIP with master ovndb_servers-master \ | |
189565c9 | 260 | score=INFINITY |