]>
Commit | Line | Data |
---|---|---|
2567fb84 SF |
1 | .. |
2 | Licensed under the Apache License, Version 2.0 (the "License"); you may | |
3 | not use this file except in compliance with the License. You may obtain | |
4 | a copy of the License at | |
5 | ||
6 | http://www.apache.org/licenses/LICENSE-2.0 | |
7 | ||
8 | Unless required by applicable law or agreed to in writing, software | |
9 | distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | |
10 | WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | |
11 | License for the specific language governing permissions and limitations | |
12 | under the License. | |
13 | ||
14 | Convention for heading levels in Open vSwitch documentation: | |
15 | ||
16 | ======= Heading 0 (reserved for the title in a document) | |
17 | ------- Heading 1 | |
18 | ~~~~~~~ Heading 2 | |
19 | +++++++ Heading 3 | |
20 | ''''''' Heading 4 | |
21 | ||
22 | Avoid deeper levels because they do not render well. | |
23 | ||
24 | ========================================= | |
25 | Integration Guide for Centralized Control | |
26 | ========================================= | |
27 | ||
28 | This document describes how to integrate Open vSwitch onto a new platform to | |
29 | expose the state of the switch and attached devices for centralized control. | |
30 | (If you are looking to port the switching components of Open vSwitch to a new | |
31 | platform, please see the PORTING document.) The focus of this guide is on | |
32 | hypervisors, but many of the interfaces are useful for hardware switches, as | |
33 | well. The XenServer integration is the most mature implementation, so most of | |
34 | the examples are drawn from it. | |
35 | ||
36 | The externally visible interface to this integration is platform-agnostic. We | |
37 | encourage anyone who integrates Open vSwitch to use the same interface, because | |
38 | keeping a uniform interface means that controllers require less customization | |
39 | for individual platforms (and perhaps no customization at all). | |
40 | ||
41 | Integration centers around the Open vSwitch database and mostly involves the | |
42 | ``external_ids`` columns in several of the tables. These columns are not | |
43 | interpreted by Open vSwitch itself. Instead, they provide information to a | |
44 | controller that permits it to associate a database record with a more | |
45 | meaningful entity. In contrast, the ``other_config`` column is used to | |
46 | configure behavior of the switch. The main job of the integrator, then, is to | |
47 | ensure that these values are correctly populated and maintained. | |
48 | ||
49 | An integrator sets the columns in the database by talking to the ovsdb-server | |
50 | daemon. A few of the columns can be set during startup by calling the ovs-ctl | |
51 | tool from inside the startup scripts. The ``xenserver/etc_init.d_openvswitch`` | |
52 | script provides examples of its use, and the ovs-ctl(8) manpage contains | |
53 | complete documentation. At runtime, ovs-vsctl can be be used to set columns in | |
54 | the database. The script ``xenserver/etc_xensource_scripts_vif`` contains | |
55 | examples of its use, and ovs-vsctl(8) manpage contains complete documentation. | |
56 | ||
57 | Python and C bindings to the database are provided if deeper integration with a | |
58 | program are needed. The XenServer ovs-xapi-sync daemon | |
59 | (``xenserver/usr_share_openvswitch_scripts_ovs-xapi-sync``) provides an example | |
60 | of using the Python bindings. More information on the python bindings is | |
61 | available at ``python/ovs/db/idl.py``. Information on the C bindings is | |
62 | available at ``lib/ovsdb-idl.h``. | |
63 | ||
64 | The following diagram shows how integration scripts fit into the Open vSwitch | |
65 | architecture: | |
66 | ||
67 | :: | |
68 | ||
69 | Diagram | |
70 | ||
71 | +----------------------------------------+ | |
72 | | Controller Cluster + | |
73 | +----------------------------------------+ | |
74 | | | |
75 | | | |
76 | +----------------------------------------------------------+ | |
77 | | | | | |
78 | | +--------------+---------------+ | | |
79 | | | | | | |
80 | | +-------------------+ +------------------+ | | |
81 | | | ovsdb-server |-----------| ovs-vswitchd | | | |
82 | | +-------------------+ +------------------+ | | |
83 | | | | | | |
84 | | +---------------------+ | | | |
85 | | | Integration scripts | | | | |
86 | | | (ex: ovs-xapi-sync) | | | | |
87 | | +---------------------+ | | | |
88 | | | Userspace | | |
89 | |----------------------------------------------------------| | |
90 | | | Kernel | | |
91 | | | | | |
92 | | +---------------------+ | | |
93 | | | OVS Kernel Module | | | |
94 | | +---------------------+ | | |
95 | +----------------------------------------------------------+ | |
96 | ||
97 | A description of the most relevant fields for integration follows. By setting | |
98 | these values, controllers are able to understand the network and manage it more | |
99 | dynamically and precisely. For more details about the database and each | |
100 | individual column, please refer to the ovs-vswitchd.conf.db(5) manpage. | |
101 | ||
102 | ``Open_vSwitch`` table | |
103 | ---------------------- | |
104 | ||
105 | The ``Open_vSwitch`` table describes the switch as a whole. The | |
106 | ``system_type`` and ``system_version`` columns identify the platform to the | |
107 | controller. The ``external_ids:system-id`` key uniquely identifies the | |
108 | physical host. In XenServer, the system-id will likely be the same as the UUID | |
109 | returned by ``xe host-list``. This key allows controllers to distinguish | |
110 | between multiple hypervisors. | |
111 | ||
112 | Most of this configuration can be done with the ovs-ctl command at startup. | |
113 | For example: | |
114 | ||
115 | :: | |
116 | ||
117 | $ ovs-ctl --system-type="XenServer" --system-version="6.0.0-50762p" \ | |
118 | --system-id="${UUID}" "${other_options}" start | |
119 | ||
120 | Alternatively, the ovs-vsctl command may be used to set a particular value at | |
121 | runtime. For example: | |
122 | ||
123 | :: | |
124 | ||
125 | $ ovs-vsctl set open_vswitch . external-ids:system-id='"${UUID}"' | |
126 | ||
127 | The ``other_config:enable-statistics`` key may be set to ``true`` to have OVS | |
128 | populate the database with statistics (e.g., number of CPUs, memory, system | |
129 | load) for the controller's use. | |
130 | ||
131 | Bridge table | |
132 | ------------ | |
133 | ||
134 | The Bridge table describes individual bridges within an Open vSwitch instance. | |
135 | The ``external-ids:bridge-id`` key uniquely identifies a particular bridge. In | |
136 | XenServer, this will likely be the same as the UUID returned by ``xe | |
137 | network-list`` for that particular bridge. | |
138 | ||
139 | For example, to set the identifier for bridge "br0", the following command can | |
140 | be used: | |
141 | ||
142 | :: | |
143 | ||
144 | $ ovs-vsctl set Bridge br0 external-ids:bridge-id='"${UUID}"' | |
145 | ||
146 | The MAC address of the bridge may be manually configured by setting it with the | |
147 | ``other_config:hwaddr`` key. For example: | |
148 | ||
149 | :: | |
150 | ||
151 | $ ovs-vsctl set Bridge br0 other_config:hwaddr="12:34:56:78:90:ab" | |
152 | ||
153 | Interface table | |
154 | --------------- | |
155 | ||
156 | The Interface table describes an interface under the control of Open vSwitch. | |
157 | The ``external_ids`` column contains keys that are used to provide additional | |
158 | information about the interface: | |
159 | ||
160 | attached-mac | |
161 | ||
162 | This field contains the MAC address of the device attached to the interface. | |
163 | On a hypervisor, this is the MAC address of the interface as seen inside a | |
164 | VM. It does not necessarily correlate to the host-side MAC address. For | |
165 | example, on XenServer, the MAC address on a VIF in the hypervisor is always | |
166 | FE:FF:FF:FF:FF:FF, but inside the VM a normal MAC address is seen. | |
167 | ||
168 | iface-id | |
169 | ||
170 | This field uniquely identifies the interface. In hypervisors, this allows | |
171 | the controller to follow VM network interfaces as VMs migrate. A well-chosen | |
172 | identifier should also allow an administrator or a controller to associate | |
173 | the interface with the corresponding object in the VM management system. For | |
174 | example, the Open vSwitch integration with XenServer by default uses the | |
175 | XenServer assigned UUID for a VIF record as the iface-id. | |
176 | ||
177 | iface-status | |
178 | ||
179 | In a hypervisor, there are situations where there are multiple interface | |
180 | choices for a single virtual ethernet interface inside a VM. Valid values | |
181 | are "active" and "inactive". A complete description is available in the | |
182 | ovs-vswitchd.conf.db(5) manpage. | |
183 | ||
184 | vm-id | |
185 | ||
186 | This field uniquely identifies the VM to which this interface belongs. A | |
187 | single VM may have multiple interfaces attached to it. | |
188 | ||
189 | As in the previous tables, the ovs-vsctl command may be used to configure the | |
190 | values. For example, to set the ``iface-id`` on eth0, the following command | |
191 | can be used: | |
192 | ||
193 | :: | |
194 | ||
195 | $ ovs-vsctl set Interface eth0 external-ids:iface-id='"${UUID}"' | |
189565c9 BS |
196 | |
197 | ||
198 | HA for OVN DB servers using pacemaker | |
199 | ------------------------------------- | |
200 | ||
201 | The ovsdb servers can work in either active or backup mode. In backup mode, db | |
202 | server will be connected to an active server and replicate the active servers | |
203 | contents. At all times, the data can be transacted only from the active server. | |
204 | When the active server dies for some reason, entire OVN operations will be | |
205 | stalled. | |
206 | ||
207 | `Pacemaker <http://clusterlabs.org/pacemaker.html>`_ is a cluster resource | |
208 | manager which can manage a defined set of resource across a set of clustered | |
209 | nodes. Pacemaker manages the resource with the help of the resource agents. | |
210 | One among the resource agent is | |
51ca1872 | 211 | `OCF <http://www.linux-ha.org/wiki/OCF_Resource_Agents>`_ |
189565c9 BS |
212 | |
213 | OCF is nothing but a shell script which accepts a set of actions and returns an | |
214 | appropriate status code. | |
215 | ||
216 | With the help of the OCF resource agent ovn/utilities/ovndb-servers.ocf, one | |
217 | can defined a resource for the pacemaker such that pacemaker will always | |
218 | maintain one running active server at any time. | |
219 | ||
220 | After creating a pacemaker cluster, use the following commands to create | |
221 | one active and multiple backup servers for OVN databases. | |
222 | ||
223 | :: | |
224 | ||
225 | pcs resource create ovndb_servers ocf:ovn:ovndb-servers \ | |
226 | master_ip=x.x.x.x \ | |
227 | ovn_ctl=<path of the ovn-ctl script> \ | |
228 | op monitor interval="10s" | |
229 | ||
230 | pcs resource master ovndb_servers-master ovndb_servers \ | |
231 | meta notify="true" | |
232 | ||
233 | The `master_ip` and `ovn_ctl` are the parameters that will be used by the | |
234 | OCF script. `ovn_ctl` is optional, if not given, it assumes a default value of | |
235 | /usr/share/openvswitch/scripts/ovn-ctl. `master_ip` is the IP address on which | |
236 | the active database server is expected to be listening. | |
237 | ||
238 | Whenever the active server dies, pacemaker is responsible to promote one of | |
239 | the backup servers to be active. Both ovn-controller and ovn-northd needs the | |
240 | ip-address at which the active server is listening. With pacemaker changing the | |
241 | node at which the active server is run, it is not efficient to instruct all the | |
242 | ovn-controllers and the ovn-northd to listen to the latest active server's ip- | |
243 | address | |
244 | ||
245 | This problem can be solved by using a native ocf resource agent | |
246 | `ocf:heartbeat:IPaddr2`. The IPAddr2 resource agent is just a resource with an | |
247 | ip-address. When we colocate this resource with the active server, pacemaker | |
248 | will enable the active server to be connected with a single ip-address all the | |
249 | time. This is the ip-address that needs to be given as the parameter while | |
250 | creating the `ovndb_servers` resource. | |
251 | ||
252 | Use the following command to create the IPAddr2 resource and colocate it | |
253 | with the active server. | |
254 | ||
255 | :: | |
256 | ||
257 | pcs resource create VirtualIP ocf:heartbeat:IPaddr2 ip=x.x.x.x \ | |
258 | op monitor interval=30s | |
259 | ||
260 | pcs constraint order VirtualIP then ovndb_servers-master | |
261 | ||
262 | pcs constraint colocation add ovndb_servers-master with master VirtualIP \ | |
263 | score=INFINITY |