]>
Commit | Line | Data |
---|---|---|
542cc9bb TG |
1 | How to Use Open vSwitch with Docker |
2 | ==================================== | |
ec8f0f0c | 3 | |
eaa923e3 | 4 | This document describes how to use Open vSwitch with Docker 1.9.0 or |
4384556d GS |
5 | later. This document assumes that you installed Open vSwitch by following |
6 | [INSTALL.md] or by using the distribution packages such as .deb or .rpm. | |
7 | Consult www.docker.com for instructions on how to install Docker. | |
ec8f0f0c | 8 | |
eaa923e3 GS |
9 | Docker 1.9.0 comes with support for multi-host networking. Integration |
10 | of Docker networking and Open vSwitch can be achieved via Open vSwitch | |
11 | virtual network (OVN). | |
12 | ||
ec8f0f0c GS |
13 | |
14 | Setup | |
eaa923e3 GS |
15 | ===== |
16 | ||
17 | For multi-host networking with OVN and Docker, Docker has to be started | |
18 | with a destributed key-value store. For e.g., if you decide to use consul | |
19 | as your distributed key-value store, and your host IP address is $HOST_IP, | |
20 | start your Docker daemon with: | |
21 | ||
22 | ``` | |
23 | docker daemon --cluster-store=consul://127.0.0.1:8500 \ | |
24 | --cluster-advertise=$HOST_IP:0 | |
25 | ``` | |
26 | ||
27 | OVN provides network virtualization to containers. OVN's integration with | |
28 | Docker currently works in two modes - the "underlay" mode or the "overlay" | |
29 | mode. | |
30 | ||
31 | In the "underlay" mode, OVN requires a OpenStack setup to provide container | |
32 | networking. In this mode, one can create logical networks and can have | |
33 | containers running inside VMs, standalone VMs (without having any containers | |
34 | running inside them) and physical machines connected to the same logical | |
35 | network. This is a multi-tenant, multi-host solution. | |
36 | ||
37 | In the "overlay" mode, OVN can create a logical network amongst containers | |
38 | running on multiple hosts. This is a single-tenant (extendable to | |
39 | multi-tenants depending on the security characteristics of the workloads), | |
40 | multi-host solution. In this mode, you do not need a pre-created OpenStack | |
41 | setup. | |
42 | ||
43 | For both the modes to work, a user has to install and start Open vSwitch in | |
44 | each VM/host that he plans to run his containers. | |
45 | ||
46 | ||
47 | The "overlay" mode | |
48 | ================== | |
49 | ||
50 | OVN in "overlay" mode needs a minimum Open vSwitch version of 2.5. | |
51 | ||
52 | * Start the central components. | |
53 | ||
54 | OVN architecture has a central component which stores your networking intent | |
55 | in a database. On one of your machines, with an IP Address of $CENTRAL_IP, | |
56 | where you have installed and started Open vSwitch, you will need to start some | |
57 | central components. | |
58 | ||
eaa923e3 GS |
59 | Start ovn-northd daemon. This daemon translates networking intent from Docker |
60 | stored in the OVN_Northbound database to logical flows in OVN_Southbound | |
61 | database. | |
62 | ||
63 | ``` | |
64 | /usr/share/openvswitch/scripts/ovn-ctl start_northd | |
65 | ``` | |
66 | ||
67 | * One time setup. | |
68 | ||
69 | On each host, where you plan to spawn your containers, you will need to | |
70 | run the following command once. (You need to run it again if your OVS database | |
71 | gets cleared. It is harmless to run it again in any case.) | |
72 | ||
73 | $LOCAL_IP in the below command is the IP address via which other hosts | |
74 | can reach this host. This acts as your local tunnel endpoint. | |
75 | ||
76 | $ENCAP_TYPE is the type of tunnel that you would like to use for overlay | |
77 | networking. The options are "geneve" or "stt". (Please note that your | |
78 | kernel should have support for your chosen $ENCAP_TYPE. Both geneve | |
79 | and stt are part of the Open vSwitch kernel module that is compiled from this | |
80 | repo. If you use the Open vSwitch kernel module from upstream Linux, | |
81 | you will need a minumum kernel version of 3.18 for geneve. There is no stt | |
82 | support in upstream Linux. You can verify whether you have the support in your | |
83 | kernel by doing a "lsmod | grep $ENCAP_TYPE".) | |
84 | ||
85 | ``` | |
d61fbedc GS |
86 | ovs-vsctl set Open_vSwitch . external_ids:ovn-remote="tcp:$CENTRAL_IP:6642" \ |
87 | external_ids:ovn-nb="tcp:$CENTRAL_IP:6641" external_ids:ovn-encap-ip=$LOCAL_IP external_ids:ovn-encap-type="$ENCAP_TYPE" | |
eaa923e3 GS |
88 | ``` |
89 | ||
90 | And finally, start the ovn-controller. (You need to run the below command | |
91 | on every boot) | |
92 | ||
93 | ``` | |
94 | /usr/share/openvswitch/scripts/ovn-ctl start_controller | |
95 | ``` | |
96 | ||
97 | * Start the Open vSwitch network driver. | |
98 | ||
99 | By default Docker uses Linux bridge for networking. But it has support | |
100 | for external drivers. To use Open vSwitch instead of the Linux bridge, | |
101 | you will need to start the Open vSwitch driver. | |
102 | ||
103 | The Open vSwitch driver uses the Python's flask module to listen to | |
104 | Docker's networking api calls. So, if your host does not have Python's | |
105 | flask module, install it with: | |
106 | ||
107 | ``` | |
108 | easy_install -U pip | |
109 | pip install Flask | |
110 | ``` | |
111 | ||
112 | Start the Open vSwitch driver on every host where you plan to create your | |
113 | containers. | |
114 | ||
115 | ``` | |
116 | ovn-docker-overlay-driver --detach | |
117 | ``` | |
118 | ||
119 | Docker has inbuilt primitives that closely match OVN's logical switches | |
120 | and logical port concepts. Please consult Docker's documentation for | |
121 | all the possible commands. Here are some examples. | |
122 | ||
123 | * Create your logical switch. | |
124 | ||
125 | To create a logical switch with name 'foo', on subnet '192.168.1.0/24' run: | |
126 | ||
127 | ``` | |
128 | NID=`docker network create -d openvswitch --subnet=192.168.1.0/24 foo` | |
129 | ``` | |
130 | ||
131 | * List your logical switches. | |
132 | ||
133 | ``` | |
134 | docker network ls | |
135 | ``` | |
136 | ||
137 | You can also look at this logical switch in OVN's northbound database by | |
138 | running the following command. | |
139 | ||
140 | ``` | |
141 | ovn-nbctl --db=tcp:$CENTRAL_IP:6640 lswitch-list | |
142 | ``` | |
143 | ||
144 | * Docker creates your logical port and attaches it to the logical network | |
145 | in a single step. | |
146 | ||
147 | For e.g., to attach a logical port to network 'foo' inside cotainer busybox, | |
148 | run: | |
149 | ||
150 | ``` | |
151 | docker run -itd --net=foo --name=busybox busybox | |
152 | ``` | |
153 | ||
154 | * List all your logical ports. | |
155 | ||
156 | Docker currently does not have a CLI command to list all your logical ports. | |
157 | But you can look at them in the OVN database, by running: | |
ec8f0f0c | 158 | |
542cc9bb | 159 | ``` |
31ed1192 | 160 | ovn-nbctl --db=tcp:$CENTRAL_IP:6640 lsp-list $NID |
542cc9bb | 161 | ``` |
ec8f0f0c | 162 | |
eaa923e3 | 163 | * You can also create a logical port and attach it to a running container. |
ec8f0f0c | 164 | |
542cc9bb | 165 | ``` |
eaa923e3 GS |
166 | docker network create -d openvswitch --subnet=192.168.2.0/24 bar |
167 | docker network connect bar busybox | |
542cc9bb | 168 | ``` |
ec8f0f0c | 169 | |
eaa923e3 GS |
170 | You can delete your logical port and detach it from a running container by |
171 | running: | |
172 | ||
173 | ``` | |
174 | docker network disconnect bar busybox | |
175 | ``` | |
ec8f0f0c | 176 | |
eaa923e3 | 177 | * You can delete your logical switch by running: |
ec8f0f0c | 178 | |
eaa923e3 GS |
179 | ``` |
180 | docker network rm bar | |
181 | ``` | |
ec8f0f0c | 182 | |
ec8f0f0c | 183 | |
eaa923e3 GS |
184 | The "underlay" mode |
185 | =================== | |
186 | ||
187 | This mode requires that you have a OpenStack setup pre-installed with OVN | |
188 | providing the underlay networking. | |
189 | ||
190 | * One time setup. | |
191 | ||
192 | A OpenStack tenant creates a VM with a single network interface (or multiple) | |
193 | that belongs to management logical networks. The tenant needs to fetch the | |
194 | port-id associated with the interface via which he plans to send the container | |
195 | traffic inside the spawned VM. This can be obtained by running the | |
196 | below command to fetch the 'id' associated with the VM. | |
ec8f0f0c | 197 | |
05444f07 | 198 | ``` |
eaa923e3 | 199 | nova list |
05444f07 | 200 | ``` |
ec8f0f0c | 201 | |
eaa923e3 | 202 | and then by running: |
ec8f0f0c | 203 | |
eaa923e3 GS |
204 | ``` |
205 | neutron port-list --device_id=$id | |
206 | ``` | |
ec8f0f0c | 207 | |
eaa923e3 GS |
208 | Inside the VM, download the OpenStack RC file that contains the tenant |
209 | information (henceforth referred to as 'openrc.sh'). Edit the file and add the | |
210 | previously obtained port-id information to the file by appending the following | |
211 | line: export OS_VIF_ID=$port_id. After this edit, the file will look something | |
212 | like: | |
ec8f0f0c | 213 | |
eaa923e3 GS |
214 | ``` |
215 | #!/bin/bash | |
216 | export OS_AUTH_URL=http://10.33.75.122:5000/v2.0 | |
217 | export OS_TENANT_ID=fab106b215d943c3bad519492278443d | |
218 | export OS_TENANT_NAME="demo" | |
219 | export OS_USERNAME="demo" | |
220 | export OS_VIF_ID=e798c371-85f4-4f2d-ad65-d09dd1d3c1c9 | |
221 | ``` | |
222 | ||
223 | * Create the Open vSwitch bridge. | |
224 | ||
225 | If your VM has one ethernet interface (e.g.: 'eth0'), you will need to add | |
226 | that device as a port to an Open vSwitch bridge 'breth0' and move its IP | |
227 | address and route related information to that bridge. (If it has multiple | |
228 | network interfaces, you will need to create and attach an Open vSwitch bridge | |
229 | for the interface via which you plan to send your container traffic.) | |
230 | ||
231 | If you use DHCP to obtain an IP address, then you should kill the DHCP client | |
232 | that was listening on the physical Ethernet interface (e.g. eth0) and start | |
233 | one listening on the Open vSwitch bridge (e.g. breth0). | |
ec8f0f0c | 234 | |
eaa923e3 GS |
235 | Depending on your VM, you can make the above step persistent across reboots. |
236 | For e.g.:, if your VM is Debian/Ubuntu, you can read | |
237 | [openvswitch-switch.README.Debian]. If your VM is RHEL based, you can read | |
238 | [README.RHEL] | |
ec8f0f0c | 239 | |
ec8f0f0c | 240 | |
eaa923e3 | 241 | * Start the Open vSwitch network driver. |
7894385a | 242 | |
eaa923e3 GS |
243 | The Open vSwitch driver uses the Python's flask module to listen to |
244 | Docker's networking api calls. The driver also uses OpenStack's | |
245 | python-neutronclient libraries. So, if your host does not have Python's | |
246 | flask module or python-neutronclient install them with: | |
247 | ||
248 | ``` | |
249 | easy_install -U pip | |
250 | pip install python-neutronclient | |
251 | pip install Flask | |
7894385a | 252 | ``` |
eaa923e3 GS |
253 | |
254 | Source the openrc file. e.g.: | |
255 | ```` | |
256 | . ./openrc.sh | |
7894385a GS |
257 | ``` |
258 | ||
eaa923e3 GS |
259 | Start the network driver and provide your OpenStack tenant password |
260 | when prompted. | |
ec8f0f0c | 261 | |
eaa923e3 GS |
262 | ``` |
263 | ovn-docker-underlay-driver --bridge breth0 --detach | |
264 | ``` | |
ec8f0f0c | 265 | |
eaa923e3 GS |
266 | From here-on you can use the same Docker commands as described in the |
267 | section 'The "overlay" mode'. | |
ec8f0f0c | 268 | |
eaa923e3 GS |
269 | Please read 'man ovn-architecture' to understand OVN's architecture in |
270 | detail. | |
9feb1017 | 271 | |
eaa923e3 GS |
272 | [INSTALL.md]: INSTALL.md |
273 | [openvswitch-switch.README.Debian]: debian/openvswitch-switch.README.Debian | |
274 | [README.RHEL]: rhel/README.RHEL |