]>
Commit | Line | Data |
---|---|---|
0bcd1f7f DM |
1 | Network Configuration |
2 | --------------------- | |
3 | include::attributes.txt[] | |
5f09af76 DM |
4 | ifdef::wiki[] |
5 | :pve-toplevel: | |
6 | endif::wiki[] | |
7 | ||
0bcd1f7f DM |
8 | {pve} uses a bridged networking model. Each host can have up to 4094 |
9 | bridges. Bridges are like physical network switches implemented in | |
10 | software. All VMs can share a single bridge, as if | |
11 | virtual network cables from each guest were all plugged into the same | |
12 | switch. But you can also create multiple bridges to separate network | |
13 | domains. | |
14 | ||
15 | For connecting VMs to the outside world, bridges are attached to | |
16 | physical network cards. For further flexibility, you can configure | |
17 | VLANs (IEEE 802.1q) and network bonding, also known as "link | |
18 | aggregation". That way it is possible to build complex and flexible | |
19 | virtual networks. | |
20 | ||
8c1189b6 FG |
21 | Debian traditionally uses the `ifup` and `ifdown` commands to |
22 | configure the network. The file `/etc/network/interfaces` contains the | |
23 | whole network setup. Please refer to to manual page (`man interfaces`) | |
0bcd1f7f DM |
24 | for a complete format description. |
25 | ||
26 | NOTE: {pve} does not write changes directly to | |
8c1189b6 FG |
27 | `/etc/network/interfaces`. Instead, we write into a temporary file |
28 | called `/etc/network/interfaces.new`, and commit those changes when | |
0bcd1f7f DM |
29 | you reboot the node. |
30 | ||
31 | It is worth mentioning that you can directly edit the configuration | |
32 | file. All {pve} tools tries hard to keep such direct user | |
33 | modifications. Using the GUI is still preferable, because it | |
34 | protect you from errors. | |
35 | ||
5eba0743 | 36 | |
0bcd1f7f DM |
37 | Naming Conventions |
38 | ~~~~~~~~~~~~~~~~~~ | |
39 | ||
40 | We currently use the following naming conventions for device names: | |
41 | ||
42 | * Ethernet devices: eth[N], where 0 ≤ N (`eth0`, `eth1`, ...) | |
43 | ||
44 | * Bridge names: vmbr[N], where 0 ≤ N ≤ 4094 (`vmbr0` - `vmbr4094`) | |
45 | ||
46 | * Bonds: bond[N], where 0 ≤ N (`bond0`, `bond1`, ...) | |
47 | ||
48 | * VLANs: Simply add the VLAN number to the device name, | |
49 | separated by a period (`eth0.50`, `bond1.30`) | |
50 | ||
51 | This makes it easier to debug networks problems, because the device | |
52 | names implies the device type. | |
53 | ||
54 | Default Configuration using a Bridge | |
55 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
56 | ||
57 | The installation program creates a single bridge named `vmbr0`, which | |
58 | is connected to the first ethernet card `eth0`. The corresponding | |
8c1189b6 | 59 | configuration in `/etc/network/interfaces` looks like this: |
0bcd1f7f DM |
60 | |
61 | ---- | |
62 | auto lo | |
63 | iface lo inet loopback | |
64 | ||
65 | iface eth0 inet manual | |
66 | ||
67 | auto vmbr0 | |
68 | iface vmbr0 inet static | |
69 | address 192.168.10.2 | |
70 | netmask 255.255.255.0 | |
71 | gateway 192.168.10.1 | |
72 | bridge_ports eth0 | |
73 | bridge_stp off | |
74 | bridge_fd 0 | |
75 | ---- | |
76 | ||
77 | Virtual machines behave as if they were directly connected to the | |
78 | physical network. The network, in turn, sees each virtual machine as | |
79 | having its own MAC, even though there is only one network cable | |
80 | connecting all of these VMs to the network. | |
81 | ||
82 | ||
83 | Routed Configuration | |
84 | ~~~~~~~~~~~~~~~~~~~~ | |
85 | ||
86 | Most hosting providers do not support the above setup. For security | |
87 | reasons, they disable networking as soon as they detect multiple MAC | |
88 | addresses on a single interface. | |
89 | ||
90 | TIP: Some providers allows you to register additional MACs on there | |
91 | management interface. This avoids the problem, but is clumsy to | |
92 | configure because you need to register a MAC for each of your VMs. | |
93 | ||
8c1189b6 | 94 | You can avoid the problem by ``routing'' all traffic via a single |
0bcd1f7f DM |
95 | interface. This makes sure that all network packets use the same MAC |
96 | address. | |
97 | ||
8c1189b6 | 98 | A common scenario is that you have a public IP (assume `192.168.10.2` |
0bcd1f7f | 99 | for this example), and an additional IP block for your VMs |
8c1189b6 | 100 | (`10.10.10.1/255.255.255.0`). We recommend the following setup for such |
0bcd1f7f DM |
101 | situations: |
102 | ||
103 | ---- | |
104 | auto lo | |
105 | iface lo inet loopback | |
106 | ||
107 | auto eth0 | |
108 | iface eth0 inet static | |
109 | address 192.168.10.2 | |
110 | netmask 255.255.255.0 | |
111 | gateway 192.168.10.1 | |
112 | post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp | |
113 | ||
114 | ||
115 | auto vmbr0 | |
116 | iface vmbr0 inet static | |
117 | address 10.10.10.1 | |
118 | netmask 255.255.255.0 | |
119 | bridge_ports none | |
120 | bridge_stp off | |
121 | bridge_fd 0 | |
122 | ---- | |
123 | ||
124 | ||
8c1189b6 FG |
125 | Masquerading (NAT) with `iptables` |
126 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
0bcd1f7f DM |
127 | |
128 | In some cases you may want to use private IPs behind your Proxmox | |
129 | host's true IP, and masquerade the traffic using NAT: | |
130 | ||
131 | ---- | |
132 | auto lo | |
133 | iface lo inet loopback | |
134 | ||
135 | auto eth0 | |
136 | #real IP adress | |
137 | iface eth0 inet static | |
138 | address 192.168.10.2 | |
139 | netmask 255.255.255.0 | |
140 | gateway 192.168.10.1 | |
141 | ||
142 | auto vmbr0 | |
143 | #private sub network | |
144 | iface vmbr0 inet static | |
145 | address 10.10.10.1 | |
146 | netmask 255.255.255.0 | |
147 | bridge_ports none | |
148 | bridge_stp off | |
149 | bridge_fd 0 | |
150 | ||
151 | post-up echo 1 > /proc/sys/net/ipv4/ip_forward | |
152 | post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE | |
153 | post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE | |
154 | ---- | |
155 | ||
b4c06a93 WL |
156 | |
157 | Linux Bond | |
158 | ~~~~~~~~~~ | |
159 | ||
3eafe338 WL |
160 | Bonding (also called NIC teaming or Link Aggregation) is a technique |
161 | for binding multiple NIC's to a single network device. It is possible | |
162 | to achieve different goals, like make the network fault-tolerant, | |
163 | increase the performance or both together. | |
164 | ||
165 | High-speed hardware like Fibre Channel and the associated switching | |
166 | hardware can be quite expensive. By doing link aggregation, two NICs | |
167 | can appear as one logical interface, resulting in double speed. This | |
168 | is a native Linux kernel feature that is supported by most | |
169 | switches. If your nodes have multiple Ethernet ports, you can | |
170 | distribute your points of failure by running network cables to | |
171 | different switches and the bonded connection will failover to one | |
172 | cable or the other in case of network trouble. | |
173 | ||
174 | Aggregated links can improve live-migration delays and improve the | |
175 | speed of replication of data between Proxmox VE Cluster nodes. | |
b4c06a93 WL |
176 | |
177 | There are 7 modes for bonding: | |
178 | ||
179 | * *Round-robin (balance-rr):* Transmit network packets in sequential | |
180 | order from the first available network interface (NIC) slave through | |
181 | the last. This mode provides load balancing and fault tolerance. | |
182 | ||
183 | * *Active-backup (active-backup):* Only one NIC slave in the bond is | |
184 | active. A different slave becomes active if, and only if, the active | |
185 | slave fails. The single logical bonded interface's MAC address is | |
186 | externally visible on only one NIC (port) to avoid distortion in the | |
187 | network switch. This mode provides fault tolerance. | |
188 | ||
189 | * *XOR (balance-xor):* Transmit network packets based on [(source MAC | |
190 | address XOR'd with destination MAC address) modulo NIC slave | |
191 | count]. This selects the same NIC slave for each destination MAC | |
192 | address. This mode provides load balancing and fault tolerance. | |
193 | ||
194 | * *Broadcast (broadcast):* Transmit network packets on all slave | |
195 | network interfaces. This mode provides fault tolerance. | |
196 | ||
197 | * *IEEE 802.3ad Dynamic link aggregation (802.3ad)(LACP):* Creates | |
198 | aggregation groups that share the same speed and duplex | |
199 | settings. Utilizes all slave network interfaces in the active | |
200 | aggregator group according to the 802.3ad specification. | |
201 | ||
202 | * *Adaptive transmit load balancing (balance-tlb):* Linux bonding | |
203 | driver mode that does not require any special network-switch | |
204 | support. The outgoing network packet traffic is distributed according | |
205 | to the current load (computed relative to the speed) on each network | |
206 | interface slave. Incoming traffic is received by one currently | |
207 | designated slave network interface. If this receiving slave fails, | |
208 | another slave takes over the MAC address of the failed receiving | |
209 | slave. | |
210 | ||
211 | * *Adaptive load balancing (balanceIEEE 802.3ad Dynamic link | |
212 | aggregation (802.3ad)(LACP):-alb):* Includes balance-tlb plus receive | |
213 | load balancing (rlb) for IPV4 traffic, and does not require any | |
214 | special network switch support. The receive load balancing is achieved | |
215 | by ARP negotiation. The bonding driver intercepts the ARP Replies sent | |
216 | by the local system on their way out and overwrites the source | |
217 | hardware address with the unique hardware address of one of the NIC | |
218 | slaves in the single logical bonded interface such that different | |
219 | network-peers use different MAC addresses for their network packet | |
220 | traffic. | |
221 | ||
222 | For the most setups the active-backup are the best choice or if your | |
223 | switch support LACP "IEEE 802.3ad" this mode should be preferred. | |
224 | ||
cd1de2c2 WL |
225 | The following bond configuration can be used as distributed/shared |
226 | storage network. The benefit would be that you get more speed and the | |
227 | network will be fault-tolerant. | |
228 | ||
b4c06a93 WL |
229 | .Example: Use bond with fixed IP address |
230 | ---- | |
231 | auto lo | |
232 | iface lo inet loopback | |
233 | ||
234 | iface eth1 inet manual | |
235 | ||
236 | iface eth2 inet manual | |
237 | ||
238 | auto bond0 | |
239 | iface bond0 inet static | |
240 | slaves eth1 eth2 | |
241 | address 192.168.1.2 | |
242 | netmask 255.255.255.0 | |
243 | bond_miimon 100 | |
244 | bond_mode 802.3ad | |
245 | bond_xmit_hash_policy layer2+3 | |
246 | ||
247 | auto vmbr0 | |
248 | iface vmbr0 inet static | |
249 | address 10.10.10.2 | |
250 | netmask 255.255.255.0 | |
251 | gateway 10.10.10.1 | |
252 | bridge_ports eth0 | |
253 | bridge_stp off | |
254 | bridge_fd 0 | |
255 | ||
256 | ---- | |
257 | ||
cd1de2c2 WL |
258 | |
259 | Another possibility it to use the bond directly as bridge port. | |
260 | This can be used to make the guest network fault-tolerant. | |
261 | ||
262 | .Example: Use a bond as bridge port | |
b4c06a93 WL |
263 | ---- |
264 | auto lo | |
265 | iface lo inet loopback | |
266 | ||
267 | iface eth1 inet manual | |
268 | ||
269 | iface eth2 inet manual | |
270 | ||
271 | auto bond0 | |
272 | iface bond0 inet maunal | |
273 | slaves eth1 eth2 | |
274 | bond_miimon 100 | |
275 | bond_mode 802.3ad | |
276 | bond_xmit_hash_policy layer2+3 | |
277 | ||
278 | auto vmbr0 | |
279 | iface vmbr0 inet static | |
280 | address 10.10.10.2 | |
281 | netmask 255.255.255.0 | |
282 | gateway 10.10.10.1 | |
283 | bridge_ports bond0 | |
284 | bridge_stp off | |
285 | bridge_fd 0 | |
286 | ||
287 | ---- | |
288 | ||
0bcd1f7f DM |
289 | //// |
290 | TODO: explain IPv6 support? | |
291 | TODO: explan OVS | |
292 | //// |