X-Git-Url: https://git.proxmox.com/?p=pve-docs.git;a=blobdiff_plain;f=pve-network.adoc;h=d5b0a483c88c8211e7a3531a5d695d50c19eb7d9;hp=82226a8a7c98e64d059d4ec8692b8668f747e77c;hb=44f38275ee66731c2ca84b288cdade61484ef5d7;hpb=8c1189b640ae7d10119ff1c046580f48749d38bd diff --git a/pve-network.adoc b/pve-network.adoc index 82226a8..d5b0a48 100644 --- a/pve-network.adoc +++ b/pve-network.adoc @@ -1,6 +1,9 @@ +[[sysadmin_network_configuration]] Network Configuration --------------------- -include::attributes.txt[] +ifdef::wiki[] +:pve-toplevel: +endif::wiki[] {pve} uses a bridged networking model. Each host can have up to 4094 bridges. Bridges are like physical network switches implemented in @@ -17,7 +20,7 @@ virtual networks. Debian traditionally uses the `ifup` and `ifdown` commands to configure the network. The file `/etc/network/interfaces` contains the -whole network setup. Please refer to to manual page (`man interfaces`) +whole network setup. Please refer to the manual page (`man interfaces`) for a complete format description. NOTE: {pve} does not write changes directly to @@ -30,42 +33,71 @@ file. All {pve} tools tries hard to keep such direct user modifications. Using the GUI is still preferable, because it protect you from errors. + Naming Conventions ~~~~~~~~~~~~~~~~~~ We currently use the following naming conventions for device names: -* Ethernet devices: eth[N], where 0 ≤ N (`eth0`, `eth1`, ...) +* New Ethernet devices: en*, systemd network interface names. + +* Legacy Ethernet devices: eth[N], where 0 ≤ N (`eth0`, `eth1`, ...) +They are available when Proxmox VE has been updated by an earlier version. * Bridge names: vmbr[N], where 0 ≤ N ≤ 4094 (`vmbr0` - `vmbr4094`) * Bonds: bond[N], where 0 ≤ N (`bond0`, `bond1`, ...) * VLANs: Simply add the VLAN number to the device name, - separated by a period (`eth0.50`, `bond1.30`) + separated by a period (`eno1.50`, `bond1.30`) This makes it easier to debug networks problems, because the device names implies the device type. + +Systemd Network Interface Names +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Systemd uses the two character prefix 'en' for Ethernet network +devices. The next characters depends on the device driver and the fact +which schema matches first. + +* o[n|d] — devices on board + +* s[f][n|d] — device by hotplug id + +* [P]ps[f][n|d] — devices by bus id + +* x — device by MAC address + +The most common patterns are: + +* eno1 — is the first on board NIC + +* enp3s0f1 — is the NIC on pcibus 3 slot 0 and use the NIC function 1. + +For more information see https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/[Predictable Network Interface Names]. + + Default Configuration using a Bridge ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The installation program creates a single bridge named `vmbr0`, which -is connected to the first ethernet card `eth0`. The corresponding -configuration in `/etc/network/interfaces` looks like this: +is connected to the first Ethernet card. The corresponding +configuration in `/etc/network/interfaces` might look like this: ---- auto lo iface lo inet loopback -iface eth0 inet manual +iface eno1 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.10.2 netmask 255.255.255.0 gateway 192.168.10.1 - bridge_ports eth0 + bridge_ports eno1 bridge_stp off bridge_fd 0 ---- @@ -100,12 +132,13 @@ situations: auto lo iface lo inet loopback -auto eth0 -iface eth0 inet static +auto eno1 +iface eno1 inet static address 192.168.10.2 netmask 255.255.255.0 gateway 192.168.10.1 - post-up echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp + post-up echo 1 > /proc/sys/net/ipv4/ip_forward + post-up echo 1 > /proc/sys/net/ipv4/conf/eno1/proxy_arp auto vmbr0 @@ -128,9 +161,9 @@ host's true IP, and masquerade the traffic using NAT: auto lo iface lo inet loopback -auto eth0 -#real IP adress -iface eth0 inet static +auto eno0 +#real IP address +iface eno1 inet static address 192.168.10.2 netmask 255.255.255.0 gateway 192.168.10.1 @@ -145,11 +178,148 @@ iface vmbr0 inet static bridge_fd 0 post-up echo 1 > /proc/sys/net/ipv4/ip_forward - post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE - post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eth0 -j MASQUERADE + post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE + post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 -j MASQUERADE +---- + + +Linux Bond +~~~~~~~~~~ + +Bonding (also called NIC teaming or Link Aggregation) is a technique +for binding multiple NIC's to a single network device. It is possible +to achieve different goals, like make the network fault-tolerant, +increase the performance or both together. + +High-speed hardware like Fibre Channel and the associated switching +hardware can be quite expensive. By doing link aggregation, two NICs +can appear as one logical interface, resulting in double speed. This +is a native Linux kernel feature that is supported by most +switches. If your nodes have multiple Ethernet ports, you can +distribute your points of failure by running network cables to +different switches and the bonded connection will failover to one +cable or the other in case of network trouble. + +Aggregated links can improve live-migration delays and improve the +speed of replication of data between Proxmox VE Cluster nodes. + +There are 7 modes for bonding: + +* *Round-robin (balance-rr):* Transmit network packets in sequential +order from the first available network interface (NIC) slave through +the last. This mode provides load balancing and fault tolerance. + +* *Active-backup (active-backup):* Only one NIC slave in the bond is +active. A different slave becomes active if, and only if, the active +slave fails. The single logical bonded interface's MAC address is +externally visible on only one NIC (port) to avoid distortion in the +network switch. This mode provides fault tolerance. + +* *XOR (balance-xor):* Transmit network packets based on [(source MAC +address XOR'd with destination MAC address) modulo NIC slave +count]. This selects the same NIC slave for each destination MAC +address. This mode provides load balancing and fault tolerance. + +* *Broadcast (broadcast):* Transmit network packets on all slave +network interfaces. This mode provides fault tolerance. + +* *IEEE 802.3ad Dynamic link aggregation (802.3ad)(LACP):* Creates +aggregation groups that share the same speed and duplex +settings. Utilizes all slave network interfaces in the active +aggregator group according to the 802.3ad specification. + +* *Adaptive transmit load balancing (balance-tlb):* Linux bonding +driver mode that does not require any special network-switch +support. The outgoing network packet traffic is distributed according +to the current load (computed relative to the speed) on each network +interface slave. Incoming traffic is received by one currently +designated slave network interface. If this receiving slave fails, +another slave takes over the MAC address of the failed receiving +slave. + +* *Adaptive load balancing (balance-alb):* Includes balance-tlb plus receive +load balancing (rlb) for IPV4 traffic, and does not require any +special network switch support. The receive load balancing is achieved +by ARP negotiation. The bonding driver intercepts the ARP Replies sent +by the local system on their way out and overwrites the source +hardware address with the unique hardware address of one of the NIC +slaves in the single logical bonded interface such that different +network-peers use different MAC addresses for their network packet +traffic. + +If your switch support the LACP (IEEE 802.3ad) protocol then we recommend using +the corresponding bonding mode (802.3ad). Otherwise you should generally use the +active-backup mode. + +// http://lists.linux-ha.org/pipermail/linux-ha/2013-January/046295.html +If you intend to run your cluster network on the bonding interfaces, then you +have to use active-passive mode on the bonding interfaces, other modes are +unsupported. + +The following bond configuration can be used as distributed/shared +storage network. The benefit would be that you get more speed and the +network will be fault-tolerant. + +.Example: Use bond with fixed IP address +---- +auto lo +iface lo inet loopback + +iface eno1 inet manual + +iface eno2 inet manual + +auto bond0 +iface bond0 inet static + slaves eno1 eno2 + address 192.168.1.2 + netmask 255.255.255.0 + bond_miimon 100 + bond_mode 802.3ad + bond_xmit_hash_policy layer2+3 + +auto vmbr0 +iface vmbr0 inet static + address 10.10.10.2 + netmask 255.255.255.0 + gateway 10.10.10.1 + bridge_ports eno1 + bridge_stp off + bridge_fd 0 + +---- + + +Another possibility it to use the bond directly as bridge port. +This can be used to make the guest network fault-tolerant. + +.Example: Use a bond as bridge port +---- +auto lo +iface lo inet loopback + +iface eno1 inet manual + +iface eno2 inet manual + +auto bond0 +iface bond0 inet manual + slaves eno1 eno2 + bond_miimon 100 + bond_mode 802.3ad + bond_xmit_hash_policy layer2+3 + +auto vmbr0 +iface vmbr0 inet static + address 10.10.10.2 + netmask 255.255.255.0 + gateway 10.10.10.1 + bridge_ports bond0 + bridge_stp off + bridge_fd 0 + ---- //// TODO: explain IPv6 support? -TODO: explan OVS +TODO: explain OVS ////