:pve-toplevel:
endif::wiki[]
-Network configuration can be done either via the GUI, or by manually
-editing the file `/etc/network/interfaces`, which contains the
-whole network configuration. The `interfaces(5)` manual page contains the
-complete format description. All {pve} tools try hard to keep direct
-user modifications, but using the GUI is still preferable, because it
+{pve} is using the Linux network stack. This provides a lot of flexibility on
+how to set up the network on the {pve} nodes. The configuration can be done
+either via the GUI, or by manually editing the file `/etc/network/interfaces`,
+which contains the whole network configuration. The `interfaces(5)` manual
+page contains the complete format description. All {pve} tools try hard to keep
+direct user modifications, but using the GUI is still preferable, because it
protects you from errors.
-Once the network is configured, you can use the Debian traditional tools `ifup`
-and `ifdown` commands to bring interfaces up and down.
+A 'vmbr' interface is needed to connect guests to the underlying physical
+network. They are a Linux bridge which can be thought of as a virtual switch
+to which the guests and physical interfaces are connected to. This section
+provides some examples on how the network can be set up to accomodate different
+use cases like redundancy with a xref:sysadmin_network_bond['bond'],
+xref:sysadmin_network_vlan['vlans'] or
+xref:sysadmin_network_routed['routed'] and
+xref:sysadmin_network_masquerading['NAT'] setups.
+
+The xref:chapter_pvesdn[Software Defined Network] is an option for more complex
+virtual networks in {pve} clusters.
+
+WARNING: It's discourage to use the Debian traditional tools `ifup` and `ifdown`
+if unsure, as they have some pitfalls like interupting all guest traffic on
+`ifdown vmbrX` but not reconnecting those guest again when doing `ifup` on the
+same bridge later.
Apply Network Changes
~~~~~~~~~~~~~~~~~~~~~
are correct before applying, as a wrong network configuration may render a node
inaccessible.
-Reboot Node to apply
-^^^^^^^^^^^^^^^^^^^^
-
-With the default installed `ifupdown` network managing package you need to
-reboot to commit any pending network changes. Most of the time, the basic {pve}
-network setup is stable and does not change often, so rebooting should not be
-required often.
-
-Reload Network with ifupdown2
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-With the optional `ifupdown2` network managing package you also can reload the
-network configuration live, without requiring a reboot.
+Live-Reload Network with ifupdown2
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-NOTE: 'ifupdown2' cannot understand 'OpenVSwitch' syntax, so reloading is *not*
-possible if OVS interfaces are configured.
+With the recommended 'ifupdown2' package (default for new installations since
+{pve} 7.0), it is possible to apply network configuration changes without a
+reboot. If you change the network configuration via the GUI, you can click the
+'Apply Configuration' button. This will move changes from the staging
+`interfaces.new` file to `/etc/network/interfaces` and apply them live.
-Since {pve} 6.1 you can apply pending network changes over the web-interface,
-using the 'Apply Configuration' button in the 'Network' panel of a node.
+If you made manual changes directly to the `/etc/network/interfaces` file, you
+can apply them by running `ifreload -a`
-To install 'ifupdown2' ensure you have the latest {pve} updates installed, then
+NOTE: If you installed {pve} on top of Debian, or upgraded to {pve} 7.0 from an
+older {pve} installation, make sure 'ifupdown2' is installed: `apt install
+ifupdown2`
-WARNING: installing 'ifupdown2' will remove 'ifupdown', but as the removal
-scripts of 'ifupdown' before version '0.8.35+pve1' have a issue where network
-is fully stopped on removal footnote:[Introduced with Debian Buster:
-https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945877] you *must* ensure
-that you have a up to date 'ifupdown' package version.
-
-For the installation itself you can then simply do:
-
- apt install ifupdown2
+Reboot Node to Apply
+^^^^^^^^^^^^^^^^^^^^
-With that you're all set. You can also switch back to the 'ifupdown' variant at
-any time, if you run into issues.
+Another way to apply a new network configuration is to reboot the node.
+In that case the systemd service `pvenetcommit` will activate the staging
+`interfaces.new` file before the `networking` service will apply that
+configuration.
Naming Conventions
~~~~~~~~~~~~~~~~~~
auto vmbr0
iface vmbr0 inet static
- address 192.168.10.2
- netmask 255.255.255.0
+ address 192.168.10.2/24
gateway 192.168.10.1
bridge-ports eno1
bridge-stp off
having its own MAC, even though there is only one network cable
connecting all of these VMs to the network.
+[[sysadmin_network_routed]]
Routed Configuration
~~~~~~~~~~~~~~~~~~~~
[thumbnail="default-network-setup-routed.svg"]
A common scenario is that you have a public IP (assume `198.51.100.5`
for this example), and an additional IP block for your VMs
-(`203.0.113.16/29`). We recommend the following setup for such
+(`203.0.113.16/28`). We recommend the following setup for such
situations:
----
auto lo
iface lo inet loopback
-auto eno1
-iface eno1 inet static
- address 198.51.100.5
- netmask 255.255.255.0
+auto eno0
+iface eno0 inet static
+ address 198.51.100.5/29
gateway 198.51.100.1
post-up echo 1 > /proc/sys/net/ipv4/ip_forward
- post-up echo 1 > /proc/sys/net/ipv4/conf/eno1/proxy_arp
+ post-up echo 1 > /proc/sys/net/ipv4/conf/eno0/proxy_arp
auto vmbr0
iface vmbr0 inet static
- address 203.0.113.17
- netmask 255.255.255.248
+ address 203.0.113.17/28
bridge-ports none
bridge-stp off
bridge-fd 0
----
+[[sysadmin_network_masquerading]]
Masquerading (NAT) with `iptables`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
auto eno1
#real IP address
iface eno1 inet static
- address 198.51.100.5
- netmask 255.255.255.0
+ address 198.51.100.5/24
gateway 198.51.100.1
auto vmbr0
#private sub network
iface vmbr0 inet static
- address 10.10.10.1
- netmask 255.255.255.0
+ address 10.10.10.1/24
bridge-ports none
bridge-stp off
bridge-fd 0
https://blog.lobraun.de/2019/05/19/prox/[Blog post with a good explanation by using TRACE in the raw table]
-
+[[sysadmin_network_bond]]
Linux Bond
~~~~~~~~~~
iface eno2 inet manual
+iface eno3 inet manual
+
auto bond0
iface bond0 inet static
- slaves eno1 eno2
- address 192.168.1.2
- netmask 255.255.255.0
+ bond-slaves eno1 eno2
+ address 192.168.1.2/24
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
auto vmbr0
iface vmbr0 inet static
- address 10.10.10.2
- netmask 255.255.255.0
+ address 10.10.10.2/24
gateway 10.10.10.1
- bridge-ports eno1
+ bridge-ports eno3
bridge-stp off
bridge-fd 0
auto bond0
iface bond0 inet manual
- slaves eno1 eno2
+ bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
auto vmbr0
iface vmbr0 inet static
- address 10.10.10.2
- netmask 255.255.255.0
+ address 10.10.10.2/24
gateway 10.10.10.1
bridge-ports bond0
bridge-stp off
----
+[[sysadmin_network_vlan]]
VLAN 802.1Q
~~~~~~~~~~~
auto vmbr0v5
iface vmbr0v5 inet static
- address 10.10.10.2
- netmask 255.255.255.0
+ address 10.10.10.2/24
gateway 10.10.10.1
bridge-ports eno1.5
bridge-stp off
auto vmbr0.5
iface vmbr0.5 inet static
- address 10.10.10.2
- netmask 255.255.255.0
+ address 10.10.10.2/24
gateway 10.10.10.1
auto vmbr0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
+ bridge-vids 2-4094
----
The next example is the same setup but a bond is used to
auto bond0
iface bond0 inet manual
- slaves eno1 eno2
+ bond-slaves eno1 eno2
bond-miimon 100
bond-mode 802.3ad
bond-xmit-hash-policy layer2+3
auto vmbr0v5
iface vmbr0v5 inet static
- address 10.10.10.2
- netmask 255.255.255.0
+ address 10.10.10.2/24
gateway 10.10.10.1
bridge-ports bond0.5
bridge-stp off
----
+Disabling IPv6 on the Node
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+{pve} works correctly in all environments, irrespective of whether IPv6 is
+deployed or not. We recommend leaving all settings at the provided defaults.
+
+Should you still need to disable support for IPv6 on your node, do so by
+creating an appropriate `sysctl.conf (5)` snippet file and setting the proper
+https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt[sysctls],
+for example adding `/etc/sysctl.d/disable-ipv6.conf` with content:
+
+----
+net.ipv6.conf.all.disable_ipv6 = 1
+net.ipv6.conf.default.disable_ipv6 = 1
+----
+
+This method is preferred to disabling the loading of the IPv6 module on the
+https://www.kernel.org/doc/Documentation/networking/ipv6.rst[kernel commandline].
+
////
TODO: explain IPv6 support?
TODO: explain OVS
projects / pve-docs.git / blobdiff