--- /dev/null
+
+////
+
+This is currently not included, because
+- it requires ifupdown2
+- routing needs more documentation
+
+////
+
+
+VXLAN layer2 with vlan unware linux bridges
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+VXLAN is an overlay network to carry Ethernet traffic over an existing IP network
+while accommodating a very large number of tenants. It is defined in RFC 7348.
+Each overlay network is known as a VXLAN Segment and identified by a unique
+24-bit segment ID called a VXLAN Network Identifier (VNI).
+
+For BUM traffic (broadcast / unknown unicast traffic, multicast),
+we have 3 differents vxlan setup modes : multicast, unicast, bgp-evpn
+
+image::images/vxlan-l2-vlanunaware.svg["vxlan l2 bridge vlan unaware",align="center"]
+
+multicast mode
+^^^^^^^^^^^^^^
+
+This scenario relies in head end replication, meaning that end host in case
+of not having any entry for the destination MAC address will send out an ARP
+to other devices / VTEPs in the VXLAN network.
+This is done by sending the request to the VXLAN multicast group,
+remote VTEPs will get the packet and answer accordingly direct to the originating VTEP.
+
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+unicast mode
+^^^^^^^^^^^^
+
+We can replace multicast by head-end replication of BUM frames to a statically configured lists of remote VTEPs.
+The VXLAN is defined without a remote multicast group.
+Instead, all the remote VTEPs are associated with the all-zero address:
+a BUM frame will be duplicated to all these destinations.
+The VXLAN device will still learn remote addresses automatically using source-address learning.
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+bgp-evpn
+^^^^^^^^
+
+VTEPs use control plane learning/distribution via BGP for remote MAC addresses instead of data plane learning.
+VTEPs have the ability to suppress ARP flooding over VXLAN tunnels.
+
+The control plane used here is FRR, a bgp routing software.
+Each node in the proxmox cluster peer with each others nodes.
+For bigger networks, or multiple proxmox clusters,
+it's possible to use external bgp route reflector servers.
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.2 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.2 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr2
+iface vmbr2 inet manual
+ bridge_ports vxlan2
+ bridge_stp off
+ bridge_fd 0
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+
+
+auto vmbr3
+iface vmbr3 inet manual
+ bridge_ports vxlan3
+ bridge_stp off
+ bridge_fd 0
+----
+
+
+/etc/frr/frr.conf
+
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.2 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.2 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+VXLAN layer2 with vlan aware linux bridges
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We use 1 vmbr bridge, each vxlan is mapped to a vlan
+
+image::images/vxlan-l2-vlanaware.svg["vxlan l2 bridge vlan aware",align="center"]
+
+multicast mode
+^^^^^^^^^^^^^^
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 3
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 3
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-svcnodeip 225.20.1.1
+ vxlan-physdev eno1
+ bridge-access 3
+----
+
+
+unicast mode
+^^^^^^^^^^^^
+
+* node1
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 3
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan_remoteip 192.168.0.1
+ vxlan_remoteip 192.168.0.3
+ bridge-access 3
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan2
+iface vxlan2 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 2
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan_remoteip 192.168.0.2
+ vxlan_remoteip 192.168.0.3
+ bridge-access 3
+----
+
+
+bgp-evpn
+^^^^^^^^
+
+Note: currently FRR is working only with 1 vlan aware bridge
+
+* node1
+
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.1
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan0
+iface vxlan0 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 2
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.1
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 3
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.2 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.2 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node2
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.2
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan0
+iface vxlan0 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 2
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.2
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 3
+----
+
+
+/etc/frr/frr.conf
+
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.3 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.3 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
+
+
+* node3
+
+----
+auto eno1
+iface eno1 inet manual
+
+auto vmbr0
+iface vmbr0 inet static
+ address 192.168.0.3
+ netmask 255.255.255.0
+ bridge_ports eno1 vxlan2 vxlan3
+ bridge_stp off
+ bridge_fd 0
+ bridge_vlan_aware yes
+
+auto vxlan0
+iface vxlan0 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 2
+
+
+auto vxlan3
+iface vxlan3 inet manual
+ vxlan-local-tunnelip 192.168.0.3
+ bridge-learning off
+ bridge-arp-nd-suppress on
+ bridge-unicast-flood off
+ bridge-multicast-flood off
+ bridge-access 3
+----
+
+
+/etc/frr/frr.conf
+----
+router bgp 1234
+ no bgp default ipv4-unicast
+ coalesce-time 1000
+ neighbor 192.168.0.1 remote-as 1234
+ neighbor 192.168.0.2 remote-as 1234
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.0.1 activate
+ neighbor 192.168.0.2 activate
+ advertise-all-vni
+ exit-address-family
+!
+line vty
+!
+----
projects / pve-docs.git / blobdiff