4 This is currently not included, because
5 - it requires ifupdown2
6 - routing needs more documentation
11 VXLAN layer2 with vlan unware linux bridges
12 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
14 VXLAN is an overlay network to carry Ethernet traffic over an existing IP network
15 while accommodating a very large number of tenants. It is defined in RFC 7348.
16 Each overlay network is known as a VXLAN Segment and identified by a unique
17 24-bit segment ID called a VXLAN Network Identifier (VNI).
19 For BUM traffic (broadcast / unknown unicast traffic, multicast),
20 we have 3 differents vxlan setup modes : multicast, unicast, bgp-evpn
22 image::images/vxlan-l2-vlanunaware.svg["vxlan l2 bridge vlan unaware",align="center"]
27 This scenario relies in head end replication, meaning that end host in case
28 of not having any entry for the destination MAC address will send out an ARP
29 to other devices / VTEPs in the VXLAN network.
30 This is done by sending the request to the VXLAN multicast group,
31 remote VTEPs will get the packet and answer accordingly direct to the originating VTEP.
38 iface eno1 inet manual
41 iface vmbr0 inet static
49 iface vxlan2 inet manual
50 vxlan-svcnodeip 225.20.1.1
54 iface vmbr2 inet manual
60 iface vxlan3 inet manual
61 vxlan-svcnodeip 225.20.1.1
65 iface vmbr3 inet manual
76 iface eno1 inet manual
79 iface vmbr0 inet static
87 iface vxlan2 inet manual
88 vxlan-svcnodeip 225.20.1.1
92 iface vmbr2 inet manual
99 iface vxlan3 inet manual
100 vxlan-svcnodeip 225.20.1.1
104 iface vmbr3 inet manual
115 iface eno1 inet manual
118 iface vmbr0 inet static
120 netmask 255.255.255.0
126 iface vxlan2 inet manual
127 vxlan-svcnodeip 225.20.1.1
131 iface vmbr2 inet manual
138 iface vxlan3 inet manual
139 vxlan-svcnodeip 225.20.1.1
143 iface vmbr3 inet manual
153 We can replace multicast by head-end replication of BUM frames to a statically configured lists of remote VTEPs.
154 The VXLAN is defined without a remote multicast group.
155 Instead, all the remote VTEPs are associated with the all-zero address:
156 a BUM frame will be duplicated to all these destinations.
157 The VXLAN device will still learn remote addresses automatically using source-address learning.
163 iface eno1 inet manual
166 iface vmbr0 inet static
168 netmask 255.255.255.0
175 iface vxlan2 inet manual
176 vxlan_remoteip 192.168.0.2
177 vxlan_remoteip 192.168.0.3
181 iface vmbr2 inet manual
188 iface vxlan2 inet manual
189 vxlan_remoteip 192.168.0.2
190 vxlan_remoteip 192.168.0.3
194 iface vmbr3 inet manual
205 iface eno1 inet manual
208 iface vmbr0 inet static
210 netmask 255.255.255.0
216 iface vxlan2 inet manual
217 vxlan_remoteip 192.168.0.1
218 vxlan_remoteip 192.168.0.3
223 iface vmbr2 inet manual
229 iface vxlan2 inet manual
230 vxlan_remoteip 192.168.0.1
231 vxlan_remoteip 192.168.0.3
235 iface vmbr3 inet manual
246 iface eno1 inet manual
249 iface vmbr0 inet static
251 netmask 255.255.255.0
257 iface vxlan2 inet manual
258 vxlan_remoteip 192.168.0.2
259 vxlan_remoteip 192.168.0.3
264 iface vmbr2 inet manual
270 iface vxlan2 inet manual
271 vxlan_remoteip 192.168.0.2
272 vxlan_remoteip 192.168.0.3
276 iface vmbr3 inet manual
286 VTEPs use control plane learning/distribution via BGP for remote MAC addresses instead of data plane learning.
287 VTEPs have the ability to suppress ARP flooding over VXLAN tunnels.
289 The control plane used here is FRR, a bgp routing software.
290 Each node in the proxmox cluster peer with each others nodes.
291 For bigger networks, or multiple proxmox clusters,
292 it's possible to use external bgp route reflector servers.
298 iface eno1 inet manual
301 iface vmbr0 inet static
303 netmask 255.255.255.0
309 iface vxlan2 inet manual
310 vxlan-local-tunnelip 192.168.0.1
312 bridge-arp-nd-suppress on
313 bridge-unicast-flood off
314 bridge-multicast-flood off
318 iface vmbr2 inet manual
325 iface vxlan3 inet manual
326 vxlan-local-tunnelip 192.168.0.1
328 bridge-arp-nd-suppress on
329 bridge-unicast-flood off
330 bridge-multicast-flood off
334 iface vmbr3 inet manual
345 no bgp default ipv4-unicast
347 neighbor 192.168.0.2 remote-as 1234
348 neighbor 192.168.0.3 remote-as 1234
350 address-family l2vpn evpn
351 neighbor 192.168.0.2 activate
352 neighbor 192.168.0.3 activate
365 iface eno1 inet manual
368 iface vmbr0 inet static
370 netmask 255.255.255.0
376 iface vxlan2 inet manual
377 vxlan-local-tunnelip 192.168.0.2
379 bridge-arp-nd-suppress on
380 bridge-unicast-flood off
381 bridge-multicast-flood off
385 iface vmbr2 inet manual
391 iface vxlan3 inet manual
392 vxlan-local-tunnelip 192.168.0.2
394 bridge-arp-nd-suppress on
395 bridge-unicast-flood off
396 bridge-multicast-flood off
400 iface vmbr3 inet manual
411 no bgp default ipv4-unicast
413 neighbor 192.168.0.1 remote-as 1234
414 neighbor 192.168.0.3 remote-as 1234
416 address-family l2vpn evpn
417 neighbor 192.168.0.1 activate
418 neighbor 192.168.0.3 activate
431 iface eno1 inet manual
434 iface vmbr0 inet static
436 netmask 255.255.255.0
442 iface vxlan2 inet manual
443 vxlan-local-tunnelip 192.168.0.3
445 bridge-arp-nd-suppress on
446 bridge-unicast-flood off
447 bridge-multicast-flood off
451 iface vmbr2 inet manual
457 iface vxlan3 inet manual
458 vxlan-local-tunnelip 192.168.0.3
460 bridge-arp-nd-suppress on
461 bridge-unicast-flood off
462 bridge-multicast-flood off
466 iface vmbr3 inet manual
478 no bgp default ipv4-unicast
480 neighbor 192.168.0.1 remote-as 1234
481 neighbor 192.168.0.2 remote-as 1234
483 address-family l2vpn evpn
484 neighbor 192.168.0.1 activate
485 neighbor 192.168.0.2 activate
494 VXLAN layer2 with vlan aware linux bridges
495 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
497 We use 1 vmbr bridge, each vxlan is mapped to a vlan
499 image::images/vxlan-l2-vlanaware.svg["vxlan l2 bridge vlan aware",align="center"]
508 iface eno1 inet manual
511 iface vmbr0 inet static
513 netmask 255.255.255.0
514 bridge_ports eno1 vxlan2 vxlan3
517 bridge_vlan_aware yes
520 iface vxlan2 inet manual
521 vxlan-svcnodeip 225.20.1.1
526 iface vxlan3 inet manual
527 vxlan-svcnodeip 225.20.1.1
537 iface eno1 inet manual
540 iface vmbr0 inet static
542 netmask 255.255.255.0
543 bridge_ports eno1 vxlan2 vxlan3
546 bridge_vlan_aware yes
549 iface vxlan2 inet manual
550 vxlan-svcnodeip 225.20.1.1
555 iface vxlan3 inet manual
556 vxlan-svcnodeip 225.20.1.1
566 iface eno1 inet manual
569 iface vmbr0 inet static
571 netmask 255.255.255.0
572 bridge_ports eno1 vxlan2 vxlan3
575 bridge_vlan_aware yes
578 iface vxlan2 inet manual
579 vxlan-svcnodeip 225.20.1.1
584 iface vxlan3 inet manual
585 vxlan-svcnodeip 225.20.1.1
598 iface eno1 inet manual
601 iface vmbr0 inet static
603 netmask 255.255.255.0
604 bridge_ports eno1 vxlan2 vxlan3
607 bridge_vlan_aware yes
610 iface vxlan2 inet manual
611 vxlan_remoteip 192.168.0.2
612 vxlan_remoteip 192.168.0.3
616 iface vxlan3 inet manual
617 vxlan_remoteip 192.168.0.2
618 vxlan_remoteip 192.168.0.3
627 iface eno1 inet manual
630 iface vmbr0 inet static
632 netmask 255.255.255.0
633 bridge_ports eno1 vxlan2 vxlan3
636 bridge_vlan_aware yes
639 iface vxlan2 inet manual
640 vxlan_remoteip 192.168.0.1
641 vxlan_remoteip 192.168.0.3
645 iface vxlan3 inet manual
646 vxlan_remoteip 192.168.0.1
647 vxlan_remoteip 192.168.0.3
656 iface eno1 inet manual
659 iface vmbr0 inet static
661 netmask 255.255.255.0
662 bridge_ports eno1 vxlan2 vxlan3
665 bridge_vlan_aware yes
668 iface vxlan2 inet manual
669 vxlan_remoteip 192.168.0.2
670 vxlan_remoteip 192.168.0.3
674 iface vxlan3 inet manual
675 vxlan_remoteip 192.168.0.2
676 vxlan_remoteip 192.168.0.3
684 Note: currently FRR is working only with 1 vlan aware bridge
691 iface eno1 inet manual
694 iface vmbr0 inet static
696 netmask 255.255.255.0
697 bridge_ports eno1 vxlan2 vxlan3
700 bridge_vlan_aware yes
703 iface vxlan0 inet manual
704 vxlan-local-tunnelip 192.168.0.1
706 bridge-arp-nd-suppress on
707 bridge-unicast-flood off
708 bridge-multicast-flood off
713 iface vxlan3 inet manual
714 vxlan-local-tunnelip 192.168.0.1
716 bridge-arp-nd-suppress on
717 bridge-unicast-flood off
718 bridge-multicast-flood off
727 no bgp default ipv4-unicast
729 neighbor 192.168.0.2 remote-as 1234
730 neighbor 192.168.0.3 remote-as 1234
732 address-family l2vpn evpn
733 neighbor 192.168.0.2 activate
734 neighbor 192.168.0.3 activate
747 iface eno1 inet manual
750 iface vmbr0 inet static
752 netmask 255.255.255.0
753 bridge_ports eno1 vxlan2 vxlan3
756 bridge_vlan_aware yes
759 iface vxlan0 inet manual
760 vxlan-local-tunnelip 192.168.0.2
762 bridge-arp-nd-suppress on
763 bridge-unicast-flood off
764 bridge-multicast-flood off
769 iface vxlan3 inet manual
770 vxlan-local-tunnelip 192.168.0.2
772 bridge-arp-nd-suppress on
773 bridge-unicast-flood off
774 bridge-multicast-flood off
783 no bgp default ipv4-unicast
785 neighbor 192.168.0.1 remote-as 1234
786 neighbor 192.168.0.3 remote-as 1234
788 address-family l2vpn evpn
789 neighbor 192.168.0.1 activate
790 neighbor 192.168.0.3 activate
803 iface eno1 inet manual
806 iface vmbr0 inet static
808 netmask 255.255.255.0
809 bridge_ports eno1 vxlan2 vxlan3
812 bridge_vlan_aware yes
815 iface vxlan0 inet manual
816 vxlan-local-tunnelip 192.168.0.3
818 bridge-arp-nd-suppress on
819 bridge-unicast-flood off
820 bridge-multicast-flood off
825 iface vxlan3 inet manual
826 vxlan-local-tunnelip 192.168.0.3
828 bridge-arp-nd-suppress on
829 bridge-unicast-flood off
830 bridge-multicast-flood off
838 no bgp default ipv4-unicast
840 neighbor 192.168.0.1 remote-as 1234
841 neighbor 192.168.0.2 remote-as 1234
843 address-family l2vpn evpn
844 neighbor 192.168.0.1 activate
845 neighbor 192.168.0.2 activate