]> git.proxmox.com Git - pve-docs.git/blame - vxlan-and-evpn.adoc
projects / pve-docs.git / blame
? search:
summary | shortlog | log | commit | commitdiff | tree
blob | blame (incremental) | history | HEAD
add documentation for vxlan layer 2 network
[pve-docs.git] / vxlan-and-evpn.adoc
CommitLineData
445822a9
DM		 1
2////
3
4This is currently not included, because
5- it requires ifupdown2
6- routing needs more documentation
7
8////
9
10
11VXLAN layer2 with vlan unware linux bridges
12~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13
14VXLAN is an overlay network to carry Ethernet traffic over an existing IP network
15while accommodating a very large number of tenants. It is defined in RFC 7348.
16Each overlay network is known as a VXLAN Segment and identified by a unique
1724-bit segment ID called a VXLAN Network Identifier (VNI).
18
19For BUM traffic (broadcast / unknown unicast traffic, multicast),
20we have 3 differents vxlan setup modes : multicast, unicast, bgp-evpn
21
22image::images/vxlan-l2-vlanunaware.svg["vxlan l2 bridge vlan unaware",align="center"]
23
24multicast mode
25^^^^^^^^^^^^^^
26
27This scenario relies in head end replication, meaning that end host in case
28of not having any entry for the destination MAC address will send out an ARP
29to other devices / VTEPs in the VXLAN network.
30This is done by sending the request to the VXLAN multicast group,
31remote VTEPs will get the packet and answer accordingly direct to the originating VTEP.
32
33
34* node1
35
36----
37auto eno1
38iface eno1 inet manual
39
40auto vmbr0
41iface vmbr0 inet static
42 address 192.168.0.1
43 netmask 255.255.255.0
44 bridge_ports eno1
45 bridge_stp off
46 bridge_fd 0
47
48auto vxlan2
49iface vxlan2 inet manual
50 vxlan-svcnodeip 225.20.1.1
51 vxlan-physdev eno1
52
53auto vmbr2
54iface vmbr2 inet manual
55 bridge_ports vxlan2
56 bridge_stp off
57 bridge_fd 0
58
59auto vxlan3
60iface vxlan3 inet manual
61 vxlan-svcnodeip 225.20.1.1
62 vxlan-physdev eno1
63
64auto vmbr3
65iface vmbr3 inet manual
66 bridge_ports vxlan3
67 bridge_stp off
68 bridge_fd 0
69----
70
71
72* node2
73
74----
75auto eno1
76iface eno1 inet manual
77
78auto vmbr0
79iface vmbr0 inet static
80 address 192.168.0.2
81 netmask 255.255.255.0
82 bridge_ports eno1
83 bridge_stp off
84 bridge_fd 0
85
86auto vxlan2
87iface vxlan2 inet manual
88 vxlan-svcnodeip 225.20.1.1
89 vxlan-physdev eno1
90
91auto vmbr2
92iface vmbr2 inet manual
93 bridge_ports vxlan2
94 bridge_stp off
95 bridge_fd 0
96
97
98auto vxlan3
99iface vxlan3 inet manual
100 vxlan-svcnodeip 225.20.1.1
101 vxlan-physdev eno1
102
103auto vmbr3
104iface vmbr3 inet manual
105 bridge_ports vxlan3
106 bridge_stp off
107 bridge_fd 0
108----
109
110
111* node3
112
113----
114auto eno1
115iface eno1 inet manual
116
117auto vmbr0
118iface vmbr0 inet static
119 address 192.168.0.3
120 netmask 255.255.255.0
121 bridge_ports eno1
122 bridge_stp off
123 bridge_fd 0
124
125auto vxlan2
126iface vxlan2 inet manual
127 vxlan-svcnodeip 225.20.1.1
128 vxlan-physdev eno1
129
130auto vmbr2
131iface vmbr2 inet manual
132 bridge_ports vxlan2
133 bridge_stp off
134 bridge_fd 0
135
136
137auto vxlan3
138iface vxlan3 inet manual
139 vxlan-svcnodeip 225.20.1.1
140 vxlan-physdev eno1
141
142auto vmbr3
143iface vmbr3 inet manual
144 bridge_ports vxlan3
145 bridge_stp off
146 bridge_fd 0
147----
148
149
150unicast mode
151^^^^^^^^^^^^
152
153We can replace multicast by head-end replication of BUM frames to a statically configured lists of remote VTEPs.
154The VXLAN is defined without a remote multicast group.
155Instead, all the remote VTEPs are associated with the all-zero address:
156a BUM frame will be duplicated to all these destinations.
157The VXLAN device will still learn remote addresses automatically using source-address learning.
158
159* node1
160
161----
162auto eno1
163iface eno1 inet manual
164
165auto vmbr0
166iface vmbr0 inet static
167 address 192.168.0.1
168 netmask 255.255.255.0
169 bridge_ports eno1
170 bridge_stp off
171 bridge_fd 0
172
173
174auto vxlan2
175iface vxlan2 inet manual
176 vxlan_remoteip 192.168.0.2
177 vxlan_remoteip 192.168.0.3
178
179
180auto vmbr2
181iface vmbr2 inet manual
182 bridge_ports vxlan2
183 bridge_stp off
184 bridge_fd 0
185
186
187auto vxlan3
188iface vxlan2 inet manual
189 vxlan_remoteip 192.168.0.2
190 vxlan_remoteip 192.168.0.3
191
192
193auto vmbr3
194iface vmbr3 inet manual
195 bridge_ports vxlan3
196 bridge_stp off
197 bridge_fd 0
198----
199
200
201* node2
202
203----
204auto eno1
205iface eno1 inet manual
206
207auto vmbr0
208iface vmbr0 inet static
209 address 192.168.0.2
210 netmask 255.255.255.0
211 bridge_ports eno1
212 bridge_stp off
213 bridge_fd 0
214
215auto vxlan2
216iface vxlan2 inet manual
217 vxlan_remoteip 192.168.0.1
218 vxlan_remoteip 192.168.0.3
219
220
221
222auto vmbr2
223iface vmbr2 inet manual
224 bridge_ports vxlan2
225 bridge_stp off
226 bridge_fd 0
227
228auto vxlan3
229iface vxlan2 inet manual
230 vxlan_remoteip 192.168.0.1
231 vxlan_remoteip 192.168.0.3
232
233
234auto vmbr3
235iface vmbr3 inet manual
236 bridge_ports vxlan3
237 bridge_stp off
238 bridge_fd 0
239----
240
241
242* node3
243
244----
245auto eno1
246iface eno1 inet manual
247
248auto vmbr0
249iface vmbr0 inet static
250 address 192.168.0.3
251 netmask 255.255.255.0
252 bridge_ports eno1
253 bridge_stp off
254 bridge_fd 0
255
256auto vxlan2
257iface vxlan2 inet manual
258 vxlan_remoteip 192.168.0.2
259 vxlan_remoteip 192.168.0.3
260
261
262
263auto vmbr2
264iface vmbr2 inet manual
265 bridge_ports vxlan2
266 bridge_stp off
267 bridge_fd 0
268
269auto vxlan3
270iface vxlan2 inet manual
271 vxlan_remoteip 192.168.0.2
272 vxlan_remoteip 192.168.0.3
273
274
275auto vmbr3
276iface vmbr3 inet manual
277 bridge_ports vxlan3
278 bridge_stp off
279 bridge_fd 0
280----
281
282
283bgp-evpn
284^^^^^^^^
285
286VTEPs use control plane learning/distribution via BGP for remote MAC addresses instead of data plane learning.
287VTEPs have the ability to suppress ARP flooding over VXLAN tunnels.
288
289The control plane used here is FRR, a bgp routing software.
290Each node in the proxmox cluster peer with each others nodes.
291For bigger networks, or multiple proxmox clusters,
292it's possible to use external bgp route reflector servers.
293
294* node1
295
296----
297auto eno1
298iface eno1 inet manual
299
300auto vmbr0
301iface vmbr0 inet static
302 address 192.168.0.1
303 netmask 255.255.255.0
304 bridge_ports eno1
305 bridge_stp off
306 bridge_fd 0
307
308auto vxlan2
309iface vxlan2 inet manual
310 vxlan-local-tunnelip 192.168.0.1
311 bridge-learning off
312 bridge-arp-nd-suppress on
313 bridge-unicast-flood off
314 bridge-multicast-flood off
315
316
317auto vmbr2
318iface vmbr2 inet manual
319 bridge_ports vxlan2
320 bridge_stp off
321 bridge_fd 0
322
323
324auto vxlan3
325iface vxlan3 inet manual
326 vxlan-local-tunnelip 192.168.0.1
327 bridge-learning off
328 bridge-arp-nd-suppress on
329 bridge-unicast-flood off
330 bridge-multicast-flood off
331
332
333auto vmbr3
334iface vmbr3 inet manual
335 bridge_ports vxlan3
336 bridge_stp off
337 bridge_fd 0
338----
339
340
341/etc/frr/frr.conf
342
343----
344router bgp 1234
345 no bgp default ipv4-unicast
346 coalesce-time 1000
347 neighbor 192.168.0.2 remote-as 1234
348 neighbor 192.168.0.3 remote-as 1234
349 !
350 address-family l2vpn evpn
351 neighbor 192.168.0.2 activate
352 neighbor 192.168.0.3 activate
353 advertise-all-vni
354 exit-address-family
355!
356line vty
357!
358----
359
360
361* node2
362
363----
364auto eno1
365iface eno1 inet manual
366
367auto vmbr0
368iface vmbr0 inet static
369 address 192.168.0.2
370 netmask 255.255.255.0
371 bridge_ports eno1
372 bridge_stp off
373 bridge_fd 0
374
375auto vxlan2
376iface vxlan2 inet manual
377 vxlan-local-tunnelip 192.168.0.2
378 bridge-learning off
379 bridge-arp-nd-suppress on
380 bridge-unicast-flood off
381 bridge-multicast-flood off
382
383
384auto vmbr2
385iface vmbr2 inet manual
386 bridge_ports vxlan2
387 bridge_stp off
388 bridge_fd 0
389
390auto vxlan3
391iface vxlan3 inet manual
392 vxlan-local-tunnelip 192.168.0.2
393 bridge-learning off
394 bridge-arp-nd-suppress on
395 bridge-unicast-flood off
396 bridge-multicast-flood off
397
398
399auto vmbr3
400iface vmbr3 inet manual
401 bridge_ports vxlan3
402 bridge_stp off
403 bridge_fd 0
404----
405
406
407/etc/frr/frr.conf
408
409----
410router bgp 1234
411 no bgp default ipv4-unicast
412 coalesce-time 1000
413 neighbor 192.168.0.1 remote-as 1234
414 neighbor 192.168.0.3 remote-as 1234
415 !
416 address-family l2vpn evpn
417 neighbor 192.168.0.1 activate
418 neighbor 192.168.0.3 activate
419 advertise-all-vni
420 exit-address-family
421!
422line vty
423!
424----
425
426
427* node3
428
429----
430auto eno1
431iface eno1 inet manual
432
433auto vmbr0
434iface vmbr0 inet static
435 address 192.168.0.2
436 netmask 255.255.255.0
437 bridge_ports eno1
438 bridge_stp off
439 bridge_fd 0
440
441auto vxlan2
442iface vxlan2 inet manual
443 vxlan-local-tunnelip 192.168.0.3
444 bridge-learning off
445 bridge-arp-nd-suppress on
446 bridge-unicast-flood off
447 bridge-multicast-flood off
448
449
450auto vmbr2
451iface vmbr2 inet manual
452 bridge_ports vxlan2
453 bridge_stp off
454 bridge_fd 0
455
456auto vxlan3
457iface vxlan3 inet manual
458 vxlan-local-tunnelip 192.168.0.3
459 bridge-learning off
460 bridge-arp-nd-suppress on
461 bridge-unicast-flood off
462 bridge-multicast-flood off
463
464
465auto vmbr3
466iface vmbr3 inet manual
467 bridge_ports vxlan3
468 bridge_stp off
469 bridge_fd 0
470----
471
472
473/etc/frr/frr.conf
474
475
476----
477router bgp 1234
478 no bgp default ipv4-unicast
479 coalesce-time 1000
480 neighbor 192.168.0.1 remote-as 1234
481 neighbor 192.168.0.2 remote-as 1234
482 !
483 address-family l2vpn evpn
484 neighbor 192.168.0.1 activate
485 neighbor 192.168.0.2 activate
486 advertise-all-vni
487 exit-address-family
488!
489line vty
490!
491----
492
493
494VXLAN layer2 with vlan aware linux bridges
495~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
496
497We use 1 vmbr bridge, each vxlan is mapped to a vlan
498
499image::images/vxlan-l2-vlanaware.svg["vxlan l2 bridge vlan aware",align="center"]
500
501multicast mode
502^^^^^^^^^^^^^^
503
504* node1
505
506----
507auto eno1
508iface eno1 inet manual
509
510auto vmbr0
511iface vmbr0 inet static
512 address 192.168.0.1
513 netmask 255.255.255.0
514 bridge_ports eno1 vxlan2 vxlan3
515 bridge_stp off
516 bridge_fd 0
517 bridge_vlan_aware yes
518
519auto vxlan2
520iface vxlan2 inet manual
521 vxlan-svcnodeip 225.20.1.1
522 vxlan-physdev eno1
523 bridge-access 2
524
525auto vxlan3
526iface vxlan3 inet manual
527 vxlan-svcnodeip 225.20.1.1
528 vxlan-physdev eno1
529 bridge-access 3
530----
531
532
533* node2
534
535----
536auto eno1
537iface eno1 inet manual
538
539auto vmbr0
540iface vmbr0 inet static
541 address 192.168.0.2
542 netmask 255.255.255.0
543 bridge_ports eno1 vxlan2 vxlan3
544 bridge_stp off
545 bridge_fd 0
546 bridge_vlan_aware yes
547
548auto vxlan2
549iface vxlan2 inet manual
550 vxlan-svcnodeip 225.20.1.1
551 vxlan-physdev eno1
552 bridge-access 2
553
554auto vxlan3
555iface vxlan3 inet manual
556 vxlan-svcnodeip 225.20.1.1
557 vxlan-physdev eno1
558 bridge-access 3
559----
560
561
562* node3
563
564----
565auto eno1
566iface eno1 inet manual
567
568auto vmbr0
569iface vmbr0 inet static
570 address 192.168.0.3
571 netmask 255.255.255.0
572 bridge_ports eno1 vxlan2 vxlan3
573 bridge_stp off
574 bridge_fd 0
575 bridge_vlan_aware yes
576
577auto vxlan2
578iface vxlan2 inet manual
579 vxlan-svcnodeip 225.20.1.1
580 vxlan-physdev eno1
581 bridge-access 2
582
583auto vxlan3
584iface vxlan3 inet manual
585 vxlan-svcnodeip 225.20.1.1
586 vxlan-physdev eno1
587 bridge-access 3
588----
589
590
591unicast mode
592^^^^^^^^^^^^
593
594* node1
595
596----
597auto eno1
598iface eno1 inet manual
599
600auto vmbr0
601iface vmbr0 inet static
602 address 192.168.0.1
603 netmask 255.255.255.0
604 bridge_ports eno1 vxlan2 vxlan3
605 bridge_stp off
606 bridge_fd 0
607 bridge_vlan_aware yes
608
609auto vxlan2
610iface vxlan2 inet manual
611 vxlan_remoteip 192.168.0.2
612 vxlan_remoteip 192.168.0.3
613 bridge-access 2
614
615auto vxlan3
616iface vxlan3 inet manual
617 vxlan_remoteip 192.168.0.2
618 vxlan_remoteip 192.168.0.3
619 bridge-access 3
620----
621
622
623* node2
624
625----
626auto eno1
627iface eno1 inet manual
628
629auto vmbr0
630iface vmbr0 inet static
631 address 192.168.0.2
632 netmask 255.255.255.0
633 bridge_ports eno1 vxlan2 vxlan3
634 bridge_stp off
635 bridge_fd 0
636 bridge_vlan_aware yes
637
638auto vxlan2
639iface vxlan2 inet manual
640 vxlan_remoteip 192.168.0.1
641 vxlan_remoteip 192.168.0.3
642 bridge-access 2
643
644auto vxlan3
645iface vxlan3 inet manual
646 vxlan_remoteip 192.168.0.1
647 vxlan_remoteip 192.168.0.3
648 bridge-access 3
649----
650
651
652* node3
653
654----
655auto eno1
656iface eno1 inet manual
657
658auto vmbr0
659iface vmbr0 inet static
660 address 192.168.0.3
661 netmask 255.255.255.0
662 bridge_ports eno1 vxlan2 vxlan3
663 bridge_stp off
664 bridge_fd 0
665 bridge_vlan_aware yes
666
667auto vxlan2
668iface vxlan2 inet manual
669 vxlan_remoteip 192.168.0.2
670 vxlan_remoteip 192.168.0.3
671 bridge-access 2
672
673auto vxlan3
674iface vxlan3 inet manual
675 vxlan_remoteip 192.168.0.2
676 vxlan_remoteip 192.168.0.3
677 bridge-access 3
678----
679
680
681bgp-evpn
682^^^^^^^^
683
684Note: currently FRR is working only with 1 vlan aware bridge
685
686* node1
687
688
689----
690auto eno1
691iface eno1 inet manual
692
693auto vmbr0
694iface vmbr0 inet static
695 address 192.168.0.1
696 netmask 255.255.255.0
697 bridge_ports eno1 vxlan2 vxlan3
698 bridge_stp off
699 bridge_fd 0
700 bridge_vlan_aware yes
701
702auto vxlan0
703iface vxlan0 inet manual
704 vxlan-local-tunnelip 192.168.0.1
705 bridge-learning off
706 bridge-arp-nd-suppress on
707 bridge-unicast-flood off
708 bridge-multicast-flood off
709 bridge-access 2
710
711
712auto vxlan3
713iface vxlan3 inet manual
714 vxlan-local-tunnelip 192.168.0.1
715 bridge-learning off
716 bridge-arp-nd-suppress on
717 bridge-unicast-flood off
718 bridge-multicast-flood off
719 bridge-access 3
720----
721
722
723/etc/frr/frr.conf
724
725----
726router bgp 1234
727 no bgp default ipv4-unicast
728 coalesce-time 1000
729 neighbor 192.168.0.2 remote-as 1234
730 neighbor 192.168.0.3 remote-as 1234
731 !
732 address-family l2vpn evpn
733 neighbor 192.168.0.2 activate
734 neighbor 192.168.0.3 activate
735 advertise-all-vni
736 exit-address-family
737!
738line vty
739!
740----
741
742
743* node2
744
745----
746auto eno1
747iface eno1 inet manual
748
749auto vmbr0
750iface vmbr0 inet static
751 address 192.168.0.2
752 netmask 255.255.255.0
753 bridge_ports eno1 vxlan2 vxlan3
754 bridge_stp off
755 bridge_fd 0
756 bridge_vlan_aware yes
757
758auto vxlan0
759iface vxlan0 inet manual
760 vxlan-local-tunnelip 192.168.0.2
761 bridge-learning off
762 bridge-arp-nd-suppress on
763 bridge-unicast-flood off
764 bridge-multicast-flood off
765 bridge-access 2
766
767
768auto vxlan3
769iface vxlan3 inet manual
770 vxlan-local-tunnelip 192.168.0.2
771 bridge-learning off
772 bridge-arp-nd-suppress on
773 bridge-unicast-flood off
774 bridge-multicast-flood off
775 bridge-access 3
776----
777
778
779/etc/frr/frr.conf
780
781----
782router bgp 1234
783 no bgp default ipv4-unicast
784 coalesce-time 1000
785 neighbor 192.168.0.1 remote-as 1234
786 neighbor 192.168.0.3 remote-as 1234
787 !
788 address-family l2vpn evpn
789 neighbor 192.168.0.1 activate
790 neighbor 192.168.0.3 activate
791 advertise-all-vni
792 exit-address-family
793!
794line vty
795!
796----
797
798
799* node3
800
801----
802auto eno1
803iface eno1 inet manual
804
805auto vmbr0
806iface vmbr0 inet static
807 address 192.168.0.3
808 netmask 255.255.255.0
809 bridge_ports eno1 vxlan2 vxlan3
810 bridge_stp off
811 bridge_fd 0
812 bridge_vlan_aware yes
813
814auto vxlan0
815iface vxlan0 inet manual
816 vxlan-local-tunnelip 192.168.0.3
817 bridge-learning off
818 bridge-arp-nd-suppress on
819 bridge-unicast-flood off
820 bridge-multicast-flood off
821 bridge-access 2
822
823
824auto vxlan3
825iface vxlan3 inet manual
826 vxlan-local-tunnelip 192.168.0.3
827 bridge-learning off
828 bridge-arp-nd-suppress on
829 bridge-unicast-flood off
830 bridge-multicast-flood off
831 bridge-access 3
832----
833
834
835/etc/frr/frr.conf
836----
837router bgp 1234
838 no bgp default ipv4-unicast
839 coalesce-time 1000
840 neighbor 192.168.0.1 remote-as 1234
841 neighbor 192.168.0.2 remote-as 1234
842 !
843 address-family l2vpn evpn
844 neighbor 192.168.0.1 activate
845 neighbor 192.168.0.2 activate
846 advertise-all-vni
847 exit-address-family
848!
849line vty
850!
851----
PVE Documentation