Installation
------------
-To enable the experimental SDN integration, you need to install
-"libpve-network-perl" package
+To enable the experimental SDN integration, you need to install the
+`libpve-network-perl` and `ifupdown2` package on every node:
----
-apt install libpve-network-perl
+apt update
+apt install libpve-network-perl ifupdown2
----
-You need to have `ifupdown2` package installed on each node to manage local
-configuration reloading without reboot:
+After that you need to add the following line:
-----
-apt install ifupdown2
-----
-
-You need to add
----
source /etc/network/interfaces.d/*
----
-at the end of /etc/network/interfaces to have the sdn config included
+at the end of the `/etc/network/interfaces` configuration file, so that the SDN
+config gets included and activated.
Basic Overview
Normally a 'VNet' shows up as a common Linux bridge with either a VLAN or
'VXLAN' tag, but some can also use layer 3 routing for control.
The 'VNets' are deployed locally on each node, after configuration was committed
-from the cluster wide datacenter SDN administration interface.
+from the cluster-wide datacenter SDN administration interface.
Main configuration
* Controller: For complex setups to control Layer 3 routing
-* Ipams: Allow to use external tools for ip managements (vm/ct ips)
+* Sub-nets: Used to defined ip networks on VNets.
-* Dns: Allow to define a dns server api for register vm/ct hostname/ip addresses
+* IPAM: Allow to use external tools for IP address management (guest IPs)
+* DNS: Allow to define a DNS server api for registering a virtual guests
+ hostname and IP-addresses
[[pvesdn_config_main_sdn]]
different nodes.
There is an 'Apply' button, to push and reload local configuration on all
-cluster nodes nodes.
+cluster nodes.
[[pvesdn_local_deployment_monitoring]]
Common options
~~~~~~~~~~~~~~
+The following options are available for all zone types.
+
nodes:: Deploy and allow to use a VNets configured for this Zone only on these
nodes.
-Ipam:: Optional, if you want to use an ipam tool to manage ips in this zone
+ipam:: Optional, if you want to use an ipam tool to manage ips in this zone
-Dns:: Optional, dns api server.
+dns:: Optional, dns api server.
-ReverseDns:: Optional, reverse dns api server.
+reversedns:: Optional, reverse dns api server.
-Dnszone:: Optional, dns domain name. Use to register hostname like <hostname>.<domain>
- The dns zone need to be already existing in dns server.
+dnszone:: Optional, dns domain name. Use to register hostname like
+`<hostname>.<domain>`. The dns zone need to be already existing in dns server.
[[pvesdn_zone_plugin_simple]]
service vlan:: The main VLAN tag of this zone
+service vlan protocol:: allow to define a 802.1q (default) or 802.1ad service vlan type.
+
mtu:: Due to the double stacking of tags you need 4 more bytes for QinQ VLANs.
For example, you reduce the MTU to `1496` if you physical interface MTU is
`1500`.
Specific EVPN configuration options:
-VRF VXLAN Tag:: This is a vxlan-id used for routing interconnect between vnets,
+VRF VXLAN tag:: This is a vxlan-id used for routing interconnect between vnets,
it must be different than VXLAN-id of VNets
controller:: an EVPN-controller need to be defined first (see controller
plugins section)
+VNet MAC address:: A unique anycast MAC address for all VNets in this zone.
+ Will be auto-generated if not defined.
-Exit Nodes:: This is used if you want to defined some proxmox nodes, as
- exit gateway from evpn network through real network. This nodes
- will announce a default route in the evpn network.
+Exit Nodes:: This is used if you want to define some proxmox nodes, as exit
+ gateway from evpn network through real network. The configured nodes will
+ announce a default route in the EVPN network.
-mtu:: because VXLAN encapsulation use 50bytes, the MTU need to be 50 bytes
-lower than the outgoing physical interface.
+Advertise Subnets:: Optional. If you have silent vms/CT (for example, multiples
+ ips by interfaces, and the anycast gateway don't see traffic from theses ips,
+ the ips addresses won't be able to be reach inside the evpn network). This
+ option will announce the full subnet in the evpn network in this case.
+
+Exit Nodes local routing:: Optional. This is a special option if you need to
+ reach a vm/ct service from an exit node. (By default, the exit nodes only
+ allow forwarding traffic between real network and evpn network).
+
+MTU:: because VXLAN encapsulation use 50 bytes, the MTU needs to be 50 bytes
+ lower than the maximal MTU of the outgoing physical interface.
[[pvesdn_config_vnet]]
[[pvesdn_config_subnet]]
-Subnets
-~~~~~~~
+Sub-Nets
+~~~~~~~~
+
+A sub-network (subnet or sub-net) allows you to define a specific IP network
+(IPv4 or IPv6). For each VNET, you can define one or more subnets.
-For each Vnet, you can define 1 or multiple subnets to define an ip network (ipv4 or ipv6).
+A subnet can be used to:
-It can be used to restrict ip addresses you can define on a specific vnet,
-assign routes/gateway on vnet in layer3 zones,
-enable snat in layer 3 zones,
-auto assign ips on vm/ct through ipam plugin && dns registration through dns plugins.
+* restrict IP-addresses you can define on a specific VNET
+* assign routes/gateway on a VNET in layer 3 zones
+* enable SNAT on a VNET in layer 3 zones
+* auto assign IPs on virtual guests (VM or CT) through IPAM plugin
+* DNS registration through DNS plugins
-If an ipam server is associated to the subnet zone, the subnet prefix will be automatically
-registered in the ipam.
+If an IPAM server is associated to the subnet zone, the subnet prefix will be
+automatically registered in the IPAM.
Subnet properties are:
ID:: a cidr network address. Ex: 10.0.0.0/8
Gateway:: ip address for the default gateway of the network.
- On layer3 zones (simple/evpn plugins), it'll be deployed on the vnet.
-
+ On layer3 zones (simple/evpn plugins), it'll be deployed on the vnet.
+
Snat:: Optional, Enable Snat for layer3 zones (simple/evpn plugins) for this subnet.
- The subnet source ip will be natted to server outgoing interface/ip.
- On evpn zone, it's done only on evpn gateway-nodes.
+ The subnet source ip will be natted to server outgoing interface/ip.
+ On evpn zone, it's done only on evpn gateway-nodes.
Dnszoneprefix:: Optional, add a prefix to domain registration, like <hostname>.prefix.<domain>
-
-
[[pvesdn_config_controllers]]
Controllers
-----------
The bgp controller is not used directly by a zone.
You can used it to configure frr to manage bgp peers.
-For Bgp-evpn, it can be use to define a different ASN by node,
-so doing ebgp.
+For BGP-evpn, it can be use to define a different ASN by node, so doing EBGP.
Configuration options:
-asn:: A unique BGP ASN number. It's highly recommended to use private ASN
-number (64512 – 65534, 4200000000 – 4294967294), as else you could end up
-breaking, or get broken, by global routing by mistake.
+node:: The node of this BGP controller
-peers:: An ip list of peers where you want to communicate for the underlay
-BGP network
+asn:: A unique BGP ASN number. It's highly recommended to use private ASN
+ number from the range (64512 - 65534) or (4200000000 - 4294967294), as else
+ you could end up breaking, or get broken, by global routing by mistake.
-ebgp:: if your peers remote-as is different, it's enabling ebgp.
+peers:: An IP list of peers you want to communicate with for the underlying
+ BGP network.
-node:: the node of this bgp controller
+ebgp:: If your peer's remote-AS is different, it's enabling EBGP.
-loopback:: If you want to use a loopback or dummy interface as source
- for the evpn network. (for multipath)
+loopback:: If you want to use a loopback or dummy interface as source for the
+ evpn network. (for multipath)
+ebgp-mutltihop:: if the peers are not directly connected or use loopback, you can increase the
+ number of hops to reach them.
[[pvesdn_config_ipam]]
-Ipams
+IPAMs
-----
IPAM (IP address management) tools, are used to manage/assign ips on your devices on the network.
It can be used to find free ip address when you create a vm/ct for example (not yet implemented).
[[pvesdn_ipam_plugin_pveipam]]
-PVEIpam plugin
-~~~~~~~~~~~~~~
+{pve} IPAM plugin
+~~~~~~~~~~~~~~~~~
-This is the default internal ipam for your proxmox cluster if you don't have external ipam software
+This is the default internal IPAM for your proxmox cluster if you don't have
+external ipam software
[[pvesdn_ipam_plugin_phpipam]]
-PHPIpam plugin
+phpIPAM plugin
~~~~~~~~~~~~~~
https://phpipam.net/
-You need to create an application in phpipam, and add an api token with admin permission
+You need to create an application in phpipam, and add an api token with admin
+permission
-PHPipam properties are:
+phpIPAM properties are:
-* Url: The rest api url : http://phpipam.domain.com/api/<appname>/
-* Token: your api token
-* Section: An integer id. Sections are group of subnets in phpipam.
- Default install have sectionid=1 for customers
+url:: The REST-API endpoint: `http://phpipam.domain.com/api/<appname>/`
+token:: An API access token
+section:: An integer ID. Sections are group of subnets in phpIPAM. Default
+ installations use `sectionid=1` for customers.
[[pvesdn_ipam_plugin_netbox]]
-Netbox Ipam plugin
+Netbox IPAM plugin
~~~~~~~~~~~~~~~~~~
+
+NetBox is an IP address management (IPAM) and data center infrastructure
+management (DCIM) tool, see the source code repository for details:
https://github.com/netbox-community/netbox
-you need to create an api token in netbox
+You need to create an api token in netbox
https://netbox.readthedocs.io/en/stable/api/authentication
-PHPipam properties are:
+NetBox properties are:
-Url:: The rest api url: http://yournetbox.domain.com/api
-Token:: your api token
+url:: The REST API endpoint: `http://yournetbox.domain.com/api`
+token:: An API access token
[[pvesdn_config_dns]]
-Dns
+DNS
---
-Dns is used to define a dns api server for registration of your hostname/ip address
-an DNS is associated to 1 or multiple zones, to provide dns registration
-for all ips in subnets defined in this zone.
+
+The DNS plugin in {pve} SDN is used to define a DNS API server for registration
+of your hostname and IP-address. A DNS configuration is associated with one or
+more zones, to provide DNS registration for all the sub-net IPs configured for
+a zone.
[[pvesdn_dns_plugin_powerdns]]
-Powerdns plugin
+PowerDNS plugin
~~~~~~~~~~~~~~~
https://doc.powerdns.com/authoritative/http-api/index.html
-you need to enable webserver && api in your powerdns config:
+You need to enable the webserver and the API in your PowerDNS config:
----
api=yes
Powerdns properties are:
-Url:: The rest api url: http://yourpowerdnserver.domain.com:8081/api/v1/servers/localhost
-key:: the api key
-ttl:: default ttl for records
+url:: The REST API endpoint: http://yourpowerdnserver.domain.com:8081/api/v1/servers/localhost
+key:: An API access key
+ttl:: The default TTL for records
Examples
Create an EVPN zone named `myevpnzone' using the previously created
EVPN-controller Define 'node1' and 'node2' as exit nodes.
-
----
id: myevpnzone
vrf vxlan tag: 10000
controller: myevpnctl
mtu: 1450
+vnet mac address: 32:F4:05:FE:6C:0A
exitnodes: node1,node2
----
id: myvnet1
zone: myevpnzone
tag: 11000
-mac address: 8C:73:B2:7B:F9:60 #random generate mac address
----
-Create a subnet 10.0.1.0/24 with 10.0.1.1 as gateway
+Create a subnet 10.0.1.0/24 with 10.0.1.1 as gateway on vnet1
+
----
-id: 10.0.1.0/24
+subnet: 10.0.1.0/24
gateway: 10.0.1.1
----
Create the second VNet named `myvnet2' using the same EVPN zone `myevpnzone', a
-different IPv4 CIDR network and a different random MAC address than `myvnet1'.
+different IPv4 CIDR network.
----
id: myvnet2
zone: myevpnzone
tag: 12000
-mac address: 8C:73:B2:7B:F9:61 #random mac, need to be different on each vnet
----
-Create a different subnet 10.0.2.0/24 with 10.0.2.1 as gateway
+Create a different subnet 10.0.2.0/24 with 10.0.2.1 as gateway on vnet2
+
----
-id: 10.0.2.0/24
+subnet: 10.0.2.0/24
gateway: 10.0.2.1
----
Apply the configuration on the main SDN web-interface panel to create VNets
locally on each nodes and generate the FRR config.
-
Create a Debian-based Virtual Machine (vm1) on node1, with a vNIC on `myvnet1'.
Use the following network configuration for this VM:
If you have configured an external BGP router, the BGP-EVPN routes (10.0.1.0/24
and 10.0.2.0/24 in this example), will be announced dynamically.
+
+
+Notes
+-----
+
+VXLAN IPSEC Encryption
+~~~~~~~~~~~~~~~~~~~~~~
+If you need to add encryption on top of VXLAN, it's possible to do so with
+IPSEC through `strongswan`. You'll need to reduce the 'MTU' by 60 bytes (IPv4)
+or 80 bytes (IPv6) to handle encryption.
+
+So with default real 1500 MTU, you need to use a MTU of 1370 (1370 + 80 (IPSEC)
++ 50 (VXLAN) == 1500).
+
+.Install strongswan
+----
+apt install strongswan
+----
+
+Add configuration in `/etc/ipsec.conf'. We only need to encrypt traffic from
+the VXLAN UDP port '4789'.
+
+----
+conn %default
+ ike=aes256-sha1-modp1024! # the fastest, but reasonably secure cipher on modern HW
+ esp=aes256-sha1!
+ leftfirewall=yes # this is necessary when using Proxmox VE firewall rules
+
+conn output
+ rightsubnet=%dynamic[udp/4789]
+ right=%any
+ type=transport
+ authby=psk
+ auto=route
+
+conn input
+ leftsubnet=%dynamic[udp/4789]
+ type=transport
+ authby=psk
+ auto=route
+----
+
+Then generate a preshared key with
+
+----
+openssl rand -base64 128
+----
+
+and copy the key in `/etc/ipsec.secrets' so that the file content looks like:
+
+----
+: PSK <generatedbase64key>
+----
+
+You need to copy the PSK and the config on other nodes.
projects / pve-docs.git / blobdiff