fixup: add missing newlines

[pve-docs.git] / pvesdn.adoc

diff --git a/pvesdn.adoc b/pvesdn.adoc
index 111653388cf6b87d31f22b4e98a8c5b892a9884a..223ba6de526ad0f535475e83824e302d4e92ab13 100644 (file)
--- a/pvesdn.adoc
+++ b/pvesdn.adoc
@@ -17,25 +17,21 @@ xref:getting_help[mailing lists or in the forum] for questions and feedback.
 Installation
 ------------
 
 Installation
 ------------
 

-To enable the experimental SDN integration, you need to install
-"libpve-network-perl" package
+To enable the experimental SDN integration, you need to install the
+`libpve-network-perl` and `ifupdown2` package on every node:

 
 ----
 
 ----

-apt install libpve-network-perl
+apt update
+apt install libpve-network-perl ifupdown2

 ----
 
 ----
 

-You need to have `ifupdown2` package installed on each node to manage local
-configuration reloading without reboot:
+After that you need to add the following line:

 
 

-----
-apt install ifupdown2
-----
-
-You need to add

 ----
 source /etc/network/interfaces.d/*
 ----
 ----
 source /etc/network/interfaces.d/*
 ----

-at the end of /etc/network/interfaces to have the sdn config included
+at the end of the `/etc/network/interfaces` configuration file, so that the SDN
+config gets included and activated.

 
 
 Basic Overview
 
 
 Basic Overview


@@ -51,7 +47,7 @@ features, advantages or disadvantages.
 Normally a 'VNet' shows up as a common Linux bridge with either a VLAN or
 'VXLAN' tag, but some can also use layer 3 routing for control.
 The 'VNets' are deployed locally on each node, after configuration was committed
 Normally a 'VNet' shows up as a common Linux bridge with either a VLAN or
 'VXLAN' tag, but some can also use layer 3 routing for control.
 The 'VNets' are deployed locally on each node, after configuration was committed

-from the cluster wide datacenter SDN administration interface.
+from the cluster-wide datacenter SDN administration interface.

 
 
 Main configuration
 
 
 Main configuration


@@ -73,10 +69,12 @@ And some options:
 
 * Controller: For complex setups to control Layer 3 routing
 
 
 * Controller: For complex setups to control Layer 3 routing
 

-* Ipams: Allow to use external tools for ip managements (vm/ct ips)
+* Sub-nets: Used to defined ip networks on VNets.

 
 

-* Dns: Allow to define a dns server api for register vm/ct hostname/ip addresses
+* IPAM: Allow to use external tools for IP address management (guest IPs)

 
 

+* DNS: Allow to define a DNS server api for registering a virtual guests
+  hostname and IP-addresses

 
 [[pvesdn_config_main_sdn]]
 
 
 [[pvesdn_config_main_sdn]]
 


@@ -87,7 +85,7 @@ This is the main status panel. Here you can see deployment status of zones on
 different nodes.
 
 There is an 'Apply' button, to push and reload local configuration on all
 different nodes.
 
 There is an 'Apply' button, to push and reload local configuration on all

-cluster nodes nodes.
+cluster nodes.

 
 
 [[pvesdn_local_deployment_monitoring]]
 
 
 [[pvesdn_local_deployment_monitoring]]


@@ -127,17 +125,19 @@ specific zone and only the VNets in that zone
 Common options
 ~~~~~~~~~~~~~~
 
 Common options
 ~~~~~~~~~~~~~~
 

+The following options are available for all zone types.
+

 nodes:: Deploy and allow to use a VNets configured for this Zone only on these
 nodes.
 
 nodes:: Deploy and allow to use a VNets configured for this Zone only on these
 nodes.
 

-Ipam:: Optional, if you want to use an ipam tool to manage ips in  this zone
+ipam:: Optional, if you want to use an ipam tool to manage ips in  this zone

 
 

-Dns:: Optional, dns api server.
+dns:: Optional, dns api server.

 
 

-ReverseDns:: Optional, reverse dns api server.
+reversedns:: Optional, reverse dns api server.

 
 

-Dnszone:: Optional, dns domain name. Use to register hostname like  <hostname>.<domain>
-           The dns zone need to be already existing in dns server.
+dnszone:: Optional, dns domain name. Use to register hostname like
+`<hostname>.<domain>`. The dns zone need to be already existing in dns server.

 
 
 [[pvesdn_zone_plugin_simple]]
 
 
 [[pvesdn_zone_plugin_simple]]


@@ -178,6 +178,8 @@ bridge:: A local VLAN-aware bridge already configured on each local node
 
 service vlan:: The main VLAN tag of this zone
 
 
 service vlan:: The main VLAN tag of this zone
 

+service vlan protocol:: allow to define a 802.1q (default) or 802.1ad service vlan type.
+

 mtu:: Due to the double stacking of tags you need 4 more bytes for QinQ VLANs.
 For example, you reduce the MTU to `1496` if you physical interface MTU is
 `1500`.
 mtu:: Due to the double stacking of tags you need 4 more bytes for QinQ VLANs.
 For example, you reduce the MTU to `1496` if you physical interface MTU is
 `1500`.


@@ -218,19 +220,30 @@ Routing and Forwarding) interface.
 
 Specific EVPN configuration options:
 
 
 Specific EVPN configuration options:
 

-VRF VXLAN Tag:: This is a vxlan-id used for routing interconnect between vnets,
+VRF VXLAN tag:: This is a vxlan-id used for routing interconnect between vnets,

 it must be different than VXLAN-id of VNets
 
 controller:: an EVPN-controller need to be defined first (see controller
 plugins section)
 
 it must be different than VXLAN-id of VNets
 
 controller:: an EVPN-controller need to be defined first (see controller
 plugins section)
 

+VNet MAC address:: A unique anycast MAC address for all VNets in this zone.
+  Will be auto-generated if not defined.

 
 

-Exit Nodes:: This is used if you want to defined some proxmox nodes, as
-             exit gateway from evpn network through real network. This nodes
-             will announce a default route in the evpn network.
+Exit Nodes:: This is used if you want to define some proxmox nodes, as exit
+  gateway from evpn network through real network. The configured nodes will
+  announce a default route in the EVPN network.

 
 

-mtu:: because VXLAN encapsulation use 50bytes, the MTU need to be 50 bytes
-lower than the outgoing physical interface.
+Advertise Subnets:: Optional. If you have silent vms/CT (for example, multiples
+  ips by interfaces, and the anycast gateway don't see traffic from theses ips,
+  the ips addresses won't be able to be reach inside the evpn network). This
+  option will announce the full subnet in the evpn network in this case.
+
+Exit Nodes local routing:: Optional. This is a special option if you need to
+  reach a vm/ct service from an exit node. (By default, the exit nodes only
+  allow forwarding traffic between real network and evpn network).
+
+MTU:: because VXLAN encapsulation use 50 bytes, the MTU needs to be 50 bytes
+  lower than the maximal MTU of the outgoing physical interface.

 
 
 [[pvesdn_config_vnet]]
 
 
 [[pvesdn_config_vnet]]


@@ -255,18 +268,22 @@ VLAN Aware:: Allow to add an extra VLAN tag in the virtual machine or
 
 [[pvesdn_config_subnet]]
 
 
 [[pvesdn_config_subnet]]
 

-Subnets
-~~~~~~~
+Sub-Nets
+~~~~~~~~
+
+A sub-network (subnet or sub-net) allows you to define a specific IP network
+(IPv4 or IPv6). For each VNET, you can define one or more subnets.

 
 

-For each Vnet, you can define 1 or multiple subnets to define an ip network (ipv4 or ipv6).
+A subnet can be used to:

 
 

-It can be used to restrict ip addresses you can define on a specific vnet,
-assign routes/gateway on vnet in layer3 zones,
-enable snat in layer 3 zones,
-auto assign ips on vm/ct through ipam plugin && dns registration through dns plugins.
+* restrict IP-addresses you can define on a specific VNET
+* assign routes/gateway on a VNET in layer 3 zones
+* enable SNAT on a VNET in layer 3 zones
+* auto assign IPs on virtual guests (VM or CT) through IPAM plugin
+* DNS registration through DNS plugins

 
 

-If an ipam server is associated to the subnet zone, the subnet prefix will be automatically
-registered in the ipam.
+If an IPAM server is associated to the subnet zone, the subnet prefix will be
+automatically registered in the IPAM.

 
 
 Subnet properties are:
 
 
 Subnet properties are:


@@ -274,17 +291,15 @@ Subnet properties are:
 ID:: a cidr network address. Ex: 10.0.0.0/8
 
 Gateway:: ip address for the default gateway of the network. 
 ID:: a cidr network address. Ex: 10.0.0.0/8
 
 Gateway:: ip address for the default gateway of the network. 

-           On layer3 zones (simple/evpn plugins), it'll be deployed on the vnet.
-           
+          On layer3 zones (simple/evpn plugins), it'll be deployed on the vnet.
+

 Snat:: Optional, Enable Snat for layer3 zones (simple/evpn plugins) for this subnet.
 Snat:: Optional, Enable Snat for layer3 zones (simple/evpn plugins) for this subnet.

-        The subnet source ip will be natted to server outgoing interface/ip.
-        On evpn zone, it's done only on evpn gateway-nodes.
+       The subnet source ip will be natted to server outgoing interface/ip.
+       On evpn zone, it's done only on evpn gateway-nodes.

 
 Dnszoneprefix:: Optional, add a prefix to domain registration, like <hostname>.prefix.<domain>
 
 
 
 Dnszoneprefix:: Optional, add a prefix to domain registration, like <hostname>.prefix.<domain>
 
 

-
-

 [[pvesdn_config_controllers]]
 Controllers
 -----------
 [[pvesdn_config_controllers]]
 Controllers
 -----------


@@ -321,28 +336,29 @@ BGP Controller
 The bgp controller is not used directly by a zone. 
 You can used it to configure frr to manage bgp peers.
 
 The bgp controller is not used directly by a zone. 
 You can used it to configure frr to manage bgp peers.
 

-For Bgp-evpn, it can be use to define a different ASN by node,
-so doing ebgp.
+For BGP-evpn, it can be use to define a different ASN by node, so doing EBGP.

 
 Configuration options:
 
 
 Configuration options:
 

-asn:: A unique BGP ASN number. It's highly recommended to use private ASN
-number (64512 – 65534, 4200000000 – 4294967294), as else you could end up
-breaking, or get broken, by global routing by mistake.
+node:: The node of this BGP controller

 
 

-peers:: An ip list of peers where you want to communicate for the underlay 
-BGP network 
+asn:: A unique BGP ASN number. It's highly recommended to use private ASN
+  number from the range (64512 - 65534) or (4200000000 - 4294967294), as else
+  you could end up breaking, or get broken, by global routing by mistake.

 
 

-ebgp:: if your peers remote-as is different, it's enabling ebgp.
+peers:: An IP list of peers you want to communicate with for the underlying
+  BGP network.

 
 

-node:: the node of this bgp controller
+ebgp:: If your peer's remote-AS is different, it's enabling EBGP.

 
 

-loopback:: If you want to use a loopback or dummy interface as source
-           for the evpn network. (for multipath)
+loopback:: If you want to use a loopback or dummy interface as source for the
+  evpn network. (for multipath)

 
 

+ebgp-mutltihop:: if the peers are not directly connected or use loopback, you can increase the
+  number of hops to reach them.

 
 [[pvesdn_config_ipam]]
 
 [[pvesdn_config_ipam]]

-Ipams
+IPAMs

 -----
 IPAM (IP address management) tools, are used to manage/assign ips on your devices on the network.
 It can be used to find free ip address when you create a vm/ct for example (not yet implemented).
 -----
 IPAM (IP address management) tools, are used to manage/assign ips on your devices on the network.
 It can be used to find free ip address when you create a vm/ct for example (not yet implemented).


@@ -351,51 +367,58 @@ An IPAM is associated to 1 or multiple zones, to provide ip addresses for all su
 
 
 [[pvesdn_ipam_plugin_pveipam]]
 
 
 [[pvesdn_ipam_plugin_pveipam]]

-PVEIpam plugin
-~~~~~~~~~~~~~~
+{pve} IPAM plugin
+~~~~~~~~~~~~~~~~~

 
 

-This is the default internal ipam for your proxmox cluster if you don't have external ipam software
+This is the default internal IPAM for your proxmox cluster if you don't have
+external ipam software

 
 [[pvesdn_ipam_plugin_phpipam]]
 
 [[pvesdn_ipam_plugin_phpipam]]

-PHPIpam plugin
+phpIPAM plugin

 ~~~~~~~~~~~~~~
 https://phpipam.net/
 
 ~~~~~~~~~~~~~~
 https://phpipam.net/
 

-You need to create an application in phpipam, and add an api token with admin permission
+You need to create an application in phpipam, and add an api token with admin
+permission

 
 

-PHPipam properties are:
+phpIPAM properties are:

 
 

-* Url: The rest api url : http://phpipam.domain.com/api/<appname>/
-* Token: your api token
-* Section: An integer id. Sections are group of subnets in phpipam. 
-           Default install have sectionid=1 for customers
+url:: The REST-API endpoint: `http://phpipam.domain.com/api/<appname>/`
+token:: An API access token
+section:: An integer ID. Sections are group of subnets in phpIPAM. Default
+ installations use `sectionid=1` for customers.

 
 [[pvesdn_ipam_plugin_netbox]]
 
 [[pvesdn_ipam_plugin_netbox]]

-Netbox Ipam plugin
+Netbox IPAM plugin

 ~~~~~~~~~~~~~~~~~~
 ~~~~~~~~~~~~~~~~~~

+
+NetBox is an IP address management (IPAM) and data center infrastructure
+management (DCIM) tool, see the source code repository for details:

 https://github.com/netbox-community/netbox
 
 https://github.com/netbox-community/netbox
 

-you need to create an api token in netbox
+You need to create an api token in netbox

 https://netbox.readthedocs.io/en/stable/api/authentication
 
 https://netbox.readthedocs.io/en/stable/api/authentication
 

-PHPipam properties are:
+NetBox properties are:

 
 

-Url:: The rest api url: http://yournetbox.domain.com/api
-Token:: your api token
+url:: The REST API endpoint: `http://yournetbox.domain.com/api`
+token:: An API access token

 
 [[pvesdn_config_dns]]
 
 [[pvesdn_config_dns]]

-Dns
+DNS

 ---
 ---

-Dns is used to define a dns api server for registration of your hostname/ip address
-an DNS is associated to 1 or multiple zones, to provide dns registration 
-for all ips in subnets defined in this zone.
+
+The DNS plugin in {pve} SDN is used to define a DNS API server for registration
+of your hostname and IP-address. A DNS configuration is associated with one or
+more zones, to provide DNS registration for all the sub-net IPs configured for
+a zone.

 
 [[pvesdn_dns_plugin_powerdns]]
 
 [[pvesdn_dns_plugin_powerdns]]

-Powerdns plugin
+PowerDNS plugin

 ~~~~~~~~~~~~~~~
 https://doc.powerdns.com/authoritative/http-api/index.html
 
 ~~~~~~~~~~~~~~~
 https://doc.powerdns.com/authoritative/http-api/index.html
 

-you need to enable webserver && api in your powerdns config:
+You need to enable the webserver and the API in your PowerDNS config:

 
 ----
 api=yes
 
 ----
 api=yes


@@ -406,9 +429,9 @@ webserver-port=8081
 
 Powerdns properties are:
 
 
 Powerdns properties are:
 

-Url:: The rest api url: http://yourpowerdnserver.domain.com:8081/api/v1/servers/localhost
-key:: the api key
-ttl:: default ttl for records
+url:: The REST API endpoint: http://yourpowerdnserver.domain.com:8081/api/v1/servers/localhost
+key:: An API access key
+ttl:: The default TTL for records

 
 
 Examples
 
 
 Examples


@@ -793,12 +816,12 @@ peers: 192.168.0.1,192.168.0.2,192.168.0.3
 Create an EVPN zone named `myevpnzone' using the previously created
 EVPN-controller Define 'node1' and 'node2' as exit nodes.
 
 Create an EVPN zone named `myevpnzone' using the previously created
 EVPN-controller Define 'node1' and 'node2' as exit nodes.
 

-

 ----
 id: myevpnzone
 vrf vxlan tag: 10000
 controller: myevpnctl
 mtu: 1450
 ----
 id: myevpnzone
 vrf vxlan tag: 10000
 controller: myevpnctl
 mtu: 1450

+vnet mac address: 32:F4:05:FE:6C:0A

 exitnodes: node1,node2
 ----
 
 exitnodes: node1,node2
 ----
 


@@ -807,28 +830,28 @@ Create the first VNet named `myvnet1' using the EVPN zone `myevpnzone'.
 id: myvnet1
 zone: myevpnzone
 tag: 11000
 id: myvnet1
 zone: myevpnzone
 tag: 11000

-mac address: 8C:73:B2:7B:F9:60 #random generate mac address

 ----
 
 ----
 

-Create a subnet 10.0.1.0/24 with 10.0.1.1 as gateway
+Create a subnet 10.0.1.0/24 with 10.0.1.1 as gateway on vnet1
+

 ----
 ----

-id: 10.0.1.0/24
+subnet: 10.0.1.0/24

 gateway: 10.0.1.1
 ----
 
 Create the second VNet named `myvnet2' using the same EVPN zone `myevpnzone', a
 gateway: 10.0.1.1
 ----
 
 Create the second VNet named `myvnet2' using the same EVPN zone `myevpnzone', a

-different IPv4 CIDR network and a different random MAC address than  `myvnet1'.
+different IPv4 CIDR network.

 
 ----
 id: myvnet2
 zone: myevpnzone
 tag: 12000
 
 ----
 id: myvnet2
 zone: myevpnzone
 tag: 12000

-mac address: 8C:73:B2:7B:F9:61  #random mac, need to be different on each vnet

 ----
 
 ----
 

-Create a different subnet 10.0.2.0/24 with 10.0.2.1 as gateway
+Create a different subnet 10.0.2.0/24 with 10.0.2.1 as gateway on vnet2
+

 ----
 ----

-id: 10.0.2.0/24
+subnet: 10.0.2.0/24

 gateway: 10.0.2.1
 ----
 
 gateway: 10.0.2.1
 ----
 


@@ -836,7 +859,6 @@ gateway: 10.0.2.1
 Apply the configuration on the main SDN web-interface panel to create VNets
 locally on each nodes and generate the FRR config.
 
 Apply the configuration on the main SDN web-interface panel to create VNets
 locally on each nodes and generate the FRR config.
 

-

 Create a Debian-based Virtual Machine (vm1) on node1, with a vNIC on `myvnet1'.
 
 Use the following network configuration for this VM:
 Create a Debian-based Virtual Machine (vm1) on node1, with a vNIC on `myvnet1'.
 
 Use the following network configuration for this VM:


@@ -876,3 +898,58 @@ public network can reply back.
 
 If you have configured an external BGP router, the BGP-EVPN routes (10.0.1.0/24
 and 10.0.2.0/24 in this example), will be announced dynamically.
 
 If you have configured an external BGP router, the BGP-EVPN routes (10.0.1.0/24
 and 10.0.2.0/24 in this example), will be announced dynamically.

+
+
+Notes
+-----
+
+VXLAN IPSEC Encryption
+~~~~~~~~~~~~~~~~~~~~~~
+If you need to add encryption on top of VXLAN, it's possible to do so with
+IPSEC through `strongswan`. You'll need to reduce the 'MTU' by 60 bytes (IPv4)
+or 80 bytes (IPv6) to handle encryption.
+
+So with default real 1500 MTU, you need to use a MTU of 1370 (1370 + 80 (IPSEC)
++ 50 (VXLAN) == 1500).
+
+.Install strongswan
+----
+apt install strongswan
+----
+
+Add configuration in `/etc/ipsec.conf'. We only need to encrypt traffic from
+the VXLAN UDP port '4789'.
+
+----
+conn %default
+    ike=aes256-sha1-modp1024!  # the fastest, but reasonably secure cipher on modern HW
+    esp=aes256-sha1!
+    leftfirewall=yes           # this is necessary when using Proxmox VE firewall rules
+
+conn output
+    rightsubnet=%dynamic[udp/4789]
+    right=%any
+    type=transport
+    authby=psk
+    auto=route
+
+conn input
+    leftsubnet=%dynamic[udp/4789]
+    type=transport
+    authby=psk
+    auto=route
+----
+
+Then generate a preshared key with
+
+----
+openssl rand -base64 128
+----
+
+and copy the key in `/etc/ipsec.secrets' so that the file content looks like:
+
+----
+: PSK <generatedbase64key>
+----
+
+You need to copy the PSK and the config on other nodes.