X-Git-Url: https://git.proxmox.com/?p=pve-docs.git;a=blobdiff_plain;f=pve-firewall.adoc;h=2bcdf6e0c93e95d7c421598e2e3b1188d215c5e7;hp=7089778ccf86e304e8adb987fa370fae6bac5750;hb=HEAD;hpb=b92c45ab817375dba8aaf2cb0a01909aada2929c diff --git a/pve-firewall.adoc b/pve-firewall.adoc index 7089778..9fb4e46 100644 --- a/pve-firewall.adoc +++ b/pve-firewall.adoc @@ -84,7 +84,7 @@ name enclosed in `[` and `]`. Cluster Wide Setup ~~~~~~~~~~~~~~~~~~ -The cluster wide firewall configuration is stored at: +The cluster-wide firewall configuration is stored at: /etc/pve/firewall/cluster.fw @@ -92,13 +92,13 @@ The configuration can contain the following sections: `[OPTIONS]`:: -This is used to set cluster wide firewall options. +This is used to set cluster-wide firewall options. include::pve-firewall-cluster-opts.adoc[] `[RULES]`:: -This sections contains cluster wide firewall rules for all nodes. +This sections contains cluster-wide firewall rules for all nodes. `[IPSET ]`:: @@ -121,7 +121,7 @@ set the enable option here: ---- [OPTIONS] -# enable firewall (cluster wide setting, default is disabled) +# enable firewall (cluster-wide setting, default is disabled) enable: 1 ---- @@ -324,7 +324,7 @@ Standard IP set `management` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This IP set applies only to host firewalls (not VM firewalls). Those -IPs are allowed to do normal management tasks (PVE GUI, VNC, SPICE, +IPs are allowed to do normal management tasks ({PVE} GUI, VNC, SPICE, SSH). The local cluster network is automatically added to this IP set (alias @@ -379,6 +379,7 @@ discovery protocol to work. ---- +[[pve_firewall_services_commands]] Services and Commands --------------------- @@ -426,7 +427,7 @@ following traffic is still allowed for all {pve} hosts in the cluster: * TCP traffic from management hosts to port 3128 for connections to the SPICE proxy * TCP traffic from management hosts to port 22 to allow ssh access -* UDP traffic in the cluster network to port 5404 and 5405 for corosync +* UDP traffic in the cluster network to ports 5405-5412 for corosync * UDP multicast traffic in the cluster network * ICMP traffic type 3 (Destination Unreachable), 4 (congestion control) or 11 (Time Exceeded) @@ -435,7 +436,7 @@ The following traffic is dropped, but not logged even with logging enabled: * TCP connections with invalid connection state * Broadcast, multicast and anycast traffic not related to corosync, i.e., not - coming through port 5404 or 5405 + coming through ports 5405-5412 * TCP traffic to port 43 * UDP traffic to ports 135 and 445 * UDP traffic to the port range 137 to 139 @@ -457,7 +458,7 @@ Please inspect the output of the system command to see the firewall chains and rules active on your system. This output is also included in a `System Report`, accessible over a node's -subscription tab in the web GUI, or through the `pvereport` command line tool. +subscription tab in the web GUI, or through the `pvereport` command-line tool. VM/CT incoming/outgoing DROP/REJECT ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -475,7 +476,7 @@ Logging of firewall rules ------------------------- By default, all logging of traffic filtered by the firewall rules is disabled. -To enable logging, the `loglevel` for incommig and/or outgoing traffic has to be +To enable logging, the `loglevel` for incoming and/or outgoing traffic has to be set in *Firewall* -> *Options*. This can be done for the host as well as for the VM/CT firewall individually. By this, logging of {PVE}'s standard firewall rules is enabled and the output can be observed in *Firewall* -> *Log*. @@ -521,14 +522,14 @@ This allows to log in a fine grained manner and independent of the log-level defined for the standard rules in *Firewall* -> *Options*. While the `loglevel` for each individual rule can be defined or changed easily -in the WebUI during creation or modification of the rule, it is possible to set +in the web UI during creation or modification of the rule, it is possible to set this also via the corresponding `pvesh` API calls. Further, the log-level can also be set via the firewall configuration file by appending a `-log ` to the selected rule (see xref:pve_firewall_log_levels[possible log-levels]). -For example, the following two are ident: +For example, the following two are identical: ---- IN REJECT -p icmp -log nolog @@ -562,7 +563,7 @@ and add `ip_conntrack_ftp` to `/etc/modules` (so that it works after a reboot). Suricata IPS integration ~~~~~~~~~~~~~~~~~~~~~~~~ -If you want to use the http://suricata-ids.org/[Suricata IPS] +If you want to use the https://suricata.io/[Suricata IPS] (Intrusion Prevention System), it's possible. Packets will be forwarded to the IPS only after the firewall ACCEPTed @@ -634,9 +635,189 @@ Ports used by {pve} * sshd (used for cluster actions): 22 (TCP) * rpcbind: 111 (UDP) * sendmail: 25 (TCP, outgoing) -* corosync cluster traffic: 5404, 5405 UDP +* corosync cluster traffic: 5405-5412 UDP * live migration (VM memory and local-disk data): 60000-60050 (TCP) + +nftables +-------- + +As an alternative to `pve-firewall` we offer `proxmox-firewall`, which is an +implementation of the Proxmox VE firewall based on the newer +https://wiki.nftables.org/wiki-nftables/index.php/What_is_nftables%3F[nftables] +rather than iptables. + +WARNING: `proxmox-firewall` is currently in tech preview. There might be bugs or +incompatibilies with the original firewall. It is currently not suited for +production use. + +This implementation uses the same configuration files and configuration format, +so you can use your old configuration when switching. It provides the exact same +functionality with a few exceptions: + +* REJECT is currently not possible for guest traffic (traffic will instead be + dropped). +* Using the `NDP`, `Router Advertisement` or `DHCP` options will *always* create + firewall rules, irregardless of your default policy. +* firewall rules for guests are evaluated even for connections that have + conntrack table entries. + + +Installation and Usage +~~~~~~~~~~~~~~~~~~~~~~ + +Install the `proxmox-firewall` package: + +---- +apt install proxmox-firewall +---- + +Enable the nftables backend via the Web UI on your hosts (Host > Firewall > +Options > nftables), or by enabling it in the configuration file for your hosts +(`/etc/pve/nodes//host.fw`): + +---- +[OPTIONS] + +nftables: 1 +---- + +NOTE: After enabling/disabling `proxmox-firewall`, all running VMs and +containers need to be restarted for the old/new firewall to work properly. + +After setting the `nftables` configuration key, the new `proxmox-firewall` +service will take over. You can check if the new service is working by +checking the systemctl status of `proxmox-firewall`: + +---- +systemctl status proxmox-firewall +---- + +You can also examine the generated ruleset. You can find more information about +this in the section xref:pve_firewall_nft_helpful_commands[Helpful Commands]. +You should also check whether `pve-firewall` is no longer generating iptables +rules, you can find the respective commands in the +xref:pve_firewall_services_commands[Services and Commands] section. + +Switching back to the old firewall can be done by simply setting the +configuration value back to 0 / No. + +Usage +~~~~~ + +`proxmox-firewall` will create two tables that are managed by the +`proxmox-firewall` service: `proxmox-firewall` and `proxmox-firewall-guests`. If +you want to create custom rules that live outside the Proxmox VE firewall +configuration you can create your own tables to manage your custom firewall +rules. `proxmox-firewall` will only touch the tables it generates, so you can +easily extend and modify the behavior of the `proxmox-firewall` by adding your +own tables. + +Instead of using the `pve-firewall` command, the nftables-based firewall uses +`proxmox-firewall`. It is a systemd service, so you can start and stop it via +`systemctl`: + +---- +systemctl start proxmox-firewall +systemctl stop proxmox-firewall +---- + +Stopping the firewall service will remove all generated rules. + +To query the status of the firewall, you can query the status of the systemctl +service: + +---- +systemctl status proxmox-firewall +---- + + +[[pve_firewall_nft_helpful_commands]] +Helpful Commands +~~~~~~~~~~~~~~~~ +You can check the generated ruleset via the following command: + +---- +nft list ruleset +---- + +If you want to debug `proxmox-firewall` you can simply run the daemon in +foreground with the `RUST_LOG` environment variable set to `trace`. This should +provide you with detailed debugging output: + +---- +RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall +---- + +You can also edit the systemctl service if you want to have detailed output for +your firewall daemon: + +---- +systemctl edit proxmox-firewall +---- + +Then you need to add the override for the `RUST_LOG` environment variable: + +---- +[Service] +Environment="RUST_LOG=trace" +---- + +This will generate a large amount of logs very quickly, so only use this for +debugging purposes. Other, less verbose, log levels are `info` and `debug`. + +Running in foreground writes the log output to STDERR, so you can redirect it +with the following command (e.g. for submitting logs to the community forum): + +---- +RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall 2> firewall_log_$(hostname).txt +---- + +It can be helpful to trace packet flow through the different chains in order to +debug firewall rules. This can be achieved by setting `nftrace` to 1 for packets +that you want to track. It is advisable that you do not set this flag for *all* +packets, in the example below we only examine ICMP packets. + +---- +#!/usr/sbin/nft -f +table bridge tracebridge +delete table bridge tracebridge + +table bridge tracebridge { + chain trace { + meta l4proto icmp meta nftrace set 1 + } + + chain prerouting { + type filter hook prerouting priority -350; policy accept; + jump trace + } + + chain postrouting { + type filter hook postrouting priority -350; policy accept; + jump trace + } +} +---- + +Saving this file, making it executable, and then running it once will create the +respective tracing chains. You can then inspect the tracing output via the +Proxmox VE Web UI (Firewall > Log) or via `nft monitor trace`. + +The above example traces traffic on all bridges, which is usually where guest +traffic flows through. If you want to examine host traffic, create those chains +in the `inet` table instead of the `bridge` table. + +NOTE: Be aware that this can generate a *lot* of log spam and slow down the +performance of your networking stack significantly. + +You can remove the tracing rules via running the following command: + +---- +nft delete table bridge tracebridge +---- + + ifdef::manvolnum[] Macro Definitions