+[[chapter_pve_firewall]]
ifdef::manvolnum[]
-PVE({manvolnum})
-================
-include::attributes.txt[]
+pve-firewall(8)
+===============
+:pve-toplevel:
NAME
----
-pve-firewall - The PVE Firewall Daemon
+pve-firewall - PVE Firewall Daemon
-SYNOPSYS
+SYNOPSIS
--------
include::pve-firewall.8-synopsis.adoc[]
DESCRIPTION
-----------
endif::manvolnum[]
-
ifndef::manvolnum[]
{pve} Firewall
==============
-include::attributes.txt[]
+:pve-toplevel:
endif::manvolnum[]
+ifdef::wiki[]
+:title: Firewall
+endif::wiki[]
-// Copied from pve wiki: Revision as of 08:45, 9 November 2015
-
-Proxmox VE Firewall provides an easy way to protect your IT
-infrastructure. You can easily setup firewall rules for all hosts
+{pve} Firewall provides an easy way to protect your IT
+infrastructure. You can setup firewall rules for all hosts
inside a cluster, or define rules for virtual machines and
containers. Features like firewall macros, security groups, IP sets
-and aliases help making that task easier.
+and aliases help to make that task easier.
While all configuration is stored on the cluster file system, the
-iptables based firewall runs on each cluster node, and thus provides
+`iptables`-based firewall service runs on each cluster node, and thus provides
full isolation between virtual machines. The distributed nature of
this system also provides much higher bandwidth than a central
firewall solution.
-NOTE: If you enable the firewall, all traffic is blocked by default,
-except WebGUI(8006) and ssh(22) from your local network.
+The firewall has full support for IPv4 and IPv6. IPv6 support is fully
+transparent, and we filter traffic for both protocols by default. So
+there is no need to maintain a different set of rules for IPv6.
Zones
outgoing traffic.
-Ports used by Proxmox VE
-------------------------
+Configuration Files
+-------------------
-* Web interface: 8006
-* VNC Web console: 5900-5999
-* SPICE proxy: 3128
-* sshd (used for cluster actions): 22
-* rpcbind: 111
-* corosync multicast (if you run a cluster): 5404, 5405 UDP
+All firewall related configuration is stored on the proxmox cluster
+file system. So those files are automatically distributed to all
+cluster nodes, and the `pve-firewall` service updates the underlying
+`iptables` rules automatically on changes.
+You can configure anything using the GUI (i.e. *Datacenter* -> *Firewall*,
+or on a *Node* -> *Firewall*), or you can edit the configuration files
+directly using your preferred editor.
-Configuration
--------------
+Firewall configuration files contain sections of key-value
+pairs. Lines beginning with a `#` and blank lines are considered
+comments. Sections start with a header line containing the section
+name enclosed in `[` and `]`.
-All firewall related configuration is stored on the proxmox cluster
-file system. So those files are automatically distributed to all
-cluster nodes, and the 'pve-firewall' service updates the underlying
-iptables rules automatically on any change. Any configuration can be
-done using the GUI (i.e. Datacenter -> Firewall -> Options tab (tabs
-at the bottom of the page), or on a Node -> Firewall), so the
-following configuration file snippets are just for completeness.
-Cluster wide configuration is stored at:
+[[pve_firewall_cluster_wide_setup]]
+Cluster Wide Setup
+~~~~~~~~~~~~~~~~~~
+
+The cluster wide firewall configuration is stored at:
/etc/pve/firewall/cluster.fw
-The firewall is completely disabled by default, so you need to set the
-enable option here:
+The configuration can contain the following sections:
+
+`[OPTIONS]`::
+
+This is used to set cluster wide firewall options.
+
+include::pve-firewall-cluster-opts.adoc[]
+
+`[RULES]`::
+
+This sections contains cluster wide firewall rules for all nodes.
+
+`[IPSET <name>]`::
+
+Cluster wide IP set definitions.
+
+`[GROUP <name>]`::
+
+Cluster wide security group definitions.
+
+`[ALIASES]`::
+
+Cluster wide Alias definitions.
+
+
+Enabling the Firewall
+^^^^^^^^^^^^^^^^^^^^^
+
+The firewall is completely disabled by default, so you need to
+set the enable option here:
----
[OPTIONS]
enable: 1
----
-The cluster wide configuration can contain the following data:
+IMPORTANT: If you enable the firewall, traffic to all hosts is blocked by
+default. Only exceptions is WebGUI(8006) and ssh(22) from your local
+network.
+
+If you want to administrate your {pve} hosts from remote, you
+need to create rules to allow traffic from those remote IPs to the web
+GUI (port 8006). You may also want to allow ssh (port 22), and maybe
+SPICE (port 3128).
+
+TIP: Please open a SSH connection to one of your {PVE} hosts before
+enabling the firewall. That way you still have access to the host if
+something goes wrong .
+
+To simplify that task, you can instead create an IPSet called
+``management'', and add all remote IPs there. This creates all required
+firewall rules to access the GUI from remote.
+
+
+[[pve_firewall_host_specific_configuration]]
+Host Specific Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Host related configuration is read from:
+
+ /etc/pve/nodes/<nodename>/host.fw
+
+This is useful if you want to overwrite rules from `cluster.fw`
+config. You can also increase log verbosity, and set netfilter related
+options. The configuration can contain the following sections:
+
+`[OPTIONS]`::
+
+This is used to set host related firewall options.
-* IP set definitions
-* Alias definitions
-* Security group definitions
-* Cluster wide firewall rules for all nodes
+include::pve-firewall-host-opts.adoc[]
+
+`[RULES]`::
+
+This sections contains host specific firewall rules.
+
+[[pve_firewall_vm_container_configuration]]
+VM/Container Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~
VM firewall configuration is read from:
and contains the following data:
-* IP set definitions
-* Alias definitions
-* Firewall rules for this VM
-* VM specific options
+`[OPTIONS]`::
-And finally, any host related configuration is read from:
+This is used to set VM/Container related firewall options.
- /etc/pve/nodes/<nodename>/host.fw
+include::pve-firewall-vm-opts.adoc[]
+
+`[RULES]`::
+
+This sections contains VM/Container firewall rules.
+
+`[IPSET <name>]`::
+
+IP set definitions.
+
+`[ALIASES]`::
+
+IP Alias definitions.
-This is useful if you want to overwrite rules from 'cluster.fw'
-config. You can also increase log verbosity, and set netfilter related
-options.
Enabling the Firewall for VMs and Containers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Each virtual network device has its own firewall enable flag. So you
+can selectively enable the firewall for each interface. This is
+required in addition to the general firewall `enable` option.
-You need to enable the firewall on the virtual network interface configuration
-in addition to the general 'Enable Firewall' option in the 'Options' tab.
Firewall Rules
-~~~~~~~~~~~~~~
+--------------
Firewall rules consists of a direction (`IN` or `OUT`) and an
action (`ACCEPT`, `DENY`, `REJECT`). You can also specify a macro
-name. Macros contain predifined sets of rules and options. Rules can be disabled by prefixing them with '|'.
+name. Macros contain predefined sets of rules and options. Rules can be
+disabled by prefixing them with `|`.
.Firewall rules syntax
----
IN SSH(ACCEPT) -i net0
IN SSH(ACCEPT) -i net0 # a comment
IN SSH(ACCEPT) -i net0 -source 192.168.2.192 # only allow SSH from 192.168.2.192
-IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for ip range
-IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for ip list
+IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for IP range
+IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for IP list
IN SSH(ACCEPT) -i net0 -source +mynetgroup # accept ssh for ipset mynetgroup
IN SSH(ACCEPT) -i net0 -source myserveralias #accept ssh for alias myserveralias
OUT ACCEPT # accept all outgoing packages
----
+
+[[pve_firewall_security_groups]]
Security Groups
-~~~~~~~~~~~~~~~
+---------------
A security group is a collection of rules, defined at cluster level, which
can be used in all VMs' rules. For example you can define a group named
-`webserver` with rules to open the http and https ports.
+``webserver'' with rules to open the 'http' and 'https' ports.
----
# /etc/pve/firewall/cluster.fw
GROUP webserver
----
-
+[[pve_firewall_ip_aliases]]
IP Aliases
-~~~~~~~~~~
+----------
IP Aliases allow you to associate IP addresses of networks with a
name. You can then refer to those names:
* inside IP set definitions
* in `source` and `dest` properties of firewall rules
-Standard IP alias `local_network`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Standard IP Alias `local_network`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This alias is automatically defined. Please use the following command
to see assigned values:
The firewall automatically sets up rules to allow everything needed
for cluster communication (corosync, API, SSH) using this alias.
-The user can overwrite these values in the cluster.fw alias
+The user can overwrite these values in the `cluster.fw` alias
section. If you use a single host on a public network, it is better to
explicitly assign the local IP address
----
# /etc/pve/firewall/cluster.fw
[ALIASES]
-local_network 1.2.3.4 # use the single ip address
+local_network 1.2.3.4 # use the single IP address
----
+[[pve_firewall_ip_sets]]
IP Sets
-~~~~~~~
+-------
IP sets can be used to define groups of networks and hosts. You can
refer to them with `+name` in the firewall rules' `source` and `dest`
IN HTTP(ACCEPT) -source +management
+
Standard IP set `management`
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This IP set applies only to host firewalls (not VM firewalls). Those
-ips are allowed to do normal management tasks (PVE GUI, VNC, SPICE,
+IPs are allowed to do normal management tasks (PVE GUI, VNC, SPICE,
SSH).
The local cluster network is automatically added to this IP set (alias
192.168.2.10/24
----
-Standard IP set 'blacklist'
-^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Traffic from these ips is dropped by every host's and VM's firewall.
+Standard IP set `blacklist`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Traffic from these IPs is dropped by every host's and VM's firewall.
----
# /etc/pve/firewall/cluster.fw
213.87.123.0/24
----
-[[ipfilter-section]]
-Standard IP set 'ipfilter-net*'
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+[[pve_firewall_ipfilter_section]]
+Standard IP set `ipfilter-net*`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
These filters belong to a VM's network interface and are mainly used to prevent
IP spoofing. If such a set exists for an interface then any outgoing traffic
be dropped.
For containers with configured IP addresses these sets, if they exist (or are
-activated via the general `IP Filter` option in the VM's firewall's 'options'
+activated via the general `IP Filter` option in the VM's firewall's *options*
tab), implicitly contain the associated IP addresses.
For both virtual machines and containers they also implicitly contain the
192.168.2.10
----
+
Services and Commands
-~~~~~~~~~~~~~~~~~~~~~
+---------------------
The firewall runs two service daemons on each node:
* pvefw-logger: NFLOG daemon (ulogd replacement).
* pve-firewall: updates iptables rules
-There is also a CLI command named 'pve-firewall', which can be used to
+There is also a CLI command named `pve-firewall`, which can be used to
start and stop the firewall service:
# pve-firewall start
# iptables-save
+[[pve_firewall_default_rules]]
+Default firewall rules
+----------------------
+
+The following traffic is filtered by the default firewall configuration:
+
+Datacenter incoming/outgoing DROP/REJECT
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the input or output policy for the firewall is set to DROP or REJECT, the
+following traffic is still allowed for all {pve} hosts in the cluster:
+
+* traffic over the loopback interface
+* already established connections
+* traffic using the IGMP protocol
+* TCP traffic from management hosts to port 8006 in order to allow access to
+ the web interface
+* TCP traffic from management hosts to the port range 5900 to 5999 allowing
+ traffic for the VNC web console
+* TCP traffic from management hosts to port 3128 for connections to the SPICE
+ proxy
+* TCP traffic from management hosts to port 22 to allow ssh access
+* UDP traffic in the cluster network to port 5404 and 5405 for corosync
+* UDP multicast traffic in the cluster network
+* ICMP traffic type 3 (Destination Unreachable), 4 (congestion control) or 11
+ (Time Exceeded)
+
+The following traffic is dropped, but not logged even with logging enabled:
+
+* TCP connections with invalid connection state
+* Broadcast, multicast and anycast traffic not related to corosync, i.e., not
+ coming through port 5404 or 5405
+* TCP traffic to port 43
+* UDP traffic to ports 135 and 445
+* UDP traffic to the port range 137 to 139
+* UDP traffic form source port 137 to port range 1024 to 65535
+* UDP traffic to port 1900
+* TCP traffic to port 135, 139 and 445
+* UDP traffic originating from source port 53
+
+The rest of the traffic is dropped or rejected, respectively, and also logged.
+This may vary depending on the additional options enabled in
+*Firewall* -> *Options*, such as NDP, SMURFS and TCP flag filtering.
+
+[[pve_firewall_iptables_inspect]]
+Please inspect the output of the
+
+----
+ # iptables-save
+----
+
+system command to see the firewall chains and rules active on your system.
+This output is also included in a `System Report`, accessible over a node's
+subscription tab in the web GUI, or through the `pvereport` command line tool.
+
+VM/CT incoming/outgoing DROP/REJECT
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This drops or rejects all the traffic to the VMs, with some exceptions for
+DHCP, NDP, Router Advertisement, MAC and IP filtering depending on the set
+configuration. The same rules for dropping/rejecting packets are inherited
+from the datacenter, while the exceptions for accepted incomming/outgoing
+traffic of the host do not apply.
+
+Again, you can use xref:pve_firewall_iptables_inspect[iptables-save (see above)]
+to inspect all rules and chains applied.
+
+Logging of firewall rules
+-------------------------
+
+By default, all logging of traffic filtered by the firewall rules is disabled.
+To enable logging, the `loglevel` for incommig and/or outgoing traffic has to be
+set in *Firewall* -> *Options*. This can be done for the host as well as for the
+VM/CT firewall individually. By this, logging of {PVE}'s standard firewall rules
+is enabled and the output can be observed in *Firewall* -> *Log*.
+Further, only some dropped or rejected packets are logged for the standard rules
+(see xref:pve_firewall_default_rules[default firewall rules]).
+
+`loglevel` does not affect how much of the filtered traffic is logged. It
+changes a `LOGID` appended as prefix to the log output for easier filtering and
+post-processing.
+
+`loglevel` is one of the following flags:
+
+[[pve_firewall_log_levels]]
+[width="25%", options="header"]
+|===================
+| loglevel | LOGID
+| nolog | --
+| emerg | 0
+| alert | 1
+| crit | 2
+| err | 3
+| warning | 4
+| notice | 5
+| info | 6
+| debug | 7
+|===================
+
+A typical firewall log output looks like this:
+
+----
+VMID LOGID CHAIN TIMESTAMP POLICY: PACKET_DETAILS
+----
+
+In case of the host firewall, `VMID` is equal to 0.
+
+
+Logging of user defined firewall rules
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In order to log packets filtered by user-defined firewall rules, it is possible
+to set a log-level parameter for each rule individually.
+This allows to log in a fine grained manner and independent of the log-level
+defined for the standard rules in *Firewall* -> *Options*.
+
+While the `loglevel` for each individual rule can be defined or changed easily
+in the WebUI during creation or modification of the rule, it is possible to set
+this also via the corresponding `pvesh` API calls.
+
+Further, the log-level can also be set via the firewall configuration file by
+appending a `-log <loglevel>` to the selected rule (see
+xref:pve_firewall_log_levels[possible log-levels]).
+
+For example, the following two are ident:
+
+----
+IN REJECT -p icmp -log nolog
+IN REJECT -p icmp
+----
+
+whereas
+
+----
+IN REJECT -p icmp -log debug
+----
+
+produces a log output flagged with the `debug` level.
+
+
Tips and Tricks
-~~~~~~~~~~~~~~~
+---------------
How to allow FTP
-^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~
FTP is an old style protocol which uses port 21 and several other dynamic ports. So you
-need a rule to accept port 21. In addition, you need to load the 'ip_conntrack_ftp' module.
+need a rule to accept port 21. In addition, you need to load the `ip_conntrack_ftp` module.
So please run:
modprobe ip_conntrack_ftp
-and add `ip_conntrack_ftp` to '/etc/modules' (so that it works after a reboot) .
+and add `ip_conntrack_ftp` to `/etc/modules` (so that it works after a reboot).
+
Suricata IPS integration
-^^^^^^^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~~~~~~~
If you want to use the http://suricata-ids.org/[Suricata IPS]
(Intrusion Prevention System), it's possible.
# modprobe nfnetlink_queue
----
-Don't forget to add `nfnetlink_queue` to '/etc/modules' for next reboot.
+Don't forget to add `nfnetlink_queue` to `/etc/modules` for next reboot.
Then, enable IPS for a specific VM with:
----
-ifdef::manvolnum[]
-include::copyright.adoc[]
-endif::manvolnum[]
-
Notes on IPv6
-^^^^^^^^^^^^^
+-------------
The firewall contains a few IPv6 specific options. One thing to note is that
IPv6 does not use the ARP protocol anymore, and instead uses NDP (Neighbor
Discovery Protocol) which works on IP level and thus needs IP addresses to
succeed. For this purpose link-local addresses derived from the interface's MAC
-address are used. By default the 'NDP' option is enabled on both host and VM
+address are used. By default the `NDP` option is enabled on both host and VM
level to allow neighbor discovery (NDP) packets to be sent and received.
Beside neighbor discovery NDP is also used for a couple of other things, like
-autoconfiguration and advertising routers.
+auto-configuration and advertising routers.
By default VMs are allowed to send out router solicitation messages (to query
-for a router), and to receive router advetisement packets. This allows them to
+for a router), and to receive router advertisement packets. This allows them to
use stateless auto configuration. On the other hand VMs cannot advertise
-themselves as routers unless the 'Allow Router Advertisement' (`radv: 1`) option
+themselves as routers unless the ``Allow Router Advertisement'' (`radv: 1`) option
is set.
-As for the link local addresses required for NDP, there's also an 'IP Filter'
+As for the link local addresses required for NDP, there's also an ``IP Filter''
(`ipfilter: 1`) option which can be enabled which has the same effect as adding
an `ipfilter-net*` ipset for each of the VM's network interfaces containing the
corresponding link local addresses. (See the
-<<ipfilter-section,Standard IP set 'ipfilter-net*'>> section for details.)
+<<pve_firewall_ipfilter_section,Standard IP set `ipfilter-net*`>> section for details.)
-Avoiding link-local addresses on tap and veth devices
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-With IPv6 enabled by default every interface gets a MAC-derived link local
-address. However, most devices on a typical {pve} setup are connected to a
-bridge and so the bridge is the only interface which really needs one.
+Ports used by {pve}
+-------------------
-To disable a link local address on an interface you can set the interface's
-`disable_ipv6` sysconf variable. Despite the name, this does not prevent IPv6
-traffic from passing through the interface when routing or bridging, so the
-only noticeable effect will be the removal of the link local address.
+* Web interface: 8006
+* VNC Web console: 5900-5999
+* SPICE proxy: 3128
+* sshd (used for cluster actions): 22
+* rpcbind: 111
+* corosync multicast (if you run a cluster): 5404, 5405 UDP
-The easiest method of achieving this setting for all newly started VMs is to
-set it for the `default` interface configuration and enabling it explicitly on
-the interfaces which need it. This is also the case for other settings such as
-`forwarding`, `accept_ra` or `autoconf`.
-Here's a possible setup:
-----
-# /etc/sysconf.d/90-ipv6.conf
+ifdef::manvolnum[]
-net.ipv6.conf.default.forwarding = 0
-net.ipv6.conf.default.proxy_ndp = 0
-net.ipv6.conf.default.autoconf = 0
-net.ipv6.conf.default.disable_ipv6 = 1
-net.ipv6.conf.default.accept_ra = 0
+Macro Definitions
+-----------------
-net.ipv6.conf.lo.disable_ipv6 = 0
-----
+include::pve-firewall-macros.adoc[]
-----
-# /etc/network/interfaces
-(...)
-iface vmbr0 inet6 static
- address fc00::31
- netmask 16
- gateway fc00::1
- accept_ra 0
- pre-up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
-(...)
-----
+
+include::pve-copyright.adoc[]
+
+endif::manvolnum[]
projects / pve-docs.git / blobdiff