use listings for file contents
[pve-docs.git] / pve-firewall.adoc
CommitLineData
c7eda5e6
DM
1ifdef::manvolnum[]
2PVE({manvolnum})
3================
38fd0958 4include::attributes.txt[]
c7eda5e6
DM
5
6NAME
7----
8
f5eb0727 9pve-firewall - PVE Firewall Daemon
c7eda5e6
DM
10
11
12SYNOPSYS
13--------
14
5f34196d 15include::pve-firewall.8-synopsis.adoc[]
c7eda5e6
DM
16
17
18DESCRIPTION
19-----------
20endif::manvolnum[]
21
22ifndef::manvolnum[]
23{pve} Firewall
24==============
38fd0958 25include::attributes.txt[]
c7eda5e6
DM
26endif::manvolnum[]
27
c7eda5e6 28Proxmox VE Firewall provides an easy way to protect your IT
89a8b6c6 29infrastructure. You can setup firewall rules for all hosts
c7eda5e6
DM
30inside a cluster, or define rules for virtual machines and
31containers. Features like firewall macros, security groups, IP sets
5eba0743 32and aliases help to make that task easier.
c7eda5e6
DM
33
34While all configuration is stored on the cluster file system, the
8c1189b6 35`iptables`-based firewall runs on each cluster node, and thus provides
c7eda5e6
DM
36full isolation between virtual machines. The distributed nature of
37this system also provides much higher bandwidth than a central
38firewall solution.
39
bd73a43e
DM
40The firewall has full support for IPv4 and IPv6. IPv6 support is fully
41transparent, and we filter traffic for both protocols by default. So
42there is no need to maintain a different set of rules for IPv6.
43
c7eda5e6
DM
44
45Zones
46-----
47
48The Proxmox VE firewall groups the network into the following logical zones:
49
50Host::
51
52Traffic from/to a cluster node
53
54VM::
55
56Traffic from/to a specific VM
57
58For each zone, you can define firewall rules for incoming and/or
59outgoing traffic.
60
61
79672214
DM
62Configuration Files
63-------------------
c7eda5e6
DM
64
65All firewall related configuration is stored on the proxmox cluster
66file system. So those files are automatically distributed to all
8c1189b6
FG
67cluster nodes, and the `pve-firewall` service updates the underlying
68`iptables` rules automatically on changes.
c7eda5e6 69
e300cf7d
FG
70You can configure anything using the GUI (i.e. *Datacenter* -> *Firewall*,
71or on a *Node* -> *Firewall*), or you can edit the configuration files
89a8b6c6
DM
72directly using your preferred editor.
73
74Firewall configuration files contains sections of key-value
8c1189b6 75pairs. Lines beginning with a `#` and blank lines are considered
79672214 76comments. Sections starts with a header line containing the section
8c1189b6 77name enclosed in `[` and `]`.
79672214 78
89a8b6c6 79
79672214
DM
80Cluster Wide Setup
81~~~~~~~~~~~~~~~~~~
82
83The cluster wide firewall configuration is stored at:
c7eda5e6
DM
84
85 /etc/pve/firewall/cluster.fw
86
79672214 87The configuration can contain the following sections:
c7eda5e6 88
8c1189b6 89`[OPTIONS]`::
79672214
DM
90
91This is used to set cluster wide firewall options.
92
c48819d1
DM
93include::pve-firewall-cluster-opts.adoc[]
94
8c1189b6 95`[RULES]`::
c7eda5e6 96
79672214
DM
97This sections contains cluster wide firewall rules for all nodes.
98
8c1189b6 99`[IPSET <name>]`::
79672214
DM
100
101Cluster wide IP set definitions.
102
8c1189b6 103`[GROUP <name>]`::
79672214
DM
104
105Cluster wide security group definitions.
106
8c1189b6 107`[ALIASES]`::
79672214
DM
108
109Cluster wide Alias definitions.
110
89a8b6c6
DM
111
112Enabling the Firewall
113^^^^^^^^^^^^^^^^^^^^^
114
115The firewall is completely disabled by default, so you need to
116set the enable option here:
117
118----
119[OPTIONS]
120# enable firewall (cluster wide setting, default is disabled)
121enable: 1
122----
123
124IMPORTANT: If you enable the firewall, traffic to all hosts is blocked by
125default. Only exceptions is WebGUI(8006) and ssh(22) from your local
126network.
127
128If you want to administrate your {pve} hosts from remote, you
129need to create rules to allow traffic from those remote IPs to the web
130GUI (port 8006). You may also want to allow ssh (port 22), and maybe
131SPICE (port 3128).
132
133TIP: Please open a SSH connection to one of your {PVE} hosts before
134enabling the firewall. That way you still have access to the host if
135something goes wrong .
136
137To simplify that task, you can instead create an IPSet called
8c1189b6 138``management'', and add all remote IPs there. This creates all required
89a8b6c6
DM
139firewall rules to access the GUI from remote.
140
141
5eba0743 142Host Specific Configuration
79672214
DM
143~~~~~~~~~~~~~~~~~~~~~~~~~~~
144
145Host related configuration is read from:
146
147 /etc/pve/nodes/<nodename>/host.fw
148
8c1189b6 149This is useful if you want to overwrite rules from `cluster.fw`
79672214 150config. You can also increase log verbosity, and set netfilter related
888c4116
DM
151options. The configuration can contain the following sections:
152
8c1189b6 153`[OPTIONS]`::
888c4116
DM
154
155This is used to set host related firewall options.
156
157include::pve-firewall-host-opts.adoc[]
158
8c1189b6 159`[RULES]`::
888c4116
DM
160
161This sections contains host specific firewall rules.
79672214
DM
162
163
5eba0743 164VM/Container Configuration
79672214 165~~~~~~~~~~~~~~~~~~~~~~~~~~
c7eda5e6
DM
166
167VM firewall configuration is read from:
168
169 /etc/pve/firewall/<VMID>.fw
170
171and contains the following data:
172
8c1189b6 173`[OPTIONS]`::
78ef35dc
DM
174
175This is used to set VM/Container related firewall options.
176
177include::pve-firewall-vm-opts.adoc[]
178
8c1189b6 179`[RULES]`::
78ef35dc
DM
180
181This sections contains VM/Container firewall rules.
182
8c1189b6 183`[IPSET <name>]`::
78ef35dc
DM
184
185IP set definitions.
186
8c1189b6 187`[ALIASES]`::
78ef35dc
DM
188
189IP Alias definitions.
c7eda5e6 190
c7eda5e6 191
58b16f71 192Enabling the Firewall for VMs and Containers
79672214 193^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
c7eda5e6 194
89a8b6c6
DM
195Each virtual network device has its own firewall enable flag. So you
196can selectively enable the firewall for each interface. This is
8c1189b6 197required in addition to the general firewall `enable` option.
89a8b6c6
DM
198
199The firewall requires a special network device setup, so you need to
200restart the VM/container after enabling the firewall on a network
201interface.
c7eda5e6 202
79672214 203
c7eda5e6 204Firewall Rules
79672214 205--------------
c7eda5e6 206
696fb448
DM
207Firewall rules consists of a direction (`IN` or `OUT`) and an
208action (`ACCEPT`, `DENY`, `REJECT`). You can also specify a macro
8c1189b6
FG
209name. Macros contain predefined sets of rules and options. Rules can be
210disabled by prefixing them with `|`.
c7eda5e6 211
696fb448 212.Firewall rules syntax
c7eda5e6
DM
213----
214[RULES]
215
696fb448
DM
216DIRECTION ACTION [OPTIONS]
217|DIRECTION ACTION [OPTIONS] # disabled rule
c7eda5e6 218
696fb448
DM
219DIRECTION MACRO(ACTION) [OPTIONS] # use predefined macro
220----
221
222The following options can be used to refine rule matches.
223
224include::pve-firewall-rules-opts.adoc[]
225
226Here are some examples:
c7eda5e6 227
696fb448
DM
228----
229[RULES]
c7eda5e6
DM
230IN SSH(ACCEPT) -i net0
231IN SSH(ACCEPT) -i net0 # a comment
696fb448 232IN SSH(ACCEPT) -i net0 -source 192.168.2.192 # only allow SSH from 192.168.2.192
c7eda5e6
DM
233IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for ip range
234IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for ip list
696fb448
DM
235IN SSH(ACCEPT) -i net0 -source +mynetgroup # accept ssh for ipset mynetgroup
236IN SSH(ACCEPT) -i net0 -source myserveralias #accept ssh for alias myserveralias
c7eda5e6
DM
237
238|IN SSH(ACCEPT) -i net0 # disabled rule
696fb448
DM
239
240IN DROP # drop all incoming packages
241OUT ACCEPT # accept all outgoing packages
c7eda5e6
DM
242----
243
8c1189b6 244
c7eda5e6 245Security Groups
79672214 246---------------
c7eda5e6 247
58b16f71
WB
248A security group is a collection of rules, defined at cluster level, which
249can be used in all VMs' rules. For example you can define a group named
8c1189b6 250``webserver'' with rules to open the 'http' and 'https' ports.
c7eda5e6
DM
251
252----
253# /etc/pve/firewall/cluster.fw
254
255[group webserver]
256IN ACCEPT -p tcp -dport 80
257IN ACCEPT -p tcp -dport 443
258----
259
58b16f71 260Then, you can add this group to a VM's firewall
c7eda5e6
DM
261
262----
263# /etc/pve/firewall/<VMID>.fw
264
265[RULES]
266GROUP webserver
267----
268
269
270IP Aliases
79672214 271----------
c7eda5e6 272
58b16f71 273IP Aliases allow you to associate IP addresses of networks with a
c7eda5e6
DM
274name. You can then refer to those names:
275
276* inside IP set definitions
277* in `source` and `dest` properties of firewall rules
278
5eba0743
FG
279
280Standard IP Alias `local_network`
79672214 281~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
c7eda5e6
DM
282
283This alias is automatically defined. Please use the following command
284to see assigned values:
285
286----
287# pve-firewall localnet
288local hostname: example
289local IP address: 192.168.2.100
290network auto detect: 192.168.0.0/20
291using detected local_network: 192.168.0.0/20
292----
293
294The firewall automatically sets up rules to allow everything needed
58b16f71 295for cluster communication (corosync, API, SSH) using this alias.
c7eda5e6 296
8c1189b6 297The user can overwrite these values in the `cluster.fw` alias
c7eda5e6
DM
298section. If you use a single host on a public network, it is better to
299explicitly assign the local IP address
300
301----
302# /etc/pve/firewall/cluster.fw
303[ALIASES]
304local_network 1.2.3.4 # use the single ip address
305----
306
5eba0743 307
c7eda5e6 308IP Sets
79672214 309-------
c7eda5e6
DM
310
311IP sets can be used to define groups of networks and hosts. You can
58b16f71 312refer to them with `+name` in the firewall rules' `source` and `dest`
c7eda5e6
DM
313properties.
314
315The following example allows HTTP traffic from the `management` IP
316set.
317
318 IN HTTP(ACCEPT) -source +management
319
5eba0743 320
c7eda5e6 321Standard IP set `management`
79672214 322~~~~~~~~~~~~~~~~~~~~~~~~~~~~
c7eda5e6
DM
323
324This IP set applies only to host firewalls (not VM firewalls). Those
5eba0743 325IPs are allowed to do normal management tasks (PVE GUI, VNC, SPICE,
c7eda5e6
DM
326SSH).
327
328The local cluster network is automatically added to this IP set (alias
329`cluster_network`), to enable inter-host cluster
330communication. (multicast,ssh,...)
331
332----
333# /etc/pve/firewall/cluster.fw
334
335[IPSET management]
336192.168.2.10
337192.168.2.10/24
338----
339
8c1189b6
FG
340
341Standard IP set `blacklist`
79672214 342~~~~~~~~~~~~~~~~~~~~~~~~~~~
c7eda5e6 343
5eba0743 344Traffic from these IPs is dropped by every host's and VM's firewall.
c7eda5e6
DM
345
346----
347# /etc/pve/firewall/cluster.fw
348
349[IPSET blacklist]
35077.240.159.182
351213.87.123.0/24
352----
353
8c1189b6 354
6300d424 355[[ipfilter-section]]
8c1189b6 356Standard IP set `ipfilter-net*`
79672214 357~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
c7eda5e6 358
a34d23e8
WB
359These filters belong to a VM's network interface and are mainly used to prevent
360IP spoofing. If such a set exists for an interface then any outgoing traffic
361with a source IP not matching its interface's corresponding ipfilter set will
362be dropped.
363
364For containers with configured IP addresses these sets, if they exist (or are
e300cf7d 365activated via the general `IP Filter` option in the VM's firewall's *options*
a34d23e8
WB
366tab), implicitly contain the associated IP addresses.
367
368For both virtual machines and containers they also implicitly contain the
369standard MAC-derived IPv6 link-local address in order to allow the neighbor
370discovery protocol to work.
c7eda5e6
DM
371
372----
373/etc/pve/firewall/<VMID>.fw
374
375[IPSET ipfilter-net0] # only allow specified IPs on net0
376192.168.2.10
377----
378
79672214 379
c7eda5e6 380Services and Commands
79672214 381---------------------
c7eda5e6
DM
382
383The firewall runs two service daemons on each node:
384
385* pvefw-logger: NFLOG daemon (ulogd replacement).
386* pve-firewall: updates iptables rules
387
8c1189b6 388There is also a CLI command named `pve-firewall`, which can be used to
c7eda5e6
DM
389start and stop the firewall service:
390
391 # pve-firewall start
392 # pve-firewall stop
393
394To get the status use:
395
396 # pve-firewall status
397
398The above command reads and compiles all firewall rules, so you will
399see warnings if your firewall configuration contains any errors.
400
401If you want to see the generated iptables rules you can use:
402
403 # iptables-save
404
79672214 405
c7eda5e6 406Tips and Tricks
79672214 407---------------
c7eda5e6
DM
408
409How to allow FTP
79672214 410~~~~~~~~~~~~~~~~
c7eda5e6
DM
411
412FTP is an old style protocol which uses port 21 and several other dynamic ports. So you
8c1189b6 413need a rule to accept port 21. In addition, you need to load the `ip_conntrack_ftp` module.
c7eda5e6
DM
414So please run:
415
416 modprobe ip_conntrack_ftp
417
8c1189b6 418and add `ip_conntrack_ftp` to `/etc/modules` (so that it works after a reboot).
c7eda5e6 419
79672214 420
c7eda5e6 421Suricata IPS integration
79672214 422~~~~~~~~~~~~~~~~~~~~~~~~
c7eda5e6
DM
423
424If you want to use the http://suricata-ids.org/[Suricata IPS]
425(Intrusion Prevention System), it's possible.
426
427Packets will be forwarded to the IPS only after the firewall ACCEPTed
428them.
429
430Rejected/Dropped firewall packets don't go to the IPS.
431
432Install suricata on proxmox host:
433
434----
435# apt-get install suricata
436# modprobe nfnetlink_queue
437----
438
8c1189b6 439Don't forget to add `nfnetlink_queue` to `/etc/modules` for next reboot.
c7eda5e6
DM
440
441Then, enable IPS for a specific VM with:
442
443----
444# /etc/pve/firewall/<VMID>.fw
445
446[OPTIONS]
447ips: 1
448ips_queues: 0
449----
450
451`ips_queues` will bind a specific cpu queue for this VM.
452
453Available queues are defined in
454
455----
456# /etc/default/suricata
457NFQUEUE=0
458----
459
8c1189b6
FG
460
461Avoiding `link-local` Addresses on `tap` and `veth` Devices
462~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
326e9652
WB
463
464With IPv6 enabled by default every interface gets a MAC-derived link local
465address. However, most devices on a typical {pve} setup are connected to a
466bridge and so the bridge is the only interface which really needs one.
467
468To disable a link local address on an interface you can set the interface's
469`disable_ipv6` sysconf variable. Despite the name, this does not prevent IPv6
470traffic from passing through the interface when routing or bridging, so the
471only noticeable effect will be the removal of the link local address.
472
473The easiest method of achieving this setting for all newly started VMs is to
474set it for the `default` interface configuration and enabling it explicitly on
475the interfaces which need it. This is also the case for other settings such as
476`forwarding`, `accept_ra` or `autoconf`.
477
083adc34 478
326e9652 479Here's a possible setup:
326e9652 480
083adc34
FG
481.File `/etc/sysconf.d/90-ipv6.conf`
482----
326e9652
WB
483net.ipv6.conf.default.forwarding = 0
484net.ipv6.conf.default.proxy_ndp = 0
485net.ipv6.conf.default.autoconf = 0
486net.ipv6.conf.default.disable_ipv6 = 1
487net.ipv6.conf.default.accept_ra = 0
488
489net.ipv6.conf.lo.disable_ipv6 = 0
490----
491
083adc34 492.File `/etc/network/interfaces`
326e9652 493----
326e9652 494(...)
f56aaa83
WB
495# Dual stack:
496iface vmbr0 inet static
497 address 1.2.3.4
498 netmask 255.255.255.128
499 gateway 1.2.3.5
326e9652
WB
500iface vmbr0 inet6 static
501 address fc00::31
502 netmask 16
503 gateway fc00::1
504 accept_ra 0
505 pre-up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
f56aaa83
WB
506
507# With IPv6-only 'pre-up' is too early and 'up' is too late.
508# Work around this by creating the bridge manually
509iface vmbr1 inet manual
510 pre-up ip link add $IFACE type bridge
511 up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6
512iface vmbr1 inet6 static
513 address fc00:b:3::1
514 netmask 96
515 bridge_ports none
516 bridge_stp off
517 bridge_fd 0
518 bridge_vlan_aware yes
519 accept_ra 0
326e9652
WB
520(...)
521----
14c06023 522
79672214
DM
523
524Notes on IPv6
525-------------
526
527The firewall contains a few IPv6 specific options. One thing to note is that
528IPv6 does not use the ARP protocol anymore, and instead uses NDP (Neighbor
529Discovery Protocol) which works on IP level and thus needs IP addresses to
530succeed. For this purpose link-local addresses derived from the interface's MAC
8c1189b6 531address are used. By default the `NDP` option is enabled on both host and VM
79672214
DM
532level to allow neighbor discovery (NDP) packets to be sent and received.
533
534Beside neighbor discovery NDP is also used for a couple of other things, like
535autoconfiguration and advertising routers.
536
537By default VMs are allowed to send out router solicitation messages (to query
5eba0743 538for a router), and to receive router advertisement packets. This allows them to
79672214 539use stateless auto configuration. On the other hand VMs cannot advertise
8c1189b6 540themselves as routers unless the ``Allow Router Advertisement'' (`radv: 1`) option
79672214
DM
541is set.
542
8c1189b6 543As for the link local addresses required for NDP, there's also an ``IP Filter''
79672214
DM
544(`ipfilter: 1`) option which can be enabled which has the same effect as adding
545an `ipfilter-net*` ipset for each of the VM's network interfaces containing the
546corresponding link local addresses. (See the
8c1189b6 547<<ipfilter-section,Standard IP set `ipfilter-net*`>> section for details.)
79672214
DM
548
549
224128ce
DM
550Ports used by Proxmox VE
551------------------------
552
553* Web interface: 8006
554* VNC Web console: 5900-5999
555* SPICE proxy: 3128
556* sshd (used for cluster actions): 22
557* rpcbind: 111
5eba0743 558* corosync multicast (if you run a cluster): 5404, 5405 UDP
224128ce 559
14c06023
DM
560
561ifdef::manvolnum[]
562
563Macro Definitions
564-----------------
565
566include::pve-firewall-macros.adoc[]
567
568
569include::pve-copyright.adoc[]
570
571endif::manvolnum[]