]>
Commit | Line | Data |
---|---|---|
80c0adcb | 1 | [[chapter_pve_firewall]] |
c7eda5e6 | 2 | ifdef::manvolnum[] |
b2f242ab DM |
3 | pve-firewall(8) |
4 | =============== | |
5f09af76 DM |
5 | :pve-toplevel: |
6 | ||
c7eda5e6 DM |
7 | NAME |
8 | ---- | |
9 | ||
f5eb0727 | 10 | pve-firewall - PVE Firewall Daemon |
c7eda5e6 DM |
11 | |
12 | ||
49a5e11c | 13 | SYNOPSIS |
c7eda5e6 DM |
14 | -------- |
15 | ||
5f34196d | 16 | include::pve-firewall.8-synopsis.adoc[] |
c7eda5e6 DM |
17 | |
18 | ||
19 | DESCRIPTION | |
20 | ----------- | |
21 | endif::manvolnum[] | |
c7eda5e6 DM |
22 | ifndef::manvolnum[] |
23 | {pve} Firewall | |
24 | ============== | |
194d2f29 | 25 | :pve-toplevel: |
c7eda5e6 | 26 | endif::manvolnum[] |
5f09af76 | 27 | ifdef::wiki[] |
cb84ed18 | 28 | :title: Firewall |
5f09af76 DM |
29 | endif::wiki[] |
30 | ||
26ca7ff5 | 31 | {pve} Firewall provides an easy way to protect your IT |
89a8b6c6 | 32 | infrastructure. You can setup firewall rules for all hosts |
c7eda5e6 DM |
33 | inside a cluster, or define rules for virtual machines and |
34 | containers. Features like firewall macros, security groups, IP sets | |
5eba0743 | 35 | and aliases help to make that task easier. |
c7eda5e6 DM |
36 | |
37 | While all configuration is stored on the cluster file system, the | |
d5621636 | 38 | `iptables`-based firewall service runs on each cluster node, and thus provides |
c7eda5e6 DM |
39 | full isolation between virtual machines. The distributed nature of |
40 | this system also provides much higher bandwidth than a central | |
41 | firewall solution. | |
42 | ||
bd73a43e DM |
43 | The firewall has full support for IPv4 and IPv6. IPv6 support is fully |
44 | transparent, and we filter traffic for both protocols by default. So | |
45 | there is no need to maintain a different set of rules for IPv6. | |
46 | ||
c7eda5e6 DM |
47 | |
48 | Zones | |
49 | ----- | |
50 | ||
51 | The Proxmox VE firewall groups the network into the following logical zones: | |
52 | ||
53 | Host:: | |
54 | ||
55 | Traffic from/to a cluster node | |
56 | ||
57 | VM:: | |
58 | ||
59 | Traffic from/to a specific VM | |
60 | ||
61 | For each zone, you can define firewall rules for incoming and/or | |
62 | outgoing traffic. | |
63 | ||
64 | ||
79672214 DM |
65 | Configuration Files |
66 | ------------------- | |
c7eda5e6 DM |
67 | |
68 | All firewall related configuration is stored on the proxmox cluster | |
69 | file system. So those files are automatically distributed to all | |
8c1189b6 FG |
70 | cluster nodes, and the `pve-firewall` service updates the underlying |
71 | `iptables` rules automatically on changes. | |
c7eda5e6 | 72 | |
e300cf7d FG |
73 | You can configure anything using the GUI (i.e. *Datacenter* -> *Firewall*, |
74 | or on a *Node* -> *Firewall*), or you can edit the configuration files | |
89a8b6c6 DM |
75 | directly using your preferred editor. |
76 | ||
d5621636 | 77 | Firewall configuration files contain sections of key-value |
8c1189b6 | 78 | pairs. Lines beginning with a `#` and blank lines are considered |
d5621636 | 79 | comments. Sections start with a header line containing the section |
8c1189b6 | 80 | name enclosed in `[` and `]`. |
79672214 | 81 | |
89a8b6c6 | 82 | |
80c0adcb | 83 | [[pve_firewall_cluster_wide_setup]] |
79672214 DM |
84 | Cluster Wide Setup |
85 | ~~~~~~~~~~~~~~~~~~ | |
86 | ||
60ed554f | 87 | The cluster-wide firewall configuration is stored at: |
c7eda5e6 DM |
88 | |
89 | /etc/pve/firewall/cluster.fw | |
90 | ||
79672214 | 91 | The configuration can contain the following sections: |
c7eda5e6 | 92 | |
8c1189b6 | 93 | `[OPTIONS]`:: |
79672214 | 94 | |
60ed554f | 95 | This is used to set cluster-wide firewall options. |
79672214 | 96 | |
c48819d1 DM |
97 | include::pve-firewall-cluster-opts.adoc[] |
98 | ||
8c1189b6 | 99 | `[RULES]`:: |
c7eda5e6 | 100 | |
60ed554f | 101 | This sections contains cluster-wide firewall rules for all nodes. |
79672214 | 102 | |
8c1189b6 | 103 | `[IPSET <name>]`:: |
79672214 DM |
104 | |
105 | Cluster wide IP set definitions. | |
106 | ||
8c1189b6 | 107 | `[GROUP <name>]`:: |
79672214 DM |
108 | |
109 | Cluster wide security group definitions. | |
110 | ||
8c1189b6 | 111 | `[ALIASES]`:: |
79672214 DM |
112 | |
113 | Cluster wide Alias definitions. | |
114 | ||
89a8b6c6 DM |
115 | |
116 | Enabling the Firewall | |
117 | ^^^^^^^^^^^^^^^^^^^^^ | |
118 | ||
119 | The firewall is completely disabled by default, so you need to | |
120 | set the enable option here: | |
121 | ||
122 | ---- | |
123 | [OPTIONS] | |
60ed554f | 124 | # enable firewall (cluster-wide setting, default is disabled) |
89a8b6c6 DM |
125 | enable: 1 |
126 | ---- | |
127 | ||
128 | IMPORTANT: If you enable the firewall, traffic to all hosts is blocked by | |
129 | default. Only exceptions is WebGUI(8006) and ssh(22) from your local | |
130 | network. | |
131 | ||
132 | If you want to administrate your {pve} hosts from remote, you | |
133 | need to create rules to allow traffic from those remote IPs to the web | |
134 | GUI (port 8006). You may also want to allow ssh (port 22), and maybe | |
135 | SPICE (port 3128). | |
136 | ||
137 | TIP: Please open a SSH connection to one of your {PVE} hosts before | |
138 | enabling the firewall. That way you still have access to the host if | |
139 | something goes wrong . | |
140 | ||
141 | To simplify that task, you can instead create an IPSet called | |
8c1189b6 | 142 | ``management'', and add all remote IPs there. This creates all required |
89a8b6c6 DM |
143 | firewall rules to access the GUI from remote. |
144 | ||
145 | ||
80c0adcb | 146 | [[pve_firewall_host_specific_configuration]] |
5eba0743 | 147 | Host Specific Configuration |
79672214 DM |
148 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
149 | ||
150 | Host related configuration is read from: | |
151 | ||
152 | /etc/pve/nodes/<nodename>/host.fw | |
153 | ||
8c1189b6 | 154 | This is useful if you want to overwrite rules from `cluster.fw` |
79672214 | 155 | config. You can also increase log verbosity, and set netfilter related |
888c4116 DM |
156 | options. The configuration can contain the following sections: |
157 | ||
8c1189b6 | 158 | `[OPTIONS]`:: |
888c4116 DM |
159 | |
160 | This is used to set host related firewall options. | |
161 | ||
162 | include::pve-firewall-host-opts.adoc[] | |
163 | ||
8c1189b6 | 164 | `[RULES]`:: |
888c4116 DM |
165 | |
166 | This sections contains host specific firewall rules. | |
79672214 | 167 | |
641cc419 | 168 | [[pve_firewall_vm_container_configuration]] |
5eba0743 | 169 | VM/Container Configuration |
79672214 | 170 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
c7eda5e6 | 171 | |
a4922e12 | 172 | VM firewall configuration is read from: |
c7eda5e6 DM |
173 | |
174 | /etc/pve/firewall/<VMID>.fw | |
175 | ||
176 | and contains the following data: | |
177 | ||
8c1189b6 | 178 | `[OPTIONS]`:: |
78ef35dc DM |
179 | |
180 | This is used to set VM/Container related firewall options. | |
181 | ||
182 | include::pve-firewall-vm-opts.adoc[] | |
183 | ||
8c1189b6 | 184 | `[RULES]`:: |
78ef35dc DM |
185 | |
186 | This sections contains VM/Container firewall rules. | |
187 | ||
8c1189b6 | 188 | `[IPSET <name>]`:: |
78ef35dc DM |
189 | |
190 | IP set definitions. | |
191 | ||
8c1189b6 | 192 | `[ALIASES]`:: |
78ef35dc DM |
193 | |
194 | IP Alias definitions. | |
c7eda5e6 | 195 | |
c7eda5e6 | 196 | |
58b16f71 | 197 | Enabling the Firewall for VMs and Containers |
79672214 | 198 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
c7eda5e6 | 199 | |
89a8b6c6 DM |
200 | Each virtual network device has its own firewall enable flag. So you |
201 | can selectively enable the firewall for each interface. This is | |
8c1189b6 | 202 | required in addition to the general firewall `enable` option. |
89a8b6c6 | 203 | |
79672214 | 204 | |
c7eda5e6 | 205 | Firewall Rules |
79672214 | 206 | -------------- |
c7eda5e6 | 207 | |
696fb448 DM |
208 | Firewall rules consists of a direction (`IN` or `OUT`) and an |
209 | action (`ACCEPT`, `DENY`, `REJECT`). You can also specify a macro | |
8c1189b6 FG |
210 | name. Macros contain predefined sets of rules and options. Rules can be |
211 | disabled by prefixing them with `|`. | |
c7eda5e6 | 212 | |
696fb448 | 213 | .Firewall rules syntax |
c7eda5e6 DM |
214 | ---- |
215 | [RULES] | |
216 | ||
696fb448 DM |
217 | DIRECTION ACTION [OPTIONS] |
218 | |DIRECTION ACTION [OPTIONS] # disabled rule | |
c7eda5e6 | 219 | |
696fb448 DM |
220 | DIRECTION MACRO(ACTION) [OPTIONS] # use predefined macro |
221 | ---- | |
222 | ||
223 | The following options can be used to refine rule matches. | |
224 | ||
225 | include::pve-firewall-rules-opts.adoc[] | |
226 | ||
227 | Here are some examples: | |
c7eda5e6 | 228 | |
696fb448 DM |
229 | ---- |
230 | [RULES] | |
c7eda5e6 DM |
231 | IN SSH(ACCEPT) -i net0 |
232 | IN SSH(ACCEPT) -i net0 # a comment | |
696fb448 | 233 | IN SSH(ACCEPT) -i net0 -source 192.168.2.192 # only allow SSH from 192.168.2.192 |
b3234584 OB |
234 | IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for IP range |
235 | IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for IP list | |
696fb448 DM |
236 | IN SSH(ACCEPT) -i net0 -source +mynetgroup # accept ssh for ipset mynetgroup |
237 | IN SSH(ACCEPT) -i net0 -source myserveralias #accept ssh for alias myserveralias | |
c7eda5e6 DM |
238 | |
239 | |IN SSH(ACCEPT) -i net0 # disabled rule | |
696fb448 DM |
240 | |
241 | IN DROP # drop all incoming packages | |
242 | OUT ACCEPT # accept all outgoing packages | |
c7eda5e6 DM |
243 | ---- |
244 | ||
8c1189b6 | 245 | |
80c0adcb | 246 | [[pve_firewall_security_groups]] |
c7eda5e6 | 247 | Security Groups |
79672214 | 248 | --------------- |
c7eda5e6 | 249 | |
58b16f71 WB |
250 | A security group is a collection of rules, defined at cluster level, which |
251 | can be used in all VMs' rules. For example you can define a group named | |
8c1189b6 | 252 | ``webserver'' with rules to open the 'http' and 'https' ports. |
c7eda5e6 DM |
253 | |
254 | ---- | |
255 | # /etc/pve/firewall/cluster.fw | |
256 | ||
257 | [group webserver] | |
258 | IN ACCEPT -p tcp -dport 80 | |
259 | IN ACCEPT -p tcp -dport 443 | |
260 | ---- | |
261 | ||
58b16f71 | 262 | Then, you can add this group to a VM's firewall |
c7eda5e6 DM |
263 | |
264 | ---- | |
265 | # /etc/pve/firewall/<VMID>.fw | |
266 | ||
267 | [RULES] | |
268 | GROUP webserver | |
269 | ---- | |
270 | ||
641cc419 | 271 | [[pve_firewall_ip_aliases]] |
c7eda5e6 | 272 | IP Aliases |
79672214 | 273 | ---------- |
c7eda5e6 | 274 | |
58b16f71 | 275 | IP Aliases allow you to associate IP addresses of networks with a |
c7eda5e6 DM |
276 | name. You can then refer to those names: |
277 | ||
278 | * inside IP set definitions | |
279 | * in `source` and `dest` properties of firewall rules | |
280 | ||
5eba0743 FG |
281 | |
282 | Standard IP Alias `local_network` | |
79672214 | 283 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
c7eda5e6 DM |
284 | |
285 | This alias is automatically defined. Please use the following command | |
286 | to see assigned values: | |
287 | ||
288 | ---- | |
289 | # pve-firewall localnet | |
290 | local hostname: example | |
291 | local IP address: 192.168.2.100 | |
292 | network auto detect: 192.168.0.0/20 | |
293 | using detected local_network: 192.168.0.0/20 | |
294 | ---- | |
295 | ||
296 | The firewall automatically sets up rules to allow everything needed | |
58b16f71 | 297 | for cluster communication (corosync, API, SSH) using this alias. |
c7eda5e6 | 298 | |
8c1189b6 | 299 | The user can overwrite these values in the `cluster.fw` alias |
c7eda5e6 DM |
300 | section. If you use a single host on a public network, it is better to |
301 | explicitly assign the local IP address | |
302 | ||
303 | ---- | |
304 | # /etc/pve/firewall/cluster.fw | |
305 | [ALIASES] | |
b3234584 | 306 | local_network 1.2.3.4 # use the single IP address |
c7eda5e6 DM |
307 | ---- |
308 | ||
641cc419 | 309 | [[pve_firewall_ip_sets]] |
c7eda5e6 | 310 | IP Sets |
79672214 | 311 | ------- |
c7eda5e6 DM |
312 | |
313 | IP sets can be used to define groups of networks and hosts. You can | |
58b16f71 | 314 | refer to them with `+name` in the firewall rules' `source` and `dest` |
c7eda5e6 DM |
315 | properties. |
316 | ||
317 | The following example allows HTTP traffic from the `management` IP | |
318 | set. | |
319 | ||
320 | IN HTTP(ACCEPT) -source +management | |
321 | ||
5eba0743 | 322 | |
c7eda5e6 | 323 | Standard IP set `management` |
79672214 | 324 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
c7eda5e6 DM |
325 | |
326 | This IP set applies only to host firewalls (not VM firewalls). Those | |
363c7a1d | 327 | IPs are allowed to do normal management tasks ({PVE} GUI, VNC, SPICE, |
c7eda5e6 DM |
328 | SSH). |
329 | ||
330 | The local cluster network is automatically added to this IP set (alias | |
331 | `cluster_network`), to enable inter-host cluster | |
332 | communication. (multicast,ssh,...) | |
333 | ||
334 | ---- | |
335 | # /etc/pve/firewall/cluster.fw | |
336 | ||
337 | [IPSET management] | |
338 | 192.168.2.10 | |
339 | 192.168.2.10/24 | |
340 | ---- | |
341 | ||
8c1189b6 FG |
342 | |
343 | Standard IP set `blacklist` | |
79672214 | 344 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
c7eda5e6 | 345 | |
5eba0743 | 346 | Traffic from these IPs is dropped by every host's and VM's firewall. |
c7eda5e6 DM |
347 | |
348 | ---- | |
349 | # /etc/pve/firewall/cluster.fw | |
350 | ||
351 | [IPSET blacklist] | |
352 | 77.240.159.182 | |
353 | 213.87.123.0/24 | |
354 | ---- | |
355 | ||
8c1189b6 | 356 | |
80c0adcb | 357 | [[pve_firewall_ipfilter_section]] |
8c1189b6 | 358 | Standard IP set `ipfilter-net*` |
79672214 | 359 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
c7eda5e6 | 360 | |
a34d23e8 WB |
361 | These filters belong to a VM's network interface and are mainly used to prevent |
362 | IP spoofing. If such a set exists for an interface then any outgoing traffic | |
363 | with a source IP not matching its interface's corresponding ipfilter set will | |
364 | be dropped. | |
365 | ||
366 | For containers with configured IP addresses these sets, if they exist (or are | |
e300cf7d | 367 | activated via the general `IP Filter` option in the VM's firewall's *options* |
a34d23e8 WB |
368 | tab), implicitly contain the associated IP addresses. |
369 | ||
370 | For both virtual machines and containers they also implicitly contain the | |
371 | standard MAC-derived IPv6 link-local address in order to allow the neighbor | |
372 | discovery protocol to work. | |
c7eda5e6 DM |
373 | |
374 | ---- | |
375 | /etc/pve/firewall/<VMID>.fw | |
376 | ||
377 | [IPSET ipfilter-net0] # only allow specified IPs on net0 | |
378 | 192.168.2.10 | |
379 | ---- | |
380 | ||
79672214 | 381 | |
d00d6f56 | 382 | [[pve_firewall_services_commands]] |
c7eda5e6 | 383 | Services and Commands |
79672214 | 384 | --------------------- |
c7eda5e6 DM |
385 | |
386 | The firewall runs two service daemons on each node: | |
387 | ||
388 | * pvefw-logger: NFLOG daemon (ulogd replacement). | |
389 | * pve-firewall: updates iptables rules | |
390 | ||
8c1189b6 | 391 | There is also a CLI command named `pve-firewall`, which can be used to |
c7eda5e6 DM |
392 | start and stop the firewall service: |
393 | ||
394 | # pve-firewall start | |
395 | # pve-firewall stop | |
396 | ||
397 | To get the status use: | |
398 | ||
399 | # pve-firewall status | |
400 | ||
401 | The above command reads and compiles all firewall rules, so you will | |
402 | see warnings if your firewall configuration contains any errors. | |
403 | ||
404 | If you want to see the generated iptables rules you can use: | |
405 | ||
406 | # iptables-save | |
407 | ||
afde3bac CE |
408 | [[pve_firewall_default_rules]] |
409 | Default firewall rules | |
410 | ---------------------- | |
411 | ||
412 | The following traffic is filtered by the default firewall configuration: | |
413 | ||
3f41b2c5 TL |
414 | Datacenter incoming/outgoing DROP/REJECT |
415 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
afde3bac | 416 | |
3f41b2c5 TL |
417 | If the input or output policy for the firewall is set to DROP or REJECT, the |
418 | following traffic is still allowed for all {pve} hosts in the cluster: | |
afde3bac CE |
419 | |
420 | * traffic over the loopback interface | |
421 | * already established connections | |
3f41b2c5 TL |
422 | * traffic using the IGMP protocol |
423 | * TCP traffic from management hosts to port 8006 in order to allow access to | |
424 | the web interface | |
425 | * TCP traffic from management hosts to the port range 5900 to 5999 allowing | |
426 | traffic for the VNC web console | |
427 | * TCP traffic from management hosts to port 3128 for connections to the SPICE | |
428 | proxy | |
429 | * TCP traffic from management hosts to port 22 to allow ssh access | |
337a2d42 | 430 | * UDP traffic in the cluster network to ports 5405-5412 for corosync |
3f41b2c5 TL |
431 | * UDP multicast traffic in the cluster network |
432 | * ICMP traffic type 3 (Destination Unreachable), 4 (congestion control) or 11 | |
433 | (Time Exceeded) | |
afde3bac CE |
434 | |
435 | The following traffic is dropped, but not logged even with logging enabled: | |
436 | ||
3f41b2c5 TL |
437 | * TCP connections with invalid connection state |
438 | * Broadcast, multicast and anycast traffic not related to corosync, i.e., not | |
337a2d42 | 439 | coming through ports 5405-5412 |
3f41b2c5 TL |
440 | * TCP traffic to port 43 |
441 | * UDP traffic to ports 135 and 445 | |
442 | * UDP traffic to the port range 137 to 139 | |
443 | * UDP traffic form source port 137 to port range 1024 to 65535 | |
444 | * UDP traffic to port 1900 | |
445 | * TCP traffic to port 135, 139 and 445 | |
446 | * UDP traffic originating from source port 53 | |
447 | ||
448 | The rest of the traffic is dropped or rejected, respectively, and also logged. | |
afde3bac CE |
449 | This may vary depending on the additional options enabled in |
450 | *Firewall* -> *Options*, such as NDP, SMURFS and TCP flag filtering. | |
451 | ||
3f41b2c5 TL |
452 | [[pve_firewall_iptables_inspect]] |
453 | Please inspect the output of the | |
afde3bac | 454 | |
3f41b2c5 | 455 | ---- |
afde3bac | 456 | # iptables-save |
3f41b2c5 | 457 | ---- |
afde3bac | 458 | |
3f41b2c5 TL |
459 | system command to see the firewall chains and rules active on your system. |
460 | This output is also included in a `System Report`, accessible over a node's | |
ff4ae052 | 461 | subscription tab in the web GUI, or through the `pvereport` command-line tool. |
afde3bac | 462 | |
3f41b2c5 TL |
463 | VM/CT incoming/outgoing DROP/REJECT |
464 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
afde3bac | 465 | |
3f41b2c5 TL |
466 | This drops or rejects all the traffic to the VMs, with some exceptions for |
467 | DHCP, NDP, Router Advertisement, MAC and IP filtering depending on the set | |
468 | configuration. The same rules for dropping/rejecting packets are inherited | |
5f318cc0 | 469 | from the datacenter, while the exceptions for accepted incoming/outgoing |
3f41b2c5 | 470 | traffic of the host do not apply. |
afde3bac | 471 | |
3f41b2c5 TL |
472 | Again, you can use xref:pve_firewall_iptables_inspect[iptables-save (see above)] |
473 | to inspect all rules and chains applied. | |
afde3bac | 474 | |
7d47064e CE |
475 | Logging of firewall rules |
476 | ------------------------- | |
477 | ||
afde3bac | 478 | By default, all logging of traffic filtered by the firewall rules is disabled. |
3a433e9b | 479 | To enable logging, the `loglevel` for incoming and/or outgoing traffic has to be |
afde3bac CE |
480 | set in *Firewall* -> *Options*. This can be done for the host as well as for the |
481 | VM/CT firewall individually. By this, logging of {PVE}'s standard firewall rules | |
482 | is enabled and the output can be observed in *Firewall* -> *Log*. | |
483 | Further, only some dropped or rejected packets are logged for the standard rules | |
484 | (see xref:pve_firewall_default_rules[default firewall rules]). | |
485 | ||
486 | `loglevel` does not affect how much of the filtered traffic is logged. It | |
487 | changes a `LOGID` appended as prefix to the log output for easier filtering and | |
488 | post-processing. | |
489 | ||
490 | `loglevel` is one of the following flags: | |
491 | ||
492 | [[pve_firewall_log_levels]] | |
493 | [width="25%", options="header"] | |
494 | |=================== | |
495 | | loglevel | LOGID | |
3f41b2c5 | 496 | | nolog | -- |
afde3bac CE |
497 | | emerg | 0 |
498 | | alert | 1 | |
499 | | crit | 2 | |
500 | | err | 3 | |
501 | | warning | 4 | |
502 | | notice | 5 | |
503 | | info | 6 | |
504 | | debug | 7 | |
505 | |=================== | |
506 | ||
507 | A typical firewall log output looks like this: | |
508 | ||
509 | ---- | |
510 | VMID LOGID CHAIN TIMESTAMP POLICY: PACKET_DETAILS | |
511 | ---- | |
512 | ||
513 | In case of the host firewall, `VMID` is equal to 0. | |
7d47064e | 514 | |
afde3bac CE |
515 | |
516 | Logging of user defined firewall rules | |
517 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
73b78e5e | 518 | |
7d47064e CE |
519 | In order to log packets filtered by user-defined firewall rules, it is possible |
520 | to set a log-level parameter for each rule individually. | |
521 | This allows to log in a fine grained manner and independent of the log-level | |
afde3bac CE |
522 | defined for the standard rules in *Firewall* -> *Options*. |
523 | ||
524 | While the `loglevel` for each individual rule can be defined or changed easily | |
e2b3622a | 525 | in the web UI during creation or modification of the rule, it is possible to set |
afde3bac | 526 | this also via the corresponding `pvesh` API calls. |
7d47064e | 527 | |
afde3bac CE |
528 | Further, the log-level can also be set via the firewall configuration file by |
529 | appending a `-log <loglevel>` to the selected rule (see | |
530 | xref:pve_firewall_log_levels[possible log-levels]). | |
7d47064e | 531 | |
604a393a | 532 | For example, the following two are identical: |
7d47064e CE |
533 | |
534 | ---- | |
535 | IN REJECT -p icmp -log nolog | |
7d47064e CE |
536 | IN REJECT -p icmp |
537 | ---- | |
538 | ||
539 | whereas | |
540 | ||
541 | ---- | |
542 | IN REJECT -p icmp -log debug | |
543 | ---- | |
544 | ||
545 | produces a log output flagged with the `debug` level. | |
546 | ||
79672214 | 547 | |
c7eda5e6 | 548 | Tips and Tricks |
79672214 | 549 | --------------- |
c7eda5e6 DM |
550 | |
551 | How to allow FTP | |
79672214 | 552 | ~~~~~~~~~~~~~~~~ |
c7eda5e6 DM |
553 | |
554 | FTP is an old style protocol which uses port 21 and several other dynamic ports. So you | |
8c1189b6 | 555 | need a rule to accept port 21. In addition, you need to load the `ip_conntrack_ftp` module. |
c7eda5e6 DM |
556 | So please run: |
557 | ||
558 | modprobe ip_conntrack_ftp | |
559 | ||
8c1189b6 | 560 | and add `ip_conntrack_ftp` to `/etc/modules` (so that it works after a reboot). |
c7eda5e6 | 561 | |
79672214 | 562 | |
c7eda5e6 | 563 | Suricata IPS integration |
79672214 | 564 | ~~~~~~~~~~~~~~~~~~~~~~~~ |
c7eda5e6 | 565 | |
bf7e21c9 | 566 | If you want to use the https://suricata.io/[Suricata IPS] |
c7eda5e6 DM |
567 | (Intrusion Prevention System), it's possible. |
568 | ||
569 | Packets will be forwarded to the IPS only after the firewall ACCEPTed | |
570 | them. | |
571 | ||
572 | Rejected/Dropped firewall packets don't go to the IPS. | |
573 | ||
574 | Install suricata on proxmox host: | |
575 | ||
576 | ---- | |
577 | # apt-get install suricata | |
578 | # modprobe nfnetlink_queue | |
579 | ---- | |
580 | ||
8c1189b6 | 581 | Don't forget to add `nfnetlink_queue` to `/etc/modules` for next reboot. |
c7eda5e6 DM |
582 | |
583 | Then, enable IPS for a specific VM with: | |
584 | ||
585 | ---- | |
586 | # /etc/pve/firewall/<VMID>.fw | |
587 | ||
588 | [OPTIONS] | |
589 | ips: 1 | |
590 | ips_queues: 0 | |
591 | ---- | |
592 | ||
593 | `ips_queues` will bind a specific cpu queue for this VM. | |
594 | ||
595 | Available queues are defined in | |
596 | ||
597 | ---- | |
598 | # /etc/default/suricata | |
599 | NFQUEUE=0 | |
600 | ---- | |
601 | ||
8c1189b6 | 602 | |
79672214 DM |
603 | Notes on IPv6 |
604 | ------------- | |
605 | ||
606 | The firewall contains a few IPv6 specific options. One thing to note is that | |
607 | IPv6 does not use the ARP protocol anymore, and instead uses NDP (Neighbor | |
608 | Discovery Protocol) which works on IP level and thus needs IP addresses to | |
609 | succeed. For this purpose link-local addresses derived from the interface's MAC | |
8c1189b6 | 610 | address are used. By default the `NDP` option is enabled on both host and VM |
79672214 DM |
611 | level to allow neighbor discovery (NDP) packets to be sent and received. |
612 | ||
613 | Beside neighbor discovery NDP is also used for a couple of other things, like | |
b3234584 | 614 | auto-configuration and advertising routers. |
79672214 DM |
615 | |
616 | By default VMs are allowed to send out router solicitation messages (to query | |
5eba0743 | 617 | for a router), and to receive router advertisement packets. This allows them to |
79672214 | 618 | use stateless auto configuration. On the other hand VMs cannot advertise |
8c1189b6 | 619 | themselves as routers unless the ``Allow Router Advertisement'' (`radv: 1`) option |
79672214 DM |
620 | is set. |
621 | ||
8c1189b6 | 622 | As for the link local addresses required for NDP, there's also an ``IP Filter'' |
79672214 DM |
623 | (`ipfilter: 1`) option which can be enabled which has the same effect as adding |
624 | an `ipfilter-net*` ipset for each of the VM's network interfaces containing the | |
625 | corresponding link local addresses. (See the | |
80c0adcb | 626 | <<pve_firewall_ipfilter_section,Standard IP set `ipfilter-net*`>> section for details.) |
79672214 DM |
627 | |
628 | ||
26ca7ff5 FG |
629 | Ports used by {pve} |
630 | ------------------- | |
224128ce | 631 | |
b92c45ab TL |
632 | * Web interface: 8006 (TCP, HTTP/1.1 over TLS) |
633 | * VNC Web console: 5900-5999 (TCP, WebSocket) | |
634 | * SPICE proxy: 3128 (TCP) | |
635 | * sshd (used for cluster actions): 22 (TCP) | |
636 | * rpcbind: 111 (UDP) | |
637 | * sendmail: 25 (TCP, outgoing) | |
337a2d42 | 638 | * corosync cluster traffic: 5405-5412 UDP |
b92c45ab | 639 | * live migration (VM memory and local-disk data): 60000-60050 (TCP) |
14c06023 | 640 | |
d00d6f56 SH |
641 | |
642 | nftables | |
643 | -------- | |
644 | ||
645 | As an alternative to `pve-firewall` we offer `proxmox-firewall`, which is an | |
646 | implementation of the Proxmox VE firewall based on the newer | |
647 | https://wiki.nftables.org/wiki-nftables/index.php/What_is_nftables%3F[nftables] | |
648 | rather than iptables. | |
649 | ||
650 | WARNING: `proxmox-firewall` is currently in tech preview. There might be bugs or | |
651 | incompatibilies with the original firewall. It is currently not suited for | |
652 | production use. | |
653 | ||
654 | This implementation uses the same configuration files and configuration format, | |
655 | so you can use your old configuration when switching. It provides the exact same | |
656 | functionality with a few exceptions: | |
657 | ||
658 | * REJECT is currently not possible for guest traffic (traffic will instead be | |
659 | dropped). | |
660 | * Using the `NDP`, `Router Advertisement` or `DHCP` options will *always* create | |
661 | firewall rules, irregardless of your default policy. | |
662 | * firewall rules for guests are evaluated even for connections that have | |
663 | conntrack table entries. | |
664 | ||
665 | ||
666 | Installation and Usage | |
667 | ~~~~~~~~~~~~~~~~~~~~~~ | |
668 | ||
669 | Install the `proxmox-firewall` package: | |
670 | ||
671 | ---- | |
672 | apt install proxmox-firewall | |
673 | ---- | |
674 | ||
675 | Enable the nftables backend via the Web UI on your hosts (Host > Firewall > | |
676 | Options > nftables), or by enabling it in the configuration file for your hosts | |
677 | (`/etc/pve/nodes/<node_name>/host.fw`): | |
678 | ||
679 | ---- | |
680 | [OPTIONS] | |
681 | ||
682 | nftables: 1 | |
683 | ---- | |
684 | ||
685 | NOTE: After enabling/disabling `proxmox-firewall`, all running VMs and | |
686 | containers need to be restarted for the old/new firewall to work properly. | |
687 | ||
688 | After setting the `nftables` configuration key, the new `proxmox-firewall` | |
689 | service will take over. You can check if the new service is working by | |
690 | checking the systemctl status of `proxmox-firewall`: | |
691 | ||
692 | ---- | |
693 | systemctl status proxmox-firewall | |
694 | ---- | |
695 | ||
696 | You can also examine the generated ruleset. You can find more information about | |
697 | this in the section xref:pve_firewall_nft_helpful_commands[Helpful Commands]. | |
698 | You should also check whether `pve-firewall` is no longer generating iptables | |
699 | rules, you can find the respective commands in the | |
700 | xref:pve_firewall_services_commands[Services and Commands] section. | |
701 | ||
702 | Switching back to the old firewall can be done by simply setting the | |
703 | configuration value back to 0 / No. | |
704 | ||
705 | Usage | |
706 | ~~~~~ | |
707 | ||
708 | `proxmox-firewall` will create two tables that are managed by the | |
709 | `proxmox-firewall` service: `proxmox-firewall` and `proxmox-firewall-guests`. If | |
710 | you want to create custom rules that live outside the Proxmox VE firewall | |
711 | configuration you can create your own tables to manage your custom firewall | |
712 | rules. `proxmox-firewall` will only touch the tables it generates, so you can | |
713 | easily extend and modify the behavior of the `proxmox-firewall` by adding your | |
714 | own tables. | |
715 | ||
716 | Instead of using the `pve-firewall` command, the nftables-based firewall uses | |
717 | `proxmox-firewall`. It is a systemd service, so you can start and stop it via | |
718 | `systemctl`: | |
719 | ||
720 | ---- | |
721 | systemctl start proxmox-firewall | |
722 | systemctl stop proxmox-firewall | |
723 | ---- | |
724 | ||
725 | Stopping the firewall service will remove all generated rules. | |
726 | ||
727 | To query the status of the firewall, you can query the status of the systemctl | |
728 | service: | |
729 | ||
730 | ---- | |
731 | systemctl status proxmox-firewall | |
732 | ---- | |
733 | ||
734 | ||
735 | [[pve_firewall_nft_helpful_commands]] | |
736 | Helpful Commands | |
737 | ~~~~~~~~~~~~~~~~ | |
738 | You can check the generated ruleset via the following command: | |
739 | ||
740 | ---- | |
741 | nft list ruleset | |
742 | ---- | |
743 | ||
744 | If you want to debug `proxmox-firewall` you can simply run the daemon in | |
745 | foreground with the `RUST_LOG` environment variable set to `trace`. This should | |
746 | provide you with detailed debugging output: | |
747 | ||
748 | ---- | |
749 | RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall | |
750 | ---- | |
751 | ||
752 | You can also edit the systemctl service if you want to have detailed output for | |
753 | your firewall daemon: | |
754 | ||
755 | ---- | |
756 | systemctl edit proxmox-firewall | |
757 | ---- | |
758 | ||
759 | Then you need to add the override for the `RUST_LOG` environment variable: | |
760 | ||
761 | ---- | |
762 | [Service] | |
763 | Environment="RUST_LOG=trace" | |
764 | ---- | |
765 | ||
766 | This will generate a large amount of logs very quickly, so only use this for | |
767 | debugging purposes. Other, less verbose, log levels are `info` and `debug`. | |
768 | ||
769 | Running in foreground writes the log output to STDERR, so you can redirect it | |
770 | with the following command (e.g. for submitting logs to the community forum): | |
771 | ||
772 | ---- | |
773 | RUST_LOG=trace /usr/libexec/proxmox/proxmox-firewall 2> firewall_log_$(hostname).txt | |
774 | ---- | |
775 | ||
776 | It can be helpful to trace packet flow through the different chains in order to | |
777 | debug firewall rules. This can be achieved by setting `nftrace` to 1 for packets | |
778 | that you want to track. It is advisable that you do not set this flag for *all* | |
779 | packets, in the example below we only examine ICMP packets. | |
780 | ||
781 | ---- | |
782 | #!/usr/sbin/nft -f | |
783 | table bridge tracebridge | |
784 | delete table bridge tracebridge | |
785 | ||
786 | table bridge tracebridge { | |
787 | chain trace { | |
788 | meta l4proto icmp meta nftrace set 1 | |
789 | } | |
790 | ||
791 | chain prerouting { | |
792 | type filter hook prerouting priority -350; policy accept; | |
793 | jump trace | |
794 | } | |
795 | ||
796 | chain postrouting { | |
797 | type filter hook postrouting priority -350; policy accept; | |
798 | jump trace | |
799 | } | |
800 | } | |
801 | ---- | |
802 | ||
803 | Saving this file, making it executable, and then running it once will create the | |
804 | respective tracing chains. You can then inspect the tracing output via the | |
805 | Proxmox VE Web UI (Firewall > Log) or via `nft monitor trace`. | |
806 | ||
807 | The above example traces traffic on all bridges, which is usually where guest | |
808 | traffic flows through. If you want to examine host traffic, create those chains | |
809 | in the `inet` table instead of the `bridge` table. | |
810 | ||
811 | NOTE: Be aware that this can generate a *lot* of log spam and slow down the | |
812 | performance of your networking stack significantly. | |
813 | ||
814 | You can remove the tracing rules via running the following command: | |
815 | ||
816 | ---- | |
817 | nft delete table bridge tracebridge | |
818 | ---- | |
819 | ||
820 | ||
14c06023 DM |
821 | ifdef::manvolnum[] |
822 | ||
823 | Macro Definitions | |
824 | ----------------- | |
825 | ||
826 | include::pve-firewall-macros.adoc[] | |
827 | ||
828 | ||
829 | include::pve-copyright.adoc[] | |
830 | ||
831 | endif::manvolnum[] |