| 1 | ifdef::manvolnum[] |
| 2 | PVE({manvolnum}) |
| 3 | ================ |
| 4 | include::attributes.txt[] |
| 5 | |
| 6 | NAME |
| 7 | ---- |
| 8 | |
| 9 | pve-firewall - The PVE Firewall Daemon |
| 10 | |
| 11 | |
| 12 | SYNOPSYS |
| 13 | -------- |
| 14 | |
| 15 | include::pve-firewall.8-synopsis.adoc[] |
| 16 | |
| 17 | |
| 18 | DESCRIPTION |
| 19 | ----------- |
| 20 | endif::manvolnum[] |
| 21 | |
| 22 | ifndef::manvolnum[] |
| 23 | {pve} Firewall |
| 24 | ============== |
| 25 | include::attributes.txt[] |
| 26 | endif::manvolnum[] |
| 27 | |
| 28 | // Copied from pve wiki: Revision as of 08:45, 9 November 2015 |
| 29 | |
| 30 | Proxmox VE Firewall provides an easy way to protect your IT |
| 31 | infrastructure. You can easily setup firewall rules for all hosts |
| 32 | inside a cluster, or define rules for virtual machines and |
| 33 | containers. Features like firewall macros, security groups, IP sets |
| 34 | and aliases help making that task easier. |
| 35 | |
| 36 | While all configuration is stored on the cluster file system, the |
| 37 | iptables based firewall runs on each cluster node, and thus provides |
| 38 | full isolation between virtual machines. The distributed nature of |
| 39 | this system also provides much higher bandwidth than a central |
| 40 | firewall solution. |
| 41 | |
| 42 | NOTE: If you enable the firewall, all traffic is blocked by default, |
| 43 | except WebGUI(8006) and ssh(22) from your local network. |
| 44 | |
| 45 | The firewall has full support for IPv4 and IPv6. IPv6 support is fully |
| 46 | transparent, and we filter traffic for both protocols by default. So |
| 47 | there is no need to maintain a different set of rules for IPv6. |
| 48 | |
| 49 | |
| 50 | Zones |
| 51 | ----- |
| 52 | |
| 53 | The Proxmox VE firewall groups the network into the following logical zones: |
| 54 | |
| 55 | Host:: |
| 56 | |
| 57 | Traffic from/to a cluster node |
| 58 | |
| 59 | VM:: |
| 60 | |
| 61 | Traffic from/to a specific VM |
| 62 | |
| 63 | For each zone, you can define firewall rules for incoming and/or |
| 64 | outgoing traffic. |
| 65 | |
| 66 | |
| 67 | Configuration |
| 68 | ------------- |
| 69 | |
| 70 | All firewall related configuration is stored on the proxmox cluster |
| 71 | file system. So those files are automatically distributed to all |
| 72 | cluster nodes, and the 'pve-firewall' service updates the underlying |
| 73 | iptables rules automatically on any change. Any configuration can be |
| 74 | done using the GUI (i.e. Datacenter -> Firewall -> Options tab (tabs |
| 75 | at the bottom of the page), or on a Node -> Firewall), so the |
| 76 | following configuration file snippets are just for completeness. |
| 77 | |
| 78 | Cluster wide configuration is stored at: |
| 79 | |
| 80 | /etc/pve/firewall/cluster.fw |
| 81 | |
| 82 | The firewall is completely disabled by default, so you need to set the |
| 83 | enable option here: |
| 84 | |
| 85 | ---- |
| 86 | [OPTIONS] |
| 87 | # enable firewall (cluster wide setting, default is disabled) |
| 88 | enable: 1 |
| 89 | ---- |
| 90 | |
| 91 | The cluster wide configuration can contain the following data: |
| 92 | |
| 93 | * IP set definitions |
| 94 | * Alias definitions |
| 95 | * Security group definitions |
| 96 | * Cluster wide firewall rules for all nodes |
| 97 | |
| 98 | VM firewall configuration is read from: |
| 99 | |
| 100 | /etc/pve/firewall/<VMID>.fw |
| 101 | |
| 102 | and contains the following data: |
| 103 | |
| 104 | * IP set definitions |
| 105 | * Alias definitions |
| 106 | * Firewall rules for this VM |
| 107 | * VM specific options |
| 108 | |
| 109 | And finally, any host related configuration is read from: |
| 110 | |
| 111 | /etc/pve/nodes/<nodename>/host.fw |
| 112 | |
| 113 | This is useful if you want to overwrite rules from 'cluster.fw' |
| 114 | config. You can also increase log verbosity, and set netfilter related |
| 115 | options. |
| 116 | |
| 117 | Enabling the Firewall for VMs and Containers |
| 118 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 119 | |
| 120 | You need to enable the firewall on the virtual network interface configuration |
| 121 | in addition to the general 'Enable Firewall' option in the 'Options' tab. |
| 122 | |
| 123 | Firewall Rules |
| 124 | ~~~~~~~~~~~~~~ |
| 125 | |
| 126 | Firewall rules consists of a direction (`IN` or `OUT`) and an |
| 127 | action (`ACCEPT`, `DENY`, `REJECT`). You can also specify a macro |
| 128 | name. Macros contain predifined sets of rules and options. Rules can be disabled by prefixing them with '|'. |
| 129 | |
| 130 | .Firewall rules syntax |
| 131 | ---- |
| 132 | [RULES] |
| 133 | |
| 134 | DIRECTION ACTION [OPTIONS] |
| 135 | |DIRECTION ACTION [OPTIONS] # disabled rule |
| 136 | |
| 137 | DIRECTION MACRO(ACTION) [OPTIONS] # use predefined macro |
| 138 | ---- |
| 139 | |
| 140 | The following options can be used to refine rule matches. |
| 141 | |
| 142 | include::pve-firewall-rules-opts.adoc[] |
| 143 | |
| 144 | Here are some examples: |
| 145 | |
| 146 | ---- |
| 147 | [RULES] |
| 148 | IN SSH(ACCEPT) -i net0 |
| 149 | IN SSH(ACCEPT) -i net0 # a comment |
| 150 | IN SSH(ACCEPT) -i net0 -source 192.168.2.192 # only allow SSH from 192.168.2.192 |
| 151 | IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for ip range |
| 152 | IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for ip list |
| 153 | IN SSH(ACCEPT) -i net0 -source +mynetgroup # accept ssh for ipset mynetgroup |
| 154 | IN SSH(ACCEPT) -i net0 -source myserveralias #accept ssh for alias myserveralias |
| 155 | |
| 156 | |IN SSH(ACCEPT) -i net0 # disabled rule |
| 157 | |
| 158 | IN DROP # drop all incoming packages |
| 159 | OUT ACCEPT # accept all outgoing packages |
| 160 | ---- |
| 161 | |
| 162 | Security Groups |
| 163 | ~~~~~~~~~~~~~~~ |
| 164 | |
| 165 | A security group is a collection of rules, defined at cluster level, which |
| 166 | can be used in all VMs' rules. For example you can define a group named |
| 167 | `webserver` with rules to open the http and https ports. |
| 168 | |
| 169 | ---- |
| 170 | # /etc/pve/firewall/cluster.fw |
| 171 | |
| 172 | [group webserver] |
| 173 | IN ACCEPT -p tcp -dport 80 |
| 174 | IN ACCEPT -p tcp -dport 443 |
| 175 | ---- |
| 176 | |
| 177 | Then, you can add this group to a VM's firewall |
| 178 | |
| 179 | ---- |
| 180 | # /etc/pve/firewall/<VMID>.fw |
| 181 | |
| 182 | [RULES] |
| 183 | GROUP webserver |
| 184 | ---- |
| 185 | |
| 186 | |
| 187 | IP Aliases |
| 188 | ~~~~~~~~~~ |
| 189 | |
| 190 | IP Aliases allow you to associate IP addresses of networks with a |
| 191 | name. You can then refer to those names: |
| 192 | |
| 193 | * inside IP set definitions |
| 194 | * in `source` and `dest` properties of firewall rules |
| 195 | |
| 196 | Standard IP alias `local_network` |
| 197 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 198 | |
| 199 | This alias is automatically defined. Please use the following command |
| 200 | to see assigned values: |
| 201 | |
| 202 | ---- |
| 203 | # pve-firewall localnet |
| 204 | local hostname: example |
| 205 | local IP address: 192.168.2.100 |
| 206 | network auto detect: 192.168.0.0/20 |
| 207 | using detected local_network: 192.168.0.0/20 |
| 208 | ---- |
| 209 | |
| 210 | The firewall automatically sets up rules to allow everything needed |
| 211 | for cluster communication (corosync, API, SSH) using this alias. |
| 212 | |
| 213 | The user can overwrite these values in the cluster.fw alias |
| 214 | section. If you use a single host on a public network, it is better to |
| 215 | explicitly assign the local IP address |
| 216 | |
| 217 | ---- |
| 218 | # /etc/pve/firewall/cluster.fw |
| 219 | [ALIASES] |
| 220 | local_network 1.2.3.4 # use the single ip address |
| 221 | ---- |
| 222 | |
| 223 | IP Sets |
| 224 | ~~~~~~~ |
| 225 | |
| 226 | IP sets can be used to define groups of networks and hosts. You can |
| 227 | refer to them with `+name` in the firewall rules' `source` and `dest` |
| 228 | properties. |
| 229 | |
| 230 | The following example allows HTTP traffic from the `management` IP |
| 231 | set. |
| 232 | |
| 233 | IN HTTP(ACCEPT) -source +management |
| 234 | |
| 235 | Standard IP set `management` |
| 236 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 237 | |
| 238 | This IP set applies only to host firewalls (not VM firewalls). Those |
| 239 | ips are allowed to do normal management tasks (PVE GUI, VNC, SPICE, |
| 240 | SSH). |
| 241 | |
| 242 | The local cluster network is automatically added to this IP set (alias |
| 243 | `cluster_network`), to enable inter-host cluster |
| 244 | communication. (multicast,ssh,...) |
| 245 | |
| 246 | ---- |
| 247 | # /etc/pve/firewall/cluster.fw |
| 248 | |
| 249 | [IPSET management] |
| 250 | 192.168.2.10 |
| 251 | 192.168.2.10/24 |
| 252 | ---- |
| 253 | |
| 254 | Standard IP set 'blacklist' |
| 255 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 256 | |
| 257 | Traffic from these ips is dropped by every host's and VM's firewall. |
| 258 | |
| 259 | ---- |
| 260 | # /etc/pve/firewall/cluster.fw |
| 261 | |
| 262 | [IPSET blacklist] |
| 263 | 77.240.159.182 |
| 264 | 213.87.123.0/24 |
| 265 | ---- |
| 266 | |
| 267 | [[ipfilter-section]] |
| 268 | Standard IP set 'ipfilter-net*' |
| 269 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 270 | |
| 271 | These filters belong to a VM's network interface and are mainly used to prevent |
| 272 | IP spoofing. If such a set exists for an interface then any outgoing traffic |
| 273 | with a source IP not matching its interface's corresponding ipfilter set will |
| 274 | be dropped. |
| 275 | |
| 276 | For containers with configured IP addresses these sets, if they exist (or are |
| 277 | activated via the general `IP Filter` option in the VM's firewall's 'options' |
| 278 | tab), implicitly contain the associated IP addresses. |
| 279 | |
| 280 | For both virtual machines and containers they also implicitly contain the |
| 281 | standard MAC-derived IPv6 link-local address in order to allow the neighbor |
| 282 | discovery protocol to work. |
| 283 | |
| 284 | ---- |
| 285 | /etc/pve/firewall/<VMID>.fw |
| 286 | |
| 287 | [IPSET ipfilter-net0] # only allow specified IPs on net0 |
| 288 | 192.168.2.10 |
| 289 | ---- |
| 290 | |
| 291 | Services and Commands |
| 292 | ~~~~~~~~~~~~~~~~~~~~~ |
| 293 | |
| 294 | The firewall runs two service daemons on each node: |
| 295 | |
| 296 | * pvefw-logger: NFLOG daemon (ulogd replacement). |
| 297 | * pve-firewall: updates iptables rules |
| 298 | |
| 299 | There is also a CLI command named 'pve-firewall', which can be used to |
| 300 | start and stop the firewall service: |
| 301 | |
| 302 | # pve-firewall start |
| 303 | # pve-firewall stop |
| 304 | |
| 305 | To get the status use: |
| 306 | |
| 307 | # pve-firewall status |
| 308 | |
| 309 | The above command reads and compiles all firewall rules, so you will |
| 310 | see warnings if your firewall configuration contains any errors. |
| 311 | |
| 312 | If you want to see the generated iptables rules you can use: |
| 313 | |
| 314 | # iptables-save |
| 315 | |
| 316 | Tips and Tricks |
| 317 | ~~~~~~~~~~~~~~~ |
| 318 | |
| 319 | How to allow FTP |
| 320 | ^^^^^^^^^^^^^^^^ |
| 321 | |
| 322 | FTP is an old style protocol which uses port 21 and several other dynamic ports. So you |
| 323 | need a rule to accept port 21. In addition, you need to load the 'ip_conntrack_ftp' module. |
| 324 | So please run: |
| 325 | |
| 326 | modprobe ip_conntrack_ftp |
| 327 | |
| 328 | and add `ip_conntrack_ftp` to '/etc/modules' (so that it works after a reboot) . |
| 329 | |
| 330 | Suricata IPS integration |
| 331 | ^^^^^^^^^^^^^^^^^^^^^^^^ |
| 332 | |
| 333 | If you want to use the http://suricata-ids.org/[Suricata IPS] |
| 334 | (Intrusion Prevention System), it's possible. |
| 335 | |
| 336 | Packets will be forwarded to the IPS only after the firewall ACCEPTed |
| 337 | them. |
| 338 | |
| 339 | Rejected/Dropped firewall packets don't go to the IPS. |
| 340 | |
| 341 | Install suricata on proxmox host: |
| 342 | |
| 343 | ---- |
| 344 | # apt-get install suricata |
| 345 | # modprobe nfnetlink_queue |
| 346 | ---- |
| 347 | |
| 348 | Don't forget to add `nfnetlink_queue` to '/etc/modules' for next reboot. |
| 349 | |
| 350 | Then, enable IPS for a specific VM with: |
| 351 | |
| 352 | ---- |
| 353 | # /etc/pve/firewall/<VMID>.fw |
| 354 | |
| 355 | [OPTIONS] |
| 356 | ips: 1 |
| 357 | ips_queues: 0 |
| 358 | ---- |
| 359 | |
| 360 | `ips_queues` will bind a specific cpu queue for this VM. |
| 361 | |
| 362 | Available queues are defined in |
| 363 | |
| 364 | ---- |
| 365 | # /etc/default/suricata |
| 366 | NFQUEUE=0 |
| 367 | ---- |
| 368 | |
| 369 | Notes on IPv6 |
| 370 | ^^^^^^^^^^^^^ |
| 371 | |
| 372 | The firewall contains a few IPv6 specific options. One thing to note is that |
| 373 | IPv6 does not use the ARP protocol anymore, and instead uses NDP (Neighbor |
| 374 | Discovery Protocol) which works on IP level and thus needs IP addresses to |
| 375 | succeed. For this purpose link-local addresses derived from the interface's MAC |
| 376 | address are used. By default the 'NDP' option is enabled on both host and VM |
| 377 | level to allow neighbor discovery (NDP) packets to be sent and received. |
| 378 | |
| 379 | Beside neighbor discovery NDP is also used for a couple of other things, like |
| 380 | autoconfiguration and advertising routers. |
| 381 | |
| 382 | By default VMs are allowed to send out router solicitation messages (to query |
| 383 | for a router), and to receive router advetisement packets. This allows them to |
| 384 | use stateless auto configuration. On the other hand VMs cannot advertise |
| 385 | themselves as routers unless the 'Allow Router Advertisement' (`radv: 1`) option |
| 386 | is set. |
| 387 | |
| 388 | As for the link local addresses required for NDP, there's also an 'IP Filter' |
| 389 | (`ipfilter: 1`) option which can be enabled which has the same effect as adding |
| 390 | an `ipfilter-net*` ipset for each of the VM's network interfaces containing the |
| 391 | corresponding link local addresses. (See the |
| 392 | <<ipfilter-section,Standard IP set 'ipfilter-net*'>> section for details.) |
| 393 | |
| 394 | Avoiding link-local addresses on tap and veth devices |
| 395 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 396 | |
| 397 | With IPv6 enabled by default every interface gets a MAC-derived link local |
| 398 | address. However, most devices on a typical {pve} setup are connected to a |
| 399 | bridge and so the bridge is the only interface which really needs one. |
| 400 | |
| 401 | To disable a link local address on an interface you can set the interface's |
| 402 | `disable_ipv6` sysconf variable. Despite the name, this does not prevent IPv6 |
| 403 | traffic from passing through the interface when routing or bridging, so the |
| 404 | only noticeable effect will be the removal of the link local address. |
| 405 | |
| 406 | The easiest method of achieving this setting for all newly started VMs is to |
| 407 | set it for the `default` interface configuration and enabling it explicitly on |
| 408 | the interfaces which need it. This is also the case for other settings such as |
| 409 | `forwarding`, `accept_ra` or `autoconf`. |
| 410 | |
| 411 | Here's a possible setup: |
| 412 | ---- |
| 413 | # /etc/sysconf.d/90-ipv6.conf |
| 414 | |
| 415 | net.ipv6.conf.default.forwarding = 0 |
| 416 | net.ipv6.conf.default.proxy_ndp = 0 |
| 417 | net.ipv6.conf.default.autoconf = 0 |
| 418 | net.ipv6.conf.default.disable_ipv6 = 1 |
| 419 | net.ipv6.conf.default.accept_ra = 0 |
| 420 | |
| 421 | net.ipv6.conf.lo.disable_ipv6 = 0 |
| 422 | ---- |
| 423 | |
| 424 | ---- |
| 425 | # /etc/network/interfaces |
| 426 | (...) |
| 427 | iface vmbr0 inet6 static |
| 428 | address fc00::31 |
| 429 | netmask 16 |
| 430 | gateway fc00::1 |
| 431 | accept_ra 0 |
| 432 | pre-up echo 0 > /proc/sys/net/ipv6/conf/$IFACE/disable_ipv6 |
| 433 | (...) |
| 434 | ---- |
| 435 | |
| 436 | Ports used by Proxmox VE |
| 437 | ------------------------ |
| 438 | |
| 439 | * Web interface: 8006 |
| 440 | * VNC Web console: 5900-5999 |
| 441 | * SPICE proxy: 3128 |
| 442 | * sshd (used for cluster actions): 22 |
| 443 | * rpcbind: 111 |
| 444 | * corosync multicast (if you run a cluster): 5404, 5405 UDP |
| 445 | |
| 446 | |
| 447 | ifdef::manvolnum[] |
| 448 | |
| 449 | Macro Definitions |
| 450 | ----------------- |
| 451 | |
| 452 | include::pve-firewall-macros.adoc[] |
| 453 | |
| 454 | |
| 455 | include::pve-copyright.adoc[] |
| 456 | |
| 457 | endif::manvolnum[] |