selected in the installer.
For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is
-used. All other deployments use the standard `grub` bootloader (this usually
-also applies to systems which are installed on top of Debian).
+used, unless Secure Boot is enabled. All other deployments use the standard
+`grub` bootloader (this usually also applies to systems which are installed on
+top of Debian).
[[sysboot_installer_part_scheme]]
used for the chosen storage type
Systems using ZFS as root filesystem are booted with a kernel and initrd image
-stored on the 512 MB EFI System Partition. For legacy BIOS systems, `grub` is
-used, for EFI systems `systemd-boot` is used. Both are installed and configured
-to point to the ESPs.
+stored on the 512 MB EFI System Partition. For legacy BIOS systems, and EFI
+systems with Secure Boot enabled, `grub` is used, for EFI systems without
+Secure Boot, `systemd-boot` is used. Both are installed and configured to point
+to the ESPs.
`grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
Partition of all selected disks on all systems booted with `grub`
To format and initialize a partition as synced ESP, e.g., after replacing a
failed vdev in an rpool, or when converting an existing system that pre-dates
-the sync mechanism, `proxmox-boot-tool` from `pve-kernel-helpers` can be used.
+the sync mechanism, `proxmox-boot-tool` from `proxmox-kernel-helper` can be used.
WARNING: the `format` command will format the `<partition>`, make sure to pass
in the right device/partition!
# proxmox-boot-tool init /dev/sda2
----
+or
+
+----
+# proxmox-boot-tool init /dev/sda2 grub
+----
+
+to force initialization with Grub instead of systemd-boot, for example for
+Secure Boot support.
+
Afterwards `/etc/kernel/proxmox-boot-uuids` should contain a new line with the
UUID of the newly added partition. The `init` command will also automatically
trigger a refresh of all configured ESPs.
To apply your changes, run `proxmox-boot-tool refresh`, which sets it as the
`option` line for all config files in `loader/entries/proxmox-*.conf`.
+A complete list of kernel parameters can be found at
+'https://www.kernel.org/doc/html/v<YOUR-KERNEL-VERSION>/admin-guide/kernel-parameters.html'.
+replace <YOUR-KERNEL-VERSION> with the major.minor version, for example, for
+kernels based on version 6.5 the URL would be:
+https://www.kernel.org/doc/html/v6.5/admin-guide/kernel-parameters.html
+
+You can find your kernel version by checking the web interface ('Node ->
+Summary'), or by running
+
+----
+# uname -r
+----
+
+Use the first two numbers at the front of the output.
[[sysboot_kernel_pin]]
-Selecting the kernel-version for booting
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In addition to actively watching the boot process to select an older kernel
-version to boot into, you can also use `proxmox-boot-tool` to `pin` the kernel
-version the system should use. This should help you to work around
-incompatibilities between a newer kernel version and the hardware. Such a
-`pin` should be removed as soon as possible in order to have all the latest
-security patches in place.
-
-NOTE: The pinning functionality works for all {pve} systems, not only those
-using `proxmox-boot-tool` to synchronize the contents of the ESPs, if your
-system does not use `proxmox-boot-tool` for synchronizing you can also skip the
-`proxmox-boot-tool refresh` call in the end.
+Override the Kernel-Version for next Boot
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To select a kernel that is not currently the default kernel, you can either:
+
+* use the boot loader menu that is displayed at the beginning of the boot
+ process
+* use the `proxmox-boot-tool` to `pin` the system to a kernel version either
+ once or permanently (until pin is reset).
+
+This should help you work around incompatibilities between a newer kernel
+version and the hardware.
-To permanently select the version `5.15.30-1-pve` for booting run:
+NOTE: Such a pin should be removed as soon as possible so that all current
+security patches of the latest kernel are also applied to the system.
+
+For example: To permanently select the version `5.15.30-1-pve` for booting you
+would run:
----
# proxmox-boot-tool kernel pin 5.15.30-1-pve
----
-You can also set a kernel version to be booted on the next system boot only
-(e.g. to test if an updated kernel has resolved an issue, which caused you
-to `pin` a version in the first place):
+TIP: The pinning functionality works for all {pve} systems, not only those using
+`proxmox-boot-tool` to synchronize the contents of the ESPs, if your system
+does not use `proxmox-boot-tool` for synchronizing you can also skip the
+`proxmox-boot-tool refresh` call in the end.
+
+You can also set a kernel version to be booted on the next system boot only.
+This is for example useful to test if an updated kernel has resolved an issue,
+which caused you to `pin` a version in the first place:
----
-# proxmox-boot-tool kernel pin 5.15.31-1-pve --next-boot
+# proxmox-boot-tool kernel pin 5.15.30-1-pve --next-boot
----
To remove any pinned version configuration use the `unpin` subcommand:
----
While `unpin` has a `--next-boot` option as well, it is used to clear a pinned
-version set with `--next-boot` and manually invoking is of little use.
+version set with `--next-boot`. As that happens already automatically on boot,
+invonking it manually is of little use.
+
+After setting, or clearing pinned versions you also need to synchronize the
+content and configuration on the ESPs by running the `refresh` subcommand.
-After setting or clearing pinned versions you also need to synchronize the
-content and configuration on the ESPs by running the `refresh` subcommand:
+TIP: You will be prompted to automatically do for `proxmox-boot-tool` managed
+systems if you call the tool interactively.
----
# proxmox-boot-tool refresh
----
+
+[[sysboot_secure_boot]]
+Secure Boot
+~~~~~~~~~~~
+
+Since {pve} 8.1, Secure Boot is supported out of the box via signed packages
+and integration in `proxmox-boot-tool`.
+
+The following packages need to be installed for Secure Boot to be enabled:
+
+- shim-signed (shim bootloader signed by Microsoft)
+- shim-helpers-amd64-signed (fallback bootloader and MOKManager, signed by Proxmox)
+- grub-efi-amd64-signed (Grub EFI bootloader, signed by Proxmox)
+- proxmox-kernel-6.X.Y-Z-pve-signed (Kernel image, signed by Proxmox)
+
+Only Grub as bootloader is supported out of the box, since there are no other
+pre-signed bootloader packages available. Any new installation of {pve} will
+automatically have all of the above packages included.
+
+More details about how Secure Boot works, and how to customize the setup, are
+available in https://pve.proxmox.com/wiki/Secure_Boot_Setup[our wiki].
+
+Switching an Existing Installation to Secure Boot
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+WARNING: This can lead to an unbootable installation in some cases if not done
+correctly. Reinstalling the host will setup Secure Boot automatically if
+available, without any extra interactions. **Make sure you have a working and
+well-tested backup of your {pve} host!**
+
+An existing UEFI installation can be switched over to Secure Boot if desired,
+without having to reinstall {pve} from scratch.
+
+First, ensure all your system is up-to-date. Next, install all the required
+pre-signed packages as listed above. Grub automatically creates the needed EFI
+boot entry for booting via the default shim.
+
+.systemd-boot
+
+If `systemd-boot` is used as a bootloader (see
+xref:sysboot_determine_bootloader_used[Determine which Bootloader is used]),
+some additional setup is needed. This is only the case if {pve} was installed
+with ZFS-on-root.
+
+To check the latter, run:
+----
+# findmnt /
+----
+
+If the host is indeed running using ZFS as root filesystem, the `FSTYPE` column
+should contain `zfs`:
+----
+TARGET SOURCE FSTYPE OPTIONS
+/ rpool/ROOT/pve-1 zfs rw,relatime,xattr,noacl,casesensitive
+----
+
+Next, a suitable potential ESP (EFI system partition) must be found. This can be
+done using the `lsblk` command as following:
+----
+# lsblk -o +FSTYPE
+----
+
+The output should look something like this:
+----
+NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS FSTYPE
+sda 8:0 0 32G 0 disk
+├─sda1 8:1 0 1007K 0 part
+├─sda2 8:2 0 512M 0 part vfat
+└─sda3 8:3 0 31.5G 0 part zfs_member
+sdb 8:16 0 32G 0 disk
+├─sdb1 8:17 0 1007K 0 part
+├─sdb2 8:18 0 512M 0 part vfat
+└─sdb3 8:19 0 31.5G 0 part zfs_member
+----
+
+In this case, the partitions `sda2` and `sdb2` are the targets. They can be
+identified by the their size of 512M and their `FSTYPE` being `vfat`, in this
+case on a ZFS RAID-1 installation.
+
+These partitions must be properly set up for booting through Grub using
+`proxmox-boot-tool`. This command (using `sda2` as an example) must be run
+separately for each individual ESP:
+----
+# proxmox-boot-tool init /dev/sda2 grub
+----
+
+Afterwards, you can sanity-check the setup by running the following command:
+----
+# efibootmgr -v
+----
+
+This list should contain an entry looking similar to this:
+----
+[..]
+Boot0009* proxmox HD(2,GPT,..,0x800,0x100000)/File(\EFI\proxmox\shimx64.efi)
+[..]
+----
+
+NOTE: The old `systemd-boot` bootloader will be kept, but Grub will be
+preferred. This way, if booting using Grub in Secure Boot mode does not work for
+any reason, the system can still be booted using `systemd-boot` with Secure Boot
+turned off.
+
+Now the host can be rebooted and Secure Boot enabled in the UEFI firmware setup
+utility.
+
+On reboot, a new entry named `proxmox` should be selectable in the UEFI firmware
+boot menu, which boots using the pre-signed EFI shim.
+
+If, for any reason, no `proxmox` entry can be found in the UEFI boot menu, you
+can try adding it manually (if supported by the firmware), by adding the file
+`\EFI\proxmox\shimx64.efi` as a custom boot entry.
+
+NOTE: Some UEFI firmwares are known to drop the `proxmox` boot option on reboot.
+This can happen if the `proxmox` boot entry is pointing to a Grub installation
+on a disk, where the disk itself not a boot option. If possible, try adding the
+disk as a boot option in the UEFI firmware setup utility and run
+`proxmox-boot-tool` again.
+
+TIP: To enroll custom keys, see the accompanying
+https://pve.proxmox.com/wiki/Secure_Boot_Setup#Setup_instructions_for_db_key_variant[Secure
+Boot wiki page].
+
+Using DKMS/third party modules with Secure Boot
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+On systems with Secure Boot enabled, the kernel will refuse to load modules
+which are not signed by a trusted key. The default set of modules shipped with
+the kernel packages is signed with an ephemeral key embedded in the kernel
+image which is trusted by that specific version of the kernel image.
+
+In order to load other modules, such as those built with DKMS or manually, they
+need to be signed with a key trusted by the Secure Boot stack. The easiest way
+to achieve this is to enroll them as Machine Owner Key (`MOK`) with `mokutil`.
+
+The `dkms` tool will automatically generate a keypair and certificate in
+`/var/lib/dkms/mok.key` and `/var/lib/dkms/mok.pub` and use it for signing
+the kernel modules it builds and installs.
+
+You can view the certificate contents with
+
+----
+# openssl x509 -in /var/lib/dkms/mok.pub -noout -text
+----
+
+and enroll it on your system using the following command:
+
+----
+# mokutil --import /var/lib/dkms/mok.pub
+input password:
+input password again:
+----
+
+The `mokutil` command will ask for a (temporary) password twice, this password
+needs to be entered one more time in the next step of the process! Rebooting
+the system should automatically boot into the `MOKManager` EFI binary, which
+allows you to verify the key/certificate and confirm the enrollment using the
+password selected when starting the enrollment using `mokutil`. Afterwards, the
+kernel should allow loading modules built with DKMS (which are signed with the
+enrolled `MOK`). The `MOK` can also be used to sign custom EFI binaries and
+kernel images if desired.
+
+The same procedure can also be used for custom/third-party modules not managed
+with DKMS, but the key/certificate generation and signing steps need to be done
+manually in that case.
projects / pve-docs.git / blobdiff