For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is
used, unless Secure Boot is enabled. All other deployments use the standard
-`grub` bootloader (this usually also applies to systems which are installed on
+GRUB bootloader (this usually also applies to systems which are installed on
top of Debian).
Systems using ZFS as root filesystem are booted with a kernel and initrd image
stored on the 512 MB EFI System Partition. For legacy BIOS systems, and EFI
-systems with Secure Boot enabled, `grub` is used, for EFI systems without
+systems with Secure Boot enabled, GRUB is used, for EFI systems without
Secure Boot, `systemd-boot` is used. Both are installed and configured to point
to the ESPs.
-`grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
-Partition of all selected disks on all systems booted with `grub`
+GRUB in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
+Partition of all selected disks on all systems booted with GRUB
footnote:[These are all installs with root on `ext4` or `xfs` and installs
with root on ZFS on non-EFI systems].
versions to all ESPs and configures the respective bootloader to boot from
the `vfat` formatted ESPs. In the context of ZFS as root filesystem this means
that you can use all optional features on your root pool instead of the subset
-which is also present in the ZFS implementation in `grub` or having to create a
-separate small boot-pool footnote:[Booting ZFS on root with grub
+which is also present in the ZFS implementation in GRUB or having to create a
+separate small boot-pool footnote:[Booting ZFS on root with GRUB
https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS].
In setups with redundancy all disks are partitioned with an ESP, by the
# proxmox-boot-tool init /dev/sda2 grub
----
-to force initialization with Grub instead of systemd-boot, for example for
+to force initialization with GRUB instead of `systemd-boot`, for example for
Secure Boot support.
Afterwards `/etc/kernel/proxmox-boot-uuids` should contain a new line with the
The simplest and most reliable way to determine which bootloader is used, is to
watch the boot process of the {pve} node.
-You will either see the blue box of `grub` or the simple black on white
+You will either see the blue box of GRUB or the simple black on white
`systemd-boot`.
[thumbnail="screenshot/boot-systemdboot.png"]
# efibootmgr -v
----
-If it returns a message that EFI variables are not supported, `grub` is used in
+If it returns a message that EFI variables are not supported, GRUB is used in
BIOS/Legacy mode.
-If the output contains a line that looks similar to the following, `grub` is
+If the output contains a line that looks similar to the following, GRUB is
used in UEFI mode.
----
[[sysboot_grub]]
-Grub
+GRUB
~~~~
-`grub` has been the de-facto standard for booting Linux systems for many years
+GRUB has been the de-facto standard for booting Linux systems for many years
and is quite well documented
-footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
+footnote:[GRUB Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
Configuration
^^^^^^^^^^^^^
-Changes to the `grub` configuration are done via the defaults file
+Changes to the GRUB configuration are done via the defaults file
`/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate
the configuration file after a change to the configuration run:
footnote:[Systems using `proxmox-boot-tool` will call `proxmox-boot-tool
You can modify the kernel commandline in the following places, depending on the
bootloader used:
-.Grub
+.GRUB
The kernel commandline needs to be placed in the variable
`GRUB_CMDLINE_LINUX_DEFAULT` in the file `/etc/default/grub`. Running
Since {pve} 8.1, Secure Boot is supported out of the box via signed packages
and integration in `proxmox-boot-tool`.
-The following packages need to be installed for Secure Boot to be enabled:
+The following packages are required for secure boot to work. You can
+install them all at once by using the `proxmox-secure-boot-support'
+meta-package.
-- shim-signed (shim bootloader signed by Microsoft)
-- shim-helpers-amd64-signed (fallback bootloader and MOKManager, signed by Proxmox)
-- grub-efi-amd64-signed (Grub EFI bootloader, signed by Proxmox)
-- proxmox-kernel-6.X.Y-Z-pve-signed (Kernel image, signed by Proxmox)
+- `shim-signed` (shim bootloader signed by Microsoft)
+- `shim-helpers-amd64-signed` (fallback bootloader and MOKManager, signed by
+ Proxmox)
+- `grub-efi-amd64-signed` (GRUB EFI bootloader, signed by Proxmox)
+- `proxmox-kernel-6.X.Y-Z-pve-signed` (Kernel image, signed by Proxmox)
-Only Grub as bootloader is supported out of the box, since there are no other
-pre-signed bootloader packages available. Any new installation of {pve} will
-automatically have all of the above packages included.
+Only GRUB is supported as bootloader out of the box, since other bootloader are
+currently not eligible for secure boot code-signing.
+
+Any new installation of {pve} will automatically have all of the above packages
+included.
More details about how Secure Boot works, and how to customize the setup, are
available in https://pve.proxmox.com/wiki/Secure_Boot_Setup[our wiki].
-Switching an existing installation to Secure Boot
+Switching an Existing Installation to Secure Boot
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-WARNING: This can lead to unbootable installation in some cases if not done
+WARNING: This can lead to an unbootable installation in some cases if not done
correctly. Reinstalling the host will setup Secure Boot automatically if
available, without any extra interactions. **Make sure you have a working and
well-tested backup of your {pve} host!**
An existing UEFI installation can be switched over to Secure Boot if desired,
without having to reinstall {pve} from scratch.
-First, ensure all your system is up-to-date. Next, install all the required
-pre-signed packages as listed above. Grub automatically creates the needed EFI
-boot entry for booting via the default shim.
+First, ensure all your system is up-to-date. Next, install
+`proxmox-secure-boot-support`. GRUB automatically creates the needed EFI boot
+entry for booting via the default shim.
.systemd-boot
# findmnt /
----
-If the host is indeed running using ZFS as root filesystem, the `FSTYPE` column
+If the host is indeed using ZFS as root filesystem, the `FSTYPE` column
should contain `zfs`:
----
TARGET SOURCE FSTYPE OPTIONS
identified by the their size of 512M and their `FSTYPE` being `vfat`, in this
case on a ZFS RAID-1 installation.
-These partitions must be properly set up for booting through Grub using
+These partitions must be properly set up for booting through GRUB using
`proxmox-boot-tool`. This command (using `sda2` as an example) must be run
separately for each individual ESP:
----
[..]
----
-NOTE: The old `systemd-boot` bootloader will be kept, but Grub will be
-preferred. This way, if booting using Grub in Secure Boot mode does not work for
+NOTE: The old `systemd-boot` bootloader will be kept, but GRUB will be
+preferred. This way, if booting using GRUB in Secure Boot mode does not work for
any reason, the system can still be booted using `systemd-boot` with Secure Boot
turned off.
can try adding it manually (if supported by the firmware), by adding the file
`\EFI\proxmox\shimx64.efi` as a custom boot entry.
+NOTE: Some UEFI firmwares are known to drop the `proxmox` boot option on reboot.
+This can happen if the `proxmox` boot entry is pointing to a GRUB installation
+on a disk, where the disk itself is not a boot option. If possible, try adding
+the disk as a boot option in the UEFI firmware setup utility and run
+`proxmox-boot-tool` again.
+
TIP: To enroll custom keys, see the accompanying
https://pve.proxmox.com/wiki/Secure_Boot_Setup#Setup_instructions_for_db_key_variant[Secure
Boot wiki page].
+
+Using DKMS/Third Party Modules With Secure Boot
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+On systems with Secure Boot enabled, the kernel will refuse to load modules
+which are not signed by a trusted key. The default set of modules shipped with
+the kernel packages is signed with an ephemeral key embedded in the kernel
+image which is trusted by that specific version of the kernel image.
+
+In order to load other modules, such as those built with DKMS or manually, they
+need to be signed with a key trusted by the Secure Boot stack. The easiest way
+to achieve this is to enroll them as Machine Owner Key (`MOK`) with `mokutil`.
+
+The `dkms` tool will automatically generate a keypair and certificate in
+`/var/lib/dkms/mok.key` and `/var/lib/dkms/mok.pub` and use it for signing
+the kernel modules it builds and installs.
+
+You can view the certificate contents with
+
+----
+# openssl x509 -in /var/lib/dkms/mok.pub -noout -text
+----
+
+and enroll it on your system using the following command:
+
+----
+# mokutil --import /var/lib/dkms/mok.pub
+input password:
+input password again:
+----
+
+The `mokutil` command will ask for a (temporary) password twice, this password
+needs to be entered one more time in the next step of the process! Rebooting
+the system should automatically boot into the `MOKManager` EFI binary, which
+allows you to verify the key/certificate and confirm the enrollment using the
+password selected when starting the enrollment using `mokutil`. Afterwards, the
+kernel should allow loading modules built with DKMS (which are signed with the
+enrolled `MOK`). The `MOK` can also be used to sign custom EFI binaries and
+kernel images if desired.
+
+The same procedure can also be used for custom/third-party modules not managed
+with DKMS, but the key/certificate generation and signing steps need to be done
+manually in that case.