From 62de1c14a7188a76b206ba7f761ec5a83b1df196 Mon Sep 17 00:00:00 2001 From: Stoiko Ivanov Date: Wed, 10 Jul 2019 14:40:16 +0200 Subject: [PATCH] Add documentation on bootloaders (systemd-boot) With the recently added support for booting ZFS on root on EFI systems via `systemd-boot` the documentation needs adapting (mostly related to editing the kernel commandline). This patch adds a short section on Bootloaders to the sysadmin chapter describing both `grub` and PVE's use of `systemd-boot` Signed-off-by: Stoiko Ivanov --- sysadmin.adoc | 2 + system-booting.adoc | 174 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 176 insertions(+) create mode 100644 system-booting.adoc diff --git a/sysadmin.adoc b/sysadmin.adoc index 21537f1..e045610 100644 --- a/sysadmin.adoc +++ b/sysadmin.adoc @@ -74,6 +74,8 @@ include::local-zfs.adoc[] include::certificate-management.adoc[] +include::system-booting.adoc[] + endif::wiki[] diff --git a/system-booting.adoc b/system-booting.adoc new file mode 100644 index 0000000..38dacb1 --- /dev/null +++ b/system-booting.adoc @@ -0,0 +1,174 @@ +[[system_booting]] +Bootloaders +----------- +ifdef::wiki[] +:pve-toplevel: +endif::wiki[] + +{pve} uses one of two bootloaders depending on the disk setup selected in the +installer. + +For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is +used. All other deployments use the standard `grub` bootloader (this usually +also applies to systems which are installed on top of Debian). + +[[installer_partitioning_scheme]] +Partitioning scheme used by the installer +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The {pve} installer creates 3 partitions on the bootable disks selected for +installation. The bootable disks are: + +* For Installations with `ext4` or `xfs` the selected disk + +* For ZFS installations all disks belonging to the first `vdev`: +** The first disk for RAID0 +** All disks for RAID1, RAIDZ1, RAIDZ2, RAIDZ3 +** The first two disks for RAID10 + +The created partitions are: + +* a 1 MB BIOS Boot Partition (gdisk type EF02) + +* a 512 MB EFI System Partition (ESP, gdisk type EF00) + +* a third partition spanning the set `hdsize` parameter or the remaining space + used for the chosen storage type + +`grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot +Partition of all bootable disks for supporting older systems. + + +Grub +~~~~ + +`grub` has been the de-facto standard for booting Linux systems for many years +and is quite well documented +footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html]. + +The kernel and initrd images are taken from `/boot` and its configuration file +`/boot/grub/grub.cfg` gets updated by the kernel installation process. + +Configuration +^^^^^^^^^^^^^ +Changes to the `grub` configuration are done via the defaults file +`/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate +the `/boot/grub/grub.cfg` after a change to the configuration run: + +---- +`update-grub`. +---- + +Systemd-boot +~~~~~~~~~~~~ + +`systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd +images directly from the EFI Service Partition (ESP) where it is installed. +The main advantage of directly loading the kernel from the ESP is that it does +not need to reimplement the drivers for accessing the storage. In the context +of ZFS as root filesystem this means that you can use all optional features on +your root pool instead of the subset which is also present in the ZFS +implementation in `grub` or having to create a separate small boot-pool +footnote:[Booting ZFS on root with grub https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS]. + +In setups with redundancy (RAID1, RAID10, RAIDZ*) all bootable disks (those +being part of the first `vdev`) are partitioned with an ESP. This ensures the +system boots even if the first boot device fails. The ESPs are kept in sync by +a kernel postinstall hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. The +script copies certain kernel versions and the initrd images to `EFI/proxmox/` +on the root of each ESP and creates the appropriate config files in +`loader/entries/proxmox-*.conf`. + +The following kernel versions are configured by default: + +* the currently running kernel +* the version being newly installed on package updates +* the two latest kernels +* the latest version of each kernel series (e.g. 4.15, 5.0). + +The ESPs are not kept mounted during regular operation, in contrast to `grub`, +which keeps an ESP mounted on `/boot/efi`. This helps to prevent filesystem +corruption to the `vfat` formatted ESPs in case of a system crash, and removes +the need to manually adapt `/etc/fstab` in case the primary boot device fails. + +[[systemd_boot_config]] +Configuration +^^^^^^^^^^^^^ + +`systemd-boot` is configured via the file `loader/loader.conf` in the root +directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage +for details. + +Each bootloader entry is placed in a file of its own in the directory +`loader/entries/` + +An example entry.conf looks like this (`/` refers to the root of the ESP): + +---- +title Proxmox +version 5.0.15-1-pve +options root=ZFS=rpool/ROOT/pve-1 boot=zfs +linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve +initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve +---- + + +.Manually keeping a kernel bootable + +Should you wish to add a certain kernel and initrd image to the list of +bootable kernels you need to: + +* create a directory on the ESP (e.g. `/EFI/personalkernel`) +* copy the kernel and initrd image to that directory +* create a entry for this kernel in `/loader/entries/*.conf` + +NOTE: do not use `/EFI/proxmox` as directory since all entries there can be +removed by `/etc/kernel/postinst.d/zz-pve-efiboot` + +Example (keeping kernel 5.0.15-1-pve and copying to an ESP mounted on +`/mnt/esp`): + +---- +mkdir /mnt/esp/EFI/preferred-kernel +cp /boot/initrd.img-5.0.15-1-pve /boot/vmlinuz-5.0.15-1-pve /mnt/esp/EFI/preferred-kernel +echo -n "title Preferred Kernel +version 5.0.15-1-pve +linux /mnt/esp/EFI/preferred-kernel/vmlinuz-5.0.15-1-pve +initrd /mnt/esp/EFI/preferred-kernel/initrd.img-5.0.15-1-pve +options " > /mnt/esp/loader/entries/preferred.conf +cat /etc/kernel/cmdline >> /mnt/esp/loader/entries/preferred.conf +---- + + +[[systemd-boot-refresh]] +.Updating the configuration on all ESPs + +To copy and configure all bootable kernels and keep all ESPs in sync you just +need to run the kernel hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. +(The equivalent to running `update-grub` on Systems being booted with `grub`). + +This is necessary should you make changes to the kernel commandline, or if you +want to add another ESP (e.g. when replacing a failed disk in a redundant ZFS +pool). + +[[edit_kernel_cmdline]] +Editing the kernel commandline +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can modify the kernel commandline in the following places, depending on the +bootloarder used: + +.Grub + +The kernel commandline needs to be placed in the variable +`GRUB_CMDLINE_LINUX_DEFAULT` in the file `/etc/default/grub`. Running +`update-grub` appends its content to all `linux` entries in +`/boot/grub/grub.cfg`. + +.Systemd-boot + +The kernel commandline needs to be placed as line in `/etc/kernel/cmdline` +Running `/etc/kernel/postinst.d/zz-pve-efiboot` sets it as `option` line for +all config files in `loader/entries/proxmox-*.conf`. + + -- 2.39.2