[[sysboot]] Host Bootloader --------------- ifdef::wiki[] :pve-toplevel: endif::wiki[] {pve} currently uses one of two bootloaders depending on the disk setup selected in the installer. For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is used. All other deployments use the standard `grub` bootloader (this usually also applies to systems which are installed on top of Debian). [[sysboot_installer_part_scheme]] Partitioning scheme used by the installer ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The {pve} installer creates 3 partitions on the bootable disks selected for installation. The bootable disks are: * For Installations with `ext4` or `xfs` the selected disk * For ZFS installations all disks belonging to the first `vdev`: ** The first disk for RAID0 ** All disks for RAID1, RAIDZ1, RAIDZ2, RAIDZ3 ** The first two disks for RAID10 The created partitions are: * a 1 MB BIOS Boot Partition (gdisk type EF02) * a 512 MB EFI System Partition (ESP, gdisk type EF00) * a third partition spanning the set `hdsize` parameter or the remaining space used for the chosen storage type `grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot Partition of all bootable disks for supporting older systems. [[sysboot_grub]] Grub ~~~~ `grub` has been the de-facto standard for booting Linux systems for many years and is quite well documented footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html]. The kernel and initrd images are taken from `/boot` and its configuration file `/boot/grub/grub.cfg` gets updated by the kernel installation process. Configuration ^^^^^^^^^^^^^ Changes to the `grub` configuration are done via the defaults file `/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate the `/boot/grub/grub.cfg` after a change to the configuration run: ---- `update-grub`. ---- [[sysboot_systemd_boot]] Systemd-boot ~~~~~~~~~~~~ `systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd images directly from the EFI Service Partition (ESP) where it is installed. The main advantage of directly loading the kernel from the ESP is that it does not need to reimplement the drivers for accessing the storage. In the context of ZFS as root filesystem this means that you can use all optional features on your root pool instead of the subset which is also present in the ZFS implementation in `grub` or having to create a separate small boot-pool footnote:[Booting ZFS on root with grub https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS]. In setups with redundancy (RAID1, RAID10, RAIDZ*) all bootable disks (those being part of the first `vdev`) are partitioned with an ESP. This ensures the system boots even if the first boot device fails. The ESPs are kept in sync by a kernel postinstall hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. The script copies certain kernel versions and the initrd images to `EFI/proxmox/` on the root of each ESP and creates the appropriate config files in `loader/entries/proxmox-*.conf`. The following kernel versions are configured by default: * the currently running kernel * the version being newly installed on package updates * the two latest kernels * the latest version of each kernel series (e.g. 4.15, 5.0). The ESPs are not kept mounted during regular operation, in contrast to `grub`, which keeps an ESP mounted on `/boot/efi`. This helps to prevent filesystem corruption to the `vfat` formatted ESPs in case of a system crash, and removes the need to manually adapt `/etc/fstab` in case the primary boot device fails. [[sysboot_systemd_boot_config]] Configuration ^^^^^^^^^^^^^ `systemd-boot` is configured via the file `loader/loader.conf` in the root directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage for details. Each bootloader entry is placed in a file of its own in the directory `loader/entries/` An example entry.conf looks like this (`/` refers to the root of the ESP): ---- title Proxmox version 5.0.15-1-pve options root=ZFS=rpool/ROOT/pve-1 boot=zfs linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve ---- .Manually keeping a kernel bootable Should you wish to add a certain kernel and initrd image to the list of bootable kernels you need to: * create a directory on the ESP (e.g. `/EFI/personalkernel`) * copy the kernel and initrd image to that directory * create a entry for this kernel in `/loader/entries/*.conf` NOTE: do not use `/EFI/proxmox` as directory since all entries there can be removed by `/etc/kernel/postinst.d/zz-pve-efiboot` Example (keeping kernel 5.0.15-1-pve and copying to an ESP mounted on `/mnt/esp`): ---- mkdir /mnt/esp/EFI/preferred-kernel cp /boot/initrd.img-5.0.15-1-pve /boot/vmlinuz-5.0.15-1-pve /mnt/esp/EFI/preferred-kernel echo -n "title Preferred Kernel version 5.0.15-1-pve linux /mnt/esp/EFI/preferred-kernel/vmlinuz-5.0.15-1-pve initrd /mnt/esp/EFI/preferred-kernel/initrd.img-5.0.15-1-pve options " > /mnt/esp/loader/entries/preferred.conf cat /etc/kernel/cmdline >> /mnt/esp/loader/entries/preferred.conf ---- [[sysboot_systemd_boot_refresh]] .Updating the configuration on all ESPs To copy and configure all bootable kernels and keep all ESPs in sync you just need to run the kernel hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. (The equivalent to running `update-grub` on Systems being booted with `grub`). This is necessary should you make changes to the kernel commandline, or if you want to add another ESP (e.g. when replacing a failed disk in a redundant ZFS pool). [[sysboot_edit_kernel_cmdline]] Editing the kernel commandline ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can modify the kernel commandline in the following places, depending on the bootloarder used: .Grub The kernel commandline needs to be placed in the variable `GRUB_CMDLINE_LINUX_DEFAULT` in the file `/etc/default/grub`. Running `update-grub` appends its content to all `linux` entries in `/boot/grub/grub.cfg`. .Systemd-boot The kernel commandline needs to be placed as line in `/etc/kernel/cmdline` Running `/etc/kernel/postinst.d/zz-pve-efiboot` sets it as `option` line for all config files in `loader/entries/proxmox-*.conf`.