system-booting.adoc

   1 [[system_booting]]
   2 Bootloaders
   3 -----------
   4 ifdef::wiki[]
   5 :pve-toplevel:
   6 endif::wiki[]
   7
   8 {pve} uses one of two bootloaders depending on the disk setup selected in the
   9 installer.
  10
  11 For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is
  12 used. All other deployments use the standard `grub` bootloader (this usually
  13 also applies to systems which are installed on top of Debian).
  14
  15 [[installer_partitioning_scheme]]
  16 Partitioning scheme used by the installer
  17 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  18
  19 The {pve} installer creates 3 partitions on the bootable disks selected for
  20 installation. The bootable disks are:
  21
  22 * For Installations with `ext4` or `xfs` the selected disk
  23
  24 * For ZFS installations all disks belonging to the first `vdev`:
  25 ** The first disk for RAID0
  26 ** All disks for RAID1, RAIDZ1, RAIDZ2, RAIDZ3
  27 ** The first two disks for RAID10
  28
  29 The created partitions are:
  30
  31 * a 1 MB BIOS Boot Partition (gdisk type EF02)
  32
  33 * a 512 MB EFI System Partition (ESP, gdisk type EF00)
  34
  35 * a third partition spanning the set `hdsize` parameter or the remaining space
  36     used for the chosen storage type
  37
  38 `grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
  39 Partition of all bootable disks for supporting older systems.
  40
  41
  42 Grub
  43 ~~~~
  44
  45 `grub` has been the de-facto standard for booting Linux systems for many years
  46 and is quite well documented
  47 footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
  48
  49 The kernel and initrd images are taken from `/boot` and its configuration file
  50 `/boot/grub/grub.cfg` gets updated by the kernel installation process.
  51
  52 Configuration
  53 ^^^^^^^^^^^^^
  54 Changes to the `grub` configuration are done via the defaults file
  55 `/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate
  56 the `/boot/grub/grub.cfg` after a change to the configuration run:
  57
  58 ----
  59 `update-grub`.
  60 ----
  61
  62 Systemd-boot
  63 ~~~~~~~~~~~~
  64
  65 `systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd
  66 images directly from the EFI Service Partition (ESP) where it is installed.
  67 The main advantage of directly loading the kernel from the ESP is that it does
  68 not need to reimplement the drivers for accessing the storage.  In the context
  69 of ZFS as root filesystem this means that you can use all optional features on
  70 your root pool instead of the subset which is also present in the ZFS
  71 implementation in `grub` or having to create a separate small boot-pool
  72 footnote:[Booting ZFS on root with grub https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS].
  73
  74 In setups with redundancy (RAID1, RAID10, RAIDZ*) all bootable disks (those
  75 being part of the first `vdev`) are partitioned with an ESP. This ensures the
  76 system boots even if the first boot device fails.  The ESPs are kept in sync by
  77 a kernel postinstall hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. The
  78 script copies certain kernel versions and the initrd images to `EFI/proxmox/`
  79 on the root of each ESP and creates the appropriate config files in
  80 `loader/entries/proxmox-*.conf`.
  81
  82 The following kernel versions are configured by default:
  83
  84 * the currently running kernel
  85 * the version being newly installed on package updates
  86 * the two latest kernels
  87 * the latest version of each kernel series (e.g. 4.15, 5.0).
  88
  89 The ESPs are not kept mounted during regular operation, in contrast to `grub`,
  90 which keeps an ESP mounted on `/boot/efi`. This helps to prevent filesystem
  91 corruption to the `vfat` formatted ESPs in case of a system crash, and removes
  92 the need to manually adapt `/etc/fstab` in case the primary boot device fails.
  93
  94 [[systemd_boot_config]]
  95 Configuration
  96 ^^^^^^^^^^^^^
  97
  98 `systemd-boot` is configured via the file `loader/loader.conf` in the root
  99 directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage
 100 for details.
 101
 102 Each bootloader entry is placed in a file of its own in the directory
 103 `loader/entries/`
 104
 105 An example entry.conf looks like this (`/` refers to the root of the ESP):
 106
 107 ----
 108 title    Proxmox
 109 version  5.0.15-1-pve
 110 options   root=ZFS=rpool/ROOT/pve-1 boot=zfs
 111 linux    /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve
 112 initrd   /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve
 113 ----
 114
 115
 116 .Manually keeping a kernel bootable
 117
 118 Should you wish to add a certain kernel and initrd image to the list of
 119 bootable kernels you need to:
 120
 121 * create a directory on the ESP (e.g. `/EFI/personalkernel`)
 122 * copy the kernel and initrd image to that directory
 123 * create a entry for this kernel in `/loader/entries/*.conf`
 124
 125 NOTE: do not use `/EFI/proxmox` as directory since all entries there can be
 126 removed by `/etc/kernel/postinst.d/zz-pve-efiboot`
 127
 128 Example (keeping kernel 5.0.15-1-pve and copying to an ESP mounted on
 129 `/mnt/esp`):
 130
 131 ----
 132 mkdir /mnt/esp/EFI/preferred-kernel
 133 cp /boot/initrd.img-5.0.15-1-pve /boot/vmlinuz-5.0.15-1-pve /mnt/esp/EFI/preferred-kernel
 134 echo -n "title Preferred Kernel
 135 version 5.0.15-1-pve
 136 linux /mnt/esp/EFI/preferred-kernel/vmlinuz-5.0.15-1-pve
 137 initrd /mnt/esp/EFI/preferred-kernel/initrd.img-5.0.15-1-pve
 138 options " > /mnt/esp/loader/entries/preferred.conf
 139 cat /etc/kernel/cmdline >> /mnt/esp/loader/entries/preferred.conf
 140 ----
 141
 142
 143 [[systemd-boot-refresh]]
 144 .Updating the configuration on all ESPs
 145
 146 To copy and configure all bootable kernels and keep all ESPs in sync you just
 147 need to run the kernel hook script `/etc/kernel/postinst.d/zz-pve-efiboot`.
 148 (The equivalent to running `update-grub` on Systems being booted with `grub`).
 149
 150 This is necessary should you make changes to the kernel commandline, or if you
 151 want to add another ESP (e.g. when replacing a failed disk in a redundant ZFS
 152 pool).
 153
 154 [[edit_kernel_cmdline]]
 155 Editing the kernel commandline
 156 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 157
 158 You can modify the kernel commandline in the following places, depending on the
 159 bootloarder used:
 160
 161 .Grub
 162
 163 The kernel commandline needs to be placed in the variable
 164 `GRUB_CMDLINE_LINUX_DEFAULT` in the file `/etc/default/grub`. Running
 165 `update-grub` appends its content to all `linux` entries in
 166 `/boot/grub/grub.cfg`.
 167
 168 .Systemd-boot
 169
 170 The kernel commandline needs to be placed as line in `/etc/kernel/cmdline`
 171 Running `/etc/kernel/postinst.d/zz-pve-efiboot` sets it as `option` line for
 172 all config files in `loader/entries/proxmox-*.conf`.
 173
 174