system-booting.adoc

   1 [[sysboot]]
   2 Host Bootloader
   3 ---------------
   4 ifdef::wiki[]
   5 :pve-toplevel:
   6 endif::wiki[]
   7
   8 {pve} currently uses one of two bootloaders depending on the disk setup
   9 selected in the installer.
  10
  11 For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is
  12 used. All other deployments use the standard `grub` bootloader (this usually
  13 also applies to systems which are installed on top of Debian).
  14
  15 [[sysboot_installer_part_scheme]]
  16 Partitioning scheme used by the installer
  17 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  18
  19 The {pve} installer creates 3 partitions on the bootable disks selected for
  20 installation. The bootable disks are:
  21
  22 * For Installations with `ext4` or `xfs` the selected disk
  23
  24 * For ZFS installations all disks belonging to the first `vdev`:
  25 ** The first disk for RAID0
  26 ** All disks for RAID1, RAIDZ1, RAIDZ2, RAIDZ3
  27 ** The first two disks for RAID10
  28
  29 The created partitions are:
  30
  31 * a 1 MB BIOS Boot Partition (gdisk type EF02)
  32
  33 * a 512 MB EFI System Partition (ESP, gdisk type EF00)
  34
  35 * a third partition spanning the set `hdsize` parameter or the remaining space
  36     used for the chosen storage type
  37
  38 `grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
  39 Partition of all bootable disks for supporting older systems.
  40
  41
  42 [[sysboot_grub]]
  43 Grub
  44 ~~~~
  45
  46 `grub` has been the de-facto standard for booting Linux systems for many years
  47 and is quite well documented
  48 footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
  49
  50 The kernel and initrd images are taken from `/boot` and its configuration file
  51 `/boot/grub/grub.cfg` gets updated by the kernel installation process.
  52
  53 Configuration
  54 ^^^^^^^^^^^^^
  55 Changes to the `grub` configuration are done via the defaults file
  56 `/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate
  57 the `/boot/grub/grub.cfg` after a change to the configuration run:
  58
  59 ----
  60 `update-grub`.
  61 ----
  62
  63 [[sysboot_systemd_boot]]
  64 Systemd-boot
  65 ~~~~~~~~~~~~
  66
  67 `systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd
  68 images directly from the EFI Service Partition (ESP) where it is installed.
  69 The main advantage of directly loading the kernel from the ESP is that it does
  70 not need to reimplement the drivers for accessing the storage.  In the context
  71 of ZFS as root filesystem this means that you can use all optional features on
  72 your root pool instead of the subset which is also present in the ZFS
  73 implementation in `grub` or having to create a separate small boot-pool
  74 footnote:[Booting ZFS on root with grub https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS].
  75
  76 In setups with redundancy (RAID1, RAID10, RAIDZ*) all bootable disks (those
  77 being part of the first `vdev`) are partitioned with an ESP. This ensures the
  78 system boots even if the first boot device fails.  The ESPs are kept in sync by
  79 a kernel postinstall hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. The
  80 script copies certain kernel versions and the initrd images to `EFI/proxmox/`
  81 on the root of each ESP and creates the appropriate config files in
  82 `loader/entries/proxmox-*.conf`.
  83
  84 The following kernel versions are configured by default:
  85
  86 * the currently running kernel
  87 * the version being newly installed on package updates
  88 * the two latest kernels
  89 * the latest version of each kernel series (e.g. 4.15, 5.0).
  90
  91 The ESPs are not kept mounted during regular operation, in contrast to `grub`,
  92 which keeps an ESP mounted on `/boot/efi`. This helps to prevent filesystem
  93 corruption to the `vfat` formatted ESPs in case of a system crash, and removes
  94 the need to manually adapt `/etc/fstab` in case the primary boot device fails.
  95
  96 [[sysboot_systemd_boot_config]]
  97 Configuration
  98 ^^^^^^^^^^^^^
  99
 100 `systemd-boot` is configured via the file `loader/loader.conf` in the root
 101 directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage
 102 for details.
 103
 104 Each bootloader entry is placed in a file of its own in the directory
 105 `loader/entries/`
 106
 107 An example entry.conf looks like this (`/` refers to the root of the ESP):
 108
 109 ----
 110 title    Proxmox
 111 version  5.0.15-1-pve
 112 options   root=ZFS=rpool/ROOT/pve-1 boot=zfs
 113 linux    /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve
 114 initrd   /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve
 115 ----
 116
 117
 118 .Manually keeping a kernel bootable
 119
 120 Should you wish to add a certain kernel and initrd image to the list of
 121 bootable kernel use `pve-efiboot-tool kernel add`.
 122
 123 For example run the following to add the kernel with ABI version `5.0.15-1-pve`
 124 to the list of kernels to keep installed and synced to all ESPs:
 125
 126 ----
 127 pve-efiboot-tool kernel add 5.0.15-1-pve
 128 ----
 129
 130 `pve-efiboot-tool kernel list` will list all kernel versions currently selected
 131 for booting:
 132
 133 ----
 134 # pve-efiboot-tool kernel list
 135 Manually selected kernels:
 136 5.0.15-1-pve
 137
 138 Automatically selected kernels:
 139 5.0.12-1-pve
 140 4.15.18-18-pve
 141 ----
 142
 143 Run `pve-efiboot-tool remove` to remove a kernel from the list of manually
 144 selected kernels, for example:
 145
 146 ----
 147 pve-efiboot-tool kernel remove 5.0.15-1-pve
 148 ----
 149
 150
 151 [[sysboot_systemd_boot_setup]]
 152 .Setting up a new partition for use as synced ESP
 153
 154 To format and initialize a partition as synced ESP, e.g., after replacing a
 155 failed vdev in an rpool, or when converting an existing system that pre-dates
 156 the sync mechanism, `pve-efiboot-tool` from `pve-kernel-helpers` can be used.
 157
 158 WARNING: the `format` command will format the `<partition>`, make sure to pass
 159 in the right device/partition!
 160
 161 For example, to format an empty partition `/dev/sda2` as ESP, run the following:
 162
 163 ----
 164 pve-efiboot-tool format /dev/sda2
 165 ----
 166
 167 To setup an existing, unmounted ESP located on `/dev/sda2` for inclusion in
 168 {pve}'s kernel update synchronization mechanism, use the following:
 169
 170 ----
 171 pve-efiboot-tool init /dev/sda2
 172 ----
 173
 174 Afterwards `/etc/kernel/pve-efiboot-uuids` should contain a new line with the
 175 UUID of the newly added partition. The `init` command will also automatically
 176 trigger a refresh of all configured ESPs.
 177
 178 [[sysboot_systemd_boot_refresh]]
 179 .Updating the configuration on all ESPs
 180
 181 To copy and configure all bootable kernels and keep all ESPs listed in
 182 `/etc/kernel/pve-efiboot-uuids` in sync you just need to run `pve-efiboot-tool
 183 refresh`.
 184 (The equivalent to running `update-grub` on Systems being booted with `grub`).
 185
 186 This is necessary should you make changes to the kernel commandline, or want to
 187 sync all kernels and initrds after regenerating the latter.
 188
 189 [[sysboot_edit_kernel_cmdline]]
 190 Editing the kernel commandline
 191 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 192
 193 You can modify the kernel commandline in the following places, depending on the
 194 bootloarder used:
 195
 196 .Grub
 197
 198 The kernel commandline needs to be placed in the variable
 199 `GRUB_CMDLINE_LINUX_DEFAULT` in the file `/etc/default/grub`. Running
 200 `update-grub` appends its content to all `linux` entries in
 201 `/boot/grub/grub.cfg`.
 202
 203 .Systemd-boot
 204
 205 The kernel commandline needs to be placed as line in `/etc/kernel/cmdline`
 206 Running `/etc/kernel/postinst.d/zz-pve-efiboot` sets it as `option` line for
 207 all config files in `loader/entries/proxmox-*.conf`.