used. All other deployments use the standard `grub` bootloader (this usually
also applies to systems which are installed on top of Debian).
+
[[sysboot_installer_part_scheme]]
-Partitioning scheme used by the installer
+Partitioning Scheme Used by the Installer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The {pve} installer creates 3 partitions on the bootable disks selected for
-installation. The bootable disks are:
-
-* For Installations with `ext4` or `xfs` the selected disk
-
-* For ZFS installations all disks belonging to the first `vdev`:
-** The first disk for RAID0
-** All disks for RAID1, RAIDZ1, RAIDZ2, RAIDZ3
-** The first two disks for RAID10
+The {pve} installer creates 3 partitions on all disks selected for
+installation.
The created partitions are:
* a third partition spanning the set `hdsize` parameter or the remaining space
used for the chosen storage type
+Systems using ZFS as root filesystem are booted with a kernel and initrd image
+stored on the 512 MB EFI System Partition. For legacy BIOS systems, `grub` is
+used, for EFI systems `systemd-boot` is used. Both are installed and configured
+to point to the ESPs.
+
`grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot
-Partition of all bootable disks for supporting older systems.
+Partition of all selected disks on all systems booted with `grub`
+footnote:[These are all installs with root on `ext4` or `xfs` and installs
+with root on ZFS on non-EFI systems].
-[[sysboot_grub]]
-Grub
-~~~~
+[[sysboot_proxmox_boot_tool]]
+Synchronizing the content of the ESP with `proxmox-boot-tool`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-`grub` has been the de-facto standard for booting Linux systems for many years
-and is quite well documented
-footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
+`proxmox-boot-tool` is a utility used to keep the contents of the EFI System
+Partitions properly configured and synchronized. It copies certain kernel
+versions to all ESPs and configures the respective bootloader to boot from
+the `vfat` formatted ESPs. In the context of ZFS as root filesystem this means
+that you can use all optional features on your root pool instead of the subset
+which is also present in the ZFS implementation in `grub` or having to create a
+separate small boot-pool footnote:[Booting ZFS on root with grub
+https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS].
-The kernel and initrd images are taken from `/boot` and its configuration file
-`/boot/grub/grub.cfg` gets updated by the kernel installation process.
+In setups with redundancy all disks are partitioned with an ESP, by the
+installer. This ensures the system boots even if the first boot device fails
+or if the BIOS can only boot from a particular disk.
-Configuration
-^^^^^^^^^^^^^
-Changes to the `grub` configuration are done via the defaults file
-`/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate
-the `/boot/grub/grub.cfg` after a change to the configuration run:
+The ESPs are not kept mounted during regular operation. This helps to prevent
+filesystem corruption to the `vfat` formatted ESPs in case of a system crash,
+and removes the need to manually adapt `/etc/fstab` in case the primary boot
+device fails.
+
+`proxmox-boot-tool` handles the following tasks:
+
+* formatting and setting up a new partition
+* copying and configuring new kernel images and initrd images to all listed ESPs
+* synchronizing the configuration on kernel upgrades and other maintenance tasks
+* managing the list of kernel versions which are synchronized
+
+
+You can view the currently configured ESPs and their state by running:
----
-`update-grub`.
+# proxmox-boot-tool status
----
-[[sysboot_systemd_boot]]
-Systemd-boot
-~~~~~~~~~~~~
+[[sysboot_proxmox_boot_setup]]
+.Setting up a new partition for use as synced ESP
-`systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd
-images directly from the EFI Service Partition (ESP) where it is installed.
-The main advantage of directly loading the kernel from the ESP is that it does
-not need to reimplement the drivers for accessing the storage. In the context
-of ZFS as root filesystem this means that you can use all optional features on
-your root pool instead of the subset which is also present in the ZFS
-implementation in `grub` or having to create a separate small boot-pool
-footnote:[Booting ZFS on root with grub https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS].
-
-In setups with redundancy (RAID1, RAID10, RAIDZ*) all bootable disks (those
-being part of the first `vdev`) are partitioned with an ESP. This ensures the
-system boots even if the first boot device fails. The ESPs are kept in sync by
-a kernel postinstall hook script `/etc/kernel/postinst.d/zz-pve-efiboot`. The
-script copies certain kernel versions and the initrd images to `EFI/proxmox/`
-on the root of each ESP and creates the appropriate config files in
-`loader/entries/proxmox-*.conf`.
+To format and initialize a partition as synced ESP, e.g., after replacing a
+failed vdev in an rpool, or when converting an existing system that pre-dates
+the sync mechanism, `proxmox-boot-tool` from `pve-kernel-helpers` can be used.
+WARNING: the `format` command will format the `<partition>`, make sure to pass
+in the right device/partition!
+
+For example, to format an empty partition `/dev/sda2` as ESP, run the following:
+
+----
+# proxmox-boot-tool format /dev/sda2
+----
+
+To setup an existing, unmounted ESP located on `/dev/sda2` for inclusion in
+{pve}'s kernel update synchronization mechanism, use the following:
+
+----
+# proxmox-boot-tool init /dev/sda2
+----
+
+Afterwards `/etc/kernel/proxmox-boot-uuids` should contain a new line with the
+UUID of the newly added partition. The `init` command will also automatically
+trigger a refresh of all configured ESPs.
+
+[[sysboot_proxmox_boot_refresh]]
+.Updating the configuration on all ESPs
+
+To copy and configure all bootable kernels and keep all ESPs listed in
+`/etc/kernel/proxmox-boot-uuids` in sync you just need to run:
+
+----
+# proxmox-boot-tool refresh
+----
+(The equivalent to running `update-grub` systems with `ext4` or `xfs` on root).
+
+This is necessary should you make changes to the kernel commandline, or want to
+sync all kernels and initrds.
+
+NOTE: Both `update-initramfs` and `apt` (when necessary) will automatically
+trigger a refresh.
+
+.Kernel Versions considered by `proxmox-boot-tool`
The following kernel versions are configured by default:
* the currently running kernel
* the version being newly installed on package updates
-* the two latest kernels
-* the latest version of each kernel series (e.g. 4.15, 5.0).
+* the two latest already installed kernels
+* the latest version of the second-to-last kernel series (e.g. 5.0, 5.3), if applicable
+* any manually selected kernels
-The ESPs are not kept mounted during regular operation, in contrast to `grub`,
-which keeps an ESP mounted on `/boot/efi`. This helps to prevent filesystem
-corruption to the `vfat` formatted ESPs in case of a system crash, and removes
-the need to manually adapt `/etc/fstab` in case the primary boot device fails.
+.Manually keeping a kernel bootable
-[[sysboot_systemd_boot_config]]
-Configuration
-^^^^^^^^^^^^^
+Should you wish to add a certain kernel and initrd image to the list of
+bootable kernels use `proxmox-boot-tool kernel add`.
-`systemd-boot` is configured via the file `loader/loader.conf` in the root
-directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage
-for details.
+For example run the following to add the kernel with ABI version `5.0.15-1-pve`
+to the list of kernels to keep installed and synced to all ESPs:
-Each bootloader entry is placed in a file of its own in the directory
-`loader/entries/`
+----
+# proxmox-boot-tool kernel add 5.0.15-1-pve
+----
-An example entry.conf looks like this (`/` refers to the root of the ESP):
+`proxmox-boot-tool kernel list` will list all kernel versions currently selected
+for booting:
----
-title Proxmox
-version 5.0.15-1-pve
-options root=ZFS=rpool/ROOT/pve-1 boot=zfs
-linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve
-initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve
+# proxmox-boot-tool kernel list
+Manually selected kernels:
+5.0.15-1-pve
+
+Automatically selected kernels:
+5.0.12-1-pve
+4.15.18-18-pve
----
+Run `proxmox-boot-tool kernel remove` to remove a kernel from the list of
+manually selected kernels, for example:
-.Manually keeping a kernel bootable
+----
+# proxmox-boot-tool kernel remove 5.0.15-1-pve
+----
-Should you wish to add a certain kernel and initrd image to the list of
-bootable kernels you need to:
+NOTE: It's required to run `proxmox-boot-tool refresh` to update all EFI System
+Partitions (ESPs) after a manual kernel addition or removal from above.
+
+
+[[sysboot_determine_bootloader_used]]
+Determine which Bootloader is Used
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+[thumbnail="screenshot/boot-grub.png", float="left"]
-* create a directory on the ESP (e.g. `/EFI/personalkernel`)
-* copy the kernel and initrd image to that directory
-* create a entry for this kernel in `/loader/entries/*.conf`
+The simplest and most reliable way to determine which bootloader is used, is to
+watch the boot process of the {pve} node.
-NOTE: do not use `/EFI/proxmox` as directory since all entries there can be
-removed by `/etc/kernel/postinst.d/zz-pve-efiboot`
+You will either see the blue box of `grub` or the simple black on white
+`systemd-boot`.
+
+[thumbnail="screenshot/boot-systemdboot.png"]
+
+Determining the bootloader from a running system might not be 100% accurate. The
+safest way is to run the following command:
-Example (keeping kernel 5.0.15-1-pve and copying to an ESP mounted on
-`/mnt/esp`):
----
-mkdir /mnt/esp/EFI/preferred-kernel
-cp /boot/initrd.img-5.0.15-1-pve /boot/vmlinuz-5.0.15-1-pve /mnt/esp/EFI/preferred-kernel
-echo -n "title Preferred Kernel
-version 5.0.15-1-pve
-linux /mnt/esp/EFI/preferred-kernel/vmlinuz-5.0.15-1-pve
-initrd /mnt/esp/EFI/preferred-kernel/initrd.img-5.0.15-1-pve
-options " > /mnt/esp/loader/entries/preferred.conf
-cat /etc/kernel/cmdline >> /mnt/esp/loader/entries/preferred.conf
+# efibootmgr -v
----
-[[sysboot_systemd_boot_setup]]
-.Setting up a new partition for use as synced ESP
+If it returns a message that EFI variables are not supported, `grub` is used in
+BIOS/Legacy mode.
-To format and initialize a partition as synced ESP, e.g., after replacing a
-failed vdev in an rpool, or when converting an existing system that pre-dates
-the sync mechanism, `pve-efiboot-tool` from `pve-kernel-helpers` can be used.
+If the output contains a line that looks similar to the following, `grub` is
+used in UEFI mode.
-WARNING: the `format` command will format the `<partition>`, make sure to pass
-in the right device/partition!
+----
+Boot0005* proxmox [...] File(\EFI\proxmox\grubx64.efi)
+----
-For example, to format an empty partition `/dev/sda2` as ESP, run the following:
+If the output contains a line similar to the following, `systemd-boot` is used.
----
-pve-efiboot-tool format /dev/sda2
+Boot0006* Linux Boot Manager [...] File(\EFI\systemd\systemd-bootx64.efi)
----
-To setup an existing, unmounted ESP located on `/dev/sda2` for inclusion in
-{pve}'s kernel update synchronization mechanism, use the following:
+By running:
----
-pve-efiboot-tool init /dev/sda2
+# proxmox-boot-tool status
----
-Afterwards `/etc/kernel/pve-efiboot-uuids` should contain a new line with the
-UUID of the newly added partition. The `init` command will also automatically
-trigger a refresh of all configured ESPs.
+you can find out if `proxmox-boot-tool` is configured, which is a good
+indication of how the system is booted.
-[[sysboot_systemd_boot_refresh]]
-.Updating the configuration on all ESPs
-To copy and configure all bootable kernels and keep all ESPs listed in
-`/etc/kernel/pve-efiboot-uuids` in sync you just need to run `pve-efiboot-tool
-refresh`.
-(The equivalent to running `update-grub` on Systems being booted with `grub`).
+[[sysboot_grub]]
+Grub
+~~~~
+
+`grub` has been the de-facto standard for booting Linux systems for many years
+and is quite well documented
+footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html].
+
+Configuration
+^^^^^^^^^^^^^
+Changes to the `grub` configuration are done via the defaults file
+`/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate
+the configuration file after a change to the configuration run:
+footnote:[Systems using `proxmox-boot-tool` will call `proxmox-boot-tool
+refresh` upon `update-grub`.]
+
+----
+# update-grub
+----
+
+
+[[sysboot_systemd_boot]]
+Systemd-boot
+~~~~~~~~~~~~
+
+`systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd
+images directly from the EFI Service Partition (ESP) where it is installed.
+The main advantage of directly loading the kernel from the ESP is that it does
+not need to reimplement the drivers for accessing the storage. In {pve}
+xref:sysboot_proxmox_boot_tool[`proxmox-boot-tool`] is used to keep the
+configuration on the ESPs synchronized.
+
+[[sysboot_systemd_boot_config]]
+Configuration
+^^^^^^^^^^^^^
+
+`systemd-boot` is configured via the file `loader/loader.conf` in the root
+directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage
+for details.
+
+Each bootloader entry is placed in a file of its own in the directory
+`loader/entries/`
+
+An example entry.conf looks like this (`/` refers to the root of the ESP):
+
+----
+title Proxmox
+version 5.0.15-1-pve
+options root=ZFS=rpool/ROOT/pve-1 boot=zfs
+linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve
+initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve
+----
-This is necessary should you make changes to the kernel commandline, or want to
-sync all kernels and initrds after regenerating the latter.
[[sysboot_edit_kernel_cmdline]]
-Editing the kernel commandline
+Editing the Kernel Commandline
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can modify the kernel commandline in the following places, depending on the
-bootloarder used:
+bootloader used:
.Grub
.Systemd-boot
-The kernel commandline needs to be placed as line in `/etc/kernel/cmdline`
-Running `/etc/kernel/postinst.d/zz-pve-efiboot` sets it as `option` line for
-all config files in `loader/entries/proxmox-*.conf`.
+The kernel commandline needs to be placed as one line in `/etc/kernel/cmdline`.
+To apply your changes, run `proxmox-boot-tool refresh`, which sets it as the
+`option` line for all config files in `loader/entries/proxmox-*.conf`.
projects / pve-docs.git / blobdiff