]>
Commit | Line | Data |
---|---|---|
9eb80400 SI |
1 | |
2 | .. _chapter-systembooting: | |
3 | ||
4 | Host Bootloader | |
5 | --------------- | |
6 | ||
6481fd24 | 7 | `Proxmox Backup`_ currently uses one of two bootloaders, depending on the disk setup |
9eb80400 SI |
8 | selected in the installer. |
9 | ||
10 | For EFI Systems installed with ZFS as the root filesystem ``systemd-boot`` is | |
11 | used. All other deployments use the standard ``grub`` bootloader (this usually | |
12 | also applies to systems which are installed on top of Debian). | |
13 | ||
14 | ||
15 | .. _systembooting-installer-part-scheme: | |
16 | ||
17 | Partitioning Scheme Used by the Installer | |
18 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
19 | ||
20 | The `Proxmox Backup`_ installer creates 3 partitions on all disks selected for | |
21 | installation. | |
22 | ||
23 | The created partitions are: | |
24 | ||
6481fd24 | 25 | * A 1 MB BIOS Boot Partition (gdisk type EF02) |
9eb80400 | 26 | |
6481fd24 | 27 | * A 512 MB EFI System Partition (ESP, gdisk type EF00) |
9eb80400 | 28 | |
6481fd24 DW |
29 | * A third partition spanning the configured ``hdsize`` parameter or the |
30 | remaining space available for the chosen storage type | |
9eb80400 | 31 | |
6481fd24 | 32 | Systems using ZFS as a root filesystem are booted with a kernel and initrd image |
9eb80400 SI |
33 | stored on the 512 MB EFI System Partition. For legacy BIOS systems, ``grub`` is |
34 | used, for EFI systems ``systemd-boot`` is used. Both are installed and configured | |
35 | to point to the ESPs. | |
36 | ||
37 | ``grub`` in BIOS mode (``--target i386-pc``) is installed onto the BIOS Boot | |
6481fd24 DW |
38 | Partition of all selected disks on all systems booted with ``grub`` (that is, |
39 | all installs with root on ``ext4`` or ``xfs``, and installs with root on ZFS on | |
9eb80400 SI |
40 | non-EFI systems). |
41 | ||
42 | ||
43 | .. _systembooting-proxmox-boot-tool: | |
44 | ||
6481fd24 | 45 | Synchronizing the Content of the ESP with ``proxmox-boot-tool`` |
9eb80400 SI |
46 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
47 | ||
48 | ``proxmox-boot-tool`` is a utility used to keep the contents of the EFI System | |
49 | Partitions properly configured and synchronized. It copies certain kernel | |
50 | versions to all ESPs and configures the respective bootloader to boot from | |
6481fd24 DW |
51 | the ``vfat`` formatted ESPs. In the context of ZFS as root filesystem, this means |
52 | that you can use all the optional features on your root pool, instead of the subset | |
9eb80400 | 53 | which is also present in the ZFS implementation in ``grub`` or having to create a |
6481fd24 | 54 | small, separate boot-pool (see: `Booting ZFS on root with grub |
9eb80400 SI |
55 | <https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS>`_). |
56 | ||
6481fd24 DW |
57 | In setups with redundancy, all disks are partitioned with an ESP by the |
58 | installer. This ensures the system boots, even if the first boot device fails | |
9eb80400 SI |
59 | or if the BIOS can only boot from a particular disk. |
60 | ||
61 | The ESPs are not kept mounted during regular operation. This helps to prevent | |
6481fd24 | 62 | filesystem corruption in the ``vfat`` formatted ESPs in case of a system crash, |
9eb80400 SI |
63 | and removes the need to manually adapt ``/etc/fstab`` in case the primary boot |
64 | device fails. | |
65 | ||
66 | ``proxmox-boot-tool`` handles the following tasks: | |
67 | ||
6481fd24 DW |
68 | * Formatting and setting up a new partition |
69 | * Copying and configuring new kernel images and initrd images to all listed ESPs | |
70 | * Synchronizing the configuration on kernel upgrades and other maintenance tasks | |
71 | * Managing the list of kernel versions which are synchronized | |
72 | * Configuring the boot-loader to boot a particular kernel version (pinning) | |
9eb80400 SI |
73 | |
74 | ||
75 | You can view the currently configured ESPs and their state by running: | |
76 | ||
77 | .. code-block:: console | |
78 | ||
79 | # proxmox-boot-tool status | |
80 | ||
81 | .. _systembooting-proxmox-boot-setup: | |
82 | ||
6481fd24 | 83 | Setting up a New Partition for use as Synced ESP |
9eb80400 SI |
84 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
85 | ||
6481fd24 | 86 | To format and initialize a partition as synced ESP, for example, after replacing a |
9eb80400 SI |
87 | failed vdev in an rpool, ``proxmox-boot-tool`` from ``pve-kernel-helper`` can be used. |
88 | ||
6481fd24 | 89 | WARNING: the ``format`` command will format the ``<partition>``. Make sure to pass |
9eb80400 SI |
90 | in the right device/partition! |
91 | ||
92 | For example, to format an empty partition ``/dev/sda2`` as ESP, run the following: | |
93 | ||
94 | .. code-block:: console | |
95 | ||
96 | # proxmox-boot-tool format /dev/sda2 | |
97 | ||
98 | To setup an existing, unmounted ESP located on ``/dev/sda2`` for inclusion in | |
99 | `Proxmox Backup`_'s kernel update synchronization mechanism, use the following: | |
100 | ||
101 | .. code-block:: console | |
102 | ||
103 | # proxmox-boot-tool init /dev/sda2 | |
104 | ||
6481fd24 | 105 | Following this, `/etc/kernel/proxmox-boot-uuids`` should contain a new line with the |
9eb80400 SI |
106 | UUID of the newly added partition. The ``init`` command will also automatically |
107 | trigger a refresh of all configured ESPs. | |
108 | ||
109 | .. _systembooting-proxmox-boot-refresh: | |
110 | ||
6481fd24 | 111 | Updating the Configuration on all ESPs |
9eb80400 SI |
112 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
113 | ||
114 | To copy and configure all bootable kernels and keep all ESPs listed in | |
6481fd24 | 115 | ``/etc/kernel/proxmox-boot-uuids`` in sync, you just need to run: |
9eb80400 SI |
116 | |
117 | .. code-block:: console | |
118 | ||
119 | # proxmox-boot-tool refresh | |
120 | ||
6481fd24 | 121 | (Equivalent to running ``update-grub`` on systems with ``ext4`` or ``xfs`` on root). |
9eb80400 | 122 | |
6481fd24 DW |
123 | This is necessary after making changes to the kernel commandline, or if you want |
124 | to sync all kernels and initrds. | |
9eb80400 SI |
125 | |
126 | .. NOTE:: Both ``update-initramfs`` and ``apt`` (when necessary) will automatically | |
127 | trigger a refresh. | |
128 | ||
6481fd24 | 129 | Kernel Versions Considered by ``proxmox-boot-tool`` |
9eb80400 SI |
130 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
131 | ||
132 | The following kernel versions are configured by default: | |
133 | ||
6481fd24 DW |
134 | * The currently running kernel |
135 | * The version being newly installed on package updates | |
136 | * The two latest, already installed kernels | |
137 | * The latest version of the second-to-last kernel series (e.g. 5.0, 5.3), if applicable | |
138 | * Any manually selected kernels | |
9eb80400 | 139 | |
6481fd24 | 140 | Manually Keeping a Kernel Bootable |
9eb80400 SI |
141 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
142 | ||
143 | Should you wish to add a certain kernel and initrd image to the list of | |
6481fd24 | 144 | bootable kernels, use ``proxmox-boot-tool kernel add``. |
9eb80400 | 145 | |
6481fd24 | 146 | For example, run the following to add the kernel with ABI version ``5.0.15-1-pve`` |
9eb80400 SI |
147 | to the list of kernels to keep installed and synced to all ESPs: |
148 | ||
149 | .. code-block:: console | |
150 | ||
151 | # proxmox-boot-tool kernel add 5.0.15-1-pve | |
152 | ||
153 | ||
154 | ``proxmox-boot-tool kernel list`` will list all kernel versions currently selected | |
155 | for booting: | |
156 | ||
157 | .. code-block:: console | |
158 | ||
159 | # proxmox-boot-tool kernel list | |
160 | Manually selected kernels: | |
161 | 5.0.15-1-pve | |
162 | ||
163 | Automatically selected kernels: | |
164 | 5.0.12-1-pve | |
165 | 4.15.18-18-pve | |
166 | ||
167 | Run ``proxmox-boot-tool kernel remove`` to remove a kernel from the list of | |
168 | manually selected kernels, for example: | |
169 | ||
170 | .. code-block:: console | |
171 | ||
172 | # proxmox-boot-tool kernel remove 5.0.15-1-pve | |
173 | ||
174 | ||
175 | .. NOTE:: It's required to run ``proxmox-boot-tool refresh`` to update all EFI System | |
176 | Partitions (ESPs) after a manual kernel addition or removal from above. | |
177 | ||
178 | ||
179 | .. _systembooting-determine-bootloader: | |
180 | ||
181 | Determine which Bootloader is Used | |
182 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
183 | ||
09d90303 SI |
184 | .. image:: images/screenshots/boot-grub.png |
185 | :target: _images/boot-grub.png | |
186 | :align: left | |
187 | :alt: Grub boot screen | |
9eb80400 SI |
188 | |
189 | The simplest and most reliable way to determine which bootloader is used, is to | |
190 | watch the boot process of the `Proxmox Backup`_ node. | |
191 | ||
09d90303 | 192 | |
9eb80400 SI |
193 | You will either see the blue box of ``grub`` or the simple black on white |
194 | ``systemd-boot``. | |
195 | ||
09d90303 SI |
196 | .. image:: images/screenshots/boot-systemdboot.png |
197 | :target: _images/boot-systemdboot.png | |
198 | :align: right | |
199 | :alt: systemd-boot screen | |
9eb80400 SI |
200 | |
201 | Determining the bootloader from a running system might not be 100% accurate. The | |
6481fd24 | 202 | most reliable way is to run the following command: |
9eb80400 SI |
203 | |
204 | ||
205 | .. code-block:: console | |
206 | ||
207 | # efibootmgr -v | |
208 | ||
209 | ||
210 | If it returns a message that EFI variables are not supported, ``grub`` is used in | |
211 | BIOS/Legacy mode. | |
212 | ||
213 | If the output contains a line that looks similar to the following, ``grub`` is | |
214 | used in UEFI mode. | |
215 | ||
216 | .. code-block:: console | |
217 | ||
218 | Boot0005* proxmox [...] File(\EFI\proxmox\grubx64.efi) | |
219 | ||
220 | ||
221 | If the output contains a line similar to the following, ``systemd-boot`` is used. | |
222 | ||
223 | .. code-block:: console | |
224 | ||
225 | Boot0006* Linux Boot Manager [...] File(\EFI\systemd\systemd-bootx64.efi) | |
226 | ||
227 | ||
6481fd24 DW |
228 | By running the following command, you can find out if ``proxmox-boot-tool`` is |
229 | configured, which is a good indication of how the system is booted: | |
9eb80400 SI |
230 | |
231 | .. code-block:: console | |
232 | ||
233 | # proxmox-boot-tool status | |
234 | ||
235 | ||
9eb80400 SI |
236 | .. _systembooting-grub: |
237 | ||
238 | Grub | |
239 | ~~~~ | |
240 | ||
6481fd24 | 241 | ``grub`` has been the de facto standard for booting Linux systems for many years |
9eb80400 SI |
242 | and is quite well documented |
243 | (see the `Grub Manual | |
244 | <https://www.gnu.org/software/grub/manual/grub/grub.html>`_). | |
245 | ||
246 | Configuration | |
247 | ^^^^^^^^^^^^^ | |
6481fd24 | 248 | |
9eb80400 | 249 | Changes to the ``grub`` configuration are done via the defaults file |
6481fd24 DW |
250 | ``/etc/default/grub`` or via config snippets in ``/etc/default/grub.d``. To |
251 | regenerate the configuration file after a change to the configuration, run: | |
9eb80400 SI |
252 | |
253 | .. code-block:: console | |
254 | ||
255 | # update-grub | |
256 | ||
257 | .. NOTE:: Systems using ``proxmox-boot-tool`` will call | |
258 | ``proxmox-boot-tool refresh`` upon ``update-grub`` | |
259 | ||
260 | .. _systembooting-systemdboot: | |
261 | ||
262 | Systemd-boot | |
263 | ~~~~~~~~~~~~ | |
264 | ||
265 | ``systemd-boot`` is a lightweight EFI bootloader. It reads the kernel and initrd | |
266 | images directly from the EFI Service Partition (ESP) where it is installed. | |
267 | The main advantage of directly loading the kernel from the ESP is that it does | |
268 | not need to reimplement the drivers for accessing the storage. In `Proxmox | |
6481fd24 | 269 | Backup`_, :ref:`proxmox-boot-tool <systembooting-proxmox-boot-tool>` is used to |
9eb80400 SI |
270 | keep the configuration on the ESPs synchronized. |
271 | ||
272 | .. _systembooting-systemd-boot-config: | |
273 | ||
274 | Configuration | |
275 | ^^^^^^^^^^^^^ | |
276 | ||
277 | ``systemd-boot`` is configured via the file ``loader/loader.conf`` in the root | |
278 | directory of an EFI System Partition (ESP). See the ``loader.conf(5)`` manpage | |
279 | for details. | |
280 | ||
6481fd24 | 281 | Each bootloader entry is placed in a file of its own, in the directory |
9eb80400 SI |
282 | ``loader/entries/`` |
283 | ||
284 | An example entry.conf looks like this (``/`` refers to the root of the ESP): | |
285 | ||
286 | .. code-block:: console | |
287 | ||
288 | title Proxmox | |
289 | version 5.0.15-1-pve | |
290 | options root=ZFS=rpool/ROOT/pve-1 boot=zfs | |
291 | linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve | |
292 | initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve | |
293 | ||
294 | ||
295 | .. _systembooting-edit-kernel-cmdline: | |
296 | ||
297 | Editing the Kernel Commandline | |
298 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
299 | ||
300 | You can modify the kernel commandline in the following places, depending on the | |
301 | bootloader used: | |
302 | ||
303 | Grub | |
304 | ^^^^ | |
305 | ||
306 | The kernel commandline needs to be placed in the variable | |
307 | ``GRUB_CMDLINE_LINUX_DEFAULT`` in the file ``/etc/default/grub``. Running | |
308 | ``update-grub`` appends its content to all ``linux`` entries in | |
309 | ``/boot/grub/grub.cfg``. | |
310 | ||
6481fd24 | 311 | systemd-boot |
9eb80400 SI |
312 | ^^^^^^^^^^^^ |
313 | ||
314 | The kernel commandline needs to be placed as one line in ``/etc/kernel/cmdline``. | |
315 | To apply your changes, run ``proxmox-boot-tool refresh``, which sets it as the | |
316 | ``option`` line for all config files in ``loader/entries/proxmox-*.conf``. | |
317 | ||
318 | ||
319 | .. _systembooting-kernel-pin: | |
320 | ||
321 | Override the Kernel-Version for next Boot | |
322 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
323 | ||
324 | To select a kernel that is not currently the default kernel, you can either: | |
325 | ||
6481fd24 | 326 | * Use the boot loader menu that is displayed at the beginning of the boot |
9eb80400 | 327 | process |
6481fd24 | 328 | * Use the ``proxmox-boot-tool`` to ``pin`` the system to a kernel version either |
9eb80400 SI |
329 | once or permanently (until pin is reset). |
330 | ||
331 | This should help you work around incompatibilities between a newer kernel | |
332 | version and the hardware. | |
333 | ||
6481fd24 DW |
334 | .. NOTE:: Such a pin should be removed as soon as possible, so that all recent |
335 | security patches from the latest kernel are also applied to the system. | |
9eb80400 | 336 | |
6481fd24 | 337 | For example, to permanently select the version ``5.15.30-1-pve`` for booting, you |
9eb80400 SI |
338 | would run: |
339 | ||
340 | .. code-block:: console | |
341 | ||
342 | # proxmox-boot-tool kernel pin 5.15.30-1-pve | |
343 | ||
344 | ||
345 | .. TIP:: The pinning functionality works for all `Proxmox Backup`_ systems, not only those using | |
346 | ``proxmox-boot-tool`` to synchronize the contents of the ESPs, if your system | |
6481fd24 | 347 | does not use ``proxmox-boot-tool`` for synchronizing, you can also skip the |
9eb80400 SI |
348 | ``proxmox-boot-tool refresh`` call in the end. |
349 | ||
350 | You can also set a kernel version to be booted on the next system boot only. | |
6481fd24 | 351 | This is useful, for example, to test if an updated kernel has resolved an issue, |
9eb80400 SI |
352 | which caused you to ``pin`` a version in the first place: |
353 | ||
354 | .. code-block:: console | |
355 | ||
356 | # proxmox-boot-tool kernel pin 5.15.30-1-pve --next-boot | |
357 | ||
358 | ||
6481fd24 | 359 | To remove any pinned version configuration, use the ``unpin`` subcommand: |
9eb80400 SI |
360 | |
361 | .. code-block:: console | |
362 | ||
363 | # proxmox-boot-tool kernel unpin | |
364 | ||
365 | While ``unpin`` has a ``--next-boot`` option as well, it is used to clear a pinned | |
366 | version set with ``--next-boot``. As that happens already automatically on boot, | |
6481fd24 | 367 | invoking it manually is of little use. |
9eb80400 | 368 | |
6481fd24 | 369 | After setting or clearing pinned versions, you also need to synchronize the |
9eb80400 SI |
370 | content and configuration on the ESPs by running the ``refresh`` subcommand. |
371 | ||
372 | .. TIP:: You will be prompted to automatically do for ``proxmox-boot-tool`` managed | |
373 | systems if you call the tool interactively. | |
374 | ||
375 | .. code-block:: console | |
376 | ||
377 | # proxmox-boot-tool refresh |