]>
Commit | Line | Data |
---|---|---|
1 | [[sysboot]] | |
2 | Host Bootloader | |
3 | --------------- | |
4 | ifdef::wiki[] | |
5 | :pve-toplevel: | |
6 | endif::wiki[] | |
7 | ||
8 | {pve} currently uses one of two bootloaders depending on the disk setup | |
9 | selected in the installer. | |
10 | ||
11 | For EFI Systems installed with ZFS as the root filesystem `systemd-boot` is | |
12 | used. All other deployments use the standard `grub` bootloader (this usually | |
13 | also applies to systems which are installed on top of Debian). | |
14 | ||
15 | ||
16 | [[sysboot_installer_part_scheme]] | |
17 | Partitioning Scheme Used by the Installer | |
18 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
19 | ||
20 | The {pve} installer creates 3 partitions on all disks selected for | |
21 | installation. | |
22 | ||
23 | The created partitions are: | |
24 | ||
25 | * a 1 MB BIOS Boot Partition (gdisk type EF02) | |
26 | ||
27 | * a 512 MB EFI System Partition (ESP, gdisk type EF00) | |
28 | ||
29 | * a third partition spanning the set `hdsize` parameter or the remaining space | |
30 | used for the chosen storage type | |
31 | ||
32 | Systems using ZFS as root filesystem are booted with a kernel and initrd image | |
33 | stored on the 512 MB EFI System Partition. For legacy BIOS systems, `grub` is | |
34 | used, for EFI systems `systemd-boot` is used. Both are installed and configured | |
35 | to point to the ESPs. | |
36 | ||
37 | `grub` in BIOS mode (`--target i386-pc`) is installed onto the BIOS Boot | |
38 | Partition of all selected disks on all systems booted with `grub` | |
39 | footnote:[These are all installs with root on `ext4` or `xfs` and installs | |
40 | with root on ZFS on non-EFI systems]. | |
41 | ||
42 | ||
43 | [[sysboot_proxmox_boot_tool]] | |
44 | Synchronizing the content of the ESP with `proxmox-boot-tool` | |
45 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
46 | ||
47 | `proxmox-boot-tool` is a utility used to keep the contents of the EFI System | |
48 | Partitions properly configured and synchronized. It copies certain kernel | |
49 | versions to all ESPs and configures the respective bootloader to boot from | |
50 | the `vfat` formatted ESPs. In the context of ZFS as root filesystem this means | |
51 | that you can use all optional features on your root pool instead of the subset | |
52 | which is also present in the ZFS implementation in `grub` or having to create a | |
53 | separate small boot-pool footnote:[Booting ZFS on root with grub | |
54 | https://github.com/zfsonlinux/zfs/wiki/Debian-Stretch-Root-on-ZFS]. | |
55 | ||
56 | In setups with redundancy all disks are partitioned with an ESP, by the | |
57 | installer. This ensures the system boots even if the first boot device fails | |
58 | or if the BIOS can only boot from a particular disk. | |
59 | ||
60 | The ESPs are not kept mounted during regular operation. This helps to prevent | |
61 | filesystem corruption to the `vfat` formatted ESPs in case of a system crash, | |
62 | and removes the need to manually adapt `/etc/fstab` in case the primary boot | |
63 | device fails. | |
64 | ||
65 | `proxmox-boot-tool` handles the following tasks: | |
66 | ||
67 | * formatting and setting up a new partition | |
68 | * copying and configuring new kernel images and initrd images to all listed ESPs | |
69 | * synchronizing the configuration on kernel upgrades and other maintenance tasks | |
70 | * managing the list of kernel versions which are synchronized | |
71 | ||
72 | ||
73 | You can view the currently configured ESPs and their state by running: | |
74 | ||
75 | ---- | |
76 | # proxmox-boot-tool status | |
77 | ---- | |
78 | ||
79 | [[sysboot_proxmox_boot_setup]] | |
80 | .Setting up a new partition for use as synced ESP | |
81 | ||
82 | To format and initialize a partition as synced ESP, e.g., after replacing a | |
83 | failed vdev in an rpool, or when converting an existing system that pre-dates | |
84 | the sync mechanism, `proxmox-boot-tool` from `pve-kernel-helpers` can be used. | |
85 | ||
86 | WARNING: the `format` command will format the `<partition>`, make sure to pass | |
87 | in the right device/partition! | |
88 | ||
89 | For example, to format an empty partition `/dev/sda2` as ESP, run the following: | |
90 | ||
91 | ---- | |
92 | # proxmox-boot-tool format /dev/sda2 | |
93 | ---- | |
94 | ||
95 | To setup an existing, unmounted ESP located on `/dev/sda2` for inclusion in | |
96 | {pve}'s kernel update synchronization mechanism, use the following: | |
97 | ||
98 | ---- | |
99 | # proxmox-boot-tool init /dev/sda2 | |
100 | ---- | |
101 | ||
102 | Afterwards `/etc/kernel/proxmox-boot-uuids` should contain a new line with the | |
103 | UUID of the newly added partition. The `init` command will also automatically | |
104 | trigger a refresh of all configured ESPs. | |
105 | ||
106 | [[sysboot_proxmox_boot_refresh]] | |
107 | .Updating the configuration on all ESPs | |
108 | ||
109 | To copy and configure all bootable kernels and keep all ESPs listed in | |
110 | `/etc/kernel/proxmox-boot-uuids` in sync you just need to run: | |
111 | ||
112 | ---- | |
113 | # proxmox-boot-tool refresh | |
114 | ---- | |
115 | (The equivalent to running `update-grub` systems with `ext4` or `xfs` on root). | |
116 | ||
117 | This is necessary should you make changes to the kernel commandline, or want to | |
118 | sync all kernels and initrds. | |
119 | ||
120 | NOTE: Both `update-initramfs` and `apt` (when necessary) will automatically | |
121 | trigger a refresh. | |
122 | ||
123 | .Kernel Versions considered by `proxmox-boot-tool` | |
124 | The following kernel versions are configured by default: | |
125 | ||
126 | * the currently running kernel | |
127 | * the version being newly installed on package updates | |
128 | * the two latest already installed kernels | |
129 | * the latest version of the second-to-last kernel series (e.g. 5.0, 5.3), if applicable | |
130 | * any manually selected kernels | |
131 | ||
132 | .Manually keeping a kernel bootable | |
133 | ||
134 | Should you wish to add a certain kernel and initrd image to the list of | |
135 | bootable kernels use `proxmox-boot-tool kernel add`. | |
136 | ||
137 | For example run the following to add the kernel with ABI version `5.0.15-1-pve` | |
138 | to the list of kernels to keep installed and synced to all ESPs: | |
139 | ||
140 | ---- | |
141 | # proxmox-boot-tool kernel add 5.0.15-1-pve | |
142 | ---- | |
143 | ||
144 | `proxmox-boot-tool kernel list` will list all kernel versions currently selected | |
145 | for booting: | |
146 | ||
147 | ---- | |
148 | # proxmox-boot-tool kernel list | |
149 | Manually selected kernels: | |
150 | 5.0.15-1-pve | |
151 | ||
152 | Automatically selected kernels: | |
153 | 5.0.12-1-pve | |
154 | 4.15.18-18-pve | |
155 | ---- | |
156 | ||
157 | Run `proxmox-boot-tool kernel remove` to remove a kernel from the list of | |
158 | manually selected kernels, for example: | |
159 | ||
160 | ---- | |
161 | # proxmox-boot-tool kernel remove 5.0.15-1-pve | |
162 | ---- | |
163 | ||
164 | NOTE: It's required to run `proxmox-boot-tool refresh` to update all EFI System | |
165 | Partitions (ESPs) after a manual kernel addition or removal from above. | |
166 | ||
167 | ||
168 | [[sysboot_determine_bootloader_used]] | |
169 | Determine which Bootloader is Used | |
170 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
171 | ||
172 | [thumbnail="screenshot/boot-grub.png", float="left"] | |
173 | ||
174 | The simplest and most reliable way to determine which bootloader is used, is to | |
175 | watch the boot process of the {pve} node. | |
176 | ||
177 | You will either see the blue box of `grub` or the simple black on white | |
178 | `systemd-boot`. | |
179 | ||
180 | [thumbnail="screenshot/boot-systemdboot.png"] | |
181 | ||
182 | Determining the bootloader from a running system might not be 100% accurate. The | |
183 | safest way is to run the following command: | |
184 | ||
185 | ||
186 | ---- | |
187 | # efibootmgr -v | |
188 | ---- | |
189 | ||
190 | If it returns a message that EFI variables are not supported, `grub` is used in | |
191 | BIOS/Legacy mode. | |
192 | ||
193 | If the output contains a line that looks similar to the following, `grub` is | |
194 | used in UEFI mode. | |
195 | ||
196 | ---- | |
197 | Boot0005* proxmox [...] File(\EFI\proxmox\grubx64.efi) | |
198 | ---- | |
199 | ||
200 | If the output contains a line similar to the following, `systemd-boot` is used. | |
201 | ||
202 | ---- | |
203 | Boot0006* Linux Boot Manager [...] File(\EFI\systemd\systemd-bootx64.efi) | |
204 | ---- | |
205 | ||
206 | By running: | |
207 | ||
208 | ---- | |
209 | # proxmox-boot-tool status | |
210 | ---- | |
211 | ||
212 | you can find out if `proxmox-boot-tool` is configured, which is a good | |
213 | indication of how the system is booted. | |
214 | ||
215 | ||
216 | [[sysboot_grub]] | |
217 | Grub | |
218 | ~~~~ | |
219 | ||
220 | `grub` has been the de-facto standard for booting Linux systems for many years | |
221 | and is quite well documented | |
222 | footnote:[Grub Manual https://www.gnu.org/software/grub/manual/grub/grub.html]. | |
223 | ||
224 | Configuration | |
225 | ^^^^^^^^^^^^^ | |
226 | Changes to the `grub` configuration are done via the defaults file | |
227 | `/etc/default/grub` or config snippets in `/etc/default/grub.d`. To regenerate | |
228 | the configuration file after a change to the configuration run: | |
229 | footnote:[Systems using `proxmox-boot-tool` will call `proxmox-boot-tool | |
230 | refresh` upon `update-grub`.] | |
231 | ||
232 | ---- | |
233 | # update-grub | |
234 | ---- | |
235 | ||
236 | ||
237 | [[sysboot_systemd_boot]] | |
238 | Systemd-boot | |
239 | ~~~~~~~~~~~~ | |
240 | ||
241 | `systemd-boot` is a lightweight EFI bootloader. It reads the kernel and initrd | |
242 | images directly from the EFI Service Partition (ESP) where it is installed. | |
243 | The main advantage of directly loading the kernel from the ESP is that it does | |
244 | not need to reimplement the drivers for accessing the storage. In {pve} | |
245 | xref:sysboot_proxmox_boot_tool[`proxmox-boot-tool`] is used to keep the | |
246 | configuration on the ESPs synchronized. | |
247 | ||
248 | [[sysboot_systemd_boot_config]] | |
249 | Configuration | |
250 | ^^^^^^^^^^^^^ | |
251 | ||
252 | `systemd-boot` is configured via the file `loader/loader.conf` in the root | |
253 | directory of an EFI System Partition (ESP). See the `loader.conf(5)` manpage | |
254 | for details. | |
255 | ||
256 | Each bootloader entry is placed in a file of its own in the directory | |
257 | `loader/entries/` | |
258 | ||
259 | An example entry.conf looks like this (`/` refers to the root of the ESP): | |
260 | ||
261 | ---- | |
262 | title Proxmox | |
263 | version 5.0.15-1-pve | |
264 | options root=ZFS=rpool/ROOT/pve-1 boot=zfs | |
265 | linux /EFI/proxmox/5.0.15-1-pve/vmlinuz-5.0.15-1-pve | |
266 | initrd /EFI/proxmox/5.0.15-1-pve/initrd.img-5.0.15-1-pve | |
267 | ---- | |
268 | ||
269 | ||
270 | [[sysboot_edit_kernel_cmdline]] | |
271 | Editing the Kernel Commandline | |
272 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
273 | ||
274 | You can modify the kernel commandline in the following places, depending on the | |
275 | bootloader used: | |
276 | ||
277 | .Grub | |
278 | ||
279 | The kernel commandline needs to be placed in the variable | |
280 | `GRUB_CMDLINE_LINUX_DEFAULT` in the file `/etc/default/grub`. Running | |
281 | `update-grub` appends its content to all `linux` entries in | |
282 | `/boot/grub/grub.cfg`. | |
283 | ||
284 | .Systemd-boot | |
285 | ||
286 | The kernel commandline needs to be placed as one line in `/etc/kernel/cmdline`. | |
287 | To apply your changes, run `proxmox-boot-tool refresh`, which sets it as the | |
288 | `option` line for all config files in `loader/entries/proxmox-*.conf`. |