Daniel Kral [Wed, 20 Nov 2024 18:24:14 +0000 (19:24 +0100)]
common: make btrfs disk options uppercase for consistency
As XFS and ZFS are spelled in uppercase letters in the installer UI and
BTRFS is usually written in this way too, make the BTRFS string
uppercase too for consistency wrt to the other options.
Daniel Kral [Wed, 20 Nov 2024 18:24:13 +0000 (19:24 +0100)]
common: allow lowercase and uppercase btrfs raid levels
Allows the BTRFS RAID levels to be either lowercase or uppercase when
deserializing them from string values, i.e. currently only the config
value of `btrfs.raid` in auto-installer answer files.
Daniel Kral [Wed, 20 Nov 2024 18:24:12 +0000 (19:24 +0100)]
common: allow lowercase and uppercase zfs raid levels
Allows the ZFS RAID levels to be either lowercase or uppercase when
deserializing them from string values, i.e. currently only the config
value of `zfs.raid` in auto-installer answer files.
This partly fixes a regression, where deserializing the `zfs.raid`
property in answer files were only possible with uppercase values for
the ZFS RAID Z-levels, opposed to only lowercase as in previous
versions. This breaks the user API, as users cannot use the same answer
files as before for ZFS RAID Z-levels (the prepare-iso command fails).
Fixes: 510b0c008fb1 ("common: simplifying filesystem type serializing & Display trait impl") Signed-off-by: Daniel Kral <d.kral@proxmox.com>
Christoph Heiss [Tue, 19 Nov 2024 15:13:44 +0000 (16:13 +0100)]
first-boot: multi-user: start after product-specific API proxy
First of, multi-user.target does not seem to really provide any (strong)
ordering guarantee, it seems.
Instead, let the "fully-up" ordering from the auto-installer depend on
the product-specific API proxy instead.
That way, it is ensured that 1) the system really is fully up and 2)
that users could even use the API / CLI commands, or write files to
pmxcfs (in case of PVE).
After= and Wants= ignore non-existent units, so we can just specify all
three API proxy units here w/o any conditional.
Suggested-by: Shannon Sterz <s.sterz@proxmox.com> Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Filip Schauer [Tue, 19 Nov 2024 15:56:09 +0000 (16:56 +0100)]
fetch-answer: encode unsafe characters in partition label
Ensure potentially unsafe characters in the partition label are encoded,
preventing the installer from failing to find the answer partition when
the label contains whitespaces or !"$%&'()*,/;<>?[\]^`{|}~
The encoding is done according to `blkid_encode_string` [0] in the blkid
util, which is used by `/lib/udev/rules.d/60-persistent-storage.rules`
to create a symlink under `/dev/disk/by-label/`.
For example: "ANSWER PART" is encoded to "ANSWER\x20PART"
Users can specifying an optional file - either fetched from an URL or
backed into the ISO - to execute on the first boot after the
installation, using the 'proxmox-first-boot' oneshot service.
Essentially adds an (optional) `[first-boot]` section to the answer
file. If specified, the `source` key must be at least set, which gives
the location of the hook script.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Mon, 18 Nov 2024 12:38:37 +0000 (13:38 +0100)]
fix #5579: first-boot: add initial service packaging
While there is the `systemd-first-boot.service`, it uses the
non-existence of `/etc/machine-id` as condition to run. As we already
set up that file in the installer ourselves, we cannot use that.
Instead our service depends on a custom flag file in
/var/lib/proxmox-first-boot and will only run if that is present.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christian Ebner [Fri, 15 Nov 2024 15:35:00 +0000 (16:35 +0100)]
install: zfs: rename pre-existing `rpool` also if in degraded state
If a subset of disks associated with a pre-existing ZFS pool are
selected for installation, the pool might still be importable
(required for the rename) but will be in a `degraded` state.
Currently, only pools in `online` state will be considered for
renameing, leading a possibly clashing pool named `rpool` behind.
Therefore, a reboot after installation will fail because of the
duplicate names.
To partially fix this behaviour, also rename `rpool` in `degraded`
state.
Note:
This however does not cover the case when a pool can no longer be
imported because the number of required replicas is not available.
Renaming by zpool import is not possible for that case.
Partially-fixes: 43591049 ("low-level: install: check for already-existing `rpool` on install") Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Fri, 15 Nov 2024 12:43:18 +0000 (13:43 +0100)]
install: ask for patience when making system bootable
Making the system bootable can take some time if many disks are used
for installation, which could be misinterpreted as a hanging
installer. Add a please be patient output when more than 3 disks are
used.
Output changes from `make system bootable` to
`make system bootable (please be patient)`
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
[ TL: include hint for why user needs to be patient ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Thu, 14 Nov 2024 20:53:48 +0000 (21:53 +0100)]
auto-install: rename 'posthook' to 'post-installation-webhook'
Webhooks are nowadays pretty clear in what they do generally, so the
updated name should paint a clearer picture about what this is about
and avoid confusion with a under-development feature that executes a
admin provided program after the installation on the target system,
which could be also seen as "posthook".
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Christoph Heiss [Wed, 13 Nov 2024 13:59:03 +0000 (14:59 +0100)]
common: add function for issuing HTTP GET requests
Factors out the user-agent building into a separate function and then
re-uses that for get().
This has the side-effect that now for all requests issued by post() a
timeout of 60s is applied. Previously, this was only done when an
explicit fingerprint was given. Minute change and shouldn't effect
anything.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Thu, 14 Nov 2024 20:01:20 +0000 (21:01 +0100)]
fetch-answer, post-hook: normalize key for meta info to '$format-info'
This is something that we should not change, and it might be good to
use similar semantics and naming for similar things, so use a generic
"$format-info" property to save the schema/meta information.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Christoph Heiss [Wed, 13 Nov 2024 09:06:36 +0000 (10:06 +0100)]
fetch-answer: add `$fetchinfo` meta-field to HTTP POST payload
This adds a metadata-field `$fetchinfo` containing a single key
`version` (for now) to the POST payload json, indicating which schema
version (and thus structure) this document uses.
The version field follows the format "<major>.<minor>" and applies
semantic versioning meaning for both the major and minor number. A patch
version is left out here, as it doesn't make much sense in this context.
Works in the same manner as the post-hook does it [0]. Useful to have it
for this too, as we might change/expand this structure too in the
future.
In the resulting JSON, this will look like this:
{
"$fetchinfo": {
"version": "1.0"
},
"product": ..,
..
}
Christoph Heiss [Thu, 14 Nov 2024 14:15:28 +0000 (15:15 +0100)]
assistant: avoid regex for simple prefix matching
udev properties are very easy to parse and can be done by doing a
line-based scan and matching the prefix, splitting once for properties.
Avoids the use of regexes and signicantly reduces binary size by about
-46%(!).
Tested by comparing the output of `proxmox-auto-install-assistant
device-info`, running it before and after the changes.
Christoph Heiss [Tue, 12 Nov 2024 14:53:59 +0000 (15:53 +0100)]
post-hook: add `$hook` field describing document schema version
This adds a metadata-field `$hook` containing a single key `version`
(for now) to the post-hook json, indicating which schema version (and
thus structure) this document uses.
In the resulting JSON, this will look like this:
{
"$hook": {
"version": "1.0"
},
"debian-version": ..,
..
}
The field follows the format "<major>.<minor>" and applies semantic
versioning meaning for both the major and minor number. A patch version
is left out here, as it doesn't make much sense in this context.
This was suggested by Thomas when originally introducing the post-hook
functionality in [0].
Christoph Heiss [Tue, 12 Nov 2024 12:48:17 +0000 (13:48 +0100)]
tui: tests: fix reading ui messages from low-level installer
This actually broke with commit 723afe2 - this patch was quite old
(18.10.2023) and these tests were introduced some time after sending it,
thus is not adjusted for it.
Fix itself is pretty simple, simply ignore non-JSON/invalid message from
the low-level installer, much like the actual progression code does it.
Fixes: 723afe2 ("run env: always re-create run env file in test mode") Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Wed, 18 Oct 2023 08:17:19 +0000 (10:17 +0200)]
run env: always re-create run env file in test mode
When debugging or otherwise deliberately running the `dump-env`
low-level installer command in test mode, chances are that you'd want
the run env file to be re-created as well.
No impact on the normal installation flow.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Mon, 11 Nov 2024 13:15:07 +0000 (14:15 +0100)]
fix #5536: post-hook: add utility for sending notifications after auto-install
This utility can be called with the low-level install config after a
successful installation to send a notification via a HTTP POST request,
if the user has configured an endpoint for that in the answer file.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Wed, 14 Aug 2024 13:25:39 +0000 (15:25 +0200)]
tui: NumericEditView: add optional placeholder value
Enables to add an optional placeholder value to `NumericEditView`, which
will be displayed in a different (darker) color and not returned by
`.get_content*()`.
Can be used for having default values in the TUI, but with different
handling in the back.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Tue, 13 Aug 2024 16:15:31 +0000 (18:15 +0200)]
fix #5250: install: write btrfs `compress` option to fstab
`compress` instead of `compress-force` is used, as the latter can have
unindented (performance) implications, as the name implies. That would
be neither expected by users nor should such a decision made without the
user explicitly opting for it.
Others do the same, e.g. the installer for RedHat/Fedora systems (aka.
Anaconda) opts for `compress` too.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Wed, 21 Aug 2024 09:40:06 +0000 (11:40 +0200)]
common: simplify filesystem type serializing & Display trait impl
Implements the proper de-/serializer directly on the type and then
use serde_plain::derive_display_from_serialize where applicable, instead
of separate serializer functions somewhere else.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Wed, 21 Aug 2024 09:40:04 +0000 (11:40 +0200)]
fetch-answer: partition: fix clippy warning
warning: the borrowed expression implements the required traits
--> proxmox-fetch-answer/src/fetch_plugins/partition.rs:34:44
|
34 | let path = Path::new(search_path).join(&file_name);
| ^^^^^^^^^^ help: change this to: `file_name`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrows_for_generic_args
= note: `#[warn(clippy::needless_borrows_for_generic_args)]` on by default
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Fri, 8 Nov 2024 13:05:36 +0000 (14:05 +0100)]
fetch-answer: use partition label from fetch config instead of hardcoded
This has been requested by at least one user one user [0] and definitely
makes sense, esp. for BMCs/IPMIs where one might not be able to control
the partition label.
Christoph Heiss [Fri, 8 Nov 2024 13:05:35 +0000 (14:05 +0100)]
fetch-answer: partition: also search for exact-matching partition label
While some filesystems - such as FAT(32) - might not supported/allow
mixed-case labels, some implementations still handle them correctly,
such as Linux. Thus, also search for that variant.
Stoiko Ivanov [Wed, 30 Oct 2024 09:59:24 +0000 (10:59 +0100)]
installer-common: do not drop nomodeset from target kernel cmdline
dropping the `nomodeset` here makes little sense:
* currently users need to explicitly add it when booting the kernel,
which is probably only done when the system needs it to show any
output when booting
* it was originally removed, because the installer had a grub-entry
that explicitly disabled it - and the reasoning was to remove
everything installer-specific - but I assume that it should cause
less problems to keep it in place in all cases.
This was also partially pointed to in:
https://bugzilla.proxmox.com/show_bug.cgi?id=4230#c38
Fixes: a02a78a865fc37f7bdea077a4421319b84c635c9 Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Christoph Heiss [Wed, 2 Oct 2024 12:27:23 +0000 (14:27 +0200)]
tui: fix new clippy lint
warning: unnecessary hashes around raw string literal
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_raw_string_hashes
= note: `#[warn(clippy::needless_raw_string_hashes)]` on by default
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Tue, 13 Aug 2024 10:53:19 +0000 (12:53 +0200)]
tui: switch cursive to crossterm backend
Development of crossterm seems to be more active, has more frequent
releases and is also the default backend for cursive - so overall also
probably better supported/tested.
Additionally, it feels a bit more snappier/less flickery, probably due
to buffering writes to the TTY layer unlike termion.
As this effects only the way cursive talks to the TTY, this has no
visible change on the TUI itself.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Tue, 23 Jul 2024 09:24:57 +0000 (11:24 +0200)]
auto-installer: tests: fix wrong option name in expected output
The name of this configuration option has been changed with commit 0e1d973 [0]. The patch of the commit introducing this tests [1] was
posted earlier and wasn't rebased properly before applying.
[0] 0e1d973 ("install: config: rename option lvm_auto_rename -> existing_storage_auto_rename")
[1] 89314a4 ("auto-installer: add test for hashed root password option")
Fixes: 89314a4 ("auto-installer: add test for hashed root password option") Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Christoph Heiss [Mon, 15 Jul 2024 07:56:03 +0000 (09:56 +0200)]
low-level: change root password option to contain either plaintext or hash
A hashed password can be created e.g. using the `mkpasswd(1)`.
This then will allow the auto-installer to pass along a
already-hashed password from the user, instead of simple plaintext.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com> Tested-by: Theodor Fumics <theodor.fumics@gmx.net>
Christoph Heiss [Tue, 16 Jul 2024 08:18:07 +0000 (10:18 +0200)]
proxmox: add zfs module for retrieving importable zpool info
Will be used for prompting the user to rename existing "rpool" ZFS
pools, similar to what we do for an existing LVM "pve" volume group.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com> Reviewed-by: Aaron Lauterer <a.lauterer@proxmox.com> Tested-by: Aaron Lauterer <a.lauterer@proxmox.com>
[ TL: added a bit context for what this will be used and merge in the
tests into this preparatory commit ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Christoph Heiss [Wed, 10 Jul 2024 13:27:42 +0000 (15:27 +0200)]
tree-wide: collect hardcoded installer runtime directory strings into constant
No functional changes.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com> Reviewed-By: Stefan Hanreich <s.hanreich@proxmox.com>