Thomas Lamprecht [Fri, 24 Apr 2020 13:17:26 +0000 (15:17 +0200)]
unconfigured: make agetty listen on tty9
it's a bit brittle and login is weird (enter root user twice, pw
doesn't matters) and once exited it's gone, but it's just for
"emergency debugging" anyway
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Aaron Lauterer [Mon, 24 Feb 2020 08:37:23 +0000 (09:37 +0100)]
fix behavior if zfs disks have no by-id path
in some situations it is possible, that a disk does not have a
/dev/disk/by-id path. Reported cases are KVM with virtio-blk [0] and
with VMware player / workstation when using the paravirtual SCSI disk
types [1]. It can be reproduced in Proxmox VE by using virtio-blk disks
for the VM.
> The issue at hand happens because udev does not create the
> /dev/disk/by-id symlink in
> '/lib/udev/rules.d/60-persistent-storage.rules', since it does
> not have a serial-number for the device (which in the virtio-blk case
> is in turn maybe caused by qemu-server not adding one to the
> commandline).
Quoted from [2]
This regression was introduced with commit e1b490865f750e08f6c9c6b7e162e7def9dcc411 which always replaced the disks
in the $vdev variable even if the result of the by-id lookup was
`undef`. Thus the paths did not point to any valid disk and the zpool
creation failed.
This patch replaces the disk paths with by-id paths only if they are
present.
`policy-rc.d` invocation is still used by deb-systemd-invoke (1p), despite
it's by now unfitting name.
The policy-rc.d script the installer placed into the chroot prevents the
starting of services by (debianized) maintainer-scripts [0].
This should reduce installation time on one hand, and on the other hand does
mask errors in maintainer-scripts, which error out if no systemd is running
(e.g. inside the chroot)
Noticed the problem, when testing an unrelated patch and running into
an aborted installtion, due to a, already fixed, glitch (missing '||true')
in pve-lxc's postinst script.
[0] https://people.debian.org/~hmh/invokerc.d-policyrc.d-specification.txt
(did not find anything more current, contradicting the information there)
Packages which we have under control have to ensure themself that the
correct permissions are set, else the "install on top of Debian" also
fails. It just crowd proxinstall unecessarily, so avoid to add new
ones if possible. This specific one is handled in the respective
package since quite a bit.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stoiko Ivanov [Fri, 29 Nov 2019 10:53:44 +0000 (11:53 +0100)]
umount testdir recursively in check-* and clean
recursively unmount testdir before removing it. This prevents the subsequent
call to try to remove files in potentially still bindmounted /proc, /sys, /dev
filesystems.
Dominik Csapak [Thu, 28 Nov 2019 12:39:56 +0000 (13:39 +0100)]
restrict admin email to same format as the JSONSchema
Use the same regex as the 'email' format in JSONSchema (copied from there).
Otherwise, users can enter an email address which does not match our
schema and breaks the userlist API Call with a 'return value
verification failed', and one cannot rectify this via the gui
(the user has to manually edit the user.cfg or set the email via
pveum)
i noticed this while testing and using the 'insert emoji' menu in the
context menu
Stoiko Ivanov [Wed, 27 Nov 2019 16:06:59 +0000 (17:06 +0100)]
use by-id paths for all vdevs on pool creation
currently only the first vdev of a ZFS rpool gets created with the stable
/dev/by-id paths, all later vdevs (in a RAID0 or RAID10 setup) are left as
sdX.
By iterating over all disks in the pool and replacing their occurence with
their by-id path we get consistent and recommended names for all vdevs.
Stoiko Ivanov [Wed, 27 Nov 2019 16:06:58 +0000 (17:06 +0100)]
wipe partitiontable after early exits
by wiping the partition table after the initial sanity checks regarding
minimal size and blocksize of the device, no data is destroyed for an
install that would fail in any case.
Stoiko Ivanov [Wed, 27 Nov 2019 16:06:57 +0000 (17:06 +0100)]
fix #1211: allow install on 4kn disks
Installation on disks with 4k logical blocksize (4kn) failed, because
the bios_boot (a.k.a. gdisk partitiontype EF02, place for grub in
legacy BIOS boot mode) partition is created using start and end
sectors (and sector 2047 is not at 1M, where the ESP starts)
This patch addresses the issue by not creating the bios_boot
partition on 4kn disks at all - legacy boot from 4kn disks is not
supported by most BIOS implementations and grub does not support it
[0].
Checks for 4kn disks, when booted in legacy mode are added, and
prevent from leaving the harddisk selection page, if an not bootable
selection was made.
The partition numbering was kept (esp is partition 2, data is
partition 3, for consistency with other installations)
If any of the bootable disks is 4kn then the installation of the grub
legacy installation is skipped altogether.
Additionally the invocation of mkfs.vfat needs to add the parameter
'-s1' to create a bootable ESP on 4kn disks.
For a 512M vfat partition on a 512n or 512e disk, mkfs.vfat
calculates a default value of 8 sectors per cluster. For a 512M
partition on a 4kn disk, we need to scale this by 512/4096=8 and
explicitly set '-s' to 1 accordingly, since the calculation in
mkfs.vfat is brokenly assuming 512b sectors[1].
Tested with a qemu-machine by passing
'logical_block_size=4096,physical_block_size=4096' to the disk's device lines
and installing in UEFI and legacy booted mode:
* ZFS RAIDZ1
* ZFS single-disk
* ZFS RAID10 (in legacy mode grub fails to install, if any 4kn disk is in the
pool, even if it's not in the first vdev)
* EXT4
Stoiko Ivanov [Wed, 23 Oct 2019 16:10:26 +0000 (18:10 +0200)]
raise postifx main.cf compatibility_level to 2
otherwise a warning was issued (when missing /etc/aliases.db) that the system
is using the backward compatible setting of $mydestination for $relay_domain
(see [0]).
by collecting them, but proceeding with the rest of the installation. it
is possible that only either legacy or EFI boot fails, and even if both
do, manually fixing the bootloader setup by booting a live CD is very
often possible.
still display the error screen at the end if bootloader setup was not
successful, so that expectations are not set too high..
prompt user if a vgrename is OK for exisiting 'pve'/'pmg' VGs
If one has a 'pve' VG on a disks not selected as install target
(e.g., on re-installation to different disk or if putting a used disk
into another server where PVE was installed on) the vgcreate call
errored out, as for creation the VG names must be unique.
Cope with that by asking the users if a rename to a
'<vgname>-OLD-<short-uid>' name is OK in this case. It ensures that
no data is lost and that we can safely continue with the
installation. The admin can then later on wipe the renamed VG if it
was really decommissioned or save data from it (or actually use it)
This can cope (tested) with:
* a single 'pve' VG on another device
* a single 'pve' VG spanning multiple devices
* multiple 'pve' VGs spanning different sets of devices
This is achieved by using the VG UUID for rename, and by recording
all PVs with said UUID as index.
Note, while this commit message talks mostly about 'pve' VG the patch
itself is actually agnostic of the specific name, works for 'pmg' and
possible (future) other VG names too.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
/etc/hostid is used by ZFS (spl.ko) to determine which host last imported a
pool creating and importing a pool with one hostid during install and booting
with a different one (or none) leads to the system refusing to import the pool
see spl-module-parameters(5) and zpool(8).
by copying the /etc/hostid from the installer into the target system we ensure
that it stays consistent
mount efivarfs to ensure we can read bigger variables
In short, EFI variables can get quite big, and the old sysfs
interface was made for when they couldn't. A few firmwares out there
have such big variables, and if those are accessed through the sysfs
backed interface one gets a "Input/Output Error". 'grub-install'
chokes on that error when it iterates over all variables to do it's
stuff, and thus fails our installation. When we mount the efivarfs,
which does not has this limitations, one can read all variables just
fine - at least as long as the NVRAM backing them is not broken.
from Linux Kernel Documentation/filesystems/efivarfs.txt:
> The efivarfs filesystem was created to address the shortcomings of
> using entries in sysfs to maintain EFI variables. The old sysfs EFI
> variables code only supported variables of up to 1024 bytes. This
> limitation existed in version 0.99 of the EFI specification, but was
> removed before any full releases. Since variables can now be larger
> than a single page, sysfs isn't the best interface for this.
> Variables can be created, deleted and modified with the efivarfs
> filesystem.
Also mount it in the installer environment for debugging purpose.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
underline that we detected less than 1024 MB of *useable* memory
firmwar, kernel, and some PCI(e) devices can use some memory, which
then is not available for userspace, thus the total memory installed
for this server normally differs at least about 50 MB from the
useable one.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This ensures that when booting from a USB stick, or similar device,
we do not show that device in the "Target Harddisk" selector.
While there may be some constructed cases where this actually worked
and was wanted, e.g., one created a partition on the end of a USB
storage device, put the ISO on that and used the beginning of that
device as target, I'd say that even if that worked with luck, it's
better not actively not support it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Oguz Bektas [Tue, 2 Jul 2019 13:17:32 +0000 (15:17 +0200)]
abort installation if memory is less than 1GB
show a message on screen about memory requirement, then die and abort
the installation. (Note: followup removes the die again - Thomas)
The "magical" limit seems around 850 - 900 MiB memory needed, it's
a bit strange, as we do not need that much, proxinstall + webkit +
base system are around 300 MiB, the rest is page cache, but it seems
that the memory pressure gets to big.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With the recent fixes to the installers initial /dev creation, the installer
and the kernel should now have the same view on the created devices and links
along with `pvremove -ff` and writing zeroes to the first 16M of all
partitions, which belong to disks selected in the installer run `zpool
labelclear`.
This prevents a failure to boot after installation, if the disks were
previously also used as a zpool called rpool, but in a different configuration
(e.g. installing a raidzX first and then changing to raid10).
update systemd-boot config after initializing esps
run the kernel-postinst hook once (like we do for update-grub) instead
of once per ESP. This fixes an error (the hook does not write entries for
mounted partitions), and save quite a bit of (un)mounting and copying in case
of multiple ESPs
Thomas Lamprecht [Mon, 25 Mar 2019 20:18:23 +0000 (21:18 +0100)]
unconfigured: mount /dev/shm as tmpfs inside chroot
as preparation for GKT3 WebKit2 perl bindings, which wants to make a
shared memory file there, but fails and hangs the installer as
/dev/shm didn't exists in the chroot environment.
Doesn't hurt to just create it now already
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Mon, 25 Mar 2019 18:47:05 +0000 (19:47 +0100)]
tell udev to ignore being in a chroot
since udev/systemd v241 udev actively ignores requests if it detects
running in a chroot environment[0], as we do exactly this in our
setup process we need to tell udev that it is OK and we know what we
do, so set SYSTEMD_IGNORE_CHROOT=1 which makes the
"running_in_chroot" function lie[1].
Thomas Lamprecht [Thu, 13 Dec 2018 10:13:32 +0000 (11:13 +0100)]
buildsys: do not copy test disk images over to build dir
this is unecessary and rsycn does no sparse copy if the --sparse
option is not passed.
As I want to introduce easier testing with multiple disks this helps
alot in reducing test setup and package build time.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>