Alwin Antreich [Wed, 12 Jun 2019 10:32:05 +0000 (12:32 +0200)]
cephfs: Exclude _netdev when mounting with fuse
Since ceph-fuse is called directly in the CephFS storage plugin, which
can not process the _netdev option, mounting the CephFS storage fails
when fuse is set in the storage.cfg.
This patch moves the _netdev option into the else part of the if fuse is
set statement. _netdev is only added if the CephFS kernel client mounts
the storage.
It seems _netdev is not needed anyway for the fuse mount, as the
connection is closed, once the fuse process gets killed on shutdown.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
(cherry picked from commit 9553b55c4f2e9f77caa59c5fab2d21fd151f8f70) Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Thu, 13 Jun 2019 09:59:58 +0000 (11:59 +0200)]
followup: reword comment a bit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 243f413ad28c6a8e70eeff52d835482d4fc18bd4) Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Reiter [Thu, 13 Jun 2019 09:33:28 +0000 (11:33 +0200)]
fix #1427: Set file mode on uploaded templates/ISOs
simply chmod the temp file before copying to the "correct" permission
mode, where all users with access to the directory can read the file,
to mirror the behavior one gets for a apl_download call.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
(cherry picked from commit 5c0720d6c99754540c23cc02092902ef89f3a890) Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
(cherry picked from commit 3ed611874ee24be08758d1e620a320f2f3224c38) Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Mira Limbeck [Mon, 29 Apr 2019 13:00:38 +0000 (15:00 +0200)]
map_volume: fall back to 'path'
Adds a fallback to 'Plugin::path' in the default implementation of
'map_volume' to simplify a common case of calling 'map_volume' followed
by a defined-check and a call to path if it is not. The path is now
always returned if the plugin in question does not override
'map_volume'.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Thomas Lamprecht [Wed, 17 Apr 2019 15:00:20 +0000 (15:00 +0000)]
zpool: handle race with other zpool imports
The underlying issue is that a zpool can get imported only once, so
we first check if it's in `zpool list`, and thus imported, and only
if it does not shows up there we try to import it.
But, this can race with either:
* parallel running activate_storage call, through CLI/API/daemon
* a zpool import from an admin (a bit unlikely, but hey that's the
thing with race conditions ;))
So refactor the "is pool imported" check into a closure, and call it
addditionally if the import failed, and silent the error if the pool
is now listed, and thus imported. This makes it a little bit nicer to
read too, IMO.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Fix #318: Delete vzdump log when deleting a backup
Vzdump log files were not deleted when a backup was deleted.
Consequently, the folder continuously filled with .log files.
Now they get deleted after the backup is removed.
Since zfsutils are not a hard dependency of our stack it is possible to not have
`zpool` available.
Checking for existance of `zpool` before calling it suppresses spurious warnings
in the logs (e.g. when creating Ceph OSDs or accessing the 'Disk' Tab in the
GUI).
1) checking that the empty storage list is treated correctly (only override
and datacenter config limit considered)
2) checking that the empty storage list is treated correctly (as with 1).
3) checking that undef can be passed as one element of the storage list (it is
ignored)
If one of the storages passed in $storage_list was not defined
get_bandwidth_limit died (see [0], of an occurence of this).
This patch changes the behavior to ignore undef as storage instead.
zfs: don't generate/update cachefile on pool import
during storage activation.
for pools that don't get imported at boot (e.g. because their vdevs are
not available when zfs-import-*.service runs) it is fatal to include
them in the cachefile, for those that do get imported at boot this code
should never run anyway as they are already imported.
in any case, a fallback to import without cachefile is the safe variant.
Mira Limbeck [Fri, 29 Mar 2019 09:13:10 +0000 (10:13 +0100)]
workaround zfs create -V error for unaligned sizes
fixes the 'cannot create 'nvme/foo': volume size must be a multiple of
volume block size' error by always rounding the size up to the next 1M
boundary. this is a workaround until
https://github.com/zfsonlinux/zfs/issues/8541 is solved.
the current manpage says 128k is the maximum blocksize, but a local test
showed that values up to 1M are allowed. it might be possible to
increase it even further (see f1512ee61).
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Christian Ebner [Mon, 4 Mar 2019 15:47:06 +0000 (16:47 +0100)]
fix #585: remove leftover disks/directory after VM creation failed
When trying to create a qcow2 disk image with a size larger than available on the
storage, this will fail.
As qemu-img does not clean up the disk afterwards, it needs to be deleted
explicitly. Further, the vmid folder is cleaned up once it is empty.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Stoiko Ivanov [Thu, 17 Jan 2019 15:31:51 +0000 (16:31 +0100)]
Fix #2050: only provide 'conv=sparse' for LvmThin
LVMPlugin->volume_import (used by storage_migrate on either offline
migration with local disks, or online migration with storage-only
referenced disks) passed 'conv=sparse' to `dd`. This can lead to
data-corruption, if the target volume is not zero-initialized.
dropping the sparse argument completely would fix the problem, but
breaks keeping data sparse for LvmThinPlugin.
This patch moves the dd out into (LVM*) plugin specific sub so that
each can control the parameters.
Steps for reproducing the issue:
* create a cluster with (at least) 2 nodes A and B, with a free
disk-device (/dev/sdx)
* write a recognizable pattern to /dev/sdx on B:
`dd if=/dev/zero bs=10M | tr '\000' '\255' | dd of=/dev/sdb bs=10M`
(would be grateful for alternatives to the dd| tr| dd)
* on both A and B create a lvm-vg (pvcreate, vgcreate)
* add it as _not_ shared storage, which is available on nodes A and B
* create a small guest on A
* fill a file in the guest with zeros
`dd if=/dev/zero of=/zerofil bs=10M`
* stop the guest, migrate it to B
* start the guest - check that the file `/zerofil` contains `ad`
instead of `00`
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stoiko Ivanov [Fri, 4 Jan 2019 13:06:25 +0000 (14:06 +0100)]
LVM: Add '--refresh' when activating volumes
From `man 8 lvchange`:
--refresh
If the logical volume is active, reload its metadata. This is not
necessary in normal operation, but may be useful ... if you're doing
clustering manually without a clustered lock manager.
Fixes migration in a shared LVM (iscsi) setup, where a disk gets resized on one
node A and the guest is afterwards migrated to another node B: B still presents
the old size to the guest, leading to data corruption.
It is necessary to run `lvchange` twice because the options `-ay` and
`--refresh` are mutually exclusive.
Stoiko Ivanov [Fri, 4 Jan 2019 13:06:24 +0000 (14:06 +0100)]
fix #2046 add volume_size_info to LVMPlugin
Without volume_size_info a Storage plugin falls back to the Implementation
in PVE/Storage/Plugin.pm, which relies on `qemu-img info`.
`qemu-img info` returns wrong results on a node in the case of shared volume
groups (e.g. when sharing disks via iSCSI), if a disk was resized on another
node (it lseeks to the end of the block-device, and this yields the old size).
Using lvs directly fixes the issue, since the LVM metadata gets updated when
invoking lvs.
Dominik Csapak [Wed, 19 Dec 2018 10:24:39 +0000 (11:24 +0100)]
move Storage/CephTools to CephConfig
it is not a storage plugin, and it makes more sense to have it
top-level, but there we cannot name it CephTools because of the
existing ones in pve-manager
`nvmeX` devices nodes are apparently allocated independently
from their namespace block devices `nvmeXnY` and therefore
they are not strictly related by name. For instance:
$ readlink /sys/block/nvme0n1/device
../../nvme1
$ readlink /sys/block/nvme1n1/device
../../nvme0
Here /dev/nvme0n1 is the first namespace of /dev/nvme1 while
/dev/nvme1n1 is the first namespace of /dev/nvme0.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Thomas Lamprecht [Tue, 27 Nov 2018 11:02:01 +0000 (12:02 +0100)]
cephfs: tell systemd that mount mount requires network
As we mount this manually and thus systemd doesn't know about any
dependency for cephFS mounts, this got umounted only at the last
stage of shutdown, where network wasn't active anymore.
But, CephFS needs to be connected to an active MDS for a clean
unmount so without network this mount would delay shutdown for quite
a bit, until after some minutes systemd gave up and forced unmount.
So tell systemd that this mount requires network, which can be done
with the '_netdev'[0] mount option, that lucky for us can be also
passed to a mount call and isn't only available for fstab.
with this a mount gets, among others:
> Wants=network-online.target
> Before=umount.target remote-fs.target
> After=remote-fs-pre.target system.slice network.target network-online.target -.mount
Thomas Lamprecht [Mon, 19 Nov 2018 13:44:46 +0000 (14:44 +0100)]
d/control: bump pve-common dependency and break older pve-manager
To ensure that we have the new SysFSTools available and as previously
we hosted the API scan endoints which pve-manager registered so
record the break of older versions.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With the addition of the map/unmap_volume() methods we made
an (actually unnecessary) API version bump.
All current users of these methods fall back to path() when
they return undef, so plugins implementing version 1 are
in fact compatible currently. (In fact, the default
Plugin::map_volume() could fall back to it on its own, but
doesn't currently).
For now let's just allow plugins older plugins to also be
loaded by introducing an API age variable. With it, if we
have a reason to break older plugins, we can have a
deprecation period during which older plugins cause a
warning instead of refusing to load altogether.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Thomas Lamprecht [Wed, 14 Nov 2018 13:43:29 +0000 (14:43 +0100)]
LVM: lock on volume_resize
This is important for shared LVM storages. As with deletes and
creates of images, as else we may have not the up-to-date metadata
and extents may get reused if another node created an image during
the same time, for example.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Wed, 14 Nov 2018 17:11:02 +0000 (18:11 +0100)]
fix #862: do not resolve portal adress on storage add
as described in #862:
> I experienced a problem with ISCSI portal when using a hostname and
> not IP.
> The GUI resolves the hostname to an IP and writes it to storage.cfg.
> As my setup requires hostnames, i needed to change the config
> manually back to the hostname which is working fine.
>
> Why is this conversion done? If I enter a hostname, i want to have a
> hostname. If i enter an IP address i want to have an IP address.
This makes sense to me, a feature of using domains is that they
are/should be resolved when actually using (i.e., connecting to them)
so resolving it once on add does not seems like a good idea (if I do
not miss something - as this is a classic "imported from SVN" I do
not have any rationale to look at).
So save the work and pass it as is.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
this was not installed, had references to non-existing modules
(e.g., PVE::ReplicationTools) and the things it probably was intended
for are done in pve-manager, which has full access to all pve perl
libs and API methods (from a dependency POV)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
projects / pve-storage.git / log