]> git.proxmox.com Git - qemu-server.git/log
qemu-server.git
3 years agod/control: bump versioned dependency on pve-common
Fabian Grünbichler [Wed, 8 Jul 2020 08:24:54 +0000 (10:24 +0200)]
d/control: bump versioned dependency on pve-common

for PVE::JSONSchema with additional validator support

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agofix #2671: include CPU format in man page again
Stefan Reiter [Thu, 25 Jun 2020 11:35:41 +0000 (13:35 +0200)]
fix #2671: include CPU format in man page again

Use the new register_format(3) call to use a validator (instead of a
parser) for 'pve-(vm-)?cpu-conf'. This way the $cpu_fmt hash can be used for
generating the documentation, while still applying the same verification
rules as before.

Since the function no longer parses but only verifies, the parsing in
print_cpu_device/get_cpu_options has to go via JSONSchema directly.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agobump version to 6.2-6
Thomas Lamprecht [Tue, 7 Jul 2020 16:59:59 +0000 (18:59 +0200)]
bump version to 6.2-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agovzdump: log: small fixups
Thomas Lamprecht [Tue, 7 Jul 2020 15:32:25 +0000 (17:32 +0200)]
vzdump: log: small fixups

reuse can also come from the current backup - so drop the "from last
backup" as this can be very confusing if one reads it after making
the first backup ever, with no last backup existing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agovzdump: fix variable redeclaration warning
Thomas Lamprecht [Tue, 7 Jul 2020 06:41:22 +0000 (08:41 +0200)]
vzdump: fix variable redeclaration warning

happened due to moving the code from another scope which had no $res,
and not noticing as it was still working after all.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.2-5
Thomas Lamprecht [Mon, 6 Jul 2020 19:50:59 +0000 (21:50 +0200)]
bump version to 6.2-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobackup: PBS: save backup task size
Thomas Lamprecht [Mon, 6 Jul 2020 19:47:30 +0000 (21:47 +0200)]
backup: PBS: save backup task size

normally this is done centrally in the managers code, but we do not
have the info for PBS there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobackup: query status: use latest up-to-date info when finished
Thomas Lamprecht [Mon, 6 Jul 2020 19:46:03 +0000 (21:46 +0200)]
backup: query status: use latest up-to-date info when finished

we used the info from the previous loop for some summary logs, fix
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobackup: restructure log: use human-readable sizes and include PBS info
Thomas Lamprecht [Mon, 6 Jul 2020 19:45:25 +0000 (21:45 +0200)]
backup: restructure log: use human-readable sizes and include PBS info

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobackup: rename confusing incremental param to use-dirty-bitmap
Thomas Lamprecht [Mon, 6 Jul 2020 18:33:13 +0000 (20:33 +0200)]
backup: rename confusing incremental param to use-dirty-bitmap

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoupdate_disksize: make interface leaner
Fabian Ebner [Wed, 20 May 2020 08:20:37 +0000 (10:20 +0200)]
update_disksize: make interface leaner

Pass new size directly, so the function doesn't need to know about
how some hash is organized. And return a message directly, instead
of both size-strings. Also dropped the wantarray, because both
existing callers use the message anyways.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 years agosync_disks: fix check
Fabian Ebner [Wed, 20 May 2020 08:20:36 +0000 (10:20 +0200)]
sync_disks: fix check

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 years agobump version to 6.2-4
Thomas Lamprecht [Tue, 30 Jun 2020 09:34:33 +0000 (11:34 +0200)]
bump version to 6.2-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoenable dirty-bitmap incremental backups for PBS
Stefan Reiter [Thu, 25 Jun 2020 10:23:34 +0000 (12:23 +0200)]
enable dirty-bitmap incremental backups for PBS

The $total != $transferred check is changed to a log, as QEMU reports
only actually transferred bytes, and it is indeed correct for
incremental backups to have differing values from $total.

The 'incremental' parameter is always set, QEMU will figure out if it should
re-use an existing bitmap or create a new one on its own.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agoclose #2741: add VM.Config.Cloudinit permission
Mira Limbeck [Thu, 25 Jun 2020 14:48:47 +0000 (16:48 +0200)]
close #2741: add VM.Config.Cloudinit permission

This allows setting ciuser, cipassword and all other cloudinit settings that
are not part of the network without VM.Config.Network permissions.
Keep VM.Config.Network still as fallback so custom roles that add
VM.Config.Network but not VM.Config.Cloudinit don't break.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
3 years agoavoid backup command timeout with pbs
Dietmar Maurer [Fri, 26 Jun 2020 05:10:57 +0000 (07:10 +0200)]
avoid backup command timeout with pbs

3 years agopci: whitespace, indentation and formating fixes
Thomas Lamprecht [Thu, 25 Jun 2020 11:33:26 +0000 (13:33 +0200)]
pci: whitespace, indentation and formating fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agofix #2794: allow legacy IGD passthrough
Stefan Reiter [Thu, 18 Jun 2020 14:36:54 +0000 (16:36 +0200)]
fix #2794: allow legacy IGD passthrough

Legacy IGD passthrough requires address 00:1f.0 to not be assigned to
anything on QEMU startup (currently it's assigned to bridge pci.2).
Changing this in general would break live-migration, so introduce a new
hostpci parameter "legacy-igd", which if set to 1 will move that bridge
to be nested under bridge 1.

This is safe because:
* Bridge 1 is unconditionally created on i440fx, so nesting is ok
* Defaults are not changed, i.e. PCI layout only changes when the new
parameter is specified manually
* hostpci forbids migration anyway

Additionally, the PT device has to be assigned address 00:02.0 in the
guest as well, which is usually used for VGA assignment. Luckily, IGD PT
requires vga=none, so that is not an issue either.

See https://git.qemu.org/?p=qemu.git;a=blob;f=docs/igd-assign.txt

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agocfg2cmd: hostpci: move code to PCI.pm
Stefan Reiter [Thu, 18 Jun 2020 14:36:53 +0000 (16:36 +0200)]
cfg2cmd: hostpci: move code to PCI.pm

To avoid further cluttering config_to_command with subsequent changes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agotrivial whitespace followup fix
Thomas Lamprecht [Wed, 24 Jun 2020 08:59:05 +0000 (10:59 +0200)]
trivial whitespace followup fix

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agovzdump: move include logic for volumes to method
Aaron Lauterer [Mon, 22 Jun 2020 14:34:36 +0000 (16:34 +0200)]
vzdump: move include logic for volumes to method

Move the logic which volumes are included in the backup job to its own
method and adapt the VZDump code accordingly. This makes it possible to
develop other features around backup jobs.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
3 years agogen_rand_chars: handle errors properly
Fabian Grünbichler [Mon, 22 Jun 2020 10:03:01 +0000 (12:03 +0200)]
gen_rand_chars: handle errors properly

should not really happen on modern systems, but random_bytes just
returns false if it fails to generate random bytes, in which case we
want to die instead of returning an empty 'random' string.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agovncproxy: allow to request a generated VNC password
Thomas Lamprecht [Thu, 18 Jun 2020 16:20:10 +0000 (18:20 +0200)]
vncproxy: allow to request a generated VNC password

We used the VNC API $ticket as password for VNC, but QEMU limits the
password to the first 8 chars and ignores the rest[0].
As our tickets start with a static string (e.g., "PVE") the entropy
was a bit limited.

For Proxmox VE this does not matters much as the noVNC viewer
provided by has to go always over the API call, and so a valid
ticket and correct permissions for the requested VM are enforced
anyway.

This patch helps external users, which often use NoVNC-Websockify,
circumventing the API and relying solely on the VNC password to avoid
snooping on VNC sessions.

A 'generate-password' parameter is added, if set a password from good
entropy (using libopenssl) is generated.

For simplicity of mapping random bits to ranges we extract 6 bit of
entropy per character and add the integer value of '!' (first
printable ASCII char) to that. This way we get 64^8 possibilities,
which even with millions of guesses per second one would need years
of guessing and mostly just DDOS the server with websocket upgrade
requests.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
3 years agoCPUConfig: add add_cpu_json_properties()
Stefan Reiter [Thu, 18 Jun 2020 09:05:07 +0000 (11:05 +0200)]
CPUConfig: add add_cpu_json_properties()

Useful for APIs and docs.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agofix indentation
Thomas Lamprecht [Thu, 18 Jun 2020 09:26:36 +0000 (11:26 +0200)]
fix indentation

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agofollowup whitespace cleanup
Thomas Lamprecht [Wed, 17 Jun 2020 12:05:49 +0000 (14:05 +0200)]
followup whitespace cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoproperly parse 'vga' for termproxy
Fabian Grünbichler [Wed, 17 Jun 2020 08:41:17 +0000 (10:41 +0200)]
properly parse 'vga' for termproxy

'vga' is a property string, we can't just assume it starts with the default key's value here either.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agofix #2787: properly parse 'vga' for vncproxy
Fabian Grünbichler [Wed, 17 Jun 2020 08:41:16 +0000 (10:41 +0200)]
fix #2787: properly parse 'vga' for vncproxy

'vga' is a property string, we can't just assume it starts with the
default key's value.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agocfg2cmd test: hardcode/mock bridge MTU
Fabian Grünbichler [Wed, 17 Jun 2020 08:30:13 +0000 (10:30 +0200)]
cfg2cmd test: hardcode/mock bridge MTU

otherwise the netdev test reads the MTU value from the test host's vmbr0
bridge, or fails if no such bridge exists.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agobump version to 6.2-3
Thomas Lamprecht [Thu, 4 Jun 2020 09:19:17 +0000 (11:19 +0200)]
bump version to 6.2-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocfg2cmd: add simple MTU test
Thomas Lamprecht [Thu, 4 Jun 2020 09:15:52 +0000 (11:15 +0200)]
cfg2cmd: add simple MTU test

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agonetdev MTU: stricter checks and warn if not possible on netdev
Thomas Lamprecht [Thu, 4 Jun 2020 09:11:52 +0000 (11:11 +0200)]
netdev MTU: stricter checks and warn if not possible on netdev

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd virtio host_mtu feature.
Alexandre Derumier [Fri, 17 Apr 2020 05:47:20 +0000 (07:47 +0200)]
add virtio host_mtu feature.

This add a new "mtu" param to vm nic,
and force the mtu in the guest for virtio nic only.

Special value: 1 = set the same value than the bridge

3 years agofix pbs-restore call for non raw images
Dominik Csapak [Thu, 4 Jun 2020 08:52:42 +0000 (10:52 +0200)]
fix pbs-restore call for non raw images

if we have a format, give it to pbs-restore too
fixes restore of non-raw (e.g. qcow2) images

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agofix pbs backups of templates/vms without disks
Dominik Csapak [Wed, 3 Jun 2020 12:28:22 +0000 (14:28 +0200)]
fix pbs backups of templates/vms without disks

fingerprint is now given with an environment variable not as parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agofix #2774: add early check for non-managed volumes
Fabian Grünbichler [Tue, 2 Jun 2020 08:46:27 +0000 (10:46 +0200)]
fix #2774: add early check for non-managed volumes

when checking whether a to-be-added drive's and the VM's replication
status are matching. otherwise, we end up in a failing generic
'parse_volume_id' with no mention of the actual reason.

adding 'replicate=0' to the new drive string fixes the underlying issue
with and without this patch, so this is just a cosmetic/usability
improvement.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agocreate_disks: fix uninitialized warning
Fabian Grünbichler [Tue, 2 Jun 2020 08:46:26 +0000 (10:46 +0200)]
create_disks: fix uninitialized warning

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agofix #2748: make order of interfaces consistent
Mira Limbeck [Thu, 28 May 2020 14:48:36 +0000 (16:48 +0200)]
fix #2748: make order of interfaces consistent

As perl hashes have random order, sort them before iterating through.
This makes the output of 'qm cloudinit dump <vmid> network' consistent
between calls if the config has not changed.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
3 years agobump version to 6.2-2
Thomas Lamprecht [Fri, 8 May 2020 11:00:26 +0000 (13:00 +0200)]
bump version to 6.2-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agofix #2697: map netdev_add options to correct json types
Dominik Csapak [Tue, 21 Apr 2020 14:01:11 +0000 (16:01 +0200)]
fix #2697: map netdev_add options to correct json types

netdev_add is now a proper qmp command, which means that it verifies
the parameter types properly

instead of sending strings, we now have to choose the correct
types for the parameters

bool for vhost
and uint64 for queues

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agobump version to 6.2-1
Thomas Lamprecht [Thu, 7 May 2020 19:51:05 +0000 (21:51 +0200)]
bump version to 6.2-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoqmrestore: fix restore from STDIN
Fabian Grünbichler [Thu, 7 May 2020 07:15:33 +0000 (09:15 +0200)]
qmrestore: fix restore from STDIN

the special case was dropped when moving this to pve-storage.

fixes commit c6d517835abfe345c31cd889b557529d334dcf04

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.1-20
Thomas Lamprecht [Wed, 6 May 2020 15:17:03 +0000 (17:17 +0200)]
bump version to 6.1-20

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: allow listing custom and default CPU models
Stefan Reiter [Mon, 4 May 2020 10:58:39 +0000 (12:58 +0200)]
api: allow listing custom and default CPU models

More API calls will follow for this path, for now add the 'index' call to
list all custom and default CPU models.

Any user can list the default CPU models, as these are public anyway, but
custom models are restricted to users with Sys.Audit on /nodes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agoapi: check Sys.Audit permissions when setting a custom CPU model
Stefan Reiter [Mon, 4 May 2020 10:58:38 +0000 (12:58 +0200)]
api: check Sys.Audit permissions when setting a custom CPU model

Explicitly allows changing other properties than the cputype, even if
the currently set cputype is not accessible by the user. This way, an
administrator can assign a custom CPU type to a VM for a less privileged
user without breaking edit functionality for them.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agovzdump: use new 'pbs' option
Fabian Grünbichler [Wed, 6 May 2020 08:57:53 +0000 (10:57 +0200)]
vzdump: use new 'pbs' option

instead of storage config to determine whether we are in 'PBS mode'

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agomigrate: cleanup forwarding code
Fabian Grünbichler [Tue, 5 May 2020 11:07:24 +0000 (13:07 +0200)]
migrate: cleanup forwarding code

fixing the following two issues:
- the legacy code path was never converted to the new fork_tunnel
signature (which probably means that nothing triggers it in practice
anymore?)
- the NBD Unix socket got forwarded multiple times if more than one disk
was migrated via NBD (this is harmless, but wrong)

for the second issue I opted to keep the code compatible with the
possibility that Qemu starts supporting multiple NBD servers in the
future (and the target node could thus return multiple UNIX socket
paths). currently we can only start one NBD server on one socket, and
each drive-mirror simply starts a new connection over that single
socket.

I took the liberty of renaming the variables/keys since I found
'tunnel_addr' and 'sock_addr' rather confusing.

Reviewed-By: Mira Limbeck <m.limbeck@proxmox.com>
Tested-By: Mira Limbeck <m.limbeck@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agocfg2cmd: set audiodev parameter only on qemu >= 4.2
Aaron Lauterer [Wed, 6 May 2020 12:51:54 +0000 (14:51 +0200)]
cfg2cmd: set audiodev parameter only on qemu >= 4.2

fixes commit 940e2a3a06b3ea47aae144519e2aaa881a80e437

QEMU 4.1 will fail to start a guest with an audio device set with:
> Property '.audiodev' not found

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agorng: die when trying to pass through disconnected hwrng
Stefan Reiter [Tue, 5 May 2020 14:53:12 +0000 (16:53 +0200)]
rng: die when trying to pass through disconnected hwrng

If /dev/hwrng exists, but no actual generator is connected (or it is
disabled on the host), QEMU will happily start the VM but crash as soon
as the guest accesses the VirtIO RNG device.

To prevent this unfortunate behaviour, check if a useable hwrng is
connected to the host before allowing the VM to be started.

While at it, clean up config_to_command by moving new and existing rng
source checks to a seperate sub.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agovzdump: fix template backup to stdout
Fabian Grünbichler [Wed, 6 May 2020 09:56:33 +0000 (11:56 +0200)]
vzdump: fix template backup to stdout

redirecting to the saved STDOUT in case of a template backup or a VM
without any disks failed because of the erroneous '=':

Backup of VM 123123 failed - command '/usr/bin/vma create -v -c [...]' failed:
Bad filehandle: =5 at /usr/share/perl/5.28/IPC/Open3.pm line 58.

https://forum.proxmox.com/threads/vzdump-to-stdout.69364

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agocfg2cmd: fix uninitialized value warning on OVMF w/o efidisk
Stefan Reiter [Wed, 6 May 2020 12:17:52 +0000 (14:17 +0200)]
cfg2cmd: fix uninitialized value warning on OVMF w/o efidisk

It's possible to have a VM with OVMF but without an efidisk, so don't
call parse_drive on a potential undef value.

Partial revert of 818c3b8d91 ("cfg2cmd: ovmf: code cleanup")

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agobump version to 6.1-18
Fabian Grünbichler [Tue, 5 May 2020 09:22:24 +0000 (11:22 +0200)]
bump version to 6.1-18

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agod/control: depend on new libpve-guest-common-perl
Fabian Grünbichler [Tue, 5 May 2020 09:20:07 +0000 (11:20 +0200)]
d/control: depend on new libpve-guest-common-perl

with lock_config_shared

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoclone: use new config_lock_shared
Fabian Grünbichler [Tue, 5 May 2020 08:27:23 +0000 (10:27 +0200)]
clone: use new config_lock_shared

and move the lock call and decision logic closer together

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Tested-by: Fabian Ebner <f.ebner@proxmox.com>
3 years agobump version to 6.1-18
Thomas Lamprecht [Mon, 4 May 2020 15:36:49 +0000 (17:36 +0200)]
bump version to 6.1-18

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoRevert "resize_vm: request new size from storage after resizing"
Fabian Ebner [Wed, 4 Mar 2020 09:51:12 +0000 (10:51 +0100)]
Revert "resize_vm: request new size from storage after resizing"

This reverts commit b5490d8a98e5e7328eb4cebb0ae0b60e6d406c38.

When resizing a volume of a running VM, a qmp block_resize command
is issued. This is non-blocking, so the size on the storage immediately
after issuing the command might still be the old one.

This is part of the issue reported in bug #2621.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 years agomigrate: skip rescan for efidisk and shared volumes
Dominik Csapak [Thu, 30 Apr 2020 10:44:48 +0000 (12:44 +0200)]
migrate: skip rescan for efidisk and shared volumes

we really only want to rescan the disk size of the disks we actually
need, and that are only the local disks (for which we have to allocate
the correct size on the target)

also we want to always skip the efidisk, since we get the wanted
size after the loop, and this produced a confusing log line
(for details why we do not want the 'real' size,
see commit 818ce80ec1a89c4abee61145c858b9323180e31b)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agomigrate: don't accidentally take NBD code paths
Fabian Grünbichler [Thu, 30 Apr 2020 07:35:29 +0000 (09:35 +0200)]
migrate: don't accidentally take NBD code paths

by avoiding auto-vivification of $self->{online_local_volumes} via
iteration. most code paths don't care whether it's undef or a reference
to an empty list, but this caused the (already) fixed bug of calling
nbd_stop without having started an NBD server in the first place.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoapi/destroy: repeat early checks after lock
Fabian Grünbichler [Mon, 27 Apr 2020 08:24:26 +0000 (10:24 +0200)]
api/destroy: repeat early checks after lock

to protect against concurrent changes

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoQemuServer: drop unused imported locking functions
Fabian Grünbichler [Mon, 27 Apr 2020 08:24:23 +0000 (10:24 +0200)]
QemuServer: drop unused imported locking functions

lock_file is used by PVE::QemuServer::Memory, but it does properly 'use
PVE::Tools ...' itself so we can drop them in the main module.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agovzdump: assemble: fix skipping all pending and snapshot config entries
Thomas Lamprecht [Mon, 4 May 2020 15:29:09 +0000 (17:29 +0200)]
vzdump: assemble: fix skipping all pending and snapshot config entries

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.1-17
Fabian Grünbichler [Mon, 4 May 2020 12:12:21 +0000 (14:12 +0200)]
bump version to 6.1-17

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agod/control: bump build-depend on pve-storage
Fabian Grünbichler [Mon, 4 May 2020 12:09:36 +0000 (14:09 +0200)]
d/control: bump build-depend on pve-storage

with refactored archive/compressor info and RE

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoFix #2124: Add support for zstd
Alwin Antreich [Tue, 28 Apr 2020 13:58:14 +0000 (15:58 +0200)]
Fix #2124: Add support for zstd

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
3 years agorestore: replace archive format/compression
Alwin Antreich [Tue, 28 Apr 2020 13:58:12 +0000 (15:58 +0200)]
restore: replace archive format/compression

regex to reduce the code duplication, as archive_info and
decompressor_info provides the same information as well.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
3 years agobackup: never try to freeze in stop mode backup
Thomas Lamprecht [Sun, 3 May 2020 09:13:36 +0000 (11:13 +0200)]
backup: never try to freeze in stop mode backup

VM was can be true for stop mode backup, we cannot check the "is VM
currently running" as that doesn't tells us anything (could be the
backup process), so check the mode also..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years ago_do_vm_stop: only warn if QMP command failed but VM is actually stopped
Thomas Lamprecht [Sun, 3 May 2020 09:12:54 +0000 (11:12 +0200)]
_do_vm_stop: only warn if QMP command failed but VM is actually stopped

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.1-16
Thomas Lamprecht [Wed, 29 Apr 2020 14:23:26 +0000 (16:23 +0200)]
bump version to 6.1-16

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoqm nbdstop: cope graceful with errors
Thomas Lamprecht [Wed, 29 Apr 2020 14:19:41 +0000 (16:19 +0200)]
qm nbdstop: cope graceful with errors

as the nbd server could have been stopped by something else.
Further, it makes no sense to die and mark the migration thus as
failed, just because of a NBD server stop issue.

At this point the migration hand off to the target was done already,
so normally we're good, if it fails we have other (followup) problems
anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agomigrate: only stop NBD if we got a NBD url from the target
Thomas Lamprecht [Wed, 29 Apr 2020 14:18:55 +0000 (16:18 +0200)]
migrate: only stop NBD if we got a NBD url from the target

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocleanup: get rid of unnecessary closures
Fabian Ebner [Wed, 29 Apr 2020 11:34:08 +0000 (13:34 +0200)]
cleanup: get rid of unnecessary closures

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agobump version to 6.1-15
Thomas Lamprecht [Wed, 29 Apr 2020 11:37:40 +0000 (13:37 +0200)]
bump version to 6.1-15

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoInclude vmstate and unused volumes in foreach_volid
Fabian Ebner [Thu, 16 Apr 2020 12:54:43 +0000 (14:54 +0200)]
Include vmstate and unused volumes in foreach_volid

and refactor the test_volid closure. Like this get_replicatable_volumes doesn't
need a separate loop for unused volumes anymore. For get_vm_volumes, which is used
for activation/deactivation of volumes at migration and deactivation in vm_stop_cleanup,
includes those volumes now. For migration it's an improvement, because those volumes
might need to be migrated and for vm_stop_cleanup it shouldn't hurt. The last user
of foreach_volid is check_vm_disks_local used by migrate_vm_precondition,
where information about the additional volumes doesn't hurt either.

Note that replicate is (still) set by default, so the behavior for
get_replicatable_volumes for unused volumes should not change.

Hibernation vmstate files are now also included and recognized as 'is_vmstate'.
The 'size' attribute will not be overwritten by subsequent iterations for the
same volid anymore (a volid may appear both in the config and in snapshots),
so the size from the current config is now preferred.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoFix test_volid call for vmstate and fix check for snapshots on migration
Fabian Ebner [Thu, 16 Apr 2020 12:54:42 +0000 (14:54 +0200)]
Fix test_volid call for vmstate and fix check for snapshots on migration

by excluding vmstate. It is referenced by snapshots, but
is not a volume containing a snapshot.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agorestore: small code cleanup for parse_backup_hints
Thomas Lamprecht [Wed, 29 Apr 2020 09:05:19 +0000 (11:05 +0200)]
restore: small code cleanup for parse_backup_hints

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agorestore: use correct storage for format check for cloudinit drives
Dominik Csapak [Tue, 28 Apr 2020 12:52:29 +0000 (14:52 +0200)]
restore: use correct storage for format check for cloudinit drives

when a backup includes a cloudinit disk on a non-existent storage,
the restore fails with 'storage' does not exist

this happens because we want to get the format of the disk, by
checking the source storage

we fix this by using the target storage first and only the source as
fallback

this will still fail if neither storage exists
(which is ok, since we cannot restore then anyway)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agoimportovf: die with error when disk file is missing
Dominik Csapak [Thu, 23 Apr 2020 10:46:49 +0000 (12:46 +0200)]
importovf: die with error when disk file is missing

Also add missing '\n' at the end of error messages

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agoimportovf: fix import of ovfs without default namespaces
Dominik Csapak [Thu, 23 Apr 2020 10:46:48 +0000 (12:46 +0200)]
importovf: fix import of ovfs without default namespaces

Some OVF files to not declare 'rasd' as a default namespace (in the
top level Envelope element), but inline in each element (e.g.
<rasd:HostResource xmlns:rasd="foo">...</rasd:HostResource>)

This trips up our relative findvalue with
> XPath error : Undefined namespace prefix

To avoid this, search in the global XPathContext (where we register
those namespaces ourselves) and pass the item_node as context
parameter.

This works then for both cases

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agotest: add test for OVF with missing default rasd namespace
Dominik Csapak [Thu, 23 Apr 2020 10:46:47 +0000 (12:46 +0200)]
test: add test for OVF with missing default rasd namespace

sometimes vendors do not put the 'rasd' namespaces in the top level
Envelope, but in every 'rasd' element this adds a test for this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agotest: print more info when OVF parsing fails
Dominik Csapak [Thu, 23 Apr 2020 10:46:46 +0000 (12:46 +0200)]
test: print more info when OVF parsing fails

When one of the ovf tests fails to parse at all, we just get the
'die' message of the failing component, but not which file actually
failed to parse.

To get better output, convert the parsing also to a test and ok() and
fail() respectively and then printing the error.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agoapi/resume: make nocheck root-only
Fabian Grünbichler [Mon, 27 Apr 2020 07:19:13 +0000 (09:19 +0200)]
api/resume: make nocheck root-only

this is only used for migration via 'qm mtunnel', regular users should
never need to resume a VM that does not logically belong to the node it
is running on

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agomigrate: simplify replicated_volume loop
Fabian Grünbichler [Mon, 20 Apr 2020 09:24:05 +0000 (11:24 +0200)]
migrate: simplify replicated_volume loop

(no change compared to previous iteration except for readability)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agoFix live migration with replicated unused volumes
Fabian Ebner [Mon, 20 Apr 2020 08:31:55 +0000 (10:31 +0200)]
Fix live migration with replicated unused volumes

by counting only local volumes that will be live-migrated via qemu_drive_mirror,
i.e. those listed in $self->{online_local_volumes}.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agomigrate: workaround issues with format switch on storage live migration
Thomas Lamprecht [Fri, 17 Apr 2020 13:27:38 +0000 (15:27 +0200)]
migrate: workaround issues with format switch on storage live migration

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocfg2cmd: factor out audio device generation
Thomas Lamprecht [Fri, 17 Apr 2020 13:20:05 +0000 (15:20 +0200)]
cfg2cmd: factor out audio device generation

and version guard it with 4.0 machines, just do be sure

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocfg2cmd: Add the audiodev property to audio devs
Aaron Lauterer [Mon, 16 Mar 2020 10:02:38 +0000 (11:02 +0100)]
cfg2cmd: Add the audiodev property to audio devs

With Qemu 4.2 a new `audiodev` property was introduced [0] to explicitly
specify the backend to be used for the audio device. This is accompanied
with a warning that the fallback to the default audio backend is
deprecated.

[0] https://wiki.qemu.org/ChangeLog/4.2#Audio

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
4 years agocfg2cmd: ovmf: code cleanup
Thomas Lamprecht [Wed, 15 Apr 2020 16:30:53 +0000 (18:30 +0200)]
cfg2cmd: ovmf: code cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-14
Fabian Grünbichler [Thu, 9 Apr 2020 06:58:37 +0000 (08:58 +0200)]
bump version to 6.1-14

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agod/control: depend on updated guest-common/pve-storage
Fabian Grünbichler [Thu, 9 Apr 2020 06:54:42 +0000 (08:54 +0200)]
d/control: depend on updated guest-common/pve-storage

pve-storage: new storage_migrate with allow_rename
guest-common: update_volume_ids

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agosync_disks: log output of storage_migrate
Fabian Ebner [Wed, 8 Apr 2020 09:25:12 +0000 (11:25 +0200)]
sync_disks: log output of storage_migrate

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agosync_disks: be more verbose if storage_migrate fails
Fabian Ebner [Wed, 8 Apr 2020 09:25:11 +0000 (11:25 +0200)]
sync_disks: be more verbose if storage_migrate fails

If storage_migrate dies, the error message might not include the
volume ID or the target storage ID, but those might be good to know.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agosync_disks: use allow_rename to avoid collisions on the target storage
Fabian Ebner [Wed, 8 Apr 2020 09:25:10 +0000 (11:25 +0200)]
sync_disks: use allow_rename to avoid collisions on the target storage

This makes it possible to migrate a VM with volumes store1:vm-123-disk-0
store2:vm-123-disk-0 to some targetstorage. Also prevents migration failure
when there is an orphaned disk with the same volid on the target.

To avoid confusion, the name should not change for 'vmstate'-volumes.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoUpdate volume IDs in one go
Fabian Ebner [Wed, 8 Apr 2020 09:25:09 +0000 (11:25 +0200)]
Update volume IDs in one go

Use 'update_volume_ids' for the live-migrated disks as well.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoAllow specifying targetstorage for offline migration
Fabian Ebner [Wed, 8 Apr 2020 09:25:08 +0000 (11:25 +0200)]
Allow specifying targetstorage for offline migration

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoTake note of changes to the volume IDs when migrating and update the config
Fabian Ebner [Wed, 8 Apr 2020 09:25:07 +0000 (11:25 +0200)]
Take note of changes to the volume IDs when migrating and update the config

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoAllow parsing vmstate entries
Fabian Ebner [Wed, 8 Apr 2020 09:25:06 +0000 (11:25 +0200)]
Allow parsing vmstate entries

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoUse new storage_migrate interface
Fabian Ebner [Wed, 8 Apr 2020 09:25:01 +0000 (11:25 +0200)]
Use new storage_migrate interface

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoSwitch to using foreach_volume instead of foreach_drive
Fabian Ebner [Wed, 8 Apr 2020 09:24:56 +0000 (11:24 +0200)]
Switch to using foreach_volume instead of foreach_drive

It was necessary to move foreach_volid back to QemuServer.pm

In VZDump/QemuServer.pm and QemuMigrate.pm the dependency on
QemuConfig.pm was already there, just the explicit "use" was missing.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agobump version to 6.1-13
Thomas Lamprecht [Wed, 8 Apr 2020 15:13:42 +0000 (17:13 +0200)]
bump version to 6.1-13

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoconfig_to_command: indentation, whitspace and newline fixes
Thomas Lamprecht [Wed, 8 Apr 2020 15:44:16 +0000 (17:44 +0200)]
config_to_command: indentation, whitspace and newline fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>