]> git.proxmox.com Git - qemu-server.git/log
qemu-server.git
4 years agoversion_guard scsi drive count
Stefan Reiter [Mon, 10 Feb 2020 15:05:36 +0000 (16:05 +0100)]
version_guard scsi drive count

Live-migrating a VM with more than 14 SCSI disks to a node that doesn't
support it yet is broken. Use a bumped pve-version to represent that and
give the user a nice error message instead.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoUse 'QEMU version' -> '+pve-version' mapping for machine types
Stefan Reiter [Mon, 10 Feb 2020 15:05:35 +0000 (16:05 +0100)]
Use 'QEMU version' -> '+pve-version' mapping for machine types

The previously introduced approach can fail for pinned versions when a
new QEMU release is introduced. The saner approach is to use a mapping
that gives one pve-version for each QEMU release.

Fortunately, the old system has not been bumped yet, so we can still
change it without too much effort.

QEMU versions without a mapping are assumed to be pve0, 4.1 is mapped to
pve1 since thats what we had as our default previously.

Pinned machine versions (i.e. pc-i440fx-4.1) are always assumed to be
pve0, for specific pve-versions they'd have to be pinned as well (i.e.
pc-i440fx-4.1+pve1).

The new logic also makes the pve-version dynamic, and starts VMs with
the lowest possible 'feature-level', i.e. if a feature is only available
with 4.1+pve2, but the VM isn't using it, we still start it with
4.1+pve0.

We die if we don't support a version that is requested from us. This
allows us to use the pve-version as live-migration blocks (i.e. bumping
the version and then live-migrating a VM which uses the new feature (so
is running with the bumped version) to an outdated node will present the
user with a helpful error message and fail instead of silently modifying
the config and only failing *after* the migration).

$version_guard is introduced in config_to_command to use for features
that need to check pve-version, it automatically handles selecting the
newest necessary pve-version for the VM.

Tests have to be adjusted, since all of them now resolve to pve0 instead
of pve1. EXPECT_ERROR matching is changed to use 'eq' instead of regex
to allow special characters in error messages.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agobump version to 6.1-6
Thomas Lamprecht [Mon, 10 Feb 2020 05:40:45 +0000 (06:40 +0100)]
bump version to 6.1-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoresize_vm: request new size from storage after resizing
Fabian Ebner [Mon, 13 Jan 2020 10:47:43 +0000 (11:47 +0100)]
resize_vm: request new size from storage after resizing

Because of alignment and rounding in the storage backend, the effective
size might not match the 'newsize' parameter we passed along.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoqmp: use query-cpus-fast for hotplug
Stefan Reiter [Thu, 6 Feb 2020 09:53:55 +0000 (10:53 +0100)]
qmp: use query-cpus-fast for hotplug

query-cpus has been deprecated since 2.12.0 [0] in favor of
query-cpus-fast, which no longer incurs a guest performance penalty on
the guest. The returned information is the same as far as our use case
is concerned.

[0] https://qemu.weilnetz.de/doc/qemu-doc.html#Deprecated-features

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoqmp: use migrate-set-parameters in favor of deprecated options
Stefan Reiter [Thu, 6 Feb 2020 09:53:54 +0000 (10:53 +0100)]
qmp: use migrate-set-parameters in favor of deprecated options

migrate_set_downtime, migrate_set_speed and migrate-set-cachesize have
all been deprecated since 2.8 or 2.11 [0]. They still work, but no
reason not to use the correct version.

Note that the downtime-limit parameter switched from seconds to
milliseconds, so convert to that. Slightly improve log output with units
while at it.

[0] https://qemu.weilnetz.de/doc/qemu-doc.html#Deprecated-features

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoqmp: use 'id' parameter instead of 'device'
Stefan Reiter [Thu, 6 Feb 2020 09:53:53 +0000 (10:53 +0100)]
qmp: use 'id' parameter instead of 'device'

'device' is deprecated since 2.8 in favor of 'id' [0], but since we
always consistently set the id on our drives anyway we can substitute it
easily.

[0] see files qapi/block.json and qapi/block-core.json in QEMU source
code, the online documentation doesn't mention it AFAICT

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoqmp: use 'blockdev-change-medium' instead of generic 'change'
Stefan Reiter [Thu, 6 Feb 2020 09:53:52 +0000 (10:53 +0100)]
qmp: use 'blockdev-change-medium' instead of generic 'change'

...and cleanup surrounding code a bit.

'change' is deprecated, and according to the qapi definition in QEMU it
is 'strongly recommended' to avoid using it.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoRemove unused $force parameter
Stefan Reiter [Thu, 6 Feb 2020 09:53:51 +0000 (10:53 +0100)]
Remove unused $force parameter

The fixme got it right :)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agomigrate: re-order lines to improve readability
Fabian Grünbichler [Wed, 5 Feb 2020 08:43:09 +0000 (09:43 +0100)]
migrate: re-order lines to improve readability

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agoExtract volume ID before calling 'parse_volume_id'
Fabian Ebner [Wed, 29 Jan 2020 13:30:09 +0000 (14:30 +0100)]
Extract volume ID before calling 'parse_volume_id'

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agorename 'volid' to 'drivestr' where it's not only a volume ID
Fabian Ebner [Wed, 29 Jan 2020 13:30:08 +0000 (14:30 +0100)]
rename 'volid' to 'drivestr' where it's not only a volume ID

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoFix description for vm_config and change description for vm_pending
Fabian Ebner [Tue, 4 Feb 2020 08:41:00 +0000 (09:41 +0100)]
Fix description for vm_config and change description for vm_pending

The description for vm_config was out of date and from the description
for vm_pending it was hard to tell what the difference to vm_config was.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agofixup: re-add target check
Fabian Grünbichler [Wed, 5 Feb 2020 07:35:14 +0000 (08:35 +0100)]
fixup: re-add target check

that accidentally got dropped while re-factoring previous commit

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agofix #2578: check if $target is provided in clone
Oguz Bektas [Mon, 3 Feb 2020 15:23:55 +0000 (16:23 +0100)]
fix #2578: check if $target is provided in clone

regression introduced with commit a85ff91b

previously we set $target to undef if it's localnode or localhost, then
we check if node exists.

with regression commit, behaviour changes as we do the node check in
else, but $target may be undef. this causes an error:

    no such cluster node ''

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
improved readability

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agofix #2566: increase scsi limit to 31
Dominik Csapak [Fri, 31 Jan 2020 14:41:22 +0000 (15:41 +0100)]
fix #2566: increase scsi limit to 31

to achieve this we have to add 3 new scsihw addresses since lsi
controllers can only hold 7 scsi drives

we go up to 31, since this is the limit for virtio-scsi-single devices
we have reserved (we can increase this in the future)

to make it more future proof, we add a new pci bridge under pci
bridge 1, so we have to adapt the bridge adding code (we did not
need this for q35 previously)

impact on live migration:
since on older versions of qemu-server we do not have those config
settings, there is no problem from old -> new

new->old is not supported anyway and this breaks so that
the vm crashes and loses the configs for scsi15-30
(same behaviour as e.g. with audio0 and migration from new->old)

tested with 31 scsi disk on
i440fx + virtio-scsi
i440fx + lsi
q35 + virtio-scsi
q35 + lsi
with ovmf + seabios

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agomove the vmgenid device after readconfig on q35
Dominik Csapak [Fri, 31 Jan 2020 14:41:21 +0000 (15:41 +0100)]
move the vmgenid device after readconfig on q35

and adapt the tests

this does not impact live migration, since the order here does not
change the device layout

we want this to consistently have the readconfig first

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agoallow reading snapshot config for VM.Audit
Dominik Csapak [Fri, 31 Jan 2020 10:40:37 +0000 (11:40 +0100)]
allow reading snapshot config for VM.Audit

VM.Audit can see the current config and the list of snapshots
already, so there is no real reason to disallow
the config of snapshots

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agobump version to 6.1-5
Thomas Lamprecht [Thu, 30 Jan 2020 09:30:22 +0000 (10:30 +0100)]
bump version to 6.1-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agohotplug_pending: make 'ssd' option non-hotpluggable
Oguz Bektas [Thu, 16 Jan 2020 15:06:34 +0000 (16:06 +0100)]
hotplug_pending: make 'ssd' option non-hotpluggable

from hotplug_pending we go into 'vmconfig_update_disk', where we check the
hotpluggability of options.

add 'ssd' there as a non-hotpluggable option (since we'd have to unplug/plug to
change the drive type)

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
4 years agoAdd CPUConfig file and migrate some helpers
Stefan Reiter [Thu, 16 Jan 2020 15:40:48 +0000 (16:40 +0100)]
Add CPUConfig file and migrate some helpers

The package will be used for custom CPU models as a SectionConfig, hence
the name. For now we simply move some CPU related helper functions and
declarations over from QemuServer to reduce clutter there.

Exports are to avoid changing all call sites, functions have useful
names on their own.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoUse 'volname' instead of 'volid' for 'qemu_img_format'
Fabian Ebner [Mon, 20 Jan 2020 13:00:44 +0000 (14:00 +0100)]
Use 'volname' instead of 'volid' for 'qemu_img_format'

As 'qemu_img_format' just matches a regex, this doesn't make much of
a difference, but AFAICT all other calls of 'qemu_img_format' use 'volname'.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agoFix 2070: vm_start: for a migrating VM, use current format of disk if possible
Fabian Ebner [Mon, 20 Jan 2020 13:00:43 +0000 (14:00 +0100)]
Fix 2070: vm_start: for a migrating VM, use current format of disk if possible

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agovmconfig_apply_pending: remove redundant write/load config calls
Oguz Bektas [Wed, 15 Jan 2020 13:48:59 +0000 (14:48 +0100)]
vmconfig_apply_pending: remove redundant write/load config calls

since we handle errors gracefully now, we don't need to write & save
config every time we change a setting.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
4 years agoqm start: document timeout default
Thomas Lamprecht [Wed, 15 Jan 2020 16:40:12 +0000 (17:40 +0100)]
qm start: document timeout default

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoadd timeout parameter to vm_start api endpoint
Tim Marx [Tue, 14 Jan 2020 13:30:37 +0000 (14:30 +0100)]
add timeout parameter to vm_start api endpoint

Signed-off-by: Tim Marx <t.marx@proxmox.com>
4 years agoadd new helper to calculate timeout based on vm config
Tim Marx [Tue, 14 Jan 2020 13:30:36 +0000 (14:30 +0100)]
add new helper to calculate timeout based on vm config

Signed-off-by: Tim Marx <t.marx@proxmox.com>
4 years agoFixup CPU flag query to not use get_basic_machine_info
Stefan Reiter [Wed, 15 Jan 2020 14:36:54 +0000 (15:36 +0100)]
Fixup CPU flag query to not use get_basic_machine_info

get_basic_machine_info was removed by commit 045749f2fc.

Use get_host_arch to get the default machine type instead, and
optionally allow to specify architecture as parameter.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoapi: vm clone: unlink zombie target config at end of error cleanup
Thomas Lamprecht [Wed, 15 Jan 2020 07:12:22 +0000 (08:12 +0100)]
api: vm clone: unlink zombie target config at end of error cleanup

This is the guarantee that this call operates on it's created config.
A VMID cannot be reused afterall. So only remove the guarantee at the
last step, just before throwing up the error message about the clone
failure.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoapi: vm clone: remove cloned FW conf in error cleanup path
Thomas Lamprecht [Wed, 15 Jan 2020 07:06:14 +0000 (08:06 +0100)]
api: vm clone: remove cloned FW conf in error cleanup path

We clone the source VM firewall config before forking the "realcmd"
worker, but did not mind cleaning it up again if the clone failed
somewhere in the worker.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoapi/qemu: remove some empty newlines and code cleanup
Thomas Lamprecht [Wed, 15 Jan 2020 07:02:06 +0000 (08:02 +0100)]
api/qemu: remove some empty newlines and code cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoAdd QEMU CPU flag querying helpers
Stefan Reiter [Thu, 21 Nov 2019 14:53:42 +0000 (15:53 +0100)]
Add QEMU CPU flag querying helpers

* query_understood_cpu_flags returns all flags that QEMU/KVM knows about
* query_supported_cpu_flags returns all flags that QEMU/KVM can use on
  this particular host.

To get supported flags, a temporary VM is started with QEMU, so we can
issue the "query-cpu-model-expansion" QMP command. This is how libvirt
queries supported flags for its "host-passthrough" CPU type.
query_supported_cpu_flags is thus rather slow and shouldn't be called
unnecessarily.

Note that KVM and TCG accelerators provide different expansions for the
"host" CPU type, so we need to query both.

Currently only supports x86_64, because QEMU-aarch64 doesn't provide the
necessary querying functions.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agovmconfig_apply_pending: add error handling
Oguz Bektas [Tue, 7 Jan 2020 15:55:18 +0000 (16:55 +0100)]
vmconfig_apply_pending: add error handling

wrap around code which can possibly fail in evals to handle them
gracefully, and log errors.

note: this results in a change of behavior in the API. since errors
are handled gracefully instead of "die"ing, when there is a pending
change which cannot be applied for some reason, it will get logged in
the tasklog but the vm will continue booting regardless. the
non-applied change will stay in the pending section of the
configuration.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoRemove unused 'sharedvm' variable
Fabian Ebner [Thu, 9 Jan 2020 09:23:07 +0000 (10:23 +0100)]
Remove unused 'sharedvm' variable

AFAICT this one hasn't been in use since commit
'4530494bf9f3d45c4a405c53ef3688e641f6bd8e'

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 years agohotplug_pending: remove redundant write/load config calls
Oguz Bektas [Fri, 13 Dec 2019 11:41:51 +0000 (12:41 +0100)]
hotplug_pending: remove redundant write/load config calls

instead of writing the config after every change, we can do it once for
all the changes in the end to avoid redundant i/o.

we also don't need to load_config after writing fastplug changes.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
4 years agobump version to 6.1-4
Thomas Lamprecht [Mon, 16 Dec 2019 15:03:30 +0000 (16:03 +0100)]
bump version to 6.1-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoapi/restore: do not trigger autostart task from locked context
Thomas Lamprecht [Mon, 16 Dec 2019 14:56:46 +0000 (15:56 +0100)]
api/restore: do not trigger autostart task from locked context

Do the same as for the "create" case, only trigger the "start after
create/restore" task after the locked "realcmd" was done. Else, the
start can never succeed, it also acquires a lock, but restore only
release it once outside of realcmd.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agogrammar fix: s/does not exists/does not exist/g
Thomas Lamprecht [Fri, 13 Dec 2019 11:11:58 +0000 (12:11 +0100)]
grammar fix: s/does not exists/does not exist/g

bump versioned build-dependency, as qemu-server has tests checking
for errors, and we fixed an grammar error in pve-storage, so we need
the newer version to ensure our test go through

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofollowup: simplify log callback
Thomas Lamprecht [Thu, 12 Dec 2019 12:38:55 +0000 (13:38 +0100)]
followup: simplify log callback

run_command only passes defined and chomped strings to the callback,
so no need to do that twice.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofix #2493: show QEMU errors in migration log
Stefan Reiter [Mon, 9 Dec 2019 15:14:09 +0000 (16:14 +0100)]
fix #2493: show QEMU errors in migration log

QEMU usually only prints warnings and errors and stays silent otherwise,
so it makes sense to just log all of it's output.

Prefix it with '[<target_hostname>]' to indicate that the output is
coming from the remote node, so users know where to search for the
error.

Side effect is that the 'VM start' task created by the migration will
now show the "QEMU:" prefix, but it's still very readable IMHO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agohide long commandline on vm_start/migrate failure
Stefan Reiter [Mon, 9 Dec 2019 15:14:08 +0000 (16:14 +0100)]
hide long commandline on vm_start/migrate failure

By default run_command prints the entire commandline executed when an
error occurs, but QEMU and our migrate command are not only
uninteresting to the user[*] but also annoyingly long. Hide them and only
print the exit code.

[*] Especially our migrate command, since it can't be manually executed
anyway. QEMU's commandline *might* contain something interesting, but is
so long that it's tricky to parse anyway, any a user can always call 'qm
showcmd --pretty'.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agoupdate disk size before local disk migration
Stefan Reiter [Mon, 9 Dec 2019 13:08:09 +0000 (14:08 +0100)]
update disk size before local disk migration

Split out 'update_disksize' from the renamed 'update_disk_config' to
allow code reuse in QemuMigrate.

Remove dots after messages to keep style consistent for migration log.

After updating in sync_disks (phase1) of migration, write out updated
config. This means that even if migration fails or is aborted in later
stages, we keep the fixed config - this is not an issue, as it would
have been fixed on the next attempt anyway, and it can't hurt to have
the correct size instead of a wrong one either way.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agomove vmstate check to check_vm_modify_config_perm
Dominik Csapak [Mon, 9 Dec 2019 14:27:00 +0000 (15:27 +0100)]
move vmstate check to check_vm_modify_config_perm

so that it gets checked before the special case of removing the vmstate

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agosuspend to disk: check more permissions
Dominik Csapak [Mon, 9 Dec 2019 14:26:59 +0000 (15:26 +0100)]
suspend to disk: check more permissions

only VM.PowerMgmt is not enough, since we allocate space on a storage,
so we need VM.Config.Disk on the vm and Datastore.AllocateSpace on the storage

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agorefactor finding of vmstate storage
Dominik Csapak [Mon, 9 Dec 2019 14:26:58 +0000 (15:26 +0100)]
refactor finding of vmstate storage

we need that on another place, so refactor in its own sub

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agofix wrong punctuation in error msg
Stefan Reiter [Mon, 9 Dec 2019 15:14:07 +0000 (16:14 +0100)]
fix wrong punctuation in error msg

...leading to ugly line info being printed.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agocfg2cmd: allow to test for expected error messages
Thomas Lamprecht [Tue, 10 Dec 2019 10:07:00 +0000 (11:07 +0100)]
cfg2cmd: allow to test for expected error messages

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoqemu-server: make nodename mock-able for tests
Thomas Lamprecht [Tue, 10 Dec 2019 10:05:39 +0000 (11:05 +0100)]
qemu-server: make nodename mock-able for tests

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofollowup: improve error message for outdated QEMU version
Thomas Lamprecht [Tue, 10 Dec 2019 10:06:17 +0000 (11:06 +0100)]
followup: improve error message for outdated QEMU version

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocheck if QEMU version is recent enough for machine type
Stefan Reiter [Mon, 9 Dec 2019 15:14:10 +0000 (16:14 +0100)]
check if QEMU version is recent enough for machine type

...and show the user a more meaningful error otherwise.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agobump version to 6.1-3
Thomas Lamprecht [Mon, 9 Dec 2019 09:34:43 +0000 (10:34 +0100)]
bump version to 6.1-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoremove $vmid param from print_drive
Stefan Reiter [Thu, 5 Dec 2019 15:11:01 +0000 (16:11 +0100)]
remove $vmid param from print_drive

It isn't used in the sub, but suggest it is needed. No users outside
qemu-server found.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agocfg2cmd: test runs_at_least_qemu_version and version_cmp explicitly
Stefan Reiter [Wed, 4 Dec 2019 12:15:51 +0000 (13:15 +0100)]
cfg2cmd: test runs_at_least_qemu_version and version_cmp explicitly

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agocfg2cmd: minor cleanup
Stefan Reiter [Wed, 4 Dec 2019 12:15:50 +0000 (13:15 +0100)]
cfg2cmd: minor cleanup

We never shipped a 4.1.0 QEMU, so it makes more sense to test as 4.1.1

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agofollowup: minor cleanup/error message spelling
Thomas Lamprecht [Mon, 9 Dec 2019 10:29:26 +0000 (11:29 +0100)]
followup: minor cleanup/error message spelling

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofix #2510: hostpci: always check if device exists
Dominik Csapak [Mon, 9 Dec 2019 09:10:04 +0000 (10:10 +0100)]
fix #2510: hostpci: always check if device exists

if the user set a device as hostpci with the 'shorthand' syntax:

hostpciX: 00:12

we ignored it on starting and showcmd and continued.
Since the user explicitly wanted to passthrough a device, we now check
if there is actually a device with that id

for explicitly configured devices (00:12.1), we did not check if it exists,
but the kvm call failed with a non-obvious error message

now we always call 'lspci' from SysFSTools to check if it actually exists,
and fail if not. With this, we can drop the workaround for adding
'0000' if no domain was given, since lspci does it already for us

this fixes #2510, an issue with using mediated devices where the users did not have
the domain in the config, since we forgot to add the default domain there

the only issue with this patch is that it changes the behaviour of
'showcmd' slightly, as in now, we die if the device was explicitly
given, but did not exists (we showed the commandline, now we fail)

this also slightly changes the commandline for qemu (adding always
the domain), which is not a problem since we cannot live migrate
or snapshot such vms, but we have to adapt the tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agocfg2cmd test: add tests for multifunction devices
Dominik Csapak [Mon, 9 Dec 2019 09:10:03 +0000 (10:10 +0100)]
cfg2cmd test: add tests for multifunction devices

by mocking the lspci call

the mocked lspci code is basically the same as the real one,
only difference is the source of the devices and
there is no verbose flag

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agoRevert "fix #2478: record python3-minimal as build dependency"
Thomas Lamprecht [Mon, 9 Dec 2019 09:32:46 +0000 (10:32 +0100)]
Revert "fix #2478: record python3-minimal as build dependency"

This reverts commit 951b1783cdd328990b46b47a60a457e5acc33cb0.

4 years agofix #2469: fix qemu-img convert src_format detection
Thomas Lamprecht [Mon, 9 Dec 2019 09:31:33 +0000 (10:31 +0100)]
fix #2469: fix qemu-img convert src_format detection

This reverts commit c5151cb8bb5f919c9e621065224c3d455b60f833 which is
a revert of the wrongly done revert of
commit e2414e73ce75893a143dc32cf6a4e58159cc25ff.

4 years agocreate_efidisk: poll the real size after volume creation
Thomas Lamprecht [Wed, 4 Dec 2019 17:40:31 +0000 (18:40 +0100)]
create_efidisk: poll the real size after volume creation

some storage backends have bigger granularity than the default 128k
size from the EFIVARS template file, so we actually need to poll the
real created disk size, as it will be used to create the target
volume for local storage migration on running VMs, if it's to small
the target will be to small and migration will fail.

Just a fix for newly created EFIDISKS, for others we need to rescan
the size after we've got the migrate lock and write the updated info
out, so that the target node has the correct one (protected from
migrate lock).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agorefactor storage whitelist in sync_disks to regex
Thomas Lamprecht [Wed, 4 Dec 2019 17:40:03 +0000 (18:40 +0100)]
refactor storage whitelist in sync_disks to regex

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-2
Thomas Lamprecht [Sat, 30 Nov 2019 17:38:50 +0000 (18:38 +0100)]
bump version to 6.1-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoapi: allow to clear vmstate actively
Thomas Lamprecht [Sat, 30 Nov 2019 17:17:17 +0000 (18:17 +0100)]
api: allow to clear vmstate actively

Sometimes, a user wants to remove the 'suspended' state without
resuming the vm from that state. Since the vm is locked with
'suspended', this was not possible without help from root@pam

This patch allows to delete the vmstate and the suspended lock and
related config entries with it. The user still has to have the right
priviliges and the vm cannot be 'protected' for this to work

Inspired-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoimplement removal of vmstate via api
Dominik Csapak [Fri, 29 Nov 2019 10:06:48 +0000 (11:06 +0100)]
implement removal of vmstate via api

we did not actually delete the state if we deleted the 'vmstate' config,
leaving stray vmstates on the disks

actually implement the removal, requiring 'VM.Config.Disk' and
'VM.PowerMgmt' privs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agodelete suspended lock when we have no vmstate
Dominik Csapak [Fri, 29 Nov 2019 10:06:47 +0000 (11:06 +0100)]
delete suspended lock when we have no vmstate

if a user removed the vmstate from the config for whatever reason,
a vmstart did not remove the 'suspended' lock
so always delete it and delete the vmstate only if it really was there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agologging vmstate into Resume/Start Task
Dominik Csapak [Fri, 29 Nov 2019 10:06:46 +0000 (11:06 +0100)]
logging vmstate into Resume/Start Task

so that the user knows what got loaded as vmstate

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agomigrate phase 3 cleanup: add error into error propagation message
Thomas Lamprecht [Sat, 30 Nov 2019 16:27:14 +0000 (17:27 +0100)]
migrate phase 3 cleanup: add error into error propagation message

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agomon_cmd: add explicit return
Thomas Lamprecht [Sat, 30 Nov 2019 16:24:55 +0000 (17:24 +0100)]
mon_cmd: add explicit return

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agovzdump: log QGA FS freeze/thaw try in task log
Thomas Lamprecht [Wed, 27 Nov 2019 19:18:15 +0000 (20:18 +0100)]
vzdump: log QGA FS freeze/thaw try in task log

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoskip efidisk0 in hotplug
Oguz Bektas [Tue, 26 Nov 2019 16:07:49 +0000 (17:07 +0100)]
skip efidisk0 in hotplug

while it's a disk from our storage POV, in QEMU it's a pflash, and
those cannot be hot-plugged

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocleanup build artificats before doc clean
Thomas Lamprecht [Tue, 26 Nov 2019 12:53:18 +0000 (13:53 +0100)]
cleanup build artificats before doc clean

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.1-1
Thomas Lamprecht [Tue, 26 Nov 2019 12:04:59 +0000 (13:04 +0100)]
bump version to 6.1-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoRevert "fix #2469: fix qemu-img convert src_format detection"
Thomas Lamprecht [Tue, 26 Nov 2019 12:05:54 +0000 (13:05 +0100)]
Revert "fix #2469: fix qemu-img convert src_format detection"

This reverts commit e2414e73ce75893a143dc32cf6a4e58159cc25ff.

4 years agofix #2478: record python3-minimal as build dependency
Thomas Lamprecht [Tue, 26 Nov 2019 10:37:18 +0000 (11:37 +0100)]
fix #2478: record python3-minimal as build dependency

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoimplement PVE Version addition for QEMU machine
Thomas Lamprecht [Mon, 25 Nov 2019 10:18:13 +0000 (11:18 +0100)]
implement PVE Version addition for QEMU machine

With our QEMU 4.1.1 package we can pass a additional internal version
to QEMU's machine, it will be split out there and ignored, but
returned on a QMP 'query-machines' call.

This allows us to use it for increasing the granularity with which we
can roll-out HW layout changes/additions for VMs. Until now we
required a machine version bump, happening normally every major
release of QEMU, with seldom, for us irrelevant, exceptions.
This often delays rolling out a feature, which would break
live-migration, by several months. That can now be avoided, the new
"pve-version" component of the machine can be bumped at will, and
thus we are much more flexible.

That versions orders after the ($major, $minor) version components
from an stable release - it can thus also be reset on the next
release.

The implementation extends the qemu-machine REGEX, remembers
"pve-version" when doing a "query-machines" and integrates support
into the min_version and extract_version helpers.

We start out with a version of 1.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agofix #2469: fix qemu-img convert src_format detection
Fabian Grünbichler [Mon, 25 Nov 2019 11:03:28 +0000 (12:03 +0100)]
fix #2469: fix qemu-img convert src_format detection

if we don't know which format the source volume/file has, let qemu-img
decide.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoversion_cmp: give info about caller on error
Thomas Lamprecht [Mon, 25 Nov 2019 10:16:38 +0000 (11:16 +0100)]
version_cmp: give info about caller on error

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoapi: use get_vm_arch instead of depreacted basic_machine_info
Thomas Lamprecht [Mon, 25 Nov 2019 09:51:32 +0000 (10:51 +0100)]
api: use get_vm_arch instead of depreacted basic_machine_info

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoremove wrong comment
Fabian Grünbichler [Mon, 25 Nov 2019 09:38:13 +0000 (10:38 +0100)]
remove wrong comment

the change to encode pending changes differently is not yet included in
qemu-server.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agoapi: catch 'pending' as snapshot name early
Fabian Grünbichler [Mon, 25 Nov 2019 09:02:46 +0000 (10:02 +0100)]
api: catch 'pending' as snapshot name early

instead of only at write_config time.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 years agofix #2367: do not allow snapshot with name PENDING
Oguz Bektas [Thu, 24 Oct 2019 11:53:09 +0000 (13:53 +0200)]
fix #2367: do not allow snapshot with name PENDING

or any other variant of the word 'pending'.

note that we can actually allow this snapshot after PVE 7.0, since
pending section and snapshots will be properly namespaced.
([pve:pending] and [snap:$snapname] or similar).

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
4 years agosplit get_basic_machine_info into get_vm_arch and get_vm_machine
Thomas Lamprecht [Mon, 25 Nov 2019 07:56:58 +0000 (08:56 +0100)]
split get_basic_machine_info into get_vm_arch and get_vm_machine

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agovm start use new pve-qemu-machine standard option
Thomas Lamprecht [Mon, 25 Nov 2019 07:54:07 +0000 (08:54 +0100)]
vm start use new pve-qemu-machine standard option

'pve-qm-machine' is auto-registered, but for re-use for a new
runningmachine we added the newer pve-qemu-machine standard option.
Use that one to avoid confusion.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocfg2cmd test: fix spice enhancement test
Thomas Lamprecht [Mon, 25 Nov 2019 06:45:44 +0000 (07:45 +0100)]
cfg2cmd test: fix spice enhancement test

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.0-17
Thomas Lamprecht [Sat, 23 Nov 2019 08:52:17 +0000 (09:52 +0100)]
bump version to 6.0-17

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoreally fallback to PCI standard domain "0000" again
Thomas Lamprecht [Sat, 23 Nov 2019 08:47:56 +0000 (09:47 +0100)]
really fallback to PCI standard domain "0000" again

fixes commit 70740c1a4d60b002249a648cc6588ecddb44ec70

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agobump version to 6.0-16
Thomas Lamprecht [Fri, 22 Nov 2019 13:19:01 +0000 (14:19 +0100)]
bump version to 6.0-16

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofixup: use correct version_cmp
Thomas Lamprecht [Fri, 22 Nov 2019 13:18:02 +0000 (14:18 +0100)]
fixup: use correct version_cmp

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofix #2473: use of unitialized value
Mira Limbeck [Thu, 21 Nov 2019 10:04:25 +0000 (11:04 +0100)]
fix #2473: use of unitialized value

With the noerr flag set in parse_volume_id we have to check if
$volname is defined before comparing it to 'cloudinit'.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
4 years agobump version to 6.0-15
Thomas Lamprecht [Wed, 20 Nov 2019 18:41:05 +0000 (19:41 +0100)]
bump version to 6.0-15

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agoadd 'tags' option
Dominik Csapak [Thu, 31 Oct 2019 12:36:25 +0000 (13:36 +0100)]
add 'tags' option

this is useful as meta information for e.g., provisioning or config
management systems

adding the info also to the 'status' api call to make it easier to show
it in the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agod/control: bump versioned dependency on pve-common
Thomas Lamprecht [Wed, 20 Nov 2019 18:08:33 +0000 (19:08 +0100)]
d/control: bump versioned dependency on pve-common

to ensure the sysfs PCI tools can cope with our new more flexible PCI
addr (domain != 0000) fix

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofix #2436: pci: do not hardcode pci domain to 0000
Dominik Csapak [Tue, 12 Nov 2019 13:23:03 +0000 (14:23 +0100)]
fix #2436: pci: do not hardcode pci domain to 0000

relax the regex for hostpci to allow different pci domains than 0000

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
4 years agod/control: break old ha-manager, pve-manager
Thomas Lamprecht [Wed, 20 Nov 2019 18:06:57 +0000 (19:06 +0100)]
d/control: break old ha-manager, pve-manager

due to the refactoring file/module movements we made

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agocleanup: improve locallity of variable declaration/use
Thomas Lamprecht [Wed, 20 Nov 2019 18:06:15 +0000 (19:06 +0100)]
cleanup: improve locallity of variable declaration/use

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agofollowup snapshot-test: pass class to cfs_config_path
Thomas Lamprecht [Wed, 20 Nov 2019 18:05:14 +0000 (19:05 +0100)]
followup snapshot-test: pass class to cfs_config_path

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agorefactor: split qemu_machine_feature_enabled
Stefan Reiter [Tue, 19 Nov 2019 11:23:49 +0000 (12:23 +0100)]
refactor: split qemu_machine_feature_enabled

...into:

* PVE::QemuServer::Helpers::min_version: check a major.minor version
  string with a given major/minor version (this is equivalent to calling
  the old qemu_machine_feature_enabled with only $kvmver)
* PVE::QemuServer::Machine::extract_version: get major.minor version
  string from arbitrary machine type (e.g. pc-q35-4.0, ...)
* PVE::QemuServer::Machine::machine_version: helper to call
  extract_version automatically before min_version

Includes a cfg2cmd test case with pinned machine version.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 years agorefactor: extract QEMU machine related helpers to package
Stefan Reiter [Tue, 19 Nov 2019 11:23:48 +0000 (12:23 +0100)]
refactor: extract QEMU machine related helpers to package

...PVE::QemuServer::Machine.

qemu_machine_feature_enabled is exported since it has a *lot* of users
in PVE::QemuServer and a long enough name as it is.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agorefactor: create QemuServer::Monitor for high-level QMP access
Stefan Reiter [Tue, 19 Nov 2019 11:23:47 +0000 (12:23 +0100)]
refactor: create QemuServer::Monitor for high-level QMP access

QMP and monitor helpers are moved from QemuServer.pm.

By using only vm_running_locally instead of check_running, a cyclic
dependency to QemuConfig is avoided. This also means that the $nocheck
parameter serves no more purpose, and has thus been removed along with
vm_mon_cmd_nocheck.

Care has been taken to avoid errors resulting from this, and
occasionally a manual check for a VM's existance inserted on the
callsite.

Methods have been renamed to avoid redundant naming:
* vm_qmp_command -> qmp_cmd
* vm_mon_cmd -> mon_cmd
* vm_human_monitor_command -> hmp_cmd

mon_cmd is exported since it has many users. This patch also changes all
non-package users of vm_qmp_command to use the mon_cmd helper. Includes
mocking for tests.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 years agorefactor: split check_running into _exists_ and _running_
Stefan Reiter [Tue, 19 Nov 2019 11:23:46 +0000 (12:23 +0100)]
refactor: split check_running into _exists_ and _running_

vm_exists_on_node in PVE::QemuConfig checks if a config file for a vmid
exists

vm_running_locally in PVE::QemuServer::Helpers checks if a VM is running
on the local machine by probing its pidfile and checking /proc/.../cmdline

check_running is left in QemuServer for compatibility, but changed to
simply call the two new helper functions.

Both methods are also correctly mocked for testing snapshots.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>