qemu-server.git
9 months agod/control: update dh version dependency and standard version
Thomas Lamprecht [Wed, 25 Sep 2019 12:58:05 +0000 (14:58 +0200)]
d/control: update dh version dependency and standard version

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agobuildsys: add dsc target
Thomas Lamprecht [Wed, 25 Sep 2019 12:50:45 +0000 (14:50 +0200)]
buildsys: add dsc target

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agobuildsys: use BUILDDIR variable
Thomas Lamprecht [Wed, 25 Sep 2019 12:47:26 +0000 (14:47 +0200)]
buildsys: use BUILDDIR variable

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agobuildsys: remove unused or duplicate directories
Thomas Lamprecht [Tue, 24 Sep 2019 17:04:12 +0000 (19:04 +0200)]
buildsys: remove unused or duplicate directories

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agoremove pcitest, now irrelevant
Thomas Lamprecht [Tue, 24 Sep 2019 16:14:14 +0000 (18:14 +0200)]
remove pcitest, now irrelevant

PCI pass through was greatly advanced since this got changed and is
all in QemuServer.

If anybody needs it, it's in git history anyway..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agomove VM network (hook) scripts to own directory
Thomas Lamprecht [Tue, 24 Sep 2019 16:12:25 +0000 (18:12 +0200)]
move VM network (hook) scripts to own directory

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agomove qemu-configs to own directory
Thomas Lamprecht [Tue, 24 Sep 2019 16:11:22 +0000 (18:11 +0200)]
move qemu-configs to own directory

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agomove qmeventd to own directory
Thomas Lamprecht [Tue, 24 Sep 2019 16:08:48 +0000 (18:08 +0200)]
move qmeventd to own directory

It's really not nice if such many files, source code, meta-files, …
linger around in the top level directory..

Also, cleanup the build a bit, i.e., use LDFLAGS as dpkg-buildpackage
can set some LDFLAGS so it'd be nice if both CFLAFGS and LDFLAGS have
the same (related) ones.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agobuildsys: use $(MAKE) instead of make
Thomas Lamprecht [Tue, 24 Sep 2019 16:06:16 +0000 (18:06 +0200)]
buildsys: use $(MAKE) instead of make

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agobuildsys: omit manual manpage gzip, done by debhelpers
Thomas Lamprecht [Tue, 24 Sep 2019 16:03:46 +0000 (18:03 +0200)]
buildsys: omit manual manpage gzip, done by debhelpers

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agofollowup: reorder error so that $volid is last (potential long)
Thomas Lamprecht [Mon, 23 Sep 2019 16:26:15 +0000 (18:26 +0200)]
followup: reorder error so that $volid is last (potential long)

and add "current" as keyword, further remove the parenthesis for the
post-if, to adapt to Proxmox general perl code style

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agoadd check to resize disk if current size could not be determined
Tim Marx [Thu, 12 Sep 2019 10:56:18 +0000 (12:56 +0200)]
add check to resize disk if current size could not be determined

This check ensures that disks aren't unintentionally shrunken, if the
size is zero due to an underlying problem.

9 months agofollowup: add newline in warn to avoid extra perl-internal info
Thomas Lamprecht [Mon, 23 Sep 2019 11:15:30 +0000 (13:15 +0200)]
followup: add newline in warn to avoid extra perl-internal info

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agoImprove handling and description of migrate with --online
Fabian Ebner [Mon, 23 Sep 2019 07:56:01 +0000 (09:56 +0200)]
Improve handling and description of migrate with --online

Thanks to Stefan and Thomas for the suggestions.

Changes from v1:
* update parameter description
* warn instead of die

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
9 months agoapi: config deletion: avoid regex on undefined values
Thomas Lamprecht [Mon, 23 Sep 2019 10:53:04 +0000 (12:53 +0200)]
api: config deletion: avoid regex on undefined values

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agoapi: deletion: check also pending values for serial/usb
Dominik Csapak [Mon, 23 Sep 2019 08:50:21 +0000 (10:50 +0200)]
api: deletion: check also pending values for serial/usb

otherwise a user with only VM.Config.HWType cannot
delete a 'pending' usbX: spice or serial: socket option

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
9 months agofollowup: fix usb3 description, SPICE is OK now
Thomas Lamprecht [Mon, 23 Sep 2019 08:25:01 +0000 (10:25 +0200)]
followup: fix usb3 description, SPICE is OK now

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agousb: Fix local resource check of Spice USB devices
Aaron Lauterer [Wed, 11 Sep 2019 12:43:33 +0000 (14:43 +0200)]
usb: Fix local resource check of Spice USB devices

The check relied on the fact that in a regular use case a usb device of
type spice would not have any other settings like `usb3=1`. An exact
match worked fine for this.

This patch changes the behaviour and makes it possible to migrate VMs
with Spice USB devices that have additional usb options set.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
9 months agocfg2cmd: mock SPICE/VNC port allocation methods
Thomas Lamprecht [Sat, 21 Sep 2019 11:41:06 +0000 (13:41 +0200)]
cfg2cmd: mock SPICE/VNC port allocation methods

to make them independent of environment, else other running VMs or
quickly sequential test runs may result in false negative test
results, as the port differed.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agocfg2cmd: add config test for SPICE with USB3
Thomas Lamprecht [Sat, 21 Sep 2019 11:35:01 +0000 (13:35 +0200)]
cfg2cmd: add config test for SPICE with USB3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
9 months agousb: Add USB3 capabilities to Spice USB devices
Aaron Lauterer [Wed, 11 Sep 2019 12:43:32 +0000 (14:43 +0200)]
usb: Add USB3 capabilities to Spice USB devices

To not change current behaviour and thus breaking live migration USB3
for a Spice USB device requires Qemu v4.1.

The old behavior was that even though technically it was possible to
the set `usb3=1` setting, it was ignored. The bus was hardcoded to
ehci. If another USB2 device was added or the machine type was set to
Q35 an ehci controller was present and the VM was able to boot.

With this patch the behaviour is changing and the bus is set to xhci if
USB3 is set for the Spice USB device and the VM is running under Qemu
v4.1.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
9 months agousb: Cleanup redundant if condition
Aaron Lauterer [Wed, 11 Sep 2019 12:43:31 +0000 (14:43 +0200)]
usb: Cleanup redundant if condition

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
9 months agoapi: add reboot api call
Dominik Csapak [Wed, 11 Sep 2019 12:07:45 +0000 (14:07 +0200)]
api: add reboot api call

this creates a reboot request file (inspired by pve-container)
and relies on the 'qm cleanup' call by the qmeventd to detect
and restart the vm afterwards

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
9 months agorefactor vm_stop locked code
Dominik Csapak [Wed, 11 Sep 2019 12:07:44 +0000 (14:07 +0200)]
refactor vm_stop locked code

we want to reuse most of the code in the locked context of vm_stop
for vm_reboot (since it really is just a vm_stop with a
create_reboot_request in there) so we factor that out into
_do_vm_stop
and note that it has to be called in a locked context

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
9 months agoqm: cleanup: detect and handle reboot trigger
Dominik Csapak [Wed, 11 Sep 2019 12:07:43 +0000 (14:07 +0200)]
qm: cleanup: detect and handle reboot trigger

if the reboot trigger file was set, start the vm again
also cleanup the reboot trigger on vm startup, to prevent
leftover files to trigger a reboot at the next shutdown

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
9 months agoadd create and remove reboot trigger subs
Dominik Csapak [Wed, 11 Sep 2019 12:07:42 +0000 (14:07 +0200)]
add create and remove reboot trigger subs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
10 months agoapi: add missing index child links
Dominik Csapak [Fri, 6 Sep 2019 12:24:05 +0000 (14:24 +0200)]
api: add missing index child links

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
10 months agotest: compare also with our qemu cfgs for pci addr conflicts
Thomas Lamprecht [Tue, 10 Sep 2019 08:31:38 +0000 (10:31 +0200)]
test: compare also with our qemu cfgs for pci addr conflicts

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agotest: pci addr checker: add slurp_qemu_config skeleton
Thomas Lamprecht [Mon, 9 Sep 2019 14:24:20 +0000 (16:24 +0200)]
test: pci addr checker: add slurp_qemu_config skeleton

it works™, but yet unused, add nonetheless to make it easier to pick
up and complete this testing work.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agocfg2cmd test: hostpci, also specify exact PCIe devices
Thomas Lamprecht [Mon, 9 Sep 2019 06:40:07 +0000 (08:40 +0200)]
cfg2cmd test: hostpci, also specify exact PCIe devices

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agocfg2cmd test: fix hostpci tests, specify exact PCI device
Thomas Lamprecht [Mon, 9 Sep 2019 06:26:45 +0000 (08:26 +0200)]
cfg2cmd test: fix hostpci tests, specify exact PCI device

as a hack to avoid that we hit a code path where the current ones
from the hosts are checked, and the command line option is added
depending if it exists or not.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agocfg2cmd tests: improve hostpci test by marking some as PCIe
Thomas Lamprecht [Fri, 6 Sep 2019 17:45:33 +0000 (19:45 +0200)]
cfg2cmd tests: improve hostpci test by marking some as PCIe

and add a linux based one, which acts a bit different

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agoconfig_to_command: refactor/cleanup try for hostpci part
Thomas Lamprecht [Fri, 6 Sep 2019 17:24:01 +0000 (19:24 +0200)]
config_to_command: refactor/cleanup try for hostpci part

try to increase variable declaration to use locality, reduce
duplication of same variable names used, and try to cleanup a bit in
general..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agocfg2cmd tests: config with PC i440fx, pci passthrough, NUMA, EFI
Thomas Lamprecht [Fri, 6 Sep 2019 17:26:54 +0000 (19:26 +0200)]
cfg2cmd tests: config with PC i440fx, pci passthrough, NUMA, EFI

identical with the Q35 one, solely the machine type is different

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agocfg2cmd tests: config with q35, pci passthrough, NUMA, EFI
Thomas Lamprecht [Fri, 6 Sep 2019 17:21:08 +0000 (19:21 +0200)]
cfg2cmd tests: config with q35, pci passthrough, NUMA, EFI

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agoPCIe passthrough: fixup: avoid addr conflict and cleanup a bit
Thomas Lamprecht [Fri, 6 Sep 2019 17:22:27 +0000 (19:22 +0200)]
PCIe passthrough: fixup: avoid addr conflict and cleanup a bit

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agopci: add conflict tests
Thomas Lamprecht [Fri, 6 Sep 2019 17:10:34 +0000 (19:10 +0200)]
pci: add conflict tests

best viewed with: git show -w

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agoAdd support for up to 16 PCI(e) devices
Aaron Lauterer [Thu, 5 Sep 2019 16:13:00 +0000 (18:13 +0200)]
Add support for up to 16 PCI(e) devices

For non pci express passthrough additional addresses are reserved.
For pcie passthrough pcie root ports are needed (unless guest is like
windows 7).

The first 4 pcie root ports are defined by default in the pve-q35*.cfg
files. If more than 4 pcie devices are passed through the needed root
ports are created on demand. This helps to keep live migration possible
without adding a new pve-q35*.cfg file.

For the windows 7 like guests additional addresses are reserved as well.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
10 months agoNewline cleanup
Christian Ebner [Mon, 1 Jul 2019 13:43:51 +0000 (15:43 +0200)]
Newline cleanup

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
10 months agospice enhancements: document default values
Thomas Lamprecht [Wed, 4 Sep 2019 12:44:04 +0000 (14:44 +0200)]
spice enhancements: document default values

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agofollowup: code cleanup
Thomas Lamprecht [Wed, 4 Sep 2019 11:26:11 +0000 (13:26 +0200)]
followup: code cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
10 months agofix #2041, #2272: Add Spice enhancements
Aaron Lauterer [Thu, 22 Aug 2019 15:33:18 +0000 (17:33 +0200)]
fix #2041, #2272: Add Spice enhancements

This adds a new config option called `spice_enhancements` with two
optional settings:

* videostreaming
* foldersharing

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
10 months agoclose #2263: die on live migration with local cloudinit disk
Mira Limbeck [Fri, 12 Jul 2019 12:23:15 +0000 (14:23 +0200)]
close #2263: die on live migration with local cloudinit disk

Live migration with a local cloudinit disk was never intended to work. It did
however work to an extent that the migration completed but the disk on the
source node could not be deleted. Now die if a live migration is started with
a local cloudinit disk.

With the GUI changes live migration is already disabled as it recognizes the
cloudinit disk as a local resource.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
10 months agoCPU flags: Add aes flag
Peter Keresztes Schmidt [Thu, 1 Aug 2019 17:22:23 +0000 (19:22 +0200)]
CPU flags: Add aes flag

Signed-off-by: Peter Keresztes Schmidt <peterke@sos.ethz.ch>
10 months agofix remote viewer live migration
Dominik Csapak [Tue, 20 Aug 2019 08:45:32 +0000 (10:45 +0200)]
fix remote viewer live migration

for some reason not setting port results in a port of '65535' which
triggers an execption in http-server anyevent, so we set the port to 0

also, we have to read the ticket from stdin even for 'unix' type secure
migration

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
10 months agoqemu_machine_pxe: remove unused parameter
Dominik Csapak [Tue, 13 Aug 2019 13:19:08 +0000 (15:19 +0200)]
qemu_machine_pxe: remove unused parameter

no caller used that parameter, ever

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
10 months agorework kvm_user_version cache mechanism
Dominik Csapak [Tue, 13 Aug 2019 13:19:07 +0000 (15:19 +0200)]
rework kvm_user_version cache mechanism

we now also save the mtime of the binary and cache per binary (for each
arch; this is done so we have it already when we sometime decide
that we want to split the qemu package for each arch)
so that we get the real version if only pve-qemu-kvm was updated

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
11 months agofix: qemu: uninitialized value in multiplication
Alwin Antreich [Wed, 24 Jul 2019 13:56:43 +0000 (15:56 +0200)]
fix: qemu: uninitialized value in multiplication

A generated VM with default values does not set the memory key in the
configuration. Hence the size of the state file for a suspend had only
the default size of the state itself and not in addition twice the
configured memory.

The patch uses the static defaults from the JSON schema if the memory
key is not set.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
11 months agoAdd a FIXME to the Q35 config to rm audio0 dev
Aaron Lauterer [Wed, 24 Jul 2019 13:57:29 +0000 (15:57 +0200)]
Add a FIXME to the Q35 config to rm audio0 dev

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
11 months agobump version to 6.0-7
Thomas Lamprecht [Wed, 24 Jul 2019 13:13:44 +0000 (15:13 +0200)]
bump version to 6.0-7

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoq35 already has an 'audio0' device, rename our to "audiodev$id"
Thomas Lamprecht [Wed, 24 Jul 2019 13:06:20 +0000 (15:06 +0200)]
q35 already has an 'audio0' device, rename our to "audiodev$id"

The latest changes to our audio device implemenation [0] changed the
naming of the device id to "audio<id>" which in practice resulted in
"audio0".

This conflicts with the predefined audio device in the Q35 configs
that is also using "audio0". The result is that a VM with a
configured audio device and Q35 type will not start.

While we just would had removed the audio0 device if we had detected
this earlier in the new 4.0 q35 config, we cannot do so anymore due
to migration compatibility.

So rename the device from "audio$id" to audiodev$id".

Co-authored-by: Aaron Lauterer <a.lauterer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobump version to 6.0-6
Thomas Lamprecht [Tue, 23 Jul 2019 16:20:49 +0000 (18:20 +0200)]
bump version to 6.0-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoMake hv-tlbflush and hv-evmcs optional and merge flag list
Stefan Reiter [Tue, 23 Jul 2019 11:41:28 +0000 (13:41 +0200)]
Make hv-tlbflush and hv-evmcs optional and merge flag list

Both flags were causing issues and have been made into optional flags to
be manually selected by the user if needed.

The supported flags have been merged into a single list for convenience.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
11 months agoaudio follouwp: pull out parsing into a conf_has_audio
Thomas Lamprecht [Tue, 23 Jul 2019 16:09:32 +0000 (18:09 +0200)]
audio follouwp: pull out parsing into a conf_has_audio

now that we finally now what the ID from the audiodev is for let's
factor some format string parsing out making a eventual future multi
audio device support with differen backends quite easy to support
here - once QEMU itself supports that fully.[0][1]

[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038400.html
[1]: https://lists.nongnu.org/archive/html/qemu-devel/2019-07/msg03583.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoaudio dev: specify qemu audio backend
Aaron Lauterer [Fri, 19 Jul 2019 13:15:44 +0000 (15:15 +0200)]
audio dev: specify qemu audio backend

QEMU 4 adds the `-audiodev` parameter to explicitly specify the audio
backend. Setting it avoids occasional error messages when starting a
virtual machine with an audio device and qemu wants to connect it to the
physical audio device.

For now only SPICE is supported as it's also the biggest use case.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
11 months agoremove unused variable
Dominik Csapak [Mon, 15 Jul 2019 12:35:30 +0000 (14:35 +0200)]
remove unused variable

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
11 months agofix #2275: die on invalid sendkey
Dominik Csapak [Mon, 15 Jul 2019 12:35:29 +0000 (14:35 +0200)]
fix #2275: die on invalid sendkey

sendkey does not return anything if it is succesful and the error otherwise

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
11 months agoaudio dev: follouwp code cleanup and error handling
Thomas Lamprecht [Thu, 18 Jul 2019 07:08:59 +0000 (09:08 +0200)]
audio dev: follouwp code cleanup and error handling

die if we get some unknown value, which is either a bug (new dev type
added to enum but not here) or a user manual edit error, good to
ccatch both

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoaudio dev: add note that this is useful for SPICE
Thomas Lamprecht [Thu, 18 Jul 2019 07:08:40 +0000 (09:08 +0200)]
audio dev: add note that this is useful for SPICE

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoAdd audio device to HW permissions
Aaron Lauterer [Wed, 17 Jul 2019 13:59:01 +0000 (15:59 +0200)]
Add audio device to HW permissions

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
11 months agoAdd PCI address for audio device
Aaron Lauterer [Wed, 17 Jul 2019 13:59:00 +0000 (15:59 +0200)]
Add PCI address for audio device

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
11 months agoAdd audio device support
Aaron Lauterer [Wed, 17 Jul 2019 13:58:57 +0000 (15:58 +0200)]
Add audio device support

Until now audio devices had to be added manually with the `args` option
in the VMs config file. This adds a new config option to define an audio
device. It is called `audio0` with the extra `0` to be prepared should
we ever implement support for more than one audio device.

Supported devices are:

* ich9-intel-hda: Intel HD Audio Controller, emulates ICH9
* intel-hda: Intel HD Audio Controller, emulates ICH6
* AC97: useful for older OSs like Win XP

If any of the `intel-hda`s is selected two additional devices will be
created: hda-micro and hda-duplex. The `*intel-hda` controllers need at
least one of them as they emulate microphones and speakers (hda-micro)
or line in and out devices (hda-duplex).

Having both is deemed best practice to avoid problems if some software
needs one of the other kind of output/input ports.

I also cleaned up some old, commented out, code regarding audio devices.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
12 months agobump version to 6.0-5
Thomas Lamprecht [Thu, 11 Jul 2019 17:45:38 +0000 (19:45 +0200)]
bump version to 6.0-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agouse new pcie port hardware
Dominik Csapak [Mon, 8 Jul 2019 09:25:10 +0000 (11:25 +0200)]
use new pcie port hardware

with qemu 4.0 we can make use of the new pcie-root-ports with settings
for the width/speed which can resolve issues with some hardware combinations
when negioating link speed

so we add a new q35 cfg that we include with machine types >= 4.0
to preserve live migration of machines without passthrough but q35

for details about the link speeds see:

pcie: Enhanced link speed and width support
https://lists.gnu.org/archive/html/qemu-devel/2018-12/msg02827.html

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
12 months agoFix: rtcflags: ignore startdate 'now'
Alwin Antreich [Thu, 4 Jul 2019 09:00:58 +0000 (11:00 +0200)]
Fix: rtcflags: ignore startdate 'now'

If 'now' is passed to the startdate option, the kvm start fails with
below failure.
 kvm: invalid datetime format
 valid formats: '2006-06-17T16:01:21' or '2006-06-17'

With this patch, 'now' is ignored and not passed to the rtcflags (-rtc).

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
12 months agobump version to 6.0-4
Thomas Lamprecht [Fri, 28 Jun 2019 18:36:39 +0000 (20:36 +0200)]
bump version to 6.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agocheck_local_storage_availability: only count each unavailable storage once
Thomas Lamprecht [Fri, 28 Jun 2019 15:23:44 +0000 (17:23 +0200)]
check_local_storage_availability: only count each unavailable storage once

and some general cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agomigrate: add size to volume attributes
Tim Marx [Fri, 28 Jun 2019 13:13:47 +0000 (15:13 +0200)]
migrate: add size to volume attributes

Signed-off-by: Tim Marx <t.marx@proxmox.com>
12 months agomigrate: handle storage not selected manually in storage config
Tim Marx [Fri, 28 Jun 2019 13:13:45 +0000 (15:13 +0200)]
migrate: handle storage not selected manually in storage config

e.g. local storage was considered not allowed for offline migration
even if it is available on all nodes, this should now be fixed as it
is now considered available on all nodes if a local storage isn't
restricted to a specific subset of the available nodes. The user is
responseable to make sure that the datacenter storage config reflects
the actual setup, so there is no additional check for local storages
which aren't available on all nodes if they are not explicitly marked
at datacenter level.

Signed-off-by: Tim Marx <t.marx@proxmox.com>
12 months agocloudinit: set iso-level in genisoimage call
Mira Limbeck [Thu, 27 Jun 2019 16:01:05 +0000 (18:01 +0200)]
cloudinit: set iso-level in genisoimage call

This is required for Windows to recognize the ISO and as a result the cloudinit
config. This is the minimum to get any config working at all for windows.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
12 months agoFix guest agent shutdown without timeout
Stefan Reiter [Tue, 25 Jun 2019 14:44:31 +0000 (16:44 +0200)]
Fix guest agent shutdown without timeout

Regression from change allowing timeouts to be set, now shutting down
also works without timeouts again (previously qmp failed because of
the unknown "timeout" parameter passed to it).

We always delete the timeout value from the arguments, regardless of
truthiness. "delete" returns the deleted element, deleting a
non-existant hash entry returns undef, which is fine after this point:

"deleting non-existent elements returns the undefined value in their
corresponding positions."
- https://perldoc.perl.org/functions/delete.html

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
12 months agobump version to 6.0-3
Thomas Lamprecht [Tue, 25 Jun 2019 12:34:27 +0000 (14:34 +0200)]
bump version to 6.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agoadd_hyperv_enlightenments: do not add hv_evmcs for now at all
Thomas Lamprecht [Tue, 25 Jun 2019 12:32:05 +0000 (14:32 +0200)]
add_hyperv_enlightenments: do not add hv_evmcs for now at all

as Windows VMs on hosts with AMD CPUs do not start at all with
that...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agobump version to 6.0-2
Thomas Lamprecht [Mon, 24 Jun 2019 15:50:01 +0000 (17:50 +0200)]
bump version to 6.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agofollowup: move hv_tlbflush under 3.1 machine feature guard
Thomas Lamprecht [Mon, 24 Jun 2019 15:23:42 +0000 (17:23 +0200)]
followup: move hv_tlbflush under 3.1 machine feature guard

while it really should not pose an issue for live migrations lets be
on the safe side and also add this only with 3.1 or later machines

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agofix #2083: Add hv_tlbflush, hv_ipi, hv_evmcs enlightenments
Stefan Reiter [Wed, 19 Jun 2019 08:23:20 +0000 (10:23 +0200)]
fix #2083: Add hv_tlbflush, hv_ipi, hv_evmcs enlightenments

Kernels 4.18+ (4.17+ for evmcs) support new Hyper-V enlightenments for
Windows KVM guests. QEMU supports these since 3.0 and 3.1 respectively.
tlbflush and ipi improve performance on overcommitted systems, evmcs
improves nested virtualization.

It's not entirely clear to me if Win7 already supports these, but since
it doesn't cause any performance penalties (and it works fine without
crashing, which makes sense either way, because Hyper-V enlightenments
are opt-in by the guest OS), enabling it regardless should be fine.
(As opposed to adding a new if branch for win8+)

Feature explanations to the best of my understanding:

hv_tlbflush allows the guest OS to trigger tlb shootdowns via a
hypercall. This allows CPUs to be identified via their vpindex (which
makes hv_vpindex a prerequisite to hv_tlbflush, but that is already
handled in our code). In overcommited configurations, where multiple
vCPUs reside on one pCPU, this increases performance of guest tlb
flushes, by only flushing each pCPU once. It also allows multiple tlb
flushes with only one vmexit.

hv_ipi allows sending inter-processor interrupts via vpindex, once again
making it a prerequisite. Benefits are pretty much as with tlbflush.

hv_evmcs is a VM control structure in L1 guest memory, allowing an L1 guest
to modify L2 VMCS and entering L2 without having the L0 host perform an
expensive VMCS update on trapping the nested vmenter.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
12 months agofix #2244: Allow timeout for guest-agent shutdown
Stefan Reiter [Mon, 24 Jun 2019 08:33:07 +0000 (10:33 +0200)]
fix #2244: Allow timeout for guest-agent shutdown

The "guest-shutdown" guest agent call is blocking for some reason, so if
it fails (e.g. agent not installed on guest) only the default timeout of
10 minutes (see QMPClient.pm, sub cmd) would apply.

With this change, if (and only if) a timeout is specified via CLI/API,
it is used instead. In case it is not specified, behaviour stays the
same (default 10 min timeout).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
12 months agod/control: bump version dependency to libpve-common-perl
Thomas Lamprecht [Mon, 24 Jun 2019 15:20:05 +0000 (17:20 +0200)]
d/control: bump version dependency to libpve-common-perl

for the new wait_for_unit_removed helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agod/control: wrap and sort + remove duplicates
Thomas Lamprecht [Mon, 24 Jun 2019 15:19:19 +0000 (17:19 +0200)]
d/control: wrap and sort + remove duplicates

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agowait for vm scopes to be gone
Wolfgang Bumiller [Wed, 19 Jun 2019 07:21:22 +0000 (09:21 +0200)]
wait for vm scopes to be gone

This should help with the rare case where stop mode backups
fail to restart due to the $vmid.scope not being completely
gone when we want to restart. This queries systemd via dbus,
and if the scope is still there, awaits a UnitRemoved signal
for the scope from dbus.

For now with a 5 second timeout... (given that the processes
are already gone and it's really just waiting for systemd to
wake up, this should be plenty...)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
12 months agoadd migration precondition api endpoint
Tim Marx [Fri, 14 Jun 2019 12:35:35 +0000 (14:35 +0200)]
add migration precondition api endpoint

Signed-off-by: Tim Marx <t.marx@proxmox.com>
12 months agobump version to 6.0-1
Thomas Lamprecht [Fri, 14 Jun 2019 18:59:24 +0000 (20:59 +0200)]
bump version to 6.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
12 months agod/control: bump qemu versioned dependency to 3.0.1-62
Thomas Lamprecht [Fri, 14 Jun 2019 18:55:28 +0000 (20:55 +0200)]
d/control: bump qemu versioned dependency to 3.0.1-62

to ensure we have the md-clear flag available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agoSMBIOS: followup: allow now 512 characters for full format string
Thomas Lamprecht [Tue, 11 Jun 2019 15:39:20 +0000 (17:39 +0200)]
SMBIOS: followup: allow now 512 characters for full format string

to acutally allow to set the values requested by #2190 we need to
alter the limit a bit, as else the requested values cannot be save.

Just double it to 512, which should be really enough for this, else
go complain to your software vendor for needing such _hacks_ in the
first place..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agofix #2190: Base64 encode SMBIOS value strings in order to allow more characters
Christian Ebner [Tue, 11 Jun 2019 10:13:52 +0000 (12:13 +0200)]
fix #2190: Base64 encode SMBIOS value strings in order to allow more characters

On some occasions e.g. license checking, the manufacturer string in the
SMBIOS settings edit has to allow characters such as whitespaces.
https://forum.proxmox.com/threads/proxmox-and-windows-rok-license-for-dell.53236/
In principle SMBIOS allows to pass any zero terminated string to the
corresponding fields in the structure type 1 (System Information).

By base64 encoding the values clashing of the config is avoided.

Relies on the corresponding patch to pve-manager to obtain base64 encoded values.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
13 months agofollowup: mention new md-clear flag also in description
Thomas Lamprecht [Thu, 6 Jun 2019 15:47:41 +0000 (17:47 +0200)]
followup: mention new md-clear flag also in description

that should really be autogenerated!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agoadd md-clear cpu flag
Oguz Bektas [Mon, 3 Jun 2019 13:17:11 +0000 (15:17 +0200)]
add md-clear cpu flag

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
13 months agoadd qm command for cloudinit config dump
Mira Limbeck [Wed, 5 Jun 2019 09:09:43 +0000 (11:09 +0200)]
add qm command for cloudinit config dump

Adds the command 'qm cloudinit dump <vmid> <type>' where type is either
'user', 'network' or 'meta' as specified in the API.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
13 months agoadd new API for dumping cloudinit config
Mira Limbeck [Wed, 5 Jun 2019 09:09:42 +0000 (11:09 +0200)]
add new API for dumping cloudinit config

Adds the path '{vmid}/cloudinit/dump' and requires the parameter 'type'
that's either 'user', 'network' or 'meta'. Returns the generated config as
string.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
13 months agoadd function to dump cloudinit config
Mira Limbeck [Wed, 5 Jun 2019 09:09:41 +0000 (11:09 +0200)]
add function to dump cloudinit config

This adds a function to dump the generated cloudinit config. Only one
can be dumped at a time, either 'user', 'network' or 'meta'.

The logic to get user, network and metadata is copied from the other
path that also creates the ISO image to keep it simple and not
complicate the other code path further.

The hash generation for the metadata config is unified between nocloud
and configdrive2 formats. We need it a 3rd time with the new dump
functions so it makes sense to combine it and the metadata config
generation in a single function. The <format>_gen_metadata functions are
each used twice now.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
13 months agodrop vnc x509 param, deprecated in 2.5 removed in 4.0
Thomas Lamprecht [Thu, 6 Jun 2019 07:49:47 +0000 (09:49 +0200)]
drop vnc x509 param, deprecated in 2.5 removed in 4.0

while we have 3.0.1 for now, 4.x will come sooner or later so do this
now already..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agoapi/qemu: start: add helper for root-only param extraction
Thomas Lamprecht [Thu, 6 Jun 2019 07:43:43 +0000 (09:43 +0200)]
api/qemu: start: add helper for root-only param extraction

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agoapi/qemu: extra line cleanup
Thomas Lamprecht [Thu, 6 Jun 2019 07:41:52 +0000 (09:41 +0200)]
api/qemu: extra line cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agoFix #1999: cli: listsnapshot: handle multiple roots and mark orphaned as root
Rhonda D'Vine [Wed, 5 Jun 2019 13:30:48 +0000 (15:30 +0200)]
Fix #1999: cli: listsnapshot: handle multiple roots and mark orphaned as root

This commit addresses the following things:

* There can be multiple independent snapshot tree roots, display them
  all
* If a snapshot defines a parent that doesn't exist, it is now
  considered a snapshot root

There is a potential issue (which was also before and also affects the
GUI): circular snapshot trees.  That plays into the second mentioned
issue above.  If you manage to have a snapshot that defines a
non-existing root in the config, and then create a snapshot with that
exact name as a child of that snapshot, it would create a circular
dependency.  This would have to get addressed in the GUI too.

Signed-off-by: Rhonda D'Vine <rhonda@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agodrop references to un-maintained sheepdog plugin
Thomas Lamprecht [Tue, 4 Jun 2019 15:42:33 +0000 (17:42 +0200)]
drop references to un-maintained sheepdog plugin

we remove it from pve-storage as it's abandoned

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agorefactor do_snapshots_with_qemu a bit cleaner
Thomas Lamprecht [Tue, 4 Jun 2019 15:40:42 +0000 (17:40 +0200)]
refactor do_snapshots_with_qemu a bit cleaner

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agowhitespace cleanup
Tim Marx [Wed, 22 May 2019 13:25:34 +0000 (15:25 +0200)]
whitespace cleanup

Signed-off-by: Tim Marx <t.marx@proxmox.com>
13 months agocommon sources.list: move from stretch to buster
Thomas Lamprecht [Mon, 27 May 2019 11:58:01 +0000 (13:58 +0200)]
common sources.list: move from stretch to buster

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agovm_resume: correctly honor $nocheck
Fabian Grünbichler [Thu, 23 May 2019 19:22:22 +0000 (21:22 +0200)]
vm_resume: correctly honor $nocheck

for both vm_mon_cmd calls. under certain circumstances, the following
sequence of events can otherwise fail when live-migrating under load:

S...source node
T...target node

0: migration is complete, handover from S to T starts
1: S: logically move VM config file from S to T via rename()
2: S: rename returns, config file is (visibly) moved on S
3: S: trigger resume on T via mtunnel
4a: T: call vm_resume while config file move is not yet visible on T
4b: T: call vm_resume while config file move is already visible on T

4a instead of 4b means vm_mon_cmd will die in check_running unless
vm_mon_cmd_nocheck is used.

under heavy pmxcfs load and a slow cluster/corosync network, there can
be a few seconds of delay between 1 and 2, with a subsequent race ending in 4a
instead of 4b.

this issue was reported to occur on bulk migrations.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
13 months agod/changelog: correct distribution
Thomas Lamprecht [Wed, 22 May 2019 17:12:54 +0000 (19:12 +0200)]
d/changelog: correct distribution

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>