qemu-server.git
34 hours agobump version to 6.2-18 master
Thomas Lamprecht [Thu, 29 Oct 2020 17:23:23 +0000 (18:23 +0100)]
bump version to 6.2-18

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
34 hours agopartially fix #3056: try to cancel backup without uuid
Dominik Csapak [Mon, 19 Oct 2020 14:11:38 +0000 (16:11 +0200)]
partially fix #3056: try to cancel backup without uuid

if the 'backup' qmp call itself times out or fails, we still want to
try to cancel the backup, else it can happen that there is still
a backup running even when vzdump thinks it was canceled

qapi docs says that backup cancel always returns success, even
if no backup is running

since we hold a global and a per vm lock for the backup, this should be
ok, since we should not reach this code without that lock

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
34 hours agomigrate: enable dirty-bitmap migration
Stefan Reiter [Thu, 22 Oct 2020 15:34:20 +0000 (17:34 +0200)]
migrate: enable dirty-bitmap migration

We query QEMU if it's safe before enabling it, as on versions without
the necessary patches it not only would be useless, but can actually
lead to hangs.

PBS state is always migrated, as it's a small amount of data anyway, so
we don't need to set a specific flag for it.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
8 days agobump version to 6.2-17
Thomas Lamprecht [Thu, 22 Oct 2020 14:09:03 +0000 (16:09 +0200)]
bump version to 6.2-17

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 days agobootorder: don't print empty 'order=' property
Stefan Reiter [Wed, 21 Oct 2020 09:00:25 +0000 (11:00 +0200)]
bootorder: don't print empty 'order=' property

Specifying 'boot: order=' was intended to be used for an empty bootorder
(i.e. no boot devices), but as it turns out our format parser doesn't
like empty '-list' properties if they are nested in a subformat.

Fixing this in JSONSchema sounds like a risky move, so instead just
write 'boot: ' (without 'order=') to indicate an empty bootorder. The
rest of the code handles it just fine, as this was valid before too.

Incidentally also fixes a bug where you couldn't create a new VM without
any disks if no explicit 'boot' property was specified (i.e. a simple
'qm create 100' without any parameters would fail).

Reported-by: Dominic Jäger <d.jaeger@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
11 days agobump version to 6.2-16
Thomas Lamprecht [Mon, 19 Oct 2020 13:51:00 +0000 (15:51 +0200)]
bump version to 6.2-16

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 days agotests: make module truthy
Thomas Lamprecht [Mon, 19 Oct 2020 13:33:16 +0000 (15:33 +0200)]
tests: make module truthy

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 days agotests: do not use for-loop for globs
Thomas Lamprecht [Mon, 19 Oct 2020 13:32:25 +0000 (15:32 +0200)]
tests: do not use for-loop for globs

they are rather inefficient for this

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 days agocloudinit: use normal grep
Thomas Lamprecht [Mon, 19 Oct 2020 13:32:03 +0000 (15:32 +0200)]
cloudinit: use normal grep

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 days agocomment out USB hotplug code
Thomas Lamprecht [Mon, 19 Oct 2020 13:31:38 +0000 (15:31 +0200)]
comment out USB hotplug code

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 days agofix some FH close
Thomas Lamprecht [Mon, 19 Oct 2020 13:30:21 +0000 (15:30 +0200)]
fix some FH close

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agoPCI: use warnings/strict and fix setting $vga from config2command
Thomas Lamprecht [Fri, 16 Oct 2020 15:53:28 +0000 (17:53 +0200)]
PCI: use warnings/strict and fix setting $vga from config2command

fixes commit 74c17b7a23c8a953d1dcec9bd53449d71d88cd5d which moved
this code here, but forgot to pass $vga ref, as the module was not
using warning nor strict mode this was not caught..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agotree wide cleanups
Thomas Lamprecht [Fri, 16 Oct 2020 15:52:51 +0000 (17:52 +0200)]
tree wide cleanups

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agoapi: factor out common vm_is_paused code in private helper
Thomas Lamprecht [Fri, 16 Oct 2020 15:51:24 +0000 (17:51 +0200)]
api: factor out common vm_is_paused code in private helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agoapi: fix conditional variable declaration
Thomas Lamprecht [Fri, 16 Oct 2020 15:51:05 +0000 (17:51 +0200)]
api: fix conditional variable declaration

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agodrop legacy __read_avail code
Thomas Lamprecht [Fri, 16 Oct 2020 14:52:29 +0000 (16:52 +0200)]
drop legacy __read_avail code

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agofix various conditionally declared vars
Thomas Lamprecht [Fri, 16 Oct 2020 14:52:11 +0000 (16:52 +0200)]
fix various conditionally declared vars

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agotree wide cleanup of s/return undef/return/
Thomas Lamprecht [Fri, 16 Oct 2020 14:20:05 +0000 (16:20 +0200)]
tree wide cleanup of s/return undef/return/

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 weeks agofix clone_disk failing for nonexistent cloudinit disk
Mira Limbeck [Mon, 28 Sep 2020 08:36:31 +0000 (10:36 +0200)]
fix clone_disk failing for nonexistent cloudinit disk

After migration or a rollback the cloudinit disk might not be allocated, so
volume_size_info() fails. As we override the value anyway for cloudinit
and efi disks simply move the volume_size_info() call into the 'else'
branch.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2 weeks agofix VM clone from snapshot with cloudinit disk
Mira Limbeck [Mon, 28 Sep 2020 08:36:30 +0000 (10:36 +0200)]
fix VM clone from snapshot with cloudinit disk

All volumes contained in $vollist are activated. In this case a snapshot
of the volume. For cloudinit disks no snapshots are created so don't add
it to the list of volumes to activate as it otherwise fails with no
logical volume found.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2 weeks agofix typos
Fabian Ebner [Wed, 14 Oct 2020 11:40:17 +0000 (13:40 +0200)]
fix typos

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 weeks agocfg2cmd: add tests for new boot order property
Stefan Reiter [Tue, 6 Oct 2020 13:32:17 +0000 (15:32 +0200)]
cfg2cmd: add tests for new boot order property

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 weeks agoapi: add handling for new boot order format
Stefan Reiter [Tue, 6 Oct 2020 13:32:16 +0000 (15:32 +0200)]
api: add handling for new boot order format

The API is updated to handle the deprecation correctly, i.e. when
updating the 'order' attribute, the old 'legacy' (default_key) values
are removed (would now be ignored anyway).

When removing a device that is in the bootorder list, it will be removed
from the aforementioned. Note that non-existing devices in the list will
not cause an error - they will simply be ignored - but it's still nice
to not have them in there.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 weeks agofix #3010: add 'bootorder' parameter for better control of boot devices
Stefan Reiter [Tue, 6 Oct 2020 13:32:15 +0000 (15:32 +0200)]
fix #3010: add 'bootorder' parameter for better control of boot devices

(also fixes #3011)

Deprecates the old-style 'boot' and 'bootdisk' options by adding a new
'order=' subproperty to 'boot'.

This allows a user to specify more than one disk in the boot order,
helping with newer versions of SeaBIOS/OVMF where disks without a
bootindex won't be initialized at all (breaks soft-raid and some LVM
setups).

This also allows specifying a bootindex for USB and hostpci devices,
which was not possible before. Floppy boot support is not supported in
the new model, but I doubt that will be a problem (AFAICT we can't even
attach floppy disks to a VM?).

Default behaviour is intended to stay the same, i.e. while new VMs will
receive the new 'order' property, it will be set so the VM starts the
same as before (using get_default_bootorder).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 weeks agoadd new 'boot' property format and introduce legacy conversion helpers
Stefan Reiter [Tue, 6 Oct 2020 13:32:14 +0000 (15:32 +0200)]
add new 'boot' property format and introduce legacy conversion helpers

The format is unused in this commit, but will replace the current
string-based format of the 'boot' property. It is included since the
parameter of bootorder_from_legacy follows it.

Two helper methods are introduced:
* bootorder_from_legacy: Parses the legacy format into a hash closer to
    what the new format represents
* get_default_bootdevices: Encapsulates the legacy default behaviour if
    nothing is specified in the boot order

resolve_first_disk is simplified and gets a new $cdrom parameter to
control the behaviour of excluding CD-ROMs or instead searching for only
them.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 weeks agocfg2cmd: add test for legacy-style bootorder
Stefan Reiter [Tue, 6 Oct 2020 13:32:13 +0000 (15:32 +0200)]
cfg2cmd: add test for legacy-style bootorder

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 weeks agofix indentation
Stefan Reiter [Tue, 6 Oct 2020 13:32:12 +0000 (15:32 +0200)]
fix indentation

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 weeks agobump version to 6.2-15
Thomas Lamprecht [Tue, 29 Sep 2020 15:44:31 +0000 (17:44 +0200)]
bump version to 6.2-15

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 weeks agofollowup: s/verification/validation/ to avoid confusion
Thomas Lamprecht [Tue, 29 Sep 2020 15:37:31 +0000 (17:37 +0200)]
followup: s/verification/validation/ to avoid confusion

We use verification for something more in-depth on the PBS server, so
avoid that term to avoid misunderstandings.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 weeks agovzdump: log 'finishing' state
Stefan Reiter [Mon, 28 Sep 2020 15:48:36 +0000 (17:48 +0200)]
vzdump: log 'finishing' state

...and avoid printing 100% status twice

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
7 weeks agod/control: update
Thomas Lamprecht [Tue, 8 Sep 2020 17:12:36 +0000 (19:12 +0200)]
d/control: update

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
7 weeks agod/copyright: update
Thomas Lamprecht [Tue, 8 Sep 2020 17:01:54 +0000 (19:01 +0200)]
d/copyright: update

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 weeks agoline length and formatting fixes
Thomas Lamprecht [Wed, 2 Sep 2020 12:07:02 +0000 (14:07 +0200)]
line length and formatting fixes

In accord to updated perl style guide
https://pve.proxmox.com/wiki/Perl_Style_Guide#Breaking_long_lines_and_strings

Mostly try to avoid >100 column lines and update some descriptions
from 80 to 100 column limit, improving readability.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 weeks agotests: add cfg2cmd test for virtio-blk disk with iothread on
Thomas Lamprecht [Wed, 2 Sep 2020 11:27:27 +0000 (13:27 +0200)]
tests: add cfg2cmd test for virtio-blk disk with iothread on

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
8 weeks agofix #2570: add 'keephugepages' config
Stefan Reiter [Wed, 2 Sep 2020 09:03:37 +0000 (11:03 +0200)]
fix #2570: add 'keephugepages' config

We already keep hugepages if they are created with the kernel
commandline (hugepagesz=x hugepages=y), but some setups (specifically
hugepages across multiple NUMA nodes) cannot be configured that way.
Since we always clear these hugepages at VM shutdown, rebooting a VM
that uses them might not work, since the requested count might not be
available anymore by the time we want to use them (also, we would then
no longer allocate them correctly on the NUMA nodes).

Add a 'keephugepages' parameter to skip cleanup and simply leave them
untouched.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agobump version to 6.2-14
Thomas Lamprecht [Mon, 24 Aug 2020 17:34:27 +0000 (19:34 +0200)]
bump version to 6.2-14

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agovzdump: do not log per disk dirty bitmap state for templates
Thomas Lamprecht [Fri, 21 Aug 2020 12:11:48 +0000 (14:11 +0200)]
vzdump: do not log per disk dirty bitmap state for templates

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agofix #2862: properly backup (all) VM templates
Fabian Grünbichler [Thu, 6 Aug 2020 11:13:48 +0000 (13:13 +0200)]
fix #2862: properly backup (all) VM templates

until we maybe have a 'pbs-backup' that links Qemu and PBS like
'pbs-restore', we need to do a regular backup for the template case to
support all storage types and image formats.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 months agofollowup cleanup
Thomas Lamprecht [Fri, 21 Aug 2020 08:44:00 +0000 (10:44 +0200)]
followup cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agotests: cfg2cmd: check also warnings
Thomas Lamprecht [Fri, 21 Aug 2020 08:38:27 +0000 (10:38 +0200)]
tests: cfg2cmd: check also warnings

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agotests: cfg2cmd: get testname earlier
Thomas Lamprecht [Fri, 21 Aug 2020 08:38:08 +0000 (10:38 +0200)]
tests: cfg2cmd: get testname earlier

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agoprint_vga_device: check if bios is defined
Stoiko Ivanov [Fri, 21 Aug 2020 08:07:38 +0000 (10:07 +0200)]
print_vga_device: check if bios is defined

Otherwise a warning is printed if the bios is not set in the config.

reported via community forum:
https://forum.proxmox.com/threads/warning-in-qemuserver.74683/

reproduced and tested that the patch fixes the issue.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 months agovzdump: show "reused" message even if nothing was transferred
Stefan Reiter [Thu, 20 Aug 2020 13:32:23 +0000 (15:32 +0200)]
vzdump: show "reused" message even if nothing was transferred

This still works even if all drives were clean. It then shows the very
magical line:

  INFO: backup was done incrementally, reused 34.00 GiB (100%)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agovzdump: allow bandwidth limit for PBS backup too
Stefan Reiter [Thu, 20 Aug 2020 13:32:22 +0000 (15:32 +0200)]
vzdump: allow bandwidth limit for PBS backup too

QEMU handles it just as well as with VMA, so this was probably just
forgotten to implement for PBS.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agobump version to 6.2-13
Thomas Lamprecht [Thu, 20 Aug 2020 09:42:53 +0000 (11:42 +0200)]
bump version to 6.2-13

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agovzdump: restore log behavior for VMs running with older QEMU
Thomas Lamprecht [Thu, 20 Aug 2020 08:45:34 +0000 (10:45 +0200)]
vzdump: restore log behavior for VMs running with older QEMU

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agovdzump: padd percentagde done counter, format duration for humans
Thomas Lamprecht [Thu, 20 Aug 2020 08:23:18 +0000 (10:23 +0200)]
vdzump: padd percentagde done counter, format duration for humans

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agovzdump: output sparseness again
Thomas Lamprecht [Thu, 20 Aug 2020 08:22:44 +0000 (10:22 +0200)]
vzdump: output sparseness again

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agovzdump: log reused information also if we run only <1s
Thomas Lamprecht [Wed, 19 Aug 2020 18:31:09 +0000 (20:31 +0200)]
vzdump: log reused information also if we run only <1s

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agovzdump: followup: log total dirtiness again, minor output adaptions
Thomas Lamprecht [Wed, 19 Aug 2020 18:30:30 +0000 (20:30 +0200)]
vzdump: followup: log total dirtiness again, minor output adaptions

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agobump version to 6.2-12
Thomas Lamprecht [Wed, 19 Aug 2020 17:46:28 +0000 (19:46 +0200)]
bump version to 6.2-12

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 months agocfg2cmd: vga: fix #2749: disable edid for Win+BIOS+VGA machines
Aaron Lauterer [Mon, 27 Jul 2020 13:55:25 +0000 (15:55 +0200)]
cfg2cmd: vga: fix #2749: disable edid for Win+BIOS+VGA machines

Edid support was added with Qemu 5. Windows guests seem to not be able
to get all possible resolutions if the default std VGA device is used as
GPU and the VM boots in BIOS mode. The result is that only one of the
following three resolutions can be configured:

800x600
1024x768
1920x1080

It is important to note that just booting a Windows VM with the edid=off
parameter will not make the large list of resolutions available. It
seems that Windows is caching the list of possible resolutions
somewhere [0].

Uninstalling the 'Microsoft Basic Display Adapter' in the device manager
and rebooting the VM is one way I found to force Windows to recreate the
list of possible resolutions.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
[0] https://lists.nongnu.org/archive/html/qemu-devel/2020-07/msg07128.html

2 months agovzdump: don't use dirty bitmap when VM was off
Stefan Reiter [Wed, 19 Aug 2020 15:02:04 +0000 (17:02 +0200)]
vzdump: don't use dirty bitmap when VM was off

There can't be a dirty bitmap when the VM was off, and if it was off we
will also shut it down after the backup, so no point in creating one.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agovzdump: log 100% percent in case $target is 0
Stefan Reiter [Wed, 19 Aug 2020 15:02:03 +0000 (17:02 +0200)]
vzdump: log 100% percent in case $target is 0

When $target is 0, that means we don't have to upload any data, in which
case we're immediately done.

Otherwise incremental backups with no changes display a really weird
  status: 0% (0.0 B of 0.0 B), duration 0, read: 0 B/s, write: 0 B/s
when they're actually done already.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agovzdump: display actually uploaded chunks as 'write' speed
Stefan Reiter [Wed, 19 Aug 2020 15:02:02 +0000 (17:02 +0200)]
vzdump: display actually uploaded chunks as 'write' speed

Previously 'read' and 'write' would always show the same value, which is
of little use. Change it so 'write' excludes reused bytes, thus
displaying the actual upload speed.

$last_reused needs to be initialized to contain reused data from 'clean'
dirty bitmaps to ensure the first output line is correct.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agovzdump: improve logging output with dirty bitmaps
Stefan Reiter [Wed, 19 Aug 2020 15:02:01 +0000 (17:02 +0200)]
vzdump: improve logging output with dirty bitmaps

Uses the new 'query-pbs-bitmap-info' QMP call to retrieve additional
information about each drive's dirty bitmap. Returned info is also used
to calculate $target by simply adding all the dirty values (dirty is
equal to size in case the entire drive will be backed up).

"Backup is sparse" message is suppressed for PBS, since it makes little
sense (if zero chunks appear in the clean area of a bitmap, they won't
be counted, and a user is probably more interested in the 'reused' data
anyway).

Also removes the need for the hacky $first_round query-backup handling.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agovalidate_cpu_conf: return config
Fabian Ebner [Wed, 12 Aug 2020 11:40:12 +0000 (13:40 +0200)]
validate_cpu_conf: return config

as parse_property_string expects it to.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>
2 months agoAdd missing die statements
Fabian Ebner [Tue, 4 Aug 2020 11:32:44 +0000 (13:32 +0200)]
Add missing die statements

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 months agoAdd dot character to CPU commandline regular expression
Fabian Ebner [Tue, 11 Aug 2020 10:47:03 +0000 (12:47 +0200)]
Add dot character to CPU commandline regular expression

to be able to parse CPU features like "+sse4.2" correctly.

Reported here: https://forum.proxmox.com/threads/zsf-snapshot-unable-to-parse-value-of-runningcpu-got-undefined-value.74094/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 months agod/control: bump versioned dep on pve-storage
Fabian Grünbichler [Thu, 6 Aug 2020 07:49:17 +0000 (09:49 +0200)]
d/control: bump versioned dep on pve-storage

needed for PBS plugin methods that are called in backup/restore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2 months agoUse correct option for storage_migrate
Fabian Ebner [Wed, 29 Jul 2020 09:44:15 +0000 (11:44 +0200)]
Use correct option for storage_migrate

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 months agobump version to 6.2-11
Fabian Grünbichler [Fri, 24 Jul 2020 06:14:08 +0000 (08:14 +0200)]
bump version to 6.2-11

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 months agoFix #2728: die/warn if target is not a replication target when live-migrating
Fabian Ebner [Mon, 18 May 2020 07:37:57 +0000 (09:37 +0200)]
Fix #2728: die/warn if target is not a replication target when live-migrating

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Tested-by: Aaron Lauterer <a.lauterer@proxmox.com>
3 months agofix #2857: restore: pass keyfile to pbs-restore
Fabian Grünbichler [Mon, 20 Jul 2020 08:26:21 +0000 (10:26 +0200)]
fix #2857: restore: pass keyfile to pbs-restore

if configured. otherwise restoring encrypted backups will be quite
hard..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 months agobump version to 6.2-10
Thomas Lamprecht [Mon, 13 Jul 2020 11:38:42 +0000 (13:38 +0200)]
bump version to 6.2-10

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agoset resume parameter for vm_start_nolock if there is a vmstate
Fabian Ebner [Tue, 7 Jul 2020 08:20:03 +0000 (10:20 +0200)]
set resume parameter for vm_start_nolock if there is a vmstate

In config_to_command, '-loadstate' will be added whenever there is a
vmstate in the config. But in vm_start_nolock, the resume parameter
is used to calculate the appropriate timeout and to remove the vmstate
after the start. The resume parameter was only set if there is a
'suspended' lock, but apparently [0] we cannot rely on the lock to be
set if and only if there is a vmstate.

[0]: https://forum.proxmox.com/threads/task-error-start-failed.72450

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 months agod/control: update build-dependency of pve-doc-generator
Thomas Lamprecht [Mon, 13 Jul 2020 10:25:33 +0000 (12:25 +0200)]
d/control: update build-dependency of pve-doc-generator

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agoAdd man page cpu-models.conf(5)
Stefan Reiter [Thu, 18 Jun 2020 13:44:57 +0000 (15:44 +0200)]
Add man page cpu-models.conf(5)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agofix mdev cmdline generation
Dominik Csapak [Mon, 13 Jul 2020 07:27:16 +0000 (09:27 +0200)]
fix mdev cmdline generation

during refactoring, the vmid got lost, but is necessary to get
the correct mdev id

Fixes commit 74c17b7a23c8a953d1dcec9bd53449d71d88cd5d
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ reference fixed commit ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agopass -X to ssh on upload
Wolfgang Bumiller [Fri, 10 Jul 2020 12:24:36 +0000 (14:24 +0200)]
pass -X to ssh on upload

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 months agobump version to 6.2-9
Wolfgang Bumiller [Fri, 10 Jul 2020 12:24:03 +0000 (14:24 +0200)]
bump version to 6.2-9

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 months agosupport for encrypted pbs backups
Wolfgang Bumiller [Fri, 10 Jul 2020 09:53:03 +0000 (11:53 +0200)]
support for encrypted pbs backups

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 months agoAdd comment about pbs env vars
Wolfgang Bumiller [Fri, 10 Jul 2020 09:44:53 +0000 (11:44 +0200)]
Add comment about pbs env vars

pbs-restore might not stay there like that forever and if
this code path changes we should remember to also remove the
environment variables

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 months agobump version to 6.2-8
Thomas Lamprecht [Thu, 9 Jul 2020 13:14:41 +0000 (15:14 +0200)]
bump version to 6.2-8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agofixup: use parse_property_string instead of parse_cpu_conf_basic
Stefan Reiter [Thu, 9 Jul 2020 12:41:14 +0000 (14:41 +0200)]
fixup: use parse_property_string instead of parse_cpu_conf_basic

The latter was removed and replaced with a validator.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agobump version to 6.2-7
Thomas Lamprecht [Thu, 9 Jul 2020 11:51:25 +0000 (13:51 +0200)]
bump version to 6.2-7

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobackup: detect PBS features and use only supported
Stefan Reiter [Wed, 8 Jul 2020 09:57:54 +0000 (11:57 +0200)]
backup: detect PBS features and use only supported

If 'query-proxmox-support' is not known to QEMU, assume that no other
features are supported either.

If 'pbs' is not supported at all, error out with a nice message.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agod/control: bump versioned dependency on pve-common
Fabian Grünbichler [Wed, 8 Jul 2020 08:24:54 +0000 (10:24 +0200)]
d/control: bump versioned dependency on pve-common

for PVE::JSONSchema with additional validator support

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 months agofix #2671: include CPU format in man page again
Stefan Reiter [Thu, 25 Jun 2020 11:35:41 +0000 (13:35 +0200)]
fix #2671: include CPU format in man page again

Use the new register_format(3) call to use a validator (instead of a
parser) for 'pve-(vm-)?cpu-conf'. This way the $cpu_fmt hash can be used for
generating the documentation, while still applying the same verification
rules as before.

Since the function no longer parses but only verifies, the parsing in
print_cpu_device/get_cpu_options has to go via JSONSchema directly.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 months agobump version to 6.2-6
Thomas Lamprecht [Tue, 7 Jul 2020 16:59:59 +0000 (18:59 +0200)]
bump version to 6.2-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agovzdump: log: small fixups
Thomas Lamprecht [Tue, 7 Jul 2020 15:32:25 +0000 (17:32 +0200)]
vzdump: log: small fixups

reuse can also come from the current backup - so drop the "from last
backup" as this can be very confusing if one reads it after making
the first backup ever, with no last backup existing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agovzdump: fix variable redeclaration warning
Thomas Lamprecht [Tue, 7 Jul 2020 06:41:22 +0000 (08:41 +0200)]
vzdump: fix variable redeclaration warning

happened due to moving the code from another scope which had no $res,
and not noticing as it was still working after all.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobump version to 6.2-5
Thomas Lamprecht [Mon, 6 Jul 2020 19:50:59 +0000 (21:50 +0200)]
bump version to 6.2-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobackup: PBS: save backup task size
Thomas Lamprecht [Mon, 6 Jul 2020 19:47:30 +0000 (21:47 +0200)]
backup: PBS: save backup task size

normally this is done centrally in the managers code, but we do not
have the info for PBS there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobackup: query status: use latest up-to-date info when finished
Thomas Lamprecht [Mon, 6 Jul 2020 19:46:03 +0000 (21:46 +0200)]
backup: query status: use latest up-to-date info when finished

we used the info from the previous loop for some summary logs, fix
that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobackup: restructure log: use human-readable sizes and include PBS info
Thomas Lamprecht [Mon, 6 Jul 2020 19:45:25 +0000 (21:45 +0200)]
backup: restructure log: use human-readable sizes and include PBS info

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 months agobackup: rename confusing incremental param to use-dirty-bitmap
Thomas Lamprecht [Mon, 6 Jul 2020 18:33:13 +0000 (20:33 +0200)]
backup: rename confusing incremental param to use-dirty-bitmap

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 months agoupdate_disksize: make interface leaner
Fabian Ebner [Wed, 20 May 2020 08:20:37 +0000 (10:20 +0200)]
update_disksize: make interface leaner

Pass new size directly, so the function doesn't need to know about
how some hash is organized. And return a message directly, instead
of both size-strings. Also dropped the wantarray, because both
existing callers use the message anyways.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 months agosync_disks: fix check
Fabian Ebner [Wed, 20 May 2020 08:20:36 +0000 (10:20 +0200)]
sync_disks: fix check

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
4 months agobump version to 6.2-4
Thomas Lamprecht [Tue, 30 Jun 2020 09:34:33 +0000 (11:34 +0200)]
bump version to 6.2-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 months agoenable dirty-bitmap incremental backups for PBS
Stefan Reiter [Thu, 25 Jun 2020 10:23:34 +0000 (12:23 +0200)]
enable dirty-bitmap incremental backups for PBS

The $total != $transferred check is changed to a log, as QEMU reports
only actually transferred bytes, and it is indeed correct for
incremental backups to have differing values from $total.

The 'incremental' parameter is always set, QEMU will figure out if it should
re-use an existing bitmap or create a new one on its own.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 months agoclose #2741: add VM.Config.Cloudinit permission
Mira Limbeck [Thu, 25 Jun 2020 14:48:47 +0000 (16:48 +0200)]
close #2741: add VM.Config.Cloudinit permission

This allows setting ciuser, cipassword and all other cloudinit settings that
are not part of the network without VM.Config.Network permissions.
Keep VM.Config.Network still as fallback so custom roles that add
VM.Config.Network but not VM.Config.Cloudinit don't break.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
4 months agoavoid backup command timeout with pbs
Dietmar Maurer [Fri, 26 Jun 2020 05:10:57 +0000 (07:10 +0200)]
avoid backup command timeout with pbs

4 months agopci: whitespace, indentation and formating fixes
Thomas Lamprecht [Thu, 25 Jun 2020 11:33:26 +0000 (13:33 +0200)]
pci: whitespace, indentation and formating fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 months agofix #2794: allow legacy IGD passthrough
Stefan Reiter [Thu, 18 Jun 2020 14:36:54 +0000 (16:36 +0200)]
fix #2794: allow legacy IGD passthrough

Legacy IGD passthrough requires address 00:1f.0 to not be assigned to
anything on QEMU startup (currently it's assigned to bridge pci.2).
Changing this in general would break live-migration, so introduce a new
hostpci parameter "legacy-igd", which if set to 1 will move that bridge
to be nested under bridge 1.

This is safe because:
* Bridge 1 is unconditionally created on i440fx, so nesting is ok
* Defaults are not changed, i.e. PCI layout only changes when the new
parameter is specified manually
* hostpci forbids migration anyway

Additionally, the PT device has to be assigned address 00:02.0 in the
guest as well, which is usually used for VGA assignment. Luckily, IGD PT
requires vga=none, so that is not an issue either.

See https://git.qemu.org/?p=qemu.git;a=blob;f=docs/igd-assign.txt

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 months agocfg2cmd: hostpci: move code to PCI.pm
Stefan Reiter [Thu, 18 Jun 2020 14:36:53 +0000 (16:36 +0200)]
cfg2cmd: hostpci: move code to PCI.pm

To avoid further cluttering config_to_command with subsequent changes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
4 months agotrivial whitespace followup fix
Thomas Lamprecht [Wed, 24 Jun 2020 08:59:05 +0000 (10:59 +0200)]
trivial whitespace followup fix

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
4 months agovzdump: move include logic for volumes to method
Aaron Lauterer [Mon, 22 Jun 2020 14:34:36 +0000 (16:34 +0200)]
vzdump: move include logic for volumes to method

Move the logic which volumes are included in the backup job to its own
method and adapt the VZDump code accordingly. This makes it possible to
develop other features around backup jobs.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
4 months agogen_rand_chars: handle errors properly
Fabian Grünbichler [Mon, 22 Jun 2020 10:03:01 +0000 (12:03 +0200)]
gen_rand_chars: handle errors properly

should not really happen on modern systems, but random_bytes just
returns false if it fails to generate random bytes, in which case we
want to die instead of returning an empty 'random' string.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
4 months agovncproxy: allow to request a generated VNC password
Thomas Lamprecht [Thu, 18 Jun 2020 16:20:10 +0000 (18:20 +0200)]
vncproxy: allow to request a generated VNC password

We used the VNC API $ticket as password for VNC, but QEMU limits the
password to the first 8 chars and ignores the rest[0].
As our tickets start with a static string (e.g., "PVE") the entropy
was a bit limited.

For Proxmox VE this does not matters much as the noVNC viewer
provided by has to go always over the API call, and so a valid
ticket and correct permissions for the requested VM are enforced
anyway.

This patch helps external users, which often use NoVNC-Websockify,
circumventing the API and relying solely on the VNC password to avoid
snooping on VNC sessions.

A 'generate-password' parameter is added, if set a password from good
entropy (using libopenssl) is generated.

For simplicity of mapping random bits to ranges we extract 6 bit of
entropy per character and add the integer value of '!' (first
printable ASCII char) to that. This way we get 64^8 possibilities,
which even with millions of guesses per second one would need years
of guessing and mostly just DDOS the server with websocket upgrade
requests.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>