]> git.proxmox.com Git - pve-qemu.git/log
pve-qemu.git
11 months agodrop patch for custom get_link_status QMP command
Fiona Ebner [Wed, 24 May 2023 13:56:52 +0000 (15:56 +0200)]
drop patch for custom get_link_status QMP command

There doesn't seem to be any Proxmox VE code using this.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agobump version to 8.0.2-1
Thomas Lamprecht [Tue, 6 Jun 2023 14:35:20 +0000 (16:35 +0200)]
bump version to 8.0.2-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoupdate reentrancy patches to version in upstream git
Fiona Ebner [Tue, 6 Jun 2023 08:58:50 +0000 (10:58 +0200)]
update reentrancy patches to version in upstream git

The previous version was picked from the mailing list and still had
an object_dynamic_cast call in a hot path, which is avoided with the
version that landed in git.

Also adds a few more exceptions for devices that need reentrancy.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agoupdate submodule and patches to QEMU 8.0.2
Fiona Ebner [Tue, 6 Jun 2023 08:58:49 +0000 (10:58 +0200)]
update submodule and patches to QEMU 8.0.2

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agobuildsys: remove edk2 source tree when assembling build-dir
Thomas Lamprecht [Wed, 24 May 2023 08:37:07 +0000 (10:37 +0200)]
buildsys: remove edk2 source tree when assembling build-dir

we ship it via pve-edk2-firmware anyway and it only results in bigger
source tar balls and lintian yelling at us due to edk2 not being the
simplest repo to ensure DFSG compat.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobump version to 8.0.0-1
Thomas Lamprecht [Mon, 22 May 2023 11:49:22 +0000 (13:49 +0200)]
bump version to 8.0.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: avoid handling noopt locally, rather extend CFLAGS
Thomas Lamprecht [Tue, 23 May 2023 12:09:03 +0000 (14:09 +0200)]
buildsys: avoid handling noopt locally, rather extend CFLAGS

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/rules: add identation for configure switches for readability
Thomas Lamprecht [Mon, 22 May 2023 13:23:20 +0000 (15:23 +0200)]
d/rules: add identation for configure switches for readability

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/control: drop obsolete build dependencies
Thomas Lamprecht [Mon, 22 May 2023 13:09:36 +0000 (15:09 +0200)]
d/control: drop obsolete build dependencies

drop autotools-dev, texi2html and texinfo build dependencies, they
are not used and have no effect

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: auto-generate dbgsym package
Thomas Lamprecht [Mon, 22 May 2023 11:51:22 +0000 (13:51 +0200)]
buildsys: auto-generate dbgsym package

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agosquash related patches
Fiona Ebner [Mon, 15 May 2023 13:39:56 +0000 (15:39 +0200)]
squash related patches

where there is no good reason to keep them separate. It's a pain
during rebase if there are multiple patches changing the same code
over and over again. This was especially bad for the backup-related
patches. If the history of patches really is needed, it can be
extracted via git. Additionally, compilation with partial application
of patches was broken since a long time, because one of the master key
changes became part of an earlier patch during a past rebase.

If only the same files were changed by a subsequent patch and the
changes felt to belong together (obvious for later bug fixes, but also
done for features e.g. adding master key support for PBS), the patches
were squashed together.

The PBS namespace support patch was split into the individual parts
it changes, i.e. PBS block driver, pbs-restore binary and QMP backup
infrastructure, and squashed into the respective patches.

No code change is intended, git diff in the submodule should not show
any difference between applying all patches before this commit and
applying all patches after this commit.

The query-proxmox-support QMP function has been left as part of the
"PVE-Backup: Proxmox backup patches for QEMU" patch, because it's
currently only used there. If it ever is used elsewhere too, it can
be split out from there.

The recent alloc-track and BQL-related savevm-async changes have been
left separate for now, because it's not 100% clear they are the best
approach yet. This depends on what upstream decides about the BQL
stuff and whether and what kind of issues with the changes pop up.

The qemu-img dd snapshot patch has been re-ordered to after the other
qemu-img dd patches.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agoPVE backup: don't call no_co_wrapper function from coroutine
Fiona Ebner [Mon, 15 May 2023 13:39:55 +0000 (15:39 +0200)]
PVE backup: don't call no_co_wrapper function from coroutine

Namely, pvebackup_co_prepare() needs to call bdrv_co_open() rather
than bdrv_open(), because it is a coroutine itself.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agoadd stable patches for 8.0.0
Fiona Ebner [Mon, 15 May 2023 13:39:54 +0000 (15:39 +0200)]
add stable patches for 8.0.0

Changes to other patches are all just metadata/context changes except
for pvebackup_co_prepare() needing to call bdrv_co_unref() rather than
bdrv_unref(), because it is a coroutine itself. This is documented in
d6ee2e324e ("block-coroutine-wrapper: Introduce no_co_wrapper"). The
change is necessary, because one of the stable fixes converts
bdrv_unref and blk_unref into no_co_wrappers (in preparation for a
second patch to fix a hang with the block resize QMP command).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agoupdate submodule and patches to QEMU 8.0.0
Fiona Ebner [Mon, 15 May 2023 13:39:53 +0000 (15:39 +0200)]
update submodule and patches to QEMU 8.0.0

Many changes were necessary this time around:

* QAPI was changed to avoid redundant has_* variables, see commit
44ea9d9be3 ("qapi: Start to elide redundant has_FOO in generated C")
for details. This affected many QMP commands added by Proxmox too.

* Pending querying for migration got split into two functions, one to
estimate, one for exact value, see commit c8df4a7aef ("migration:
Split save_live_pending() into state_pending_*") for details. Relevant
for savevm-async and PBS dirty bitmap.

* Some block (driver) functions got converted to coroutines, so the
Proxmox block drivers needed to be adapted.

* Alloc track auto-detaching during PBS live restore got broken by
AioContext-related changes resulting in a deadlock. The current, hacky
method was replaced by a simpler one. Stefan apparently ran into a
problem with that when he wrote the driver, but there were
improvements in the stream job code since then and I didn't manage to
reproduce the issue. It's a separate patch "alloc-track: fix deadlock
during drop" for now, you can find the details there.

* Async snapshot-related changes:
  - The pending querying got adapted to the above-mentioned split and
  a patch is added to optimize it/make it more similar to what
  upstream code does.
  - Added initialization of the compression counters (for
    future-proofing).
  - It's necessary the hold the BQL (big QEMU lock = iothread mutex)
  during the setup phase, because block layer functions are used there
  and not doing so leads to racy, hard-to-debug crashes or hangs. It's
  necessary to change some upstream code too for this, a version of
  the patch "migration: for snapshots, hold the BQL during setup
  callbacks" is intended to be upstreamed.
  - Need to take the bdrv graph read lock before flushing.

* hmp_info_balloon was moved to a different file.

* Needed to include a new headers from time to time to still get the
correct functions.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agod/rules: drop virtiofsd switch
Fiona Ebner [Mon, 15 May 2023 13:39:50 +0000 (15:39 +0200)]
d/rules: drop virtiofsd switch

virtiofsd is no longer part of QEMU 8.0. It got replaced by a separate
implementation written in Rust, which will be its own package.

See QEMU commit 0aaf44776e ("Merge tag 'pull-virtiofs-20230216b' of
https://gitlab.com/dagrh/qemu into staging").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
11 months agomakefile: convert to use simple parenthesis
Thomas Lamprecht [Mon, 22 May 2023 11:53:03 +0000 (13:53 +0200)]
makefile: convert to use simple parenthesis

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/lintian-overrides: ignore groff line breakage/adjustment warnings
Thomas Lamprecht [Mon, 22 May 2023 13:08:38 +0000 (15:08 +0200)]
d/lintian-overrides: ignore groff line breakage/adjustment warnings

not much we can do here anyway..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/lintian-overrides: sort
Thomas Lamprecht [Mon, 22 May 2023 13:07:15 +0000 (15:07 +0200)]
d/lintian-overrides: sort

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/parse-machines: produce stable json output
Thomas Lamprecht [Mon, 22 May 2023 13:02:42 +0000 (15:02 +0200)]
d/parse-machines: produce stable json output

Enabling the "canonical" option the keys will be sorted, improving
build reproducibility.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/control: define compat level via build-depends and raise to 13
Fiona Ebner [Fri, 19 May 2023 11:20:13 +0000 (13:20 +0200)]
d/control: define compat level via build-depends and raise to 13

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoalso exclude hppa-firmware.img ROM from build
Thomas Lamprecht [Mon, 22 May 2023 10:01:05 +0000 (12:01 +0200)]
also exclude hppa-firmware.img ROM from build

We don't use it and with debhelper compat level >= 11, the switch
from detecting files for strip through patters to checking for an ELF
header caused a build failure with the hppa-firmware.img ROM, as some
tools cannot cope with HP PARISC files.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agomove cleanup of unused ROMs from d/rules to build-dir generation
Thomas Lamprecht [Mon, 22 May 2023 10:00:03 +0000 (12:00 +0200)]
move cleanup of unused ROMs from d/rules to build-dir generation

this way we save a bit of space and should make build also slightly
faster, otherwise nothing should change.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/rules: fix debian-rules-missing-required-target
Thomas Lamprecht [Mon, 22 May 2023 10:24:06 +0000 (12:24 +0200)]
d/rules: fix debian-rules-missing-required-target

until we switch fully over to the dh sequencer

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/rules: cleanup cruft and use dpkg makefile fragements
Thomas Lamprecht [Mon, 22 May 2023 09:53:07 +0000 (11:53 +0200)]
d/rules: cleanup cruft and use dpkg makefile fragements

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: update lintian overrides
Thomas Lamprecht [Mon, 22 May 2023 10:04:22 +0000 (12:04 +0200)]
buildsys: update lintian overrides

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agoadd .gitignore file
Thomas Lamprecht [Mon, 22 May 2023 09:52:12 +0000 (11:52 +0200)]
add .gitignore file

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: use shorter variable name $@ in $(BUILDIR) target
Thomas Lamprecht [Mon, 22 May 2023 08:55:57 +0000 (10:55 +0200)]
buildsys: use shorter variable name $@ in $(BUILDIR) target

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: only run lintian for phony dsc target
Thomas Lamprecht [Mon, 22 May 2023 08:30:49 +0000 (10:30 +0200)]
buildsys: only run lintian for phony dsc target

This allows the sbuild to start much faster (lintian takes ~ minutes
for such big packages), and that without loss as sbuild will run
lintian on both binary and source package anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/control: avoid versioned build-dependcies with a -1 revision
Thomas Lamprecht [Sun, 21 May 2023 13:48:16 +0000 (15:48 +0200)]
d/control: avoid versioned build-dependcies with a -1 revision

no effect besides making it harder to build this for an eventual
backport.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agodebian: normalize packaging files with wrap-and-sort -tkn
Thomas Lamprecht [Sun, 21 May 2023 13:47:38 +0000 (15:47 +0200)]
debian: normalize packaging files with wrap-and-sort -tkn

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/rules: fix comment about when clean target is executed
Fiona Ebner [Fri, 19 May 2023 12:31:30 +0000 (14:31 +0200)]
d/rules: fix comment about when clean target is executed

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/rules: move copying config.guess and config.sub to config.status target
Fiona Ebner [Fri, 19 May 2023 12:21:11 +0000 (14:21 +0200)]
d/rules: move copying config.guess and config.sub to config.status target

It causes problems when done as part of the clean target when building
the dsc with the following error due to the additional files:
dpkg-source: error: aborting due to unexpected upstream changes

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: fix lintian overrides
Fiona Ebner [Mon, 15 May 2023 13:28:47 +0000 (15:28 +0200)]
buildsys: fix lintian overrides

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1007002 for more
information.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agod/rules: set job flag for make based on DEB_BUILD_OPTIONS
Fiona Ebner [Fri, 12 May 2023 11:39:08 +0000 (13:39 +0200)]
d/rules: set job flag for make based on DEB_BUILD_OPTIONS

Copied from Debian's QEMU package's d/rules. Otherwise, ninja will end
up using only a single job (in Debian Bookworm/Proxmox VE 8).

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: expand clean target
Fiona Ebner [Fri, 19 May 2023 11:22:00 +0000 (13:22 +0200)]
buildsys: expand clean target

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 [ T: remove all tarballs for a package and any .deb ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: create build directory atomically
Fiona Ebner [Fri, 19 May 2023 11:41:49 +0000 (13:41 +0200)]
buildsys: create build directory atomically

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: add sbuild convenience target
Fiona Ebner [Fri, 19 May 2023 11:31:05 +0000 (13:31 +0200)]
buildsys: add sbuild convenience target

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: add dsc target
Fiona Ebner [Fri, 19 May 2023 11:30:23 +0000 (13:30 +0200)]
buildsys: add dsc target

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
11 months agobuildsys: derive upload dist automatically
Fiona Ebner [Fri, 19 May 2023 11:24:32 +0000 (13:24 +0200)]
buildsys: derive upload dist automatically

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agobump version to 7.2.0-8
Thomas Lamprecht [Fri, 17 Mar 2023 14:48:12 +0000 (15:48 +0100)]
bump version to 7.2.0-8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agocherry-pick TCG-related stable fixes for 7.2
Fiona Ebner [Fri, 17 Mar 2023 12:47:11 +0000 (13:47 +0100)]
cherry-pick TCG-related stable fixes for 7.2

When turning off the "KVM hardware virtualization" checkbox in Proxmox
VE, the TCG accelerator is used, so these fixes are relevant then.

The first patch is included to allow cherry-picking the others without
changes.

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
13 months agobump version to 7.2.0-8
Thomas Lamprecht [Fri, 17 Mar 2023 11:12:02 +0000 (12:12 +0100)]
bump version to 7.2.0-8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
13 months agoadd patch fixing ACPI CPU hotplug issue with TCG
Fiona Ebner [Fri, 17 Mar 2023 10:39:52 +0000 (11:39 +0100)]
add patch fixing ACPI CPU hotplug issue with TCG

Required for the debian/edk2-vars-generator.py script in the
pve-edk2-firmware repository when building the edk2-stable202302
release. Without this patch, the QEMU process spawned by the script
would hang indefinietly.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agobump version to 7.2.0-7
Thomas Lamprecht [Mon, 13 Mar 2023 16:42:52 +0000 (17:42 +0100)]
bump version to 7.2.0-7

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
14 months agoadd patches to fix regression with LSI SCSI controller
Fiona Ebner [Mon, 13 Mar 2023 11:43:54 +0000 (12:43 +0100)]
add patches to fix regression with LSI SCSI controller

The patch 0008-memory-prevent-dma-reentracy-issues.patch introduced a
regression for the LSI SCSI controller leading to boot failures [0],
because, in its current form, it relies on reentrancy for a particular
ram_io region.

[0]: https://forum.proxmox.com/threads/123843

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agoadd more stable fixes
Fiona Ebner [Thu, 9 Mar 2023 13:37:35 +0000 (14:37 +0100)]
add more stable fixes

The patches were selected from the recent "Patch Round-up for stable
7.2.1" [0]. Those that should be relevant for our supported use-cases
(and the upcoming nvme use-case) were picked. Most of the patches
added now have not been submitted to qemu-stable before.

The follow-up for the virtio-rng-pci migration fix will break
migration between versions with the fix and without the fix when a
virtio-pci-rng(-non)-transitional device is used. Luckily Proxmox VE
only uses the virtio-pci-rng device, and this was fixed by
0006-virtio-rng-pci-fix-migration-compat-for-vectors.patch which was
applied before any public version of Proxmox VE's QEMU 7.2 package was
released.

[0]: https://lists.nongnu.org/archive/html/qemu-stable/2023-03/msg00010.html
[1]: https://bugzilla.redhat.com/show_bug.cgi?id=2162569

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agofixup patch "ide: avoid potential deadlock when draining during trim"
Fiona Ebner [Thu, 9 Mar 2023 13:37:34 +0000 (14:37 +0100)]
fixup patch "ide: avoid potential deadlock when draining during trim"

The patch was incomplete and (re-)introduced an issue with a potential
failing assertion upon cancelation of the DMA request.

There is a patch on qemu-devel now[0], and it's the same as this one
code-wise (except for comments). But the discussion is still ongoing.
While there shouldn't be a real issue with the patch, there might be
better approaches. The plan is to use this as a stop-gap for now and
pick up the proper solution once it's ready.

[0]: https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg03325.html

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agobump version to 7.2.0-6
Thomas Lamprecht [Wed, 8 Mar 2023 13:32:22 +0000 (14:32 +0100)]
bump version to 7.2.0-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
14 months agoadd patch to avoid potential deadlock with trim for IDE/SATA and draining
Fiona Ebner [Wed, 8 Mar 2023 11:51:05 +0000 (12:51 +0100)]
add patch to avoid potential deadlock with trim for IDE/SATA and draining

In particular, the deadlock can occur, together with unlucky timing
between the QEMU threads, when the guest is issuing trim requests
during the start of a backup operation.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 [ T: resolve trivial merge conflict in series file ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
14 months agoadd patch fixing Linux boot failures with megasas SCSI
Fiona Ebner [Mon, 6 Mar 2023 07:41:37 +0000 (08:41 +0100)]
add patch fixing Linux boot failures with megasas SCSI

A regression in 7.2 and easily reproduced.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agobump version to 7.2.0-5
Thomas Lamprecht [Tue, 21 Feb 2023 12:50:08 +0000 (13:50 +0100)]
bump version to 7.2.0-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
14 months agoadd fix for iscsi double free issue leading to crashes
Fiona Ebner [Tue, 21 Feb 2023 12:21:02 +0000 (13:21 +0100)]
add fix for iscsi double free issue leading to crashes

Reported here[0] and here[1].

[0]: https://gitlab.com/qemu-project/qemu/-/issues/1378
[1]: https://forum.proxmox.com/threads/122776/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agoadd patch fixing DMA reentrancy issues
Fiona Ebner [Tue, 21 Feb 2023 09:08:59 +0000 (10:08 +0100)]
add patch fixing DMA reentrancy issues

that could lead to use-after-frees and stack overflows with a
malicious (or buggy) guest. See [0] for a good summary:

[0]: https://lore.kernel.org/qemu-devel/CAFEAcA_23vc7hE3iaM-JVA6W38LK4hJoWae5KcknhPRD5fPBZA@mail.gmail.com

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agoQMP backup: use correct errno when getting blockdrive length fails
Fiona Ebner [Mon, 30 Jan 2023 10:20:42 +0000 (11:20 +0100)]
QMP backup: use correct errno when getting blockdrive length fails

di->size would only be set later. The errno is minus the return value
from the function.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agosavevm-async: keep more free space when entering final stage
Fiona Ebner [Thu, 26 Jan 2023 13:46:14 +0000 (14:46 +0100)]
savevm-async: keep more free space when entering final stage

In qemu-server, we already allocate 2 * $mem_size + 500 MiB for driver
state (which was 32 MiB long ago according to git history). It seems
likely that the 30 MiB cutoff in the savevm-async implementation was
chosen based on that.

In bug #4476 [0], another issue caused the iteration to not make any
progress and the state file filled up all the way to the 30 MiB +
pending_size cutoff. Since the guest is not stopped immediately after
the check, it can still dirty some RAM and the current cutoff is not
enough for a reproducer VM (was done while bug #4476 still was not
fixed), dirtying memory with
> stress-ng -B 2 --bigheap-growth 64.0M'
After entering the final stage, savevm actually filled up the state
file completely, leading to an I/O error. It's probably the same
scenario as reported in the bug report, the error message was fixed in
commit a020815 ("savevm-async: fix function name in error message")
after the bug report.

If not for the bug, the cutoff will only be reached by a VM that's
dirtying RAM faster than can be written to the storage, so increase
the cutoff to 100 MiB to have a bigger chance to finish successfully,
while still trying to not increase downtime too much for
non-hibernation snapshots.

[0]: https://bugzilla.proxmox.com/show_bug.cgi?id=4476

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
14 months agofix #4476: savevm-async: avoid looping without progress
Fiona Ebner [Thu, 26 Jan 2023 13:46:13 +0000 (14:46 +0100)]
fix #4476: savevm-async: avoid looping without progress

when pend_postcopy is large. By definition, pend_postcopy won't
decrease when iterating, so a value larger than the cutoff of 400000
would lead to essentially empty iterations, filling up the state file
until only 30 MiB + pending_size remain and the second half of the
check would trigger.

Avoid this, by not considering pend_postcopy for the cutoff to enter
the final phase.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
15 months agod/rules: add note explaining why using noopt doesn't currenlty work
Fiona Ebner [Fri, 10 Feb 2023 09:39:24 +0000 (10:39 +0100)]
d/rules: add note explaining why using noopt doesn't currenlty work

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
15 months agod/rules: add missing export for CFLAGS
Fiona Ebner [Fri, 10 Feb 2023 09:39:23 +0000 (10:39 +0100)]
d/rules: add missing export for CFLAGS

Otherwise, they don't affect the build of QEMU at all.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
15 months agobump version to 7.2.0-4
Thomas Lamprecht [Fri, 27 Jan 2023 08:37:53 +0000 (09:37 +0100)]
bump version to 7.2.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
15 months agoadd patch to fix issue with VirtIO disk using detect-zeroes=unmap
Fiona Ebner [Fri, 27 Jan 2023 08:13:35 +0000 (09:13 +0100)]
add patch to fix issue with VirtIO disk using detect-zeroes=unmap

Affects Proxmox VE, when the discard disk setting is used for a
VirtIO disk.

Upstream bug report:
https://gitlab.com/qemu-project/qemu/-/issues/1404

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
15 months agod/control: also conflict with "qemu-system-data"
Fabian Grünbichler [Fri, 20 Jan 2023 08:45:51 +0000 (09:45 +0100)]
d/control: also conflict with "qemu-system-data"

it ships files also shipped by our qemu package, switching from Debian qemu to
ours doesn't work without manual intervention otherwise..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
15 months agosavevm-async: fix function name in error message
Fiona Ebner [Mon, 23 Jan 2023 11:43:23 +0000 (12:43 +0100)]
savevm-async: fix function name in error message

which also makes it distinguishable from the other
"qemu_savevm_state_iterate error" message.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agobump version to 7.2.0-3
Thomas Lamprecht [Thu, 12 Jan 2023 12:13:23 +0000 (13:13 +0100)]
bump version to 7.2.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
16 months agoadd regression fix for migration with virtio-rng device
Fiona Ebner [Thu, 12 Jan 2023 10:09:08 +0000 (11:09 +0100)]
add regression fix for migration with virtio-rng device

between QEMU less than 7.2 and QEMU 7.2 without the fix (both
directions are affected).

As mentioned in the patch message, this fix itself will break
migration between QEMU 7.2 and QEMU 7.2 with the fix (in both
directions, if a virtio-rng device is attached), but this is fine,
because no pve-qemu-kvm package with QEMU 7.2 has been publicly
released yet.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agobump version to 7.2.0-2
Thomas Lamprecht [Tue, 10 Jan 2023 14:47:52 +0000 (15:47 +0100)]
bump version to 7.2.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
16 months agocherry-pick stable fixes for 7.2
Fiona Ebner [Tue, 10 Jan 2023 09:34:25 +0000 (10:34 +0100)]
cherry-pick stable fixes for 7.2

Two for virtio-mem and one for vIOMMU. Both features are not yet
exposed in PVE's qemu-server, but planned to be added.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agoadd QAPI naming exceptions in patches introducing them
Fiona Ebner [Tue, 10 Jan 2023 08:40:57 +0000 (09:40 +0100)]
add QAPI naming exceptions in patches introducing them

Avoids a patch and is required to compile when not all patches are
applied. No functional change is intended.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agod/control: drop outdated jemalloc dependencies
Fiona Ebner [Tue, 20 Dec 2022 08:19:53 +0000 (09:19 +0100)]
d/control: drop outdated jemalloc dependencies

Commit 3d785ea ("disable jemalloc") disabled jemalloc support, so
these are not needed anymore.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agod/control: add libslirp-dev as a build dependency
Fiona Ebner [Tue, 20 Dec 2022 08:19:52 +0000 (09:19 +0100)]
d/control: add libslirp-dev as a build dependency

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agod/rules: enable slirp again
Fiona Ebner [Tue, 20 Dec 2022 08:19:51 +0000 (09:19 +0100)]
d/rules: enable slirp again

Commit d03e1b3 ("update submodule and patches to 7.2.0") argued that
slirp is not explicitly supported in PVE, but that is not true. In
qemu-server, user networking is supported (via CLI/API) when no bridge
is set on a virtual NIC. So slirp needs to stay to keep such NICs
working.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agobump version to 7.2.0-1
Wolfgang Bumiller [Fri, 16 Dec 2022 12:18:54 +0000 (13:18 +0100)]
bump version to 7.2.0-1

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
16 months agod/rules: explicitly disable building slirp
Fiona Ebner [Wed, 14 Dec 2022 14:16:33 +0000 (15:16 +0100)]
d/rules: explicitly disable building slirp

Otherwise, it depends on whether libslirp-devel is installed or not.
See the previous commit message for more context.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
16 months agoupdate submodule and patches to 7.2.0
Fiona Ebner [Wed, 14 Dec 2022 14:16:32 +0000 (15:16 +0100)]
update submodule and patches to 7.2.0

User-facing breaking change:

The slirp submodule for user networking got removed. It would be
necessary to add the --enable-slirp option to the build and/or install
the appropriate library to continue building it. Since PVE is not
explicitly supporting it, it would require additionally installing the
libslirp0 package on all installations and there is *very* little
mention on the community forum when searching for "slirp" or
"netdev user", the plan is to only enable it again if there is some
real demand for it.

Notable changes:

* The big change for this release is the rework of job locking, using
  a job mutex and introducing _locked() variants of job API functions
  moving away from call-side AioContext locking. See (in the qemu
  submodule) commit 6f592e5aca ("job.c: enable job lock/unlock and
  remove Aiocontext locks") and previous commits for context.

  Changes required for the backup patches:
  * Use WITH_JOB_LOCK_GUARD() and call the _locked() variant of job
    API functions where appropriate (many are only availalbe as
    a _locked() variant).
  * Remove acquiring/releasing AioContext around functions taking the
    job mutex lock internally.

  The patch introducing sequential transaction support for jobs needs
  to temporarily unlock the job mutex to call job_start() when
  starting the next job in the transaction.

* The zeroinit block driver now marks its child as primary.

  The documentation in include/block/block-common.h states:
  > Filter node has exactly one FILTERED|PRIMARY child, and may have
  > other children which must not have these bits

  Without this, an assert will trigger when copying to a zeroinit target
  with qemu-img convert, because bdrv_child_cb_attach() expects any
  non-PRIMARY child to be not FILTERED:
  > qemu-img convert -n -p -f raw -O raw input.raw zeroinit:output.raw
  > qemu-img: ../block.c:1476: bdrv_child_cb_attach: Assertion
  > `!(child->role & BDRV_CHILD_FILTERED)' failed.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
17 months agobump version to 7.1.0-4
Thomas Lamprecht [Tue, 22 Nov 2022 08:21:10 +0000 (09:21 +0100)]
bump version to 7.1.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
17 months agocherry-pick "block/block-backend: blk_set_enable_write_cache is IO_CODE"
Thomas Lamprecht [Tue, 22 Nov 2022 08:18:56 +0000 (09:18 +0100)]
cherry-pick "block/block-backend: blk_set_enable_write_cache is IO_CODE"

albeit I was short from disarming that GLOBAL_STATE_CODE assert
completely, as its just bogus to assert that on runtime for a lot of
call sites, rather it should be verified on compilation (function
coloring with attributes and maybe a compiler plugin).

But, as this is already solved upstream lets take in that patch.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
18 months agobump version to 7.1.0-3
Thomas Lamprecht [Fri, 28 Oct 2022 08:27:54 +0000 (10:27 +0200)]
bump version to 7.1.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
18 months agoinit: daemonize: defuse PID file resolve error to warning
Thomas Lamprecht [Fri, 28 Oct 2022 08:22:21 +0000 (10:22 +0200)]
init: daemonize: defuse PID file resolve error to warning

fixes file restore, where we actively unlink the PID file of the
transient VM ourself after opening it - while we use it only for
tracking when the QEMU process itself has finished start up, it's
easier and cleaner to fix this regression now, than to rework that to
something that doesn't depends on the PID file at all.

Applying Fiona's patch as patch-patch tracked under extra, as I
expect that something similar to this gets accepted upstreamed.

Link: https://lists.proxmox.com/pipermail/pve-devel/2022-October/054448.html
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
18 months agobump version to 7.1.0-2
Wolfgang Bumiller [Tue, 18 Oct 2022 13:35:09 +0000 (15:35 +0200)]
bump version to 7.1.0-2

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
18 months agosavevm async IO channel: channel writev: fix return value in error case
Fiona Ebner [Mon, 17 Oct 2022 07:18:33 +0000 (09:18 +0200)]
savevm async IO channel: channel writev: fix return value in error case

The documentation in include/io/channel.h states that -1 or
QIO_CHANNEL_ERR_BLOCK should be returned upon error. Simply passing
along the return value from the blk-functions has the potential to
confuse the call sides. Non-blocking mode is not implemented
currently, so -1 it is.

The "return ret" was mistakenly left over from the previous
QEMUFileOps based implementation. Also, use error_setg_errno(), since
the blk(_co)_p{readv,writev} functions return errno codes.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
19 months agobump version to 7.1.0-1
Wolfgang Bumiller [Fri, 14 Oct 2022 12:55:53 +0000 (14:55 +0200)]
bump version to 7.1.0-1

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
19 months ago{alloc track, pbs} block driver: bdrv_co_preadv: adapt return values
Fiona Ebner [Fri, 14 Oct 2022 12:07:16 +0000 (14:07 +0200)]
{alloc track, pbs} block driver: bdrv_co_preadv: adapt return values

to be in-line with what other implementations in QEMU do. Commit
1d39c7098bbfa6862cb96066c4f8f6735ea397c5 mentions the EIO bit and
the function is expected to return 0 upon success (see other
implementations).

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
19 months agosavevm async: cleaner initialization of target_close_wait member
Fiona Ebner [Fri, 14 Oct 2022 12:07:15 +0000 (14:07 +0200)]
savevm async: cleaner initialization of target_close_wait member

Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
19 months agocherry-pick upstream fixes for 7.1.0
Fiona Ebner [Fri, 14 Oct 2022 12:07:14 +0000 (14:07 +0200)]
cherry-pick upstream fixes for 7.1.0

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
19 months agoupdate submodule and patches to 7.1.0
Fiona Ebner [Fri, 14 Oct 2022 12:07:13 +0000 (14:07 +0200)]
update submodule and patches to 7.1.0

Notable changes:
* The only big change is the switch to using a custom QIOChannel for
  savevm-async, because the previously used QEMUFileOps was dropped.

  Changes to the current implementation:

  * Switch to vector based methods as required for an IO channel. For
    short reads the passed-in IO vector is stuffed with zeroes at the
    end, just to be sure.

  * For reading: The documentation in include/io/channel.h states that
    at least one byte should be read, so also error out when whe are
    at the very end instead of returning 0.

  * For reading: Fix off-by-one error when request goes beyond end.

    The wrong code piece was:
    if ((pos + size) > maxlen) {
        size = maxlen - pos - 1;
    }

    Previously, the last byte would not be read. It's actually
    possible to get a snapshot .raw file that has content all the way
    up the final 512 byte (= BDRV_SECTOR_SIZE) boundary without any
    trailing zero bytes (I wrote a script to do it).

    Luckily, it didn't cause a real issue, because qemu_loadvm_state()
    is not interested in the final (i.e. QEMU_VM_VMDESCRIPTION)
    section. The buffer for reading it is simply freed up afterwards
    and the function will assume that it read the whole section, even
    if that's not the case.

  * For writing: Make use of the generated blk_pwritev() wrapper
    instead of manually wrapping the coroutine to simplify and save a
    few lines.

* Adapt to changed interfaces for blk_{pread,pwrite}:
  * a9262f551e ("block: Change blk_{pread,pwrite}() param order")
  * 3b35d4542c ("block: Add a 'flags' param to blk_pread()")
  * bf5b16fa40 ("block: Make blk_{pread,pwrite}() return 0 on success")
  Those changes especially affected the qemu-img dd patches, because
  the context also changed, but also some of our block drivers used
  the functions.

* Drop qemu-common.h include: it got renamed after essentially
  everything was moved to other headers. The only remaining user I
  could find for things dropped from the header between 7.0 and 7.1
  was qemu_get_vm_name() in the iscsi-initiatorname patch, but it
  already includes the header to which the function was moved.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
19 months agobump version to 7.0.0-4
Wolfgang Bumiller [Mon, 10 Oct 2022 09:56:27 +0000 (11:56 +0200)]
bump version to 7.0.0-4

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
19 months agoadd patch: PVE Backup: allow passing max-workers performance setting
Wolfgang Bumiller [Mon, 10 Oct 2022 09:55:15 +0000 (11:55 +0200)]
add patch: PVE Backup: allow passing max-workers performance setting

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
19 months agod/rules: add revision to package version
Fiona Ebner [Fri, 30 Sep 2022 12:18:31 +0000 (14:18 +0200)]
d/rules: add revision to package version

This version string can be queried with $BINARY --version as well as
the query-version QMP command.

Useful for qemu-server to be able to report the running QEMU version
exactly. Could also be used to version guard against features as an
alternative to the query-proxmox-support QMP command.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
20 months agobump version to 7.0.0-3
Thomas Lamprecht [Tue, 30 Aug 2022 10:54:12 +0000 (12:54 +0200)]
bump version to 7.0.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
20 months agosavevm-async: set SAVE_STATE_DONE when closing state file was successful
Fiona Ebner [Thu, 18 Aug 2022 11:44:17 +0000 (13:44 +0200)]
savevm-async: set SAVE_STATE_DONE when closing state file was successful

Without this change, it's necessary to send a second savevm-end QMP
command after aborting a snaphsot, before a new savevm-start QMP
command can succeed.

In process_savevm_finalize(), no longer set an error in the abort
scenario. If there already is another error, there's no need to
override it. If canceling was done intentionally, qmp_savevm_end()
is responsible for setting the state now.

Reported-by: Mira Limbeck <m.limbeck@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
20 months agosavevm-async: avoid segfault when aborting snapshot
Fiona Ebner [Thu, 18 Aug 2022 11:44:16 +0000 (13:44 +0200)]
savevm-async: avoid segfault when aborting snapshot

Reported in the community forum[0].

For 6.1.0, there were a few changes to the coroutine-sleep API, but
the adaptations in f376b2b ("update and rebase to QEMU v6.1.0") made
a mistake.

Currently, target_close_wait is NULL when passed to
qemu_co_sleep_ns_wakeable(), which further passes it to
qemu_co_sleep(), but there, it is dereferenced when trying to access
the 'to_wake' member:

> Thread 1 "kvm" received signal SIGSEGV, Segmentation fault.
> qemu_co_sleep (w=0x0) at ../util/qemu-coroutine-sleep.c:57

To fix it, create a proper struct and pass its address instead. Also
call qemu_co_sleep_wake unconditionally, because the NULL check (for
the 'to_wake' member) is done inside the function itself.

This patch is based on what the QEMU commits introducing the changes
to the coroutine-sleep API did to the callers in QEMU:
eaee072085 ("coroutine-sleep: allow qemu_co_sleep_wake that wakes nothing")
29a6ea24eb ("coroutine-sleep: replace QemuCoSleepState pointer with struct in the API")

[0]: https://forum.proxmox.com/threads/112130/

Tested-by: Mira Limbeck <m.limbeck@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
21 months agobump version to 7.0.0-2
Thomas Lamprecht [Wed, 20 Jul 2022 07:17:13 +0000 (09:17 +0200)]
bump version to 7.0.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
21 months agoadd two more stable patches
Fabian Ebner [Tue, 19 Jul 2022 08:19:31 +0000 (10:19 +0200)]
add two more stable patches

For the io_uring patch, it's not very clear which configurations can
trigger it, but it should be rather uncommon. See qemu commit
be6a166fde652589761cf70471bcde623e9bd72a for a bit more information.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
22 months agobump version to 7.0.0-1
Wolfgang Bumiller [Thu, 30 Jun 2022 09:08:36 +0000 (11:08 +0200)]
bump version to 7.0.0-1

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
22 months agocherry-pick upstream fixes for 7.0.0
Fabian Ebner [Mon, 27 Jun 2022 11:05:43 +0000 (13:05 +0200)]
cherry-pick upstream fixes for 7.0.0

coming in via qemu-stable (except for the vdmk fix, which was tagged
for-7.0 on the qemu-devel list, but didn't make it into the release).

Also took the chance to switch the gluster fix to the version that
made it into upstream.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
22 months agod/rules: adapt to changed opensbi riscv filenames in 7.0.0
Fabian Ebner [Mon, 27 Jun 2022 11:05:42 +0000 (13:05 +0200)]
d/rules: adapt to changed opensbi riscv filenames in 7.0.0

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
22 months agod/rules: drop outdated configure flags
Fabian Ebner [Mon, 27 Jun 2022 11:05:41 +0000 (13:05 +0200)]
d/rules: drop outdated configure flags

See QEMU commits 9e8be4c546ce8469ca9702715bf8f198d604b685 and
a5730b8bd3675f484ed0eacea052452048eeb35d for more information.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
22 months agoupdate submodule and patches to 7.0.0
Fabian Ebner [Mon, 27 Jun 2022 11:05:40 +0000 (13:05 +0200)]
update submodule and patches to 7.0.0

Only very minor changes needed:
* Most patches in extra (or some version of them) are part of 7.0.0.
* aio_set_fd_handler got an extra parameter, but can just pass NULL
  like we did for the related 'poll' parameter. See QEMU commit
  826cc32423db2a99d184dbf4f507c737d7e7a4ae for more.
* Add include for qemu/memalign.h in vma.c and vma-writer.c.
* Add reverts for fixups of already reverted 0347a8fd4c ("block/rbd:
  implement bdrv_co_block_status") that came in with 7.0.0. Those
  fixups are not enough, see Proxmox bugzilla #4047.
* Two trivial context changes for bitmap-mirror patches.
* block_int.h got split up into multiple headers.
* Some context changes in configure and meson.build.
* Used the oppurtunity to squash fixup of bdrv_backuo_dump_create typo
  in a later patch into the patch introducing the function (had to
  move code to new header during rebase).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
22 months agobump version to 6.2.0-11
Thomas Lamprecht [Wed, 22 Jun 2022 13:54:58 +0000 (15:54 +0200)]
bump version to 6.2.0-11

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
22 months agovma/alloc-track improvements
Thomas Lamprecht [Wed, 22 Jun 2022 13:52:16 +0000 (15:52 +0200)]
vma/alloc-track improvements

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
22 months agore-export patches in normalized form
Thomas Lamprecht [Wed, 22 Jun 2022 13:47:34 +0000 (15:47 +0200)]
re-export patches in normalized form

iow. using:

git format-patch --zero-commit --no-signature --no-numbered --diff-algorithm=myers ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>