Fix dirty-bitmap PBS backup with multiple drives

"PVE backup: rename incremental to use-dirty-bitmap" merged two
variables (use_dirty_bitmap and incremental) into one, but they served
two different purposes. Rename the original use_dirty_bitmap to
"expect_only_dirty" so the new one doesn't conflict, and rework "PVE:
use proxmox_backup_check_incremental" around that semantic.

In practice, this had the effect that only one disk at a time would
have a bitmap added, as after the first "use_dirty_bitmap" would be set
to one and the rest would behave as if the QMP parameter of the same
name was unset.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

bump version to 5.0.0-10

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

fix backup qmp parameters to pass along encryption info

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 5.0.0-9

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

d/control: update build-dependency of libproxmox-backup-qemu0-dev

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

PBS patches: block driver, adapat encrypt/compress param, add query-proxmox-support QMP cmd

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-8

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fixup: proxmox_backup_check_incremental is negated

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-7

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

update/add PBS integration patches

* rename "incremental" param to "use-dirty-bitmap", avoids confusion
  as the backup can be incrementally also with that param set to
  false.
* use new proxmox_backup_check_incremental
* fix setting dirty counter and adapt to new connect API semantic

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-6

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

pbs: query-backup: set reused field also for dirty-bitmap

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

debian/patches: squash some followup patches and regroup a bit more together

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fixup: qemu submodule should be at v5.0.0

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

backup: improve QAPI info and remove all dirty-bitmaps on failed drive-job

effectively two commits merged as one:
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044185.html
https://pve.proxmox.com/pipermail/pve-devel/2020-July/044194.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

d/control: build depend on newer libproxmox-backup-qemu0-dev

to ensure we have the new 'incremental' parameter for the
proxmox_backup_register_image and proxmox_backup_register_image_async
functions available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

add incremental backup patches

and fix typo: s/BPS/PBS/

fix #2794: Include legacy-igd passthrough fix

See https://bugs.launchpad.net/qemu/+bug/1882784

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

Fix backup for not 64k-aligned storages

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Roland Kammerer <roland.kammerer@linbit.com>

pbs-restore: flush verbose log before calling into library

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

pbs-restore: be more verbose if asked to

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

small cleanups for pbs-restore

Add trailing newline to two error messages, and drop an extra
unconditional `qdict_put_str(options, "driver", format);`
Besides that it's just formatting.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

install pbs-restore to usr/bin

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fix vmstate-snapshots with iothread=1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

add optional buffer size to QEMUFile

and use 4M for our savevm-async buffer size

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

experimentally move savevm-async back into a coroutine

Move qemu_savevm_state_{header,setup} into the main loop and
the rest of the iteration into a coroutine. The former need
to lock the iothread (and we can't unlock it in the
coroutine), and the latter can't deal with being in a
separate thread, so a coroutine it must be.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 5.0.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0~rc3-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

avoid using QemuRecMutex inside coroutines, QemuMutex to lock outside

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0~rc3-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

d/control: build deps; switch libcap to libcap-ng, add libattr

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

pve backup: avoid using recursive mutex

Release the lock on qemu_coroutine_yield, so coroutines don't
deadlock.

Originally-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

rebase patches onto v5.0.0-rc3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 5.0.0~rc2-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

import QEMU 5.0.0-rc2 and rebase patches

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.2.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

patches: move upstream cherry picked to extra folder, like we did in the past

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

add bitmap drive-mirror patches

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

vma_writer: Display more error information

Also print the reason why the function vma_writer_register_stream failed to
help debug errors like in [0].

[0] https://forum.proxmox.com/threads/backup-error-vma_writer_register_stream-drive-scsi0-failed-pve-6-1-7.65925/

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
Reviewed-by: Stefan Reiter <s.reiter@proxmox.com>

slirp: re-add security patches

the first two patches were mistakenly left out during the 4.2 qemu
rebase.

also adds another patch for issue CVE-2019-14378 (heap-based BOF)

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>

fixup backup with io-threads

bump version to 4.2.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

update submodule and patches for 4.2.0

The long overdue nice rebase+cleanup was done by Dietmar

Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.1.1-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

security patches for libslirp CVE-2020-8608

original commits and email can be found here[0]

A out-of-bounds heap buffer access issue was found in the SLiRP
networking implementation of the QEMU emulator. It occurs in tcp_emu()
routine while emulating IRC and other protocols due to unsafe usage of
snprintf(3) function.

A user/process could use this flaw to crash the Qemu process on the host
resulting in DoS or potentially execute arbitrary code with privileges
of the QEMU process on the host.

[0]: https://seclists.org/oss-sec/2020/q1/64

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

add patch for CVE-2019-20382 (vnc disconnect memory leak)

oss-security email can be found here[0]

upstream commit here[1]

this effects our vncproxy. dominik and me tested if the issue is present
on our branch and it appears that it is.
in essence when we disconnect from a vnc connection, the memory isn't
free'd afterwards which causes the qemu process to use more and more
memory with each disconnect, which could lead to a dos scenario.

we tested the patch and it seems to mitigate the problem.

[0]: https://seclists.org/oss-sec/2020/q1/105
[1]: https://git.qemu.org/?p=qemu.git;a=commitdiff;h=6bf21f3d83e95bcc4ba35a7a07cc6655e8b010b0

Tested-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>

fix changelog release

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.1.1-3

improve qemu backup by reducing lock contention

- reducing lock contention by using CoRwLock
- correctly call aio_wait_kick()

followup: fix indentation

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fix hmp info backup command

Signed-off-by: Tim Marx <t.marx@proxmox.com>

correct link to vma spec to new repository url

Signed-off-by: Tim Marx <t.marx@proxmox.com>

grammar fix: s/does not exists/does not exist/g

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.1.1-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fix #2478: record python3-minimal as build dependency

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fix #2486: include fix for disk hot-resize regression (w/ iothread=1)

See: https://lists.nongnu.org/archive/html/qemu-devel/2019-09/msg03201.html

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

fixup patch author

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.1.1-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

allow to pass PVE version to machine

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

handle virtio-balloon cfg size compat with our 4.0 machines

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

update diffs/indexes from patches

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

Update and rebase to QEMU 4.1.1

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

Apply fixups for 4.1

* Fix VMA tool build
* Change PVE code to new blockjob API
* Acquire missing lock for block_job_add_bdrv

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

Update and rebase to QEMU 4.1

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>

bump version to 4.0.1-5

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

fix backup speed regression

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 4.0.1-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

various small pve backup co-routine related fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.0.1-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

backup: add various cleanups

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.0.1-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

run backup related code inside co-routines and improve locking

Patches-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.0.1-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

update sources for v4.0.1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fix #1071: use correct AioContext to cope with IOThreads

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>

Write recognized CPUID flags into static file

located at /usr/share/kvm/recognized-CPUID-flags-x86_64

It's a simple one flag per line list of all flags the build of QEMU
can understand for x86_64 CPUs.

It will be used in qemu-server for the custom CPU model feature.

For now, only x86_64 is implemented, since aarch64 doesn't print any flags when
called this way.

Co-developed-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.0.0-7

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

disable oob again

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 4.0.0-6

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

merge monitor oob fixup

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 4.0.0-5

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

fix #2269: build: adapt to glusterfs changes

build-depend on Buster's re-introduced libglusterfs-dev, and depend on
either libglusterfs0 / libgfFOO (Debian repos) or glusterfs-common
(upstream community repos), until the latter gets fixed upstream.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>

bump version to 4.0.0-4

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

merge fix for CVE-2019-14378

merge: slirp: Fix heap overflow in ip_reass on big packet input

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 4.0.0-3

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

update live migration patches

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>

bump version to 4.0.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

virtio-balloon: use smaller config on older guests

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

fix qemu PKGVERSION

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 4.0.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

buildsys: ensure docs (man pages) are build

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

debian/rules: disable building with SDL integration

This was added once by accident, we do not really need it for Proxmox
VE, it could bring some value for testing but here one can build qemu
oneself too.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

update patches for v4.0.0

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

update qemu submodule to v4.0.0

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

bump version to 3.0.1-62

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>

d/control: add libcurl4-gnutls-dev to build dependencies

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>