]> git.proxmox.com Git - mirror_qemu.git/log
mirror_qemu.git
17 months agotcg/ppc: Reorg tcg_out_tlb_read
Richard Henderson [Mon, 3 Apr 2023 22:25:06 +0000 (01:25 +0300)]
tcg/ppc: Reorg tcg_out_tlb_read

Allocate TCG_REG_TMP2.  Use R0, TMP1, TMP2 instead of any of
the normally allocated registers for the tlb load.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/mips: Simplify constraints on qemu_ld/st
Richard Henderson [Mon, 3 Apr 2023 05:06:53 +0000 (22:06 -0700)]
tcg/mips: Simplify constraints on qemu_ld/st

The softmmu tlb uses TCG_REG_TMP[0-3], not any of the normally available
registers.  Now that we handle overlap betwen inputs and helper arguments,
and have eliminated use of A0, we can allow any allocatable reg.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/mips: Reorg tlb load within prepare_host_addr
Richard Henderson [Mon, 24 Apr 2023 03:29:40 +0000 (04:29 +0100)]
tcg/mips: Reorg tlb load within prepare_host_addr

Compare the address vs the tlb entry with sign-extended values.
This simplifies the page+alignment mask constant, and the
generation of the last byte address for the misaligned test.

Move the tlb addend load up, and the zero-extension down.

This frees up a register, which allows us use TMP3 as the returned base
address register instead of A0, which we were using as a 5th temporary.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/mips: Remove MO_BSWAP handling
Richard Henderson [Sat, 8 Apr 2023 05:15:41 +0000 (06:15 +0100)]
tcg/mips: Remove MO_BSWAP handling

While performing the load in the delay slot of the call to the common
bswap helper function is cute, it is not worth the added complexity.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/loongarch64: Simplify constraints on qemu_ld/st
Richard Henderson [Mon, 3 Apr 2023 17:36:02 +0000 (17:36 +0000)]
tcg/loongarch64: Simplify constraints on qemu_ld/st

The softmmu tlb uses TCG_REG_TMP[0-2], not any of the normally available
registers.  Now that we handle overlap betwen inputs and helper arguments,
we can allow any allocatable reg.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/s390x: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 10 Apr 2023 19:23:08 +0000 (12:23 -0700)]
tcg/s390x: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/riscv: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 10 Apr 2023 18:59:54 +0000 (11:59 -0700)]
tcg/riscv: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/ppc: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 10 Apr 2023 18:09:42 +0000 (11:09 -0700)]
tcg/ppc: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/mips: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 10 Apr 2023 17:30:21 +0000 (10:30 -0700)]
tcg/mips: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.  This allows our local
tcg_out_arg_* infrastructure to be removed.

We are no longer filling the call or return branch
delay slots, nor are we tail-calling for the store,
but this seems a small price to pay.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/loongarch64: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 24 Apr 2023 02:54:06 +0000 (03:54 +0100)]
tcg/loongarch64: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/arm: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 10 Apr 2023 07:15:24 +0000 (00:15 -0700)]
tcg/arm: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.  This allows our local
tcg_out_arg_* infrastructure to be removed.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/aarch64: Convert tcg_out_qemu_{ld,st}_slow_path
Richard Henderson [Mon, 10 Apr 2023 06:21:20 +0000 (23:21 -0700)]
tcg/aarch64: Convert tcg_out_qemu_{ld,st}_slow_path

Use tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/i386: Convert tcg_out_qemu_st_slow_path
Richard Henderson [Mon, 10 Apr 2023 06:08:12 +0000 (23:08 -0700)]
tcg/i386: Convert tcg_out_qemu_st_slow_path

Use tcg_out_st_helper_args.  This eliminates the use of a tail call to
the store helper.  This may or may not be an improvement, depending on
the call/return branch prediction of the host microarchitecture.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/i386: Convert tcg_out_qemu_ld_slow_path
Richard Henderson [Mon, 10 Apr 2023 06:03:55 +0000 (23:03 -0700)]
tcg/i386: Convert tcg_out_qemu_ld_slow_path

Use tcg_out_ld_helper_args and tcg_out_ld_helper_ret.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg: Add routines for calling slow-path helpers
Richard Henderson [Mon, 10 Apr 2023 05:59:09 +0000 (22:59 -0700)]
tcg: Add routines for calling slow-path helpers

Add tcg_out_ld_helper_args, tcg_out_ld_helper_ret,
and tcg_out_st_helper_args.  These and their subroutines
use the existing knowledge of the host function call abi
to load the function call arguments and return results.

These will be used to simplify the backends in turn.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/s390x: Introduce prepare_host_addr
Richard Henderson [Sun, 23 Apr 2023 20:46:46 +0000 (21:46 +0100)]
tcg/s390x: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment,
tcg_prepare_user_ldst, and some code that lived in both tcg_out_qemu_ld
and tcg_out_qemu_st into one function that returns HostAddress and
TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/riscv: Introduce prepare_host_addr
Richard Henderson [Sun, 23 Apr 2023 20:06:38 +0000 (21:06 +0100)]
tcg/riscv: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment,
and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st
into one function that returns TCGReg and TCGLabelQemuLdst.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/ppc: Introduce prepare_host_addr
Richard Henderson [Sun, 23 Apr 2023 19:26:05 +0000 (20:26 +0100)]
tcg/ppc: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment,
and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st
into one function that returns HostAddress and TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/mips: Introduce prepare_host_addr
Richard Henderson [Sun, 23 Apr 2023 17:34:35 +0000 (18:34 +0100)]
tcg/mips: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment,
and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st
into one function that returns HostAddress and TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/loongarch64: Introduce prepare_host_addr
Richard Henderson [Sun, 23 Apr 2023 16:02:44 +0000 (17:02 +0100)]
tcg/loongarch64: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment,
tcg_out_zext_addr_if_32_bit, and some code that lived in both
tcg_out_qemu_ld and tcg_out_qemu_st into one function that returns
HostAddress and TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/arm: Introduce prepare_host_addr
Richard Henderson [Sun, 23 Apr 2023 14:54:41 +0000 (15:54 +0100)]
tcg/arm: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, and some code that lived
in both tcg_out_qemu_ld and tcg_out_qemu_st into one function that
returns HostAddress and TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/aarch64: Introduce prepare_host_addr
Richard Henderson [Fri, 21 Apr 2023 11:37:20 +0000 (12:37 +0100)]
tcg/aarch64: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label, tcg_out_test_alignment,
and some code that lived in both tcg_out_qemu_ld and tcg_out_qemu_st
into one function that returns HostAddress and TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/i386: Use indexed addressing for softmmu fast path
Richard Henderson [Sun, 16 Apr 2023 07:00:18 +0000 (09:00 +0200)]
tcg/i386: Use indexed addressing for softmmu fast path

Since tcg_out_{ld,st}_helper_args, the slow path no longer requires
the address argument to be set up by the tlb load sequence.  Use a
plain load for the addend and indexed addressing with the original
input address register.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotcg/i386: Introduce prepare_host_addr
Richard Henderson [Wed, 19 Apr 2023 17:10:27 +0000 (19:10 +0200)]
tcg/i386: Introduce prepare_host_addr

Merge tcg_out_tlb_load, add_qemu_ldst_label,
tcg_out_test_alignment, and some code that lived in both
tcg_out_qemu_ld and tcg_out_qemu_st into one function
that returns HostAddress and TCGLabelQemuLdst structures.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agoaccel/tcg/tcg-accel-ops-rr: ensure fairness with icount
Jamie Iles [Thu, 27 Apr 2023 02:09:25 +0000 (03:09 +0100)]
accel/tcg/tcg-accel-ops-rr: ensure fairness with icount

The round-robin scheduler will iterate over the CPU list with an
assigned budget until the next timer expiry and may exit early because
of a TB exit.  This is fine under normal operation but with icount
enabled and SMP it is possible for a CPU to be starved of run time and
the system live-locks.

For example, booting a riscv64 platform with '-icount
shift=0,align=off,sleep=on -smp 2' we observe a livelock once the kernel
has timers enabled and starts performing TLB shootdowns.  In this case
we have CPU 0 in M-mode with interrupts disabled sending an IPI to CPU
1.  As we enter the TCG loop, we assign the icount budget to next timer
interrupt to CPU 0 and begin executing where the guest is sat in a busy
loop exhausting all of the budget before we try to execute CPU 1 which
is the target of the IPI but CPU 1 is left with no budget with which to
execute and the process repeats.

We try here to add some fairness by splitting the budget across all of
the CPUs on the thread fairly before entering each one.  The CPU count
is cached on CPU list generation ID to avoid iterating the list on each
loop iteration.  With this change it is possible to boot an SMP rv64
guest with icount enabled and no hangs.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230427020925.51003-3-quic_jiles@quicinc.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agocpu: expose qemu_cpu_list_lock for lock-guard use
Jamie Iles [Thu, 27 Apr 2023 02:09:24 +0000 (03:09 +0100)]
cpu: expose qemu_cpu_list_lock for lock-guard use

Expose qemu_cpu_list_lock globally so that we can use
WITH_QEMU_LOCK_GUARD and QEMU_LOCK_GUARD to simplify a few code paths
now and in future.

Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230427020925.51003-2-quic_jiles@quicinc.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agodisas: Move disas.c into the target-independent source set
Thomas Huth [Tue, 9 May 2023 16:27:48 +0000 (17:27 +0100)]
disas: Move disas.c into the target-independent source set

Use target_words_bigendian() instead of an ifdef.

Remove CONFIG_RISCV_DIS from the check for riscv as a host; this is
a poisoned identifier, and anyway will always be set by meson.build
when building on a riscv host.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230508133745.109463-3-thuth@redhat.com>
[rth: Type change done in a separate patch]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agodisas: Move softmmu specific code to separate file
Thomas Huth [Tue, 9 May 2023 16:14:36 +0000 (17:14 +0100)]
disas: Move softmmu specific code to separate file

We'd like to move disas.c into the common code source set, where
CONFIG_USER_ONLY is not available anymore. So we have to move
the related code into a separate file instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230508133745.109463-2-thuth@redhat.com>
[rth: Type change done in a separate patch]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agodisas: Remove target-specific headers
Richard Henderson [Wed, 3 May 2023 07:23:29 +0000 (08:23 +0100)]
disas: Remove target-specific headers

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503072331.1747057-83-richard.henderson@linaro.org>

17 months agodisas: Remove target_ulong from the interface
Richard Henderson [Wed, 3 May 2023 07:23:27 +0000 (08:23 +0100)]
disas: Remove target_ulong from the interface

Use uint64_t for the pc, and size_t for the size.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503072331.1747057-81-richard.henderson@linaro.org>

17 months agodisas: Move disas.c to disas/
Richard Henderson [Wed, 3 May 2023 07:23:26 +0000 (08:23 +0100)]
disas: Move disas.c to disas/

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503072331.1747057-80-richard.henderson@linaro.org>

17 months agodisas: Fix tabs and braces in disas.c
Richard Henderson [Wed, 10 May 2023 17:04:45 +0000 (18:04 +0100)]
disas: Fix tabs and braces in disas.c

Fix these before moving the file, for checkpatch.pl.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230510170812.663149-1-richard.henderson@linaro.org>

17 months agoaccel/tcg: Fix atomic_mmu_lookup for reads
Richard Henderson [Fri, 5 May 2023 20:40:49 +0000 (21:40 +0100)]
accel/tcg: Fix atomic_mmu_lookup for reads

A copy-paste bug had us looking at the victim cache for writes.

Cc: qemu-stable@nongnu.org
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Fixes: 08dff435e2 ("tcg: Probe the proper permissions for atomic ops")
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20230505204049.352469-1-richard.henderson@linaro.org>

17 months agotarget/m68k: Fix gen_load_fp for OS_LONG
Richard Henderson [Mon, 8 May 2023 14:07:47 +0000 (15:07 +0100)]
target/m68k: Fix gen_load_fp for OS_LONG

Case was accidentally dropped in b7a94da9550b.

Tested-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agoMerge tag 'migration-20230509-pull-request' of https://gitlab.com/juan.quintela/qemu...
Richard Henderson [Thu, 11 May 2023 04:55:12 +0000 (05:55 +0100)]
Merge tag 'migration-20230509-pull-request' of https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request (20230509 vintage) take 2

Hi

In this take 2:
- Change uint -> uint32_t to fix mingw32 compilation.

Please apply.
[take 1]
In this PULL request:
- 1st part of colo support for multifd (lukas)
- 1st part of disabling colo option (vladimir)

Please, apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmRb3dgACgkQ9IfvGFhy
# 1yNLBxAAwHiAOdSPS7TqJXH2/PkBKsd42XMtWzC9UowZ6SUdQi0Q2bQUBnygJ8BA
# 59yLOTPdwUhaPWk4KsyKM2znOCJ+f9MF5V4QXbyILf1WCAq6d+mtPwArnYF1TRwi
# XIewVDeRopdOO5lnWGcfAKZZ5WIDzA/bn6NiGLi+pQa5HGyk84Bk+tFa8kJI6xBL
# 5CWfhNTcxDNYRFg/z/9YVirkuxIXEEL6VEeRFV+pmFuj05q9bysWJkLFoEcFNawO
# gp1foHDkU7wHmHDJ3D4AVTm3TW641ft1wdlHIHZRoOiIIu3EUOoDEVVsaCfdxrY8
# pPJZ5m37wb52GIaCJmigG8rkHxIJ8xKLk4HKu4umDqFq5jZQ2krnnj7AkQhpp7p2
# aEIOXJQQq7XCsKpuvSUIexPv4gbN5SEYKi7XKoOPe3sZ03Rkn0I5xY3KSyMQMamP
# jtk8tNlRA+9Wug82eb/FtIKDj3//4SbuQOJEdRXjKJBldd3mtWTT/FRj/8oo96/p
# hmTu/cGDrP5qgtWpz0kKI/xaBf8at1nwpDgdEzOjRw4zf6xQHFjbXgJ7tQBH/JUI
# T3A9pdiXN6QdRupcWUSV0iJsfS/5i3mOUTA/C529qGXabSnZzfMK+unL/I8N02yt
# 83o7jSg22etMjaS1c+VuDmzKCAfuZloDZv2Bms/+yM/8k8Xe5S4=
# =vbqf
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 10 May 2023 07:09:28 PM BST
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [undefined]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* tag 'migration-20230509-pull-request' of https://gitlab.com/juan.quintela/qemu:
  migration: block incoming colo when capability is disabled
  migration: disallow change capabilities in COLO state
  migration: process_incoming_migration_co: simplify code flow around ret
  migration: drop colo_incoming_thread from MigrationIncomingState
  build: move COLO under CONFIG_REPLICATION
  colo: make colo_checkpoint_notify static and provide simpler API
  block/meson.build: prefer positive condition for replication
  multifd: Add the ramblock to MultiFDRecvParams
  ram: Let colo_flush_ram_cache take the bitmap_mutex
  ram: Add public helper to set colo bitmap

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agomigration: block incoming colo when capability is disabled
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:28 +0000 (22:49 +0300)]
migration: block incoming colo when capability is disabled

We generally require same set of capabilities on source and target.
Let's require x-colo capability to use COLO on target.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20230428194928.1426370-11-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agomigration: disallow change capabilities in COLO state
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:27 +0000 (22:49 +0300)]
migration: disallow change capabilities in COLO state

COLO is not listed as running state in migrate_is_running(), so, it's
theoretically possible to disable colo capability in COLO state and the
unexpected error in migration_iteration_finish() is reachable.

Let's disallow that in qmp_migrate_set_capabilities. Than the error
becomes absolutely unreachable: we can get into COLO state only with
enabled capability and can't disable it while we are in COLO state. So
substitute the error by simple assertion.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20230428194928.1426370-10-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agomigration: process_incoming_migration_co: simplify code flow around ret
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:24 +0000 (22:49 +0300)]
migration: process_incoming_migration_co: simplify code flow around ret

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20230428194928.1426370-7-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agomigration: drop colo_incoming_thread from MigrationIncomingState
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:23 +0000 (22:49 +0300)]
migration: drop colo_incoming_thread from MigrationIncomingState

have_colo_incoming_thread variable is unused. colo_incoming_thread can
be local.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20230428194928.1426370-6-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agobuild: move COLO under CONFIG_REPLICATION
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:21 +0000 (22:49 +0300)]
build: move COLO under CONFIG_REPLICATION

We don't allow to use x-colo capability when replication is not
configured. So, no reason to build COLO when replication is disabled,
it's unusable in this case.

Note also that the check in migrate_caps_check() is not the only
restriction: some functions in migration/colo.c will just abort if
called with not defined CONFIG_REPLICATION, for example:

    migration_iteration_finish()
       case MIGRATION_STATUS_COLO:
           migrate_start_colo_process()
               colo_process_checkpoint()
                   abort()

It could probably make sense to have possibility to enable COLO without
REPLICATION, but this requires deeper audit of colo & replication code,
which may be done later if needed.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Acked-by: Dr. David Alan Gilbert <dave@treblig.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230428194928.1426370-4-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agocolo: make colo_checkpoint_notify static and provide simpler API
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:20 +0000 (22:49 +0300)]
colo: make colo_checkpoint_notify static and provide simpler API

colo_checkpoint_notify() is mostly used in colo.c. Outside we use it
once when x-checkpoint-delay migration parameter is set. So, let's
simplify the external API to only that function - notify COLO that
parameter was set. This make external API more robust and hides
implementation details from external callers. Also this helps us to
make COLO module optional in further patch (i.e. we are going to add
possibility not build the COLO module).

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20230428194928.1426370-3-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agoblock/meson.build: prefer positive condition for replication
Vladimir Sementsov-Ogievskiy [Fri, 28 Apr 2023 19:49:19 +0000 (22:49 +0300)]
block/meson.build: prefer positive condition for replication

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Zhang Chen <chen.zhang@intel.com>
Message-Id: <20230428194928.1426370-2-vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agomultifd: Add the ramblock to MultiFDRecvParams
Lukas Straub [Mon, 8 May 2023 19:11:07 +0000 (21:11 +0200)]
multifd: Add the ramblock to MultiFDRecvParams

This will be used in the next commits to add colo support to multifd.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <88135197411df1a71d7832962b39abf60faf0021.1683572883.git.lukasstraub2@web.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agoram: Let colo_flush_ram_cache take the bitmap_mutex
Lukas Straub [Mon, 8 May 2023 19:10:55 +0000 (21:10 +0200)]
ram: Let colo_flush_ram_cache take the bitmap_mutex

This is not required, colo_flush_ram_cache does not run concurrently
with the multifd threads since the cache is only flushed after
everything has been received. But it makes me more comfortable.

This will be used in the next commits to add colo support to multifd.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <35cb23ba854151d38a31e3a5c8a1020e4283cb4a.1683572883.git.lukasstraub2@web.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agoram: Add public helper to set colo bitmap
Lukas Straub [Mon, 8 May 2023 19:10:52 +0000 (21:10 +0200)]
ram: Add public helper to set colo bitmap

The overhead of the mutex in non-multifd mode is negligible,
because in that case its just the single thread taking the mutex.

This will be used in the next commits to add colo support to multifd.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <22d83cb428f37929563155531bfb69fd8953cc61.1683572883.git.lukasstraub2@web.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
17 months agoMerge tag 'pull-testing-updates-100523-1' of https://gitlab.com/stsquad/qemu into...
Richard Henderson [Wed, 10 May 2023 15:43:01 +0000 (16:43 +0100)]
Merge tag 'pull-testing-updates-100523-1' of https://gitlab.com/stsquad/qemu into staging

Testing updates:

  - fix up xtensa docker container base to current Debian
  - document breakpoint and watchpoint support
  - clean up the ansible scripts for Ubuntu 22.04
  - add a minimal device profile
  - drop https on mipsdistros URL
  - fix Kconfig bug for XLNX_VERSAL

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmRbspsACgkQ+9DbCVqe
# KkSBowf+JjcVxZMb2kS8pV8WEdAq+fceBYI7mDBSEu0DFqZF+w0XSM+T+VZHyZ8+
# QmPeE+McKBUXvq/V4osPnDVVZfBKmwzFN548M6qIMLUbHjbDp94DtudNkAZ0ejhc
# +Ack73vzTiTWsGmBaqQxZlcYkZNZiZAhQsTF6cPwna74cDkcRghvd/Zxzy831rVB
# gVWhbEkk7SBQhJ+PqRIeso60DbWvCaVDMrkPc2WX8kup6QltbUpoayS/eNOtBkfA
# C557eOBxoM8s0cu33O780K5mCPCyk1IaIynvZtmkty0DXUSd5y9SNpsofhAY7BGy
# 4QdlolLygDgEC3s4bMULGy04nzaylw==
# =a+97
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 10 May 2023 04:04:59 PM BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-testing-updates-100523-1' of https://gitlab.com/stsquad/qemu:
  hw/arm: Select XLNX_USB_SUBSYS for xlnx-zcu102 machine
  tests/avocado: use http for mipsdistros.mips.com
  gitlab: enable minimal device profile for aarch64 --disable-tcg
  gitlab: add ubuntu-22.04-aarch64-without-defaults
  scripts/ci: clean-up the 20.04/22.04 confusion in ansible
  scripts/ci: add gitlab-runner to kvm group
  docs: document breakpoint and watchpoint support
  tests/docker: bump the xtensa base to debian:11-slim

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agohw/arm: Select XLNX_USB_SUBSYS for xlnx-zcu102 machine
Fabiano Rosas [Wed, 3 May 2023 09:12:29 +0000 (10:12 +0100)]
hw/arm: Select XLNX_USB_SUBSYS for xlnx-zcu102 machine

This machine hardcodes initialization of the USB device, so select the
corresponding Kconfig. It is not enough to have it as "default y if
XLNX_VERSAL" at usb/Kconfig because building --without-default-devices
disables the default selection resulting in:

$ ./qemu-system-aarch64 -M xlnx-zcu102
qemu-system-aarch64: missing object type 'usb_dwc3'
Aborted (core dumped)

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230208192654.8854-8-farosas@suse.de>
Message-Id: <20230503091244.1450613-8-alex.bennee@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotests/avocado: use http for mipsdistros.mips.com
Alex Bennée [Tue, 2 May 2023 16:54:03 +0000 (17:54 +0100)]
tests/avocado: use http for mipsdistros.mips.com

As the cached assets have fallen out of our cache new attempts to
fetch these binaries fail hard due to certificate expiry. It's hard
to find a contact email for the domain as the root page of mipsdistros
throws up some random XML. I suspect Amazon are merely the hosts.

The checksums should protect us from any man-in-the-middle type
attacks.

Message-Id: <20230503091244.1450613-22-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
17 months agogitlab: enable minimal device profile for aarch64 --disable-tcg
Alex Bennée [Tue, 2 May 2023 16:20:46 +0000 (17:20 +0100)]
gitlab: enable minimal device profile for aarch64 --disable-tcg

As this is likely the most common configuration people will want once
the --disable-tcg patches land.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Fabiano Rosas <farosas@suse.de>
Message-Id: <20230503091244.1450613-21-alex.bennee@linaro.org>

17 months agogitlab: add ubuntu-22.04-aarch64-without-defaults
Alex Bennée [Tue, 2 May 2023 15:59:17 +0000 (16:59 +0100)]
gitlab: add ubuntu-22.04-aarch64-without-defaults

This does a very minimal build without default devices or features. I
chose the aarch64 runner as it doesn't count towards CI minutes and is
a fairly under-utilised builder.

Message-Id: <20230503091244.1450613-20-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
17 months agoscripts/ci: clean-up the 20.04/22.04 confusion in ansible
Alex Bennée [Tue, 2 May 2023 15:54:54 +0000 (16:54 +0100)]
scripts/ci: clean-up the 20.04/22.04 confusion in ansible

We have a bunch of references to 20.04 (which s390x is still on)
although we are basically building on 22.04 now. Clean up the textual
references and use lcitool to generate the full package list to be
consistent.

We can drop "Install packages to build QEMU on Ubuntu on non-s390x" as
when we upgrade the s390x builder to 22.04 it won't need this
workaround.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230503091244.1450613-19-alex.bennee@linaro.org>

17 months agoscripts/ci: add gitlab-runner to kvm group
Alex Bennée [Tue, 2 May 2023 15:11:50 +0000 (16:11 +0100)]
scripts/ci: add gitlab-runner to kvm group

One of the main reasons to have custom runners it so we can run KVM
tests. Enable the "kvm" additional group so we can access the feature
on the kernel.

Message-Id: <20230503091244.1450613-5-alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
17 months agodocs: document breakpoint and watchpoint support
Alex Bennée [Tue, 2 May 2023 14:23:50 +0000 (15:23 +0100)]
docs: document breakpoint and watchpoint support

This varies by accelerator. Also mention the modern bear trap that is
ASLR.

Message-Id: <20230503091244.1450613-4-alex.bennee@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
17 months agotests/docker: bump the xtensa base to debian:11-slim
Alex Bennée [Tue, 2 May 2023 14:20:59 +0000 (15:20 +0100)]
tests/docker: bump the xtensa base to debian:11-slim

Stretch is going out of support so things like security updates will
fail. As the toolchain itself is binary it hopefully won't mind the
underlying OS being updated.

Message-Id: <20230503091244.1450613-3-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Richard Henderson <richard.henderson@linaro.org>
17 months agoMerge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging
Richard Henderson [Wed, 10 May 2023 13:52:03 +0000 (14:52 +0100)]
Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging

Block layer patches

- Graph locking, part 3 (more block drivers)
- Compile out assert_bdrv_graph_readable() by default
- Add configure options for vmdk, vhdx and vpc
- Fix use after free in blockdev_mark_auto_del()
- migration: Attempt disk reactivation in more failure scenarios
- Coroutine correctness fixes

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmRbi6ERHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9Y66A//ZRk/0M6EZUJPAKG6m/XLTDNrOCNBZ1Tu
# kBGvxXsVQZMt4gGpBad4l2INN6IQKTIdIf+lK71EpxMPmFG6xK32btn38yywCAfQ
# lr1p5nR0Y/zSlT+XzP4yKy/CtQl6U0rkysmjCIk35bZc7uLy6eo4oFR4vmhRRt2M
# UGltB50/Nicx12YFufVjodbhv+apxTGwS2XHatmwqtjKeYReSz8mJHslEy6DvC8m
# ziNThD6YBy7hMktAhNaqUqtZD0OSWz66VMObco/4i2++sOAMZIspXQkjv3AjH74e
# lmgMhNc/xgJKPwFBPsj6F7dOKxwhdKD9jzZlx3yaBtAU18hpWX54QWuA3/CFlySc
# 5QbbqIstFTC8lqoRWThQrcHHRKbDBJCP4ImRXUIKhuPaxEzXA9zb3+f3QPTIjLSA
# KO7nxuSmO+tC7hQ1K9kAjRZHWlxxAk4clk+7UrK4UrWgGxfCUKgFg4Tyx7RrpwA6
# j4L5vwAY60LW74tikWe9xJx2QbdRoWBTTZhUyirbO7rLX1e8mS1nUWmtIsFSQxAq
# Z7nX7ygN0WEF+8qIsk3jTGaEeJoCM7+7B+X2RpSy0sftFjFYmybIiUgLMO7e+ozK
# rvUPnwlHAbGCVIJOKrUDj3cGt6k3/xnrTajUc7pCB3KKqG4pe+IlZuHyKIUMActb
# dBLaBnj0M2o=
# =hw9E
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 10 May 2023 01:18:41 PM BST
# gpg:                using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg:                issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]

* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (28 commits)
  block: compile out assert_bdrv_graph_readable() by default
  block: Mark bdrv_refresh_limits() and callers GRAPH_RDLOCK
  block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK
  block: Mark bdrv_query_block_graph_info() and callers GRAPH_RDLOCK
  block: Mark bdrv_query_bds_stats() and callers GRAPH_RDLOCK
  block: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK
  block: Mark bdrv_co_debug_event() GRAPH_RDLOCK
  block: Mark bdrv_co_get_info() and callers GRAPH_RDLOCK
  block: Mark bdrv_co_get_allocated_file_size() and callers GRAPH_RDLOCK
  mirror: Require GRAPH_RDLOCK for accessing a node's parent list
  vhdx: Require GRAPH_RDLOCK for accessing a node's parent list
  nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK
  nbd: Remove nbd_co_flush() wrapper function
  block: .bdrv_open is non-coroutine and unlocked
  graph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock
  graph-lock: Add GRAPH_UNLOCKED(_PTR)
  test-bdrv-drain: Don't modify the graph in coroutines
  iotests: Test resizing image attached to an iothread
  block: Don't call no_coroutine_fns in qmp_block_resize()
  block: bdrv/blk_co_unref() for calls in coroutine context
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agoblock: compile out assert_bdrv_graph_readable() by default
Stefan Hajnoczi [Mon, 1 May 2023 17:34:43 +0000 (13:34 -0400)]
block: compile out assert_bdrv_graph_readable() by default

reader_count() is a performance bottleneck because the global
aio_context_list_lock mutex causes thread contention. Put this debugging
assertion behind a new ./configure --enable-debug-graph-lock option and
disable it by default.

The --enable-debug-graph-lock option is also enabled by the more general
--enable-debug option.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230501173443.153062-1-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_refresh_limits() and callers GRAPH_RDLOCK
Kevin Wolf [Thu, 4 May 2023 11:57:50 +0000 (13:57 +0200)]
block: Mark bdrv_refresh_limits() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_refresh_limits() need to hold a reader lock for the graph because
it accesses the children list of a node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-21-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK
Kevin Wolf [Thu, 4 May 2023 11:57:49 +0000 (13:57 +0200)]
block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_recurse_can_replace() need to hold a reader lock for the graph
because it accesses the children list of a node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-20-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_query_block_graph_info() and callers GRAPH_RDLOCK
Kevin Wolf [Thu, 4 May 2023 11:57:48 +0000 (13:57 +0200)]
block: Mark bdrv_query_block_graph_info() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_query_block_graph_info() need to hold a reader lock for the graph
because it accesses the children list of a node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-19-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_query_bds_stats() and callers GRAPH_RDLOCK
Kevin Wolf [Thu, 4 May 2023 11:57:47 +0000 (13:57 +0200)]
block: Mark bdrv_query_bds_stats() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_query_bds_stats() need to hold a reader lock for the graph because
it accesses the children list of a node.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-18-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK
Emanuele Giuseppe Esposito [Thu, 4 May 2023 11:57:46 +0000 (13:57 +0200)]
block: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of amend
callbacks in BlockDriver need to hold a reader lock for the graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-17-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_co_debug_event() GRAPH_RDLOCK
Emanuele Giuseppe Esposito [Thu, 4 May 2023 11:57:45 +0000 (13:57 +0200)]
block: Mark bdrv_co_debug_event() GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_debug_event() need to hold a reader lock for the graph.

Unfortunately we cannot use a co_wrapper_bdrv_rdlock (i.e. make the
coroutine wrapper a no_coroutine_fn), because the function is called
(using the BLKDBG_EVENT macro) by mixed functions that run both in
coroutine and non-coroutine context (for example many of the functions
in qcow2-cluster.c and qcow2-refcount.c).

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-16-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_co_get_info() and callers GRAPH_RDLOCK
Emanuele Giuseppe Esposito [Thu, 4 May 2023 11:57:44 +0000 (13:57 +0200)]
block: Mark bdrv_co_get_info() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_get_info() need to hold a reader lock for the graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-15-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Mark bdrv_co_get_allocated_file_size() and callers GRAPH_RDLOCK
Emanuele Giuseppe Esposito [Thu, 4 May 2023 11:57:43 +0000 (13:57 +0200)]
block: Mark bdrv_co_get_allocated_file_size() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
bdrv_co_get_allocated_file_size() need to hold a reader lock for the
graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230504115750.54437-14-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agomirror: Require GRAPH_RDLOCK for accessing a node's parent list
Kevin Wolf [Thu, 4 May 2023 11:57:42 +0000 (13:57 +0200)]
mirror: Require GRAPH_RDLOCK for accessing a node's parent list

This adds GRAPH_RDLOCK annotations to declare that functions accessing
the parent list of a node need to hold a reader lock for the graph. As
it happens, they already do.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230504115750.54437-13-kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agovhdx: Require GRAPH_RDLOCK for accessing a node's parent list
Kevin Wolf [Thu, 4 May 2023 11:57:41 +0000 (13:57 +0200)]
vhdx: Require GRAPH_RDLOCK for accessing a node's parent list

This adds GRAPH_RDLOCK annotations to declare that functions accessing
the parent list of a node need to hold a reader lock for the graph. As
it happens, they already do.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-12-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agonbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK
Emanuele Giuseppe Esposito [Thu, 4 May 2023 11:57:40 +0000 (13:57 +0200)]
nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK

This adds GRAPH_RDLOCK annotations to declare that callers of
nbd_co_do_establish_connection() need to hold a reader lock for the
graph.

Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-11-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agonbd: Remove nbd_co_flush() wrapper function
Kevin Wolf [Thu, 4 May 2023 11:57:39 +0000 (13:57 +0200)]
nbd: Remove nbd_co_flush() wrapper function

The only thing nbd_co_flush() does is call nbd_client_co_flush(). Just
use that function directly in the BlockDriver definitions and remove the
wrapper.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-10-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: .bdrv_open is non-coroutine and unlocked
Kevin Wolf [Thu, 4 May 2023 11:57:38 +0000 (13:57 +0200)]
block: .bdrv_open is non-coroutine and unlocked

Drivers were a bit confused about whether .bdrv_open can run in a
coroutine and whether or not it holds a graph lock.

It cannot keep a graph lock from the caller across the whole function
because it both changes the graph (requires a writer lock) and does I/O
(requires a reader lock). Therefore, it should take these locks
internally as needed.

The functions used to be called in coroutine context during image
creation. This was buggy for other reasons, and as of commit 32192301,
all block drivers go through no_co_wrappers. So it is not called in
coroutine context any more.

Fix qcow2 and qed to work with the correct assumptions: The graph lock
needs to be taken internally instead of just assuming it's already
there, and the coroutine path is dead code that can be removed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-9-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agograph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock
Kevin Wolf [Thu, 4 May 2023 11:57:37 +0000 (13:57 +0200)]
graph-lock: Fix GRAPH_RDLOCK_GUARD*() to be reader lock

GRAPH_RDLOCK_GUARD() and GRAPH_RDLOCK_GUARD_MAINLOOP() only take a
reader lock for the graph, so the correct annotation for them to use is
TSA_ASSERT_SHARED rather than TSA_ASSERT.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230504115750.54437-8-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agograph-lock: Add GRAPH_UNLOCKED(_PTR)
Kevin Wolf [Thu, 4 May 2023 11:57:36 +0000 (13:57 +0200)]
graph-lock: Add GRAPH_UNLOCKED(_PTR)

For some functions, it is part of their interface to be called without
holding the graph lock. Add a new macro to document this.

The macro expands to TSA_EXCLUDES(), which is a relatively weak check
because it passes in cases where the compiler just doesn't know if the
lock is held. Function pointers can't be checked at all. Therefore, its
primary purpose is documentation.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-7-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agotest-bdrv-drain: Don't modify the graph in coroutines
Kevin Wolf [Thu, 4 May 2023 11:57:35 +0000 (13:57 +0200)]
test-bdrv-drain: Don't modify the graph in coroutines

test-bdrv-drain contains a few test cases that are run both in coroutine
and non-coroutine context. Running the entire code including the setup
and shutdown in coroutines is incorrect because graph modifications can
generally not happen in coroutines.

Change the test so that creating and destroying the test nodes and
BlockBackends always happens outside of coroutine context.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230504115750.54437-6-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoiotests: Test resizing image attached to an iothread
Kevin Wolf [Tue, 9 May 2023 13:41:33 +0000 (15:41 +0200)]
iotests: Test resizing image attached to an iothread

This tests that trying to resize an image with QMP block_resize doesn't
hang or otherwise fail when the image is attached to a device running in
an iothread.

This is a regression test for the recent fix that changed
qmp_block_resize, which is a coroutine based QMP handler, to avoid
calling no_coroutine_fns directly.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230509134133.373408-1-kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Don't call no_coroutine_fns in qmp_block_resize()
Kevin Wolf [Thu, 4 May 2023 11:57:34 +0000 (13:57 +0200)]
block: Don't call no_coroutine_fns in qmp_block_resize()

This QMP handler runs in a coroutine, so it must use the corresponding
no_co_wrappers instead.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2185688
Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-5-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: bdrv/blk_co_unref() for calls in coroutine context
Kevin Wolf [Thu, 4 May 2023 11:57:33 +0000 (13:57 +0200)]
block: bdrv/blk_co_unref() for calls in coroutine context

These functions must not be called in coroutine context, because they
need write access to the graph.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-4-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Consistently call bdrv_activate() outside coroutine
Kevin Wolf [Thu, 4 May 2023 11:57:32 +0000 (13:57 +0200)]
block: Consistently call bdrv_activate() outside coroutine

Migration code can call bdrv_activate() in coroutine context, whereas
other callers call it outside of coroutines. As it calls other code that
is not supposed to run in coroutines, standardise on running outside of
coroutines.

This adds a no_co_wrapper to switch to the main loop before calling
bdrv_activate().

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-3-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoqcow2: Don't call bdrv_getlength() in coroutine_fns
Kevin Wolf [Thu, 4 May 2023 11:57:31 +0000 (13:57 +0200)]
qcow2: Don't call bdrv_getlength() in coroutine_fns

There is a bdrv_co_getlength() now, which should be used in coroutine
context.

This requires adding GRAPH_RDLOCK to some functions so that this still
compiles with TSA because bdrv_co_getlength() is GRAPH_RDLOCK.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230504115750.54437-2-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agomigration: Attempt disk reactivation in more failure scenarios
Eric Blake [Tue, 2 May 2023 20:52:12 +0000 (15:52 -0500)]
migration: Attempt disk reactivation in more failure scenarios

Commit fe904ea824 added a fail_inactivate label, which tries to
reactivate disks on the source after a failure while s->state ==
MIGRATION_STATUS_ACTIVE, but didn't actually use the label if
qemu_savevm_state_complete_precopy() failed.  This failure to
reactivate is also present in commit 6039dd5b1c (also covering the new
s->state == MIGRATION_STATUS_DEVICE state) and 403d18ae (ensuring
s->block_inactive is set more reliably).

Consolidate the two labels back into one - no matter HOW migration is
failed, if there is any chance we can reach vm_start() after having
attempted inactivation, it is essential that we have tried to restart
disks before then.  This also makes the cleanup more like
migrate_fd_cancel().

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230502205212.134680-1-eblake@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoiotests/nbd-reconnect-on-open: Fix NBD socket path
Kevin Wolf [Wed, 3 May 2023 16:50:19 +0000 (18:50 +0200)]
iotests/nbd-reconnect-on-open: Fix NBD socket path

Socket paths need to be short to avoid failures. This is why there is a
iotests.sock_dir (defaulting to /tmp) separate from the disk image base
directory.

Make use of it to fix failures in too deeply nested test directories.

Fixes: ab7f7e67a7e7b49964109501dfcde4ec29bae60e
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230503165019.8867-1-kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: Fix use after free in blockdev_mark_auto_del()
Kevin Wolf [Wed, 3 May 2023 14:01:42 +0000 (16:01 +0200)]
block: Fix use after free in blockdev_mark_auto_del()

job_cancel_locked() drops the job list lock temporarily and it may call
aio_poll(). We must assume that the list has changed after this call.
Also, with unlucky timing, it can end up freeing the job during
job_completed_txn_abort_locked(), making the job pointer invalid, too.

For both reasons, we can't just continue at block_job_next_locked(job).
Instead, start at the head of the list again after job_cancel_locked()
and skip those jobs that we already cancelled (or that are completing
anyway).

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230503140142.474404-1-kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoaio-wait: avoid AioContext lock in aio_wait_bh_oneshot()
Stefan Hajnoczi [Tue, 4 Apr 2023 15:33:07 +0000 (11:33 -0400)]
aio-wait: avoid AioContext lock in aio_wait_bh_oneshot()

There is no need for the AioContext lock in aio_wait_bh_oneshot().
It's easy to remove the lock from existing callers and then switch from
AIO_WAIT_WHILE() to AIO_WAIT_WHILE_UNLOCKED() in aio_wait_bh_oneshot().

Document that the AioContext lock should not be held across
aio_wait_bh_oneshot(). Holding a lock across aio_poll() can cause
deadlock so we don't want callers to do that.

This is a step towards getting rid of the AioContext lock.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230404153307.458883-1-stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: add missing coroutine_fn annotations
Paolo Bonzini [Thu, 6 Apr 2023 10:17:52 +0000 (12:17 +0200)]
block: add missing coroutine_fn annotations

After the recent introduction of many new coroutine callbacks,
a couple calls from non-coroutine_fn to coroutine_fn have sneaked
in; fix them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20230406101752.242125-1-pbonzini@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoblock: add configure options for excluding vmdk, vhdx and vpc
Vladimir Sementsov-Ogievskiy [Fri, 21 Apr 2023 09:27:58 +0000 (12:27 +0300)]
block: add configure options for excluding vmdk, vhdx and vpc

Let's add --enable / --disable configure options for these formats,
so that those who don't need them may not build them.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <20230421092758.814122-1-vsementsov@yandex-team.ru>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
17 months agoMerge tag 'pull-qapi-2023-05-09-v2' of https://repo.or.cz/qemu/armbru into staging
Richard Henderson [Wed, 10 May 2023 12:11:29 +0000 (13:11 +0100)]
Merge tag 'pull-qapi-2023-05-09-v2' of https://repo.or.cz/qemu/armbru into staging

QAPI patches patches for 2023-05-09

# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmRbUEYSHGFybWJydUBy
# ZWRoYXQuY29tAAoJEDhwtADrkYZTmzEP/3pDpVxpP7xXLevl2vFqkFyHEjc0L3N4
# x//ljgQojAdM6WU3e0qqOfp/NE2ktUg5D3z+QNiVP1/xXv/dtMGATdG+X9AZs0US
# XnhdicYdBng8bGuhlNuNY8QJ/I4ALwUR44LVOYibVohv2RVYWBapGiHowpyTyABq
# sFSHrj/cgvTMUn53yp7veZTo6rWG6RU/D5uUTOMsvKeAoHoOXMyBxV01SCt84t/J
# pcelINcriP6cQVzgfm1B39UNa0IxinGxEx/IIaxz5Ju66G05HTs4CsBFAF6/0QI/
# 3YerGWPt9fF6+qYNn21Gg9CL1fHHppNqTXkcuTeGn/Ohg53bosktti5Ysn73vtpR
# GWsJr6M4KQ1SwEbZIiFZCS3A4VTbRcr7WkXets39pcpxGDlNisi+zfV95kNo09xR
# hxi8SuWgb2OfQpVs/71eunp+PM1ZQsODurcy4x0/rlYJfhk53kQSMRtlfy5Cn6uY
# +weWUgygBSWG/w0qanWWK5TF1DNlRKzbix6cmMuGGKcpyF7EMWE1kqmjmmu7CQvM
# a3aPTqGtUt0LeqBQIhmeq/jEwd3vxQa1R85gd0/0sWxEMHkPXVfVoaryiaWAykye
# 7r+c8o/41c44zs8YxdZrz72su9fqKC/TeVf5soU46ZucmH8D6f7QHy+s1ec2PEjY
# l6cRIXTXHeQe
# =j6cJ
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 10 May 2023 09:05:26 AM BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [undefined]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* tag 'pull-qapi-2023-05-09-v2' of https://repo.or.cz/qemu/armbru:
  qapi: Reformat doc comments to conform to current conventions
  qga/qapi-schema: Reformat doc comments to conform to current conventions
  docs/devel/qapi-code-gen: Update doc comment conventions
  qapi: Section parameter @indent is no longer used, drop
  qapi: Relax doc string @name: description indentation rules
  qapi: Rewrite parsing of doc comment section symbols and tags
  qapi: Fix argument description indentation stripping
  tests/qapi-schema/doc-good: Improve argument description tests
  tests/qapi-schema/doc-good: Improve a comment
  qapi/dump: Indent bulleted lists consistently
  qapi: Tidy up a slightly awkward TODO comment
  sphinx/qapidoc: Do not emit TODO sections into user manuals
  Revert "qapi: BlockExportRemoveMode: move comments to TODO"
  meson: Fix to make QAPI generator output depend on main.py
  qapi: Fix crash on stray double quote character
  docs/devel/qapi-code-gen: Turn FIXME admonitions into comments
  docs/devel/qapi-code-gen: Clean up use of quotes a bit

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agoMerge tag 'vfio-updates-20230509.0' of https://gitlab.com/alex.williamson/qemu into...
Richard Henderson [Wed, 10 May 2023 10:20:35 +0000 (11:20 +0100)]
Merge tag 'vfio-updates-20230509.0' of https://gitlab.com/alex.williamson/qemu into staging

VFIO updates 2023-05-09

 * Add vf-token device option allowing QEMU to assign VFs where the PF
   is managed by a userspace driver. (Minwoo Im)

 * Skip log_sync during migration setup as a potential source of failure
   and likely source of redundancy. (Avihai Horon)

 * Virtualize PCIe Resizable BAR capability rather than hiding it,
   exposing only the current size as available. (Alex Williamson)

# -----BEGIN PGP SIGNATURE-----
#
# iQJPBAABCAA5FiEEQvbATlQL0amee4qQI5ubbjuwiyIFAmRaqfobHGFsZXgud2ls
# bGlhbXNvbkByZWRoYXQuY29tAAoJECObm247sIsiwNYP/2KtCbKqylnGPuwLbRMP
# HC4Id4mme7jUribmhM7FP57nQrb0tgnQoGvalkmB6M3833e3p4ivH2ezTyPxIawx
# UH4mAEBtR03rxh54eVBbOvDVf+XHd6qll/rFw5dBI0C5s7JQyMOourNRLTZLvqzD
# 2bwI7dfQzWbXWPj8QGPmDti9wbeATZ3RjqC7onoWq6A6Cw4aRGj1gHBQH9v81iA+
# m8hnZh+e5eFkQRc4mPXxFjm1Kw6ZYXWGoEEZrYPXvQn9+3MDCLcNb++KIrLsGujP
# qOnZG534vs+EZtUsGI8F02CBBXMAQFuBZhxCtuuG8iI9OQSE6R3E29iIc0Lpz5aO
# s8rN5OW4m7wXPdGkU1/7/N7kdeZvg+R8Jc4ozx3Mez3eSFbVkABSSX9vyvdHAezi
# 02Np1+ZBldZWBbBhYbWfqhvcg4iYNnHknSkS2CYY8jdsGttbrNY2f7Xllf3KC/Iv
# 6Un5WpU//0LuJjmH6onzswUUEmulchzR7OpBj68jFsB8rnTaZWM4Sqb/Jx+KXlRB
# BnNck0PCPoblpT8lgjAD3H9NaXx3mdVsml8i/7YIZjx8Zc4eanRGlsH9DmnHbB7U
# i4orDvL3SR3ZKVy6Zssti5jt8GwrEnqg97uTbS/jiTai1tOCP9n6U4T/wslHIUR4
# rIxvyJnmqrPAiWtVF+0cvGmT
# =VTJU
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 09 May 2023 09:15:54 PM BST
# gpg:                using RSA key 42F6C04E540BD1A99E7B8A90239B9B6E3BB08B22
# gpg:                issuer "alex.williamson@redhat.com"
# gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>" [undefined]
# gpg:                 aka "Alex Williamson <alex@shazbot.org>" [undefined]
# gpg:                 aka "Alex Williamson <alwillia@redhat.com>" [undefined]
# gpg:                 aka "Alex Williamson <alex.l.williamson@gmail.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 42F6 C04E 540B D1A9 9E7B  8A90 239B 9B6E 3BB0 8B22

* tag 'vfio-updates-20230509.0' of https://gitlab.com/alex.williamson/qemu:
  vfio/pci: Static Resizable BAR capability
  vfio/migration: Skip log_sync during migration SETUP state
  vfio/pci: add support for VF token

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agotarget/loongarch: Terminate vmstate subsections list
Richard Henderson [Wed, 10 May 2023 06:15:44 +0000 (06:15 +0000)]
target/loongarch: Terminate vmstate subsections list

This list requires a NULL terminator.

Fixes: 16f5396cec23 ("target/loongarch: Add LSX data type VReg")
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230510062405.127260-1-richard.henderson@linaro.org>

17 months agoqapi: Reformat doc comments to conform to current conventions
Markus Armbruster [Fri, 28 Apr 2023 10:54:29 +0000 (12:54 +0200)]
qapi: Reformat doc comments to conform to current conventions

Change

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #        do eiusmod tempor incididunt ut labore et dolore magna aliqua.

to

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.

See recent commit "qapi: Relax doc string @name: description
indentation rules" for rationale.

Reflow paragraphs to 70 columns width, and consistently use two spaces
to separate sentences.

To check the generated documentation does not change, I compared the
generated HTML before and after this commit with "wdiff -3".  Finds no
differences.  Comparing with diff is not useful, as the reflown
paragraphs are visible there.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-18-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Acked-by: Lukas Straub <lukasstraub2@web.de>
[Straightforward conflicts in qapi/audio.json qapi/misc-target.json
qapi/run-state.json resolved]

17 months agoqga/qapi-schema: Reformat doc comments to conform to current conventions
Markus Armbruster [Fri, 28 Apr 2023 10:54:28 +0000 (12:54 +0200)]
qga/qapi-schema: Reformat doc comments to conform to current conventions

Change

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #        do eiusmod tempor incididunt ut labore et dolore magna aliqua.

to

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed
    #     do eiusmod tempor incididunt ut labore et dolore magna aliqua.

See recent commit "qapi: Relax doc string @name: description
indentation rules" for rationale.

Reflow paragraphs to 70 columns width, and consistently use two spaces
to separate sentences.

To check the generated documentation does not change, I compared the
generated HTML before and after this commit with "wdiff -3".  Finds no
differences.  Comparing with diff is not useful, as the reflown
paragraphs are visible there.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-17-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
17 months agodocs/devel/qapi-code-gen: Update doc comment conventions
Markus Armbruster [Fri, 28 Apr 2023 10:54:27 +0000 (12:54 +0200)]
docs/devel/qapi-code-gen: Update doc comment conventions

The commit before previous relaxed the indentation rules to let us
improve the doc comment conventions.  This commit changes the written
conventions.  The next commits will update QAPI schemas to conform to
them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-16-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
17 months agoqapi: Section parameter @indent is no longer used, drop
Markus Armbruster [Fri, 28 Apr 2023 10:54:26 +0000 (12:54 +0200)]
qapi: Section parameter @indent is no longer used, drop

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-15-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
17 months agoqapi: Relax doc string @name: description indentation rules
Markus Armbruster [Fri, 28 Apr 2023 10:54:25 +0000 (12:54 +0200)]
qapi: Relax doc string @name: description indentation rules

The QAPI schema doc comment language provides special syntax for
command and event arguments, struct and union members, alternate
branches, enumeration values, and features: descriptions starting with
"@name:".

By convention, we format them like this:

    # @name: Lorem ipsum dolor sit amet, consectetur adipiscing elit,
    #        sed do eiusmod tempor incididunt ut labore et dolore
    #        magna aliqua.

Okay for names as short as "name", but we have much longer ones.  Their
description gets squeezed against the right margin, like this:

    # @dirty-sync-missed-zero-copy: Number of times dirty RAM synchronization could
    #                               not avoid copying dirty pages. This is between
    #                               0 and @dirty-sync-count * @multifd-channels.
    #                               (since 7.1)

The description text is effectively just 50 characters wide.  Easy
enough to read, but can be cumbersome to write.

The awkward squeeze against the right margin makes people go beyond it,
which produces two undesirables: arguments about style, and descriptions
that are unnecessarily hard to read, like this one:

    # @postcopy-vcpu-blocktime: list of the postcopy blocktime per vCPU.  This is
    #                           only present when the postcopy-blocktime migration capability
    #                           is enabled. (Since 3.0)

We could instead format it like

    # @postcopy-vcpu-blocktime:
    # list of the postcopy blocktime per vCPU.  This is only present
    # when the postcopy-blocktime migration capability is
    # enabled. (Since 3.0)

or, since the commit before previous, like

    # @postcopy-vcpu-blocktime:
    #    list of the postcopy blocktime per vCPU.  This is only present
    #    when the postcopy-blocktime migration capability is
    #    enabled. (Since 3.0)

However, I'd rather have

    # @postcopy-vcpu-blocktime: list of the postcopy blocktime per vCPU.
    #     This is only present when the postcopy-blocktime migration
    #     capability is enabled.  (Since 3.0)

because this is how rST field and option lists work.

To get this, we need to let the first non-blank line after the
"@name:" line determine expected indentation.

This fills up the indentation pitfall mentioned in
docs/devel/qapi-code-gen.rst.  A related pitfall still exists.  Update
the text to show it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-14-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Work around lack of walrus operator in Python 3.7 and older]

17 months agoqapi: Rewrite parsing of doc comment section symbols and tags
Markus Armbruster [Fri, 28 Apr 2023 10:54:24 +0000 (12:54 +0200)]
qapi: Rewrite parsing of doc comment section symbols and tags

To recognize a line starting with a section symbol and or tag, we
first split it at the first space, then examine the part left of the
space.  We can just as well examine the unsplit line, so do that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-13-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
[Work around lack of walrus operator in Python 3.7 and older]

17 months agoMerge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging
Richard Henderson [Tue, 9 May 2023 16:21:39 +0000 (17:21 +0100)]
Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* target/i386: improved EPYC models
* more removal of mb_read/mb_set
* bump _WIN32_WINNT to the Windows 8 API
* fix for modular builds with --disable-system

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmRZK7wUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroObngf8D6A5l1QQAnImRrZAny6HZV/9xseD
# 9QhkUW3fxXlUhb8tXomv2BlT8h9GzLIN6aWvcCotT+xK3kAX7mRcYKgPMr9CYL7y
# vev/hh+B6RY1CJ/xPT09/BMVjkj50AL0O/OuWMhcQ5nCO7F2sdMjMrsYqqeZcjYf
# zx9RTX7gVGt+wWFHxgCgdfL0kfgzexK55YuZU0vLzcA+pYsZWoEfW+fKBIf4rzDV
# r9M6mDBUkHBQ0rIVC3QFloAXnYb1JrpeqqL2i2qwhAkLz8LyGqk3lZF20hE/04im
# XZcZjWO5pxAxIEPeTken+2x1n8tn2BLkMtvwJdV5TpvICCFRtPZlbH79qw==
# =rXLN
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 08 May 2023 06:05:00 PM BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [undefined]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu:
  meson: leave unnecessary modules out of the build
  docs: clarify --without-default-devices
  target/i386: Add EPYC-Genoa model to support Zen 4 processor series
  target/i386: Add VNMI and automatic IBRS feature bits
  target/i386: Add missing feature bits in EPYC-Milan model
  target/i386: Add feature bits for CPUID_Fn80000021_EAX
  target/i386: Add a couple of feature bits in 8000_0008_EBX
  target/i386: Add new EPYC CPU versions with updated cache_info
  target/i386: allow versioned CPUs to specify new cache_info
  include/qemu/osdep.h: Bump _WIN32_WINNT to the Windows 8 API
  MAINTAINERS: add stanza for Kconfig files
  tb-maint: do not use mb_read/mb_set
  call_rcu: stop using mb_set/mb_read
  test-aio-multithread: simplify test_multi_co_schedule
  test-aio-multithread: do not use mb_read/mb_set for simple flags
  rcu: remove qatomic_mb_set, expand comments

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
17 months agovfio/pci: Static Resizable BAR capability
Alex Williamson [Thu, 4 May 2023 20:42:48 +0000 (14:42 -0600)]
vfio/pci: Static Resizable BAR capability

The PCI Resizable BAR (ReBAR) capability is currently hidden from the
VM because the protocol for interacting with the capability does not
support a mechanism for the device to reject an advertised supported
BAR size.  However, when assigned to a VM, the act of resizing the
BAR requires adjustment of host resources for the device, which
absolutely can fail.  Linux does not currently allow us to reserve
resources for the device independent of the current usage.

The only writable field within the ReBAR capability is the BAR Size
register.  The PCIe spec indicates that when written, the device
should immediately begin to operate with the provided BAR size.  The
spec however also notes that software must only write values
corresponding to supported sizes as indicated in the capability and
control registers.  Writing unsupported sizes produces undefined
results.  Therefore, if the hypervisor were to virtualize the
capability and control registers such that the current size is the
only indicated available size, then a write of anything other than
the current size falls into the category of undefined behavior,
where we can essentially expose the modified ReBAR capability as
read-only.

This may seem pointless, but users have reported that virtualizing
the capability in this way not only allows guest software to expose
related features as available (even if only cosmetic), but in some
scenarios can resolve guest driver issues.  Additionally, no
regressions in behavior have been reported for this change.

A caveat here is that the PCIe spec requires for compatibility that
devices report support for a size in the range of 1MB to 512GB,
therefore if the current BAR size falls outside that range we revert
to hiding the capability.

Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/r/20230505232308.2869912-1-alex.williamson@redhat.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
17 months agovfio/migration: Skip log_sync during migration SETUP state
Avihai Horon [Mon, 3 Apr 2023 13:00:00 +0000 (16:00 +0300)]
vfio/migration: Skip log_sync during migration SETUP state

Currently, VFIO log_sync can be issued while migration is in SETUP
state. However, doing this log_sync is at best redundant and at worst
can fail.

Redundant -- all RAM is marked dirty in migration SETUP state and is
transferred only after migration is set to ACTIVE state, so doing
log_sync during migration SETUP is pointless.

Can fail -- there is a time window, between setting migration state to
SETUP and starting dirty tracking by RAM save_live_setup handler, during
which dirty tracking is still not started. Any VFIO log_sync call that
is issued during this time window will fail. For example, this error can
be triggered by migrating a VM when a GUI is active, which constantly
calls log_sync.

Fix it by skipping VFIO log_sync while migration is in SETUP state.

Fixes: 758b96b61d5c ("vfio/migrate: Move switch of dirty tracking into vfio_memory_listener")
Signed-off-by: Avihai Horon <avihaih@nvidia.com>
Link: https://lore.kernel.org/r/20230403130000.6422-1-avihaih@nvidia.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
17 months agovfio/pci: add support for VF token
Minwoo Im [Mon, 20 Mar 2023 07:35:22 +0000 (16:35 +0900)]
vfio/pci: add support for VF token

VF token was introduced [1] to kernel vfio-pci along with SR-IOV
support [2].  This patch adds support VF token among PF and VF(s). To
passthu PCIe VF to a VM, kernel >= v5.7 needs this.

It can be configured with UUID like:

  -device vfio-pci,host=DDDD:BB:DD:F,vf-token=<uuid>,...

[1] https://lore.kernel.org/linux-pci/158396393244.5601.10297430724964025753.stgit@gimli.home/
[2] https://lore.kernel.org/linux-pci/158396044753.5601.14804870681174789709.stgit@gimli.home/

Cc: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Minwoo Im <minwoo.im@samsung.com>
Reviewed-by: Klaus Jensen <k.jensen@samsung.com>
Link: https://lore.kernel.org/r/20230320073522epcms2p48f682ecdb73e0ae1a4850ad0712fd780@epcms2p4
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
17 months agoqapi: Fix argument description indentation stripping
Markus Armbruster [Fri, 28 Apr 2023 10:54:23 +0000 (12:54 +0200)]
qapi: Fix argument description indentation stripping

When an argument's description starts on the line after the "#arg: "
line, indentation is stripped only from the description's first line,
as demonstrated by the previous commit.  Moreover, subsequent lines
with less indentation are not rejected.

Make the first line's indentation the expected indentation for the
remainder of the description.  This fixes indentation stripping, and
also requires at least that much indentation.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-12-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
17 months agotests/qapi-schema/doc-good: Improve argument description tests
Markus Armbruster [Fri, 28 Apr 2023 10:54:22 +0000 (12:54 +0200)]
tests/qapi-schema/doc-good: Improve argument description tests

Improve the comments to better describe what they test.

Cover argument description starting on a new line indented.  This
style isn't documented in docs/devel/qapi-code-gen.rst.  qapi-gen.py
accepts it, but messes up indentation: it's stripped from the first
line, not subsequent ones.  The next commit will fix this.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-11-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
17 months agotests/qapi-schema/doc-good: Improve a comment
Markus Armbruster [Fri, 28 Apr 2023 10:54:21 +0000 (12:54 +0200)]
tests/qapi-schema/doc-good: Improve a comment

The QAPI generator doesn't reject undocumented members and
features (yet).  doc-good.json covers this, with clear "is
undocumented" notes to signal intent.

Except for @Variant1 member @var1, where it's "(but no @var: line)".
Less clear.  Replace by "@var1 is undocumented".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-10-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
17 months agoqapi/dump: Indent bulleted lists consistently
Markus Armbruster [Fri, 28 Apr 2023 10:54:20 +0000 (12:54 +0200)]
qapi/dump: Indent bulleted lists consistently

Documentation of dump-guest-memory contains two bulleted lists.  The
first one is indented, the second one isn't.  Delete the first one's
indentation for a more consistent look.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20230428105429.1687850-9-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>