Eric Blake [Mon, 22 May 2023 19:04:37 +0000 (14:04 -0500)]
cutils: Set value in all qemu_strtosz* error paths
Making callers determine whether or not *value was populated on error
is not nice for usability. Pre-patch, we have unit tests that check
that *result is left unchanged on most EINVAL errors and set to 0 on
many ERANGE errors. This is subtly different from libc strtoumax()
behavior which returns UINT64_MAX on ERANGE errors, as well as
different from our parse_uint() which slams to 0 on EINVAL on the
grounds that we want our functions to be harder to mis-use than
strtoumax().
Let's audit callers:
- hw/core/numa.c:parse_numa() fixed in the previous patch to check for
errors
- migration/migration-hmp-cmds.c:hmp_migrate_set_parameter(),
monitor/hmp.c:monitor_parse_arguments(),
qapi/opts-visitor.c:opts_type_size(),
qapi/qobject-input-visitor.c:qobject_input_type_size_keyval(),
qemu-img.c:cvtnum_full(), qemu-io-cmds.c:cvtnum(),
target/i386/cpu.c:x86_cpu_parse_featurestr(), and
util/qemu-option.c:parse_option_size() appear to reject all failures
(although some with distinct messages for ERANGE as opposed to
EINVAL), so it doesn't matter what is in the value parameter on
error.
- All remaining callers are in the testsuite, where we can tweak our
expectations to match our new desired behavior.
Advancing to the end of the string parsed on overflow (ERANGE), while
still returning 0, makes sense (UINT64_MAX as a size is unlikely to be
useful); likewise, our size parsing code is complex enough that it's
easier to always return 0 when endptr is NULL but trailing garbage was
found, rather than trying to return the value of the prefix actually
parsed (no current caller cared about the value of the prefix).
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-16-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:36 +0000 (14:04 -0500)]
test-cutils: Add more coverage to qemu_strtosz
Add some more strings that the user might send our way. In
particular, some of these additions include FIXME comments showing
where our parser doesn't quite behave the way we want.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-15-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:35 +0000 (14:04 -0500)]
numa: Check for qemu_strtosz_MiB error
As shown in the previous commit, qemu_strtosz_MiB sometimes leaves the
result value untouched (we have to audit further to learn that in that
case, the QAPI generator says that visit_type_NumaOptions() will have
zero-initialized it), and sometimes leaves it with the value of a
partial parse before -EINVAL occurs because of trailing garbage.
Rather than blindly treating any string the user may throw at us as
valid, we should check for parse failures.
Fixes: cc001888 ("numa: fixup parsed NumaNodeOptions earlier", v2.11.0) Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-14-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:34 +0000 (14:04 -0500)]
cutils: Allow NULL str in qemu_strtosz
All the other qemu_strto* and parse_uint allow a NULL str. Having
qemu_strtosz not crash on qemu_strtosz(NULL, NULL, &value) is an easy
fix that adds some consistency between our string parsers.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-13-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:33 +0000 (14:04 -0500)]
test-cutils: Refactor qemu_strtosz tests for less boilerplate
No need to copy-and-paste lots of boilerplate per string tested, when
we can consolidate that behind helper functions. Plus, this adds a
bit more coverage (we now test all strings both with and without
endptr, whereas before some tests skipped the NULL endptr case), which
exposed a SEGFAULT on qemu_strtosz(NULL, NULL, &val) that will be
fixed in an upcoming patch.
Note that duplicating boilerplate has one advantage lost here - a
failed test tells you which line number failed; but a helper function
does not show the call stack that reached the failure. Since we call
the helper more than once within many of the "unit tests", even the
unit test name doesn't point out which call is failing. But that only
matters when tests fail (they normally pass); at which point I'm
debugging the failures under gdb anyways, so I'm not too worried about
it.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-12-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:32 +0000 (14:04 -0500)]
test-cutils: Prepare for upcoming semantic change in qemu_strtosz
A quick search for 'qemu_strtosz' in the code base shows that outside
of the testsuite, the ONLY place that passes a non-NULL pointer to
@endptr of any variant of a size parser is in hmp.c (the 'o' parser of
monitor_parse_arguments), and that particular caller warns of
"extraneous characters at the end of line" unless the trailing bytes
are purely whitespace. Thus, it makes no semantic difference at the
high level whether we parse "1.5e1k" as "1" + ".5e1" + "k" (an attempt
to use scientific notation in strtod with a scaling suffix of 'k' with
no trailing junk, but which qemu_strtosz says should fail with
EINVAL), or as "1.5e" + "1k" (a valid size with scaling suffix of 'e'
for exabytes, followed by two junk bytes) - either way, any user
passing such a string will get an error message about a parse failure.
However, an upcoming patch to qemu_strtosz will fix other corner case
bugs in handling the fractional portion of a size, and in doing so, it
is easier to declare that qemu_strtosz() itself stops parsing at the
first 'e' rather than blindly consuming whatever strtod() will
recognize. Once that is fixed, the difference will be visible at the
low level (getting a valid parse with trailing garbage when @endptr is
non-NULL, while continuing to get -EINVAL when @endptr is NULL); this
is easier to demonstrate by moving the affected strings from
test_qemu_strtosz_invalid() (which declares them as always -EINVAL) to
test_qemu_strtosz_trailing() (where @endptr affects behavior, for now
with FIXME comments).
Note that a similar argument could be made for having "0x1.5" or
"0x1M" parse as 0x1 with ".5" or "M" as trailing junk, instead of
blindly treating it as -EINVAL; however, as these cases do not suffer
from the same problems as floating point, they are not worth changing
at this time.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-11-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:31 +0000 (14:04 -0500)]
test-cutils: Add coverage of qemu_strtod
It's hard to tweak code for consistency if I can't prove what will or
won't break from those tweaks. Time to add unit tests for
qemu_strtod() and qemu_strtod_finite().
Among other things, I wrote a check whether we have C99 semantics for
strtod("0x1") (which MUST parse hex numbers) rather than C89 (which
must stop parsing at 'x'). These days, I suspect that is okay; but if
it fails CI checks, knowing the difference will help us decide what we
want to do about it. Note that C2x, while not final at the time of
this patch, has been considering whether to make strtol("0b1") parse
as 1 with no slop instead of the C17 parse of 0 with slop "b1"; that
decision may also bleed over to strtod(). But for now, I didn't think
it worth adding unit tests on that front (to strtol or strtod) as
things may still change.
Likewise, there are plenty more corner cases of strtod proper that I
don't explicitly test here, but there are enough unit tests added here
that it covers all the branches reached in our wrappers. In
particular, it demonstrates the difference on when *value is left
uninitialized, which an upcoming patch will normalize.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-10-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:30 +0000 (14:04 -0500)]
cutils: Allow NULL endptr in parse_uint()
All the qemu_strto*() functions permit a NULL endptr, just like their
libc counterparts, leaving parse_uint() as the oddball that caused
SEGFAULT on NULL and required the user to call parse_uint_full()
instead. Relax things for consistency, even though the testsuite is
the only impacted caller. Add one more unit test to ensure even
parse_uint_full(NULL, 0, &value) works. This also fixes our code to
uniformly favor EINVAL over ERANGE when both apply.
Also fixes a doc mismatch @v vs. a parameter named value.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-9-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:29 +0000 (14:04 -0500)]
cutils: Adjust signature of parse_uint[_full]
It's already confusing that we have two very similar functions for
wrapping the parse of a 64-bit unsigned value, differing mainly on
whether they permit leading '-'. Adjust the signature of parse_uint()
and parse_uint_full() to be like all of qemu_strto*(): put the result
parameter last, use the same types (uint64_t and unsigned long long
have the same width, but are not always the same type), and mark
endptr const (this latter change only affects the rare caller of
parse_uint). Adjust all callers in the tree.
While at it, note that since cutils.c already includes:
we are guaranteed that the result of parse_uint* cannot exceed
UINT64_MAX (or the build would have failed), so we can drop
pre-existing dead comparisons in opts-visitor.c that were never false.
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-8-eblake@redhat.com>
[eblake: Drop dead code spotted by Markus] Signed-off-by: Eric Blake <eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:28 +0000 (14:04 -0500)]
cutils: Document differences between parse_uint and qemu_strtou64
These two functions are subtly different, and not just because of
swapped parameter order. It took me adding better unit tests to
figure out why. Document the differences to make it more obvious to
developers trying to pick which one to use, as well as to aid in
upcoming semantic changes.
While touching the documentation, adjust a mis-statement: parse_uint
does not return -EINVAL on invalid base, but assert()s, like all the
other qemu_strto* functions that take a base argument.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-7-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:27 +0000 (14:04 -0500)]
cutils: Fix wraparound parsing in qemu_strtoui
While we were matching 32-bit strtol in qemu_strtoi, our use of a
64-bit parse was leaking through for some inaccurate answers in
qemu_strtoui in comparison to a 32-bit strtoul (see the unit test for
examples). The comment for that function even described what we have
to do for a correct parse, but didn't implement it correctly: since
strtoull checks for overflow against the wrong values and then
negates, we have to temporarily undo negation before checking for
overflow against our desired value.
Our int wrappers would be a lot easier to write if libc had a
guaranteed 32-bit parser even on platforms with 64-bit long.
Whether we parse C2x binary strings like "0b1000" is currently up to
what libc does; our unit tests intentionally don't cover that at the
moment, though.
Fixes: 473a2a331e ("cutils: add qemu_strtoi & qemu_strtoui parsers for int/unsigned int types", v2.12.0) Signed-off-by: Eric Blake <eblake@redhat.com> CC: qemu-stable@nongnu.org
Message-Id: <20230522190441.64278-6-eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:26 +0000 (14:04 -0500)]
test-cutils: Test more integer corner cases
We have quite a few undertested and underdocumented integer parsing
corner cases. To ensure that any changes we make in the code are
intentional rather than accidental semantic changes, it is time to add
more unit tests of existing behavior.
In particular, this demonstrates that parse_uint() and qemu_strtou64()
behave differently. For "-0", it's hard to argue why parse_uint needs
to reject it (it's not a negative integer), but the documentation sort
of mentions it; but it is intentional that all other negative values
are treated as ERANGE with value 0 (compared to qemu_strtou64()
treating "-2" as success and UINT64_MAX-1, for example).
Also, when mixing overflow/underflow with a check for no trailing
junk, parse_uint_full favors ERANGE over EINVAL, while qemu_strto[iu]*
favor EINVAL. This behavior is outside the C standard, so we can pick
whatever we want, but it would be nice to be consistent.
Note that C requires that "9223372036854775808" fail strtoll() with
ERANGE/INT64_MAX, but "-9223372036854775808" pass with INT64_MIN; we
weren't testing this. For strtol(), the behavior depends on whether
long is 32- or 64-bits (the cutoff point either being the same as
strtoll() or at "-2147483648"). Meanwhile, C is clear that
"-18446744073709551615" pass stroull() (but not strtoll) with value 1,
even though we want it to fail parse_uint(). And although
qemu_strtoui() has no C counterpart, it makes more sense if we design
it like 32-bit strtoul() (that is, where "-4294967296" be an alternate
acceptable spelling for "1", but "-0xffffffff00000001" should be
treated as overflow and return 0xffffffff rather than 1). We aren't
there yet, so some of the tests added in this patch have FIXME
comments.
However, note that C2x will (likely) be adding a SILENT semantic
change, where C17 strtol("0b1", &ep, 2) returns 0 with ep="b1", but
C2x will have it return 1 with ep="". I did not feel like adding
testing for those corner cases, in part because the next version of C
is not standard and libc support for binary parsing is not yet
wide-spread (as of this patch, glibc.git still misparses bare "0b":
https://sourceware.org/bugzilla/show_bug.cgi?id=30371).
Message-Id: <20230522190441.64278-5-eblake@redhat.com>
[eblake: fix a few typos spotted by Hanna] Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
[eblake: fix typo on platforms with 32-bit long] Signed-off-by: Eric Blake <eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 19:04:25 +0000 (14:04 -0500)]
test-cutils: Test integral qemu_strto* value on failures
We are inconsistent on the contents of *value after a strto* parse
failure. I found the following behaviors:
- parse_uint() and parse_uint_full(), which document that *value is
slammed to 0 on all EINVAL failures and 0 or UINT_MAX on ERANGE
failures, and has unit tests for that (note that parse_uint requires
non-NULL endptr, and does not fail with EINVAL for trailing junk)
- qemu_strtosz(), which leaves *value untouched on all failures (both
EINVAL and ERANGE), and has unit tests but not documentation for
that
- qemu_strtoi() and other integral friends, which document *value on
ERANGE failures but is unspecified on EINVAL (other than implicitly
by comparison to libc strto*); there, *value is untouched for NULL
string, slammed to 0 on no conversion, and left at the prefix value
on NULL endptr; unit tests do not consistently check the value
- qemu_strtod(), which documents *value on ERANGE failures but is
unspecified on EINVAL; there, *value is untouched for NULL string,
slammed to 0.0 for no conversion, and left at the prefix value on
NULL endptr; there are no unit tests (other than indirectly through
qemu_strtosz)
- qemu_strtod_finite(), which documents *value on ERANGE failures but
is unspecified on EINVAL; there, *value is left at the prefix for
'inf' or 'nan' and untouched in all other cases; there are no unit
tests (other than indirectly through qemu_strtosz)
Upcoming patches will change behaviors for consistency, but it's best
to first have more unit test coverage to see the impact of those
changes.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-Id: <20230522190441.64278-4-eblake@redhat.com>
Eric Blake [Fri, 12 May 2023 02:10:16 +0000 (21:10 -0500)]
test-cutils: Use g_assert_cmpuint where appropriate
When debugging test failures, seeing unsigned values as large positive
values rather than negative values matters (assuming glib 2.78+; given
that I just fixed a bug in glib 2.76 [1] where g_assert_cmpuint
displays signed instead of unsigned values). No impact when the test
is passing, but using a consistent style will matter more in upcoming
test additions. Also, some tests are better with cmphex.
While at it, fix some spacing and minor typing issues spotted nearby.
Eric Blake [Fri, 12 May 2023 02:10:15 +0000 (21:10 -0500)]
test-cutils: Avoid g_assert in unit tests
glib documentation[1] is clear: g_assert() should be avoided in unit
tests because it is ineffective if G_DISABLE_ASSERT is defined; unit
tests should stick to constructs based on g_assert_true() instead.
Note that since commit 262a69f428, we intentionally state that you
cannot define G_DISABLE_ASSERT while building qemu; but our code can
be copied to other projects without that restriction, so we should be
consistent.
For most of the replacements in this patch, using g_assert_cmpstr()
would be a regression in quality - although it would helpfully display
the string contents of both pointers on test failure, here, we really
do care about pointer equality, not just string content equality. But
when a NULL pointer is expected, g_assert_null works fine.
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Hanna Czenczek <hreitz@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230522190441.64278-2-eblake@redhat.com>
Eric Blake [Mon, 22 May 2023 18:46:31 +0000 (13:46 -0500)]
qcow2: Explicit mention of padding bytes
Although we already covered the need for padding bytes with our
changes in commit 3ae3fcfa, commit 66fcbca5 (both v5.0.0) added one
byte and relied on the rest of the text for implicitly covering 7
padding bytes. For consistency with other parts of the header (such
as the header extension format listing padding from n - m, or the
snapshot table entry listing variable padding), we might as well call
out the remaining 7 bytes as padding until such time (as any) as they
gain another meaning.
Signed-off-by: Eric Blake <eblake@redhat.com> CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-Id: <20230522184631.47211-1-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Eric Blake [Fri, 19 May 2023 15:02:16 +0000 (10:02 -0500)]
iotests: Fix test 104 under NBD
In the past, commit a231cb27 ("iotests: Fix 104 for NBD", v2.3.0)
added an additional filter to _filter_img_info to rewrite NBD URIs
into the expected output form. This recently broke when we tweaked
tests to run in a per-format directory, which did not match the regex,
because _img_info itself is now already changing
SOCK_DIR=/tmp/tmpphjfbphd/raw-nbd-104 into
/tmp/tmpphjfbphd/IMGFMT-nbd-104 prior to _img_info_filter getting a
chance to further filter things.
While diagnosing the problem, I also noticed some filter lines
rendered completely useless by a typo when we switched from TCP to
Unix sockets for NBD (in shell, '\\+' is different from "\\+" (one
gives two backslash to the regex, matching the literal 2-byte sequence
<\+> after a single digit; the other gives one backslash to the regex,
as the metacharacter \+ to match one or more of <[0-9]>); since the
literal string <nbd://127.0.0.1:0\+> is not a valid URI, that regex
hasn't been matching anything for years so it is fine to just drop it
rather than fix the typo.
Fixes: f3923a72 ("iotests: Switch nbd tests to use Unix rather than TCP", v4.2.0) Fixes: 5ba7db09 ("iotests: always use a unique sub-directory per test", v8.0.0) Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20230519150216.2599189-1-eblake@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
In this series:
- improve background migration (fiona)
- improve vmstate failure states (vladimir)
- dropped all the RDMA cleanups
Please, apply.
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmR5LAoACgkQ9IfvGFhy
# 1yPOuxAA5QzUfswCIWehrkY8FApDjsecPDq5R6hS1p5pDZkHTF5y6j49J93I65o2
# E4qr4l0+DJvBvnxTW29JQ1i0RPqJHnJBFC9Ib4o0NaA/7iRP1sTwYxIN4wWZz6H/
# pqG3oQC0WPPqgj9tHSgDW4TNkFjETYaezTN8nddNmyiaO9UxNuR5ZKbeYMroVlfp
# KbnAYfXV6CyXKUZFT32BYcajYBDZAqBCO6y3gEn77KPlT1/TqnucoYEVuNudq5SE
# jeCamTzoAQ6SIRFM/eY+aASxdsSryqDS/WLqBFsleXs1kkJ6mkDnNels4HqS+xs9
# p2Vhv/59ktoC57XsRgTgzEklAaSHunZivcQkc5szyGVE5TZyKtWg8WhA6rvlqbjK
# lb3kKpvtVi73+pAWU0hhKFdnCrB6ieCHI70CJ5mpiIu3MzLUyrNJOK4FoKNoJDOD
# Dp45DK+W/EMg51pXyHJZZqHM1p0GGj0fmhv5T05nJ590fIWV4iqDdHFxsMZ9vEPN
# iEvAB7/pXz+yECznDFrp2e047rshOGaKKNSW3zl3/7D32Ds2FKur76dL4BoymztW
# HHLxmRWn8HmHMoKYLoawWVmCBsDqy8BFct+rHbA6h/0nSCPYIUCmMrSYVqajUbXD
# Ulkh4KNQGoBCzp5Toa0dYEXVc891wVOw4k8PTARwf2OYskSkT88=
# =Km5X
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 01 Jun 2023 04:38:50 PM PDT
# gpg: using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [unknown]
# gpg: aka "Juan Quintela <quintela@trasno.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723
* tag 'migration-20230601-pull-request' of https://gitlab.com/juan.quintela/qemu:
migration: stop tracking ram writes when cancelling background migration
migration: restore vmstate on migration failure
migration: switch from .vm_was_running to .vm_old_state
runstate: drop unused runstate_store()
migration: never fail in global_state_store()
runstate: add runstate_get()
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Fiona Ebner [Fri, 26 May 2023 11:59:08 +0000 (13:59 +0200)]
migration: stop tracking ram writes when cancelling background migration
Currently, it is only done when the iteration finishes successfully.
Not cleaning up the userfaultfd write protection can lead to
symptoms/issues such as the process hanging in memmove or GDB not
being able to attach.
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Message-Id: <20230526115908.196171-1-f.ebner@proxmox.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
1. Otherwise failed migration just drops guest-panicked state, which is
not good for management software.
2. We do keep different paused states like guest-panicked during
migration with help of global_state state.
3. We do restore running state on source when migration is cancelled or
failed.
4. "postmigrate" state is documented as "guest is paused following a
successful 'migrate'", so originally it's only for successful path
and we never documented current behavior.
Let's restore paused states like guest-panicked in case of cancel or
fail too. Allow same transitions like for inmigrate state.
This commit changes the behavior that was introduced by commit 42da5550d6 "migration: set state to post-migrate on failure" and
provides a bit different fix on related
https://bugzilla.redhat.com/show_bug.cgi?id=1355683
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230517123752.21615-6-vsementsov@yandex-team.ru> Signed-off-by: Juan Quintela <quintela@redhat.com>
migration: switch from .vm_was_running to .vm_old_state
No logic change here, only refactoring. That's a preparation for next
commit where we finally restore the stopped vm state on migration
failure or cancellation.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230517123752.21615-5-vsementsov@yandex-team.ru> Signed-off-by: Juan Quintela <quintela@redhat.com>
The function is unused since previous commit. Drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230517123752.21615-4-vsementsov@yandex-team.ru> Signed-off-by: Juan Quintela <quintela@redhat.com>
Actually global_state_store() can never fail. Let's get rid of extra
error paths.
To make things clear, use new runstate_get() and use same approach for
global_state_store() and global_state_store_running().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230517123752.21615-3-vsementsov@yandex-team.ru> Signed-off-by: Juan Quintela <quintela@redhat.com>
It's necessary to restore the state after failed/cancelled migration in
further commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20230517123752.21615-2-vsementsov@yandex-team.ru> Signed-off-by: Juan Quintela <quintela@redhat.com>
Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging
Pull request
- Stefano Garzarella's blkio block driver 'fd' parameter
- My thread-local blk_io_plug() series
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmR4uHoACgkQnKSrs4Gr
# c8hFBAgAo+SFrOteYgdELM9s0EWb0AU39MTOyNXW7i5mPZNXrn5J7pfRD/5wvI6l
# wl5GNMQ+M5HVYO7CumKWr4M1IpKV5Jin6FN/2h15fWkeg17lBOmNHUF+LctLYQbq
# HwtNA4hdw1+SEv8kQLBgiqSJMqWcn80X09emgPMCIwET9zxokRYwVjQJx2alM5bd
# SqgitDp5qlHyj5HQPX2orT9KrXYWQdGr8i50bn0S67r1wdqTRMu93wrWdEUUncId
# 7otlUaq8cARbRMJzIwDmy/cF24Ynr0wCJb4aHW+trRtf+PNgx1Ki+YOiz+LFyjq7
# t6KOMeignzhz9Uzq8EVG4XW8SHpGkw==
# =Ms48
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 01 Jun 2023 08:25:46 AM PDT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
* tag 'block-pull-request' of https://gitlab.com/stefanha/qemu:
qapi: add '@fdset' feature for BlockdevOptionsVirtioBlkVhostVdpa
block/blkio: use qemu_open() to support fd passing for virtio-blk
block: remove bdrv_co_io_plug() API
block/linux-aio: convert to blk_io_plug_call() API
block/io_uring: convert to blk_io_plug_call() API
block/blkio: convert to blk_io_plug_call() API
block/nvme: convert to blk_io_plug_call() API
block: add blk_io_plug_call() API
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Merge tag 'tracing-pull-request' of https://gitlab.com/stefanha/qemu into staging
Pull request
This pull request contains Alex Bennée's vcpu trace events removal patches.
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmR4tAMACgkQnKSrs4Gr
# c8ht/AgAiVslnH4vmD5IZloBHVRNEZKifODZbHW75yDgIirj/MhqlXPZ7bWoGwTN
# MLsTVuihhYnJBQKknN7lKyhkoQjgiJSkYhQbXSlcN7T3UE0+iG47FSudYTLDZSov
# M5wu1Edzi4q1uWr7ZIn/NS39iHVvQ7fdDMosHQmI0HKl25yx5936c0T2A4yyj96e
# LEtg4wLKo1uRgEMvCWrpiDz8ohNVwexAxCggwHE17tCebBmik+2cBEWAS+fcTbSr
# Nx3yWRat5VbqHOe3ghudLMNXHySQjNYrexULOVzyUUoaqUDt2eWCr9A4312BflEl
# 8U9FFl99BZX5rWkyUzsHxEmPlRsazQ==
# =oMRe
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 01 Jun 2023 08:06:43 AM PDT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
* tag 'tracing-pull-request' of https://gitlab.com/stefanha/qemu:
accel/tcg: include cs_base in our hash calculations
hw/9pfs: use qemu_xxhash4
tcg: remove the final vestiges of dstate
trace: remove control-vcpu.h
trace: remove code that depends on setting vcpu
qapi: make the vcpu parameters deprecated for 8.1
docs/deprecated: move QMP events bellow QMP command section
scripts/qapi: document the tool that generated the file
trace: remove vcpu_id from the TraceEvent structure
trace-events: remove the remaining vcpu trace events
*-user: remove the guest_user_syscall tracepoints
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
qapi: add '@fdset' feature for BlockdevOptionsVirtioBlkVhostVdpa
The virtio-blk-vhost-vdpa driver in libblkio 1.3.0 supports the fd
passing through the new 'fd' property.
Since now we are using qemu_open() on '@path' if the virtio-blk driver
supports the fd passing, let's announce it.
In this way, the management layer can pass the file descriptor of an
already opened vhost-vdpa character device. This is useful especially
when the device can only be accessed with certain privileges.
Add the '@fdset' feature only when the virtio-blk-vhost-vdpa driver
in libblkio supports it.
Suggested-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20230530071941.8954-3-sgarzare@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
block/blkio: use qemu_open() to support fd passing for virtio-blk
Some virtio-blk drivers (e.g. virtio-blk-vhost-vdpa) supports the fd
passing. Let's expose this to the user, so the management layer
can pass the file descriptor of an already opened path.
If the libblkio virtio-blk driver supports fd passing, let's always
use qemu_open() to open the `path`, so we can handle fd passing
from the management layer through the "/dev/fdset/N" special path.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20230530071941.8954-2-sgarzare@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:54:01 +0000 (17:54 +0100)]
accel/tcg: include cs_base in our hash calculations
We weren't using cs_base in the hash calculations before. Since the
arm front end moved a chunk of flags in a378206a20 (target/arm: Move
mode specific TB flags to tb->cs_base) they comprise of an important
part of the execution state.
Widen the tb_hash_func to include cs_base and expand to qemu_xxhash8()
to accommodate it.
My initial benchmark shows very little difference in the
runtime.
Alex Bennée [Fri, 26 May 2023 16:54:00 +0000 (17:54 +0100)]
hw/9pfs: use qemu_xxhash4
No need to pass zeros as we have helpers that do that for us.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-11-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-10-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:58 +0000 (17:53 +0100)]
trace: remove control-vcpu.h
Now we no longer have vcpu controlled trace events we can excise the
code that allows us to query its status.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-9-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-8-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:57 +0000 (17:53 +0100)]
trace: remove code that depends on setting vcpu
Now we no longer have any events that are for vcpus we can start
excising the code from the trace control. As the vcpu parameter is
encoded as part of QMP we just stub out the has_vcpu/vcpu parameters
rather than alter the API.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-8-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-7-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:56 +0000 (17:53 +0100)]
qapi: make the vcpu parameters deprecated for 8.1
I don't think I can remove the parameters directly but certainly mark
them as deprecated.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-7-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-6-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Also rename the section to make the fact this is part of the
management protocol even clearer.
Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-6-alex.bennee@linaro.org Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:54 +0000 (17:53 +0100)]
scripts/qapi: document the tool that generated the file
This makes it a little easier for developers to find where things
where being generated.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-5-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-5-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:53 +0000 (17:53 +0100)]
trace: remove vcpu_id from the TraceEvent structure
This does involve temporarily stubbing out some helper functions
before we excise the rest of the code.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-4-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-4-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:52 +0000 (17:53 +0100)]
trace-events: remove the remaining vcpu trace events
While these are all in helper functions being designated vcpu events
complicates the removal of the dynamic vcpu state code. TCG plugins
allow you to instrument vcpu_[init|exit|idle].
We rename cpu_reset and make it a normal trace point.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-3-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-3-alex.bennee@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Alex Bennée [Fri, 26 May 2023 16:53:51 +0000 (17:53 +0100)]
*-user: remove the guest_user_syscall tracepoints
This is pure duplication now. Both bsd-user and linux-user have
builtin strace support and we can also track syscalls via the plugins
system.
Reviewed-by: Warner Losh <imp@bsdimp.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20230526165401.574474-2-alex.bennee@linaro.org
Message-Id: <20230524133952.3971948-2-alex.bennee@linaro.org>
[Remove unused variable in do_freebsd_syscall() reported by Richard
Henderson.
--Stefan] Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Tue, 30 May 2023 18:09:59 +0000 (14:09 -0400)]
block: remove bdrv_co_io_plug() API
No block driver implements .bdrv_co_io_plug() anymore. Get rid of the
function pointers.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-7-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Tue, 30 May 2023 18:09:58 +0000 (14:09 -0400)]
block/linux-aio: convert to blk_io_plug_call() API
Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission instead.
Note that a dev_max_batch check is dropped in laio_io_unplug() because
the semantics of unplug_fn() are different from .bdrv_co_unplug():
1. unplug_fn() is only called when the last blk_io_unplug() call occurs,
not every time blk_io_unplug() is called.
2. unplug_fn() is per-thread, not per-BlockDriverState, so there is no
way to get per-BlockDriverState fields like dev_max_batch.
Therefore this condition cannot be moved to laio_unplug_fn(). It is not
obvious that this condition affects performance in practice, so I am
removing it instead of trying to come up with a more complex mechanism
to preserve the condition.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-id: 20230530180959.1108766-6-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Tue, 30 May 2023 18:09:57 +0000 (14:09 -0400)]
block/io_uring: convert to blk_io_plug_call() API
Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission instead.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-5-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Tue, 30 May 2023 18:09:56 +0000 (14:09 -0400)]
block/blkio: convert to blk_io_plug_call() API
Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission instead.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-4-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Tue, 30 May 2023 18:09:55 +0000 (14:09 -0400)]
block/nvme: convert to blk_io_plug_call() API
Stop using the .bdrv_co_io_plug() API because it is not multi-queue
block layer friendly. Use the new blk_io_plug_call() API to batch I/O
submission instead.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Stefan Hajnoczi [Tue, 30 May 2023 18:09:54 +0000 (14:09 -0400)]
block: add blk_io_plug_call() API
Introduce a new API for thread-local blk_io_plug() that does not
traverse the block graph. The goal is to make blk_io_plug() multi-queue
friendly.
Instead of having block drivers track whether or not we're in a plugged
section, provide an API that allows them to defer a function call until
we're unplugged: blk_io_plug_call(fn, opaque). If blk_io_plug_call() is
called multiple times with the same fn/opaque pair, then fn() is only
called once at the end of the function - resulting in batching.
This patch introduces the API and changes blk_io_plug()/blk_io_unplug().
blk_io_plug()/blk_io_unplug() no longer require a BlockBackend argument
because the plug state is now thread-local.
Later patches convert block drivers to blk_io_plug_call() and then we
can finally remove .bdrv_co_io_plug() once all block drivers have been
converted.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Acked-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20230530180959.1108766-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Using "-o /dev/null" fails on Windows. Rather that working
around this in meson, add a separate command-line option so
that we can use python's os.devnull.
Reported-by: Thomas Huth <thuth@redhat.com> Fixes: 656666dc7d1b ("tests/decode: Convert tests to meson") Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230531232510.66985-1-richard.henderson@linaro.org>
Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging
Python: synchronize python-qemu-qmp
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmR3sUwACgkQfe+BBqr8
# OQ75GA/+NkO60LR6G7M68Wk2eaLlArlpWKa66lVTUPzIV+ERTzccjzFmEFhYM42z
# lJkkVieysuW/cFpjQqILSokfjiMOEH0XqC2R545vx0shZlExG6j6ky74jpFXxLCY
# 1tnZ/wOB97D1nO273T6taGfS6ZPBULaL+J2iCBuruEQiM9OKDQTjojLQYn+hRokL
# BZHF5fkMDts92GHBJdUo8ftYDyblDQ2SUQqdq0pgBgkt+kHlQ4wFB7O39HUwKwT7
# rnAYz7EGbumYGwDkuNSIbpJ2pPiX7SxSPmmrebVPlQ79town3XATLraVbllls5eO
# 8BvvkDakO7GvTkzcRvqcFnsnytWJvbEr0jPs1m8lQ2dMTv+NdZmsoItqGDP3LzVZ
# RU/Dr/8biKAbMXpSRH0Waddvmpb18I9I4U2NrVWDZ/vp6DqOFMgx/wUAVz0y0+3O
# M9o9Bj93YZhqBXhpShc75xjvaqJ10IzqG0roR0JbbskdbPmtIEvFlparxGDyH3cX
# UaQPKk8WdRCVOjtodqM28C441zMSUdL5ZCHB1LnMEhbTeV/MkR8W5KAXcYIzy2ay
# gh0FBYoiI8QNBGMR5AEpxdc3XKSSYFXlPGMz74yhlO1hWP1KZM1rE8OWHVfAPcwl
# T4xkw+Hoio9T6SlOWH4qwANaiaX9BII5Dv+L+UqqYt4+neeN8yk=
# =FvG2
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 31 May 2023 01:42:52 PM PDT
# gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB
# Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E
* tag 'python-pull-request' of https://gitlab.com/jsnow/qemu:
Revert "python/qmp/protocol: add open_with_socket()"
python/qmp/legacy: remove open_with_socket() calls
python/machine: use connect-based interface for existing sockets
python/qmp/legacy: allow using sockets for connect()
python/qmp: allow sockets to be passed to connect()
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Favor using connect() when passing a socket instead of
open_with_socket(). Simultaneously, update constructor calls to use the
combined address argument for QEMUMonitorProtocol().
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20230517163406.2593480-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
John Snow [Wed, 17 May 2023 16:34:04 +0000 (12:34 -0400)]
python/machine: use connect-based interface for existing sockets
Instead of using accept() with sockets (which uses open_with_socket()),
use calls to connect() to utilize existing sockets instead. A benefit of
this is more robust error handling already present within the connect()
call that isn't present in open_with_socket().
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20230517163406.2593480-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
John Snow [Wed, 17 May 2023 16:34:02 +0000 (12:34 -0400)]
python/qmp: allow sockets to be passed to connect()
Allow existing sockets to be passed to connect(). The changes are pretty
minimal, and this allows for far greater flexibility in setting up
communications with an endpoint.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20230517163406.2593480-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
gitlab: switch from 'stable' to 'latest' docker container tags
The 'stable' and 'stable-dind' tags are not documented as supported
tags at:
https://hub.docker.com/_/docker
Looking at their content they reflect docker 19.x.x release series,
were last built in Dec 2020, and have 3 critical and 20 high rated
CVEs unfixed. This obsolete status is attested by this commit:
The 'stable-dind' tag in particular appears buggy as it is unable to
resolve DNS for Fedora repos:
- Curl error (6): Couldn't resolve host name for https://mirrors.fedoraproject.org/metalink?repo=fedora-37&arch=x86_64&countme=1 [getaddrinfo() thread failed to start]
We used the 'stable' tag previously at the recommendation of GitLab
docs, but those docs are wrong and pending a fix:
Merge tag 'pull-tcg-20230530' of https://gitlab.com/rth7680/qemu into staging
Improvements to 128-bit atomics:
- Separate __int128_t type and arithmetic detection
- Support 128-bit load/store in backend for i386, aarch64, ppc64, s390x
- Accelerate atomics via host/include/
Decodetree:
- Add named field syntax
- Move tests to meson
# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmR2R10dHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV/bsgf/XLi8q+ITyoEAKwG4
# 6ML7DktLAdIs9Euah9twqe16U0BM0YzpKfymBfVVBKKaIa0524N4ZKIT3h6EeJo+
# f+ultqrpsnH+aQh4wc3ZCkEvRdhzhFT8VcoRTunJuJrbL3Y8n2ZSgODUL2a0tahT
# Nn+zEPm8rzQanSKQHq5kyNBLpgTUKjc5wKfvy/WwttnFmkTnqzcuEA6nPVOVwOHC
# lZBQCByIQWsHfFHUVJFvsFzBQbm0mAiW6FNKzPBkoXon0h/UZUI1lV+xXzgutFs+
# zR2O8IZwLYRu2wOWiTF8Nn2qQafkB3Dhwoq3JTEXhOqosOPExbIiWlsZDlPiKRJk
# bwmQlg==
# =XQMb
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 30 May 2023 11:58:37 AM PDT
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [ultimate]
* tag 'pull-tcg-20230530' of https://gitlab.com/rth7680/qemu: (27 commits)
tests/decode: Add tests for various named-field cases
scripts/decodetree: Implement named field support
scripts/decodetree: Implement a topological sort
scripts/decodetree: Pass lvalue-formatter function to str_extract()
docs: Document decodetree named field syntax
tests/decode: Convert tests to meson
decodetree: Do not remove output_file from /dev
decodetree: Diagnose empty pattern group
decodetree: Fix recursion in prop_format and build_tree
decodetree: Add --test-for-error
tcg: Remove TCG_TARGET_TLB_DISPLACEMENT_BITS
accel/tcg: Add aarch64 store_atom_insert_al16
accel/tcg: Add aarch64 lse2 load_atom_extract_al16_or_al8
accel/tcg: Add x86_64 load_atom_extract_al16_or_al8
accel/tcg: Extract store_atom_insert_al16 to host header
accel/tcg: Extract load_atom_extract_al16_or_al8 to host header
tcg/s390x: Support 128-bit load/store
tcg/ppc: Support 128-bit load/store
tcg/aarch64: Support 128-bit load/store
tcg/aarch64: Simplify constraints on qemu_ld/st
...
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Thomas Huth [Tue, 30 May 2023 18:03:30 +0000 (20:03 +0200)]
tests/avocado/virtio-gpu: Cancel test if drm rendering is not available
The test_vhost_user_vga_virgl test currently fails on some CI
machines with:
qemu-system-x86_64: egl: no drm render node available
qemu-system-x86_64: egl: render node init failed
The other test in this file already checks whether there is
an error while starting QEMU - we should do the same for the
test_vhost_user_vga_virgl test, too.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230530180330.48722-1-thuth@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Peter Maydell [Tue, 23 May 2023 12:04:47 +0000 (13:04 +0100)]
tests/decode: Add tests for various named-field cases
Add some tests for various cases of named-field use, both ones that
should work and ones that should be diagnosed as errors.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230523120447.728365-7-peter.maydell@linaro.org>
Peter Maydell [Tue, 23 May 2023 12:04:46 +0000 (13:04 +0100)]
scripts/decodetree: Implement named field support
Implement support for named fields, i.e. where one field is defined
in terms of another, rather than directly in terms of bits extracted
from the instruction.
The new method referenced_fields() on all the Field classes returns a
list of fields that this field references. This just passes through,
except for the new NamedField class.
We can then use referenced_fields() to:
* construct a list of 'dangling references' for a format or
pattern, which is the fields that the format/pattern uses but
doesn't define itself
* do a topological sort, so that we output "field = value"
assignments in an order that means that we assign a field before
we reference it in a subsequent assignment
* check when we output the code for a pattern whether we need to
fill in the format fields before or after the pattern fields, and
do other error checking
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230523120447.728365-6-peter.maydell@linaro.org>
Peter Maydell [Tue, 23 May 2023 12:04:45 +0000 (13:04 +0100)]
scripts/decodetree: Implement a topological sort
To support named fields, we will need to be able to do a topological
sort (so that we ensure that we output the assignment to field A
before the assignment to field B if field B refers to field A by
name). The good news is that there is a tsort in the python standard
library; the bad news is that it was only added in Python 3.9.
To bridge the gap between our current minimum supported Python
version and 3.9, provide a local implementation that has the
same API as the stdlib version for the parts we care about.
In future when QEMU's minimum Python version requirement reaches
3.9 we can delete this code and replace it with an 'import' line.
The core of this implementation is based on
https://code.activestate.com/recipes/578272-topological-sort/
which is MIT-licensed.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230523120447.728365-5-peter.maydell@linaro.org>
Peter Maydell [Tue, 23 May 2023 12:04:44 +0000 (13:04 +0100)]
scripts/decodetree: Pass lvalue-formatter function to str_extract()
To support referring to other named fields in field definitions, we
need to pass the str_extract() method a function which tells it how
to emit the code for a previously initialized named field. (In
Pattern::output_code() the other field will be "u.f_foo.field", and
in Format::output_extract() it is "a->field".)
Refactor the two callsites that currently do "output code to
initialize each field", and have them pass a lambda that defines how
to format the lvalue in each case. This is then used both in
emitting the LHS of the assignment and also passed down to
str_extract() as a new argument (unused at the moment, but will be
used in the following patch).
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230523120447.728365-4-peter.maydell@linaro.org>
Peter Maydell [Tue, 23 May 2023 12:04:43 +0000 (13:04 +0100)]
docs: Document decodetree named field syntax
Document the named field syntax that we want to implement for the
decodetree script. This allows a field to be defined in terms of
some other field that the instruction pattern has already set, for
example:
%sz_imm 10:3 sz:3 !function=expand_sz_imm
to allow a function to be passed both an immediate field from the
instruction and also a sz value which might have been specified by
the instruction pattern directly (sz=1, etc) rather than being a
simple field within the instruction.
Note that the restriction on not having the format referring to the
pattern and the pattern referring to the format simultaneously is a
restriction of the decoder generator rather than inherently being a
silly thing to do.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230523120447.728365-3-peter.maydell@linaro.org>
Test err_pattern_group_empty.decode failed with exception:
Traceback (most recent call last):
File "./scripts/decodetree.py", line 1424, in <module> main()
File "./scripts/decodetree.py", line 1342, in main toppat.build_tree()
File "./scripts/decodetree.py", line 627, in build_tree
self.tree = self.__build_tree(self.pats, self.fixedbits,
File "./scripts/decodetree.py", line 607, in __build_tree
fb = i.fixedbits & innermask
TypeError: unsupported operand type(s) for &: 'NoneType' and 'int'
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Adjust the softmmu tlb to use TMP[0-2], not any of the normally available
registers. Since we handle overlap betwen inputs and helper arguments,
we can allow any allocatable reg.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
meson: Split test for __int128_t type from __int128_t arithmetic
Older versions of clang have missing runtime functions for arithmetic
with -fsanitize=undefined (see 464e3671f9d5c), so we cannot use
__int128_t for implementing Int128. But __int128_t is present,
data movement works, and it can be used for atomic128.
Probe for both CONFIG_INT128_TYPE and CONFIG_INT128, adjust
qemu/int128.h to define Int128Alias if CONFIG_INT128_TYPE,
and adjust the meson probe for atomics to use has_int128_type.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
accel/tcg: Fix check for page writeability in load_atomic16_or_exit
PAGE_WRITE is current writability, as modified by TB protection;
PAGE_WRITE_ORG is the original page writability.
Fixes: cdfac37be0d ("accel/tcg: Honor atomicity of loads") Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
tcg: Fix register move type in tcg_out_ld_helper_ret
The first move was incorrectly using TCG_TYPE_I32 while the second
move was correctly using TCG_TYPE_REG. This prevents a 64-bit host
from moving all 128-bits of the return value.
Fixes: ebebea53ef8 ("tcg: Support TCG_TYPE_I128 in tcg_out_{ld,st}_helper_{args,ret}") Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Merge tag 'for-upstream' of https://repo.or.cz/qemu/kevin into staging
Block layer patches
- Fix blockdev-create with iothreads
- Remove aio_disable_external() API
# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEE3D3rFZqa+V09dFb+fwmycsiPL9YFAmR2JIARHGt3b2xmQHJl
# ZGhhdC5jb20ACgkQfwmycsiPL9brtA/9HVdAdtJxW78J60TE2lTqE9XlqMOEHBZl
# 8GN72trjP2geY/9mVsv/XoFie4ecqFsYjwAWWUuXZwLgAo53jh7oFN7gBH5iGyyD
# +EukYEfjqoykX5BkoK0gbMZZUe5Y4Dr2CNXYw4bNg8kDzj2RLifGA1XhdL3HoiVt
# PHZrhwBR7ddww6gVOnyJrfGL8fMkW/ZNeKRhrTZuSP+63oDOeGTsTumD+YKJzfPs
# p5WlwkuPjcqbO+w32FeVOHVhNI4swkN5svz3fkr8NuflfA7kH6nBQ5wymObbaTLc
# Erx03lrtP1+6nw43V11UnYt6iDMg4EBUQwtzNaKFnk3rMIdjoQYxIM5FTBWL2rYD
# Dg6PhkncXQ1WNWhUaFqpTFLB52XAYsSa4/y2QAGP6nWbqAUAUknQ3exaMvWiq7Z0
# nZeyyhIWvpJIHGCArWRdqqh+zsBdsmUVuPGyZnZgL/cXoJboYiHMyMJSUWE0XxML
# NGrncwxdsBXkVGGwTdHpBT64dcu3ENRgwtraqRLQm+tp5MKNTJB/+Ug2/p1vonHT
# UOoHz//UPskn8sHIyevoHXeu2Ns0uIHzrAXr+7Ay+9UYyIH6a07F4b2BGqkfyi/i
# 8wQsDmJ/idx5C4q1+jS+GuIbpnjIx6nxXwXMqpscUXZmM4Am8OMkiKxQAa1wExGF
# paId+HHwyks=
# =yuER
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 30 May 2023 09:29:52 AM PDT
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
* tag 'for-upstream' of https://repo.or.cz/qemu/kevin: (32 commits)
aio: remove aio_disable_external() API
virtio: do not set is_external=true on host notifiers
virtio-scsi: implement BlockDevOps->drained_begin()
virtio-blk: implement BlockDevOps->drained_begin()
virtio: make it possible to detach host notifier from any thread
block/fuse: do not set is_external=true on FUSE fd
block/export: don't require AioContext lock around blk_exp_ref/unref()
block/export: rewrite vduse-blk drain code
hw/xen: do not set is_external=true on evtchn fds
xen-block: implement BlockDevOps->drained_begin()
block: drain from main loop thread in bdrv_co_yield_to_drain()
block: add blk_in_drain() API
hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore
block/export: stop using is_external in vhost-user-blk server
block/export: wait for vhost-user-blk requests when draining
util/vhost-user-server: rename refcount to in_flight counter
virtio-scsi: stop using aio_disable_external() during unplug
virtio-scsi: avoid race between unplug and transport event
hw/qdev: introduce qdev_is_realized() helper
block-backend: split blk_do_set_aio_context()
...
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:38 +0000 (15:02 -0400)]
aio: remove aio_disable_external() API
All callers now pass is_external=false to aio_set_fd_handler() and
aio_set_event_notifier(). The aio_disable_external() API that
temporarily disables fd handlers that were registered is_external=true
is therefore dead code.
Remove aio_disable_external(), aio_enable_external(), and the
is_external arguments to aio_set_fd_handler() and
aio_set_event_notifier().
The entire test-fdmon-epoll test is removed because its sole purpose was
testing aio_disable_external().
Parts of this patch were generated using the following coccinelle
(https://coccinelle.lip6.fr/) semantic patch:
Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-21-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:37 +0000 (15:02 -0400)]
virtio: do not set is_external=true on host notifiers
Host notifiers can now use is_external=false since virtio-blk and
virtio-scsi no longer rely on is_external=true for drained sections.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-20-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The virtio-scsi Host Bus Adapter provides access to devices on a SCSI
bus. Those SCSI devices typically have a BlockBackend. When the
BlockBackend enters a drained section, the SCSI device must temporarily
stop submitting new I/O requests.
Implement this behavior by temporarily stopping virtio-scsi virtqueue
processing when one of the SCSI devices enters a drained section. The
new scsi_device_drained_begin() API allows scsi-disk to message the
virtio-scsi HBA.
scsi_device_drained_begin() uses a drain counter so that multiple SCSI
devices can have overlapping drained sections. The HBA only sees one
pair of .drained_begin/end() calls.
After this commit, virtio-scsi no longer depends on hw/virtio's
ioeventfd aio_set_event_notifier(is_external=true). This commit is a
step towards removing the aio_disable_external() API.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-19-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Detach ioeventfds during drained sections to stop I/O submission from
the guest. virtio-blk is no longer reliant on aio_disable_external()
after this patch. This will allow us to remove the
aio_disable_external() API once all other code that relies on it is
converted.
Take extra care to avoid attaching/detaching ioeventfds if the data
plane is started/stopped during a drained section. This should be rare,
but maybe the mirror block job can trigger it.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-18-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:34 +0000 (15:02 -0400)]
virtio: make it possible to detach host notifier from any thread
virtio_queue_aio_detach_host_notifier() does two things:
1. It removes the fd handler from the event loop.
2. It processes the virtqueue one last time.
The first step can be peformed by any thread and without taking the
AioContext lock.
The second step may need the AioContext lock (depending on the device
implementation) and runs in the thread where request processing takes
place. virtio-blk and virtio-scsi therefore call
virtio_queue_aio_detach_host_notifier() from a BH that is scheduled in
AioContext.
The next patch will introduce a .drained_begin() function that needs to
call virtio_queue_aio_detach_host_notifier(). .drained_begin() functions
cannot call aio_poll() to wait synchronously for the BH. It is possible
for a .drained_poll() callback to asynchronously wait for the BH, but
that is more complex than necessary here.
Move the virtqueue processing out to the callers of
virtio_queue_aio_detach_host_notifier() so that the function can be
called from any thread. This is in preparation for the next patch.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20230516190238.8401-17-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:33 +0000 (15:02 -0400)]
block/fuse: do not set is_external=true on FUSE fd
This is part of ongoing work to remove the aio_disable_external() API.
Use BlockDevOps .drained_begin/end/poll() instead of
aio_set_fd_handler(is_external=true).
As a side-effect the FUSE export now follows AioContext changes like the
other export types.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-16-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:32 +0000 (15:02 -0400)]
block/export: don't require AioContext lock around blk_exp_ref/unref()
The FUSE export calls blk_exp_ref/unref() without the AioContext lock.
Instead of fixing the FUSE export, adjust blk_exp_ref/unref() so they
work without the AioContext lock. This way it's less error-prone.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-15-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:31 +0000 (15:02 -0400)]
block/export: rewrite vduse-blk drain code
vduse_blk_detach_ctx() waits for in-flight requests using
AIO_WAIT_WHILE(). This is not allowed according to a comment in
bdrv_set_aio_context_commit():
/*
* Take the old AioContex when detaching it from bs.
* At this point, new_context lock is already acquired, and we are now
* also taking old_context. This is safe as long as bdrv_detach_aio_context
* does not call AIO_POLL_WHILE().
*/
Use this opportunity to rewrite the drain code in vduse-blk:
- Use the BlockExport refcount so that vduse_blk_exp_delete() is only
called when there are no more requests in flight.
- Implement .drained_poll() so in-flight request coroutines are stopped
by the time .bdrv_detach_aio_context() is called.
- Remove AIO_WAIT_WHILE() from vduse_blk_detach_ctx() to solve the
.bdrv_detach_aio_context() constraint violation. It's no longer
needed due to the previous changes.
- Always handle the VDUSE file descriptor, even in drained sections. The
VDUSE file descriptor doesn't submit I/O, so it's safe to handle it in
drained sections. This ensures that the VDUSE kernel code gets a fast
response.
- Suspend virtqueue fd handlers in .drained_begin() and resume them in
.drained_end(). This eliminates the need for the
aio_set_fd_handler(is_external=true) flag, which is being removed from
QEMU.
This is a long list but splitting it into individual commits would
probably lead to git bisect failures - the changes are all related.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-14-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:30 +0000 (15:02 -0400)]
hw/xen: do not set is_external=true on evtchn fds
is_external=true suspends fd handlers between aio_disable_external() and
aio_enable_external(). The block layer's drain operation uses this
mechanism to prevent new I/O from sneaking in between
bdrv_drained_begin() and bdrv_drained_end().
The previous commit converted the xen-block device to use BlockDevOps
.drained_begin/end() callbacks. It no longer relies on is_external=true
so it is safe to pass is_external=false.
This is part of ongoing work to remove the aio_disable_external() API.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-13-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:29 +0000 (15:02 -0400)]
xen-block: implement BlockDevOps->drained_begin()
Detach event channels during drained sections to stop I/O submission
from the ring. xen-block is no longer reliant on aio_disable_external()
after this patch. This will allow us to remove the
aio_disable_external() API once all other code that relies on it is
converted.
Extend xen_device_set_event_channel_context() to allow ctx=NULL. The
event channel still exists but the event loop does not monitor the file
descriptor. Event channel processing can resume by calling
xen_device_set_event_channel_context() with a non-NULL ctx.
Factor out xen_device_set_event_channel_context() calls in
hw/block/dataplane/xen-block.c into attach/detach helper functions.
Incidentally, these don't require the AioContext lock because
aio_set_fd_handler() is thread-safe.
It's safer to register BlockDevOps after the dataplane instance has been
created. The BlockDevOps .drained_begin/end() callbacks depend on the
dataplane instance, so move the blk_set_dev_ops() call after
xen_block_dataplane_create().
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-12-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:28 +0000 (15:02 -0400)]
block: drain from main loop thread in bdrv_co_yield_to_drain()
For simplicity, always run BlockDevOps .drained_begin/end/poll()
callbacks in the main loop thread. This makes it easier to implement the
callbacks and avoids extra locks.
Move the function pointer declarations from the I/O Code section to the
Global State section for BlockDevOps, BdrvChildClass, and BlockDriver.
Narrow IO_OR_GS_CODE() to GLOBAL_STATE_CODE() where appropriate.
The test-bdrv-drain test case calls bdrv_drain() from an IOThread. This
is now only allowed from coroutine context, so update the test case to
run in a coroutine.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-11-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:27 +0000 (15:02 -0400)]
block: add blk_in_drain() API
The BlockBackend quiesce_counter is greater than zero during drained
sections. Add an API to check whether the BlockBackend is in a drained
section.
The next patch will use this API.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-10-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:26 +0000 (15:02 -0400)]
hw/xen: do not use aio_set_fd_handler(is_external=true) in xen_xenstore
There is no need to suspend activity between aio_disable_external() and
aio_enable_external(), which is mainly used for the block layer's drain
operation.
This is part of ongoing work to remove the aio_disable_external() API.
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk> Reviewed-by: Paul Durrant <paul@xen.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-9-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:25 +0000 (15:02 -0400)]
block/export: stop using is_external in vhost-user-blk server
vhost-user activity must be suspended during bdrv_drained_begin/end().
This prevents new requests from interfering with whatever is happening
in the drained section.
Previously this was done using aio_set_fd_handler()'s is_external
argument. In a multi-queue block layer world the aio_disable_external()
API cannot be used since multiple AioContext may be processing I/O, not
just one.
Switch to BlockDevOps->drained_begin/end() callbacks.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-8-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:24 +0000 (15:02 -0400)]
block/export: wait for vhost-user-blk requests when draining
Each vhost-user-blk request runs in a coroutine. When the BlockBackend
enters a drained section we need to enter a quiescent state. Currently
any in-flight requests race with bdrv_drained_begin() because it is
unaware of vhost-user-blk requests.
When blk_co_preadv/pwritev()/etc returns it wakes the
bdrv_drained_begin() thread but vhost-user-blk request processing has
not yet finished. The request coroutine continues executing while the
main loop thread thinks it is in a drained section.
One example where this is unsafe is for blk_set_aio_context() where
bdrv_drained_begin() is called before .aio_context_detached() and
.aio_context_attach(). If request coroutines are still running after
bdrv_drained_begin(), then the AioContext could change underneath them
and they race with new requests processed in the new AioContext. This
could lead to virtqueue corruption, for example.
(This example is theoretical, I came across this while reading the
code and have not tried to reproduce it.)
It's easy to make bdrv_drained_begin() wait for in-flight requests: add
a .drained_poll() callback that checks the VuServer's in-flight counter.
VuServer just needs an API that returns true when there are requests in
flight. The in-flight counter needs to be atomic.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-7-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:23 +0000 (15:02 -0400)]
util/vhost-user-server: rename refcount to in_flight counter
The VuServer object has a refcount field and ref/unref APIs. The name is
confusing because it's actually an in-flight request counter instead of
a refcount.
Normally a refcount destroys the object upon reaching zero. The VuServer
counter is used to wake up the vhost-user coroutine when there are no
more requests.
Avoid confusing by renaming refcount and ref/unref to in_flight and
inc/dec.
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-6-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Stefan Hajnoczi [Tue, 16 May 2023 19:02:22 +0000 (15:02 -0400)]
virtio-scsi: stop using aio_disable_external() during unplug
This patch is part of an effort to remove the aio_disable_external()
API because it does not fit in a multi-queue block layer world where
many AioContexts may be submitting requests to the same disk.
The SCSI emulation code is already in good shape to stop using
aio_disable_external(). It was only used by commit 9c5aad84da1c
("virtio-scsi: fixed virtio_scsi_ctx_check failed when detaching scsi
disk") to ensure that virtio_scsi_hotunplug() works while the guest
driver is submitting I/O.
Ensure virtio_scsi_hotunplug() is safe as follows:
1. qdev_simple_device_unplug_cb() -> qdev_unrealize() ->
device_set_realized() calls qatomic_set(&dev->realized, false) so
that future scsi_device_get() calls return NULL because they exclude
SCSIDevices with realized=false.
That means virtio-scsi will reject new I/O requests to this
SCSIDevice with VIRTIO_SCSI_S_BAD_TARGET even while
virtio_scsi_hotunplug() is still executing. We are protected against
new requests!
2. scsi_qdev_unrealize() already contains a call to
scsi_device_purge_requests() so that in-flight requests are cancelled
synchronously. This ensures that no in-flight requests remain once
qdev_simple_device_unplug_cb() returns.
Thanks to these two conditions we don't need aio_disable_external()
anymore.
Cc: Zhengui Li <lizhengui@huawei.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Daniil Tatianin <d-tatianin@yandex-team.ru> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20230516190238.8401-5-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
projects / mirror_qemu.git / log