- Bryan's fix on multifd compression level API
- Fabiano's mapped-ram series (base + multifd only)
- Steve's amend on cpr document in qapi/
# -----BEGIN PGP SIGNATURE-----
#
# iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCZeUjKhIccGV0ZXJ4QHJl
# ZGhhdC5jb20ACgkQO1/MzfOr1wbv5QD/ZexBUsmZA5qyxgGvZ2yvlUBEGNOvtmKY
# kRdiYPU7khMA/0N43rn4LcqKCoq4+T+EAnYizGjIyhH/7BRUyn4DUxgO
# =AeEn
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 04 Mar 2024 01:26:02 GMT
# gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706
# gpg: issuer "peterx@redhat.com"
# gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [marginal]
# gpg: aka "Peter Xu <peterx@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706
* tag 'migration-next-pull-request' of https://gitlab.com/peterx/qemu: (27 commits)
migration/multifd: Document two places for mapped-ram
tests/qtest/migration: Add a multifd + mapped-ram migration test
migration/multifd: Add mapped-ram support to fd: URI
migration/multifd: Support incoming mapped-ram stream format
migration/multifd: Support outgoing mapped-ram stream format
migration/multifd: Prepare multifd sync for mapped-ram migration
migration/multifd: Add incoming QIOChannelFile support
migration/multifd: Add outgoing QIOChannelFile support
migration/multifd: Add a wrapper for channels_created
migration/multifd: Allow receiving pages without packets
migration/multifd: Allow multifd without packets
migration/multifd: Decouple recv method from pages
migration/multifd: Rename MultiFDSend|RecvParams::data to compress_data
tests/qtest/migration: Add tests for mapped-ram file-based migration
migration/ram: Add incoming 'mapped-ram' migration
migration/ram: Add outgoing 'mapped-ram' migration
migration: Add mapped-ram URI compatibility check
migration/ram: Introduce 'mapped-ram' migration capability
migration/qemu-file: add utility methods for working with seekable channels
io: fsync before closing a file channel
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
# Conflicts:
# migration/ram.c
Peter Maydell [Mon, 4 Mar 2024 16:01:33 +0000 (16:01 +0000)]
Merge tag 'hppa-latest-pull-request' of https://github.com/hdeller/qemu-hppa into staging
HPPA64 updates
# -----BEGIN PGP SIGNATURE-----
#
# iHUEABYKAB0WIQS86RI+GtKfB8BJu973ErUQojoPXwUCZeQOtQAKCRD3ErUQojoP
# Xzh6AP9Gs8sJzRbR+HLlbrs3ACR7kQEVGOYuuBEJpKqkTyDbLwD/YdOjzgtTRKLd
# imxaWOJaLhpvBrmFLxn3Q58K3n9u1wY=
# =bnMy
# -----END PGP SIGNATURE-----
# gpg: Signature made Sun 03 Mar 2024 05:46:29 GMT
# gpg: using EDDSA key BCE9123E1AD29F07C049BBDEF712B510A23A0F5F
# gpg: Good signature from "Helge Deller <deller@gmx.de>" [unknown]
# gpg: aka "Helge Deller <deller@kernel.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 4544 8228 2CD9 10DB EF3D 25F8 3E5F 3D04 A7A2 4603
# Subkey fingerprint: BCE9 123E 1AD2 9F07 C049 BBDE F712 B510 A23A 0F5F
* tag 'hppa-latest-pull-request' of https://github.com/hdeller/qemu-hppa:
roms/hppa: Add build rules for hppa-firmware
pc-bios/README: Add information about hppa-firmware
pc-bios/meson: Add hppa-firmware64.img blob
target/hppa: Restore unwind_breg before calculating ior
target: hppa: Fix unaligned double word accesses for hppa64
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Cc: qemu-stable@nongnu.org Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20240304144456.3825935-3-berrange@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
meson: detect broken clang 17 with -fzero-call-used-regs
Clang 17 will segv if given -fzero-call-used-regs and optimization
is enabled. Since upstream hasn't triaged the bug, distros are
increasingly shipping with broken Clang.
https://github.com/llvm/llvm-project/issues/75168
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=277474 Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20240304144456.3825935-2-berrange@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Thomas Huth [Mon, 4 Mar 2024 13:04:03 +0000 (14:04 +0100)]
docs/conf.py: Remove usage of distutils
The macOS jobs in our CI recently started failing, complaining that
the distutils module is not available anymore. And indeed, according to
https://peps.python.org/pep-0632/ it's been deprecated since a while
and now likely got removed in recent Python versions.
Fortunately, we only use it for a version check via LooseVersion here
which we don't really need anymore - according to Repology.org, these
are the versions of sphinx-rtd-theme that are currently used by the
various distros:
So except for CentOS 8, all distros are using a newer version of
sphinx-rtd-theme, and for CentOS 8 we don't support compiling with
the Sphinx of the distro anymore anyway, since it's based on the
Python 3.6 interpreter there. For compiling on CentOS 8, you have
to use the alternative Python 3.8 interpreter which comes without
Sphinx, so that needs the Sphinx installed via pip in the venv
instead, and that is using a newer version, too, according to our
pythondeps.toml file.
Thus we can simply drop the version check now to get rid of the
distutils dependency here.
Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Message-id: 20240304130403.129543-1-thuth@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Steve Sistare [Tue, 27 Feb 2024 15:33:21 +0000 (16:33 +0100)]
migration: simplify exec migration functions
Simplify the exec migration code by using list utility functions.
As a side effect, this also fixes a minor memory leak. On function return,
"g_auto(GStrv) argv" frees argv and each element, which is wrong, because
the function does not own the individual elements. To compensate, the code
uses g_steal_pointer which NULLs argv and prevents the destructor from
running, but argv is leaked.
docs/devel/writing-monitor-commands: Minor improvements
Avoid "JSON" when talking about the QAPI schema syntax. Capitalize
QEMU. Don't claim all HMP commands live in monitor/hmp-cmds.c (this
was never true). Fix punctuation and drop inappropriate "the" here
and there.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227115617.237875-3-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
docs/devel/writing-monitor-commands: Repair a decade of rot
The tutorial doesn't match reality since at least 2013. Repairing it
involves fixing the following issues:
* Update for commit 6d327171551 (aio / timers: Remove alarm timers):
replace the broken examples. Instead of having one for returning a
struct and another for returning a list of structs, do just one for
the latter. This resolves the FIXME added in commit e218052f928 (aio / timers: De-document -clock) back in 2014.
* Update for commit 895a2a80e0e (qapi: Use 'struct' instead of 'type'
in schema).
* Update for commit 3313b6124b5 (qapi: add qapi2texi script): add
required documentation to the schema snippets, and drop section
"Command Documentation".
* Update for commit a3c45b3e629 (qapi: New special feature flag
"unstable"): supply the required feature, deemphasize the x- prefix.
* Update for commit dd98234c059 (qapi: introduce x-query-roms QMP
command): rephrase from "add new command" to "examine existing
command".
* Update for commit 9492718b7c0 (qapi misc: Elide redundant has_FOO in
generated C): hello-world's message argument no longer comes with a
has_message, add a second argument that does.
* Update for moved and renamed files.
While there, update QMP version output to current output.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227115617.237875-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
[Whitespace tidied up, typo fixed]
Documentation claims the command can "return NULL". "NULL" doesn't
exist in JSON. "null" does, but the command returns lists, and null
isn't. Correct documentation to "return an empty list".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-13-armbru@redhat.com>
qga/qapi-schema: Tweak documentation of fsfreeze commands
"Returns:" sections of guest-fsfreeze-freeze and
guest-fsfreeze-freeze-list describe both command behavior and success
response. Move behavior out, so "Returns:" is only about success
response.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-12-armbru@redhat.com>
Helge Deller [Sat, 2 Mar 2024 21:02:38 +0000 (22:02 +0100)]
target/hppa: Restore unwind_breg before calculating ior
When calculating the IOR for the exception handlers, the current
unwind_breg value is needed on 64-bit hppa machines.
Restore that value by calling cpu_restore_state() earlier, which in turn
calls hppa_restore_state_to_opc() which restores the unwind_breg for the
current instruction.
Signed-off-by: Helge Deller <deller@gmx.de> Fixes: 3824e0d643f3 ("target/hppa: Export function hppa_set_ior_and_isr()") Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Guenter Roeck [Sat, 17 Feb 2024 01:58:11 +0000 (17:58 -0800)]
target: hppa: Fix unaligned double word accesses for hppa64
Unaligned 64-bit accesses were found in Linux to clobber carry bits,
resulting in bad results if an arithmetic operation involving a
carry bit was executed after an unaligned 64-bit operation.
hppa 2.0 defines additional carry bits in PSW register bits 32..39.
When restoring PSW after executing an unaligned instruction trap, those
bits were not cleared and ended up to be active all the time. Since there
are no bits other than the upper carry bits needed in the upper 32 bit of
env->psw and since those are stored in env->psw_cb, just clear the entire
upper 32 bit when storing psw to solve the problem unconditionally.
Fixes: 931adff31478 ("target/hppa: Update cpu_hppa_get/put_psw for hppa64") Cc: Richard Henderson <richard.henderson@linaro.org> Cc: Charlie Jenkins <charlie@rivosinc.com> Cc: Helge Deller <deller@gmx.de> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Guenter Roeck <linux@roeck-us.net> Signed-off-by: Helge Deller <deller@gmx.de>
Ilya Leoshkevich [Fri, 23 Feb 2024 11:31:40 +0000 (12:31 +0100)]
tests/tcg: Check that shmat() does not break /proc/self/maps
Add a regression test for a recently fixed issue, where shmat()
desynced the guest and the host view of the address space and caused
open("/proc/self/maps") to SEGV.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <jwyuvao4apydvykmsnvacwshdgy3ixv7qvkh4dbxm3jkwgnttw@k4wpaayou7oq> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Handle combined host and guest alignment requirements.
Handle host and guest page size differences.
Handle SHM_EXEC.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/115 Tested-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Peter Maydell [Fri, 1 Mar 2024 10:14:32 +0000 (10:14 +0000)]
Merge tag 'pull-request-2024-03-01' of https://gitlab.com/thuth/qemu into staging
* Fix some bugs in the vring setup of libqos
* Fix GIC settings when using --without-default-devices
* Fix USB PCAP streams on Windows
* Remove temporary files from test-util-sockets
* Fix TLS io channels sending too much data to the backend
* tag 'pull-request-2024-03-01' of https://gitlab.com/thuth/qemu:
chardev/char-socket: Fix TLS io channels sending too much data to the backend
tests/unit/test-util-sockets: Remove temporary file after test
hw/usb/bus.c: PCAP adding 0xA in Windows version
hw/intc/Kconfig: Fix GIC settings when using "--without-default-devices"
libqos/virtio.c: fix 'avail_event' offset in qvring_init()
libqos/virtio.c: init all elems in qvring_indirect_desc_setup()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
# -----BEGIN PGP SIGNATURE-----
#
# iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZeBrwAAKCRBAov/yOSY+
# 33YXA/4+A5Bpe/3+mSAWZSUlluGTqUi0ILBYRMyX1RXovMx4uCRGr7PXzAf03yKS
# MZzlVzTuOK69WmTm/iTdYWOxkXisC3gzxL/wm8hP4lzh4c0dHrHRsKHqq6gR3+t2
# ojdZn7TefeflnNqIhxXxgxb1OETofhBNnBJ74pvqxO7XV5SWnA==
# =J2Kb
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 29 Feb 2024 11:34:24 GMT
# gpg: using RSA key B8FF1DA0D2FDCB2DA09C6C2C40A2FFF239263EDF
# gpg: Good signature from "Song Gao <m17746591750@163.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B8FF 1DA0 D2FD CB2D A09C 6C2C 40A2 FFF2 3926 3EDF
* tag 'pull-loongarch-20240229' of https://gitlab.com/gaosong/qemu:
loongarch: Change the UEFI loading mode to loongarch
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Fabiano Rosas [Thu, 29 Feb 2024 15:30:16 +0000 (12:30 -0300)]
migration/multifd: Add mapped-ram support to fd: URI
If we receive a file descriptor that points to a regular file, there's
nothing stopping us from doing multifd migration with mapped-ram to
that file.
Enable the fd: URI to work with multifd + mapped-ram.
Note that the fds passed into multifd are duplicated because we want
to avoid cross-thread effects when doing cleanup (i.e. close(fd)). The
original fd doesn't need to be duplicated because monitor_get_fd()
transfers ownership to the caller.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:15 +0000 (12:30 -0300)]
migration/multifd: Support incoming mapped-ram stream format
For the incoming mapped-ram migration we need to read the ramblock
headers, get the pages bitmap and send the host address of each
non-zero page to the multifd channel thread for writing.
Usage on HMP is:
(qemu) migrate_set_capability multifd on
(qemu) migrate_set_capability mapped-ram on
(qemu) migrate_incoming file:migfile
(the ram.h include needs to move because we've been previously relying
on it being included from migration.c. Now file.h will start including
multifd.h before migration.o is processed)
Fabiano Rosas [Thu, 29 Feb 2024 15:30:14 +0000 (12:30 -0300)]
migration/multifd: Support outgoing mapped-ram stream format
The new mapped-ram stream format uses a file transport and puts ram
pages in the migration file at their respective offsets and can be
done in parallel by using the pwritev system call which takes iovecs
and an offset.
Add support to enabling the new format along with multifd to make use
of the threading and page handling already in place.
This requires multifd to stop sending headers and leaving the stream
format to the mapped-ram code. When it comes time to write the data, we
need to call a version of qio_channel_write that can take an offset.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:13 +0000 (12:30 -0300)]
migration/multifd: Prepare multifd sync for mapped-ram migration
The mapped-ram migration can be performed live or non-live, but it is
always asynchronous, i.e. the source machine and the destination
machine are not migrating at the same time. We only need some pieces
of the multifd sync operations.
multifd_send_sync_main()
------------------------
Issued by the ram migration code on the migration thread, causes the
multifd send channels to synchronize with the migration thread and
makes the sending side emit a packet with the MULTIFD_FLUSH flag.
With mapped-ram we want to maintain the sync on the sending side
because that provides ordering between the rounds of dirty pages when
migrating live.
MULTIFD_FLUSH
-------------
On the receiving side, the presence of the MULTIFD_FLUSH flag on a
packet causes the receiving channels to start synchronizing with the
main thread.
We're not using packets with mapped-ram, so there's no MULTIFD_FLUSH
flag and therefore no channel sync on the receiving side.
multifd_recv_sync_main()
------------------------
Issued by the migration thread when the ram migration flag
RAM_SAVE_FLAG_MULTIFD_FLUSH is received, causes the migration thread
on the receiving side to start synchronizing with the recv
channels. Due to compatibility, this is also issued when
RAM_SAVE_FLAG_EOS is received.
For mapped-ram we only need to synchronize the channels at the end of
migration to avoid doing cleanup before the channels have finished
their IO.
Make sure the multifd syncs are only issued at the appropriate times.
Note that due to pre-existing backward compatibility issues, we have
the multifd_flush_after_each_section property that can cause a sync to
happen at EOS. Since the EOS flag is needed on the stream, allow
mapped-ram to just ignore it.
Also emit an error if any other unexpected flags are found on the
stream.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:12 +0000 (12:30 -0300)]
migration/multifd: Add incoming QIOChannelFile support
On the receiving side we don't need to differentiate between main
channel and threads, so whichever channel is defined first gets to be
the main one. And since there are no packets, use the atomic channel
count to index into the params array.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:11 +0000 (12:30 -0300)]
migration/multifd: Add outgoing QIOChannelFile support
Allow multifd to open file-backed channels. This will be used when
enabling the mapped-ram migration stream format which expects a
seekable transport.
The QIOChannel read and write methods will use the preadv/pwritev
versions which don't update the file offset at each call so we can
reuse the fd without re-opening for every channel.
Contrary to the socket migration, the file migration doesn't need an
asynchronous channel creation process, so expose
multifd_channel_connect() and call it directly.
Note that this is just setup code and multifd cannot yet make use of
the file channels.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:09 +0000 (12:30 -0300)]
migration/multifd: Allow receiving pages without packets
Currently multifd does not need to have knowledge of pages on the
receiving side because all the information needed is within the
packets that come in the stream.
We're about to add support to mapped-ram migration, which cannot use
packets because it expects the ramblock section in the migration file
to contain only the guest pages data.
Add a data structure to transfer pages between the ram migration code
and the multifd receiving threads.
We don't want to reuse MultiFDPages_t for two reasons:
a) multifd threads don't really need to know about the data they're
receiving.
b) the receiving side has to be stopped to load the pages, which means
we can experiment with larger granularities than page size when
transferring data.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:08 +0000 (12:30 -0300)]
migration/multifd: Allow multifd without packets
For the upcoming support to the new 'mapped-ram' migration stream
format, we cannot use multifd packets because each write into the
ramblock section in the migration file is expected to contain only the
guest pages. They are written at their respective offsets relative to
the ramblock section header.
There is no space for the packet information and the expected gains
from the new approach come partly from being able to write the pages
sequentially without extraneous data in between.
The new format also simply doesn't need the packets and all necessary
information can be taken from the standard migration headers with some
(future) changes to multifd code.
Use the presence of the mapped-ram capability to decide whether to
send packets.
This only moves code under multifd_use_packets(), it has no effect for
now as mapped-ram cannot yet be enabled with multifd.
Fabiano Rosas [Thu, 29 Feb 2024 15:30:07 +0000 (12:30 -0300)]
migration/multifd: Decouple recv method from pages
Next patches will abstract the type of data being received by the
channels, so do some cleanup now to remove references to pages and
dependency on 'normal_num'.
Add the necessary code to parse the format changes for the
'mapped-ram' capability.
One of the more notable changes in behavior is that in the
'mapped-ram' case ram pages are restored in one go rather than
constantly looping through the migration stream.
Implement the outgoing migration side for the 'mapped-ram' capability.
A bitmap is introduced to track which pages have been written in the
migration file. Pages are written at a fixed location for every
ramblock. Zero pages are ignored as they'd be zero in the destination
migration as well.
The migration stream is altered to put the dirty pages for a ramblock
after its header instead of having a sequential stream of pages that
follow the ramblock headers.
Without mapped-ram (current): With mapped-ram (new):
where:
- ramblock header: the generic information for a ramblock, such as
idstr, used_len, etc.
- ramblock mapped-ram header: the new information added by this
feature: bitmap of pages written, bitmap size and offset of pages
in the migration file.
The core of the feature is to ensure that RAM pages are mapped
directly to offsets in the resulting migration file instead of being
streamed at arbitrary points.
The reasons why we'd want such behavior are:
- The resulting file will have a bounded size, since pages which are
dirtied multiple times will always go to a fixed location in the
file, rather than constantly being added to a sequential
stream. This eliminates cases where a VM with, say, 1G of RAM can
result in a migration file that's 10s of GBs, provided that the
workload constantly redirties memory.
- It paves the way to implement O_DIRECT-enabled save/restore of the
migration stream as the pages are ensured to be written at aligned
offsets.
- It allows the usage of multifd so we can write RAM pages to the
migration file in parallel.
For now, enabling the capability has no effect. The next couple of
patches implement the core functionality.
Fabiano Rosas [Thu, 29 Feb 2024 15:29:59 +0000 (12:29 -0300)]
io: fsync before closing a file channel
Make sure the data is flushed to disk before closing file
channels. This is to ensure data is on disk and not lost in the event
of a host crash.
This is currently being implemented to affect the migration code when
migrating to a file, but all QIOChannelFile users should benefit from
the change.
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Acked-by: "Daniel P. Berrangé" <berrange@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-6-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
Nikolay Borisov [Thu, 29 Feb 2024 15:29:57 +0000 (12:29 -0300)]
io: Add generic pwritev/preadv interface
Introduce basic pwritev/preadv support in the generic channel layer.
Specific implementation will follow for the file channel as this is
required in order to support migration streams with fixed location of
each ram page.
Signed-off-by: Nikolay Borisov <nborisov@suse.com> Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-4-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
Nikolay Borisov [Thu, 29 Feb 2024 15:29:56 +0000 (12:29 -0300)]
io: add and implement QIO_CHANNEL_FEATURE_SEEKABLE for channel file
Add a generic QIOChannel feature SEEKABLE which would be used by the
qemu_file* apis. For the time being this will be only implemented for
file channels.
Signed-off-by: Nikolay Borisov <nborisov@suse.com> Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Fabiano Rosas <farosas@suse.de> Link: https://lore.kernel.org/r/20240229153017.2221-3-farosas@suse.de Signed-off-by: Peter Xu <peterx@redhat.com>
Fabiano Rosas [Thu, 29 Feb 2024 15:29:55 +0000 (12:29 -0300)]
migration/multifd: Cleanup multifd_recv_sync_main
Some minor cleanups and documentation for multifd_recv_sync_main.
Use thread_count as done in other parts of the code. Remove p->id from
the multifd_recv_state sync, since that is global and not tied to a
channel. Add documentation for the sync steps.
Thomas Huth [Thu, 29 Feb 2024 10:43:37 +0000 (11:43 +0100)]
chardev/char-socket: Fix TLS io channels sending too much data to the backend
Commit ffda5db65a ("io/channel-tls: fix handling of bigger read buffers")
changed the behavior of the TLS io channels to schedule a second reading
attempt if there is still incoming data pending. This caused a regression
with backends like the sclpconsole that check in their read function that
the sender does not try to write more bytes to it than the device can
currently handle.
The problem can be reproduced like this:
1) In one terminal, do this:
mkdir qemu-pki
cd qemu-pki
openssl genrsa 2048 > ca-key.pem
openssl req -new -x509 -nodes -days 365000 -key ca-key.pem -out ca-cert.pem
# enter some dummy value for the cert
openssl genrsa 2048 > server-key.pem
openssl req -new -x509 -nodes -days 365000 -key server-key.pem \
-out server-cert.pem
# enter some other dummy values for the cert
It looks like the second read does not trigger the chr_can_read() function
to be called before the second read, which should normally always be done
before sending bytes to a character device to see how much it can handle,
so the s->max_size in tcp_chr_read() still contains the old value from the
previous read. Let's make sure that we use the up-to-date value by calling
tcp_chr_read_poll() again here.
Fixes: ffda5db65a ("io/channel-tls: fix handling of bigger read buffers") Buglink: https://issues.redhat.com/browse/RHEL-24614 Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Message-ID: <20240229104339.42574-1-thuth@redhat.com> Reviewed-by: Antoine Damhet <antoine.damhet@blade-group.com> Tested-by: Antoine Damhet <antoine.damhet@blade-group.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
Since Windows text files use CRLFs for all \n, the Windows version of QEMU
inserts a CR in the PCAP stream when a LF is encountered when using USB PCAP
files. This is due to the fact that the PCAP file is opened as TEXT instead
of BINARY.
To show an example, when using a very common protocol to USB disks, the BBB
protocol uses a 10-byte command packet. For example, the READ_CAPACITY(10)
command will have a command block length of 10 (0xA). When this 10-byte
command (part of the 31-byte CBW) is placed into the PCAP file, the Windows
file manager inserts a 0xD before the 0xA, turning the 31-byte CBW into a
32-byte CBW.
I believe simply opening the PCAP file as BINARY instead of TEXT will fix
this issue.
Resolves: https://bugs.launchpad.net/qemu/+bug/2054889 Signed-off-by: Benjamin David Lunt <benlunt@fysnet.net>
Message-ID: <000101da6823$ce1bbf80$6a533e80$@fysnet.net>
[thuth: Break long line to avoid checkpatch.pl error] Signed-off-by: Thomas Huth <thuth@redhat.com>
Thomas Huth [Wed, 21 Feb 2024 11:00:59 +0000 (12:00 +0100)]
hw/intc/Kconfig: Fix GIC settings when using "--without-default-devices"
When using "--without-default-devices", the ARM_GICV3_TCG and ARM_GIC_KVM
settings currently get disabled, though the arm virt machine is only of
very limited use in that case. This also causes the migration-test to
fail in such builds. Let's make sure that we always keep the GIC switches
enabled in the --without-default-devices builds, too.
Message-ID: <20240221110059.152665-1-thuth@redhat.com> Tested-by: Fabiano Rosas <farosas@suse.de> Signed-off-by: Thomas Huth <thuth@redhat.com>
libqos/virtio.c: fix 'avail_event' offset in qvring_init()
In qvring_init() we're writing vq->used->avail_event at "vq->used + 2 +
array_size". The struct pointed by vq->used is, from virtio_ring.h
Linux header):
* // A ring of used descriptor heads with free-running index.
* __virtio16 used_flags;
* __virtio16 used_idx;
* struct vring_used_elem used[num];
* __virtio16 avail_event_idx;
So 'flags' is the word right at vq->used. 'idx' is vq->used + 2. We need
to skip 'used_idx' by adding + 2 bytes, and then sum the vector size, to
reach avail_event_idx. An example on how to properly access this field
can be found in qvirtqueue_kick():
This error was detected when enabling the RISC-V 'virt' libqos machine.
The 'idx' test from vhost-user-blk-test.c errors out with a timeout in
qvirtio_wait_used_elem(). The timeout happens because when processing
the first element, 'avail_event' is read in qvirtqueue_kick() as non-zero
because we didn't initialize it properly (and the memory at that point
happened to be non-zero). 'idx' is 0.
All of this makes this condition fail because "idx - avail_event" will
overflow and be non-zero:
/* < 1 because we add elements to avail queue one by one */
if ((flags & VRING_USED_F_NO_NOTIFY) == 0 &&
(!vq->event || (uint16_t)(idx-avail_event) < 1)) {
d->bus->virtqueue_kick(d, vq);
}
As a result the virtqueue is never kicked and we'll timeout waiting for it.
Fixes: 1053587c3f ("libqos: Added EVENT_IDX support") Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240217192607.32565-3-dbarboza@ventanamicro.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
libqos/virtio.c: init all elems in qvring_indirect_desc_setup()
The loop isn't setting the values for the last element. Every other
element is being initialized with addr = 0, flags = VRING_DESC_F_NEXT
and next = i + 1. The last elem is never touched.
This became a problem when enabling a RISC-V 'virt' libqos machine in
the 'indirect' test of virti-blk-test.c. The 'flags' for the last
element will end up being an odd number (since we didn't touch it).
Being an odd number it will be mistaken by VRING_DESC_F_NEXT, which
happens to be 1.
Deep into hw/virt/virtio.c, in virtqueue_split_pop(), into
virtqueue_split_read_next_desc(), a check for VRING_DESC_F_NEXT will be
made to see if we're supposed to chain. The code will keep up chaining
in the last element because the uninitialized value happens to be odd.
We'll error out right after that because desc->next (which is also
uninitialized) will be >= max. A VIRTQUEUE_READ_DESC_ERROR will be
returned, with an error message like this in the stderr:
qemu-system-riscv64: Desc next is 49391
Since we never returned, we'll end up timing out at qvirtio_wait_used_elem():
The root cause is using uninitialized values from guest_alloc() in
qvring_indirect_desc_setup(). There's no guarantee that the memory pages
retrieved will be zeroed, so we can't make assumptions. In fact, commit 5b4f72f5e8 ("tests/qtest: properly initialise the vring used idx") fixed a
similar problem stating "It is probably not wise to assume guest memory
is zeroed anyway". I concur.
Initialize all elems in qvring_indirect_desc_setup().
Fixes: f294b029aa ("libqos: Added indirect descriptor support to virtio implementation") Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240217192607.32565-2-dbarboza@ventanamicro.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
Some glue code was missing, so that using `qmp_migrate_set_parameters`
to set `multifd-zstd-level` or `multifd-zlib-level` did not work. This
commit adds the glue code to fix that.
Steve Sistare [Thu, 29 Feb 2024 14:54:22 +0000 (06:54 -0800)]
migration: massage cpr-reboot documentation
Re-wrap the cpr-reboot documentation to 70 columns, use '@' for
cpr-reboot references, capitalize COLO and VFIO, and tweak the
wording.
Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Steve Sistare <steven.sistare@oracle.com> Link: https://lore.kernel.org/r/1709218462-3640-1-git-send-email-steven.sistare@oracle.com
[peterx: s/qemu/QEMU per Markus's suggestion] Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com>
linux-user/x86_64: Handle the vsyscall page in open_self_maps_{2,4}
This is the only case in which we expect to have no host memory backing
for a guest memory page, because in general linux user processes cannot
map any pages in the top half of the 64-bit address space.
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2170 Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Paolo Bonzini [Wed, 28 Feb 2024 11:06:41 +0000 (12:06 +0100)]
tcg/optimize: fix uninitialized variable
The variables uext_opc and sext_opc are used without initialization if
TCG_TARGET_extract_i{32,64}_valid returns false. The result, depending
on the compiler, might be the generation of extract and sextract opcodes
with invalid offset and count, or just random data in the TCG opcode
stream.
Fixes: ceb9ee06b71 ("tcg/optimize: Handle TCG_COND_TST{EQ,NE}", 2024-02-03) Cc: Richard Henderson <pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240228110641.287205-1-pbonzini@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The assertion was never correct, because the alignment is a composite
of the image alignment and SHMLBA. Even if the image alignment didn't
match the image address, an assertion would not be correct -- more
appropriate would be an error message about an ill formed image. But
the image cannot be held to SHMLBA under any circumstances.
Fixes: ee94743034b ("linux-user: completely re-write init_guest_space")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2157 Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reported-by: Alexey Sheplyakov <asheplyakov@yandex.ru> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
This option controls the host page size. From the mis-usage in
our own testsuite, this is easily confused with guest page size.
The only thing that occurs when changing the host page size is
that stuff breaks, because one cannot actually change the host
page size. Therefore reject all but the no-op setting as part
of the deprecation process.
Reviewed-by: Warner Losh <imp@bsdimp.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20240102015808.132373-27-richard.henderson@linaro.org>
When guest page size > host page size, this test can fail
due to the SIGBUS protection hack. Avoid this by making
sure that the file size is at least one guest page.
Visible with alpha guest on x86_64 host.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com> Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20240102015808.132373-26-richard.henderson@linaro.org>
Work much harder to get alignment and mapping beyond the end
of the file correct. Both of which are excercised by our
test-mmap for alpha (8k pages) on any 4k page host.
We cannot skip over the_end1 to the_end, because we fail to
record the validity of the guest page with the interval tree.
Remove "the_end" and rename "the_end1" to "the_end".
projects / mirror_qemu.git / log