]> git.proxmox.com Git - mirror_qemu.git/log
mirror_qemu.git
4 years agoqcow2: Add qcow2_check_fix_snapshot_table()
Max Reitz [Fri, 11 Oct 2019 15:28:07 +0000 (17:28 +0200)]
qcow2: Add qcow2_check_fix_snapshot_table()

qcow2_check_read_snapshot_table() can perform consistency checks, but it
cannot fix everything.  Specifically, it cannot allocate new clusters,
because that should wait until the refcount structures are known to be
consistent (i.e., after qcow2_check_refcounts()).  Thus, it cannot call
qcow2_write_snapshots().

Do that in qcow2_check_fix_snapshot_table(), which is called after
qcow2_check_refcounts().

Currently, there is nothing that would set result->corruptions, so this
is a no-op.  A follow-up patch will change that.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Separate qcow2_check_read_snapshot_table()
Max Reitz [Fri, 11 Oct 2019 15:28:06 +0000 (17:28 +0200)]
qcow2: Separate qcow2_check_read_snapshot_table()

Reading the snapshot table can fail.  That is a problem when we want to
repair the image.

Therefore, stop reading the snapshot table in qcow2_do_open() in check
mode.  Instead, add a new function qcow2_check_read_snapshot_table()
that reads the snapshot table at a later point.  In the future, we want
to handle errors here and fix them.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Write v3-compliant snapshot list on upgrade
Max Reitz [Fri, 11 Oct 2019 15:28:05 +0000 (17:28 +0200)]
qcow2: Write v3-compliant snapshot list on upgrade

qcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size.  Both are
optional for v2 images, so they may not be present.

qcow2_upgrade() therefore should update the snapshot table to ensure all
entries have these extra data fields.

Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1727347
Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Put qcow2_upgrade() into its own function
Max Reitz [Fri, 11 Oct 2019 15:28:04 +0000 (17:28 +0200)]
qcow2: Put qcow2_upgrade() into its own function

This does not make sense right now, but it will make sense once we need
to do more than to just update s->qcow_version.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Make qcow2_write_snapshots() public
Max Reitz [Fri, 11 Oct 2019 15:28:03 +0000 (17:28 +0200)]
qcow2: Make qcow2_write_snapshots() public

Updating the snapshot list will be useful when upgrading a v2 image to
v3, so we will need to call this function in qcow2.c.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-6-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Keep unknown extra snapshot data
Max Reitz [Fri, 11 Oct 2019 15:28:02 +0000 (17:28 +0200)]
qcow2: Keep unknown extra snapshot data

The qcow2 specification says to ignore unknown extra data fields in
snapshot table entries.  Currently, we discard it whenever we update the
image, which is a bit different from "ignore".

This patch makes the qcow2 driver keep all unknown extra data fields
when updating an image's snapshot table.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-5-mreitz@redhat.com
[mreitz: Adjusted comments as proposed by Eric]
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Add Error ** to qcow2_read_snapshots()
Max Reitz [Fri, 11 Oct 2019 15:28:01 +0000 (17:28 +0200)]
qcow2: Add Error ** to qcow2_read_snapshots()

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoqcow2: Use endof()
Max Reitz [Fri, 11 Oct 2019 15:28:00 +0000 (17:28 +0200)]
qcow2: Use endof()

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoinclude: Move endof() up from hw/virtio/virtio.h
Max Reitz [Fri, 11 Oct 2019 15:27:59 +0000 (17:27 +0200)]
include: Move endof() up from hw/virtio/virtio.h

endof() is a useful macro, we can make use of it outside of virtio.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-2-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agomirror: Do not dereference invalid pointers
Max Reitz [Mon, 14 Oct 2019 15:39:28 +0000 (17:39 +0200)]
mirror: Do not dereference invalid pointers

mirror_exit_common() may be called twice (if it is called from
mirror_prepare() and fails, it will be called from mirror_abort()
again).

In such a case, many of the pointers in the MirrorBlockJob object will
already be freed.  This can be seen most reliably for s->target, which
is set to NULL (and then dereferenced by blk_bs()).

Cc: qemu-stable@nongnu.org
Fixes: 737efc1eda23b904fbe0e66b37715fb0e5c3e58b
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191014153931.20699-2-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/nvme: add support for discard
Maxim Levitsky [Fri, 13 Sep 2019 13:36:27 +0000 (16:36 +0300)]
block/nvme: add support for discard

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20190913133627.28450-3-mlevitsk@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/nvme: add support for write zeros
Maxim Levitsky [Fri, 13 Sep 2019 13:36:26 +0000 (16:36 +0300)]
block/nvme: add support for write zeros

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20190913133627.28450-2-mlevitsk@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/block-copy: increase buffered copy request
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:05 +0000 (14:18 +0300)]
block/block-copy: increase buffered copy request

No reason to limit buffered copy to one cluster. Let's allow up to 1
MiB.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-7-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/block-copy: add memory limit
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:04 +0000 (14:18 +0300)]
block/block-copy: add memory limit

Currently total allocation for parallel requests to block-copy instance
is unlimited. Let's limit it to 128 MiB.

For now block-copy is used only in backup, so actually we limit total
allocation for backup job.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-6-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoutil: introduce SharedResource
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:03 +0000 (14:18 +0300)]
util: introduce SharedResource

Introduce an API for some shared splittable resource, like memory.
It's going to be used by backup. Backup uses both read/write io and
copy_range. copy_range may consume memory implictly, so the new API is
abstract: it doesn't allocate any real memory but only hands out
tickets.

The idea is that we have some total amount of something and callers
should wait in coroutine queue if there is not enough of the resource
at the moment.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-5-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/block-copy: refactor copying
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:02 +0000 (14:18 +0300)]
block/block-copy: refactor copying

Merge copying code into one function block_copy_do_copy, which only
calls bdrv_ io functions and don't do any synchronization (like dirty
bitmap set/reset).

Refactor block_copy() function so that it takes full decision about
size of chunk to be copied and does all the synchronization (checking
intersecting requests, set/reset dirty bitmaps).

It will help:
 - introduce parallel processing of block_copy iterations: we need to
   calculate chunk size, start async chunk copying and go to the next
   iteration
 - simplify synchronization improvement (like memory limiting in
   further commit and reducing critical section (now we lock the whole
   requested range, when actually we need to lock only dirty region
   which we handle at the moment))

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/block-copy: limit copy_range_size to 16 MiB
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:01 +0000 (14:18 +0300)]
block/block-copy: limit copy_range_size to 16 MiB

Large copy range may imply memory allocation and large io effort, so
using 2G copy range request may be bad idea. Let's limit it to 16 MiB.
It also helps the following patch to refactor copy-with-offload
fallback to copy-with-bounce-buffer.

Note, that total memory usage of backup is still not limited, it will
be fixed in further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/block-copy: allocate buffer in block_copy_with_bounce_buffer
Vladimir Sementsov-Ogievskiy [Tue, 22 Oct 2019 11:18:00 +0000 (14:18 +0300)]
block/block-copy: allocate buffer in block_copy_with_bounce_buffer

Move bounce_buffer allocation block_copy_with_bounce_buffer. This
commit simplifies further work on implementing copying by larger chunks
(of different size) and further asynchronous handling of block_copy
iterations (with help of block/aio_task API).

Allocation works fast, a lot faster than disk io, so it's not a problem
that we now allocate/free bounce_buffer more times. And we anyway will
have to allocate several bounce_buffers for parallel execution of loop
iterations in future.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Drop TEST_DIR filter from _filter_nbd
Max Reitz [Thu, 17 Oct 2019 13:31:55 +0000 (15:31 +0200)]
iotests: Drop TEST_DIR filter from _filter_nbd

Sockets should be placed into $SOCK_DIR instead of $TEST_DIR, so remove
the $TEST_DIR filter from _filter_nbd.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-24-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/267: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:54 +0000 (15:31 +0200)]
iotests/267: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-23-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/240: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:53 +0000 (15:31 +0200)]
iotests/240: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-22-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/223: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:52 +0000 (15:31 +0200)]
iotests/223: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-21-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/222: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:51 +0000 (15:31 +0200)]
iotests/222: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-20-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/209: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:50 +0000 (15:31 +0200)]
iotests/209: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-19-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/208: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:49 +0000 (15:31 +0200)]
iotests/208: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-18-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/205: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:48 +0000 (15:31 +0200)]
iotests/205: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-17-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/201: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:47 +0000 (15:31 +0200)]
iotests/201: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-16-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/194: Create sockets in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:46 +0000 (15:31 +0200)]
iotests/194: Create sockets in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-15-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/192: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:45 +0000 (15:31 +0200)]
iotests/192: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-14-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/183: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:44 +0000 (15:31 +0200)]
iotests/183: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-13-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/182: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:43 +0000 (15:31 +0200)]
iotests/182: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/181: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:42 +0000 (15:31 +0200)]
iotests/181: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-11-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/147: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:41 +0000 (15:31 +0200)]
iotests/147: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/143: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:40 +0000 (15:31 +0200)]
iotests/143: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/140: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:39 +0000 (15:31 +0200)]
iotests/140: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests/083: Create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:38 +0000 (15:31 +0200)]
iotests/083: Create socket in $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Let common.nbd create socket in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:37 +0000 (15:31 +0200)]
iotests: Let common.nbd create socket in $SOCK_DIR

In addition, drop the nbd_unix_socket assignment in 241 because it does
not really do anything.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-6-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Filter $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:36 +0000 (15:31 +0200)]
iotests: Filter $SOCK_DIR

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-5-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests.py: Add @base_dir to FilePaths etc.
Max Reitz [Thu, 17 Oct 2019 13:31:35 +0000 (15:31 +0200)]
iotests.py: Add @base_dir to FilePaths etc.

Specifying this optional parameter allows creating temporary files in
other directories than the test_dir; for example in sock_dir.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191017133155.5327-4-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests.py: Store socket files in $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:34 +0000 (15:31 +0200)]
iotests.py: Store socket files in $SOCK_DIR

iotests.py itself does not store socket files, but machine.py and
qtest.py do.  iotests.py needs to pass the respective path to them, and
they need to adhere to it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20191017133155.5327-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Introduce $SOCK_DIR
Max Reitz [Thu, 17 Oct 2019 13:31:33 +0000 (15:31 +0200)]
iotests: Introduce $SOCK_DIR

Unix sockets generally have a maximum path length.  Depending on your
$TEST_DIR, it may be exceeded and then all tests that create and use
Unix sockets there may fail.

Circumvent this by adding a new scratch directory specifically for
Unix socket files.  It defaults to a temporary directory (mktemp -d)
that is completely removed after the iotests are done.

(By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which
should be short enough for our use cases.)

Use mkdir -p to create the directory (because it seems right), and do
the same for $TEST_DIR (because there is no reason for that to be
created in any different way).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191017133155.5327-2-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoRevert "mirror: Only mirror granularity-aligned chunks"
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:11 +0000 (12:07 +0300)]
Revert "mirror: Only mirror granularity-aligned chunks"

This reverts commit 9adc1cb49af8d4e54f57980b1eed5c0a4b2dafa6.
    "mirror: Only mirror granularity-aligned chunks"

Since previous commit unaligned chunks are supported by
do_sync_target_write.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191011090711.19940-6-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/mirror: support unaligned write in active mirror
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:10 +0000 (12:07 +0300)]
block/mirror: support unaligned write in active mirror

Prior 9adc1cb49af8d do_sync_target_write had a bug: it reset aligned-up
region in the dirty bitmap, which means that we may not copy some bytes
and assume them copied, which actually leads to producing corrupted
target.

So 9adc1cb49af8d forced dirty bitmap granularity to be
request_alignment for mirror-top filter, so we are not working with
unaligned requests. However forcing large alignment obviously decreases
performance of unaligned requests.

This commit provides another solution for the problem: if unaligned
padding is already dirty, we can safely ignore it, as
1. It's dirty, it will be copied by mirror_iteration anyway
2. It's dirty, so skipping it now we don't increase dirtiness of the
   bitmap and therefore don't damage "synchronicity" of the
   write-blocking mirror.

If unaligned padding is not dirty, we just write it, no reason to touch
dirty bitmap if we succeed (on failure we'll set the whole region
ofcourse, but we loss "synchronicity" on failure anyway).

Note: we need to disable dirty_bitmap, otherwise we will not be able to
see in do_sync_target_write bitmap state before current operation. We
may of course check dirty bitmap before the operation in
bdrv_mirror_top_do_write and remember it, but we don't need active
dirty bitmap for write-blocking mirror anyway.

New code-path is unused until the following commit reverts
9adc1cb49af8d.

Suggested-by: Denis V. Lunev <den@openvz.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191011090711.19940-5-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/block-backend: add blk_co_pwritev_part
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:09 +0000 (12:07 +0300)]
block/block-backend: add blk_co_pwritev_part

Add blk write function with qiov_offset parameter. It's needed for the
following commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191011090711.19940-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoblock/mirror: simplify do_sync_target_write
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:08 +0000 (12:07 +0300)]
block/mirror: simplify do_sync_target_write

do_sync_target_write is called from bdrv_mirror_top_do_write after
write/discard operation, all inside active_write/active_write_settle
protecting us from mirror iteration. So the whole area is dirty for
sure, no reason to examine dirty bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191011090711.19940-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agohbitmap: handle set/reset with zero length
Vladimir Sementsov-Ogievskiy [Fri, 11 Oct 2019 09:07:07 +0000 (12:07 +0300)]
hbitmap: handle set/reset with zero length

Passing zero length to these functions leads to unpredicted results.
Zero-length set/reset may occur in active-mirror, on zero-length write
(which is unlikely, but not guaranteed to never happen).

Let's just do nothing on zero-length request.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191011090711.19940-2-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Cache supported_formats()
Max Reitz [Tue, 17 Sep 2019 09:20:04 +0000 (11:20 +0200)]
iotests: Cache supported_formats()

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190917092004.999-8-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Test driver whitelisting in 136
Max Reitz [Tue, 17 Sep 2019 09:20:03 +0000 (11:20 +0200)]
iotests: Test driver whitelisting in 136

null-aio may not be whitelisted.  Skip all test cases that require it.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190917092004.999-7-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Test driver whitelisting in 093
Max Reitz [Tue, 17 Sep 2019 09:20:02 +0000 (11:20 +0200)]
iotests: Test driver whitelisting in 093

null-aio may not be whitelisted.  Skip all test cases that require it.

(And skip the whole test if null-co is not whitelisted.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190917092004.999-6-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Let skip_if_unsupported accept a function
Max Reitz [Tue, 17 Sep 2019 09:20:01 +0000 (11:20 +0200)]
iotests: Let skip_if_unsupported accept a function

This lets tests use skip_if_unsupported() with a potentially variable
list of required formats.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190917092004.999-5-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Use case_skip() in skip_if_unsupported()
Max Reitz [Tue, 17 Sep 2019 09:20:00 +0000 (11:20 +0200)]
iotests: Use case_skip() in skip_if_unsupported()

skip_if_unsupported() should use the stronger variant case_skip(),
because this allows it to be used even with setUp() (in a meaningful
way).

In the process, make it explicit what we expect the first argument of
the func_wrapper to be (namely something derived of QMPTestCase).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190917092004.999-4-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Allow skipping test cases
Max Reitz [Tue, 17 Sep 2019 09:19:59 +0000 (11:19 +0200)]
iotests: Allow skipping test cases

case_notrun() does not actually skip the current test case.  It just
adds a "notrun" note and then returns to the caller, who manually has to
skip the test.  Generally, skipping a test case is as simple as
returning from the current function, but not always: For example, this
model does not allow skipping tests already in the setUp() function.

Thus, add a QMPTestCase.case_skip() function that invokes case_notrun()
and then self.skipTest().  To make this work, we need to filter the
information on how many test cases were skipped from the unittest
output.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-id: 20190917092004.999-3-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoiotests: Prefer null-co over null-aio
Max Reitz [Tue, 17 Sep 2019 09:19:58 +0000 (11:19 +0200)]
iotests: Prefer null-co over null-aio

We use null-co basically everywhere in the iotests.  Unless we want to
test null-aio specifically, we should use it instead (for consistency).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190917092004.999-2-mreitz@redhat.com
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
4 years agoMerge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-251019-3' into...
Peter Maydell [Sat, 26 Oct 2019 09:13:48 +0000 (10:13 +0100)]
Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-251019-3' into staging

Testing updates (split from mega PR)

  - various Travis dependency updates
  - enable tcg debug for check-tcg
  - additional Xcode build for Cirrus
  - dependency tweak for gitlab

# gpg: Signature made Fri 25 Oct 2019 20:35:56 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-next-251019-3:
  tests/docker: update Travis image to a more current version
  tests/docker: set HOST_ARCH if we don't have ARCH
  travis.yml: --enable-debug-tcg to check-tcg
  gitlab-ci.yml: Use libvdeplug-dev to compile-test the VDE network backend
  travis.yml: cache the clang sanitizer build
  tests/vm/netbsd: Disable IPv6
  tests/vm: Let subclasses disable IPv6
  cirrus.yml: add latest Xcode build target
  travis.yml: bump Xcode 10 to latest dot release
  travis.yml: Test the release tarball
  travis.yml: Fix the ccache lines
  travis.yml: Use newer version of libgnutls and libpng
  travis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image
  travis.yml: Add libvdeplug-dev to compile-test net/vde.c
  travis.yml: reduce scope of the --enable-debug build

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
4 years agoMerge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging
Peter Maydell [Fri, 25 Oct 2019 20:57:41 +0000 (21:57 +0100)]
Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Fri 25 Oct 2019 20:18:23 BST
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  yield_until_fd_readable: make it work with any AioContect
  virtio-blk: Add blk_drain() to virtio_blk_device_unrealize()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
4 years agotests/docker: update Travis image to a more current version
Alex Bennée [Thu, 24 Oct 2019 10:14:32 +0000 (11:14 +0100)]
tests/docker: update Travis image to a more current version

This isn't the latest one available on hub.docker.com but it does
match the ID reported by the Xenial builds running on Travis:

  instance: ... travis-ci-sardonyx-xenial-1553530528-f909ac5

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotests/docker: set HOST_ARCH if we don't have ARCH
Alex Bennée [Mon, 7 Oct 2019 19:03:44 +0000 (19:03 +0000)]
tests/docker: set HOST_ARCH if we don't have ARCH

As the docker rules want to be able to be run on a virgin unconfigured
checkout add a fallback and use it if we need to.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: --enable-debug-tcg to check-tcg
Alex Bennée [Wed, 23 Oct 2019 16:47:51 +0000 (17:47 +0100)]
travis.yml: --enable-debug-tcg to check-tcg

This adds a whole bunch of asserts which will catch bugs you might
introduce into the TCG code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agogitlab-ci.yml: Use libvdeplug-dev to compile-test the VDE network backend
Thomas Huth [Wed, 16 Oct 2019 13:10:02 +0000 (15:10 +0200)]
gitlab-ci.yml: Use libvdeplug-dev to compile-test the VDE network backend

The libvdeplug-dev package is required to compile-test net/vde.c.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20191016131002.29663-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: cache the clang sanitizer build
Alex Bennée [Tue, 22 Oct 2019 13:07:40 +0000 (14:07 +0100)]
travis.yml: cache the clang sanitizer build

Hopefully we'll see the same benefits as the other builds.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agotests/vm/netbsd: Disable IPv6
Eduardo Habkost [Fri, 18 Oct 2019 18:17:05 +0000 (15:17 -0300)]
tests/vm/netbsd: Disable IPv6

Workaround for issues when the host has no IPv6 connectivity.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20191018181705.17957-4-ehabkost@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotests/vm: Let subclasses disable IPv6
Eduardo Habkost [Fri, 18 Oct 2019 18:17:04 +0000 (15:17 -0300)]
tests/vm: Let subclasses disable IPv6

The mechanism will be used to work around issues related to IPv6
on the netbsd image builder.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20191018181705.17957-3-ehabkost@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agocirrus.yml: add latest Xcode build target
Alex Bennée [Wed, 16 Oct 2019 18:27:13 +0000 (19:27 +0100)]
cirrus.yml: add latest Xcode build target

CirrusCI provides a mojave-xcode alias for the latest Xcode available.
Let's use it to make sure we track the latest releases.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: bump Xcode 10 to latest dot release
Alex Bennée [Wed, 16 Oct 2019 18:17:08 +0000 (19:17 +0100)]
travis.yml: bump Xcode 10 to latest dot release

According to:

  https://docs.travis-ci.com/user/reference/osx/#macos-version

we have 10.3 available so lets use it. I don't know what Apple's
deprecation policy is for Xcode because it requires an AppleID to find
out.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agotravis.yml: Test the release tarball
Philippe Mathieu-Daudé [Mon, 7 Oct 2019 16:04:49 +0000 (18:04 +0200)]
travis.yml: Test the release tarball

Add a job to generate the release tarball and build/install few
QEMU targets from it.

Ideally we should build the 'efi' target from the 'roms' directory,
but it is too time consuming.

This job is only triggered when a tag starting with 'v' is pushed,
which is the case with release candidate tags.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20191007160450.3619-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: Fix the ccache lines
Thomas Huth [Wed, 9 Oct 2019 17:07:00 +0000 (19:07 +0200)]
travis.yml: Fix the ccache lines

The "command -v ccache && ccache ..." likely were supposed to test
the availability of ccache before running the program. But this
shell construct causes Travis to abort if ccache is not available.
Use an if-statement instead to fix this problem.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20191009170701.14756-5-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: Use newer version of libgnutls and libpng
Thomas Huth [Wed, 9 Oct 2019 17:06:59 +0000 (19:06 +0200)]
travis.yml: Use newer version of libgnutls and libpng

libgnutls-dev and libpng12-dev are not available in newer versions
of Ubuntu anymore, so installing these packages fails e.g. in the
new arm64 containers on Travis. Let's use newer versions of these
packages by default instead. (The old versions still get tested in
the "gcc-9" build).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20191009170701.14756-4-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image
Thomas Huth [Wed, 9 Oct 2019 17:06:58 +0000 (19:06 +0200)]
travis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image

We've removed support for SDL 1.2 quite a while ago already, so let's
use SDL 2 now in Travis to get test coverage for SDL again.
And while we're at it, also add libsdl2-image-dev which can be used
by QEMU nowadays, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20191009170701.14756-3-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
4 years agotravis.yml: Add libvdeplug-dev to compile-test net/vde.c
Thomas Huth [Wed, 9 Oct 2019 17:06:57 +0000 (19:06 +0200)]
travis.yml: Add libvdeplug-dev to compile-test net/vde.c

This library is needed to compile the VDE network backend.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20191009170701.14756-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agotravis.yml: reduce scope of the --enable-debug build
Alex Bennée [Wed, 2 Oct 2019 10:23:29 +0000 (11:23 +0100)]
travis.yml: reduce scope of the --enable-debug build

Adding debug makes things run a bit slower so lets not hammer all the
targets.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agoMerge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-oct-24-2019-v2' into...
Peter Maydell [Fri, 25 Oct 2019 17:32:26 +0000 (18:32 +0100)]
Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-oct-24-2019-v2' into staging

MIPS queue for October 24th, 2019 - v2

# gpg: Signature made Fri 25 Oct 2019 17:37:29 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-oct-24-2019-v2:
  tests/ssh_linux_malta: Fix 64-bit target tests
  tests/ssh_linux_malta: Refactor how to get image/kernel info
  tests/ssh_linux_malta: Match stricter console output
  tests/ssh_linux_malta: Remove duplicated test
  tests/ssh_linux_malta: Run tests using a snapshot image
  target/mips: Refactor handling of vector compare 'less than' (signed) instructions
  target/mips: Refactor handling of vector compare 'equal' instructions
  target/mips: Demacro LMI decoder
  target/mips: msa: Split helpers for ASUB_<S|U>.<B|H|W|D>
  target/mips: msa: Split helpers for HSUB_<S|U>.<H|W|D>
  target/mips: msa: Split helpers for PCK<EV|OD>.<B|H|W|D>
  target/mips: msa: Split helpers for S<LL|RA|RAR|RL|RLR>.<B|H|W|D>
  target/mips: msa: Split helpers for HADD_<S|U>.<H|W|D>
  target/mips: msa: Split helpers for ADD<_A|S_A|S_S|S_U|V>.<B|H|W|D>
  target/mips: msa: Split helpers for ILV<EV|OD|L|R>.<B|H|W|D>
  target/mips: msa: Split helpers for <MAX|MIN>_<S|U>.<B|H|W|D>
  target/mips: msa: Split helpers for <MAX|MIN>_A.<B|H|W|D>
  MAINTAINERS: Update mail address of Aleksandar Rikalo
  target/mips: Clean up op_helper.c
  target/mips: Clean up helper.c

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
4 years agotests/ssh_linux_malta: Fix 64-bit target tests
Philippe Mathieu-Daudé [Sat, 19 Oct 2019 15:34:37 +0000 (17:34 +0200)]
tests/ssh_linux_malta: Fix 64-bit target tests

Commit 9090d3332cdcc added tests for specific to the 32-bit
machines, which inadvertently make the 64-bit tests failing.
Now than we have this information available in the CPU_INFO
array, use it to have the 64-bit tests back.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <20191019153437.9820-12-f4bug@amsat.org>

4 years agotests/ssh_linux_malta: Refactor how to get image/kernel info
Philippe Mathieu-Daudé [Sat, 19 Oct 2019 15:34:36 +0000 (17:34 +0200)]
tests/ssh_linux_malta: Refactor how to get image/kernel info

The qcow and kernel images use a similar pattern regarding they
are for big/little endianess, or 32/64 bit.
Refactor using more dictionary keys.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <20191019153437.9820-11-f4bug@amsat.org>

4 years agotests/ssh_linux_malta: Match stricter console output
Philippe Mathieu-Daudé [Sat, 19 Oct 2019 15:34:35 +0000 (17:34 +0200)]
tests/ssh_linux_malta: Match stricter console output

Match on stricter console output.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <20191019153437.9820-10-f4bug@amsat.org>

4 years agotests/ssh_linux_malta: Remove duplicated test
Philippe Mathieu-Daudé [Sat, 19 Oct 2019 15:34:34 +0000 (17:34 +0200)]
tests/ssh_linux_malta: Remove duplicated test

Remove duplicated test (probably copy/paste error in
commit 9090d3332cdcc).

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <20191019153437.9820-9-f4bug@amsat.org>

4 years agotests/ssh_linux_malta: Run tests using a snapshot image
Philippe Mathieu-Daudé [Sat, 19 Oct 2019 15:34:33 +0000 (17:34 +0200)]
tests/ssh_linux_malta: Run tests using a snapshot image

If a test fails, it can corrupt the underlying QCow2 image,
making further tests failing.
Fix this by running each test with a snapshot.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <20191019153437.9820-8-f4bug@amsat.org>

4 years agotarget/mips: Refactor handling of vector compare 'less than' (signed) instructions
Filip Bozuta [Wed, 23 Oct 2019 13:37:05 +0000 (15:37 +0200)]
target/mips: Refactor handling of vector compare 'less than' (signed) instructions

Remove unnecessary argument and provide separate function for each
instruction.

Signed-off-by: Filip Bozuta <Filip.Bozuta@rt-rk.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571837825-24438-3-git-send-email-Filip.Bozuta@rt-rk.com>

4 years agotarget/mips: Refactor handling of vector compare 'equal' instructions
Filip Bozuta [Wed, 23 Oct 2019 13:37:04 +0000 (15:37 +0200)]
target/mips: Refactor handling of vector compare 'equal' instructions

Remove unnecessary argument and provide separate function for each
instruction.

Signed-off-by: Filip Bozuta <Filip.Bozuta@rt-rk.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571837825-24438-2-git-send-email-Filip.Bozuta@rt-rk.com>

4 years agotarget/mips: Demacro LMI decoder
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:47 +0000 (12:23 +0200)]
target/mips: Demacro LMI decoder

This makes searches for instances of opcode usages easier.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-15-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for ASUB_<S|U>.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:45 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for ASUB_<S|U>.<B|H|W|D>

Achieves clearer code and slightly better performance.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Message-Id: <1571826227-10583-13-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for HSUB_<S|U>.<H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:44 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for HSUB_<S|U>.<H|W|D>

Achieves clearer code and slightly better performance.

Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Message-Id: <1571826227-10583-12-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for PCK<EV|OD>.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:43 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for PCK<EV|OD>.<B|H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-11-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for S<LL|RA|RAR|RL|RLR>.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:42 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for S<LL|RA|RAR|RL|RLR>.<B|H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-10-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for HADD_<S|U>.<H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:41 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for HADD_<S|U>.<H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-9-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for ADD<_A|S_A|S_S|S_U|V>.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:40 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for ADD<_A|S_A|S_S|S_U|V>.<B|H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-8-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for ILV<EV|OD|L|R>.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:39 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for ILV<EV|OD|L|R>.<B|H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-7-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for <MAX|MIN>_<S|U>.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:38 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for <MAX|MIN>_<S|U>.<B|H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-6-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: msa: Split helpers for <MAX|MIN>_A.<B|H|W|D>
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:37 +0000 (12:23 +0200)]
target/mips: msa: Split helpers for <MAX|MIN>_A.<B|H|W|D>

Achieves clearer code and slightly better performance.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-5-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agoMAINTAINERS: Update mail address of Aleksandar Rikalo
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:36 +0000 (12:23 +0200)]
MAINTAINERS: Update mail address of Aleksandar Rikalo

Aleksandar Rikalo wishes to change his primary mail address for QEMU.
Some minor line order is corrected in .mailmap to be alphabetical,
too.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-4-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: Clean up op_helper.c
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:35 +0000 (12:23 +0200)]
target/mips: Clean up op_helper.c

Mostly fix errors and warnings reported by 'checkpatch.pl -f'.

Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-3-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agotarget/mips: Clean up helper.c
Aleksandar Markovic [Wed, 23 Oct 2019 10:23:34 +0000 (12:23 +0200)]
target/mips: Clean up helper.c

Mostly fix errors and warnings reported by 'checkpatch.pl -f'.

Cc: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Aleksandar Rikalo <aleksandar.rikalo@rt-rk.com>
Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-Id: <1571826227-10583-2-git-send-email-aleksandar.markovic@rt-rk.com>

4 years agoMerge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Peter Maydell [Fri, 25 Oct 2019 13:59:53 +0000 (14:59 +0100)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Fix data corruption bug that is triggered in partial cluster
  allocation with default options
- qapi: add support for blkreplay driver
- doc: Describe missing generic -blockdev options
- iotests: Fix 118 when run as root
- Minor code cleanups

# gpg: Signature made Fri 25 Oct 2019 14:19:04 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation()
  coroutine: Add qemu_co_mutex_assert_locked()
  doc: Describe missing generic -blockdev options
  block/backup: drop dead code from backup_job_create
  blockdev: Use error_report() in hmp_commit()
  iotests: Skip read-only cases in 118 when run as root
  qapi: add support for blkreplay driver

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
4 years agoqcow2: Fix corruption bug in qcow2_detect_metadata_preallocation()
Kevin Wolf [Thu, 24 Oct 2019 14:26:58 +0000 (16:26 +0200)]
qcow2: Fix corruption bug in qcow2_detect_metadata_preallocation()

qcow2_detect_metadata_preallocation() calls qcow2_get_refcount() which
requires s->lock to be taken to protect its accesses to the refcount
table and refcount blocks. However, nothing in this code path actually
took the lock. This could cause the same cache entry to be used by two
requests at the same time, for different tables at different offsets,
resulting in image corruption.

As it would be preferable to base the detection on consistent data (even
though it's just heuristics), let's take the lock not only around the
qcow2_get_refcount() calls, but around the whole function.

This patch takes the lock in qcow2_co_block_status() earlier and asserts
in qcow2_detect_metadata_preallocation() that we hold the lock.

Fixes: 69f47505ee66afaa513305de0c1895a224e52c45
Cc: qemu-stable@nongnu.org
Reported-by: Michael Weiser <michael.weiser@gmx.de>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Michael Weiser <michael.weiser@gmx.de>
Reviewed-by: Michael Weiser <michael.weiser@gmx.de>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
4 years agocoroutine: Add qemu_co_mutex_assert_locked()
Kevin Wolf [Thu, 24 Oct 2019 14:26:57 +0000 (16:26 +0200)]
coroutine: Add qemu_co_mutex_assert_locked()

Some functions require that the caller holds a certain CoMutex for them
to operate correctly. Add a function so that they can assert the lock is
really held.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Michael Weiser <michael.weiser@gmx.de>
Reviewed-by: Michael Weiser <michael.weiser@gmx.de>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
4 years agoMerge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into...
Peter Maydell [Fri, 25 Oct 2019 13:17:08 +0000 (14:17 +0100)]
Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

Fix typos and docs, trivial changes and RTC devices split

# gpg: Signature made Fri 25 Oct 2019 09:35:02 BST
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
  hw/rtc/aspeed_rtc: Remove unused includes
  hw/rtc/xlnx-zynqmp-rtc: Remove unused "ptimer.h" include
  hw/rtc/mc146818: Include mc146818rtc_regs.h a bit less
  hw: Move Aspeed RTC from hw/timer/ to hw/rtc/ subdirectory
  hw: Move Exynos4210 RTC from hw/timer/ to hw/rtc/ subdirectory
  hw: Move Xilinx ZynqMP RTC from hw/timer/ to hw/rtc/ subdirectory
  hw: Move DS1338 device from hw/timer/ to hw/rtc/ subdirectory
  hw: Move TWL92230 device from hw/timer/ to hw/rtc/ subdirectory
  hw: Move sun4v hypervisor RTC from hw/timer/ to hw/rtc/ subdirectory
  hw: Move M41T80 device from hw/timer/ to hw/rtc/ subdirectory
  hw: Move M48T59 device from hw/timer/ to hw/rtc/ subdirectory
  hw: Move MC146818 device from hw/timer/ to hw/rtc/ subdirectory
  hw: Move PL031 device from hw/timer/ to hw/rtc/ subdirectory
  hw/timer: Compile devices not target-dependent as common object
  qemu-timer: reuse MIN macro in qemu_timeout_ns_to_ms
  event_notifier: avoid dandling file descriptor in event_notifier_cleanup
  util/async: avoid useless cast
  pci_bridge: fix a typo in comment
  qemu-options.hx: Update for reboot-timeout parameter

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
# Conflicts:
# hw/timer/trace-events

4 years agodoc: Describe missing generic -blockdev options
Kevin Wolf [Tue, 15 Oct 2019 10:29:58 +0000 (12:29 +0200)]
doc: Describe missing generic -blockdev options

We added more generic options after introducing -blockdev and forgot to
update the documentation (man page and --help output) accordingly. Do
that now.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
4 years agoblock/backup: drop dead code from backup_job_create
Vladimir Sementsov-Ogievskiy [Thu, 17 Oct 2019 14:21:22 +0000 (17:21 +0300)]
block/backup: drop dead code from backup_job_create

After commit 00e30f05de1d195, there is no more "goto error" points
after job creation, so after "error:" @job is always NULL and we don't
need roll-back job creation.

Reported-by: Coverity (CID 1406402)
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
4 years agoblockdev: Use error_report() in hmp_commit()
Kevin Wolf [Mon, 27 May 2019 16:01:41 +0000 (18:01 +0200)]
blockdev: Use error_report() in hmp_commit()

Instead of using monitor_printf() to report errors, hmp_commit() should
use error_report() like other places do.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agoiotests: Skip read-only cases in 118 when run as root
Kevin Wolf [Fri, 18 Oct 2019 11:46:42 +0000 (13:46 +0200)]
iotests: Skip read-only cases in 118 when run as root

Some tests in 118 use chmod to remove write permissions from the file
and assume that the image can indeed not be opened read-write
afterwards. This doesn't work when the test is run as root, because root
can still open the file as writable even when the permission bit isn't
set.

Introduce a @skip_if_root decorator and use it in 118 to skip the tests
in question when the script is run as root.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
4 years agoqapi: add support for blkreplay driver
Pavel Dovgalyuk [Wed, 16 Oct 2019 08:40:39 +0000 (11:40 +0300)]
qapi: add support for blkreplay driver

This patch adds support for blkreplay driver to the blockdev options.
Now blkreplay can be used with -blockdev command line option
in the following format:
-blockdev driver=blkreplay,image=file-node-name,node-name=replay-node-name

This option makes possible implementation of the better command
line support for record/replay invocations.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>