Eduardo Habkost [Wed, 8 Jun 2016 20:50:24 +0000 (17:50 -0300)]
pxa2xx: Unconditionally enable USB controller
Simplify initialization logic by removing the usb_enabled()
check. The USB controller is part of the SoC, so it doesn't make
sense to create a system where it is not present.
Cc: Peter Maydell <peter.maydell@linaro.org> Cc: Andrzej Zaborowski <balrogg@gmail.com> Cc: qemu-arm@nongnu.org, Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 1465419025-21519-2-git-send-email-ehabkost@redhat.com Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Peter Maydell [Fri, 10 Jun 2016 15:37:57 +0000 (16:37 +0100)]
hw/usb/dev-network.c: Use ldl_le_p() and stl_le_p()
Use stl_le_p() and ldl_le_p() to read and write data from
buffers, rather than using pointer casts and cpu_to_le32()
for writes and le32_to_cpup() for reads. This:
* avoids lots of casts
* works even if the buffer isn't as aligned as the host would like
* avoids using the *_to_cpup() functions which we want to get rid of
Note that there may still be some places where a pointer from the
guest is cast to a pointer to a host structure; these would also
have to be changed for the device to work on a host CPU which
enforces alignment restrictions.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1465573077-29221-1-git-send-email-peter.maydell@linaro.org Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Gerd Hoffmann [Fri, 3 Jun 2016 09:12:55 +0000 (11:12 +0200)]
usb-host: add special case for bus+addr
This patch changes usb-host behavior in case we hostbus= and hostaddr=
properties are used to identify the usb device in question. Instead of
adding the device to the hotplug watchlist we try to open directly using
the given bus number and device address.
Putting a device specified by hostaddr to the hotplug watchlist isn't
a great idea as the address isn't a fixed property. It changes every
time the device is plugged in. So considering this case as "use the
device at bus:addr _now_" is more sane. Also usb-host will throw errors
in case it can't initialize the host device.
Note: For devices on the hotplug watchlist (hostport or vendorid or
productid specified) qemu continues to ignore errors and keeps
monitoring the usb bus to see if the device eventually shows up.
Peter Maydell [Mon, 13 Jun 2016 09:12:44 +0000 (10:12 +0100)]
Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20160611' into staging
TB hashing improvements
# gpg: Signature made Sun 12 Jun 2016 01:12:50 BST
# gpg: using RSA key 0xAD1270CC4DD0279B
# gpg: Good signature from "Richard Henderson <rth7680@gmail.com>"
# gpg: aka "Richard Henderson <rth@redhat.com>"
# gpg: aka "Richard Henderson <rth@twiddle.net>"
# Primary key fingerprint: 9CB1 8DDA F8E8 49AD 2AFC 16A4 AD12 70CC 4DD0 279B
* remotes/rth/tags/pull-tcg-20160611:
translate-all: add tb hash bucket info to 'info jit' dump
tb hash: track translated blocks with qht
qht: add test-qht-par to invoke qht-bench from 'check' target
qht: add qht-bench, a performance benchmark
qht: add test program
qht: QEMU's fast, resizable and scalable Hash Table
qdist: add test program
qdist: add module to represent frequency distributions of data
tb hash: hash phys_pc, pc, and flags with xxhash
exec: add tb_hash_func5, derived from xxhash
qemu-thread: add simple test-and-set spinlock
include/processor.h: define cpu_relax()
seqlock: rename write_lock/unlock to write_begin/end
seqlock: remove optional mutex
compiler.h: add QEMU_ALIGNED() to enforce struct alignment
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Emilio G. Cota [Wed, 8 Jun 2016 18:55:32 +0000 (14:55 -0400)]
tb hash: track translated blocks with qht
Having a fixed-size hash table for keeping track of all translation blocks
is suboptimal: some workloads are just too big or too small to get maximum
performance from the hash table. The MRU promotion policy helps improve
performance when the hash table is a little undersized, but it cannot
make up for severely undersized hash tables.
Furthermore, frequent MRU promotions result in writes that are a scalability
bottleneck. For scalability, lookups should only perform reads, not writes.
This is not a big deal for now, but it will become one once MTTCG matures.
The appended fixes these issues by using qht as the implementation of
the TB hash table. This solution is superior to other alternatives considered,
namely:
- master: implementation in QEMU before this patchset
- xxhash: before this patch, i.e. fixed buckets + xxhash hashing + MRU.
- xxhash-rcu: fixed buckets + xxhash + RCU list + MRU.
MRU is implemented here by adding an intermediate struct
that contains the u32 hash and a pointer to the TB; this
allows us, on an MRU promotion, to copy said struct (that is not
at the head), and put this new copy at the head. After a grace
period, the original non-head struct can be eliminated, and
after another grace period, freed.
- qht-fixed-nomru: fixed buckets + xxhash + qht without auto-resize +
no MRU for lookups; MRU for inserts.
The appended solution is the following:
- qht-dyn-nomru: dynamic number of buckets + xxhash + qht w/ auto-resize +
no MRU for lookups; MRU for inserts.
The plots below compare the considered solutions. The Y axis shows the
boot time (in seconds) of a debian jessie image with arm-softmmu; the X axis
sweeps the number of buckets (or initial number of buckets for qht-autoresize).
The plots in PNG format (and with errorbars) can be seen here:
http://imgur.com/a/Awgnq
Each test runs 5 times, and the entire QEMU process is pinned to a
single core for repeatability of results.
Note that the original point before this patch series is X=15 for "master";
the little sensitivity to the increased number of buckets is due to the
poor hashing function in master.
xxhash-rcu has significant overhead due to the constant churn of allocating
and deallocating intermediate structs for implementing MRU. An alternative
would be do consider failed lookups as "maybe not there", and then
acquire the external lock (tb_lock in this case) to really confirm that
there was indeed a failed lookup. This, however, would not be enough
to implement dynamic resizing--this is more complex: see
"Resizable, Scalable, Concurrent Hash Tables via Relativistic
Programming" by Triplett, McKenney and Walpole. This solution was
discarded due to the very coarse RCU read critical sections that we have
in MTTCG; resizing requires waiting for readers after every pointer update,
and resizes require many pointer updates, so this would quickly become
prohibitive.
qht-fixed-nomru shows that MRU promotion is advisable for undersized
hash tables.
However, qht-dyn-mru shows that MRU promotion is not important if the
hash table is properly sized: there is virtually no difference in
performance between qht-dyn-nomru and qht-dyn-mru.
Before this patch, we're at X=15 on "xxhash"; after this patch, we're at
X=15 @ qht-dyn-nomru. This patch thus matches the best performance that we
can achieve with optimum sizing of the hash table, while keeping the hash
table scalable for readers.
The improvement we get before and after this patch for booting debian jessie
with arm-softmmu is:
- Intel Xeon E5-2690: 10.5% less time
- Intel i7-4790K: 5.2% less time
We could get this same improvement _for this particular workload_ by
statically increasing the size of the hash table. But this would hurt
workloads that do not need a large hash table. The dynamic (upward)
resizing allows us to start small and enlarge the hash table as needed.
A quick note on downsizing: the table is resized back to 2**15 buckets
on every tb_flush; this makes sense because it is not guaranteed that the
table will reach the same number of TBs later on (e.g. most bootup code is
thrown away after boot); it makes sense to grow the hash table as
more code blocks are translated. This also avoids the complication of
having to build downsizing hysteresis logic into qht.
Reviewed-by: Sergey Fedorov <serge.fedorov@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <rth@twiddle.net> Signed-off-by: Emilio G. Cota <cota@braap.org>
Message-Id: <1465412133-3029-15-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson <rth@twiddle.net>
Emilio G. Cota [Wed, 8 Jun 2016 18:55:30 +0000 (14:55 -0400)]
qht: add qht-bench, a performance benchmark
This serves as a performance benchmark as well as a stress test
for QHT. We can tweak quite a number of things, including the
number of resize threads and how frequently resizes are triggered.
A performance comparison of QHT vs CLHT[1] and ck_hs[2] using
this same benchmark program can be found here:
http://imgur.com/a/0Bms4
The tests are run on a 64-core AMD Opteron 6376, pinning threads
to cores favoring same-socket cores. For each run, qht-bench is
invoked with:
$ tests/qht-bench -d $duration -n $n -u $u -g $range
, where $duration is in seconds, $n is the number of threads,
$u is the update rate (0.0 to 100.0), and $range is the number
of keys.
Note that ck_hs's performance drops significantly as writes go
up, since it requires an external lock (I used a ck_spinlock)
around every write.
Also, note that CLHT instead of using a seqlock, relies on an
allocator that does not ever return the same address during the
same read-critical section. This gives it a slight performance
advantage over QHT on read-heavy workloads, since the seqlock
writes aren't there.
Emilio G. Cota [Wed, 8 Jun 2016 18:55:28 +0000 (14:55 -0400)]
qht: QEMU's fast, resizable and scalable Hash Table
This is a fast, scalable chained hash table with optional auto-resizing, allowing
reads that are concurrent with reads, and reads/writes that are concurrent
with writes to separate buckets.
A hash table with these features will be necessary for the scalability
of the ongoing MTTCG work; before those changes arrive we can already
benefit from the single-threaded speedup that qht also provides.
Signed-off-by: Emilio G. Cota <cota@braap.org>
Message-Id: <1465412133-3029-11-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson <rth@twiddle.net>
Emilio G. Cota [Wed, 8 Jun 2016 18:55:26 +0000 (14:55 -0400)]
qdist: add module to represent frequency distributions of data
Sometimes it is useful to have a quick histogram to represent a certain
distribution -- for example, when investigating a performance regression
in a hash table due to inadequate hashing.
The appended allows us to easily represent a distribution using Unicode
characters. Further, the data structure keeping track of the distribution
is so simple that obtaining its values for off-line processing is trivial.
Reviewed-by: Richard Henderson <rth@twiddle.net> Signed-off-by: Emilio G. Cota <cota@braap.org>
Message-Id: <1465412133-3029-9-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson <rth@twiddle.net>
Emilio G. Cota [Wed, 8 Jun 2016 18:55:25 +0000 (14:55 -0400)]
tb hash: hash phys_pc, pc, and flags with xxhash
For some workloads such as arm bootup, tb_phys_hash is performance-critical.
The is due to the high frequency of accesses to the hash table, originated
by (frequent) TLB flushes that wipe out the cpu-private tb_jmp_cache's.
More info:
https://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg05098.html
To dig further into this I modified an arm image booting debian jessie to
immediately shut down after boot. Analysis revealed that quite a bit of time
is unnecessarily spent in tb_phys_hash: the cause is poor hashing that
results in very uneven loading of chains in the hash table's buckets;
the longest observed chain had ~550 elements.
The appended addresses this with two changes:
1) Use xxhash as the hash table's hash function. xxhash is a fast,
high-quality hashing function.
2) Feed the hashing function with not just tb_phys, but also pc and flags.
This improves performance over using just tb_phys for hashing, since that
resulted in some hash buckets having many TB's, while others getting very few;
with these changes, the longest observed chain on a single hash bucket is
brought down from ~550 to ~40.
Tests show that the other element checked for in tb_find_physical,
cs_base, is always a match when tb_phys+pc+flags are a match,
so hashing cs_base is wasteful. It could be that this is an ARM-only
thing, though. UPDATE:
On Tue, Apr 05, 2016 at 08:41:43 -0700, Richard Henderson wrote:
> The cs_base field is only used by i386 (in 16-bit modes), and sparc (for a TB
> consisting of only a delay slot).
> It may well still turn out to be reasonable to ignore cs_base for hashing.
BTW, after this change the hash table should not be called "tb_hash_phys"
anymore; this is addressed later in this series.
This change gives consistent bootup time improvements. I tested two
host machines:
- Intel Xeon E5-2690: 11.6% less time
- Intel i7-4790K: 19.2% less time
Increasing the number of hash buckets yields further improvements. However,
using a larger, fixed number of buckets can degrade performance for other
workloads that do not translate as many blocks (600K+ for debian-jessie arm
bootup). This is dealt with later in this series.
Reviewed-by: Sergey Fedorov <sergey.fedorov@linaro.org> Reviewed-by: Richard Henderson <rth@twiddle.net> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Emilio G. Cota <cota@braap.org>
Message-Id: <1465412133-3029-8-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson <rth@twiddle.net>
Reviewed-by: Sergey Fedorov <sergey.fedorov@linaro.org> Signed-off-by: Guillaume Delbergue <guillaume.delbergue@greensocs.com>
[Rewritten. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
[Emilio's additions: use TAS instead of atomic_xchg; emit acquire/release
barriers; return bool from trylock; call cpu_relax() while spinning;
optimize for uncontended locks by acquiring the lock with TAS instead
of TATAS; add qemu_spin_locked().] Signed-off-by: Emilio G. Cota <cota@braap.org>
Message-Id: <1465412133-3029-6-git-send-email-cota@braap.org> Signed-off-by: Richard Henderson <rth@twiddle.net>
* remotes/kraxel/tags/pull-ui-20160610-1:
console: ignore ui_info updates which don't actually update something
ui/console-gl: Add support for big endian display surfaces
gtk: fix vte version check
ui: fix regression in printing VNC host/port on startup
vnc: drop unused depth arg for set_pixel_format
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reported-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1465382576-25552-1-git-send-email-berrange@redhat.com Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Peter Maydell [Tue, 17 May 2016 14:18:07 +0000 (15:18 +0100)]
target-i386: Move user-mode exception actions out of user-exec.c
The exception_action() function in user-exec.c is just a call to
cpu_loop_exit() for every target CPU except i386. Since this
function is only called if the target's handle_mmu_fault() hook has
indicated an MMU fault, and that hook is only called from the
handle_cpu_signal() code path, we can simply move the x86-specific
setup into that hook, which allows us to remove the TARGET_I386
ifdef from user-exec.c.
Of the actions that were done by the call to raise_interrupt_err():
* cpu_svm_check_intercept_param() is a no-op in user mode
* check_exception() is a no-op since double faults are impossible
for user-mode
* assignments to cs->exception_index and env->error_code are no-ops
* assigning to env->exception_next_eip is unnecessary because it
is not used unless env->exception_is_int is true
* cpu_loop_exit_restore() is equivalent to cpu_loop_exit() since
pc is 0
which leaves just setting env_>exception_is_int as the action that
needs to be added to x86_cpu_handle_mmu_fault().
Peter Maydell [Tue, 17 May 2016 14:18:06 +0000 (15:18 +0100)]
target-i386: Add comment about do_interrupt_user() next_eip argument
Add a comment to do_interrupt_user() along the same lines as the
existing one for do_interrupt_all() noting that the next_eip
argument is not used unless is_int is true or intno is EXCP_SYSCALL.
Peter Maydell [Tue, 17 May 2016 14:18:05 +0000 (15:18 +0100)]
user-exec: Don't reextract sigmask from usercontext pointer
Extracting the old signal mask from the usercontext pointer passed to
a signal handler is a pain because it is OS and CPU dependent.
Since we've already done it once and passed it to handle_cpu_signal(),
there's no need to do it again in cpu_exit_tb_from_sighandler().
This then means we don't need to pass a usercontext pointer in to
handle_cpu_signal() at all.
Peter Maydell [Tue, 17 May 2016 14:18:04 +0000 (15:18 +0100)]
cpu-exec: Rename cpu_resume_from_signal() to cpu_loop_exit_noexc()
The function cpu_resume_from_signal() is now always called with a
NULL puc argument, and is rather misnamed since it is never called
from a signal handler. It is essentially forcing an exit to the
top level cpu loop but without raising any exception, so rename
it to cpu_loop_exit_noexc() and drop the useless unused argument.
Peter Maydell [Tue, 17 May 2016 14:18:03 +0000 (15:18 +0100)]
user-exec: Push resume-from-signal code out to handle_cpu_signal()
Since the only caller of page_unprotect() which might cause it to
need to call cpu_resume_from_signal() is handle_cpu_signal() in
the user-mode code, push the longjump handling out to that function.
Since this is the only caller of cpu_resume_from_signal() which
passes a non-NULL puc argument, split the non-NULL handling into
a new cpu_exit_tb_from_sighandler() function. This allows us
to merge the softmmu and usermode implementations of the
cpu_resume_from_signal() function, which are now identical.
Peter Maydell [Tue, 17 May 2016 14:18:02 +0000 (15:18 +0100)]
translate-all.c: Don't pass puc, locked to tb_invalidate_phys_page()
The user-mode-only function tb_invalidate_phys_page() is only
called from two places:
* page_unprotect(), which passes in a non-zero pc, a puc pointer
and the value 'true' for the locked argument
* page_set_flags(), which passes in a zero pc, a NULL puc pointer
and a 'false' locked argument
If the pc is non-zero then we may call cpu_resume_from_signal(),
which does a longjmp out of the calling code (and out of the
signal handler); this is to cover the case of a target CPU with
"precise self-modifying code" (currently only x86) executing
a store instruction which modifies code in the same TB as the
store itself. Rather than doing the longjump directly here,
return a flag to the caller which indicates whether the current
TB was modified, and move the longjump to page_unprotect.
Peter Maydell [Wed, 8 Jun 2016 17:34:32 +0000 (18:34 +0100)]
Merge remote-tracking branch 'remotes/riku/tags/pull-linux-user-20160608' into staging
linux-user pull request for June 2016
# gpg: Signature made Wed 08 Jun 2016 14:27:14 BST
# gpg: using RSA key 0xB44890DEDE3C9BC0
# gpg: Good signature from "Riku Voipio <riku.voipio@iki.fi>"
# gpg: aka "Riku Voipio <riku.voipio@linaro.org>"
* remotes/riku/tags/pull-linux-user-20160608: (44 commits)
linux-user: In fork_end(), remove correct CPUs from CPU list
linux-user: Special-case ERESTARTSYS in target_strerror()
linux-user: Make target_strerror() return 'const char *'
linux-user: Correct signedness of target_flock l_start and l_len fields
linux-user: Use safe_syscall wrapper for ioctl
linux-user: Use safe_syscall wrapper for accept and accept4 syscalls
linux-user: Use safe_syscall wrapper for semop
linux-user: Use safe_syscall wrapper for epoll_wait syscalls
linux-user: Use safe_syscall wrapper for poll and ppoll syscalls
linux-user: Use safe_syscall wrapper for sleep syscalls
linux-user: Use safe_syscall wrapper for rt_sigtimedwait syscall
linux-user: Use safe_syscall wrapper for flock
linux-user: Use safe_syscall wrapper for mq_timedsend and mq_timedreceive
linux-user: Use safe_syscall wrapper for msgsnd and msgrcv
linux-user: Use safe_syscall wrapper for send* and recv* syscalls
linux-user: Use safe_syscall wrapper for connect syscall
linux-user: Use safe_syscall wrapper for readv and writev syscalls
linux-user: Fix error conversion in 64-bit fadvise syscall
linux-user: Fix NR_fadvise64 and NR_fadvise64_64 for 32-bit guests
linux-user: Fix handling of arm_fadvise64_64 syscall
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Conflicts:
configure
scripts/qemu-binfmt-conf.sh
Peter Maydell [Wed, 8 Jun 2016 15:31:53 +0000 (16:31 +0100)]
Merge remote-tracking branch 'remotes/famz/tags/pull-docker-20160608' into staging
Docker testing fixes by Paolo.
# gpg: Signature made Wed 08 Jun 2016 08:20:54 BST
# gpg: using RSA key 0xCA35624C6A9171C6
# gpg: Good signature from "Fam Zheng <famz@redhat.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6
* remotes/famz/tags/pull-docker-20160608:
tests/docker: build all targets in test-clang
tests/docker: support travis test with fedora image
tests/docker: remove unused feature "ccache"
tests/docker: fix test-mingw
tests/docker: make test-full build all targets, not none
tests/docker: fix make-archive-maybe
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
# gpg: Signature made Tue 07 Jun 2016 18:06:22 BST
# gpg: using RSA key 0xBFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>"
* remotes/bonzini/tags/for-upstream:
vnc: list the 'to' parameter of '-vnc' in the qemu man page
scsi-disk: add missing break
Makefile: Derive "PKGVERSION" from "git describe" by default
Makefile: add dependency on scripts/hxtool
Makefile: add dependency on scripts/make_device_config.sh
Makefile: add dependency on scripts/create_config
Makefile: Add a "FORCE" target
scsi: megasas: null terminate bios version buffer
scsi: mark TYPE_SCSI_DISK_BASE as abstract
scsi: esp: check TI buffer index before read/write
hw/char: QOM'ify escc.c (fix)
pc: allow raising low memory via max-ram-below-4g option
tests: Rename tests/Makefile to tests/Makefile.include
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Tue, 7 Jun 2016 16:31:04 +0000 (17:31 +0100)]
linux-user: In fork_end(), remove correct CPUs from CPU list
In fork_end(), we must fix the list of current CPUs to match the fact
that the child of the fork has only one thread. Unfortunately we were
removing the wrong CPUs from the list, which meant that if the child
subsequently did an exclusive operation it would deadlock in
start_exclusive() waiting for a sibling CPU which didn't exist.
In particular this could cause hangs doing git submodule init
operations, as reported in https://bugs.launchpad.net/qemu/+bug/955379
comment #47.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Peter Maydell [Mon, 6 Jun 2016 18:58:19 +0000 (19:58 +0100)]
linux-user: Special-case ERESTARTSYS in target_strerror()
Since TARGET_ERESTARTSYS and TARGET_ESIGRETURN are internal-to-QEMU
error numbers, handle them specially in target_strerror(), to avoid
confusing strace output like:
Peter Maydell [Mon, 6 Jun 2016 18:58:16 +0000 (19:58 +0100)]
linux-user: Correct signedness of target_flock l_start and l_len fields
The l_start and l_len fields in the various target_flock structures are
supposed to be '__kernel_off_t' or '__kernel_loff_t', which means they
should be signed, not unsigned. Correcting the structure definitions means
that __get_user() and __put_user() will correctly sign extend them if
the guest is using 32 bit offsets and the host is using 64 bit offsets.
This fixes failures in the LTP 'fcntl14' tests where it checks that
negative seek offsets work correctly.
We reindent the structures to drop hard tabs since we're touching 40%
of the fields anyway.
RV: long long -> abi_llong as suggested by Laurent Vivier <laurent@vivier.eu> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Kevin Wolf [Mon, 13 Jul 2015 11:13:17 +0000 (13:13 +0200)]
qemu-img bench: Implement -S (step size)
With this new option, qemu-img bench can be told to advance the current
offset after each request by a different value than the buffer size.
This is useful for controlling the conditions for cluster allocation in
image formats (e.g. qcow2 cluster allocation with COW in front of the
request, or COW areas that aren't overwritten immediately).
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Drivers that implement .bdrv_co_pwritev() get the flags passed as an
argument to said function, but we also unconditionally emulate the flags
anyway. We shouldn't do that.
Fix this by clearing all flags that the driver supports natively after
it returns from .bdrv_co_pwritev().
Fixes: 4df863f3 ('block: Make supported_write_flags a per-bds property') Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Colin Lord [Mon, 6 Jun 2016 18:15:22 +0000 (14:15 -0400)]
blockdev: clean up error handling in do_open_tray
Returns negative error codes and accompanying error messages in cases where
the device has no tray or the tray is locked and isn't forced open. This
extra information should result in better flexibility in functions that
call do_open_tray.
Suggested by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Colin Lord <clord@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The code to exit the loop after bdrv_snapshot_delete_by_id_or_name()
returned failure was duplicated. The first copy of it was too early so
that the AioContext lock would not be freed. This patch removes it so
that only the second, correct copy remains.
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Denis V. Lunev [Thu, 2 Jun 2016 15:58:15 +0000 (18:58 +0300)]
qcow2: avoid extra flushes in qcow2
The problem with excessive flushing was found by a couple of performance
tests:
- parallel directory tree creation (from 2 processes)
- 32 cached writes + fsync at the end in a loop
For the first one results improved from 2.6 loops/sec to 3.5 loops/sec.
Each loop creates 10^3 directories with 10 files in each.
For the second one results improved from ~600 fsync/sec to ~1100
fsync/sec. Though, it was run on SSD so it probably won't show such
performance gain on rotational media.
qcow2_cache_flush() calls bdrv_flush() unconditionally after writing
cache entries of a particular cache. This can lead to as many as
2 additional fdatasyncs inside bdrv_flush.
We can simply skip all fdatasync calls inside qcow2_co_flush_to_os
as bdrv_flush for sure will do the job. These flushes are necessary to
keep the right order of writes to the different caches. Though this is
not necessary in the current code base as this ordering is ensured through
the flush in qcow2_cache_flush_dependency().
Signed-off-by: Denis V. Lunev <den@openvz.org> CC: Pavel Borzenkov <pborzenkov@virtuozzo.com> CC: Kevin Wolf <kwolf@redhat.com> CC: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Kevin Wolf [Fri, 27 May 2016 17:50:37 +0000 (19:50 +0200)]
migration/block: Convert saving to BlockBackend
This creates a new BlockBackend for copying data from an images to the
migration stream on the source host. All I/O for block migration goes
through BlockBackend now.
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Kevin Wolf [Wed, 25 May 2016 15:20:06 +0000 (17:20 +0200)]
migration/block: Convert load to BlockBackend
This converts the loading part of block migration to use BlockBackend
interfaces rather than accessing the BlockDriverState directly.
Note that this takes a lazy shortcut. We should really use a separate
BlockBackend that is configured for the migration rather than for the
guest (e.g. writethrough caching is unnecessary) and holds its own
reference to the BlockDriverState, but the impact isn't that big and we
didn't have a separate migration reference before either, so it must be
good enough, I guess...
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Eric Blake [Wed, 1 Jun 2016 21:10:10 +0000 (15:10 -0600)]
raw-posix: Convert to bdrv_co_pwrite_zeroes()
Another step on our continuing quest to switch to byte-based
interfaces.
Signed-off-by: Eric Blake <eblake@redhat.com>
[ kwolf: Fixed up trace_paio_submit_co() call for qiov == NULL ] Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Wed, 1 Jun 2016 21:10:09 +0000 (15:10 -0600)]
qed: Convert to bdrv_co_pwrite_zeroes()
Another step on our continuing quest to switch to byte-based
interfaces.
Kill an abuse of the comma operator while at it (fortunately,
the semantics were still right). Also, the test for requests
not aligned to clusters should be applied always, not just
when a backing file is present.
Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Wed, 1 Jun 2016 21:10:05 +0000 (15:10 -0600)]
iscsi: Convert to bdrv_co_pwrite_zeroes()
Another step on our continuing quest to switch to byte-based
interfaces.
As this is the first byte-based iscsi interface, convert
is_request_lun_aligned() into two versions, one for sectors
and one for bytes. Also, change from outright -EINVAL failure
on an unaligned request, to instead failing with -ENOTSUP to
trigger a read-modify-write fallback, particularly since the
block layer should be honoring bs->request_alignment to avoid
-EINVAL on read/write requests.
Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Wed, 1 Jun 2016 21:10:03 +0000 (15:10 -0600)]
block: Add .bdrv_co_pwrite_zeroes()
Update bdrv_co_do_write_zeroes() to be byte-based, and select
between the new byte-based bdrv_co_pwrite_zeroes() or the old
bdrv_co_write_zeroes(). The next patches will convert drivers,
then remove the old interface.
Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Wed, 1 Jun 2016 21:10:02 +0000 (15:10 -0600)]
block: Track write zero limits in bytes
Another step towards removing sector-based interfaces: convert
the maximum write and minimum alignment values from sectors to
bytes. Rename the variables to let the compiler check that all
users are converted to the new semantics.
The maximum remains an int as long as BDRV_REQUEST_MAX_SECTORS
is constrained by INT_MAX (this means that we can't even
support a 2G write_zeroes, but just under it) - changing
operation lengths to unsigned or to 64-bits is a much bigger
audit, and debatable if we even want to do it (since at the
core, a 32-bit platform will still have ssize_t as its
underlying limit on write()).
Meanwhile, alignment is changed to 'uint32_t', since it makes no
sense to have an alignment larger than the maximum write, and
less painful to use an unsigned type with well-defined behavior
in bit operations than to have to worry about what happens if
a driver mistakenly supplies a negative alignment.
Add an assert that no one was trying to use sectors to get a
write zeroes larger than 2G, and therefore that a later conversion
to bytes won't be impacted by keeping the limit at 32 bits.
Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Wed, 1 Jun 2016 21:10:01 +0000 (15:10 -0600)]
iscsi: Use block size as minimum zero/discard alignment
If hardware does not advertise a minimum zero/discard
alignment, we still want to guarantee that the block layer
will align requests to our blocks, rather than the arbitrary
512-byte BDRV sector size.
Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Thu, 26 May 2016 03:48:49 +0000 (21:48 -0600)]
qcow2: Catch more unaligned write_zero into zero cluster
is_zero_cluster() and is_zero_cluster_top_locked() are used only
by qcow2_co_write_zeroes(). The former is too broad (we don't
care if the sectors we are about to overwrite are non-zero, only
that all other sectors in the cluster are zero), so it needs to
be called up to twice but with smaller limits - rename it along
with adding the neeeded parameter. The latter can be inlined for
more compact code.
The testsuite change shows that we now have a sparser top file
when an unaligned write_zeroes overwrites the only portion of
the backing file with data.
Based on a patch proposal by Denis V. Lunev.
CC: Denis V. Lunev <den@openvz.org> Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Eric Blake [Thu, 26 May 2016 03:48:48 +0000 (21:48 -0600)]
qemu-iotests: Test one more spot for optimizing write_zeroes
Add another test to 154, showing that we currently allocate a
data cluster in the top layer if any sector of the backing file
was allocated. The next patch will optimize this case.
Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Denis V. Lunev [Thu, 26 May 2016 03:48:47 +0000 (21:48 -0600)]
qcow2: add tracepoints for qcow2_co_write_zeroes
This patch follows guidelines of all other tracepoints in qcow2, like ones
in qcow2_co_writev. I think that they should dump values in the same
quantities or be changed all together.
Signed-off-by: Denis V. Lunev <den@openvz.org> CC: Eric Blake <eblake@redhat.com> CC: Kevin Wolf <kwolf@redhat.com>
Message-Id: <1463476543-3087-4-git-send-email-den@openvz.org>
[eblake: typo fix in commit message] Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Denis V. Lunev [Thu, 26 May 2016 03:48:46 +0000 (21:48 -0600)]
qcow2: simplify logic in qcow2_co_write_zeroes
Unaligned requests will occupy only one cluster. This is true since the
previous commit. Simplify the code taking this consideration into
account.
In other words, the caller is now buggy if it ever passes us an unaligned
request that crosses cluster boundaries (the only requests that can cross
boundaries will be aligned).
There are no other changes so far.
Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Eric Blake <eblake@redhat.com> CC: Eric Blake <eblake@redhat.com> CC: Kevin Wolf <kwolf@redhat.com>
Message-Id: <1463476543-3087-3-git-send-email-den@openvz.org> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Denis V. Lunev [Thu, 26 May 2016 03:48:45 +0000 (21:48 -0600)]
block: split write_zeroes always
We should split requests even if they are less than write_zeroes_alignment.
For example we can have the following request:
offset 62k
size 4k
write_zeroes_alignment 64k
The original code sent 1 request covering 2 qcow2 clusters, and resulted
in both clusters being allocated. But by splitting the request, we can
cater to the case where one of the two clusters can be zeroed as a
whole, for only 1 cluster allocated after the operation.
Signed-off-by: Denis V. Lunev <den@openvz.org> CC: Eric Blake <eblake@redhat.com> CC: Kevin Wolf <kwolf@redhat.com>
Message-Id: <1463476543-3087-2-git-send-email-den@openvz.org>
[eblake: Avoid exceeding nb_sectors, hoist alignment checks out of
loop, and update testsuite to show that patch works]
Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Paolo Bonzini [Mon, 6 Jun 2016 14:46:57 +0000 (16:46 +0200)]
tests/docker: build all targets in test-clang
Warnings specific to clang may affect devices that are not build by
x86_64-softmmu and aarch64-softmmu. Build all targets since that
is also what Peter does.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1465224417-141321-7-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
Paolo Bonzini [Mon, 6 Jun 2016 14:46:52 +0000 (16:46 +0200)]
tests/docker: fix make-archive-maybe
make-archive-maybe expects an archive path relative
to $1, but receives a path relative to the current directory. Redirect
the output outside the subshell to bypass the "cd $1".
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1465224417-141321-2-git-send-email-pbonzini@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
Peter Maydell [Mon, 6 Jun 2016 18:58:12 +0000 (19:58 +0100)]
linux-user: Use safe_syscall wrapper for semop
Use the safe_syscall wrapper for the semop syscall or IPC operation.
(We implement via the semtimedop syscall to make it easier to
implement the guest semtimedop syscall later.)
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Peter Maydell [Mon, 6 Jun 2016 18:58:11 +0000 (19:58 +0100)]
linux-user: Use safe_syscall wrapper for epoll_wait syscalls
Use the safe_syscall wrapper for epoll_wait and epoll_pwait syscalls.
Since we now directly use the host epoll_pwait syscall for both
epoll_wait and epoll_pwait, we don't need the configure machinery
to check whether glibc supports epoll_pwait(). (The kernel has
supported the syscall since 2.6.19 so we can assume it's always there.)
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Peter Maydell [Mon, 6 Jun 2016 18:58:10 +0000 (19:58 +0100)]
linux-user: Use safe_syscall wrapper for poll and ppoll syscalls
Use the safe_syscall wrapper for the poll and ppoll syscalls.
Since not all host architectures will have a poll syscall, we
have to rewrite the TARGET_NR_poll handling to use ppoll instead
(we can assume everywhere has ppoll by now).
We take the opportunity to switch to the code structure
already used in the implementation of epoll_wait and epoll_pwait,
which uses a switch() to avoid interleaving #if and if (),
and to stop using a variable with a leading '_' which is in
the implementation's namespace.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Peter Maydell [Mon, 6 Jun 2016 18:58:05 +0000 (19:58 +0100)]
linux-user: Use safe_syscall wrapper for msgsnd and msgrcv
Use the safe_syscall wrapper for msgsnd and msgrcv syscalls.
This is made slightly awkward by some host architectures providing
only a single 'ipc' syscall rather than separate syscalls per
operation; we provide safe_msgsnd() and safe_msgrcv() as wrappers
around safe_ipc() to handle this if needed.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Peter Maydell [Tue, 31 May 2016 14:45:10 +0000 (15:45 +0100)]
linux-user: Fix NR_fadvise64 and NR_fadvise64_64 for 32-bit guests
Fix errors in the implementation of NR_fadvise64 and NR_fadvise64_64
for 32-bit guests, which pass their off_t values in register pairs.
We can't use the 64-bit code path for this, so split out the 32-bit
cases, so that we can correctly handle the "only offset is 64-bit"
and "both offset and length are 64-bit" syscall flavours, and
"uses aligned register pairs" and "does not" flavours of target.
Peter Maydell [Tue, 31 May 2016 14:45:09 +0000 (15:45 +0100)]
linux-user: Fix handling of arm_fadvise64_64 syscall
32-bit ARM has an odd variant of the fadvise syscall which has
rearranged arguments, which we try to implement. Unfortunately we got
the rearrangement wrong.
This is a six-argument syscall whose arguments are:
* fd
* advise parameter
* offset high half
* offset low half
* len high half
* len low half
Stop trying to share code with the standard fadvise syscalls,
and just implement the syscall with the correct argument order.
Peter Maydell [Mon, 6 Jun 2016 18:56:54 +0000 (19:56 +0100)]
linux-user: provide frame information in x86-64 safe_syscall
Use cfi directives in the x86-64 safe_syscall to allow gdb to get
backtraces right from within it. (In particular this will be
quite a common situation if the user interrupts QEMU while it's
in a blocked safe-syscall: at the point of the syscall insn RBP
is in use for something else, and so gdb can't find the frame then
without assistance.)
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <rth@twiddle.net> Reviewed-by: Richard Henderson <rth@twiddle.net> Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Peter Maydell [Tue, 7 Jun 2016 15:34:45 +0000 (16:34 +0100)]
Merge remote-tracking branch 'remotes/mjt/tags/pull-trivial-patches-2016-06-07' into staging
trivial patches for 2016-06-07
# gpg: Signature made Tue 07 Jun 2016 16:20:52 BST
# gpg: using RSA key 0xBEE59D74A4C3D7DB
# gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>"
# gpg: aka "Michael Tokarev <mjt@corpit.ru>"
# gpg: aka "Michael Tokarev <mjt@debian.org>"
* remotes/mjt/tags/pull-trivial-patches-2016-06-07: (51 commits)
hbitmap: Use DIV_ROUND_UP
qemu-timer: Use DIV_ROUND_UP
linux-user: Use DIV_ROUND_UP
slirp: Use DIV_ROUND_UP
usb: Use DIV_ROUND_UP
rocker: Use DIV_ROUND_UP
SPICE: Use DIV_ROUND_UP
audio: Use DIV_ROUND_UP
xen: Use DIV_ROUND_UP
crypto: Use DIV_ROUND_UP
block: Use DIV_ROUND_UP
qed: Use DIV_ROUND_UP
qcow/qcow2: Use DIV_ROUND_UP
parallels: Use DIV_ROUND_UP
coccinelle: use macro DIV_ROUND_UP instead of (((n) + (d) - 1) /(d))
thunk: Rename args and fields in host-target bitmask conversion code
thunk: Drop unused NO_THUNK_TYPE_SIZE guards
qemu-common.h: Drop WORDS_ALIGNED define
host-utils: Prefer 'false' for bool type
docs/multi-thread-compression: Fix wrong command string
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
projects / mirror_qemu.git / log