Peter Maydell [Tue, 29 Oct 2019 18:46:52 +0000 (18:46 +0000)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
# gpg: Signature made Tue 29 Oct 2019 02:33:36 GMT
# gpg: using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* remotes/vivier/tags/q800-branch-pull-request:
BootLinuxConsoleTest: Test the Quadra 800
hw/m68k: define Macintosh Quadra 800
hw/m68k: add a dummy SWIM floppy controller
hw/m68k: add Nubus macfb video card
hw/m68k: add Nubus support
hw/m68k: implement ADB bus support for via
hw/m68k: add VIA support
dp8393x: manage big endian bus
esp: add pseudo-DMA as used by Macintosh
esp: move get_cmd() post-DMA code to get_cmd_cb()
esp: move handle_ti_cmd() cleanup code to esp_do_dma().
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Tue, 29 Oct 2019 15:33:15 +0000 (15:33 +0000)]
Merge remote-tracking branch 'remotes/berrange/tags/crypto-luks-pull-request' into staging
crypto: improve performance of ciphers in XTS mode
Currently QEMU uses its own XTS cipher mode, however, this has
relatively poor performance.
Gcrypt now includes its own XTS cipher which is at least x2 faster than
what we get with QEMU's on Fedora/RHEL hosts. With gcrypt git master, a
further x5-6 speed up is seen.
This is essential for QEMU's LUKS performance to be viable.
# gpg: Signature made Mon 28 Oct 2019 15:48:38 GMT
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF
* remotes/berrange/tags/crypto-luks-pull-request:
crypto: add support for nettle's native XTS impl
crypto: add support for gcrypt's native XTS impl
tests: benchmark crypto with fixed data size, not time period
tests: allow filtering crypto cipher benchmark tests
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Tue, 29 Oct 2019 08:38:04 +0000 (08:38 +0000)]
Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20191028' into staging
Improvements for TARGET_PAGE_BITS_VARY
Fix for TCI ld16u_i64.
Fix for segv on icount execute from i/o memory.
Two misc cleanups.
# gpg: Signature made Mon 28 Oct 2019 14:55:08 GMT
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F
* remotes/rth/tags/pull-tcg-20191028:
translate-all: Remove tb_alloc
translate-all: fix uninitialized tb->orig_tb
cputlb: Fix tlb_vaddr_to_host
exec: Cache TARGET_PAGE_MASK for TARGET_PAGE_BITS_VARY
exec: Promote TARGET_PAGE_MASK to target_long
exec: Restrict TARGET_PAGE_BITS_VARY assert to CONFIG_DEBUG_TCG
exec: Use const alias for TARGET_PAGE_BITS_VARY
configure: Detect compiler support for __attribute__((alias))
exec: Split out variable page size support to exec-vary.c
cpu: use ROUND_UP() to define xxx_PAGE_ALIGN
cputlb: ensure _cmmu helper functions follow the naming standard
tci: Add implementation for INDEX_op_ld16u_i64
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
qapi: Check feature documentation against the schema
Commit f3ed93d545 "qapi: Allow documentation for features" neglected
to check documentation against the schema. Fix that: check them the
same way we check arguments.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-20-armbru@redhat.com>
Commit 6a8c0b5102 "qapi: Add feature flags to struct types" added
features to QAPISchemaObjectType. Commit a95daa5093 "qapi: Add
feature flags to commands in qapi" added them to QAPISchemaCommand,
duplicating the code. Tolerable, but the duplication will only get
worse as we add features to more definitions.
To de-duplicate, lift features from QAPISchemaObjectType and
QAPISchemaCommand into QAPISchemaEntity.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-18-armbru@redhat.com>
qapi: Fold normalize_features() into check_features()
check_features() is always called together with normalize_features():
the former in check_struct() and check_command(), the latter in their
caller check_exprs(). Fold the latter into the former.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-16-armbru@redhat.com>
All sub-classes of QAPISchemaEntity now override .check_doc() the same
way, except for QAPISchemaType and and QAPISchemaArrayType.
Put the overrides' code in QAPISchemaEntity.check_doc(), and drop the
overrides. QAPISchemaType doesn't care because it's abstract.
QAPISchemaArrayType doesn't care because its .doc is always None.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-14-armbru@redhat.com>
qapi: Fix doc comment checking for commands and events
When a command's 'data' is an object, its doc comment describes the
arguments defined there. When 'data' names a type, the doc comment
does not describe arguments. Instead, the doc generator inserts a
pointer to the named type.
An event's doc comment works the same.
We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked. Works because the
check only cares for the implicit argument type's members.
Not only is this needlessly hard to understand, it actually falls
apart in two cases:
* When 'data' is empty, there is nothing to forward to, and the doc
comment remains unchecked. Demonstrated by test doc-bad-event-arg.
* When 'data' names a type, we can't forward, as the type has its own
doc comment. The command or event's doc comment remains unchecked.
Demonstrated by test doc-bad-boxed-command-arg.
The forwarding goes back to commit 069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit 816a57cd6e "qapi: Fix detection of bogus member documentation". That
fix was incomplete.
To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-12-armbru@redhat.com>
qapi: Clean up doc comment checking for implicit union base
An object type's doc comment describes the type's members, less the
ones defined in a named base type. Cases:
* Struct: the members are defined in 'data' and inherited from 'base'.
Since the base type cannot be implicit, the doc comment describes
just 'data'.
* Simple union: the only member is the implicit tag member @type, and
the doc comment describes it.
* Flat union with implicit base type: the members are defined in
'base', and the doc comment describes it.
* Flat union with named base type: the members are inherited from
'base'. The doc comment describes no members.
Before we can check a doc comment with .check_doc(), we need
.connect_doc() connect each of its "argument sections" to the member
it documents.
For structs and simple unions, this is straightforward: the members in
question are in .local_members, and .connect_doc() connects them.
For flat unions with a named base type, it's trivial: .local_members
is empty, and .connect_doc() does nothing.
For flat unions with an implicit base type, it's tricky. We have
QAPISchema._make_implicit_object_type() forward the union's doc
comment to the implicit base type, so that the base type's
.connect_doc() connects the members. The union's .connect_doc() does
nothing, as .local_members is empty.
Dirt effect: we check the doc comment twice, once for the union type,
and once for the implicit base type.
This is needlessly brittle and hard to understand. Clean up as
follows. Make the union's .connect_doc() connect an implicit base's
members itself. Do not forward the union's doc comment to its
implicit base type.
Requires extending .connect_doc() so it can work with a doc comment
other than self.doc. Add an optional argument for that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-11-armbru@redhat.com>
Enumeration type documentation comments are not checked, as
demonstrated by test doc-bad-enum-member. This is because we neglect
to call self.doc.check() for enumeration types. Messed up in 816a57cd6e "qapi: Fix detection of bogus member documentation". Fix
it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-10-armbru@redhat.com>
qapi: Split .connect_doc(), .check_doc() off .check()
Splitting documentation checking off the .check() methods makes them a
bit more focused, which is welcome, as some of them are pretty big.
It also prepares the ground for the following commits.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-9-armbru@redhat.com>
QAPISchemaGenDocVisitor.visit_command() duplicates texi_entity() for
its boxed arguments case. The previous commit added another copy in
.visit_event().
Replace texi_entity() by texi_type() and texi_msg(). Use texi_msg()
for the boxed arguments case as well.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-8-armbru@redhat.com>
Generate a reference "Arguments: the members of ...", just like we do
for commands since commit c2dd311cb7 "qapi2texi: Implement boxed
argument documentation".
No change to generated QMP documentation; we don't yet use boxed
events outside tests/.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-7-armbru@redhat.com>
Commit 2ff3025797 "migrate: move max-bandwidth and downtime-limit to
migrate_set_parameter" (v2.8.0) deprecated migrate_set_downtime and
migrate_set_speed.
These wre missed in commit eb22aeca65 "docs: document deprecation
policy & deprecated features in appendix" (v2.10.0).
Commit 73af8dd8d7 "migration: Make xbzrle_cache_size a migration
parameter" (v2.11.0) deprecated migrate-set-cache-size and
query-migrate-cache-size, but neglected to update qemu-doc.
Make up for that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-6-armbru@redhat.com>
Commit 8aa3a33e44 "tests/qapi-schema: Test for good feature lists in
structs" made test-qapi.py show features, but neglected to show their
documentation. Fix that.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-5-armbru@redhat.com>
tests/qapi-schema: Demonstrate command and event doc comment bugs
Add negative tests doc-bad-boxed-command-arg and doc-bad-event-arg to
cover boxed and no arguments. They demonstrate insufficient doc
comment checking.
Update positive test doc-good to cover boxed event arguments. It
demonstrates the generated doc comment misses arguments.
These bugs will be fixed later in this series.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-3-armbru@redhat.com>
tests/qapi-schema: Demonstrate feature and enum doc comment bugs
Add negative tests doc-bad-enum-member and doc-bad-feature to cover
documentation for nonexistent enum members and features, and test
doc-undoc-feature to cover features lacking documentation. None of
them works. To be fixed later in this series.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-2-armbru@redhat.com>
Fan Yang [Tue, 24 Sep 2019 14:08:29 +0000 (22:08 +0800)]
COLO-compare: Fix incorrect `if` logic
'colo_mark_tcp_pkt' should return 'true' when packets are the same, and
'false' otherwise. However, it returns 'true' when
'colo_compare_packet_payload' returns non-zero while
'colo_compare_packet_payload' is just a 'memcmp'. The result is that
COLO-compare reports inconsistent TCP packets when they are actually
the same.
Currently offloads disabled by guest via the VIRTIO_NET_CTRL_GUEST_OFFLOADS_SET
command are not preserved on VM migration.
Instead all offloads reported by guest features (via VIRTIO_PCI_GUEST_FEATURES)
get enabled.
What happens is: first the VirtIONet::curr_guest_offloads gets restored and offloads
are getting set correctly:
#0 qemu_set_offload (nc=0x555556a11400, csum=1, tso4=0, tso6=0, ecn=0, ufo=0) at net/net.c:474
#1 virtio_net_apply_guest_offloads (n=0x555557701ca0) at hw/net/virtio-net.c:720
#2 virtio_net_post_load_device (opaque=0x555557701ca0, version_id=11) at hw/net/virtio-net.c:2334
#3 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577c80 <vmstate_virtio_net_device>, opaque=0x555557701ca0, version_id=11)
at migration/vmstate.c:168
#4 virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at hw/virtio/virtio.c:2197
#5 virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036
#6 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:143
#7 vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at migration/savevm.c:829
#8 qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2211
#9 qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2395
#10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467
#11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449
However later on the features are getting restored, and offloads get reset to
everything supported by features:
#0 qemu_set_offload (nc=0x555556a11400, csum=1, tso4=1, tso6=1, ecn=0, ufo=0) at net/net.c:474
#1 virtio_net_apply_guest_offloads (n=0x555557701ca0) at hw/net/virtio-net.c:720
#2 virtio_net_set_features (vdev=0x555557701ca0, features=5104441767) at hw/net/virtio-net.c:773
#3 virtio_set_features_nocheck (vdev=0x555557701ca0, val=5104441767) at hw/virtio/virtio.c:2052
#4 virtio_load (vdev=0x555557701ca0, f=0x5555569dc010, version_id=11) at hw/virtio/virtio.c:2220
#5 virtio_device_get (f=0x5555569dc010, opaque=0x555557701ca0, size=0, field=0x55555668cd00 <__compound_literal.5>) at hw/virtio/virtio.c:2036
#6 vmstate_load_state (f=0x5555569dc010, vmsd=0x555556577ce0 <vmstate_virtio_net>, opaque=0x555557701ca0, version_id=11) at migration/vmstate.c:143
#7 vmstate_load (f=0x5555569dc010, se=0x5555578189e0) at migration/savevm.c:829
#8 qemu_loadvm_section_start_full (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2211
#9 qemu_loadvm_state_main (f=0x5555569dc010, mis=0x5555569eee20) at migration/savevm.c:2395
#10 qemu_loadvm_state (f=0x5555569dc010) at migration/savevm.c:2467
#11 process_incoming_migration_co (opaque=0x0) at migration/migration.c:449
Fix this by preserving the state in saved_guest_offloads field and
pushing out offload initialization to the new post load hook.
Cc: qemu-stable@nongnu.org Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
Post load hook in virtio vmsd is called early while device is processed,
and when VirtIODevice core isn't fully initialized. Most device
specific code isn't ready to deal with a device in such state, and
behaves weirdly.
Add a new post_load hook in a device class instead. Devices should use
this unless they specifically want to verify the migration stream as
it's processed, e.g. for bounds checking.
Cc: qemu-stable@nongnu.org Suggested-by: "Dr. David Alan Gilbert" <dgilbert@redhat.com> Cc: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
This adds the basic functionality to emulate a Tulip NIC.
Implemented are:
- RX and TX functionality
- Perfect Frame Filtering
- Big/Little Endian descriptor support
- 93C46 EEPROM support
- LXT970 PHY
Not implemented, mostly because i had no OS using these functions:
- Imperfect frame filtering
- General Purpose Timer
- Transmit automatic polling
- Boot ROM support
- SIA interface
- Big/Little Endian data buffer conversion
Successfully tested with the following Operating Systems:
- MSDOS with Microsoft Network Client 3.0 and DEC ODI drivers
- HPPA Linux
- Windows XP
- HP-UX
Signed-off-by: Sven Schnelle <svens@stackframe.org>
Message-Id: <20191022155413.4619-1-svens@stackframe.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Jason Wang <jasowang@redhat.com>
Peter Maydell [Mon, 28 Oct 2019 21:43:06 +0000 (21:43 +0000)]
Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-4.2-sf2' into staging
RISC-V Patches for the 4.2 Soft Freeze, Part 2
This patch set contains a handful of small fixes for RISC-V targets that
I'd like to target for the 4.2 soft freeze. They include:
* A fix to allow the debugger to access the state of all privilege
modes, as opposed to just the currently executing one.
* A pair of cleanups to implement cpu_do_transaction_failed.
* Fixes to the device tree.
* The addition of various memory regions to make the sifive_u machine
more closely match the HiFive Unleashed board.
* Fixes to our GDB interface to allow CSRs to be accessed.
* A fix to a memory leak pointed out by coverity.
* A fix that prevents PMP checks from firing incorrectly.
This passes "make chcek" and boots Open Embedded for me.
# gpg: Signature made Mon 28 Oct 2019 15:47:52 GMT
# gpg: using RSA key 00CE76D1834960DFCE886DF8EF4CA1502CCBAB41
# gpg: issuer "palmer@dabbelt.com"
# gpg: Good signature from "Palmer Dabbelt <palmer@dabbelt.com>" [unknown]
# gpg: aka "Palmer Dabbelt <palmer@sifive.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 00CE 76D1 8349 60DF CE88 6DF8 EF4C A150 2CCB AB41
* remotes/palmer/tags/riscv-for-master-4.2-sf2:
target/riscv: PMP violation due to wrong size parameter
riscv/boot: Fix possible memory leak
target/riscv: Make the priv register writable by GDB
target/riscv: Expose "priv" register for GDB for reads
target/riscv: Tell gdbstub the correct number of CSRs
riscv/virt: Jump to pflash if specified
riscv/virt: Add the PFlash CFI01 device
riscv/virt: Manually define the machine
riscv/sifive_u: Add the start-in-flash property
riscv/sifive_u: Manually define the machine
riscv/sifive_u: Add QSPI memory region
riscv/sifive_u: Add L2-LIM cache memory
linux-user/riscv: Propagate fault address
riscv: sifive_u: Add ethernet0 to the aliases node
riscv: hw: Drop "clock-frequency" property of cpu nodes
RISC-V: Implement cpu_do_transaction_failed
RISC-V: Handle bus errors in the page table walker
riscv: Skip checking CSR privilege level in debugger mode
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
This test boots a Linux kernel on a Quadra 800 board
and verify the serial is working.
Example:
$ avocado --show=app,console run -t machine:q800 tests/acceptance/boot_linux_console.py
console: ABCFGHIJK
console: Linux version 5.2.0-2-m68k (debian-kernel@lists.debian.org) (gcc version 8.3.0 (Debian 8.3.0-21)) #1 Debian 5.2.9-2 (2019-08-21)
console: Detected Macintosh model: 35
console: Apple Macintosh Quadra 800
console: Built 1 zonelists, mobility grouping on. Total pages: 32448
console: Kernel command line: printk.time=0 console=ttyS0 vga=off
[...]
console: Calibrating delay loop... 1236.99 BogoMIPS (lpj=6184960)
[...]
console: NuBus: Scanning NuBus slots.
console: Slot 9: Board resource not found!
console: SCSI subsystem initialized
console: clocksource: Switched to clocksource via1
[...]
console: macfb: framebuffer at 0xf9001000, mapped to 0x(ptrval), size 468k
console: macfb: mode is 800x600x8, linelength=800
console: Console: switching to colour frame buffer device 100x37
console: fb0: DAFB frame buffer device
console: pmac_zilog: 0.6 (Benjamin Herrenschmidt <benh@kernel.crashing.org>)
console: scc.0: ttyS0 at MMIO 0x50f0c022 (irq = 4, base_baud = 230400) is a Z85c30 ESCC - Serial port
console: scc.1: ttyS1 at MMIO 0x50f0c020 (irq = 4, base_baud = 230400) is a Z85c30 ESCC - Serial port
console: Non-volatile memory driver v1.3
console: adb: Mac II ADB Driver v1.0 for Unified ADB
console: mousedev: PS/2 mouse device common for all mice
console: random: fast init done
console: Detected ADB keyboard, type <unknown>.
console: input: ADB keyboard as /devices/virtual/input/input0
console: input: ADB mouse as /devices/virtual/input/input1
console: rtc-generic rtc-generic: registered as rtc0
console: ledtrig-cpu: registered to indicate activity on CPUs
[...]
console: rtc-generic rtc-generic: setting system clock to 2019-09-10T16:20:25 UTC (1568132425)
console: List of all partitions:
console: No filesystem could mount root, tried:
JOB TIME : 2.91 s
Laurent Vivier [Sat, 26 Oct 2019 16:45:44 +0000 (18:45 +0200)]
hw/m68k: add a dummy SWIM floppy controller
SWIM (Sander-Wozniak Integrated Machine) is the floppy controller of
the 680x0 Macintosh.
This patch introduces only the basic support: it allows to switch from
IWM (Integrated WOZ Machine) mode to the SWIM mode and makes the linux
driver happy.
Laurent Vivier [Sat, 26 Oct 2019 16:45:40 +0000 (18:45 +0200)]
hw/m68k: add VIA support
Inside the 680x0 Macintosh, VIA (Versatile Interface Adapter) is used
to interface the keyboard, Mouse, and real-time clock. It also provides
control line for the floppy disk driver, video interface, sound circuitry
and serial interface.
This implementation is based on the MOS6522 object.
Laurent Vivier [Sat, 26 Oct 2019 16:45:38 +0000 (18:45 +0200)]
esp: add pseudo-DMA as used by Macintosh
There is no DMA in Quadra 800, so the CPU reads/writes the data from the
PDMA register (offset 0x100, ESP_PDMA in hw/m68k/q800.c) and copies them
to/from the memory.
There is a nice assembly loop in the kernel to do that, see
linux/drivers/scsi/mac_esp.c:MAC_ESP_PDMA_LOOP().
The start of the transfer is triggered by the DREQ interrupt (see linux
mac_esp_send_pdma_cmd()), the CPU polls on the IRQ flag to start the
transfer after a SCSI command has been sent (in Quadra 800 it goes
through the VIA2, the via2-irq line and the vIFR register)
The Macintosh hardware includes hardware handshaking to prevent the CPU
from reading invalid data or writing data faster than the peripheral
device can accept it.
This is the "blind mode", and from the doc:
"Approximate maximum SCSI transfer rates within a blocks are 1.4 MB per
second for blind transfers in the Macintosh II"
Some references can be found in:
Apple Macintosh Family Hardware Reference, ISBN 0-201-19255-1
Guide to the Macintosh Family Hardware, ISBN-0-201-52405-8
Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Co-developed-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> Signed-off-by: Laurent Vivier <laurent@vivier.eu> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20191026164546.30020-4-laurent@vivier.eu>
Laurent Vivier [Sat, 26 Oct 2019 16:45:36 +0000 (18:45 +0200)]
esp: move handle_ti_cmd() cleanup code to esp_do_dma().
To prepare following patches move do_cmd and DMA special case
from handle_ti() to esp_do_dma().
This part of the code must be only executed with real DMA, not with
pseudo-DMA. And PDMA is detected in esp_do_dma(), so move this part
of the code in esp_do_dma(). We keep the code in handle_ti_cmd()
in the case no DMA is done.
Signed-off-by: Laurent Vivier <laurent@vivier.eu> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20191026164546.30020-2-laurent@vivier.eu>
Dayeol Lee [Tue, 22 Oct 2019 21:21:29 +0000 (21:21 +0000)]
target/riscv: PMP violation due to wrong size parameter
riscv_cpu_tlb_fill() uses the `size` parameter to check PMP violation
using pmp_hart_has_privs().
However, if the size is unknown (=0), the ending address will be
`addr - 1` as it is `addr + size - 1` in `pmp_hart_has_privs()`.
This always causes a false PMP violation on the starting address of the
range, as `addr - 1` is not in the range.
In order to fix, we just assume that all bytes from addr to the end of
the page will be accessed if the size is unknown.
Signed-off-by: Dayeol Lee <dayeol@berkeley.edu> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Coverity (CID 1405786) thinks that there is a possible memory leak as
we don't guarantee that the memory allocated from riscv_find_firmware()
is freed. This is a false positive, but let's tidy up the code to fix
the warning.
Nettle 3.5.0 will add support for the XTS mode. Use this because long
term we wish to delete QEMU's XTS impl to avoid carrying private crypto
algorithm impls.
Unfortunately this degrades nettle performance from 612 MB/s to 568 MB/s
as nettle's XTS impl isn't so well optimized yet.
Libgcrypt 1.8.0 added support for the XTS mode. Use this because long
term we wish to delete QEMU's XTS impl to avoid carrying private crypto
algorithm impls.
As an added benefit, using this improves performance from 531 MB/sec to
670 MB/sec, since we are avoiding several layers of function call
indirection.
This is even more noticable with the gcrypt builds in Fedora or RHEL-8
which have a non-upstream patch for FIPS mode which does mutex locking.
This is catastrophic for encryption performance with small block sizes,
meaning this patch improves encryption from 240 MB/sec to 670 MB/sec.
Jonathan Behrens [Mon, 14 Oct 2019 15:45:29 +0000 (11:45 -0400)]
target/riscv: Make the priv register writable by GDB
Currently only PRV_U, PRV_S and PRV_M are supported, so this patch ensures that
the privilege mode is set to one of them. Once support for the H-extension is
added, this code will also need to properly update the virtualization status
when switching between VU/VS-modes and M-mode.
Signed-off-by: Jonathan Behrens <jonathan@fintelia.io> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Tested-by: Bin Meng <bmeng.cn@gmail.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Jonathan Behrens [Mon, 14 Oct 2019 15:45:28 +0000 (11:45 -0400)]
target/riscv: Expose "priv" register for GDB for reads
This patch enables a debugger to read the current privilege level via a virtual
"priv" register. When compiled with CONFIG_USER_ONLY the register is still
visible but always reports the value zero.
Signed-off-by: Jonathan Behrens <jonathan@fintelia.io> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Tested-by: Bin Meng <bmeng.cn@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Jonathan Behrens [Mon, 14 Oct 2019 15:45:27 +0000 (11:45 -0400)]
target/riscv: Tell gdbstub the correct number of CSRs
If the number of registers reported to the gdbstub code does not match the
number in the associated XML file, then the register numbers used by the stub
may get out of sync with a remote GDB instance.
Signed-off-by: Jonathan Behrens <jonathan@fintelia.io> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Add the CFI01 PFlash to the RISC-V virt board. This is the same PFlash
from the ARM Virt board and the implementation is based on the ARM Virt
board. This allows users to specify flash files from the command line.
Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Tested-by: Bin Meng <bmeng.cn@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
Instead of using the DEFINE_MACHINE() macro to define the machine let's
do it manually. This allows us to use the machine object to create
RISCVVirtState. This is required to add children and aliases to the
machine.
This patch is no functional change.
Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Tested-by: Bin Meng <bmeng.cn@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
The HiFive Unleashed uses is25wp256 SPI NOR flash. There is currently no
model of this in QEMU, so to allow boot firmware developers to use QEMU
to target the Unleashed let's add a chunk of memory to represent the QSPI0
memory mapped flash. This can be targeted using QEMU's -device loader
command line option.
In the future we can look at adding a model for the is25wp256 flash.
Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
On reset only a single L2 cache way is enabled, the others are exposed
as memory that can be used by early boot firmware. This L2 region is
generally disabled using the WayEnable register at a later stage in the
boot process. To allow firmware to target QEMU and the HiFive Unleashed
let's add the L2 LIM (LooselyIntegrated Memory).
Ideally we would want to adjust the size of this chunk of memory as the
L2 Cache Controller WayEnable register is incremented. Unfortunately I
don't see a nice way to handle reducing or blocking out the L2 LIM while
still allowing it be re returned to all enabled from a reset.
Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
The CPU loop tagged all the queued signals as QEMU_SI_KILL while it was
filling the `_sigfault` part of `siginfo`: this caused QEMU to copy the
wrong fields over to the userspace program.
Make sure the fault address recorded by the MMU is is stored in the CPU
environment structure.
In case of memory faults store the exception address into `siginfo`.
Palmer Dabbelt [Tue, 8 Oct 2019 20:51:50 +0000 (13:51 -0700)]
RISC-V: Handle bus errors in the page table walker
We directly access physical memory while walking the page tables on
RISC-V, but while doing so we were using cpu_ld*() which does not report
bus errors. This patch converts the page table walker over to use
address_space_ld*(), which allows bus errors to be detected.
Peter Maydell [Mon, 28 Oct 2019 14:40:00 +0000 (14:40 +0000)]
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' into staging
Block patches for softfreeze:
- iotest patches
- Improve performance of the mirror block job in write-blocking mode
- Limit memory usage for the backup block job
- Add discard and write-zeroes support to the NVMe host block driver
- Fix a bug in the mirror job
- Prevent the qcow2 driver from creating technically non-compliant qcow2
v3 images (where there is not enough extra data for snapshot table
entries)
- Allow callers of bdrv_truncate() (etc.) to determine whether the file
must be resized to the exact given size or whether it is OK for block
devices not to shrink
# gpg: Signature made Mon 28 Oct 2019 12:13:53 GMT
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2019-10-28: (69 commits)
qemu-iotests: restrict 264 to qcow2 only
Revert "qemu-img: Check post-truncation size"
block: Pass truncate exact=true where reasonable
block: Let format drivers pass @exact
block: Evaluate @exact in protocol drivers
block: Add @exact parameter to bdrv_co_truncate()
block: Do not truncate file node when formatting
block/cor: Drop cor_co_truncate()
block: Handle filter truncation like native impl.
iotests: Test qcow2's snapshot table handling
iotests: Add peek_file* functions
qcow2: Fix v3 snapshot table entry compliancy
qcow2: Repair snapshot table with too many entries
qcow2: Fix overly long snapshot tables
qcow2: Keep track of the snapshot table length
qcow2: Fix broken snapshot table entries
qcow2: Add qcow2_check_fix_snapshot_table()
qcow2: Separate qcow2_check_read_snapshot_table()
qcow2: Write v3-compliant snapshot list on upgrade
qcow2: Put qcow2_upgrade() into its own function
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Mon, 28 Oct 2019 13:32:40 +0000 (13:32 +0000)]
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging
virtio: features, tests
libqos update with support for virtio 1.
Packed ring support for virtio.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
# gpg: Signature made Fri 25 Oct 2019 12:47:59 BST
# gpg: using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg: aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469
* remotes/mst/tags/for_upstream: (25 commits)
virtio: drop unused virtio_device_stop_ioeventfd() function
libqos: add VIRTIO PCI 1.0 support
libqos: extract Legacy virtio-pci.c code
libqos: make the virtio-pci BAR index configurable
libqos: expose common virtqueue setup/cleanup functions
libqos: add MSI-X callbacks to QVirtioPCIDevice
libqos: pass full QVirtQueue to set_queue_address()
libqos: add iteration support to qpci_find_capability()
libqos: access VIRTIO 1.0 vring in little-endian
libqos: implement VIRTIO 1.0 FEATURES_OK step
libqos: enforce Device Initialization order
libqos: add missing virtio-9p feature negotiation
tests/virtio-blk-test: set up virtqueue after feature negotiation
virtio-scsi-test: add missing feature negotiation
libqos: extend feature bits to 64-bit
libqos: read QVIRTIO_MMIO_VERSION register
tests/virtio-blk-test: read config space after feature negotiation
virtio: add property to enable packed virtqueue
vhost_net: enable packed ring support
virtio: event suppression support for packed ring
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
tests: benchmark crypto with fixed data size, not time period
Currently the crypto benchmarks are processing data in varying chunk
sizes, over a fixed time period. This turns out to be a terrible idea
because with small chunk sizes the overhead of checking the elapsed
time on each loop iteration masks the true performance.
Benchmarking over a fixed data size avoids the loop running any system
calls which can interfere with the performance measurements.
Before this change
Enc chunk 512 bytes 2283.47 MB/sec Dec chunk 512 bytes 2236.23 MB/sec OK
Enc chunk 4096 bytes 2744.97 MB/sec Dec chunk 4096 bytes 2614.71 MB/sec OK
Enc chunk 16384 bytes 2777.53 MB/sec Dec chunk 16384 bytes 2678.44 MB/sec OK
Enc chunk 65536 bytes 2809.34 MB/sec Dec chunk 65536 bytes 2699.47 MB/sec OK
After this change
Enc chunk 512 bytes 2058.22 MB/sec Dec chunk 512 bytes 2030.11 MB/sec OK
Enc chunk 4096 bytes 2699.27 MB/sec Dec chunk 4096 bytes 2573.78 MB/sec OK
Enc chunk 16384 bytes 2748.52 MB/sec Dec chunk 16384 bytes 2653.76 MB/sec OK
Enc chunk 65536 bytes 2814.08 MB/sec Dec chunk 65536 bytes 2712.74 MB/sec OK
The actual crypto performance hasn't changed, which shows how
significant the mis-measurement has been for small data sizes.
Add support for specifying a cipher mode and chunk size as argv to
filter which combinations are benchmarked. For example to only
benchmark XTS mode with 512 byte chunks:
264 is unprepared to run with different formats, for example luks needs
handling keys, cloop doesn't support image creation, vpc creates image
larger than requested (which breaks "Backup completed: 5242880" in test
output).
The test is here to check nbd-reconnect feature and we actually don't
need it for all formats. Let's restrict it to qcow2 only.
Reported-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191025145023.6182-1-vsementsov@virtuozzo.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:43 +0000 (11:51 +0200)]
block: Pass truncate exact=true where reasonable
This is a change in behavior, so all instances need a good
justification. The comments added here should explain my reasoning.
qed already had a comment that suggests it always expected
bdrv_truncate()/blk_truncate() to behave as if exact=true were passed
(c743849bee7 came eight months before 55b949c8476), so it was simply
broken until now.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190918095144.955-8-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
[mreitz: Changed comment in qed.c to explain why a new QED file must be
empty, as requested and suggested by Maxim] Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:42 +0000 (11:51 +0200)]
block: Let format drivers pass @exact
When truncating a format node, the @exact parameter is generally handled
simply by virtue of the format storing the new size in the image
metadata. Such formats do not need to pass on the parameter to their
file nodes.
There are exceptions, though:
- raw and crypto cannot store the image size, and thus must pass on
@exact.
- When using qcow2 with an external data file, it just makes sense to
keep its size in sync with the qcow2 virtual disk (because the
external data file is the virtual disk). Therefore, we should pass
@exact when truncating it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190918095144.955-7-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:41 +0000 (11:51 +0200)]
block: Evaluate @exact in protocol drivers
We have two protocol drivers that return success when trying to shrink a
block device even though they cannot shrink it. This behavior is now
only allowed with exact=false, so they should return an error with
exact=true.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190918095144.955-6-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:40 +0000 (11:51 +0200)]
block: Add @exact parameter to bdrv_co_truncate()
We have two drivers (iscsi and file-posix) that (in some cases) return
success from their .bdrv_co_truncate() implementation if the block
device is larger than the requested offset, but cannot be shrunk. Some
callers do not want that behavior, so this patch adds a new parameter
that they can use to turn off that behavior.
This patch just adds the parameter and lets the block/io.c and
block/block-backend.c functions pass it around. All other callers
always pass false and none of the implementations evaluate it, so that
this patch does not change existing behavior. Future patches take care
of that.
Suggested-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190918095144.955-5-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:39 +0000 (11:51 +0200)]
block: Do not truncate file node when formatting
There is no reason why the format drivers need to truncate the protocol
node when formatting it. When using the old .bdrv_co_create_ops()
interface, the file will be created with no size option anyway, which
generally gives it a size of 0. (Exceptions are block devices, which
cannot be truncated anyway.)
When using blockdev-create, the user must have given the file node some
size anyway, so there is no reason why we should override that.
Max Reitz [Wed, 18 Sep 2019 09:51:38 +0000 (11:51 +0200)]
block/cor: Drop cor_co_truncate()
No other filter driver has a .bdrv_co_truncate() implementation, and
there is no need to because the general block layer code can handle it
just as well.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190918095144.955-3-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Wed, 18 Sep 2019 09:51:37 +0000 (11:51 +0200)]
block: Handle filter truncation like native impl.
Make the filter truncation (passing it through to bs->file) a
first-class citizen and handle it exactly as if it was the filter
driver's native implementation of .bdrv_co_truncate().
I do not see a reason not to, it makes the code a bit shorter, and may
be even more correct because this gets us to finish the write_req that
we prepared before (may be important to e.g. bring dirty bitmaps to the
correct size).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190918095144.955-2-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:14 +0000 (17:28 +0200)]
iotests: Test qcow2's snapshot table handling
Add a test how our qcow2 driver handles extra data in snapshot table
entries, and how it repairs overly long snapshot tables.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-17-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:13 +0000 (17:28 +0200)]
iotests: Add peek_file* functions
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-16-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:12 +0000 (17:28 +0200)]
qcow2: Fix v3 snapshot table entry compliancy
qcow2 v3 images require every snapshot table entry to have at least 16
bytes of extra data. If they do not, let qemu-img check -r all fix it.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-15-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:11 +0000 (17:28 +0200)]
qcow2: Repair snapshot table with too many entries
The user cannot choose which snapshots are removed. This is fine
because we have chosen the maximum snapshot table size to be so large
(65536 entries) that it cannot be reasonably reached. If the snapshot
table exceeds this size, the image has probably been corrupted in some
way; in this case, it is most important to just make the image usable
such that the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a
plain "check" or "check -r leaks" will not delete any data.)
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-14-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:10 +0000 (17:28 +0200)]
qcow2: Fix overly long snapshot tables
We currently refuse to open qcow2 images with overly long snapshot
tables. This patch makes qemu-img check -r all drop all offending
entries past what we deem acceptable.
The user cannot choose which snapshots are removed. This is fine
because we have chosen the maximum snapshot table size to be so large
(64 MB) that it cannot be reasonably reached. If the snapshot table
exceeds this size, the image has probably been corrupted in some way; in
this case, it is most important to just make the image usable such that
the user can copy off at least the active layer.
(Also note that the snapshots will be removed only with "-r all", so a
plain "check" or "check -r leaks" will not delete any data.)
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-13-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:09 +0000 (17:28 +0200)]
qcow2: Keep track of the snapshot table length
When repairing the snapshot table, we truncate entries that have too
much extra data. This frees up space that we do not have to count
towards the snapshot table size.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-12-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:08 +0000 (17:28 +0200)]
qcow2: Fix broken snapshot table entries
The only case where we currently reject snapshot table entries is when
they have too much extra data. Fix them with qemu-img check -r all by
counting it as a corruption, reducing their extra_data_size, and then
letting qcow2_check_fix_snapshot_table() do the rest.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-11-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:07 +0000 (17:28 +0200)]
qcow2: Add qcow2_check_fix_snapshot_table()
qcow2_check_read_snapshot_table() can perform consistency checks, but it
cannot fix everything. Specifically, it cannot allocate new clusters,
because that should wait until the refcount structures are known to be
consistent (i.e., after qcow2_check_refcounts()). Thus, it cannot call
qcow2_write_snapshots().
Do that in qcow2_check_fix_snapshot_table(), which is called after
qcow2_check_refcounts().
Currently, there is nothing that would set result->corruptions, so this
is a no-op. A follow-up patch will change that.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-10-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:06 +0000 (17:28 +0200)]
qcow2: Separate qcow2_check_read_snapshot_table()
Reading the snapshot table can fail. That is a problem when we want to
repair the image.
Therefore, stop reading the snapshot table in qcow2_do_open() in check
mode. Instead, add a new function qcow2_check_read_snapshot_table()
that reads the snapshot table at a later point. In the future, we want
to handle errors here and fix them.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-9-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:05 +0000 (17:28 +0200)]
qcow2: Write v3-compliant snapshot list on upgrade
qcow2 v3 requires every snapshot table entry to have two extra data
fields: The 64-bit VM state size, and the virtual disk size. Both are
optional for v2 images, so they may not be present.
qcow2_upgrade() therefore should update the snapshot table to ensure all
entries have these extra data fields.
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1727347 Reported-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-8-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:04 +0000 (17:28 +0200)]
qcow2: Put qcow2_upgrade() into its own function
This does not make sense right now, but it will make sense once we need
to do more than to just update s->qcow_version.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-7-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:03 +0000 (17:28 +0200)]
qcow2: Make qcow2_write_snapshots() public
Updating the snapshot list will be useful when upgrading a v2 image to
v3, so we will need to call this function in qcow2.c.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:02 +0000 (17:28 +0200)]
qcow2: Keep unknown extra snapshot data
The qcow2 specification says to ignore unknown extra data fields in
snapshot table entries. Currently, we discard it whenever we update the
image, which is a bit different from "ignore".
This patch makes the qcow2 driver keep all unknown extra data fields
when updating an image's snapshot table.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-5-mreitz@redhat.com
[mreitz: Adjusted comments as proposed by Eric] Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:01 +0000 (17:28 +0200)]
qcow2: Add Error ** to qcow2_read_snapshots()
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-4-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:28:00 +0000 (17:28 +0200)]
qcow2: Use endof()
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-3-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Fri, 11 Oct 2019 15:27:59 +0000 (17:27 +0200)]
include: Move endof() up from hw/virtio/virtio.h
endof() is a useful macro, we can make use of it outside of virtio.
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191011152814.14791-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Max Reitz [Mon, 14 Oct 2019 15:39:28 +0000 (17:39 +0200)]
mirror: Do not dereference invalid pointers
mirror_exit_common() may be called twice (if it is called from
mirror_prepare() and fails, it will be called from mirror_abort()
again).
In such a case, many of the pointers in the MirrorBlockJob object will
already be freed. This can be seen most reliably for s->target, which
is set to NULL (and then dereferenced by blk_bs()).
Cc: qemu-stable@nongnu.org Fixes: 737efc1eda23b904fbe0e66b37715fb0e5c3e58b Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191014153931.20699-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
No reason to limit buffered copy to one cluster. Let's allow up to 1
MiB.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-7-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Currently total allocation for parallel requests to block-copy instance
is unlimited. Let's limit it to 128 MiB.
For now block-copy is used only in backup, so actually we limit total
allocation for backup job.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-6-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Introduce an API for some shared splittable resource, like memory.
It's going to be used by backup. Backup uses both read/write io and
copy_range. copy_range may consume memory implictly, so the new API is
abstract: it doesn't allocate any real memory but only hands out
tickets.
The idea is that we have some total amount of something and callers
should wait in coroutine queue if there is not enough of the resource
at the moment.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-5-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Merge copying code into one function block_copy_do_copy, which only
calls bdrv_ io functions and don't do any synchronization (like dirty
bitmap set/reset).
Refactor block_copy() function so that it takes full decision about
size of chunk to be copied and does all the synchronization (checking
intersecting requests, set/reset dirty bitmaps).
It will help:
- introduce parallel processing of block_copy iterations: we need to
calculate chunk size, start async chunk copying and go to the next
iteration
- simplify synchronization improvement (like memory limiting in
further commit and reducing critical section (now we lock the whole
requested range, when actually we need to lock only dirty region
which we handle at the moment))
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-4-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
Large copy range may imply memory allocation and large io effort, so
using 2G copy range request may be bad idea. Let's limit it to 16 MiB.
It also helps the following patch to refactor copy-with-offload
fallback to copy-with-bounce-buffer.
Note, that total memory usage of backup is still not limited, it will
be fixed in further commit.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
block/block-copy: allocate buffer in block_copy_with_bounce_buffer
Move bounce_buffer allocation block_copy_with_bounce_buffer. This
commit simplifies further work on implementing copying by larger chunks
(of different size) and further asynchronous handling of block_copy
iterations (with help of block/aio_task API).
Allocation works fast, a lot faster than disk io, so it's not a problem
that we now allocate/free bounce_buffer more times. And we anyway will
have to allocate several bounce_buffers for parallel execution of loop
iterations in future.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191022111805.3432-2-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
projects / mirror_qemu.git / log