]> git.proxmox.com Git - mirror_qemu.git/log
mirror_qemu.git
7 years agodma/rc4030: translate memory accesses only when they occur
Hervé Poussineau [Fri, 18 Nov 2016 22:43:36 +0000 (23:43 +0100)]
dma/rc4030: translate memory accesses only when they occur

This simplifies the code a lot, and this fixes big memory leaks
introduced in a3d586f704609a45b6037534cb2f34da5dfd8895

Windows NT is now able to boot without using gigabytes of ram on the host.

Signed-off-by: Hervé Poussineau <hpoussin@reactos.org>
Reviewed-by: Yongbok Kim <yongbok.kim@imgtec.com>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agodma: rc4030: limit interval timer reload value
Prasad J Pandit [Wed, 12 Oct 2016 12:37:41 +0000 (18:07 +0530)]
dma: rc4030: limit interval timer reload value

The JAZZ RC4030 chipset emulator has a periodic timer and
associated interval reload register. The reload value is used
as divider when computing timer's next tick value. If reload
value is large, it could lead to divide by zero error. Limit
the interval reload value to avoid it.

Reported-by: Huawei PSIRT <psirt@huawei.com>
Signed-off-by: Prasad J Pandit <pjp@fedoraproject.org>
Tested-by: Hervé Poussineau <hpoussin@reactos.org>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agotarget/mips: fix delay slot detection in gen_msa_branch()
Yongbok Kim [Tue, 7 Mar 2017 15:37:34 +0000 (15:37 +0000)]
target/mips: fix delay slot detection in gen_msa_branch()

It is unnecessary to test R6 from delay/forbidden slot check
in gen_msa_branch().

https://bugs.launchpad.net/qemu/+bug/1663287

Reported-by: Brian Campbell <bacam@z273.org.uk>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agotarget-mips: replace few LOG_DISAS() with trace points
Philippe Mathieu-Daudé [Sat, 4 Mar 2017 18:56:52 +0000 (15:56 -0300)]
target-mips: replace few LOG_DISAS() with trace points

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Yongbok Kim <yongbok.kim@imgtec.com>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agotarget-mips: replace break by goto cp0_unimplemented
Philippe Mathieu-Daudé [Sat, 4 Mar 2017 18:56:51 +0000 (15:56 -0300)]
target-mips: replace break by goto cp0_unimplemented

this fixes many warnings like:

target/mips/translate.c:6253:13: warning: Value stored to 'rn' is never read
            rn = "invalid sel";
            ^    ~~~~~~~~~~~~~

Reported-by: Clang Static Analyzer
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Yongbok Kim <yongbok.kim@imgtec.com>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agotarget-mips: log bad coprocessor0 register accesses with LOG_UNIMP
Philippe Mathieu-Daudé [Sat, 4 Mar 2017 18:56:50 +0000 (15:56 -0300)]
target-mips: log bad coprocessor0 register accesses with LOG_UNIMP

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Yongbok Kim <yongbok.kim@imgtec.com>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agotarget-mips: remove old & unuseful comments
Philippe Mathieu-Daudé [Sat, 4 Mar 2017 18:56:49 +0000 (15:56 -0300)]
target-mips: remove old & unuseful comments

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Yongbok Kim <yongbok.kim@imgtec.com>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agotarget-mips: fix compiler warnings (clang 5)
Philippe Mathieu-Daudé [Sat, 4 Mar 2017 18:56:48 +0000 (15:56 -0300)]
target-mips: fix compiler warnings (clang 5)

static code analyzer complain:

target/mips/helper.c:453:5: warning: Function call argument is an uninitialized value
    qemu_log_mask(CPU_LOG_MMU,
    ^~~~~~~~~~~~~~~~~~~~~~~~~~

'physical' and 'prot' are uninitialized if 'ret' is not TLBRET_MATCH.

Reported-by: Clang Static Analyzer
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Yongbok Kim <yongbok.kim@imgtec.com>
Signed-off-by: Yongbok Kim <yongbok.kim@imgtec.com>
7 years agoMerge remote-tracking branch 'remotes/cohuck/tags/s390x-20170320' into staging
Peter Maydell [Mon, 20 Mar 2017 10:51:30 +0000 (10:51 +0000)]
Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20170320' into staging

One bugfix for device plug/unplug and migration in the
channel subsystem code.

# gpg: Signature made Mon 20 Mar 2017 08:45:59 GMT
# gpg:                using RSA key 0xDECF6B93C6F02FAF
# gpg: Good signature from "Cornelia Huck <huckc@linux.vnet.ibm.com>"
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>"
# Primary key fingerprint: C3D0 D66D C362 4FF6 A8C0  18CE DECF 6B93 C6F0 2FAF

* remotes/cohuck/tags/s390x-20170320:
  s390x/css: reassign subchannel if schid is changed after migration

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/kraxel/tags/pull-fixes-20170320-1' into staging
Peter Maydell [Mon, 20 Mar 2017 10:05:45 +0000 (10:05 +0000)]
Merge remote-tracking branch 'remotes/kraxel/tags/pull-fixes-20170320-1' into staging

fixes for 2.9: vnc, cirrus, tcg display updates.

# gpg: Signature made Mon 20 Mar 2017 08:52:34 GMT
# gpg:                using RSA key 0x4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/pull-fixes-20170320-1:
  vnc: fix a qio-channel leak
  cirrus: fix off-by-one in cirrus_bitblt_rop_bkwd_transp_*_16
  ui/console: ensure graphic updates don't race with TCG vCPUs

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agos390x/css: reassign subchannel if schid is changed after migration
Dong Jia Shi [Thu, 16 Feb 2017 05:27:33 +0000 (06:27 +0100)]
s390x/css: reassign subchannel if schid is changed after migration

The subchannel is a means to access a device. While the device number is
assigned by the administrator, the subchannel number is assigned by
the channel subsystem in an ascending order on cold and hot plug.
When doing unplug and replug operations, the same device may end up on
a different subchannel; for example

- We start with a device fe.1.2222, which ends up at subchannel
  fe.1.0000.
- Now we detach the device, attach a device fe.1.3333 (which would get
  the now-free subchannel fe.1.0000), re-attach fe.1.2222 (which ends
  up at subchannel fe.1.0001) and detach fe.1.3333.
- We now have the same device (fe.1.2222) available to the guest; it
  just shows up on a different subchannel.

In such a case, the subchannel numbers are different from what a
QEMU would create during cold plug when parsing the command line.

As this would cause a guest visible change on migration, we do restore
the source system's value of the subchannel number on load.

So we are now fine from the guest perspective. From the host
perspective this will cause an inconsistent state in our internal data
structures, though.

For example, the subchannel 0 might not be at array position 0. This will
lead to problems when we continue doing hot (un/re) plug operations.

Let's fix this by cleaning up our internal data structures.

Reported-by: Cornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
7 years agovnc: fix a qio-channel leak
Marc-André Lureau [Fri, 17 Mar 2017 09:28:02 +0000 (13:28 +0400)]
vnc: fix a qio-channel leak

Spotted by ASAN.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20170317092802.17973-1-marcandre.lureau@redhat.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
7 years agoconfigure: remove Cygwin
Paolo Bonzini [Fri, 17 Mar 2017 15:32:47 +0000 (16:32 +0100)]
configure: remove Cygwin

The Cygwin target is really compiling for native Win32 with -mno-cygwin.
Except, GCC 4.7.0 has finally removed the long deprecated -mno-cygwin
option, and that happened about five years ago.

Let it rest in peace.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
7 years agoxen: do not build backends for targets that do not support xen
Stefano Stabellini [Thu, 16 Mar 2017 20:01:50 +0000 (13:01 -0700)]
xen: do not build backends for targets that do not support xen

Change Makefile.objs to use CONFIG_XEN instead of CONFIG_XEN_BACKEND, so
that the Xen backends are only built for targets that support Xen.

Set CONFIG_XEN in the toplevel Makefile to ensure that files that are
built only once pick up Xen support properly.

Signed-off-by: Stefano Stabellini <stefano@aporeto.com>
Tested-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Greg Kurz <groug@kaod.org>
CC: pbonzini@redhat.com
CC: peter.maydell@linaro.org
CC: rth@twiddle.net
CC: stefanha@redhat.com
Message-Id: <1489694518-16978-1-git-send-email-sstabellini@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
7 years agoqemu-ga: obey LISTEN_PID when using systemd socket activation
Paolo Bonzini [Thu, 16 Mar 2017 15:29:45 +0000 (16:29 +0100)]
qemu-ga: obey LISTEN_PID when using systemd socket activation

qemu-ga's socket activation support was not obeying the LISTEN_PID
environment variable, which avoids that a process uses a socket-activation
file descriptor meant for its parent.

Mess can for example ensue if a process forks a children before consuming
the socket-activation file descriptor and therefore setting O_CLOEXEC
on it.

Luckily, qemu-nbd also got socket activation code, and its copy does
support LISTEN_PID.  Some extra fixups are needed to ensure that the
code can be used for both, but that's what this patch does.  The
main change is to replace get_listen_fds's "consume" argument with
the FIRST_SOCKET_ACTIVATION_FD macro from the qemu-nbd code.

Cc: "Richard W.M. Jones" <rjones@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
7 years agonios2: iic: Convert CPU prop to qom link
Marek Vasut [Fri, 17 Mar 2017 21:06:27 +0000 (22:06 +0100)]
nios2: iic: Convert CPU prop to qom link

Add a const qom link between the CPU and the IIC instead
of passing the CPU link through a qom property.

Signed-off-by: Marek Vasut <marex@denx.de>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20170317210627.23532-1-marex@denx.de
Cc: Alexander Graf <agraf@suse.de>
Cc: Chris Wulff <crwulff@gmail.com>
Cc: Igor Mammedov <imammedo@redhat.com>
Cc: Jeff Da Silva <jdasilva@altera.com>
Cc: Ley Foon Tan <lftan@altera.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Sandra Loosemore <sandra@codesourcery.com>
Cc: Yves Vandervennet <yvanderv@altera.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/xtensa/tags/20170317-xtensa' into staging
Peter Maydell [Sat, 18 Mar 2017 17:24:49 +0000 (17:24 +0000)]
Merge remote-tracking branch 'remotes/xtensa/tags/20170317-xtensa' into staging

target/xtensa fixes for 2.9:

- fix build failure when FDT support is not enabled;
- correctly pass command line arguments to semihosting guests.

# gpg: Signature made Fri 17 Mar 2017 18:14:01 GMT
# gpg:                using RSA key 0x51F9CC91F83FA044
# gpg: Good signature from "Max Filippov <filippov@cadence.com>"
# gpg:                 aka "Max Filippov <max.filippov@cogentembedded.com>"
# gpg:                 aka "Max Filippov <jcmvbkbc@gmail.com>"
# Primary key fingerprint: 2B67 854B 98E5 327D CDEB  17D8 51F9 CC91 F83F A044

* remotes/xtensa/tags/20170317-xtensa:
  target/xtensa: fix semihosting argc/argv implementation
  target/xtensa: xtfpga: load DTB only when FDT support is enabled

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agooslib-posix: fix compilation on OpenBSD
Paolo Bonzini [Fri, 17 Mar 2017 15:22:14 +0000 (16:22 +0100)]
oslib-posix: fix compilation on OpenBSD

si_band is not found in OpenBSD.   It is marked as obsolescent in
POSIX, so we can delete it without any remorse.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20170317152214.6148-1-pbonzini@redhat.com
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agocurl: fix compilation on OpenBSD
Paolo Bonzini [Fri, 17 Mar 2017 15:24:12 +0000 (16:24 +0100)]
curl: fix compilation on OpenBSD

EPROTO is not found in OpenBSD.   We usually use EIO when no better
errno is available, do that here too.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20170317152412.8472-1-pbonzini@redhat.com
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Peter Maydell [Fri, 17 Mar 2017 13:26:10 +0000 (13:26 +0000)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer fixes for 2.9.0-rc1

# gpg: Signature made Fri 17 Mar 2017 12:06:04 GMT
# gpg:                using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  block: quiesce AioContext when detaching from it
  thread-pool: add missing qemu_bh_cancel in completion function
  block: Propagate error in bdrv_open_backing_file
  blockdev: fix bitmap clear undo
  block: Always call bdrv_child_check_perm first
  file-posix: Don't leak fd in hdev_get_max_segments
  replication: clarify permissions
  file-posix: clean up max_segments buffer termination

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'mreitz/tags/pull-block-2017-03-17' into queue-block
Kevin Wolf [Fri, 17 Mar 2017 12:03:44 +0000 (13:03 +0100)]
Merge remote-tracking branch 'mreitz/tags/pull-block-2017-03-17' into queue-block

Block patches for 2.9-rc1

# gpg: Signature made Fri Mar 17 12:59:20 2017 CET
# gpg:                using RSA key 0xF407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2017-03-17:
  block: quiesce AioContext when detaching from it

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agoblock: quiesce AioContext when detaching from it
Paolo Bonzini [Tue, 14 Mar 2017 11:11:57 +0000 (12:11 +0100)]
block: quiesce AioContext when detaching from it

While it is true that bdrv_set_aio_context only works on a single
BlockDriverState subtree (see commit message for 53ec73e, "block: Use
bdrv_drain to replace uncessary bdrv_drain_all", 2015-07-07), it works
at the AioContext level rather than the BlockDriverState level.

Therefore, it is also necessary to trigger pending bottom halves too,
even if no requests are pending.

For NBD this ensures that the aio_co_schedule of a previous call to
nbd_attach_aio_context is completed before detaching from the old
AioContext; it fixes qemu-iotest 094.  Another similar bug happens
when the VM is stopped and the virtio-blk dataplane irqfd is torn down.
In this case it's possible that guest I/O gets stuck if notify_guest_bh
was scheduled but doesn't run.

Calling aio_poll from another AioContext is safe if non-blocking; races
such as the one mentioned in the commit message for c9d1a56 ("block:
only call aio_poll on the current thread's AioContext", 2016-10-28)
are a concern for blocking calls.

I considered other options, including:

- moving the bs->wakeup mechanism to AioContext, and letting the caller
check.  This might work for virtio which has a clear place to wakeup
(notify_place_bh) and check the condition (virtio_blk_data_plane_stop).
For aio_co_schedule I couldn't find a clear place to check the condition.

- adding a dummy oneshot bottom half and waiting for it to trigger.
This has the complication that bottom half list is LIFO for historical
reasons.  There were performance issues caused by bottom half ordering
in the past, so I decided against it for 2.9.

Fixes: 99723548561978da8ef44cf804fb7912698f5d88
Reported-by: Max Reitz <mreitz@redhat.com>
Reported-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Tested-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20170314111157.14464-2-pbonzini@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
7 years agothread-pool: add missing qemu_bh_cancel in completion function
Peter Lieven [Thu, 16 Mar 2017 16:02:49 +0000 (17:02 +0100)]
thread-pool: add missing qemu_bh_cancel in completion function

commit 3c80ca15 fixed a deadlock scenarion with nested aio_poll invocations.

However, the rescheduling of the completion BH introcuded unnecessary spinning
in the main-loop. On very fast file backends this can even lead to the
"WARNING: I/O thread spun for 1000 iterations" message popping up.

Callgrind reports about 3-4% less instructions with this patch running
qemu-img bench on a ramdisk based VMDK file.

Fixes: 3c80ca158c96ff902a30883a8933e755988948b1
Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Lieven <pl@kamp.de>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agoblock: Propagate error in bdrv_open_backing_file
Fam Zheng [Fri, 17 Mar 2017 01:56:30 +0000 (09:56 +0800)]
block: Propagate error in bdrv_open_backing_file

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agoblockdev: fix bitmap clear undo
John Snow [Wed, 15 Mar 2017 21:28:11 +0000 (17:28 -0400)]
blockdev: fix bitmap clear undo

Only undo the action if we actually prepared the action.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agoblock: Always call bdrv_child_check_perm first
Fam Zheng [Tue, 14 Mar 2017 02:30:50 +0000 (10:30 +0800)]
block: Always call bdrv_child_check_perm first

bdrv_child_set_perm alone is not very usable because the caller must
call bdrv_child_check_perm first. This is already encapsulated
conveniently in bdrv_child_try_set_perm, so remove the other prototypes
from the header and fix the one wrong caller, block/mirror.c.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agofile-posix: Don't leak fd in hdev_get_max_segments
Fam Zheng [Tue, 14 Mar 2017 16:12:05 +0000 (00:12 +0800)]
file-posix: Don't leak fd in hdev_get_max_segments

This fixes a leaked fd introduced in commit 9103f1ce.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agoreplication: clarify permissions
Changlong Xie [Tue, 14 Mar 2017 11:46:52 +0000 (19:46 +0800)]
replication: clarify permissions

Even if hidden_disk, secondary_disk are backing files, they all need
write permissions in replication scenario. Otherwise we will encouter
below exceptions on secondary side during adding nbd server:

{'execute': 'nbd-server-add', 'arguments': {'device': 'colo-disk', 'writable': true } }
{"error": {"class": "GenericError", "desc": "Conflicts with use by hidden-qcow2-driver as 'backing', which does not allow 'write' on sec-qcow2-driver-for-nbd"}}

CC: Zhang Hailiang <zhang.zhanghailiang@huawei.com>
CC: Zhang Chen <zhangchen.fnst@cn.fujitsu.com>
CC: Wen Congyang <wencongyang2@huawei.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agofile-posix: clean up max_segments buffer termination
Stefan Hajnoczi [Tue, 14 Mar 2017 09:09:22 +0000 (17:09 +0800)]
file-posix: clean up max_segments buffer termination

The following pattern is unsafe:

  char buf[32];
  ret = read(fd, buf, sizeof(buf));
  ...
  buf[ret] = 0;

If read(2) returns 32 then a byte beyond the end of the buffer is
zeroed.

In practice this buffer overflow does not occur because the sysfs
max_segments file only contains an unsigned short + '\n'.  The string is
always shorter than 32 bytes.

Regardless, avoid this pattern because static analysis tools might
complain and it could lead to real buffer overflows if copy-pasted
elsewhere in the codebase.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
7 years agocirrus: fix off-by-one in cirrus_bitblt_rop_bkwd_transp_*_16
Gerd Hoffmann [Fri, 17 Mar 2017 07:21:36 +0000 (08:21 +0100)]
cirrus: fix off-by-one in cirrus_bitblt_rop_bkwd_transp_*_16

The switch from pointers to addresses (commit
026aeffcb4752054830ba203020ed6eb05bcaba8 and
ffaf857778286ca54e3804432a2369a279e73aa7) added
a off-by-one bug to 16bit backward blits.  Fix.

Reported-by: 李强 <liqiang6-s@360.cn>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Li Qiang <liqiang6-s@360.cn>
Message-id: 1489735296-19047-1-git-send-email-kraxel@redhat.com

7 years agoui/console: ensure graphic updates don't race with TCG vCPUs
Alex Bennée [Wed, 15 Mar 2017 14:48:25 +0000 (14:48 +0000)]
ui/console: ensure graphic updates don't race with TCG vCPUs

Commit 8d04fb55..

  tcg: drop global lock during TCG code execution

..broke the assumption that updates to the GUI couldn't happen at the
same time as TCG vCPUs where running. As a result the TCG vCPU could
still be updating a directly mapped frame-buffer while the display
side was updating. This would cause artefacts to appear when the
update code assumed that memory block hadn't changed.

The simplest solution is to ensure the two things can't happen at the
same time like the old BQL locking scheme. Here we use the solution
introduced for MTTCG and schedule the update as async_safe_work when
we know no vCPUs can be running.

Reported-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20170315144825.3108-1-alex.bennee@linaro.org
Cc: BALATON Zoltan <balaton@eik.bme.hu>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
[ kraxel: updated comment clarifying the display adapters are buggy
          and this is a temporary workaround ]

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
7 years agoMerge remote-tracking branch 'remotes/kraxel/tags/pull-cirrus-20170316-1' into staging
Peter Maydell [Thu, 16 Mar 2017 16:40:44 +0000 (16:40 +0000)]
Merge remote-tracking branch 'remotes/kraxel/tags/pull-cirrus-20170316-1' into staging

cirrus: blitter fixes.

# gpg: Signature made Thu 16 Mar 2017 09:05:22 GMT
# gpg:                using RSA key 0x4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/pull-cirrus-20170316-1:
  cirrus: stop passing around src pointers in the blitter
  cirrus: stop passing around dst pointers in the blitter
  cirrus: fix cirrus_invalidate_region
  cirrus: add option to disable blitter
  cirrus: switch to 4 MB video memory by default
  cirrus/vnc: zap bitblit support from console code.
  fix :cirrus_vga fix OOB read case qemu Segmentation fault

# Conflicts:
# include/hw/compat.h

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/juanquintela/tags/migration/20170316' into...
Peter Maydell [Thu, 16 Mar 2017 15:32:07 +0000 (15:32 +0000)]
Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20170316' into staging

migration/next for 20170316

# gpg: Signature made Thu 16 Mar 2017 08:21:51 GMT
# gpg:                using RSA key 0xF487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration/20170316:
  postcopy: Check for shared memory
  RAMBlocks: qemu_ram_is_shared
  vmstate: fix failed iotests case 68 and 91
  migration/block: Avoid invoking blk_drain too frequently
  migration: use "" as the default for tls-creds/hostname
  Change the method to calculate dirty-pages-rate

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging
Peter Maydell [Thu, 16 Mar 2017 14:23:10 +0000 (14:23 +0000)]
Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging

Pull request

Tracing makefile fixes for QEMU 2.9.

# gpg: Signature made Thu 16 Mar 2017 06:56:10 GMT
# gpg:                using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/tracing-pull-request:
  trace: ensure $(tracetool-y) is defined in top level makefile
  makefile: generate trace-events-all upfront
  makefile: merge GENERATED_HEADERS & GENERATED_SOURCES variables

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agodtc: Revert unintentional submodule downgrade from commit c2cabb34220d63f9
Peter Maydell [Thu, 16 Mar 2017 14:11:15 +0000 (14:11 +0000)]
dtc: Revert unintentional submodule downgrade from commit c2cabb34220d63f9

Commit c2cabb34220d63f9 inadvertently downgraded the 'dtc' submodule,
undoing the increments added in earlier commits. Revert this,
returning the submodule state to where we should be.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2017-03-16' into staging
Peter Maydell [Thu, 16 Mar 2017 12:05:02 +0000 (12:05 +0000)]
Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2017-03-16' into staging

QAPI patches for 2017-03-16

# gpg: Signature made Thu 16 Mar 2017 06:18:38 GMT
# gpg:                using RSA key 0x3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2017-03-16: (49 commits)
  qapi: Fix a misleading parser error message
  qapi: Make pylint a bit happier
  qapi: Drop unused .check_clash() parameter schema
  qapi: union_types is a list used like a dict, make it one
  qapi: struct_types is a list used like a dict, make it one
  qapi: enum_types is a list used like a dict, make it one
  qapi: Factor add_name() calls out of the meta conditional
  qapi: Simplify what gets stored in enum_types
  qapi: Drop unused variable events
  qapi: Eliminate check_docs() and drop QAPIDoc.expr
  qapi: Fix detection of bogus member documentation
  tests/qapi-schema: Improve coverage of bogus member docs
  tests/qapi-schema: Rename doc-bad-args to doc-bad-command-arg
  qapi: Move empty doc section checking to doc parser
  qapi: Improve error message on @NAME: in free-form doc
  qapi: Move detection of doc / expression name mismatch
  qapi: Fix detection of doc / expression mismatch
  tests/qapi-schema: Improve doc / expression mismatch coverage
  qapi2texi: Use category "Object" for all object types
  qapi2texi: Generate descriptions for simple union tags
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging
Peter Maydell [Thu, 16 Mar 2017 11:05:47 +0000 (11:05 +0000)]
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio, pci: fixes

More fixes missed in the previous pull request.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
# gpg: Signature made Thu 16 Mar 2017 02:29:49 GMT
# gpg:                using RSA key 0x281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>"
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>"
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
  virtio-serial-bus: Delete timer from list before free it
  hw/virtio: fix Power Management Control Register for PCI Express virtio devices
  hw/virtio: fix Link Control Register for PCI Express virtio devices
  hw/virtio: fix error enabling flags in Device Control register
  hw/pcie: fix Extended Configuration Space for devices with no Extended Capabilities

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoMerge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging
Peter Maydell [Thu, 16 Mar 2017 10:04:41 +0000 (10:04 +0000)]
Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

# gpg: Signature made Thu 16 Mar 2017 00:52:41 GMT
# gpg:                using RSA key 0x7DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>"
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/ide-pull-request:
  ide: ahci: call cleanup function in ahci unit
  ide: core: add cleanup function
  ide: qdev: register ide bus unrealize function

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agopostcopy: Check for shared memory
Dr. David Alan Gilbert [Tue, 7 Mar 2017 18:36:37 +0000 (18:36 +0000)]
postcopy: Check for shared memory

Postcopy doesn't support migration of RAM shared with another process
yet (we've got a bunch of things to understand).
Check for the case and don't allow postcopy to be enabled.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
7 years agoRAMBlocks: qemu_ram_is_shared
Dr. David Alan Gilbert [Tue, 7 Mar 2017 18:36:36 +0000 (18:36 +0000)]
RAMBlocks: qemu_ram_is_shared

Provide a helper to say whether a RAMBlock was created as a
shared mapping.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
7 years agovmstate: fix failed iotests case 68 and 91
QingFeng Hao [Fri, 10 Mar 2017 04:44:02 +0000 (05:44 +0100)]
vmstate: fix failed iotests case 68 and 91

This problem affects s390x only if we are running without KVM.
Basically, S390CPU.irqstate is unused if we do not use KVM,
and thus no buffer is allocated.
This causes size=0, first_elem=NULL and n_elems=1 in
vmstate_load_state and vmstate_save_state. And the assert fails.
With this fix we can go back to the old behavior and support
VMS_VBUFFER with size 0 and nullptr.

Signed-off-by: QingFeng Hao <haoqf@linux.vnet.ibm.com>
Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
7 years agomigration/block: Avoid invoking blk_drain too frequently
Lidong Chen [Wed, 15 Mar 2017 03:37:33 +0000 (11:37 +0800)]
migration/block: Avoid invoking blk_drain too frequently

Increase bmds->cur_dirty after submit io, so reduce the frequency
involve into blk_drain, and improve the performance obviously
when block migration.

The performance test result of this patch:

During the block dirty save phase, this patch improve guest os IOPS
from 4.0K to 9.5K. and improve the migration speed from
505856 rsec/s to 855756 rsec/s.

Signed-off-by: Lidong Chen <jemmy858585@gmail.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
7 years agocirrus: stop passing around src pointers in the blitter
Gerd Hoffmann [Wed, 15 Mar 2017 13:28:07 +0000 (14:28 +0100)]
cirrus: stop passing around src pointers in the blitter

Does basically the same as "cirrus: stop passing around dst pointers in
the blitter", just for the src pointer instead of the dst pointer.

For the src we have to care about cputovideo blits though and fetch the
data from s->cirrus_bltbuf instead of vga memory.  The cirrus_src*()
helper functions handle that.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 1489584487-3489-1-git-send-email-kraxel@redhat.com

7 years agocirrus: stop passing around dst pointers in the blitter
Gerd Hoffmann [Wed, 15 Mar 2017 10:47:52 +0000 (11:47 +0100)]
cirrus: stop passing around dst pointers in the blitter

Instead pass around the address (aka offset into vga memory).  Calculate
the pointer in the rop_* functions, after applying the mask to the
address, to make sure the address stays within the valid range.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 1489574872-8679-1-git-send-email-kraxel@redhat.com

7 years agocirrus: fix cirrus_invalidate_region
Gerd Hoffmann [Wed, 15 Mar 2017 12:06:46 +0000 (13:06 +0100)]
cirrus: fix cirrus_invalidate_region

off_cur_end is exclusive, so off_cur_end == cirrus_addr_mask is valid.
Fix calculation to make sure to allow that, otherwise the assert added
by commit f153b563f8cf121aebf5a2fff5f0110faf58ccb3 can trigger for valid
blits.

Test case: boot windows nt 4.0

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 1489579606-26020-1-git-send-email-kraxel@redhat.com

7 years agocirrus: add option to disable blitter
Gerd Hoffmann [Tue, 14 Mar 2017 12:29:00 +0000 (13:29 +0100)]
cirrus: add option to disable blitter

Ok, we have this beast in the cirrus code which is not used at all by
modern guests, except when you try to find security holes in qemu.  So,
add an option to disable blitter altogether.  Guests released within
the last ten years should not show any rendering issues if you turn off
blitter support.

There are no known bugs in the cirrus blitter code.  But in the past we
hoped a few times already that we've finally nailed the last issue.  So
having some easy way to mitigate in case yet another blitter issue shows
up certainly makes me sleep a bit better at night.

For completeness:  The by far better way to mitigate is to switch away
from cirrus and use stdvga instead.  Or something more modern like
virtio-vga in case your guest has support for it.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 1489494540-15745-1-git-send-email-kraxel@redhat.com

7 years agocirrus: switch to 4 MB video memory by default
Gerd Hoffmann [Tue, 14 Mar 2017 12:28:34 +0000 (13:28 +0100)]
cirrus: switch to 4 MB video memory by default

Quoting cirrus source code:
   Follow real hardware, cirrus card emulated has 4 MB video memory.
   Also accept 8 MB/16 MB for backward compatibility.

So just use 4MB by default.  We decided to leave that at 8MB by default
a while ago, for live migration compatibility reasons.  But we have
compat properties to handle that, so that isn't a compeling reason.

This also removes some sanity check inconsistencies in the cirrus code.
Some places check against the allocated video memory, some places check
against the 4MB physical hardware has.  Guest code can trigger asserts
because of that.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 1489494514-15606-1-git-send-email-kraxel@redhat.com

7 years agocirrus/vnc: zap bitblit support from console code.
Gerd Hoffmann [Tue, 14 Mar 2017 12:26:59 +0000 (13:26 +0100)]
cirrus/vnc: zap bitblit support from console code.

There is a special code path (dpy_gfx_copy) to allow graphic emulation
notify user interface code about bitblit operations carryed out by
guests.  It is supported by cirrus and vnc server.  The intended purpose
is to optimize display scrolls and just send over the scroll op instead
of a full display update.

This is rarely used these days though because modern guests simply don't
use the cirrus blitter any more.  Any linux guest using the cirrus drm
driver doesn't.  Any windows guest newer than winxp doesn't ship with a
cirrus driver any more and thus uses the cirrus as simple framebuffer.

So this code tends to bitrot and bugs can go unnoticed for a long time.
See for example commit "3e10c3e vnc: fix qemu crash because of SIGSEGV"
which fixes a bug lingering in the code for almost a year, added by
commit "c7628bf vnc: only alloc server surface with clients connected".

Also the vnc server will throttle the frame rate in case it figures the
network can't keep up (send buffers are full).  This doesn't work with
dpy_gfx_copy, for any copy operation sent to the vnc client we have to
send all outstanding updates beforehand, otherwise the vnc client might
run the client side blit on outdated data and thereby corrupt the
display.  So this dpy_gfx_copy "optimization" might even make things
worse on slow network links.

Lets kill it once for all.

Oh, and one more reason: Turns out (after writing the patch) we have a
security bug in that code path ...

Fixes: CVE-2016-9603
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 1489494419-14340-1-git-send-email-kraxel@redhat.com

7 years agofix :cirrus_vga fix OOB read case qemu Segmentation fault
hangaohuai [Tue, 14 Mar 2017 06:39:19 +0000 (14:39 +0800)]
fix :cirrus_vga fix OOB read case qemu Segmentation fault

check the validity of parameters in cirrus_bitblt_rop_fwd_transp_xxx
and cirrus_bitblt_rop_fwd_xxx to avoid the OOB read which causes qemu Segmentation fault.

After the fix, we will touch the assert in
cirrus_invalidate_region:
assert(off_cur_end >= off_cur);

Signed-off-by: fangying <fangying1@huawei.com>
Signed-off-by: hangaohuai <hangaohuai@huawei.com>
Message-id: 20170314063919.16200-1-hangaohuai@huawei.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
7 years agomigration: use "" as the default for tls-creds/hostname
Daniel P. Berrange [Wed, 15 Mar 2017 16:16:03 +0000 (16:16 +0000)]
migration: use "" as the default for tls-creds/hostname

The tls-creds parameter has a default value of NULL indicating
that TLS should not be used. Setting it to non-NULL enables
use of TLS. Once tls-creds are set to a non-NULL value via the
monitor, it isn't possible to set them back to NULL again, due
to current implementation limitations. The empty string is not
a valid QObject identifier, so this switches to use "" as the
default, indicating that TLS will not be used

The tls-hostname parameter has a default value of NULL indicating
the the hostname from the migrate connection URI should be used.
Again, once tls-hostname is set non-NULL, to override the default
hostname for x509 cert validation, it isn't possible to reset it
back to NULL via the monitor. The empty string is not a valid
hostname, so this switches to use "" as the default, indicating
that the migrate URI hostname should be used.

Using "" as the default for both, also means that the monitor
commands "info migrate_parameters" / "query-migrate-parameters"
will report existance of tls-creds/tls-parameters even when set
to their default values.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
7 years agoChange the method to calculate dirty-pages-rate
Chao Fan [Tue, 14 Mar 2017 01:55:07 +0000 (09:55 +0800)]
Change the method to calculate dirty-pages-rate

In function cpu_physical_memory_sync_dirty_bitmap, file
include/exec/ram_addr.h:

if (src[idx][offset]) {
    unsigned long bits = atomic_xchg(&src[idx][offset], 0);
    unsigned long new_dirty;
    new_dirty = ~dest[k];
    dest[k] |= bits;
    new_dirty &= bits;
    num_dirty += ctpopl(new_dirty);
}

After these codes executed, only the pages not dirtied in bitmap(dest),
but dirtied in dirty_memory[DIRTY_MEMORY_MIGRATION] will be calculated.
For example:
When ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION] = 0b00001111,
and atomic_rcu_read(&migration_bitmap_rcu)->bmap = 0b00000011,
the new_dirty will be 0b00001100, and this function will return 2 but not
4 which is expected.
the dirty pages in dirty_memory[DIRTY_MEMORY_MIGRATION] are all new,
so these should be calculated also.

Signed-off-by: Chao Fan <fanc.fnst@cn.fujitsu.com>
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
7 years agoMerge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-signed' into staging
Peter Maydell [Thu, 16 Mar 2017 07:47:12 +0000 (07:47 +0000)]
Merge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-signed' into staging

Update OpenBIOS images

# gpg: Signature made Wed 15 Mar 2017 21:01:53 GMT
# gpg:                using RSA key 0x5BC2C56FAE0F321F
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>"
# Primary key fingerprint: CC62 1AB9 8E82 200D 915C  C9C4 5BC2 C56F AE0F 321F

* remotes/mcayland/tags/qemu-openbios-signed:
  Update OpenBIOS images to f233c3f built from submodule.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
7 years agoqapi: Fix a misleading parser error message
Markus Armbruster [Wed, 15 Mar 2017 12:57:36 +0000 (13:57 +0100)]
qapi: Fix a misleading parser error message

When choking on a token where an expression is expected, we report
'Expected "{", "[" or string'.  Close, but no cigar.  Fix it to
Expected '"{", "[", string, boolean or "null"'.

Missed in commit e53188a.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-48-git-send-email-armbru@redhat.com>

7 years agoqapi: Make pylint a bit happier
Markus Armbruster [Wed, 15 Mar 2017 12:57:35 +0000 (13:57 +0100)]
qapi: Make pylint a bit happier

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-47-git-send-email-armbru@redhat.com>

7 years agoqapi: Drop unused .check_clash() parameter schema
Markus Armbruster [Wed, 15 Mar 2017 12:57:34 +0000 (13:57 +0100)]
qapi: Drop unused .check_clash() parameter schema

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-46-git-send-email-armbru@redhat.com>

7 years agoqapi: union_types is a list used like a dict, make it one
Markus Armbruster [Wed, 15 Mar 2017 12:57:33 +0000 (13:57 +0100)]
qapi: union_types is a list used like a dict, make it one

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-45-git-send-email-armbru@redhat.com>

7 years agoqapi: struct_types is a list used like a dict, make it one
Markus Armbruster [Wed, 15 Mar 2017 12:57:32 +0000 (13:57 +0100)]
qapi: struct_types is a list used like a dict, make it one

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-44-git-send-email-armbru@redhat.com>

7 years agoqapi: enum_types is a list used like a dict, make it one
Markus Armbruster [Wed, 15 Mar 2017 12:57:31 +0000 (13:57 +0100)]
qapi: enum_types is a list used like a dict, make it one

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-43-git-send-email-armbru@redhat.com>

7 years agoqapi: Factor add_name() calls out of the meta conditional
Markus Armbruster [Wed, 15 Mar 2017 12:57:30 +0000 (13:57 +0100)]
qapi: Factor add_name() calls out of the meta conditional

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-42-git-send-email-armbru@redhat.com>

7 years agoqapi: Simplify what gets stored in enum_types
Markus Armbruster [Wed, 15 Mar 2017 12:57:29 +0000 (13:57 +0100)]
qapi: Simplify what gets stored in enum_types

Don't invent a new dictionary structure just for enum_types, simply
store the defining expression, like we do for struct_types and
union_types.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-41-git-send-email-armbru@redhat.com>

7 years agoqapi: Drop unused variable events
Markus Armbruster [Wed, 15 Mar 2017 12:57:28 +0000 (13:57 +0100)]
qapi: Drop unused variable events

Missed in commit e98859a

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-40-git-send-email-armbru@redhat.com>

7 years agoqapi: Eliminate check_docs() and drop QAPIDoc.expr
Markus Armbruster [Wed, 15 Mar 2017 12:57:27 +0000 (13:57 +0100)]
qapi: Eliminate check_docs() and drop QAPIDoc.expr

Move what's left in check_docs() to check_expr().  Delegate the actual
checking to new QAPIDoc.check_expr().

QAPIDoc.expr is now unused; drop it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-39-git-send-email-armbru@redhat.com>

7 years agoqapi: Fix detection of bogus member documentation
Markus Armbruster [Wed, 15 Mar 2017 12:57:26 +0000 (13:57 +0100)]
qapi: Fix detection of bogus member documentation

check_definition_doc() checks for member documentation without a
matching member.  It laboriously second-guesses what members
QAPISchema._def_exprs() will create.  That's a stupid game.

Move the check into QAPISchema.check(), where the members are known.
Delegate the actual checking to new QAPIDoc.check().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-38-git-send-email-armbru@redhat.com>

7 years agotests/qapi-schema: Improve coverage of bogus member docs
Markus Armbruster [Wed, 15 Mar 2017 12:57:25 +0000 (13:57 +0100)]
tests/qapi-schema: Improve coverage of bogus member docs

New test doc-bad-union-member.json shows we can fail to reject
documentation for nonexistent members.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-37-git-send-email-armbru@redhat.com>

7 years agotests/qapi-schema: Rename doc-bad-args to doc-bad-command-arg
Markus Armbruster [Wed, 15 Mar 2017 12:57:24 +0000 (13:57 +0100)]
tests/qapi-schema: Rename doc-bad-args to doc-bad-command-arg

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-36-git-send-email-armbru@redhat.com>

7 years agoqapi: Move empty doc section checking to doc parser
Markus Armbruster [Wed, 15 Mar 2017 12:57:23 +0000 (13:57 +0100)]
qapi: Move empty doc section checking to doc parser

Results in a more precise error location, but the real reason is
emptying out check_docs() step by step.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-35-git-send-email-armbru@redhat.com>

7 years agoqapi: Improve error message on @NAME: in free-form doc
Markus Armbruster [Wed, 15 Mar 2017 12:57:22 +0000 (13:57 +0100)]
qapi: Improve error message on @NAME: in free-form doc

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-34-git-send-email-armbru@redhat.com>

7 years agoqapi: Move detection of doc / expression name mismatch
Markus Armbruster [Wed, 15 Mar 2017 12:57:21 +0000 (13:57 +0100)]
qapi: Move detection of doc / expression name mismatch

Move the check whether the doc matches the expression name from
check_definition_doc() to check_exprs().  This changes the error
location from the comment to the expression.  Makes sense as the
message talks about the expression: "Definition of '%s' follows
documentation for '%s'".  It's also a step towards getting rid of
check_docs().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-33-git-send-email-armbru@redhat.com>

7 years agoqapi: Fix detection of doc / expression mismatch
Markus Armbruster [Wed, 15 Mar 2017 12:57:20 +0000 (13:57 +0100)]
qapi: Fix detection of doc / expression mismatch

This fixes the errors uncovered by the previous commit.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-32-git-send-email-armbru@redhat.com>

7 years agotests/qapi-schema: Improve doc / expression mismatch coverage
Markus Armbruster [Wed, 15 Mar 2017 12:57:19 +0000 (13:57 +0100)]
tests/qapi-schema: Improve doc / expression mismatch coverage

New tests doc-before-include.json and doc-before-pragma.json show we
fail to reject a misplaced expression comment.

New test doc-no-symbol.json shows a bad error message.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1489582656-31133-31-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
7 years agoqapi2texi: Use category "Object" for all object types
Markus Armbruster [Wed, 15 Mar 2017 12:57:18 +0000 (13:57 +0100)]
qapi2texi: Use category "Object" for all object types

At the protocol level, the distinction between struct, flat union and
simple union is meaningless, they are all JSON objects.  Document them
that way.

Example change (qemu-qmp-ref.txt):

- -- Simple Union: InputEvent
+ -- Object: InputEvent

      Input event union.

This also fixes the completely broken headings for flat and simple
unions in qemu-qmp-ref.7 and qemu-ga-ref.7, by sidestepping a bug in
texi2pod.pl.  For instance, it mistranslates "@deftp {Simple Union}
InputEvent" to "B<Union> (Simple)", but translates "@deftp Object
InputEvent" to "B<SocketAddress> (Object)".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-30-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Generate descriptions for simple union tags
Markus Armbruster [Wed, 15 Mar 2017 12:57:17 +0000 (13:57 +0100)]
qapi2texi: Generate descriptions for simple union tags

Simple union tags carry no type information, because their type is
implicit.  Their description should make up for it, but many have
none.  Generate one automatically then.

Example change (qemu-qmp-ref.txt):

  -- Simple Union: ImageInfoSpecific

      A discriminated record of image format specific information
      structures.

      Members:
      'type'
-          Not documented
+          One of "qcow2", "vmdk", "luks"
      'data: ImageInfoSpecificQCow2' when 'type' is "qcow2"
      'data: ImageInfoSpecificVmdk' when 'type' is "vmdk"
      'data: QCryptoBlockInfoLUKS' when 'type' is "luks"

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-29-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Generate documentation for variant members
Markus Armbruster [Wed, 15 Mar 2017 12:57:16 +0000 (13:57 +0100)]
qapi2texi: Generate documentation for variant members

A flat union's branch brings in the members of another type.  Generate
a suitable reference to that type.

Example change (qemu-qmp-ref.txt):

  -- Flat Union: QCryptoBlockOpenOptions

      The options that are available for all encryption formats when
      opening an existing volume

      Members:
      The members of 'QCryptoBlockOptionsBase'
+     The members of 'QCryptoBlockOptionsQCow' when 'format' is "qcow"
+     The members of 'QCryptoBlockOptionsLUKS' when 'format' is "luks"

      Since: 2.6

A simple union's branch adds a member 'data' of some other type.
Generate documentation for that member.

Example change (qemu-qmp-ref.txt):

  -- Simple Union: SocketAddress

      Captures the address of a socket, which could also be a named file
      descriptor

      Members:
      'type'
   Not documented
+     'data: InetSocketAddress' when 'type' is "inet"
+     'data: UnixSocketAddress' when 'type' is "unix"
+     'data: VsockSocketAddress' when 'type' is "vsock"
+     'data: String' when 'type' is "fd"

      Since: 1.3

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-28-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Generate reference to base type members
Markus Armbruster [Wed, 15 Mar 2017 12:57:15 +0000 (13:57 +0100)]
qapi2texi: Generate reference to base type members

The generated documentation doesn't mention object type members
inherited from a base type.  Fix that.

Example change (qemu-qmp-ref.txt):

  -- Struct: VncServerInfo

      The network connection information for server

      Members:
      'auth' (optional)
   authentication method used for the plain (non-websocket) VNC
   server
+     The members of 'VncBasicInfo'

      Since: 2.1

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-27-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Include member type in generated documentation
Markus Armbruster [Wed, 15 Mar 2017 12:57:14 +0000 (13:57 +0100)]
qapi2texi: Include member type in generated documentation

The recent merge of docs/qmp-commands.txt and docs/qmp-events.txt into
the schema lost type information.  Fix this documentation regression.

Example change (qemu-qmp-ref.txt):

  -- Struct: InputKeyEvent

      Keyboard input event.

      Members:
-     'button'
+     'button: InputButton'
           Which button this event is for.
-     'down'
+     'down: boolean'
           True for key-down and false for key-up events.

      Since: 2.0

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-26-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Implement boxed argument documentation
Markus Armbruster [Wed, 15 Mar 2017 12:57:13 +0000 (13:57 +0100)]
qapi2texi: Implement boxed argument documentation

This replaces manual references like "For the arguments, see the
documentation of ..." by a generated reference "Arguments: the members
of ...".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-25-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Don't hide undocumented members and arguments
Markus Armbruster [Wed, 15 Mar 2017 12:57:12 +0000 (13:57 +0100)]
qapi2texi: Don't hide undocumented members and arguments

Show undocumented object, alternate type members and command, event
arguments exactly like undocumented enumeration type values.

Example change (qemu-qmp-ref.txt):

  -- Command: query-rocker

      Return rocker switch information.

+     Arguments:
+     'name'
+          Not documented
+
      Returns: 'Rocker' information

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-24-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Explain enum value undocumentedness more clearly
Markus Armbruster [Wed, 15 Mar 2017 12:57:11 +0000 (13:57 +0100)]
qapi2texi: Explain enum value undocumentedness more clearly

Instead of not saying anything when we have no documentation, say "Not
documented".

Example change (qemu-qmp-ref.txt):

  -- Enum: GuestPanicAction

      An enumeration of the actions taken when guest OS panic is detected

      Values:
      'pause'
           system pauses
      'poweroff'
+          Not documented

      Since: 2.1 (poweroff since 2.8)

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-23-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Present the table of members more clearly
Markus Armbruster [Wed, 15 Mar 2017 12:57:10 +0000 (13:57 +0100)]
qapi2texi: Present the table of members more clearly

The table of members follows the main descriptive text immediately.
Makes it hard to see what it is about.  Start a new paragraph, and
lead with a line "Members:" for object and alternate types, "Values:"
for enumeration types, and "Arguments:" for commands and events.

Example change (qemu-qmp-ref.txt):

  -- Command: set_link

      Sets the link status of a virtual network adapter.
+
+     Arguments:
      'name'
           the device name of the virtual network adapter
      'up'
           true to set the link status to be up

      Returns: Nothing on success If 'name' is not a valid network
      device, DeviceNotFound

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-22-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Plainer enum value and member name formatting
Markus Armbruster [Wed, 15 Mar 2017 12:57:09 +0000 (13:57 +0100)]
qapi2texi: Plainer enum value and member name formatting

Use @code{%s} instead of @code{'%s'}.  Impact, using @id as example:

* Texinfo
  -@item @code{'id'}
  +@item @code{id}

* HTML
  -<dt><code>'id'</code></dt>
  +<dt><code>id</code></dt>

* POD (for manual pages):
  -=item C<'id'>
  +=item C<id>

* Formatted manual pages:
  -'id'
  +"id"

* Plain text:
  -     ''id''
  +     'id'

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-21-git-send-email-armbru@redhat.com>

7 years agoqapi: Prefer single-quoted strings more consistently
Markus Armbruster [Wed, 15 Mar 2017 12:57:08 +0000 (13:57 +0100)]
qapi: Prefer single-quoted strings more consistently

PEP 8 advises:

    In Python, single-quoted strings and double-quoted strings are the
    same.  This PEP does not make a recommendation for this.  Pick a
    rule and stick to it.  When a string contains single or double
    quote characters, however, use the other one to avoid backslashes
    in the string.  It improves readability.

The QAPI generators succeed at picking a rule, but fail at sticking to
it.  Convert a bunch of double-quoted strings to single-quoted ones.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-20-git-send-email-armbru@redhat.com>

7 years agoqapi: Use raw strings for regular expressions consistently
Markus Armbruster [Wed, 15 Mar 2017 12:57:07 +0000 (13:57 +0100)]
qapi: Use raw strings for regular expressions consistently

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-19-git-send-email-armbru@redhat.com>

7 years agoqapi: The #optional tag is redundant, drop
Markus Armbruster [Wed, 15 Mar 2017 12:57:06 +0000 (13:57 +0100)]
qapi: The #optional tag is redundant, drop

We traditionally mark optional members #optional in the doc comment.
Before commit 3313b61, this was entirely manual.

Commit 3313b61 added some automation because its qapi2texi.py relied
on #optional to determine whether a member is optional.  This is no
longer the case since the previous commit: the only thing qapi2texi.py
still does with #optional is stripping it out.  We still reject bogus
qapi-schema.json and six places for qga/qapi-schema.json.

Thus, you can't actually rely on #optional to see whether something is
optional.  Yet we still make people add it manually.  That's just
busy-work.

Drop the code to check, fix up and strip out #optional, along with all
instances of #optional.  To keep it out, add code to reject it, to be
dropped again once the dust settles.

No change to generated documentation.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-18-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Convert to QAPISchemaVisitor
Markus Armbruster [Wed, 15 Mar 2017 12:57:05 +0000 (13:57 +0100)]
qapi2texi: Convert to QAPISchemaVisitor

qapi2texi works with schema expression trees.  Such a tight coupling
to schema language syntax is not a good idea.  Convert it to the visitor
interface the other generators use.

No change to generated documentation.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-17-git-send-email-armbru@redhat.com>

7 years agoqapi: Conjure up QAPIDoc.ArgSection for undocumented members
Markus Armbruster [Wed, 15 Mar 2017 12:57:04 +0000 (13:57 +0100)]
qapi: Conjure up QAPIDoc.ArgSection for undocumented members

qapi2texi.py already conjures up ArgSections for undocumented
enumeration values, in texi_enum.  Drop that, and conjure them up for
all kinds of "arguments" (enumeration values, object and alternate
type members) in qapi.py instead.

Take care to keep generated documentation exactly the same for now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-16-git-send-email-armbru@redhat.com>

7 years agoqapi: Prepare for requiring more complete documentation
Markus Armbruster [Wed, 15 Mar 2017 12:57:03 +0000 (13:57 +0100)]
qapi: Prepare for requiring more complete documentation

We currently neglect to check all enumeration values, common members
of object types and members of alternate types are documented.
Unsurprisingly, many aren't.

Add the necessary plumbing to find undocumented ones, except for
variant members of object types.  Don't enforce anything just yet, but
connect each QAPIDoc.ArgSection to its QAPISchemaMember.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-15-git-send-email-armbru@redhat.com>

7 years agoqapi: Fix QAPISchemaEnumType.is_implicit() for 'QType'
Markus Armbruster [Wed, 15 Mar 2017 12:57:02 +0000 (13:57 +0100)]
qapi: Fix QAPISchemaEnumType.is_implicit() for 'QType'

Missed in commit 7264f5c.  Harmless, because nothing checks whether an
enumeration type is implicit so far.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-14-git-send-email-armbru@redhat.com>

7 years agoqapi/rocker: Fix up doc comment notes on optional members
Markus Armbruster [Wed, 15 Mar 2017 12:57:01 +0000 (13:57 +0100)]
qapi/rocker: Fix up doc comment notes on optional members

Talking about #optional like this

    # Note: fields are marked #optional to indicate that they may or may
    # not appear ...

doesn't work so well in generated documentation, because the #optional
tag is not visible there.  Replace by

    # Note: optional members may or may not appear ...

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1489582656-31133-13-git-send-email-armbru@redhat.com>

7 years agoqapi: Avoid unwanted blank lines in QAPIDoc
Markus Armbruster [Wed, 15 Mar 2017 12:57:00 +0000 (13:57 +0100)]
qapi: Avoid unwanted blank lines in QAPIDoc

We silently fix missing #optional tags for QAPIDoc by appending a line
"#optional" to the section's .content.  However, this interferes with
.__repr__ stripping trailing blank lines from .content.

Use new ArgSection instance variable .optional instead, and leave
.content alone.

To permit testing .optional in texi_body(), clean up texi_enum()'s
hack to add empty documentation for undocumented enum values: add an
ArgSection instead of ''.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1489582656-31133-12-git-send-email-armbru@redhat.com>

7 years agoqapi2texi: Fix up output around #optional
Markus Armbruster [Wed, 15 Mar 2017 12:56:59 +0000 (13:56 +0100)]
qapi2texi: Fix up output around #optional

We use tag #optional to mark optional members, like this:

    # @name: #optional The name of the guest

texi_body() strips #optional, but not whitespace around it.  For the
above, we get in qemu-qmp-qapi.texi

    @item @code{'name'} (optional)
     The name of the guest
    @end table

The extra space can lead to artifacts in output, e.g in
qemu-qmp-ref.7.pod

    =item C<'name'> (optional)

     The name of the guest

and then in qemu-qmp-ref.7

    .IX Item "name (optional)"
    .Vb 1
    \& The name of the guest
    .Ve

instead of intended plain

    .IX Item "name (optional)"
    The name of the guest

Get rid of these artifacts by removing whitespace around #optional
along with it.

This turns three minus signs in qapi-schema.json into markup, because
they're now at the beginning of the line.  Drop them, they're unwanted
there.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1489582656-31133-11-git-send-email-armbru@redhat.com>

7 years agoqapi: Fix to reject empty union base gracefully
Markus Armbruster [Wed, 15 Mar 2017 12:56:58 +0000 (13:56 +0100)]
qapi: Fix to reject empty union base gracefully

Common Python pitfall: 'assert base_members' fires on [] in addition
to None.  Correct to 'assert base_members is not None'.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-10-git-send-email-armbru@redhat.com>

7 years agotests/qapi-schema: Cover empty union base
Markus Armbruster [Wed, 15 Mar 2017 12:56:57 +0000 (13:56 +0100)]
tests/qapi-schema: Cover empty union base

The new test case shows off qapi.py choking on an empty union base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-9-git-send-email-armbru@redhat.com>

7 years agoqapi: Clean up build of generated documentation
Markus Armbruster [Wed, 15 Mar 2017 12:56:56 +0000 (13:56 +0100)]
qapi: Clean up build of generated documentation

Rename intermediate qemu-qapi.texi to qemu-qmp-qapi.texi to match its
user qemu-qmp-ref.texi, just like qemu-ga-qapi.texi matches
qemu-ga-ref.texi.

Build the intermediate .texi next to the sources and the final output
in docs/ instead of dumping them into the build root.

Fix version.texi dependencies so that only the targets that actually
need it depend on it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-8-git-send-email-armbru@redhat.com>

7 years agoqapi: Have each QAPI schema declare its name rule violations
Markus Armbruster [Wed, 15 Mar 2017 12:56:55 +0000 (13:56 +0100)]
qapi: Have each QAPI schema declare its name rule violations

qapi.py has a hardcoded white-list of type names that may violate the
rule on use of upper and lower case.  Add a new pragma directive
'name-case-whitelist', and use it to replace the hard-coded
white-list.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1489582656-31133-7-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
7 years agoqapi: Have each QAPI schema declare its returns white-list
Markus Armbruster [Wed, 15 Mar 2017 12:56:54 +0000 (13:56 +0100)]
qapi: Have each QAPI schema declare its returns white-list

qapi.py has a hardcoded white-list of command names that may violate
the rules on permitted return types.  Add a new pragma directive
'returns-whitelist', and use it to replace the hard-coded white-list.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1489582656-31133-6-git-send-email-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
7 years agodocs/qapi-code-gen.txt: Drop confusing reference to 'gen'
Markus Armbruster [Wed, 15 Mar 2017 12:56:53 +0000 (13:56 +0100)]
docs/qapi-code-gen.txt: Drop confusing reference to 'gen'

Section "Commands" qualifies its rules on permitted argument and
return types "with one exception noted below when 'gen' is used".  The
note went away in commit 2d21291.  Clean up the dangling references.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1489582656-31133-5-git-send-email-armbru@redhat.com>

7 years agoqapi: Back out doc comments added just to please qapi.py
Markus Armbruster [Wed, 15 Mar 2017 12:56:52 +0000 (13:56 +0100)]
qapi: Back out doc comments added just to please qapi.py

This reverts commit 3313b61's changes to tests/qapi-schema/, except
for tests/qapi-schema/doc-*.

We could keep some of these doc comments to serve as positive test
cases.  However, they don't actually add to what we get from doc
comment use in actual schemas, as we we don't test output matches
expectations, and don't systematically cover doc comment features.
Proper positive test coverage would be nice.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <1489582656-31133-4-git-send-email-armbru@redhat.com>

7 years agoqapi: Make doc comments optional where we don't need them
Markus Armbruster [Wed, 15 Mar 2017 12:56:51 +0000 (13:56 +0100)]
qapi: Make doc comments optional where we don't need them

Since we added the documentation generator in commit 3313b61, doc
comments are mandatory.  That's a very good idea for a schema that
needs to be documented, but has proven to be annoying for testing.

Make doc comments optional again, but add a new directive

    { 'pragma': { 'doc-required': true } }

to let a QAPI schema require them.

Add test cases for the new pragma directive.  While there, plug a
minor hole in includ directive test coverage.

Require documentation in the schemas we actually want documented:
qapi-schema.json and qga/qapi-schema.json.

We could probably make qapi2texi.py cope with incomplete
documentation, but for now, simply make it refuse to run unless the
schema has 'doc-required': true.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <1489582656-31133-3-git-send-email-armbru@redhat.com>
[qapi-code-gen.txt wording tweaked]
Reviewed-by: Eric Blake <eblake@redhat.com>
7 years agoqapi: Factor QAPISchemaParser._include() out of .__init__()
Markus Armbruster [Wed, 15 Mar 2017 12:56:50 +0000 (13:56 +0100)]
qapi: Factor QAPISchemaParser._include() out of .__init__()

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <1489582656-31133-2-git-send-email-armbru@redhat.com>

7 years agoqmp: allow setting properties to empty string in qmp-shell
Daniel P. Berrange [Thu, 2 Mar 2017 12:24:29 +0000 (12:24 +0000)]
qmp: allow setting properties to empty string in qmp-shell

The qmp-shell property parser currently rejects attempts to
set string properties to the empty string eg

  (QEMU) migrate-set-parameters  tls-hostname=
  Error while parsing command line: Expected a key=value pair, got 'tls-hostname='
command format: <command-name>  [arg-name1=arg1] ... [arg-nameN=argN]

This is caused by checking the wrong condition after splitting
the parameter on '='. The "partition" method will return "" for
the separator field, if the seperator was not present, so that
is the correct thing to check for malformed syntax.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <20170302122429.7737-1-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>