Thomas Huth [Thu, 31 Jan 2019 13:53:29 +0000 (14:53 +0100)]
ppc: Express dependencies of the Sam460EX machines with kconfig
Most of the dependencies are now directly selected by the SAM460EX
switch. We can drop CONFIG_VGA_CIRRUS since this device is already
selected automatically when CONFIG_PCI_DEVICES is set.
Reviewed-by: BALATON Zoltan <balaton@eik.bme.hu> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Thu, 31 Jan 2019 13:53:28 +0000 (14:53 +0100)]
ppc: Express dependencies of the Mac machines with kconfig
This will make it for example easier if the users want to disable
one of the two machines for their builds.
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Thu, 31 Jan 2019 13:53:27 +0000 (14:53 +0100)]
ppc: Express dependencies of the 'prep' and '40p' machines with kconfig
Select the required devices in hw/ppc/Kconfig instead, so that
ppc-softmmu.mak only contains the user-selectable PREP switch.
Plug-in devices like NE2000_ISA are pulled in automatically by the
Kconfig build system now.
Cc: Hervé Poussineau <hpoussin@reactos.org> Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Thomas Huth [Thu, 31 Jan 2019 13:53:26 +0000 (14:53 +0100)]
ppc64: Express dependencies of 'pseries' and 'powernv' machines with kconfig
The POWERNV switch should always select ISA_IPMI_BT, then the other
IPMI options are turned on automatically now.
CONFIG_DIMM should always be selected by the pseries machine,
which in turn depends on CONFIG_MEM_DEVICE since DIMM implements
this interface.
CONFIG_VIRTIO_VGA can be dropped from default-configs/ppc64-softmmu.mak
completely since this device is already automatically enabled via
hw/display/Kconfig now.
Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Yang Zhong [Wed, 23 Jan 2019 06:56:16 +0000 (14:56 +0800)]
i386-softmmu.mak: remove all CONFIG_* except boards definitions
%-softmmu.mak only keep boards definitions in Kconfig mode.
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-43-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Tue, 29 Jan 2019 11:16:25 +0000 (12:16 +0100)]
ipmi: express dependencies with kconfig
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-36-yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Yang Zhong [Wed, 23 Jan 2019 06:56:15 +0000 (14:56 +0800)]
virtio: express virtio dependencies with Kconfig
Signed-off-by: Yang Zhong <yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-42-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Yang Zhong [Wed, 23 Jan 2019 06:56:14 +0000 (14:56 +0800)]
hyperv: express dependencies with kconfig
remove default-configs/hyperv.mak and make dependencies
with Kconfig.
Signed-off-by: Yang Zhong <yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-41-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Yang Zhong [Wed, 23 Jan 2019 06:56:13 +0000 (14:56 +0800)]
display: express dependencies with kconfig
Signed-off-by: Yang Zhong <yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-40-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:12 +0000 (14:56 +0800)]
ptimer: express dependencies with Kconfig
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-39-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:11 +0000 (14:56 +0800)]
i2c: express dependencies with Kconfig
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com> Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-38-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:10 +0000 (14:56 +0800)]
i386: express dependencies with Kconfig
This way, the default-configs file only need to specify the boards
and any optional devices.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-37-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:09 +0000 (14:56 +0800)]
isa: express dependencies with kconfig
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-36-yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:07 +0000 (14:56 +0800)]
scsi: express dependencies with Kconfig
This automatically removes the SCSI subsystem from the
binary altogether if no controllers are selected.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-34-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:06 +0000 (14:56 +0800)]
build: convert usb.mak to Kconfig
Instead of including the same list of devices for each target,
let the host controllers select CONFIG_USB and make the devices
default to present whenever USB is available.
Done with the following script:
while read i; do
i=${i%=y}; i=${i#CONFIG_}
sed -i -e'/^config '$i'$/!b' -en \
-e'a\' -e' default y\' -e' depends on USB' \
`grep -lw $i hw/*/Kconfig`
done < default-configs/usb.mak
followed by adding "select USB" on the host controllers.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-33-yang.zhong@intel.com> Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:05 +0000 (14:56 +0800)]
build: convert sound.mak to Kconfig
There is really nothing special in these devices; they are just
ISA devices. Instead of including them for each target,
set CONFIG_ISA_BUS to true, and make the devices default to present
whenever ISA is available. More conversion of ISA devices will
follow.
Done with the following script:
while read i; do
i=${i%=y}; i=${i#CONFIG_}
sed -i -e'/^config '$i'$/!b' -en \
-e'a\' -e' default y\' -e' depends on ISA_BUS' \
`grep -lw $i hw/*/Kconfig`
done < default-configs/sound.mak
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-32-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:04 +0000 (14:56 +0800)]
build: convert pci.mak to Kconfig
Instead of including the same list of devices for each target,
set CONFIG_PCI to true, and make the devices default to present
whenever PCI is available. However, s390x does not want all the
PCI devices, so there is a separate symbol to enable them.
Done mostly with the following script:
while read i; do
i=${i%=y}; i=${i#CONFIG_}
sed -i -e'/^config '$i'$/!b' -en \
-e'a\' -e' default y if PCI_DEVICES\' -e' depends on PCI' \
`grep -lw $i hw/*/Kconfig`
done < default-configs/pci.mak
followed by replacing a few "depends on" clauses with "select"
whenever the symbol is not really related to PCI.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com> Cc: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-31-yang.zhong@intel.com> Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Yang Zhong [Wed, 23 Jan 2019 06:56:03 +0000 (14:56 +0800)]
hw/pci/Makefile.objs: make pcie configurable
Make pcie splited from pci and make it configurable.
Signed-off-by: Yang Zhong <yang.zhong@intel.com> Cc: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-30-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:02 +0000 (14:56 +0800)]
ide: express dependencies with Kconfig
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-29-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Fri, 25 Jan 2019 05:48:56 +0000 (06:48 +0100)]
kconfig: introduce CONFIG_TEST_DEVICES
Devices that are mostly used for testing purposes (for example in
endianness-test) will be moved under a new symbol CONFIG_TEST_DEVICES
that can be disabled in the default-configs file. This makes
it easier to drop this code from QEMU if desirable.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:17 +0000 (14:56 +0800)]
minikconfig: implement allnoconfig and defconfig modes
Apart from defconfig (which is a no-op),
allyesconfig/allnoconfig/randcondfig can be implemented simply by ignoring
the RHS of assignments and "default" statements. The RHS is replaced
respectively by "true", "false" or a random value.
However, allyesconfig and randconfig do not quite work, because all the
files for hw/ARCH/Kconfig are sourced and therefore you could end up
enabling some ARM boards in x86 or things like that. This is left for
future work, but I am leaving it in to help debugging minikconf itself.
allnoconfig mode is tied to a new configure option, --without-default-devices.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:01 +0000 (14:56 +0800)]
build: switch to Kconfig
The make_device_config.sh script is replaced by minikconf, which
is modified to support the same command line as its predecessor.
The roots of the parsing are default-configs/*.mak, Kconfig.host and
hw/Kconfig. One difference with make_device_config.sh is that all symbols
have to be defined in a Kconfig file, including those coming from the
configure script. This is the reason for the Kconfig.host file introduced
in the previous patch. Whenever a file in default-configs/*.mak used
$(...) to refer to a config-host.mak symbol, this is replaced by a
Kconfig dependency; this part must be done already in this patch
for bisectability.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com> Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-28-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:56:00 +0000 (14:56 +0800)]
kconfig: introduce kconfig files
The Kconfig files were generated mostly with this script:
for i in `grep -ho CONFIG_[A-Z0-9_]* default-configs/* | sort -u`; do
set fnord `git grep -lw $i -- 'hw/*/Makefile.objs' `
shift
if test $# = 1; then
cat >> $(dirname $1)/Kconfig << EOF
config ${i#CONFIG_}
bool
EOF
git add $(dirname $1)/Kconfig
else
echo $i $*
fi
done
sed -i '$d' hw/*/Kconfig
for i in hw/*; do
if test -d $i && ! test -f $i/Kconfig; then
touch $i/Kconfig
git add $i/Kconfig
fi
done
Whenever a symbol is referenced from multiple subdirectories, the
script prints the list of directories that reference the symbol.
These symbols have to be added manually to the Kconfig files.
Kconfig.host and hw/Kconfig were created manually.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20190123065618.3520-27-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Yang Zhong [Wed, 23 Jan 2019 06:55:59 +0000 (14:55 +0800)]
hw/display: make edid configurable
Use CONFIG_EDID to make edid-generate.c and edid-region.c
configurable.
Signed-off-by: Yang Zhong <yang.zhong@intel.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190123065618.3520-26-yang.zhong@intel.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:55:58 +0000 (14:55 +0800)]
minikconfig: add semantic analysis
There are three parts in the semantic analysis:
1) evaluating expressions. This is done as a simple visit
of the Expr nodes.
2) ordering clauses. This is done by constructing a graph of variables.
There is an edge from X to Y if Y depends on X, if X selects Y, or if
X appears in a conditional selection of Y; in other words, if the value
of X can affect the value of Y. Each clause has a "destination" variable
whose value can be affected by the clause, and clauses will be processed
according to a topological sorting of their destination variables.
Defaults are processed after all other clauses with the same destination.
3) deriving the value of the variables. This is done by processing
the clauses in the topological order provided by the previous step.
A "depends on" clause will force a variable to False, a "select" clause
will force a variable to True, an assignment will force a variable
to its RHS. A default will set a variable to its RHS if it has not
been set before. Because all variables have a default, after visiting
all clauses all variables will have been set.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190123065618.3520-25-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:55:57 +0000 (14:55 +0800)]
minikconfig: add AST
Add Python classes that represent the Kconfig abstract syntax tree.
The abstract syntax tree is stored as a list of clauses. For example:
config FOO
depends on BAR
select BAZ
is represented as three clauses:
FOO depends on BAR
FOO default n
select BAZ if FOO
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190123065618.3520-24-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo Bonzini [Wed, 23 Jan 2019 06:55:56 +0000 (14:55 +0800)]
minikconfig: add parser skeleton
This implements a scanner and recursive descent parser for Kconfig-like
configuration files. The only "action" of the parser is for now to
detect undefined variables and process include files.
The main differences between Kconfig and this are:
* only the "bool" type is supported
* variables can only be defined once
* choices are not supported (but they could be added as syntactic
sugar for multiple Boolean values)
* menus and other graphical concepts (prompts, help text) are not
supported
* assignments ("CONFIG_FOO=y", "CONFIG_FOO=n") are parsed as part
of the Kconfig language, not as a separate file.
The idea was originally by Ákos Kovács, but I could not find his
implementation so I had to redo it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190123065618.3520-23-yang.zhong@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Peter Maydell [Thu, 7 Mar 2019 16:16:02 +0000 (16:16 +0000)]
Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging
Python queue, 2019-02-22
Python:
* introduce "python" directory with module namespace
* log QEMU launch command line on qemu.QEMUMachine
Acceptance Tests:
* initrd 4GiB+ test
* migration test
* multi vm support in test class
* bump Avocado version and drop ":avocado: enable"
# gpg: Signature made Fri 22 Feb 2019 19:37:07 GMT
# gpg: using RSA key 657E8D33A5F209F3
# gpg: Good signature from "Cleber Rosa <crosa@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 7ABB 96EB 8B46 B94D 5E0F E9BB 657E 8D33 A5F2 09F3
* remotes/cleber/tags/python-next-pull-request:
Acceptance tests: expect boot to extract 2GiB+ initrd with linux-v4.16
Acceptance tests: use linux-3.6 and set vm memory to 4GiB
tests.acceptance: adds simple migration test
tests.acceptance: adds multi vm capability for acceptance tests
scripts/qemu.py: log QEMU launch command line
Introduce a Python module structure
Acceptance tests: drop usage of ":avocado: enable"
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
* remotes/pmaydell/tags/pull-sphinx-20190307:
MAINTAINERS: Add entry for Sphinx documentation infrastructure
docs/conf.py: Don't hard-code QEMU version
Makefile: Abstract out "identify the pkgversion" code
Makefile, configure: Support building rST documentation
docs: Provide separate conf.py for each manual we want
docs/conf.py: Disable option warnings
docs/conf.py: Don't include rST sources in HTML build
docs/conf.py: Configure the 'alabaster' theme
docs/conf.py: Disable unused _static directory
docs: Commit initial files from sphinx-quickstart
docs: Convert memory.txt to rst format
docs/cpu-hotplug.rst: Fix rST markup issues
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Peter Maydell [Thu, 7 Mar 2019 14:26:47 +0000 (14:26 +0000)]
MAINTAINERS: Add entry for Sphinx documentation infrastructure
Add a MAINTAINERS entry for Sphinx documentation infrastructure:
this doesn't cover actual content, only the machinery we use to
build the docs.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-13-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:47 +0000 (14:26 +0000)]
docs/conf.py: Don't hard-code QEMU version
Don't hard-code the QEMU version number into conf.py. Instead
we either pass it to sphinx-build on the command line, or
(if doing a standalone Sphinx run in a readthedocs.org setup)
extract it from the VERSION file.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-12-peter.maydell@linaro.org
Message-id: 20190228145624.24885-12-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:47 +0000 (14:26 +0000)]
Makefile: Abstract out "identify the pkgversion" code
Abstract out the "identify the pkgversion" code from the
rule for creating qemu-version.h, so it sets makefile
variables for QEMU_PKGVERSION and QEMU_FULL_VERSION.
(We will want to use these when building the Sphinx docs.)
NB: As we abstract this out, we use -e to check for .git
rather than -d, since in some situations .git may be a file
rather than a directory.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-11-peter.maydell@linaro.org
Message-id: 20190228145624.24885-11-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:46 +0000 (14:26 +0000)]
Makefile, configure: Support building rST documentation
Add support to our configure and makefile machinery for building
our rST docs into HTML files.
Building the documentation now requires that sphinx-build is
available; this seems better than allowing half the docs to
be built if it is not present but having half of them missing.
(In particular it means that assuming that distros configured with
--enable-docs they'll get a helpful error from configure telling
them the new build dependency.)
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-10-peter.maydell@linaro.org
Message-id: 20190228145624.24885-10-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:46 +0000 (14:26 +0000)]
docs: Provide separate conf.py for each manual we want
By default Sphinx wants to build a single manual at once.
For QEMU, this doesn't suit us, because we want to have
separate manuals for "Developer's Guide", "User Manual",
and so on, and we don't want to ship the Developer's Guide
to end-users. However, we don't want to completely duplicate
conf.py for each manual, and we'd like to continue to
support "build all docs in one run" for third-party sites
like readthedocs.org.
Make the top-level conf.py support two usage forms:
(1) as a common config file which is included by the conf.py
for each of QEMU's manuals: in this case sphinx-build is run
multiple times, once per subdirectory.
(2) as a top level conf file which will result in building all
the manuals into a single document: in this case sphinx-build is
run once, on the top-level docs directory.
Provide per-manual conf.py files and top level pages for
our first two manuals:
* QEMU Developer's Guide (docs/devel)
* QEMU System Emulation Management and Interoperability Guide
(docs/interop)
Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-id: 20190305172139.32662-9-peter.maydell@linaro.org
Message-id: 20190228145624.24885-9-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:46 +0000 (14:26 +0000)]
docs/conf.py: Disable option warnings
sphinx-build complains about using :option: to mark up option
flags that it doesn't know about (because they were not defined
using the "option::" directive):
docs/pr-manager.rst:68: WARNING: unknown option: -d
Suppress these warnings. This way we get the semantic markup
of the option flag but no cross-referencing hyperlink.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-8-peter.maydell@linaro.org
Message-id: 20190228145624.24885-8-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:46 +0000 (14:26 +0000)]
docs/conf.py: Don't include rST sources in HTML build
Sphinx defaults to including all the rST source files
in the HTML build and making each HTML page link to the
source file. Disable that.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-7-peter.maydell@linaro.org
Message-id: 20190228145624.24885-7-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:45 +0000 (14:26 +0000)]
docs/conf.py: Configure the 'alabaster' theme
Add the 'navigation' bar to the sidebar, which for some
reason is not enabled by default. Remove 'relations', which
is effectively disabled anyway and isn't useful for us.
This requires that we mandate having at least Sphinx 1.3,
where the theme was added.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Message-id: 20190305172139.32662-6-peter.maydell@linaro.org
Message-id: 20190228145624.24885-6-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:45 +0000 (14:26 +0000)]
docs/conf.py: Disable unused _static directory
We don't yet have any custom static files, so disable this
config file setting to avoid a warning from sphinx about
not being able to find the directory.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-5-peter.maydell@linaro.org
Message-id: 20190228145624.24885-5-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:45 +0000 (14:26 +0000)]
docs: Commit initial files from sphinx-quickstart
Commit the initial Sphinx conf.py and skeleton index.rst as
generated with sphinx-quickstart. We'll update these to
add QEMU-specific tweaks in subsequent commits.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20190305172139.32662-4-peter.maydell@linaro.org
Message-id: 20190228145624.24885-4-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:44 +0000 (14:26 +0000)]
docs: Convert memory.txt to rst format
Convert the memory API documentation from plain text
to restructured text format.
This is a very minimal conversion: all I had to change
was to mark up the ASCII art parts as Sphinx expects
for 'literal blocks', and fix up the bulleted lists
(Sphinx expects no leading space before the bullet, and
wants a blank line before after any list).
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20190305172139.32662-3-peter.maydell@linaro.org
Message-id: 20190228145624.24885-3-peter.maydell@linaro.org
Peter Maydell [Thu, 7 Mar 2019 14:26:44 +0000 (14:26 +0000)]
docs/cpu-hotplug.rst: Fix rST markup issues
sphinx-build complains:
docs/cpu-hotplug.rst:67: ERROR: Unexpected indentation.
docs/cpu-hotplug.rst:69: ERROR: Unexpected indentation.
docs/cpu-hotplug.rst:74: WARNING: Block quote ends without a blank line; unexpected unindent.
docs/cpu-hotplug.rst:75: WARNING: Block quote ends without a blank line; unexpected unindent.
docs/cpu-hotplug.rst:76: SEVERE: Unexpected section title.
}
{
docs/cpu-hotplug.rst:78: WARNING: Block quote ends without a blank line; unexpected unindent.
These are the result of not indicating one of the literal
blocks by finishing the preceding paragraph with the "::" marker.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20190305172139.32662-2-peter.maydell@linaro.org
Message-id: 20190228145624.24885-2-peter.maydell@linaro.org
* remotes/kraxel/tags/usb-20190307-pull-request:
Introduce new "no_guest_reset" parameter for usb-host device
usb-mtp: prevent null dereference while deleting objects
usb-mtp: fix some usb_mtp_write_data return paths
usb-mtp: return incomplete transfer on a lstat failure
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Introduce new "no_guest_reset" parameter for usb-host device
With certain USB devices passed through via usb-host, a guest attempting to
reset a usb-host device can trigger a reset loop that renders the USB device
unusable. In my use case, the device was an iPhone XR that was passed through to
a Mac OS X Mojave guest. Upon connecting the device, the following happens:
1) Guest recognizes new device, sends reset to emulated USB host
2) QEMU's USB host sends reset to host kernel
3) Host kernel resets device
4) After reset, host kernel determines that some part of the device descriptor
has changed ("device firmware changed" in dmesg), so host kernel decides to
re-enumerate the device.
5) Re-enumeration causes QEMU to disconnect and reconnect the device in the
guest.
6) goto 1)
Here's from the host kernel (note the "device firmware changed" lines")
[3677704.473050] usb 1-1.3: new high-speed USB device number 53 using ehci-pci
[3677704.555594] usb 1-1.3: New USB device found, idVendor=05ac, idProduct=12a8, bcdDevice=11.08
[3677704.555599] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[3677704.555602] usb 1-1.3: Product: iPhone
[3677704.555605] usb 1-1.3: Manufacturer: Apple Inc.
[3677704.555607] usb 1-1.3: SerialNumber: [[removed]]
[3677709.401040] usb 1-1.3: reset high-speed USB device number 53 using ehci-pci
[3677709.479486] usb 1-1.3: device firmware changed
[3677709.479842] usb 1-1.3: USB disconnect, device number 53
[3677709.546039] usb 1-1.3: new high-speed USB device number 54 using ehci-pci
[3677709.627471] usb 1-1.3: New USB device found, idVendor=05ac, idProduct=12a8, bcdDevice=11.08
[3677709.627476] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[3677709.627479] usb 1-1.3: Product: iPhone
[3677709.627481] usb 1-1.3: Manufacturer: Apple Inc.
[3677709.627483] usb 1-1.3: SerialNumber: [[removed]]
[3677762.320044] usb 1-1.3: reset high-speed USB device number 54 using ehci-pci
[3677762.615630] usb 1-1.3: USB disconnect, device number 54
[3677762.787043] usb 1-1.3: new high-speed USB device number 55 using ehci-pci
[3677762.869016] usb 1-1.3: New USB device found, idVendor=05ac, idProduct=12a8, bcdDevice=11.08
[3677762.869024] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[3677762.869028] usb 1-1.3: Product: iPhone
[3677762.869032] usb 1-1.3: Manufacturer: Apple Inc.
[3677762.869035] usb 1-1.3: SerialNumber: [[removed]]
[3677815.662036] usb 1-1.3: reset high-speed USB device number 55 using ehci-pci
Here's from QEMU:
libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/005/022: No such file or directory
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [_open_sysfs_attr] open /sys/bus/usb/devices/5-1/bConfigurationValue failed ret=-1 errno=2
libusb: error [_get_usbfs_fd] File doesn't exist, wait 10 ms and try again
libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/005/024: No such file or directory
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [udev_hotplug_event] ignoring udev action bind
libusb: error [_open_sysfs_attr] open /sys/bus/usb/devices/5-1/bConfigurationValue failed ret=-1 errno=2
libusb: error [_get_usbfs_fd] File doesn't exist, wait 10 ms and try again
libusb: error [_get_usbfs_fd] libusb couldn't open USB device /dev/bus/usb/005/026: No such file or directory
The result of this is that the device remains permanently unusable in the guest.
The same problem has been previously reported for an iPad:
https://stackoverflow.com/questions/52617634/how-do-i-get-qemu-usb-passthrough-to-work-for-ipad-iphone
This problem can be elegantly solved by interrupting step 2) above. Instead of
passing through the reset, QEMU simply ignores it. To allow this to be
configured on a per-device level, a new parameter "no_guest_reset" is
introduced for the usb-host device. I can confirm that the configuration
described above (iPhone XS + Mojave guest) works flawlessly with
no_guest_reset=True specified.
Working command line for my scenario:
device_add usb-host,vendorid=0x05ac,productid=0x12a8,no_guest_reset=True,id=iphone
Best regards
Alexander
Signed-off-by: Alexander Kappner <agk@godking.net> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-id: 20190128140027.9448-1-kraxel@redhat.com
Bandan Das [Wed, 6 Mar 2019 21:04:07 +0000 (16:04 -0500)]
usb-mtp: return incomplete transfer on a lstat failure
MTP writes objects in small chunks and at the end gets the
real file size to update the object metadata. If this fails for
any reason, return an INCOMPLETE_TRANSFER to the initiator
# gpg: Signature made Wed 06 Mar 2019 18:39:29 GMT
# gpg: using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6
* remotes/ehabkost/tags/machine-next-pull-request:
qdev: Provide qdev_get_bus_hotplug_handler()
qdev: Let machine hotplug handler to override bus hotplug handler
qdev: Let the hotplug_handler_unplug() caller delete the device
hostmem: fix crash when querying empty host-nodes property via QMP
qdev/core: fix qbus_is_full()
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Let's use a wrapper instead of looking it up manually. This function can
than be reused when we explicitly want to have the bus hotplug handler
(e.g. when the bus hotplug handler was overwritten by the machine
hotplug handler).
Reviewed-by: Igor Mammedov <imammedo@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-4-david@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Igor Mammedov [Thu, 28 Feb 2019 12:28:48 +0000 (13:28 +0100)]
qdev: Let machine hotplug handler to override bus hotplug handler
it will allow to return another hotplug handler than the default
one for a specific bus based device type. Which is needed to handle
non trivial plug/unplug sequences that need the access to resources
configured outside of bus where device is attached.
That will allow for returned hotplug handler to orchestrate wiring
in arbitrary order, by chaining other hotplug handlers when
it's needed.
PS:
It could be used for hybrid virtio-mem and virtio-pmem devices
where it will return machine as hotplug handler which will do
necessary wiring at machine level and then pass control down
the chain to bus specific hotplug handler.
Example of top level hotplug handler override and custom plug sequence:
some_machine_device_plug(hotplug_dev, dev) {
if (object_dynamic_cast(OBJECT(dev), TYPE_SOME_BUS_DEVICE)) {
/* do machine specific initialization */
some_machine_init_special_device(dev)
/* pass control to bus specific handler */
hotplug_handler_plug(dev->parent_bus->hotplug_handler, dev)
}
}
Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Signed-off-by: Igor Mammedov <imammedo@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-3-david@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
qdev: Let the hotplug_handler_unplug() caller delete the device
When unplugging a device, at one point the device will be destroyed
via object_unparent(). This will, one the one hand, unrealize the
removed device hierarchy, and on the other hand, destroy/free the
device hierarchy.
When chaining hotplug handlers, we want to overwrite a bus hotplug
handler by the machine hotplug handler, to be able to perform
some part of the plug/unplug and to forward the calls to the bus hotplug
handler.
For now, the bus hotplug handler would trigger an object_unparent(), not
allowing us to perform some unplug action on a device after we forwarded
the call to the bus hotplug handler. The device would be gone at that
point.
machine_unplug_handler(dev)
/* eventually do unplug stuff */
bus_unplug_handler(dev)
/* dev is gone, we can't do more unplug stuff */
So move the object_unparent() to the original caller of the unplug. For
now, keep the unrealize() at the original places of the
object_unparent(). For implicitly chained hotplug handlers (e.g. pc
code calling acpi hotplug handlers), the object_unparent() has to be
done by the outermost caller. So when calling hotplug_handler_unplug()
from inside an unplug handler, nothing is to be done.
hotplug_handler_unplug(dev) -> calls machine_unplug_handler()
machine_unplug_handler(dev) {
/* eventually do unplug stuff */
bus_unplug_handler(dev) -> calls unrealize(dev)
/* we can do more unplug stuff but device already unrealized */
}
object_unparent(dev)
In the long run, every unplug action should be factored out of the
unrealize() function into the unplug handler (especially for PCI). Then
we can get rid of the additonal unrealize() calls and object_unparent()
will properly unrealize the device hierarchy after the device has been
unplugged.
hotplug_handler_unplug(dev) -> calls machine_unplug_handler()
machine_unplug_handler(dev) {
/* eventually do unplug stuff */
bus_unplug_handler(dev) -> only unplugs, does not unrealize
/* we can do more unplug stuff */
}
object_unparent(dev) -> will unrealize
The original approach was suggested by Igor Mammedov for the PCI
part, but I extended it to all hotplug handlers. I consider this one
step into the right direction.
To summarize:
- object_unparent() on synchronous unplugs is done by common code
-- "Caller of hotplug_handler_unplug"
- object_unparent() on asynchronous unplugs ("unplug requests") has to
be done manually
-- "Caller of hotplug_handler_unplug"
Reviewed-by: Igor Mammedov <imammedo@redhat.com> Acked-by: Cornelia Huck <cohuck@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20190228122849.4296-2-david@redhat.com> Reviewed-by: Greg Kurz <groug@kaod.org> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Igor Mammedov [Thu, 14 Feb 2019 10:57:33 +0000 (05:57 -0500)]
hostmem: fix crash when querying empty host-nodes property via QMP
QEMU will crashes with
qapi/qobject-output-visitor.c:210: qobject_output_complete: Assertion `qov->root && ((&qov->stack)->slh_first == ((void *)0))' failed
when trying to get value of not set hostmem's "host-nodes"
property, HostMemoryBackend::host_nodes bitmap doesn't have
any bits set in it, which leads to find_first_bit() returning
MAX_NODES and consequently to an early return from
host_memory_backend_get_host_nodes() without calling visitor.
Fix it by calling visitor even if "host-nodes" property wasn't
set before exiting from property getter to return valid empty
list.
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Message-Id: <20190214105733.25643-1-imammedo@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Tony Krowiak [Mon, 17 Dec 2018 15:57:30 +0000 (10:57 -0500)]
qdev/core: fix qbus_is_full()
The qbus_is_full(BusState *bus) function (qdev_monitor.c) compares the max_index
value of the BusState structure with the max_dev value of the BusClass structure
to determine whether the maximum number of children has been reached for the
bus. The problem is, the max_index field of the BusState structure does not
necessarily reflect the number of devices that have been plugged into
the bus.
Whenever a child device is plugged into the bus, the bus's max_index value is
assigned to the child device and then incremented. If the child is subsequently
unplugged, the value of the max_index does not change and no longer reflects the
number of children.
When the bus's max_index value reaches the maximum number of devices
allowed for the bus (i.e., the max_dev field in the BusClass structure),
attempts to plug another device will be rejected claiming that the bus is
full -- even if the bus is actually empty.
To resolve the problem, a new 'num_children' field is being added to the
BusState structure to keep track of the number of children plugged into the
bus. It will be incremented when a child is plugged, and decremented when a
child is unplugged.
Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com> Reviewed-by: Pierre Morel<pmorel@linux.ibm.com> Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
Message-Id: <1545062250-7573-1-git-send-email-akrowiak@linux.ibm.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Peter Maydell [Wed, 6 Mar 2019 14:50:33 +0000 (14:50 +0000)]
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20190306a' into staging
Migation pull 2019-03-06
(This replaces the pull sent yesterday)
a) 4 small fixes including the cancel problem
that caused the ahci migration test to fail
intermittently
b) Yury's ignore-shared feature
c) Juan's extra tests
d) Wei Wang's free page hinting
e) Some Colo fixes from Zhang Chen
Diff from yesterdays pull:
1) A missing fix of mine (cleanup during exit)
2) Changes from Eric/Markus on 'Create socket-address parameter'
# gpg: Signature made Wed 06 Mar 2019 11:39:53 GMT
# gpg: using RSA key 0516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20190306a: (22 commits)
qapi/migration.json: Remove a variable that doesn't exist in example
Migration/colo.c: Make COLO node running after failover
Migration/colo.c: Fix double close bug when occur COLO failover
virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
migration/ram.c: add the free page optimization enable flag
migration/ram.c: add a notifier chain for precopy
migration: API to clear bits of guest free pages from the dirty bitmap
migration: use bitmap_mutex in migration_bitmap_clear_dirty
bitmap: bitmap_count_one_with_offset
bitmap: fix bitmap_count_one
tests: Add basic migration precopy tcp test
migration: Create socket-address parameter
tests: Add migration xbzrle test
migration: Add capabilities validation
tests/migration-test: Add a test for ignore-shared capability
migration: Add an ability to ignore shared RAM blocks
migration: Introduce ignore-shared capability
exec: Change RAMBlockIterFunc definition
migration/rdma: clang compilation fix
migration: Cleanup during exit
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
* remotes/vivier2/tags/trivial-branch-pull-request:
thunk: fix of malloc to g_new
hostmem-file: simplify ifdef-s in file_backend_memory_alloc()
build: Correct explanation of unnest-vars example
bswap: Fix accessors syntax in comment
doc: fix typos for documents in tree
block/pflash_cfi02: Fix memory leak and potential use-after-free
hw/acpi: remove unnecessary variable acpi_table_builtin
hw/acpi: remove unused function acpi_table_add_builtin()
hw/i386/pc.c: remove unused function pc_acpi_init()
tests: Remove (mostly) useless architecture checks
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Zhang Chen [Sun, 3 Mar 2019 14:50:19 +0000 (22:50 +0800)]
qapi/migration.json: Remove a variable that doesn't exist in example
Remove the "active" variable in example for query-colo-status.
It is a doc bug from commit f56c0065
Signed-off-by: Zhang Chen <chen.zhang@intel.com> Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190303145021.2962-6-chen.zhang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Zhang Chen [Sun, 3 Mar 2019 14:50:17 +0000 (22:50 +0800)]
Migration/colo.c: Make COLO node running after failover
Delay to close COLO for auto start VM after failover.
Signed-off-by: Zhang Chen <chen.zhang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190303145021.2962-4-chen.zhang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Zhang Chen [Sun, 3 Mar 2019 14:50:15 +0000 (22:50 +0800)]
Migration/colo.c: Fix double close bug when occur COLO failover
In migration_incoming_state_destroy(void) will check the mis->to_src_file
to double close the mis->to_src_file when occur COLO failover.
Signed-off-by: Zhang Chen <chen.zhang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190303145021.2962-2-chen.zhang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Wei Wang [Tue, 11 Dec 2018 08:24:53 +0000 (16:24 +0800)]
virtio-balloon: VIRTIO_BALLOON_F_FREE_PAGE_HINT
The new feature enables the virtio-balloon device to receive hints of
guest free pages from the free page vq.
A notifier is registered to the migration precopy notifier chain. The
notifier calls free_page_start after the migration thread syncs the dirty
bitmap, so that the free page optimization starts to clear bits of free
pages from the bitmap. It calls the free_page_stop before the migration
thread syncs the bitmap, which is the end of the current round of ram
save. The free_page_stop is also called to stop the optimization in the
case when there is an error occurred in the process of ram saving.
Note: balloon will report pages which were free at the time of this call.
As the reporting happens asynchronously, dirty bit logging must be
enabled before this free_page_start call is made. Guest reporting must be
disabled before the migration dirty bitmap is synchronized.
Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Juan Quintela <quintela@redhat.com> CC: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-8-git-send-email-wei.w.wang@intel.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Dropped kernel header update, fixed up CMD_ID_* name change
Wei Wang [Tue, 11 Dec 2018 08:24:52 +0000 (16:24 +0800)]
migration/ram.c: add the free page optimization enable flag
This patch adds the free page optimization enable flag, and a function
to set this flag. When the free page optimization is enabled, not all
the pages are needed to be sent in the bulk stage.
Why using a new flag, instead of directly disabling ram_bulk_stage when
the optimization is running?
Thanks for Peter Xu's reminder that disabling ram_bulk_stage will affect
the use of compression. Please see save_page_use_compression. When
xbzrle and compression are used, if free page optimizaion causes the
ram_bulk_stage to be disabled, save_page_use_compression will return
false, which disables the use of compression. That is, if free page
optimization avoids the sending of half of the guest pages, the other
half of pages loses the benefits of compression in the meantime. Using a
new flag to let migration_bitmap_find_dirty skip the free pages in the
bulk stage will avoid the above issue.
Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Juan Quintela <quintela@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-7-git-send-email-wei.w.wang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Wei Wang [Tue, 11 Dec 2018 08:24:51 +0000 (16:24 +0800)]
migration/ram.c: add a notifier chain for precopy
This patch adds a notifier chain for the memory precopy. This enables various
precopy optimizations to be invoked at specific places.
Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Juan Quintela <quintela@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Peter Xu <peterx@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-6-git-send-email-wei.w.wang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Wei Wang [Tue, 11 Dec 2018 08:24:50 +0000 (16:24 +0800)]
migration: API to clear bits of guest free pages from the dirty bitmap
This patch adds an API to clear bits corresponding to guest free pages
from the dirty bitmap. Spilt the free page block if it crosses the QEMU
RAMBlock boundary.
Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Juan Quintela <quintela@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Peter Xu <peterx@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-5-git-send-email-wei.w.wang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Wei Wang [Tue, 11 Dec 2018 08:24:49 +0000 (16:24 +0800)]
migration: use bitmap_mutex in migration_bitmap_clear_dirty
The bitmap mutex is used to synchronize threads to update the dirty
bitmap and the migration_dirty_pages counter. For example, the free
page optimization clears bits of free pages from the bitmap in an
iothread context. This patch makes migration_bitmap_clear_dirty update
the bitmap and counter under the mutex.
Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Juan Quintela <quintela@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Peter Xu <peterx@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-4-git-send-email-wei.w.wang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Wei Wang [Tue, 11 Dec 2018 08:24:48 +0000 (16:24 +0800)]
bitmap: bitmap_count_one_with_offset
Count the number of 1s in a bitmap starting from an offset.
Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Juan Quintela <quintela@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1544516693-5395-3-git-send-email-wei.w.wang@intel.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Wei Wang [Tue, 11 Dec 2018 08:24:47 +0000 (16:24 +0800)]
bitmap: fix bitmap_count_one
BITMAP_LAST_WORD_MASK(nbits) returns 0xffffffff when "nbits=0", which
makes bitmap_count_one fail to handle the "nbits=0" case. It appears to be
preferred to remain BITMAP_LAST_WORD_MASK identical to the kernel
implementation that it is ported from.
So this patch fixes bitmap_count_one to handle the nbits=0 case.
Inital Discussion Link:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg554316.html Signed-off-by: Wei Wang <wei.w.wang@intel.com> CC: Juan Quintela <quintela@redhat.com> CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Peter Xu <peterx@redhat.com>
Message-Id: <1544516693-5395-2-git-send-email-wei.w.wang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Juan Quintela [Wed, 27 Feb 2019 10:51:28 +0000 (11:51 +0100)]
tests: Add basic migration precopy tcp test
Not sharing code from precopy/unix because we have to read back the
tcp parameter.
Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190227105128.1655-4-quintela@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Fixup for clash with Yury's
Juan Quintela [Wed, 27 Feb 2019 10:51:27 +0000 (11:51 +0100)]
migration: Create socket-address parameter
It will be used to store the uri parameters. We want this only for
tcp, so we don't set it for other uris. We need it to know what port
is migration running.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Removed DummyStruct as suggested by Eric & Markus
Juan Quintela [Wed, 27 Feb 2019 10:51:26 +0000 (11:51 +0100)]
tests: Add migration xbzrle test
Reviewed-by: Peter Xu <peterx@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20190227105128.1655-2-quintela@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Fixup for class with Yury's series
Yury Kotov [Fri, 15 Feb 2019 17:45:48 +0000 (20:45 +0300)]
migration: Add capabilities validation
Currently we don't check which capabilities set in the source QEMU.
We just expect that the target QEMU has the same enabled capabilities.
Add explicit validation for capabilities to make sure that the target VM
has them too. This is enabled for only new capabilities to keep compatibily.
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-6-yury-kotov@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Manual merge
Yury Kotov [Fri, 15 Feb 2019 17:45:47 +0000 (20:45 +0300)]
tests/migration-test: Add a test for ignore-shared capability
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-5-yury-kotov@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
dgilbert: Disabled the test for now, not happy on aarch64
Yury Kotov [Fri, 15 Feb 2019 17:45:46 +0000 (20:45 +0300)]
migration: Add an ability to ignore shared RAM blocks
If ignore-shared capability is set then skip shared RAMBlocks during the
RAM migration.
Also, move qemu_ram_foreach_migratable_block (and rename) to the
migration code, because it requires access to the migration capabilities.
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-4-yury-kotov@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Yury Kotov [Fri, 15 Feb 2019 17:45:45 +0000 (20:45 +0300)]
migration: Introduce ignore-shared capability
We want to use local migration to update QEMU for running guests.
In this case we don't need to migrate shared (file backed) RAM.
So, add a capability to ignore such blocks during live migration.
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-3-yury-kotov@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Yury Kotov [Fri, 15 Feb 2019 17:45:44 +0000 (20:45 +0300)]
exec: Change RAMBlockIterFunc definition
Currently, qemu_ram_foreach_* calls RAMBlockIterFunc with many
block-specific arguments. But often iter func needs RAMBlock*.
This refactoring is needed for fast access to RAMBlock flags from
qemu_ram_foreach_block's callback. The only way to achieve this now
is to call qemu_ram_block_from_host (which also enumerates blocks).
So, this patch reduces complexity of
qemu_ram_foreach_block() -> cb() -> qemu_ram_block_from_host()
from O(n^2) to O(n).
Fix RAMBlockIterFunc definition and add some functions to read
RAMBlock* fields witch were passed.
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Message-Id: <20190215174548.2630-2-yury-kotov@yandex-team.ru> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC migration/rdma.o
CC migration/block.o
qemu/migration/rdma.c:3615:58: error: taking address of packed member 'rkey' of class or structure
'RDMARegisterResult' may result in an unaligned pointer value [-Werror,-Waddress-of-packed-member]
(uintptr_t)host_addr, NULL, ®_result->rkey,
^~~~~~~~~~~~~~~~
Fix it by using a temp local variable.
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
Message-Id: <20190304184923.24215-1-marcel.apfelbaum@gmail.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Currently we cleanup the migration object as we exit main after the
main_loop finishes; however if there's a migration running things
get messy and we can end up with the migration thread still trying
to access freed structures.
We now take a ref to the object around the migration thread itself,
so the act of dropping the ref during exit doesn't cause us to lose
the state until the thread quits.
Cancelling the migration during migration also tries to get the thread
to quit.
We do this a bit earlier; so hopefully migration gets out of the way
before all the devices etc are freed.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20190227164900.16378-1-dgilbert@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
If the migration fails before the channel is open (e.g. a bad
address) we end up in the cleanup with rdma->channel==NULL.
Spotted by Coverity: CID 1398634 Fixes: fbbaacab2758cb3f32a0 Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190214185351.5927-1-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
During a cancelled migration there's a race where the fd can
go into an error state before we get back around the migration loop
and migration_detect_error transitions from cancelling->failed.
Check for cancelled/cancelling and don't change the state.
Red Hat bug: https://bugzilla.redhat.com/show_bug.cgi?id=1608649
Fixes: b23c2ade250 Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190219195928.12289-1-dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com>
block/pflash_cfi02: Fix memory leak and potential use-after-free
Don't dynamically allocate the pflash's timer. But do use timer_del in
an unrealize function to make sure that the timer can't fire after the
pflash_t has been freed.
Signed-off-by: Stephen Checkoway <stephen.checkoway@oberlin.edu> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wei Yang <richardw.yang@linux.intel.com>
Message-Id: <20190219153727.62279-1-stephen.checkoway@oberlin.edu> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
projects / mirror_qemu.git / log