Mira Limbeck [Tue, 17 Mar 2020 19:56:12 +0000 (20:56 +0100)]
add unix socket support for NBD storage migration
The reuse of the tunnel, which we're opening to communicate with the target
node and to forward the unix socket for the state migration, for the NBD unix
socket requires adding support for an array of sockets to forward, not just a
single one. We also have to change the $sock_addr variable to an array
for the cleanup of the socket file as SSH does not remove the file.
To communicate to the target node the support of unix sockets for NBD
storage migration, we're specifying an nbd_protocol_version which is set
to 1. This version is then passed to the target node via STDIN. Because
we don't want to be dependent on the order of arguments being passed
via STDIN, we also prefix the spice ticket with 'spice_ticket: '. The
target side handles both the spice ticket and the nbd protocol version
with a fallback for old source nodes passing the spice ticket without a
prefix.
All arguments are line based and require a newline in between.
When the NBD server on the target node is started with a unix socket, we
get a different line containing all the information required to start
the drive-mirror. This contains the unix socket path used on the target node
which we require for forwarding and cleanup.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 17 Mar 2020 19:56:11 +0000 (20:56 +0100)]
parse nbd_protocol_version if available
For secure live migration with local disks via NBD over a unix socket,
we have to somehow communicate from the source node to the target node
if it supports it. This is because there can only be one NBD server with
exactly one socket bound.
The source node passes that information via STDIN. Support for
'spice_ticket: (...)' is added in addition to 'nbd_protocol_version:
<version>'. As old source nodes send the spice ticket without a prefix,
we still have to have a fallback for this case. New information should
always be passed via a prefix that is matched, otherwise it will be
recognized as spice ticket.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 17 Mar 2020 19:56:10 +0000 (20:56 +0100)]
add NBD server unix socket support in vm_start
As the NBD server spawned by qemu can only listen on a single socket,
we're dependent on a version being passed to vm_start that indicates
which protocol can be used, TCP or Unix, by the source node.
The change in socket type (TCP to Unix) comes with a different URI. For
unix sockets it has the form: 'nbd:unix:<path/to/socket>:exportname=<device>'.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 17 Mar 2020 19:56:09 +0000 (20:56 +0100)]
add qemu_drive_mirror_monitor completion modes
With Qemu 4.2 we encountered a problem with unix sockets and SSH socket
forwarding for drive-mirror. It seems the socket gets reopened again and
again after it closes for some reason. This can be worked around by
specifying 'block-job-cancel' instead of 'block-job-complete' when we're
not interested in swapping the disks again from NBD to their original
protocol. This is always the case when we use drive-mirror for live
migrating a VM.
qemu_drive_mirror is used for migration and for clone_disk. All in all
we have 3 cases to handle. Either the 'skip' case which skips the
completion of the job. The 'wait' case which was the default before and
still is when $completion is undefined. And the new 'wait_noswap' case
which is used for the live migration.
If 'wait_noswap' is specified, we issue a 'block-job-cancel' once the block
job is in 'ready' state. This completes the block job without swapping the
disks.
clone_disk always uses 'block-job-cancel' via the qemu_blockjobs_cancel
sub.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Fabian Ebner [Thu, 12 Mar 2020 10:19:12 +0000 (11:19 +0100)]
Change format for unused drives
and make it match with what parse_drive does. Even though the 'real' format
was pve-volume-id, callers already expected that parse_drive returns a hash
with a valid 'file' key (e.g. PVE/API2/Qemu.pm:1147ff).
Kamil Trzcinski [Mon, 9 Mar 2020 13:32:44 +0000 (14:32 +0100)]
Append newline to all QGA commands
Since the MacOS Mojave Apple ships AppleQEMUGuestAgent by default.
However, it does not fully adhere to QGA specs as they do expect each
command to be newline delimited.
This makes each command to be newline delimited which is harmless for
all other systems (Windows, Linux), but enable guest agent by default
without any changes on OSX.
Thomas Lamprecht [Tue, 10 Mar 2020 13:47:03 +0000 (14:47 +0100)]
cloudinit: make genisoimage only output errors
avoids a genisoimage output like:
> Total translation table size: 0
> Total rockridge attributes bytes: 417
> Total directory bytes: 0
> Path table size(bytes): 10
> Max brk space used 0
> 178 extents written (0 MB)
on every VM start.
Rather than that useless output, tell genisoimage to be quiet, which
still prints errors but nothing else. Additionally print a short
single line about that we're to create the cloud-init iso.
Reformat while at it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Fabian Ebner [Mon, 9 Mar 2020 12:24:50 +0000 (13:24 +0100)]
sync_disks: Always set 'snapshots' for qcow2 and vmdk volumes
This fixes an issue when migrating a VM with an unused volume with format
qcow2 or vmdk. Since 'snapshots' wasn't set, storage_migrate wanted to
export/import with format raw+size instead. Therefore it used (instead of
just 'dd') 'qemu-img convert', which fails when its output leaves through
a pipe. Upon importing, a second error is present, because the format from
the volume ID doesn't match the format of the stream and there is no
conversion yet.
Fabian Ebner [Mon, 2 Mar 2020 10:33:48 +0000 (11:33 +0100)]
Also update disk size if there was no old size
If for whatever reason there is no size in the property string
of a drive, 'qm rescan' would do nothing for that drive and
live migration would also fail.
Also adds a check to avoid potential auto-vivification of volid_hash->{$volid}
Fabian Ebner [Mon, 2 Mar 2020 10:33:44 +0000 (11:33 +0100)]
Create Drive.pm and move drive-related code there
The initialization for the drive keys in $confdesc is changed
to be a single for-loop iterating over the keys of $drivedesc_hash and
the initialization of the unusedN keys is move to directly below it.
To avoid the need to change all the call sites, functions with more than
a few callers are exported from the submodule and imported into QemuServer.pm.
For callers of the now imported functions within QemuServer.pm, the prefix
PVE::QemuServer is dropped, because it is unnecessary and now even confusing.
Fabian Ebner [Mon, 2 Mar 2020 10:33:42 +0000 (11:33 +0100)]
Replace drivename_hash with drivedesc_hash
which contains the full descriptions of the drives, and
make parse_drive not depend on $confdesc anymore.
In preparation to moving drive-related code to its own module.
Stefan Reiter [Thu, 20 Feb 2020 17:10:44 +0000 (18:10 +0100)]
fix #2264: add virtio-rng device
Allow a user to add a virtio-rng-pci (an emulated hardware random
number generator) to a VM with the rng0 setting. The setting is
version_guard()-ed.
Limit the selection of entropy source to one of three:
/dev/urandom (preferred): Non-blocking kernel entropy source
/dev/random: Blocking kernel source
/dev/hwrng: Hardware RNG on the host for passthrough
QEMU itself defaults to /dev/urandom (or the equivalent getrandom()
call) if no source file is given, but I don't fully trust that
behaviour to stay constant, considering the documentation [0] already
disagrees with the code [1], so let's always specify the file ourselves.
/dev/urandom is preferred, since it prevents host entropy starvation.
The quality of randomness is still good enough to emulate a hwrng, since
a) it's still seeded from the kernel's true entropy pool periodically
and b) it's mixed with true entropy in the guest as well.
Additionally, all sources about entropy predicition attacks I could find
mention that to predict /dev/urandom results, /dev/random has to be
accessed or manipulated in one way or the other - this is not possible
from a VM however, as the entropy we're talking about comes from the
*hosts* blocking pool.
More about the entropy and security implications of the non-blocking
interface in [2] and [3].
Note further that only one /dev/hwrng exists at any given time, if
multiple RNGs are available, only the one selected in
'/sys/devices/virtual/misc/hw_random/rng_current' will feed the file.
Selecting this is left as an exercise to the user, if at all required.
We limit the available entropy to 1 KiB/s by default, but allow the user
to override this. Interesting to note is that the limiter does not work
linearly, i.e. max_bytes=1024/period=1000 means that up to 1 KiB of data
becomes available on a 1000 millisecond timer, not that 1 KiB is
streamed to the guest over the course of one second - hence the
configurable period.
The default used here is the same as given in the QEMU documentation [0]
and has been verified to affect entropy availability in a guest by
measuring /dev/random throughput. 1 KiB/s is enough to avoid any
early-boot entropy shortages, and already has a significant impact on
/dev/random availability in the guest.
The http-server has a 64KB payload limit for post requests, so note
that explicit even if it's a theoretical maximum as the reamainig
params also need some space in the request
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Reiter [Thu, 27 Feb 2020 10:47:41 +0000 (11:47 +0100)]
fix #2612: allow input-data in guest exec and make command optional
'input-data' can be used to pass arbitrary data to a guest when running
an agent command with 'guest-exec'. Most guest-agent implementations
treat this as STDIN to the command given by "path"/"arg", but some go as
far as relying solely on this parameter, and even fail if "path" or
"arg" are set (e.g. Mikrotik Cloud Hosted Router) - thus "command" needs
to be made optional.
Via the API, an arbitrary string can be passed, on the command line ('qm
guest exec'), an additional '--pass-stdin' flag allows to forward STDIN
of the qm process to 'input-data', with a size limitation of 1 MiB to
not overwhelm QMP.
Without 'input-data' (API) or '--pass-stdin' (CLI) behaviour is unchanged.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Fabian Ebner [Wed, 19 Feb 2020 10:31:30 +0000 (11:31 +0100)]
Align size to 1 KiB bytes before doing 'qmp block_resize'
1. Avoids the error
"VM 111 qmp command 'block_resize' failed - The new size must be a multiple of 512"
for qcow2 disks.
2. Because volume_import expects disk sizes to be a multiple of 1 KiB.
Stefan Reiter [Wed, 12 Feb 2020 10:10:56 +0000 (11:10 +0100)]
Simplify QEMU version check and require 3.0+
Some of the recent QMP changes require at least 2.8.0, but since the
oldest version we officially package for 6.x is 4.0.0 anyway, checking
for at least 3.0 should not break anyone's setup.
Note that this does not affect machine version checks, only the
installed QEMU binary version.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Stefan Reiter [Mon, 10 Feb 2020 15:05:36 +0000 (16:05 +0100)]
version_guard scsi drive count
Live-migrating a VM with more than 14 SCSI disks to a node that doesn't
support it yet is broken. Use a bumped pve-version to represent that and
give the user a nice error message instead.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Stefan Reiter [Mon, 10 Feb 2020 15:05:35 +0000 (16:05 +0100)]
Use 'QEMU version' -> '+pve-version' mapping for machine types
The previously introduced approach can fail for pinned versions when a
new QEMU release is introduced. The saner approach is to use a mapping
that gives one pve-version for each QEMU release.
Fortunately, the old system has not been bumped yet, so we can still
change it without too much effort.
QEMU versions without a mapping are assumed to be pve0, 4.1 is mapped to
pve1 since thats what we had as our default previously.
Pinned machine versions (i.e. pc-i440fx-4.1) are always assumed to be
pve0, for specific pve-versions they'd have to be pinned as well (i.e.
pc-i440fx-4.1+pve1).
The new logic also makes the pve-version dynamic, and starts VMs with
the lowest possible 'feature-level', i.e. if a feature is only available
with 4.1+pve2, but the VM isn't using it, we still start it with
4.1+pve0.
We die if we don't support a version that is requested from us. This
allows us to use the pve-version as live-migration blocks (i.e. bumping
the version and then live-migrating a VM which uses the new feature (so
is running with the bumped version) to an outdated node will present the
user with a helpful error message and fail instead of silently modifying
the config and only failing *after* the migration).
$version_guard is introduced in config_to_command to use for features
that need to check pve-version, it automatically handles selecting the
newest necessary pve-version for the VM.
Tests have to be adjusted, since all of them now resolve to pve0 instead
of pve1. EXPECT_ERROR matching is changed to use 'eq' instead of regex
to allow special characters in error messages.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Stefan Reiter [Thu, 6 Feb 2020 09:53:55 +0000 (10:53 +0100)]
qmp: use query-cpus-fast for hotplug
query-cpus has been deprecated since 2.12.0 [0] in favor of
query-cpus-fast, which no longer incurs a guest performance penalty on
the guest. The returned information is the same as far as our use case
is concerned.
Stefan Reiter [Thu, 6 Feb 2020 09:53:54 +0000 (10:53 +0100)]
qmp: use migrate-set-parameters in favor of deprecated options
migrate_set_downtime, migrate_set_speed and migrate-set-cachesize have
all been deprecated since 2.8 or 2.11 [0]. They still work, but no
reason not to use the correct version.
Note that the downtime-limit parameter switched from seconds to
milliseconds, so convert to that. Slightly improve log output with units
while at it.
Dominik Csapak [Fri, 31 Jan 2020 14:41:22 +0000 (15:41 +0100)]
fix #2566: increase scsi limit to 31
to achieve this we have to add 3 new scsihw addresses since lsi
controllers can only hold 7 scsi drives
we go up to 31, since this is the limit for virtio-scsi-single devices
we have reserved (we can increase this in the future)
to make it more future proof, we add a new pci bridge under pci
bridge 1, so we have to adapt the bridge adding code (we did not
need this for q35 previously)
impact on live migration:
since on older versions of qemu-server we do not have those config
settings, there is no problem from old -> new
new->old is not supported anyway and this breaks so that
the vm crashes and loses the configs for scsi15-30
(same behaviour as e.g. with audio0 and migration from new->old)
tested with 31 scsi disk on
i440fx + virtio-scsi
i440fx + lsi
q35 + virtio-scsi
q35 + lsi
with ovmf + seabios
Stefan Reiter [Thu, 16 Jan 2020 15:40:48 +0000 (16:40 +0100)]
Add CPUConfig file and migrate some helpers
The package will be used for custom CPU models as a SectionConfig, hence
the name. For now we simply move some CPU related helper functions and
declarations over from QemuServer to reduce clutter there.
Exports are to avoid changing all call sites, functions have useful
names on their own.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Thomas Lamprecht [Wed, 15 Jan 2020 07:12:22 +0000 (08:12 +0100)]
api: vm clone: unlink zombie target config at end of error cleanup
This is the guarantee that this call operates on it's created config.
A VMID cannot be reused afterall. So only remove the guarantee at the
last step, just before throwing up the error message about the clone
failure.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Wed, 15 Jan 2020 07:06:14 +0000 (08:06 +0100)]
api: vm clone: remove cloned FW conf in error cleanup path
We clone the source VM firewall config before forking the "realcmd"
worker, but did not mind cleaning it up again if the clone failed
somewhere in the worker.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Reiter [Thu, 21 Nov 2019 14:53:42 +0000 (15:53 +0100)]
Add QEMU CPU flag querying helpers
* query_understood_cpu_flags returns all flags that QEMU/KVM knows about
* query_supported_cpu_flags returns all flags that QEMU/KVM can use on
this particular host.
To get supported flags, a temporary VM is started with QEMU, so we can
issue the "query-cpu-model-expansion" QMP command. This is how libvirt
queries supported flags for its "host-passthrough" CPU type.
query_supported_cpu_flags is thus rather slow and shouldn't be called
unnecessarily.
Note that KVM and TCG accelerators provide different expansions for the
"host" CPU type, so we need to query both.
Currently only supports x86_64, because QEMU-aarch64 doesn't provide the
necessary querying functions.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Oguz Bektas [Tue, 7 Jan 2020 15:55:18 +0000 (16:55 +0100)]
vmconfig_apply_pending: add error handling
wrap around code which can possibly fail in evals to handle them
gracefully, and log errors.
note: this results in a change of behavior in the API. since errors
are handled gracefully instead of "die"ing, when there is a pending
change which cannot be applied for some reason, it will get logged in
the tasklog but the vm will continue booting regardless. the
non-applied change will stay in the pending section of the
configuration.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Mon, 16 Dec 2019 14:56:46 +0000 (15:56 +0100)]
api/restore: do not trigger autostart task from locked context
Do the same as for the "create" case, only trigger the "start after
create/restore" task after the locked "realcmd" was done. Else, the
start can never succeed, it also acquires a lock, but restore only
release it once outside of realcmd.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Fri, 13 Dec 2019 11:11:58 +0000 (12:11 +0100)]
grammar fix: s/does not exists/does not exist/g
bump versioned build-dependency, as qemu-server has tests checking
for errors, and we fixed an grammar error in pve-storage, so we need
the newer version to ensure our test go through
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Reiter [Mon, 9 Dec 2019 15:14:08 +0000 (16:14 +0100)]
hide long commandline on vm_start/migrate failure
By default run_command prints the entire commandline executed when an
error occurs, but QEMU and our migrate command are not only
uninteresting to the user[*] but also annoyingly long. Hide them and only
print the exit code.
[*] Especially our migrate command, since it can't be manually executed
anyway. QEMU's commandline *might* contain something interesting, but is
so long that it's tricky to parse anyway, any a user can always call 'qm
showcmd --pretty'.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Stefan Reiter [Mon, 9 Dec 2019 13:08:09 +0000 (14:08 +0100)]
update disk size before local disk migration
Split out 'update_disksize' from the renamed 'update_disk_config' to
allow code reuse in QemuMigrate.
Remove dots after messages to keep style consistent for migration log.
After updating in sync_disks (phase1) of migration, write out updated
config. This means that even if migration fails or is aborted in later
stages, we keep the fixed config - this is not an issue, as it would
have been fixed on the next attempt anyway, and it can't hurt to have
the correct size instead of a wrong one either way.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
projects / qemu-server.git / log