Kernels 4.18+ (4.17+ for evmcs) support new Hyper-V enlightenments for
Windows KVM guests. QEMU supports these since 3.0 and 3.1 respectively.
tlbflush and ipi improve performance on overcommitted systems, evmcs
improves nested virtualization.
It's not entirely clear to me if Win7 already supports these, but since
it doesn't cause any performance penalties (and it works fine without
crashing, which makes sense either way, because Hyper-V enlightenments
are opt-in by the guest OS), enabling it regardless should be fine.
(As opposed to adding a new if branch for win8+)
Feature explanations to the best of my understanding:
hv_tlbflush allows the guest OS to trigger tlb shootdowns via a
hypercall. This allows CPUs to be identified via their vpindex (which
makes hv_vpindex a prerequisite to hv_tlbflush, but that is already
handled in our code). In overcommited configurations, where multiple
vCPUs reside on one pCPU, this increases performance of guest tlb
flushes, by only flushing each pCPU once. It also allows multiple tlb
flushes with only one vmexit.
hv_ipi allows sending inter-processor interrupts via vpindex, once again
making it a prerequisite. Benefits are pretty much as with tlbflush.
hv_evmcs is a VM control structure in L1 guest memory, allowing an L1 guest
to modify L2 VMCS and entering L2 without having the L0 host perform an
expensive VMCS update on trapping the nested vmenter.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Stefan Reiter [Mon, 24 Jun 2019 08:33:07 +0000 (10:33 +0200)]
fix #2244: Allow timeout for guest-agent shutdown
The "guest-shutdown" guest agent call is blocking for some reason, so if
it fails (e.g. agent not installed on guest) only the default timeout of
10 minutes (see QMPClient.pm, sub cmd) would apply.
With this change, if (and only if) a timeout is specified via CLI/API,
it is used instead. In case it is not specified, behaviour stays the
same (default 10 min timeout).
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
This should help with the rare case where stop mode backups
fail to restart due to the $vmid.scope not being completely
gone when we want to restart. This queries systemd via dbus,
and if the scope is still there, awaits a UnitRemoved signal
for the scope from dbus.
For now with a 5 second timeout... (given that the processes
are already gone and it's really just waiting for systemd to
wake up, this should be plenty...)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Christian Ebner [Tue, 11 Jun 2019 10:13:52 +0000 (12:13 +0200)]
fix #2190: Base64 encode SMBIOS value strings in order to allow more characters
On some occasions e.g. license checking, the manufacturer string in the
SMBIOS settings edit has to allow characters such as whitespaces.
https://forum.proxmox.com/threads/proxmox-and-windows-rok-license-for-dell.53236/
In principle SMBIOS allows to pass any zero terminated string to the
corresponding fields in the structure type 1 (System Information).
By base64 encoding the values clashing of the config is avoided.
Relies on the corresponding patch to pve-manager to obtain base64 encoded values.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Mira Limbeck [Wed, 5 Jun 2019 09:09:42 +0000 (11:09 +0200)]
add new API for dumping cloudinit config
Adds the path '{vmid}/cloudinit/dump' and requires the parameter 'type'
that's either 'user', 'network' or 'meta'. Returns the generated config as
string.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Wed, 5 Jun 2019 09:09:41 +0000 (11:09 +0200)]
add function to dump cloudinit config
This adds a function to dump the generated cloudinit config. Only one
can be dumped at a time, either 'user', 'network' or 'meta'.
The logic to get user, network and metadata is copied from the other
path that also creates the ISO image to keep it simple and not
complicate the other code path further.
The hash generation for the metadata config is unified between nocloud
and configdrive2 formats. We need it a 3rd time with the new dump
functions so it makes sense to combine it and the metadata config
generation in a single function. The <format>_gen_metadata functions are
each used twice now.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Rhonda D'Vine [Wed, 5 Jun 2019 13:30:48 +0000 (15:30 +0200)]
Fix #1999: cli: listsnapshot: handle multiple roots and mark orphaned as root
This commit addresses the following things:
* There can be multiple independent snapshot tree roots, display them
all
* If a snapshot defines a parent that doesn't exist, it is now
considered a snapshot root
There is a potential issue (which was also before and also affects the
GUI): circular snapshot trees. That plays into the second mentioned
issue above. If you manage to have a snapshot that defines a
non-existing root in the config, and then create a snapshot with that
exact name as a child of that snapshot, it would create a circular
dependency. This would have to get addressed in the GUI too.
Signed-off-by: Rhonda D'Vine <rhonda@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
for both vm_mon_cmd calls. under certain circumstances, the following
sequence of events can otherwise fail when live-migrating under load:
S...source node
T...target node
0: migration is complete, handover from S to T starts
1: S: logically move VM config file from S to T via rename()
2: S: rename returns, config file is (visibly) moved on S
3: S: trigger resume on T via mtunnel
4a: T: call vm_resume while config file move is not yet visible on T
4b: T: call vm_resume while config file move is already visible on T
4a instead of 4b means vm_mon_cmd will die in check_running unless
vm_mon_cmd_nocheck is used.
under heavy pmxcfs load and a slow cluster/corosync network, there can
be a few seconds of delay between 1 and 2, with a subsequent race ending in 4a
instead of 4b.
this issue was reported to occur on bulk migrations.
Thomas Lamprecht [Fri, 17 May 2019 10:09:51 +0000 (12:09 +0200)]
fixup: delete cloudinit disk before restoring
cloudinit is just completely broken...
Until we rewrite this to a sane designe (i.e., never backup, mirror,
... any CI disk anywhere - the state is in the config and gets
created on start and deleted on stop anyway) do this..
Co-developed-by: Mira Limbek <m.limbek@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Mira Limbeck [Thu, 16 May 2019 12:07:01 +0000 (14:07 +0200)]
map cloudinit disk to new vmid on restore
Resolves the issue of restoring a VM that has a cloudinit drive
configured to a new VMID. The VMID of the disk name gets now remapped
correctly and in the process the cloudinit disk is created with the same size
as in PVE/API2/Qemu.pm create_disks and in PVE/QemuServer/Cloudinit
commit_cloudinit_disk as the same constant is used.
This is done by matching any line starting with either 'ide', 'sata' or
'scsi' and then check if it is a cloudinit drive. As cloudinit drives
are attached as cdrom, only those 3 are possible. For the cloudinit
check we use 'parse_drive' followed by 'drive_is_cloudinit' so no custom
regex is necessary.
'--targetstorage' is also taken into account on restore now for
cloudinit disks. If a target storage is specified the format is either
kept if possible, or changed to the default of that storage.
This should fix #1807 completely. The restore error was already resolved
with commit 7e8ab2a, but the vmid of the disk might not have matched the new
one.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Thu, 16 May 2019 13:08:50 +0000 (15:08 +0200)]
fix clone_disk with formats other than raw/qcow2
with commit 64d1a6a it's now possible to specify a format other than raw
or qcow2 when creating VMs. This can lead to an error when cloning the
VMs and a cloudinit disk with a different format is attached (e.g.
vmdk).
We use QEMU_FORMAT_RE in drive_is_cloudinit and according to the
QEMU_FORMAT_RE we support 7 different formats.
With this change we add any format other than 'raw' as '.<format>' to the
name and no longer die on any other format. Cloudinit disks with invalid
format are not cloned as the drive is recognized as cdrom, not cloudinit.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Wed, 15 May 2019 10:53:24 +0000 (12:53 +0200)]
fix existing cloudinit volume not available for file_size_info
file_size_info can't find the file if it is not available, e.g.,
RBD storage with KRBD or LVM where the volume was not yet activated,
returns then 0, which we interpret as the disk not existing, thus
call vdisk_alloc which errors as the disk, in fact, really already exists.
With this patch we call activate_volume before trying
file_size_info, so if the volume exists we get it available and else
we can really create it.
If the disk does not exist and is created with vdisk_alloc we still
require an additional call to activate_volume for the new disk.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Tue, 30 Apr 2019 13:09:21 +0000 (13:09 +0000)]
followup: do not query size of image we just created
we know the size, and even if a storage plugin pads this up (it
mustn't alloc something smaller, but something bigger can be OK) we
know that our 4MB is OK, and can only be used anyway to make this
compatible between storage plugins.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Mira Limbeck [Tue, 30 Apr 2019 12:20:47 +0000 (14:20 +0200)]
fix #2173: use qemu-img to check cloudinit disk existence
use file_size_info to check for existence of cloudinit disk instead of
'-e'. It uses `qemu-img info` to get some file info, which can handle
rbd, and various other paths for volumes not exposed as normal file
or not mapped, yet.
this addresses a problem with rbd where the path returned available
is not checkable with '-e'.
Any size > 0 is interpreted as the image existing.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Christian Ebner [Fri, 19 Apr 2019 10:06:07 +0000 (12:06 +0200)]
fix: #1075: Restore VM template to VM and try to convert to template.
The restore of a backup from a VM template will first restore the VM and then
convert the restored VM back into a template.
This automatically performes the steps of the current behaviour, where the user
has to manually convert the restored VM back to a template.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
fix creating clone if target storage is same as source storage
the clone API calls (target) 'storage' parameter is optional as we
simply use the source storage in this case, but we did not handle
this case when we added the bandwidth_limit abillity, address that.
This patch only pushes the storage parameter into the storage_list array
if it is defined.
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The 'migrate_speed' can be set in the VM config. Additionally the 'migrate'
bwlimit from datacenter.cfg (storage-specific limits play no role for
memory+state migration) or the parameter provided to the API call can restrict
the speed. Take the lower of the two.
This patch also refactors the setting of migrate_speed and comments for clarity.
Mira Limbeck [Fri, 29 Mar 2019 15:32:03 +0000 (16:32 +0100)]
cloudinit: create disk if it does not exist on start
create a fixed size cloudinit disk if it is referenced in config and
does not exist. the size of the disk created when first added to the
config is reduced to 4MiB to match the one created in
commit_cloudinit_disk.
maximum file size per snippet file (network, user, meta) is increased to 1MiB.
preparation for offline migration without the cloudinit disk (that is
always regenerated on start).
also fixes #1807, although a further patch is required to change the
vmid on restore
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Tested-by: Dominik Csapak <d.csapak@proxmox.com>
Dominik Csapak [Thu, 14 Mar 2019 16:04:50 +0000 (17:04 +0100)]
add statestorage parameter to suspend API
this makes it possible to give a storage for state saving, if one
wants to use a different storage than for snapshots or does not
want to save this info into the config
Dominik Csapak [Thu, 14 Mar 2019 16:04:48 +0000 (17:04 +0100)]
resume suspended vm on start
if a vm has the 'suspended' lock, we resume with the saved state
and remove the lock, the saved vmstate and the saved runningmachine
after the vm started
Dominik Csapak [Thu, 14 Mar 2019 16:04:47 +0000 (17:04 +0100)]
implement suspend to disk for running vms
the idea is to have the same logic as with snapshots, but without
the snapshotting of the disks, and after saving the vm state (incl memory),
we hard shut off the guest.
this way the disks will not be touched anymore by the guest
to prevent any alteration of the vm (incl migration, hw changes, etc) we
add a config lock 'suspend'
Stoiko Ivanov [Tue, 12 Mar 2019 15:07:45 +0000 (16:07 +0100)]
config: NIC macaddr: enforce unicast MAC addresses
creating a VM with a NIC with multicast mac (see [1]) is possible, but setting
the interface's link up inside the guest fails (tested on Debian stable).
The issue was noted with LXC first (see [0,2]) and then tested with Qemu.
This patch uses the 'mac-addr' standard_option defined in PVE::JSONSchema to
ensure only unicast MAC addresses are used for netconfig.
Dominik Csapak [Wed, 13 Mar 2019 16:28:04 +0000 (17:28 +0100)]
fix #2131: get correct device when deleting iothreads
we map scsiX to virtioscsiX/scsihwX when we use virtio-scsi-single to add
and iothread so we have to map it back when we delete an iothread, else the
parsing fails with
Dominik Csapak [Thu, 7 Mar 2019 12:43:11 +0000 (13:43 +0100)]
fix #2120: use hosts initiator name with qemu-img
qemu-img uses the qemu default initiator name 'iqn.2008-11.org.linux-kvm'
since we use the one of the host (/etc/iscsi/initiatorname.iscsi) when
using it with a running vm, we want to using it also when moving a disk
with qemu-img
to do that we have give qemu-img the image in as a full option string
this fixes the issue that we could not move an zfs-over-iscsi disk
without allowing the default qemu initiator
David Limbeck [Thu, 7 Feb 2019 14:12:35 +0000 (15:12 +0100)]
cloud-init: allow custom network/user data files via snippets
Adds the 'cicustom' option to specify either or both network and user
options as property strings. Their parameters are files in a snippets
storage (e.g. local:snippets/network.yaml). If one or both are specified
they are used instead of their respective generated configuration.
This allows the use of completely custom configurations and is also a
possible solution for bug #2068 by specifying a custom user file that
contains package_upgrade: false.
Tested with Ubuntu 18.10 and cloud-init 18.4.7
Signed-off-by: David Limbeck <d.limbeck@proxmox.com>
projects / qemu-server.git / log