]>
Commit | Line | Data |
---|---|---|
1bf84a1e | 1 | Recommendations for KVM CPU model configuration on x86 hosts |
c2c7f335 | 2 | ============================================================ |
1bf84a1e KC |
3 | |
4 | The information that follows provides recommendations for configuring | |
5 | CPU models on x86 hosts. The goals are to maximise performance, while | |
6 | protecting guest OS against various CPU hardware flaws, and optionally | |
7 | enabling live migration between hosts with heterogeneous CPU models. | |
8 | ||
9 | ||
10 | Two ways to configure CPU models with QEMU / KVM | |
11 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
12 | ||
13 | (1) **Host passthrough** | |
14 | ||
15 | This passes the host CPU model features, model, stepping, exactly to | |
16 | the guest. Note that KVM may filter out some host CPU model features | |
17 | if they cannot be supported with virtualization. Live migration is | |
18 | unsafe when this mode is used as libvirt / QEMU cannot guarantee a | |
19 | stable CPU is exposed to the guest across hosts. This is the | |
20 | recommended CPU to use, provided live migration is not required. | |
21 | ||
22 | (2) **Named model** | |
23 | ||
24 | QEMU comes with a number of predefined named CPU models, that | |
25 | typically refer to specific generations of hardware released by | |
26 | Intel and AMD. These allow the guest VMs to have a degree of | |
27 | isolation from the host CPU, allowing greater flexibility in live | |
28 | migrating between hosts with differing hardware. @end table | |
29 | ||
30 | In both cases, it is possible to optionally add or remove individual CPU | |
31 | features, to alter what is presented to the guest by default. | |
32 | ||
33 | Libvirt supports a third way to configure CPU models known as "Host | |
34 | model". This uses the QEMU "Named model" feature, automatically picking | |
35 | a CPU model that is similar the host CPU, and then adding extra features | |
36 | to approximate the host model as closely as possible. This does not | |
37 | guarantee the CPU family, stepping, etc will precisely match the host | |
38 | CPU, as they would with "Host passthrough", but gives much of the | |
39 | benefit of passthrough, while making live migration safe. | |
40 | ||
41 | ||
38dec0ef DB |
42 | ABI compatibility levels for CPU models |
43 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
44 | ||
45 | The x86_64 architecture has a number of `ABI compatibility levels`_ | |
46 | defined. Traditionally most operating systems and toolchains would | |
47 | only target the original baseline ABI. It is expected that in | |
48 | future OS and toolchains are likely to target newer ABIs. The | |
49 | table that follows illustrates which ABI compatibility levels | |
50 | can be satisfied by the QEMU CPU models. Note that the table only | |
51 | lists the long term stable CPU model versions (eg Haswell-v4). | |
b980c1ae | 52 | In addition to what is listed, there are also many CPU model |
38dec0ef DB |
53 | aliases which resolve to a different CPU model version, |
54 | depending on the machine type is in use. | |
55 | ||
56 | .. _ABI compatibility levels: https://gitlab.com/x86-psABIs/x86-64-ABI/ | |
57 | ||
58 | .. csv-table:: x86-64 ABI compatibility levels | |
59 | :file: cpu-models-x86-abi.csv | |
60 | :widths: 40,15,15,15,15 | |
61 | :header-rows: 2 | |
62 | ||
63 | ||
1bf84a1e KC |
64 | Preferred CPU models for Intel x86 hosts |
65 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
66 | ||
67 | The following CPU models are preferred for use on Intel hosts. | |
68 | Administrators / applications are recommended to use the CPU model that | |
69 | matches the generation of the host CPUs in use. In a deployment with a | |
70 | mixture of host CPU models between machines, if live migration | |
71 | compatibility is required, use the newest CPU model that is compatible | |
72 | across all desired hosts. | |
73 | ||
3b2c52c0 KC |
74 | ``Cascadelake-Server``, ``Cascadelake-Server-noTSX`` |
75 | Intel Xeon Processor (Cascade Lake, 2019), with "stepping" levels 6 | |
76 | or 7 only. (The Cascade Lake Xeon processor with *stepping 5 is | |
77 | vulnerable to MDS variants*.) | |
78 | ||
79 | ``Skylake-Server``, ``Skylake-Server-IBRS``, ``Skylake-Server-IBRS-noTSX`` | |
1bf84a1e KC |
80 | Intel Xeon Processor (Skylake, 2016) |
81 | ||
3b2c52c0 | 82 | ``Skylake-Client``, ``Skylake-Client-IBRS``, ``Skylake-Client-noTSX-IBRS}`` |
1bf84a1e KC |
83 | Intel Core Processor (Skylake, 2015) |
84 | ||
85 | ``Broadwell``, ``Broadwell-IBRS``, ``Broadwell-noTSX``, ``Broadwell-noTSX-IBRS`` | |
86 | Intel Core Processor (Broadwell, 2014) | |
87 | ||
88 | ``Haswell``, ``Haswell-IBRS``, ``Haswell-noTSX``, ``Haswell-noTSX-IBRS`` | |
89 | Intel Core Processor (Haswell, 2013) | |
90 | ||
91 | ``IvyBridge``, ``IvyBridge-IBR`` | |
92 | Intel Xeon E3-12xx v2 (Ivy Bridge, 2012) | |
93 | ||
94 | ``SandyBridge``, ``SandyBridge-IBRS`` | |
95 | Intel Xeon E312xx (Sandy Bridge, 2011) | |
96 | ||
97 | ``Westmere``, ``Westmere-IBRS`` | |
98 | Westmere E56xx/L56xx/X56xx (Nehalem-C, 2010) | |
99 | ||
100 | ``Nehalem``, ``Nehalem-IBRS`` | |
101 | Intel Core i7 9xx (Nehalem Class Core i7, 2008) | |
102 | ||
103 | ``Penryn`` | |
104 | Intel Core 2 Duo P9xxx (Penryn Class Core 2, 2007) | |
105 | ||
106 | ``Conroe`` | |
107 | Intel Celeron_4x0 (Conroe/Merom Class Core 2, 2006) | |
108 | ||
109 | ||
110 | Important CPU features for Intel x86 hosts | |
111 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
112 | ||
113 | The following are important CPU features that should be used on Intel | |
114 | x86 hosts, when available in the host CPU. Some of them require explicit | |
115 | configuration to enable, as they are not included by default in some, or | |
116 | all, of the named CPU models listed above. In general all of these | |
117 | features are included if using "Host passthrough" or "Host model". | |
118 | ||
119 | ``pcid`` | |
120 | Recommended to mitigate the cost of the Meltdown (CVE-2017-5754) fix. | |
121 | ||
122 | Included by default in Haswell, Broadwell & Skylake Intel CPU models. | |
123 | ||
124 | Should be explicitly turned on for Westmere, SandyBridge, and | |
125 | IvyBridge Intel CPU models. Note that some desktop/mobile Westmere | |
126 | CPUs cannot support this feature. | |
127 | ||
128 | ``spec-ctrl`` | |
129 | Required to enable the Spectre v2 (CVE-2017-5715) fix. | |
130 | ||
131 | Included by default in Intel CPU models with -IBRS suffix. | |
132 | ||
133 | Must be explicitly turned on for Intel CPU models without -IBRS | |
134 | suffix. | |
135 | ||
136 | Requires the host CPU microcode to support this feature before it | |
137 | can be used for guest CPUs. | |
138 | ||
139 | ``stibp`` | |
140 | Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some | |
141 | operating systems. | |
142 | ||
143 | Must be explicitly turned on for all Intel CPU models. | |
144 | ||
145 | Requires the host CPU microcode to support this feature before it can | |
146 | be used for guest CPUs. | |
147 | ||
148 | ``ssbd`` | |
149 | Required to enable the CVE-2018-3639 fix. | |
150 | ||
151 | Not included by default in any Intel CPU model. | |
152 | ||
153 | Must be explicitly turned on for all Intel CPU models. | |
154 | ||
155 | Requires the host CPU microcode to support this feature before it | |
156 | can be used for guest CPUs. | |
157 | ||
158 | ``pdpe1gb`` | |
159 | Recommended to allow guest OS to use 1GB size pages. | |
160 | ||
161 | Not included by default in any Intel CPU model. | |
162 | ||
163 | Should be explicitly turned on for all Intel CPU models. | |
164 | ||
165 | Note that not all CPU hardware will support this feature. | |
166 | ||
167 | ``md-clear`` | |
168 | Required to confirm the MDS (CVE-2018-12126, CVE-2018-12127, | |
169 | CVE-2018-12130, CVE-2019-11091) fixes. | |
170 | ||
171 | Not included by default in any Intel CPU model. | |
172 | ||
173 | Must be explicitly turned on for all Intel CPU models. | |
174 | ||
175 | Requires the host CPU microcode to support this feature before it | |
176 | can be used for guest CPUs. | |
177 | ||
3b2c52c0 KC |
178 | ``mds-no`` |
179 | Recommended to inform the guest OS that the host is *not* vulnerable | |
180 | to any of the MDS variants ([MFBDS] CVE-2018-12130, [MLPDS] | |
181 | CVE-2018-12127, [MSBDS] CVE-2018-12126). | |
182 | ||
183 | This is an MSR (Model-Specific Register) feature rather than a CPUID feature, | |
184 | so it will not appear in the Linux ``/proc/cpuinfo`` in the host or | |
185 | guest. Instead, the host kernel uses it to populate the MDS | |
186 | vulnerability file in ``sysfs``. | |
187 | ||
188 | So it should only be enabled for VMs if the host reports @code{Not | |
189 | affected} in the ``/sys/devices/system/cpu/vulnerabilities/mds`` file. | |
190 | ||
191 | ``taa-no`` | |
192 | Recommended to inform that the guest that the host is ``not`` | |
193 | vulnerable to CVE-2019-11135, TSX Asynchronous Abort (TAA). | |
194 | ||
195 | This too is an MSR feature, so it does not show up in the Linux | |
196 | ``/proc/cpuinfo`` in the host or guest. | |
197 | ||
198 | It should only be enabled for VMs if the host reports ``Not affected`` | |
199 | in the ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` | |
200 | file. | |
201 | ||
202 | ``tsx-ctrl`` | |
203 | Recommended to inform the guest that it can disable the Intel TSX | |
204 | (Transactional Synchronization Extensions) feature; or, if the | |
205 | processor is vulnerable, use the Intel VERW instruction (a | |
206 | processor-level instruction that performs checks on memory access) as | |
207 | a mitigation for the TAA vulnerability. (For details, refer to | |
208 | Intel's `deep dive into MDS | |
209 | <https://software.intel.com/security-software-guidance/insights/deep-dive-intel-analysis-microarchitectural-data-sampling>`_.) | |
210 | ||
211 | Expose this to the guest OS if and only if: (a) the host has TSX | |
212 | enabled; *and* (b) the guest has ``rtm`` CPU flag enabled. | |
213 | ||
214 | By disabling TSX, KVM-based guests can avoid paying the price of | |
215 | mitigating TSX-based attacks. | |
216 | ||
217 | Note that ``tsx-ctrl`` too is an MSR feature, so it does not show | |
218 | up in the Linux ``/proc/cpuinfo`` in the host or guest. | |
219 | ||
220 | To validate that Intel TSX is indeed disabled for the guest, there are | |
221 | two ways: (a) check for the *absence* of ``rtm`` in the guest's | |
222 | ``/proc/cpuinfo``; or (b) the | |
223 | ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` file in | |
224 | the guest should report ``Mitigation: TSX disabled``. | |
225 | ||
1bf84a1e KC |
226 | |
227 | Preferred CPU models for AMD x86 hosts | |
228 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
229 | ||
2edb76ea | 230 | The following CPU models are preferred for use on AMD hosts. |
1bf84a1e KC |
231 | Administrators / applications are recommended to use the CPU model that |
232 | matches the generation of the host CPUs in use. In a deployment with a | |
233 | mixture of host CPU models between machines, if live migration | |
234 | compatibility is required, use the newest CPU model that is compatible | |
235 | across all desired hosts. | |
236 | ||
237 | ``EPYC``, ``EPYC-IBPB`` | |
238 | AMD EPYC Processor (2017) | |
239 | ||
240 | ``Opteron_G5`` | |
241 | AMD Opteron 63xx class CPU (2012) | |
242 | ||
243 | ``Opteron_G4`` | |
244 | AMD Opteron 62xx class CPU (2011) | |
245 | ||
246 | ``Opteron_G3`` | |
247 | AMD Opteron 23xx (Gen 3 Class Opteron, 2009) | |
248 | ||
249 | ``Opteron_G2`` | |
250 | AMD Opteron 22xx (Gen 2 Class Opteron, 2006) | |
251 | ||
252 | ``Opteron_G1`` | |
253 | AMD Opteron 240 (Gen 1 Class Opteron, 2004) | |
254 | ||
255 | ||
256 | Important CPU features for AMD x86 hosts | |
257 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
258 | ||
259 | The following are important CPU features that should be used on AMD x86 | |
260 | hosts, when available in the host CPU. Some of them require explicit | |
261 | configuration to enable, as they are not included by default in some, or | |
262 | all, of the named CPU models listed above. In general all of these | |
263 | features are included if using "Host passthrough" or "Host model". | |
264 | ||
265 | ``ibpb`` | |
266 | Required to enable the Spectre v2 (CVE-2017-5715) fix. | |
267 | ||
268 | Included by default in AMD CPU models with -IBPB suffix. | |
269 | ||
270 | Must be explicitly turned on for AMD CPU models without -IBPB suffix. | |
271 | ||
272 | Requires the host CPU microcode to support this feature before it | |
273 | can be used for guest CPUs. | |
274 | ||
275 | ``stibp`` | |
276 | Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some | |
277 | operating systems. | |
278 | ||
279 | Must be explicitly turned on for all AMD CPU models. | |
280 | ||
281 | Requires the host CPU microcode to support this feature before it | |
282 | can be used for guest CPUs. | |
283 | ||
284 | ``virt-ssbd`` | |
285 | Required to enable the CVE-2018-3639 fix | |
286 | ||
287 | Not included by default in any AMD CPU model. | |
288 | ||
289 | Must be explicitly turned on for all AMD CPU models. | |
290 | ||
291 | This should be provided to guests, even if amd-ssbd is also provided, | |
292 | for maximum guest compatibility. | |
293 | ||
294 | Note for some QEMU / libvirt versions, this must be force enabled when | |
295 | when using "Host model", because this is a virtual feature that | |
296 | doesn't exist in the physical host CPUs. | |
297 | ||
298 | ``amd-ssbd`` | |
299 | Required to enable the CVE-2018-3639 fix | |
300 | ||
301 | Not included by default in any AMD CPU model. | |
302 | ||
303 | Must be explicitly turned on for all AMD CPU models. | |
304 | ||
305 | This provides higher performance than ``virt-ssbd`` so should be | |
306 | exposed to guests whenever available in the host. ``virt-ssbd`` should | |
307 | none the less also be exposed for maximum guest compatibility as some | |
308 | kernels only know about ``virt-ssbd``. | |
309 | ||
310 | ``amd-no-ssb`` | |
311 | Recommended to indicate the host is not vulnerable CVE-2018-3639 | |
312 | ||
313 | Not included by default in any AMD CPU model. | |
314 | ||
315 | Future hardware generations of CPU will not be vulnerable to | |
316 | CVE-2018-3639, and thus the guest should be told not to enable | |
317 | its mitigations, by exposing amd-no-ssb. This is mutually | |
318 | exclusive with virt-ssbd and amd-ssbd. | |
319 | ||
320 | ``pdpe1gb`` | |
321 | Recommended to allow guest OS to use 1GB size pages | |
322 | ||
323 | Not included by default in any AMD CPU model. | |
324 | ||
325 | Should be explicitly turned on for all AMD CPU models. | |
326 | ||
327 | Note that not all CPU hardware will support this feature. | |
328 | ||
329 | ||
330 | Default x86 CPU models | |
331 | ^^^^^^^^^^^^^^^^^^^^^^ | |
332 | ||
333 | The default QEMU CPU models are designed such that they can run on all | |
334 | hosts. If an application does not wish to do perform any host | |
335 | compatibility checks before launching guests, the default is guaranteed | |
336 | to work. | |
337 | ||
338 | The default CPU models will, however, leave the guest OS vulnerable to | |
339 | various CPU hardware flaws, so their use is strongly discouraged. | |
340 | Applications should follow the earlier guidance to setup a better CPU | |
341 | configuration, with host passthrough recommended if live migration is | |
342 | not needed. | |
343 | ||
344 | ``qemu32``, ``qemu64`` | |
345 | QEMU Virtual CPU version 2.5+ (32 & 64 bit variants) | |
346 | ||
347 | ``qemu64`` is used for x86_64 guests and ``qemu32`` is used for i686 | |
348 | guests, when no ``-cpu`` argument is given to QEMU, or no ``<cpu>`` is | |
349 | provided in libvirt XML. | |
350 | ||
351 | Other non-recommended x86 CPUs | |
352 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
353 | ||
354 | The following CPUs models are compatible with most AMD and Intel x86 | |
355 | hosts, but their usage is discouraged, as they expose a very limited | |
356 | featureset, which prevents guests having optimal performance. | |
357 | ||
358 | ``kvm32``, ``kvm64`` | |
359 | Common KVM processor (32 & 64 bit variants). | |
360 | ||
361 | Legacy models just for historical compatibility with ancient QEMU | |
362 | versions. | |
363 | ||
364 | ``486``, ``athlon``, ``phenom``, ``coreduo``, ``core2duo``, ``n270``, ``pentium``, ``pentium2``, ``pentium3`` | |
365 | Various very old x86 CPU models, mostly predating the introduction | |
366 | of hardware assisted virtualization, that should thus not be | |
367 | required for running virtual machines. | |
368 | ||
369 | ||
370 | Syntax for configuring CPU models | |
c2c7f335 | 371 | ================================= |
1bf84a1e KC |
372 | |
373 | The examples below illustrate the approach to configuring the various | |
374 | CPU models / features in QEMU and libvirt. | |
375 | ||
376 | QEMU command line | |
377 | ^^^^^^^^^^^^^^^^^ | |
378 | ||
379 | Host passthrough: | |
380 | ||
381 | .. parsed-literal:: | |
382 | ||
383 | |qemu_system| -cpu host | |
384 | ||
385 | Host passthrough with feature customization: | |
386 | ||
387 | .. parsed-literal:: | |
388 | ||
1bd39ea9 | 389 | |qemu_system| -cpu host,vmx=off,... |
1bf84a1e KC |
390 | |
391 | Named CPU models: | |
392 | ||
393 | .. parsed-literal:: | |
394 | ||
395 | |qemu_system| -cpu Westmere | |
396 | ||
397 | Named CPU models with feature customization: | |
398 | ||
399 | .. parsed-literal:: | |
400 | ||
1bd39ea9 | 401 | |qemu_system| -cpu Westmere,pcid=on,... |
1bf84a1e KC |
402 | |
403 | Libvirt guest XML | |
404 | ^^^^^^^^^^^^^^^^^ | |
405 | ||
406 | Host passthrough:: | |
407 | ||
408 | <cpu mode='host-passthrough'/> | |
409 | ||
410 | Host passthrough with feature customization:: | |
411 | ||
412 | <cpu mode='host-passthrough'> | |
413 | <feature name="vmx" policy="disable"/> | |
414 | ... | |
415 | </cpu> | |
416 | ||
417 | Host model:: | |
418 | ||
419 | <cpu mode='host-model'/> | |
420 | ||
421 | Host model with feature customization:: | |
422 | ||
423 | <cpu mode='host-model'> | |
424 | <feature name="vmx" policy="disable"/> | |
425 | ... | |
426 | </cpu> | |
427 | ||
428 | Named model:: | |
429 | ||
430 | <cpu mode='custom'> | |
431 | <model name="Westmere"/> | |
432 | </cpu> | |
433 | ||
434 | Named model with feature customization:: | |
435 | ||
436 | <cpu mode='custom'> | |
437 | <model name="Westmere"/> | |
438 | <feature name="pcid" policy="require"/> | |
439 | ... | |
440 | </cpu> |