]>
Commit | Line | Data |
---|---|---|
1bf84a1e KC |
1 | Recommendations for KVM CPU model configuration on x86 hosts |
2 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
3 | ||
4 | The information that follows provides recommendations for configuring | |
5 | CPU models on x86 hosts. The goals are to maximise performance, while | |
6 | protecting guest OS against various CPU hardware flaws, and optionally | |
7 | enabling live migration between hosts with heterogeneous CPU models. | |
8 | ||
9 | ||
10 | Two ways to configure CPU models with QEMU / KVM | |
11 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
12 | ||
13 | (1) **Host passthrough** | |
14 | ||
15 | This passes the host CPU model features, model, stepping, exactly to | |
16 | the guest. Note that KVM may filter out some host CPU model features | |
17 | if they cannot be supported with virtualization. Live migration is | |
18 | unsafe when this mode is used as libvirt / QEMU cannot guarantee a | |
19 | stable CPU is exposed to the guest across hosts. This is the | |
20 | recommended CPU to use, provided live migration is not required. | |
21 | ||
22 | (2) **Named model** | |
23 | ||
24 | QEMU comes with a number of predefined named CPU models, that | |
25 | typically refer to specific generations of hardware released by | |
26 | Intel and AMD. These allow the guest VMs to have a degree of | |
27 | isolation from the host CPU, allowing greater flexibility in live | |
28 | migrating between hosts with differing hardware. @end table | |
29 | ||
30 | In both cases, it is possible to optionally add or remove individual CPU | |
31 | features, to alter what is presented to the guest by default. | |
32 | ||
33 | Libvirt supports a third way to configure CPU models known as "Host | |
34 | model". This uses the QEMU "Named model" feature, automatically picking | |
35 | a CPU model that is similar the host CPU, and then adding extra features | |
36 | to approximate the host model as closely as possible. This does not | |
37 | guarantee the CPU family, stepping, etc will precisely match the host | |
38 | CPU, as they would with "Host passthrough", but gives much of the | |
39 | benefit of passthrough, while making live migration safe. | |
40 | ||
41 | ||
42 | Preferred CPU models for Intel x86 hosts | |
43 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
44 | ||
45 | The following CPU models are preferred for use on Intel hosts. | |
46 | Administrators / applications are recommended to use the CPU model that | |
47 | matches the generation of the host CPUs in use. In a deployment with a | |
48 | mixture of host CPU models between machines, if live migration | |
49 | compatibility is required, use the newest CPU model that is compatible | |
50 | across all desired hosts. | |
51 | ||
3b2c52c0 KC |
52 | ``Cascadelake-Server``, ``Cascadelake-Server-noTSX`` |
53 | Intel Xeon Processor (Cascade Lake, 2019), with "stepping" levels 6 | |
54 | or 7 only. (The Cascade Lake Xeon processor with *stepping 5 is | |
55 | vulnerable to MDS variants*.) | |
56 | ||
57 | ``Skylake-Server``, ``Skylake-Server-IBRS``, ``Skylake-Server-IBRS-noTSX`` | |
1bf84a1e KC |
58 | Intel Xeon Processor (Skylake, 2016) |
59 | ||
3b2c52c0 | 60 | ``Skylake-Client``, ``Skylake-Client-IBRS``, ``Skylake-Client-noTSX-IBRS}`` |
1bf84a1e KC |
61 | Intel Core Processor (Skylake, 2015) |
62 | ||
63 | ``Broadwell``, ``Broadwell-IBRS``, ``Broadwell-noTSX``, ``Broadwell-noTSX-IBRS`` | |
64 | Intel Core Processor (Broadwell, 2014) | |
65 | ||
66 | ``Haswell``, ``Haswell-IBRS``, ``Haswell-noTSX``, ``Haswell-noTSX-IBRS`` | |
67 | Intel Core Processor (Haswell, 2013) | |
68 | ||
69 | ``IvyBridge``, ``IvyBridge-IBR`` | |
70 | Intel Xeon E3-12xx v2 (Ivy Bridge, 2012) | |
71 | ||
72 | ``SandyBridge``, ``SandyBridge-IBRS`` | |
73 | Intel Xeon E312xx (Sandy Bridge, 2011) | |
74 | ||
75 | ``Westmere``, ``Westmere-IBRS`` | |
76 | Westmere E56xx/L56xx/X56xx (Nehalem-C, 2010) | |
77 | ||
78 | ``Nehalem``, ``Nehalem-IBRS`` | |
79 | Intel Core i7 9xx (Nehalem Class Core i7, 2008) | |
80 | ||
81 | ``Penryn`` | |
82 | Intel Core 2 Duo P9xxx (Penryn Class Core 2, 2007) | |
83 | ||
84 | ``Conroe`` | |
85 | Intel Celeron_4x0 (Conroe/Merom Class Core 2, 2006) | |
86 | ||
87 | ||
88 | Important CPU features for Intel x86 hosts | |
89 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
90 | ||
91 | The following are important CPU features that should be used on Intel | |
92 | x86 hosts, when available in the host CPU. Some of them require explicit | |
93 | configuration to enable, as they are not included by default in some, or | |
94 | all, of the named CPU models listed above. In general all of these | |
95 | features are included if using "Host passthrough" or "Host model". | |
96 | ||
97 | ``pcid`` | |
98 | Recommended to mitigate the cost of the Meltdown (CVE-2017-5754) fix. | |
99 | ||
100 | Included by default in Haswell, Broadwell & Skylake Intel CPU models. | |
101 | ||
102 | Should be explicitly turned on for Westmere, SandyBridge, and | |
103 | IvyBridge Intel CPU models. Note that some desktop/mobile Westmere | |
104 | CPUs cannot support this feature. | |
105 | ||
106 | ``spec-ctrl`` | |
107 | Required to enable the Spectre v2 (CVE-2017-5715) fix. | |
108 | ||
109 | Included by default in Intel CPU models with -IBRS suffix. | |
110 | ||
111 | Must be explicitly turned on for Intel CPU models without -IBRS | |
112 | suffix. | |
113 | ||
114 | Requires the host CPU microcode to support this feature before it | |
115 | can be used for guest CPUs. | |
116 | ||
117 | ``stibp`` | |
118 | Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some | |
119 | operating systems. | |
120 | ||
121 | Must be explicitly turned on for all Intel CPU models. | |
122 | ||
123 | Requires the host CPU microcode to support this feature before it can | |
124 | be used for guest CPUs. | |
125 | ||
126 | ``ssbd`` | |
127 | Required to enable the CVE-2018-3639 fix. | |
128 | ||
129 | Not included by default in any Intel CPU model. | |
130 | ||
131 | Must be explicitly turned on for all Intel CPU models. | |
132 | ||
133 | Requires the host CPU microcode to support this feature before it | |
134 | can be used for guest CPUs. | |
135 | ||
136 | ``pdpe1gb`` | |
137 | Recommended to allow guest OS to use 1GB size pages. | |
138 | ||
139 | Not included by default in any Intel CPU model. | |
140 | ||
141 | Should be explicitly turned on for all Intel CPU models. | |
142 | ||
143 | Note that not all CPU hardware will support this feature. | |
144 | ||
145 | ``md-clear`` | |
146 | Required to confirm the MDS (CVE-2018-12126, CVE-2018-12127, | |
147 | CVE-2018-12130, CVE-2019-11091) fixes. | |
148 | ||
149 | Not included by default in any Intel CPU model. | |
150 | ||
151 | Must be explicitly turned on for all Intel CPU models. | |
152 | ||
153 | Requires the host CPU microcode to support this feature before it | |
154 | can be used for guest CPUs. | |
155 | ||
3b2c52c0 KC |
156 | ``mds-no`` |
157 | Recommended to inform the guest OS that the host is *not* vulnerable | |
158 | to any of the MDS variants ([MFBDS] CVE-2018-12130, [MLPDS] | |
159 | CVE-2018-12127, [MSBDS] CVE-2018-12126). | |
160 | ||
161 | This is an MSR (Model-Specific Register) feature rather than a CPUID feature, | |
162 | so it will not appear in the Linux ``/proc/cpuinfo`` in the host or | |
163 | guest. Instead, the host kernel uses it to populate the MDS | |
164 | vulnerability file in ``sysfs``. | |
165 | ||
166 | So it should only be enabled for VMs if the host reports @code{Not | |
167 | affected} in the ``/sys/devices/system/cpu/vulnerabilities/mds`` file. | |
168 | ||
169 | ``taa-no`` | |
170 | Recommended to inform that the guest that the host is ``not`` | |
171 | vulnerable to CVE-2019-11135, TSX Asynchronous Abort (TAA). | |
172 | ||
173 | This too is an MSR feature, so it does not show up in the Linux | |
174 | ``/proc/cpuinfo`` in the host or guest. | |
175 | ||
176 | It should only be enabled for VMs if the host reports ``Not affected`` | |
177 | in the ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` | |
178 | file. | |
179 | ||
180 | ``tsx-ctrl`` | |
181 | Recommended to inform the guest that it can disable the Intel TSX | |
182 | (Transactional Synchronization Extensions) feature; or, if the | |
183 | processor is vulnerable, use the Intel VERW instruction (a | |
184 | processor-level instruction that performs checks on memory access) as | |
185 | a mitigation for the TAA vulnerability. (For details, refer to | |
186 | Intel's `deep dive into MDS | |
187 | <https://software.intel.com/security-software-guidance/insights/deep-dive-intel-analysis-microarchitectural-data-sampling>`_.) | |
188 | ||
189 | Expose this to the guest OS if and only if: (a) the host has TSX | |
190 | enabled; *and* (b) the guest has ``rtm`` CPU flag enabled. | |
191 | ||
192 | By disabling TSX, KVM-based guests can avoid paying the price of | |
193 | mitigating TSX-based attacks. | |
194 | ||
195 | Note that ``tsx-ctrl`` too is an MSR feature, so it does not show | |
196 | up in the Linux ``/proc/cpuinfo`` in the host or guest. | |
197 | ||
198 | To validate that Intel TSX is indeed disabled for the guest, there are | |
199 | two ways: (a) check for the *absence* of ``rtm`` in the guest's | |
200 | ``/proc/cpuinfo``; or (b) the | |
201 | ``/sys/devices/system/cpu/vulnerabilities/tsx_async_abort`` file in | |
202 | the guest should report ``Mitigation: TSX disabled``. | |
203 | ||
1bf84a1e KC |
204 | |
205 | Preferred CPU models for AMD x86 hosts | |
206 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
207 | ||
208 | The following CPU models are preferred for use on Intel hosts. | |
209 | Administrators / applications are recommended to use the CPU model that | |
210 | matches the generation of the host CPUs in use. In a deployment with a | |
211 | mixture of host CPU models between machines, if live migration | |
212 | compatibility is required, use the newest CPU model that is compatible | |
213 | across all desired hosts. | |
214 | ||
215 | ``EPYC``, ``EPYC-IBPB`` | |
216 | AMD EPYC Processor (2017) | |
217 | ||
218 | ``Opteron_G5`` | |
219 | AMD Opteron 63xx class CPU (2012) | |
220 | ||
221 | ``Opteron_G4`` | |
222 | AMD Opteron 62xx class CPU (2011) | |
223 | ||
224 | ``Opteron_G3`` | |
225 | AMD Opteron 23xx (Gen 3 Class Opteron, 2009) | |
226 | ||
227 | ``Opteron_G2`` | |
228 | AMD Opteron 22xx (Gen 2 Class Opteron, 2006) | |
229 | ||
230 | ``Opteron_G1`` | |
231 | AMD Opteron 240 (Gen 1 Class Opteron, 2004) | |
232 | ||
233 | ||
234 | Important CPU features for AMD x86 hosts | |
235 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
236 | ||
237 | The following are important CPU features that should be used on AMD x86 | |
238 | hosts, when available in the host CPU. Some of them require explicit | |
239 | configuration to enable, as they are not included by default in some, or | |
240 | all, of the named CPU models listed above. In general all of these | |
241 | features are included if using "Host passthrough" or "Host model". | |
242 | ||
243 | ``ibpb`` | |
244 | Required to enable the Spectre v2 (CVE-2017-5715) fix. | |
245 | ||
246 | Included by default in AMD CPU models with -IBPB suffix. | |
247 | ||
248 | Must be explicitly turned on for AMD CPU models without -IBPB suffix. | |
249 | ||
250 | Requires the host CPU microcode to support this feature before it | |
251 | can be used for guest CPUs. | |
252 | ||
253 | ``stibp`` | |
254 | Required to enable stronger Spectre v2 (CVE-2017-5715) fixes in some | |
255 | operating systems. | |
256 | ||
257 | Must be explicitly turned on for all AMD CPU models. | |
258 | ||
259 | Requires the host CPU microcode to support this feature before it | |
260 | can be used for guest CPUs. | |
261 | ||
262 | ``virt-ssbd`` | |
263 | Required to enable the CVE-2018-3639 fix | |
264 | ||
265 | Not included by default in any AMD CPU model. | |
266 | ||
267 | Must be explicitly turned on for all AMD CPU models. | |
268 | ||
269 | This should be provided to guests, even if amd-ssbd is also provided, | |
270 | for maximum guest compatibility. | |
271 | ||
272 | Note for some QEMU / libvirt versions, this must be force enabled when | |
273 | when using "Host model", because this is a virtual feature that | |
274 | doesn't exist in the physical host CPUs. | |
275 | ||
276 | ``amd-ssbd`` | |
277 | Required to enable the CVE-2018-3639 fix | |
278 | ||
279 | Not included by default in any AMD CPU model. | |
280 | ||
281 | Must be explicitly turned on for all AMD CPU models. | |
282 | ||
283 | This provides higher performance than ``virt-ssbd`` so should be | |
284 | exposed to guests whenever available in the host. ``virt-ssbd`` should | |
285 | none the less also be exposed for maximum guest compatibility as some | |
286 | kernels only know about ``virt-ssbd``. | |
287 | ||
288 | ``amd-no-ssb`` | |
289 | Recommended to indicate the host is not vulnerable CVE-2018-3639 | |
290 | ||
291 | Not included by default in any AMD CPU model. | |
292 | ||
293 | Future hardware generations of CPU will not be vulnerable to | |
294 | CVE-2018-3639, and thus the guest should be told not to enable | |
295 | its mitigations, by exposing amd-no-ssb. This is mutually | |
296 | exclusive with virt-ssbd and amd-ssbd. | |
297 | ||
298 | ``pdpe1gb`` | |
299 | Recommended to allow guest OS to use 1GB size pages | |
300 | ||
301 | Not included by default in any AMD CPU model. | |
302 | ||
303 | Should be explicitly turned on for all AMD CPU models. | |
304 | ||
305 | Note that not all CPU hardware will support this feature. | |
306 | ||
307 | ||
308 | Default x86 CPU models | |
309 | ^^^^^^^^^^^^^^^^^^^^^^ | |
310 | ||
311 | The default QEMU CPU models are designed such that they can run on all | |
312 | hosts. If an application does not wish to do perform any host | |
313 | compatibility checks before launching guests, the default is guaranteed | |
314 | to work. | |
315 | ||
316 | The default CPU models will, however, leave the guest OS vulnerable to | |
317 | various CPU hardware flaws, so their use is strongly discouraged. | |
318 | Applications should follow the earlier guidance to setup a better CPU | |
319 | configuration, with host passthrough recommended if live migration is | |
320 | not needed. | |
321 | ||
322 | ``qemu32``, ``qemu64`` | |
323 | QEMU Virtual CPU version 2.5+ (32 & 64 bit variants) | |
324 | ||
325 | ``qemu64`` is used for x86_64 guests and ``qemu32`` is used for i686 | |
326 | guests, when no ``-cpu`` argument is given to QEMU, or no ``<cpu>`` is | |
327 | provided in libvirt XML. | |
328 | ||
329 | Other non-recommended x86 CPUs | |
330 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
331 | ||
332 | The following CPUs models are compatible with most AMD and Intel x86 | |
333 | hosts, but their usage is discouraged, as they expose a very limited | |
334 | featureset, which prevents guests having optimal performance. | |
335 | ||
336 | ``kvm32``, ``kvm64`` | |
337 | Common KVM processor (32 & 64 bit variants). | |
338 | ||
339 | Legacy models just for historical compatibility with ancient QEMU | |
340 | versions. | |
341 | ||
342 | ``486``, ``athlon``, ``phenom``, ``coreduo``, ``core2duo``, ``n270``, ``pentium``, ``pentium2``, ``pentium3`` | |
343 | Various very old x86 CPU models, mostly predating the introduction | |
344 | of hardware assisted virtualization, that should thus not be | |
345 | required for running virtual machines. | |
346 | ||
347 | ||
348 | Syntax for configuring CPU models | |
349 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
350 | ||
351 | The examples below illustrate the approach to configuring the various | |
352 | CPU models / features in QEMU and libvirt. | |
353 | ||
354 | QEMU command line | |
355 | ^^^^^^^^^^^^^^^^^ | |
356 | ||
357 | Host passthrough: | |
358 | ||
359 | .. parsed-literal:: | |
360 | ||
361 | |qemu_system| -cpu host | |
362 | ||
363 | Host passthrough with feature customization: | |
364 | ||
365 | .. parsed-literal:: | |
366 | ||
367 | |qemu_system| -cpu host,-vmx,... | |
368 | ||
369 | Named CPU models: | |
370 | ||
371 | .. parsed-literal:: | |
372 | ||
373 | |qemu_system| -cpu Westmere | |
374 | ||
375 | Named CPU models with feature customization: | |
376 | ||
377 | .. parsed-literal:: | |
378 | ||
379 | |qemu_system| -cpu Westmere,+pcid,... | |
380 | ||
381 | Libvirt guest XML | |
382 | ^^^^^^^^^^^^^^^^^ | |
383 | ||
384 | Host passthrough:: | |
385 | ||
386 | <cpu mode='host-passthrough'/> | |
387 | ||
388 | Host passthrough with feature customization:: | |
389 | ||
390 | <cpu mode='host-passthrough'> | |
391 | <feature name="vmx" policy="disable"/> | |
392 | ... | |
393 | </cpu> | |
394 | ||
395 | Host model:: | |
396 | ||
397 | <cpu mode='host-model'/> | |
398 | ||
399 | Host model with feature customization:: | |
400 | ||
401 | <cpu mode='host-model'> | |
402 | <feature name="vmx" policy="disable"/> | |
403 | ... | |
404 | </cpu> | |
405 | ||
406 | Named model:: | |
407 | ||
408 | <cpu mode='custom'> | |
409 | <model name="Westmere"/> | |
410 | </cpu> | |
411 | ||
412 | Named model with feature customization:: | |
413 | ||
414 | <cpu mode='custom'> | |
415 | <model name="Westmere"/> | |
416 | <feature name="pcid" policy="require"/> | |
417 | ... | |
418 | </cpu> |