4 CPU features are optional features that a CPU of supporting type may
5 choose to implement or not. In QEMU, optional CPU features have
6 corresponding boolean CPU proprieties that, when enabled, indicate
7 that the feature is implemented, and, conversely, when disabled,
8 indicate that it is not implemented. An example of an Arm CPU feature
9 is the Performance Monitoring Unit (PMU). CPU types such as the
10 Cortex-A15 and the Cortex-A57, which respectively implement Arm
11 architecture reference manuals ARMv7-A and ARMv8-A, may both optionally
12 implement PMUs. For example, if a user wants to use a Cortex-A15 without
13 a PMU, then the ``-cpu`` parameter should contain ``pmu=off`` on the QEMU
14 command line, i.e. ``-cpu cortex-a15,pmu=off``.
16 As not all CPU types support all optional CPU features, then whether or
17 not a CPU property exists depends on the CPU type. For example, CPUs
18 that implement the ARMv8-A architecture reference manual may optionally
19 support the AArch32 CPU feature, which may be enabled by disabling the
20 ``aarch64`` CPU property. A CPU type such as the Cortex-A15, which does
21 not implement ARMv8-A, will not have the ``aarch64`` CPU property.
23 QEMU's support may be limited for some CPU features, only partially
24 supporting the feature or only supporting the feature under certain
25 configurations. For example, the ``aarch64`` CPU feature, which, when
26 disabled, enables the optional AArch32 CPU feature, is only supported
27 when using the KVM accelerator and when running on a host CPU type that
28 supports the feature. While ``aarch64`` currently only works with KVM,
29 it could work with TCG. CPU features that are specific to KVM are
30 prefixed with "kvm-" and are described in "KVM VCPU Features".
35 Determining which CPU features are available and functional for a given
36 CPU type is possible with the ``query-cpu-model-expansion`` QMP command.
37 Below are some examples where ``scripts/qmp/qmp-shell`` (see the top comment
38 block in the script for usage) is used to issue the QMP commands.
40 1. Determine which CPU features are available for the ``max`` CPU type
41 (Note, we started QEMU with qemu-system-aarch64, so ``max`` is
42 implementing the ARMv8-A reference manual in this case)::
44 (QEMU) query-cpu-model-expansion type=full model={"name":"max"}
46 "model": { "name": "max", "props": {
47 "sve1664": true, "pmu": true, "sve1792": true, "sve1920": true,
48 "sve128": true, "aarch64": true, "sve1024": true, "sve": true,
49 "sve640": true, "sve768": true, "sve1408": true, "sve256": true,
50 "sve1152": true, "sve512": true, "sve384": true, "sve1536": true,
51 "sve896": true, "sve1280": true, "sve2048": true
54 We see that the ``max`` CPU type has the ``pmu``, ``aarch64``, ``sve``, and many
55 ``sve<N>`` CPU features. We also see that all the CPU features are
56 enabled, as they are all ``true``. (The ``sve<N>`` CPU features are all
57 optional SVE vector lengths (see "SVE CPU Properties"). While with TCG
58 all SVE vector lengths can be supported, when KVM is in use it's more
59 likely that only a few lengths will be supported, if SVE is supported at
62 (2) Let's try to disable the PMU::
64 (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"pmu":false}}
66 "model": { "name": "max", "props": {
67 "sve1664": true, "pmu": false, "sve1792": true, "sve1920": true,
68 "sve128": true, "aarch64": true, "sve1024": true, "sve": true,
69 "sve640": true, "sve768": true, "sve1408": true, "sve256": true,
70 "sve1152": true, "sve512": true, "sve384": true, "sve1536": true,
71 "sve896": true, "sve1280": true, "sve2048": true
74 We see it worked, as ``pmu`` is now ``false``.
76 (3) Let's try to disable ``aarch64``, which enables the AArch32 CPU feature::
78 (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"aarch64":false}}
80 "class": "GenericError", "desc":
81 "'aarch64' feature cannot be disabled unless KVM is enabled and 32-bit EL1 is supported"
84 It looks like this feature is limited to a configuration we do not
87 (4) Let's disable ``sve`` and see what happens to all the optional SVE
90 (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"sve":false}}
92 "model": { "name": "max", "props": {
93 "sve1664": false, "pmu": true, "sve1792": false, "sve1920": false,
94 "sve128": false, "aarch64": true, "sve1024": false, "sve": false,
95 "sve640": false, "sve768": false, "sve1408": false, "sve256": false,
96 "sve1152": false, "sve512": false, "sve384": false, "sve1536": false,
97 "sve896": false, "sve1280": false, "sve2048": false
100 As expected they are now all ``false``.
102 (5) Let's try probing CPU features for the Cortex-A15 CPU type::
104 (QEMU) query-cpu-model-expansion type=full model={"name":"cortex-a15"}
105 {"return": {"model": {"name": "cortex-a15", "props": {"pmu": true}}}}
107 Only the ``pmu`` CPU feature is available.
109 A note about CPU feature dependencies
110 -------------------------------------
112 It's possible for features to have dependencies on other features. I.e.
113 it may be possible to change one feature at a time without error, but
114 when attempting to change all features at once an error could occur
115 depending on the order they are processed. It's also possible changing
116 all at once doesn't generate an error, because a feature's dependencies
117 are satisfied with other features, but the same feature cannot be changed
118 independently without error. For these reasons callers should always
119 attempt to make their desired changes all at once in order to ensure the
122 A note about CPU models and KVM
123 -------------------------------
125 Named CPU models generally do not work with KVM. There are a few cases
126 that do work, e.g. using the named CPU model ``cortex-a57`` with KVM on a
127 seattle host, but mostly if KVM is enabled the ``host`` CPU type must be
128 used. This means the guest is provided all the same CPU features as the
129 host CPU type has. And, for this reason, the ``host`` CPU type should
130 enable all CPU features that the host has by default. Indeed it's even
131 a bit strange to allow disabling CPU features that the host has when using
132 the ``host`` CPU type, but in the absence of CPU models it's the best we can
133 do if we want to launch guests without all the host's CPU features enabled.
135 Enabling KVM also affects the ``query-cpu-model-expansion`` QMP command. The
136 affect is not only limited to specific features, as pointed out in example
137 (3) of "CPU Feature Probing", but also to which CPU types may be expanded.
138 When KVM is enabled, only the ``max``, ``host``, and current CPU type may be
139 expanded. This restriction is necessary as it's not possible to know all
140 CPU types that may work with KVM, but it does impose a small risk of users
141 experiencing unexpected errors. For example on a seattle, as mentioned
142 above, the ``cortex-a57`` CPU type is also valid when KVM is enabled.
143 Therefore a user could use the ``host`` CPU type for the current type, but
144 then attempt to query ``cortex-a57``, however that query will fail with our
145 restrictions. This shouldn't be an issue though as management layers and
146 users have been preferring the ``host`` CPU type for use with KVM for quite
147 some time. Additionally, if the KVM-enabled QEMU instance running on a
148 seattle host is using the ``cortex-a57`` CPU type, then querying ``cortex-a57``
154 After determining which CPU features are available and supported for a
155 given CPU type, then they may be selectively enabled or disabled on the
156 QEMU command line with that CPU type::
158 $ qemu-system-aarch64 -M virt -cpu max,pmu=off,sve=on,sve128=on,sve256=on
160 The example above disables the PMU and enables the first two SVE vector
161 lengths for the ``max`` CPU type. Note, the ``sve=on`` isn't actually
162 necessary, because, as we observed above with our probe of the ``max`` CPU
163 type, ``sve`` is already on by default. Also, based on our probe of
164 defaults, it would seem we need to disable many SVE vector lengths, rather
165 than only enabling the two we want. This isn't the case, because, as
166 disabling many SVE vector lengths would be quite verbose, the ``sve<N>`` CPU
167 properties have special semantics (see "SVE CPU Property Parsing
173 KVM VCPU features are CPU features that are specific to KVM, such as
174 paravirt features or features that enable CPU virtualization extensions.
175 The features' CPU properties are only available when KVM is enabled and
176 are named with the prefix "kvm-". KVM VCPU features may be probed,
177 enabled, and disabled in the same way as other CPU features. Below is
178 the list of KVM VCPU features and their descriptions.
181 By default kvm-no-adjvtime is disabled. This means that by default
182 the virtual time adjustment is enabled (vtime is not *not* adjusted).
184 When virtual time adjustment is enabled each time the VM transitions
185 back to running state the VCPU's virtual counter is updated to
186 ensure stopped time is not counted. This avoids time jumps
187 surprising guest OSes and applications, as long as they use the
188 virtual counter for timekeeping. However it has the side effect of
189 the virtual and physical counters diverging. All timekeeping based
190 on the virtual counter will appear to lag behind any timekeeping
191 that does not subtract VM stopped time. The guest may resynchronize
192 its virtual counter with other time sources as needed.
194 Enable kvm-no-adjvtime to disable virtual time adjustment, also
195 restoring the legacy (pre-5.0) behavior.
198 Since v5.2, kvm-steal-time is enabled by default when KVM is
199 enabled, the feature is supported, and the guest is 64-bit.
201 When kvm-steal-time is enabled a 64-bit guest can account for time
202 its CPUs were not running due to the host not scheduling the
203 corresponding VCPU threads. The accounting statistics may influence
204 the guest scheduler behavior and/or be exposed to the guest
210 TCG VCPU features are CPU features that are specific to TCG.
211 Below is the list of TCG VCPU features and their descriptions.
214 When ``FEAT_Pauth`` is enabled, either the *impdef* (Implementation
215 Defined) algorithm is enabled or the *architected* QARMA algorithm
216 is enabled. By default the impdef algorithm is disabled, and QARMA
219 The architected QARMA algorithm has good cryptographic properties,
220 but can be quite slow to emulate. The impdef algorithm used by QEMU
221 is non-cryptographic but significantly faster.
226 There are two types of SVE CPU properties: ``sve`` and ``sve<N>``. The first
227 is used to enable or disable the entire SVE feature, just as the ``pmu``
228 CPU property completely enables or disables the PMU. The second type
229 is used to enable or disable specific vector lengths, where ``N`` is the
230 number of bits of the length. The ``sve<N>`` CPU properties have special
231 dependencies and constraints, see "SVE CPU Property Dependencies and
232 Constraints" below. Additionally, as we want all supported vector lengths
233 to be enabled by default, then, in order to avoid overly verbose command
234 lines (command lines full of ``sve<N>=off``, for all ``N`` not wanted), we
235 provide the parsing semantics listed in "SVE CPU Property Parsing
238 SVE CPU Property Dependencies and Constraints
239 ---------------------------------------------
241 1) At least one vector length must be enabled when ``sve`` is enabled.
243 2) If a vector length ``N`` is enabled, then, when KVM is enabled, all
244 smaller, host supported vector lengths must also be enabled. If
245 KVM is not enabled, then only all the smaller, power-of-two vector
246 lengths must be enabled. E.g. with KVM if the host supports all
247 vector lengths up to 512-bits (128, 256, 384, 512), then if ``sve512``
248 is enabled, the 128-bit vector length, 256-bit vector length, and
249 384-bit vector length must also be enabled. Without KVM, the 384-bit
250 vector length would not be required.
252 3) If KVM is enabled then only vector lengths that the host CPU type
253 support may be enabled. If SVE is not supported by the host, then
254 no ``sve*`` properties may be enabled.
256 SVE CPU Property Parsing Semantics
257 ----------------------------------
259 1) If SVE is disabled (``sve=off``), then which SVE vector lengths
260 are enabled or disabled is irrelevant to the guest, as the entire
261 SVE feature is disabled and that disables all vector lengths for
262 the guest. However QEMU will still track any ``sve<N>`` CPU
263 properties provided by the user. If later an ``sve=on`` is provided,
264 then the guest will get only the enabled lengths. If no ``sve=on``
265 is provided and there are explicitly enabled vector lengths, then
266 an error is generated.
268 2) If SVE is enabled (``sve=on``), but no ``sve<N>`` CPU properties are
269 provided, then all supported vector lengths are enabled, which when
270 KVM is not in use means including the non-power-of-two lengths, and,
271 when KVM is in use, it means all vector lengths supported by the host
274 3) If SVE is enabled, then an error is generated when attempting to
275 disable the last enabled vector length (see constraint (1) of "SVE
276 CPU Property Dependencies and Constraints").
278 4) If one or more vector lengths have been explicitly enabled and at
279 least one of the dependency lengths of the maximum enabled length
280 has been explicitly disabled, then an error is generated (see
281 constraint (2) of "SVE CPU Property Dependencies and Constraints").
283 5) When KVM is enabled, if the host does not support SVE, then an error
284 is generated when attempting to enable any ``sve*`` properties (see
285 constraint (3) of "SVE CPU Property Dependencies and Constraints").
287 6) When KVM is enabled, if the host does support SVE, then an error is
288 generated when attempting to enable any vector lengths not supported
289 by the host (see constraint (3) of "SVE CPU Property Dependencies and
292 7) If one or more ``sve<N>`` CPU properties are set ``off``, but no ``sve<N>``,
293 CPU properties are set ``on``, then the specified vector lengths are
294 disabled but the default for any unspecified lengths remains enabled.
295 When KVM is not enabled, disabling a power-of-two vector length also
296 disables all vector lengths larger than the power-of-two length.
297 When KVM is enabled, then disabling any supported vector length also
298 disables all larger vector lengths (see constraint (2) of "SVE CPU
299 Property Dependencies and Constraints").
301 8) If one or more ``sve<N>`` CPU properties are set to ``on``, then they
302 are enabled and all unspecified lengths default to disabled, except
303 for the required lengths per constraint (2) of "SVE CPU Property
304 Dependencies and Constraints", which will even be auto-enabled if
305 they were not explicitly enabled.
307 9) If SVE was disabled (``sve=off``), allowing all vector lengths to be
308 explicitly disabled (i.e. avoiding the error specified in (3) of
309 "SVE CPU Property Parsing Semantics"), then if later an ``sve=on`` is
310 provided an error will be generated. To avoid this error, one must
311 enable at least one vector length prior to enabling SVE.
313 SVE CPU Property Examples
314 -------------------------
318 $ qemu-system-aarch64 -M virt -cpu max,sve=off
320 2) Implicitly enable all vector lengths for the ``max`` CPU type::
322 $ qemu-system-aarch64 -M virt -cpu max
324 3) When KVM is enabled, implicitly enable all host CPU supported vector
325 lengths with the ``host`` CPU type::
327 $ qemu-system-aarch64 -M virt,accel=kvm -cpu host
329 4) Only enable the 128-bit vector length::
331 $ qemu-system-aarch64 -M virt -cpu max,sve128=on
333 5) Disable the 512-bit vector length and all larger vector lengths,
334 since 512 is a power-of-two. This results in all the smaller,
335 uninitialized lengths (128, 256, and 384) defaulting to enabled::
337 $ qemu-system-aarch64 -M virt -cpu max,sve512=off
339 6) Enable the 128-bit, 256-bit, and 512-bit vector lengths::
341 $ qemu-system-aarch64 -M virt -cpu max,sve128=on,sve256=on,sve512=on
343 7) The same as (6), but since the 128-bit and 256-bit vector
344 lengths are required for the 512-bit vector length to be enabled,
345 then allow them to be auto-enabled::
347 $ qemu-system-aarch64 -M virt -cpu max,sve512=on
349 8) Do the same as (7), but by first disabling SVE and then re-enabling it::
351 $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve512=on,sve=on
353 9) Force errors regarding the last vector length::
355 $ qemu-system-aarch64 -M virt -cpu max,sve128=off
356 $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve128=off,sve=on
358 SVE CPU Property Recommendations
359 --------------------------------
361 The examples in "SVE CPU Property Examples" exhibit many ways to select
362 vector lengths which developers may find useful in order to avoid overly
363 verbose command lines. However, the recommended way to select vector
364 lengths is to explicitly enable each desired length. Therefore only
365 example's (1), (4), and (6) exhibit recommended uses of the properties.
367 SME CPU Property Examples
368 -------------------------
372 $ qemu-system-aarch64 -M virt -cpu max,sme=off
374 2) Implicitly enable all vector lengths for the ``max`` CPU type::
376 $ qemu-system-aarch64 -M virt -cpu max
378 3) Only enable the 256-bit vector length::
380 $ qemu-system-aarch64 -M virt -cpu max,sme256=on
382 3) Enable the 256-bit and 1024-bit vector lengths::
384 $ qemu-system-aarch64 -M virt -cpu max,sme256=on,sme1024=on
386 4) Disable the 512-bit vector length. This results in all the other
387 lengths supported by ``max`` defaulting to enabled
388 (128, 256, 1024 and 2048)::
390 $ qemu-system-aarch64 -M virt -cpu max,sve512=off
392 SVE User-mode Default Vector Length Property
393 --------------------------------------------
395 For qemu-aarch64, the cpu property ``sve-default-vector-length=N`` is
396 defined to mirror the Linux kernel parameter file
397 ``/proc/sys/abi/sve_default_vector_length``. The default length, ``N``,
398 is in units of bytes and must be between 16 and 8192.
399 If not specified, the default vector length is 64.
401 If the default length is larger than the maximum vector length enabled,
402 the actual vector length will be reduced. Note that the maximum vector
403 length supported by QEMU is 256.
405 If this property is set to ``-1`` then the default vector length
406 is set to the maximum possible length.
411 The SME CPU properties are much like the SVE properties: ``sme`` is
412 used to enable or disable the entire SME feature, and ``sme<N>`` is
413 used to enable or disable specific vector lengths. Finally,
414 ``sme_fa64`` is used to enable or disable ``FEAT_SME_FA64``, which
415 allows execution of the "full a64" instruction set while Streaming
418 SME is not supported by KVM at this time.
420 At least one vector length must be enabled when ``sme`` is enabled,
421 and all vector lengths must be powers of 2. The maximum vector
422 length supported by qemu is 2048 bits. Otherwise, there are no
423 additional constraints on the set of vector lengths supported by SME.
425 SME User-mode Default Vector Length Property
426 --------------------------------------------
428 For qemu-aarch64, the cpu property ``sme-default-vector-length=N`` is
429 defined to mirror the Linux kernel parameter file
430 ``/proc/sys/abi/sme_default_vector_length``. The default length, ``N``,
431 is in units of bytes and must be between 16 and 8192.
432 If not specified, the default vector length is 32.
434 As with ``sve-default-vector-length``, if the default length is larger
435 than the maximum vector length enabled, the actual vector length will
436 be reduced. If this property is set to ``-1`` then the default vector
437 length is set to the maximum possible length.