docs/system/ppc/pseries.rst

   1 ===================================
   2 pSeries family boards (``pseries``)
   3 ===================================
   4
   5 The Power machine para-virtualized environment described by the Linux on Power
   6 Architecture Reference ([LoPAR]_) document is called pSeries. This environment
   7 is also known as sPAPR, System p guests, or simply Power Linux guests (although
   8 it is capable of running other operating systems, such as AIX).
   9
  10 Even though pSeries is designed to behave as a guest environment, it is also
  11 capable of acting as a hypervisor OS, providing, on that role, nested
  12 virtualization capabilities.
  13
  14 Supported devices
  15 =================
  16
  17  * Multi processor support for many Power processors generations: POWER7,
  18    POWER7+, POWER8, POWER8NVL, POWER9, and Power10. Support for POWER5+ exists,
  19    but its state is unknown.
  20  * Interrupt Controller, XICS (POWER8) and XIVE (POWER9 and Power10)
  21  * vPHB PCIe Host bridge.
  22  * vscsi and vnet devices, compatible with the same devices available on a
  23    PowerVM hypervisor with VIOS managing LPARs.
  24  * Virtio based devices.
  25  * PCIe device pass through.
  26
  27 Missing devices
  28 ===============
  29
  30  * SPICE support.
  31
  32 Firmware
  33 ========
  34
  35 `SLOF <https://github.com/aik/SLOF>`_ (Slimline Open Firmware) is an
  36 implementation of the `IEEE 1275-1994, Standard for Boot (Initialization
  37 Configuration) Firmware: Core Requirements and Practices
  38 <https://standards.ieee.org/standard/1275-1994.html>`_.
  39
  40 QEMU includes a prebuilt image of SLOF which is updated when a more recent
  41 version is required.
  42
  43 Build directions
  44 ================
  45
  46 .. code-block:: bash
  47
  48   ./configure --target-list=ppc64-softmmu && make
  49
  50 Running instructions
  51 ====================
  52
  53 Someone can select the pSeries machine type by running QEMU with the following
  54 options:
  55
  56 .. code-block:: bash
  57
  58   qemu-system-ppc64 -M pseries <other QEMU arguments>
  59
  60 sPAPR devices
  61 =============
  62
  63 The sPAPR specification defines a set of para-virtualized devices, which are
  64 also supported by the pSeries machine in QEMU and can be instantiated with the
  65 ``-device`` option:
  66
  67 * ``spapr-vlan`` : a virtual network interface.
  68 * ``spapr-vscsi`` : a virtual SCSI disk interface.
  69 * ``spapr-rng`` : a pseudo-device for passing random number generator data to the
  70   guest (see the `H_RANDOM hypercall feature
  71   <https://wiki.qemu.org/Features/HRandomHypercall>`_ for details).
  72 * ``spapr-vty``: a virtual teletype.
  73 * ``spapr-pci-host-bridge``: a PCI host bridge.
  74 * ``tpm-spapr``: a Trusted Platform Module (TPM).
  75 * ``spapr-tpm-proxy``: a TPM proxy.
  76
  77 These are compatible with the devices historically available for use when
  78 running the IBM PowerVM hypervisor with LPARs.
  79
  80 However, since these devices have originally been specified with another
  81 hypervisor and non-Linux guests in mind, you should use the virtio counterparts
  82 (virtio-net, virtio-blk/scsi and virtio-rng for instance) if possible instead,
  83 since they will most probably give you better performance with Linux guests in a
  84 QEMU environment.
  85
  86 The pSeries machine in QEMU is always instantiated with the following devices:
  87
  88 * A NVRAM device (``spapr-nvram``).
  89 * A virtual teletype (``spapr-vty``).
  90 * A PCI host bridge (``spapr-pci-host-bridge``).
  91
  92 Hence, it is not needed to add them manually, unless you use the ``-nodefaults``
  93 command line option in QEMU.
  94
  95 In the case of the default ``spapr-nvram`` device, if someone wants to make the
  96 contents of the NVRAM device persistent, they will need to specify a PFLASH
  97 device when starting QEMU, i.e. either use
  98 ``-drive if=pflash,file=<filename>,format=raw`` to set the default PFLASH
  99 device, or specify one with an ID
 100 (``-drive if=none,file=<filename>,format=raw,id=pfid``) and pass that ID to the
 101 NVRAM device with ``-global spapr-nvram.drive=pfid``.
 102
 103 sPAPR specification
 104 -------------------
 105
 106 The main source of documentation on the sPAPR standard is the [LoPAR]_ document.
 107 However, documentation specific to QEMU's implementation of the specification
 108 can  also be found in QEMU documentation:
 109
 110 .. toctree::
 111    :maxdepth: 1
 112
 113    ../../specs/ppc-spapr-hotplug.rst
 114    ../../specs/ppc-spapr-hcalls.rst
 115    ../../specs/ppc-spapr-numa.rst
 116    ../../specs/ppc-spapr-uv-hcalls.rst
 117    ../../specs/ppc-spapr-xive.rst
 118
 119 Switching between the KVM-PR and KVM-HV kernel module
 120 =====================================================
 121
 122 Currently, there are two implementations of KVM on Power, ``kvm_hv.ko`` and
 123 ``kvm_pr.ko``.
 124
 125
 126 If a host supports both KVM modes, and both KVM kernel modules are loaded, it is
 127 possible to switch between the two modes with the ``kvm-type`` parameter:
 128
 129 * Use ``qemu-system-ppc64 -M pseries,accel=kvm,kvm-type=PR`` to use the
 130   ``kvm_pr.ko`` kernel module.
 131 * Use ``qemu-system-ppc64 -M pseries,accel=kvm,kvm-type=HV`` to use ``kvm_hv.ko``
 132   instead.
 133
 134 KVM-PR
 135 ------
 136
 137 KVM-PR uses the so-called **PR**\ oblem state of the PPC CPUs to run the guests,
 138 i.e. the virtual machine is run in user mode and all privileged instructions
 139 trap and have to be emulated by the host. That means you can run KVM-PR inside
 140 a pSeries guest (or a PowerVM LPAR for that matter), and that is where it has
 141 originated, as historically (prior to POWER7) it was not possible to run Linux
 142 on hypervisor mode on a Power processor (this function was restricted to
 143 PowerVM, the IBM proprietary hypervisor).
 144
 145 Because all privileged instructions are trapped, guests that use a lot of
 146 privileged instructions run quite slow with KVM-PR. On the other hand, because
 147 of that, this kernel module can run on pretty much every PPC hardware, and is
 148 able to emulate a lot of guests CPUs. This module can even be used to run other
 149 PowerPC guests like an emulated PowerMac.
 150
 151 As KVM-PR can be run inside a pSeries guest, it can also provide nested
 152 virtualization capabilities (i.e. running a guest from within a guest).
 153
 154 It is important to notice that, as KVM-HV provides a much better execution
 155 performance, maintenance work has been much more focused on it in the past
 156 years. Maintenance for KVM-PR has been minimal.
 157
 158 In order to run KVM-PR guests with POWER9 processors, someone will need to start
 159 QEMU with ``kernel_irqchip=off`` command line option.
 160
 161 KVM-HV
 162 ------
 163
 164 KVM-HV uses the hypervisor mode of more recent Power processors, that allow
 165 access to the bare metal hardware directly. Although POWER7 had this capability,
 166 it was only starting with POWER8 that this was officially supported by IBM.
 167
 168 Originally, KVM-HV was only available when running on a PowerNV platform (a.k.a.
 169 Power bare metal). Although it runs on a PowerNV platform, it can only be used
 170 to start pSeries guests. As the pSeries guest doesn't have access to the
 171 hypervisor mode of the Power CPU, it wasn't possible to run KVM-HV on a guest.
 172 This limitation has been lifted, and now it is possible to run KVM-HV inside
 173 pSeries guests as well, making nested virtualization possible with KVM-HV.
 174
 175 As KVM-HV has access to privileged instructions, guests that use a lot of these
 176 can run much faster than with KVM-PR. On the other hand, the guest CPU has to be
 177 of the same type as the host CPU this way, e.g. it is not possible to specify an
 178 embedded PPC CPU for the guest with KVM-HV. However, there is at least the
 179 possibility to run the guest in a backward-compatibility mode of the previous
 180 CPUs generations, e.g. you can run a POWER7 guest on a POWER8 host by using
 181 ``-cpu POWER8,compat=power7`` as parameter to QEMU.
 182
 183 Modules support
 184 ===============
 185
 186 As noticed in the sections above, each module can run in a different
 187 environment. The following table shows with which environment each module can
 188 run. As long as you are in a supported environment, you can run KVM-PR or KVM-HV
 189 nested. Combinations not shown in the table are not available.
 190
 191 +--------------+------------+------+-------------------+----------+--------+
 192 | Platform     | Host type  | Bits | Page table format | KVM-HV   | KVM-PR |
 193 +==============+============+======+===================+==========+========+
 194 | PowerNV      | bare metal | 32   | hash              | no       | yes    |
 195 |              |            |      +-------------------+----------+--------+
 196 |              |            |      | radix             | N/A      | N/A    |
 197 |              |            +------+-------------------+----------+--------+
 198 |              |            | 64   | hash              | yes      | yes    |
 199 |              |            |      +-------------------+----------+--------+
 200 |              |            |      | radix             | yes      | no     |
 201 +--------------+------------+------+-------------------+----------+--------+
 202 | pSeries [1]_ | PowerNV    | 32   | hash              | no       | yes    |
 203 |              |            |      +-------------------+----------+--------+
 204 |              |            |      | radix             | N/A      | N/A    |
 205 |              |            +------+-------------------+----------+--------+
 206 |              |            | 64   | hash              | no       | yes    |
 207 |              |            |      +-------------------+----------+--------+
 208 |              |            |      | radix             | yes [2]_ | no     |
 209 |              +------------+------+-------------------+----------+--------+
 210 |              | PowerVM    | 32   | hash              | no       | yes    |
 211 |              |            |      +-------------------+----------+--------+
 212 |              |            |      | radix             | N/A      | N/A    |
 213 |              |            +------+-------------------+----------+--------+
 214 |              |            | 64   | hash              | no       | yes    |
 215 |              |            |      +-------------------+----------+--------+
 216 |              |            |      | radix [3]_        | no       | yes    |
 217 +--------------+------------+------+-------------------+----------+--------+
 218
 219 .. [1] On POWER9 DD2.1 processors, the page table format on the host and guest
 220    must be the same.
 221
 222 .. [2] KVM-HV cannot run nested on POWER8 machines.
 223
 224 .. [3] Introduced on Power10 machines.
 225
 226
 227 .. _power-papr-protected-execution-facility-pef:
 228
 229 POWER (PAPR) Protected Execution Facility (PEF)
 230 -----------------------------------------------
 231
 232 Protected Execution Facility (PEF), also known as Secure Guest support
 233 is a feature found on IBM POWER9 and POWER10 processors.
 234
 235 If a suitable firmware including an Ultravisor is installed, it adds
 236 an extra memory protection mode to the CPU.  The ultravisor manages a
 237 pool of secure memory which cannot be accessed by the hypervisor.
 238
 239 When this feature is enabled in QEMU, a guest can use ultracalls to
 240 enter "secure mode".  This transfers most of its memory to secure
 241 memory, where it cannot be eavesdropped by a compromised hypervisor.
 242
 243 Launching
 244 ^^^^^^^^^
 245
 246 To launch a guest which will be permitted to enter PEF secure mode::
 247
 248   $ qemu-system-ppc64 \
 249       -object pef-guest,id=pef0 \
 250       -machine confidential-guest-support=pef0 \
 251       ...
 252
 253 Live Migration
 254 ^^^^^^^^^^^^^^
 255
 256 Live migration is not yet implemented for PEF guests.  For
 257 consistency, QEMU currently prevents migration if the PEF feature is
 258 enabled, whether or not the guest has actually entered secure mode.
 259
 260
 261 Maintainer contact information
 262 ==============================
 263
 264 Cédric Le Goater <clg@kaod.org>
 265
 266 Daniel Henrique Barboza <danielhb413@gmail.com>
 267
 268 .. [LoPAR] `Linux on Power Architecture Reference document (LoPAR) revision
 269    2.9 <https://openpowerfoundation.org/wp-content/uploads/2020/07/LoPAR-20200812.pdf>`_.