ceph/src/spdk/doc/memory.md

   1 # Memory Management for User Space Drivers {#memory}
   2
   3 The following is an attempt to explain why all data buffers passed to SPDK must
   4 be allocated using spdk_dma_malloc() or its siblings, and why SPDK relies on
   5 DPDK's proven base functionality to implement memory management.
   6
   7 Computing platforms generally carve physical memory up into 4KiB segments
   8 called pages. They number the pages from 0 to N starting from the beginning of
   9 addressable memory. Operating systems then overlay 4KiB virtual memory pages on
  10 top of these physical pages using arbitrarily complex mappings. See
  11 [Virtual Memory](https://en.wikipedia.org/wiki/Virtual_memory) for an overview.
  12
  13 Physical memory is attached on channels, where each memory channel provides
  14 some fixed amount of bandwidth. To optimize total memory bandwidth, the
  15 physical addressing is often set up to automatically interleave between
  16 channels. For instance, page 0 may be located on channel 0, page 1 on channel
  17 1, page 2 on channel 2, etc. This is so that writing to memory sequentially
  18 automatically utilizes all available channels. In practice, interleaving is
  19 done at a much more granular level than a full page.
  20
  21 Modern computing platforms support hardware acceleration for virtual to
  22 physical translation inside of their Memory Management Unit (MMU). The MMU
  23 often supports multiple different page sizes. On recent x86_64 systems, 4KiB,
  24 2MiB, and 1GiB pages are supported. Typically, operating systems use 4KiB pages
  25 by default.
  26
  27 NVMe devices transfer data to and from system memory using Direct Memory Access
  28 (DMA). Specifically, they send messages across the PCI bus requesting data
  29 transfers. In the absence of an IOMMU, these messages contain *physical* memory
  30 addresses. These data transfers happen without involving the CPU, and the MMU
  31 is responsible for making access to memory coherent.
  32
  33 NVMe devices also may place additional requirements on the physical layout of
  34 memory for these transfers. The NVMe 1.0 specification requires all physical
  35 memory to be describable by what is called a *PRP list*. To be described by a
  36 PRP list, memory must have the following properties:
  37
  38 * The memory is broken into physical 4KiB pages, which we'll call device pages.
  39 * The first device page can be a partial page starting at any 4-byte aligned
  40   address. It may extend up to the end of the current physical page, but not
  41   beyond.
  42 * If there is more than one device page, the first device page must end on a
  43   physical 4KiB page boundary.
  44 * The last device page begins on a physical 4KiB page boundary, but is not
  45   required to end on a physical 4KiB page boundary.
  46
  47 The specification allows for device pages to be other sizes than 4KiB, but all
  48 known devices as of this writing use 4KiB.
  49
  50 The NVMe 1.1 specification added support for fully flexible scatter gather lists,
  51 but the feature is optional and most devices available today do not support it.
  52
  53 User space drivers run in the context of a regular process and so have access
  54 to virtual memory. In order to correctly program the device with physical
  55 addresses, some method for address translation must be implemented.
  56
  57 The simplest way to do this on Linux is to inspect `/proc/self/pagemap` from
  58 within a process. This file contains the virtual address to physical address
  59 mappings. As of Linux 4.0, accessing these mappings requires root privileges.
  60 However, operating systems make absolutely no guarantee that the mapping of
  61 virtual to physical pages is static. The operating system has no visibility
  62 into whether a PCI device is directly transferring data to a set of physical
  63 addresses, so great care must be taken to coordinate DMA requests with page
  64 movement. When an operating system flags a page such that the virtual to
  65 physical address mapping cannot be modified, this is called **pinning** the
  66 page.
  67
  68 There are several reasons why the virtual to physical mappings may change, too.
  69 By far the most common reason is due to page swapping to disk. However, the
  70 operating system also moves pages during a process called compaction, which
  71 collapses identical virtual pages onto the same physical page to save memory.
  72 Some operating systems are also capable of doing transparent memory
  73 compression. It is also increasingly possible to hot-add additional memory,
  74 which may trigger a physical address rebalance to optimize interleaving.
  75
  76 POSIX provides the `mlock` call that forces a virtual page of memory to always
  77 be backed by a physical page. In effect, this is disabling swapping. This does
  78 *not* guarantee, however, that the virtual to physical address mapping is
  79 static. The `mlock` call should not be confused with a **pin** call, and it
  80 turns out that POSIX does not define an API for pinning memory. Therefore, the
  81 mechanism to allocate pinned memory is operating system specific.
  82
  83 SPDK relies on DPDK to allocate pinned memory. On Linux, DPDK does this by
  84 allocating `hugepages` (by default, 2MiB). The Linux kernel treats hugepages
  85 differently than regular 4KiB pages. Specifically, the operating system will
  86 never change their physical location. This is not by intent, and so things
  87 could change in future versions, but it is true today and has been for a number
  88 of years (see the later section on the IOMMU for a future-proof solution). DPDK
  89 goes through great pains to allocate hugepages such that it can string together
  90 the longest runs of physical pages possible, such that it can accommodate
  91 physically contiguous allocations larger than a single page.
  92
  93 With this explanation, hopefully it is now clear why all data buffers passed to
  94 SPDK must be allocated using spdk_dma_malloc() or its siblings. The buffers
  95 must be allocated specifically so that they are pinned and so that physical
  96 addresses are known.
  97
  98 # IOMMU Support
  99
 100 Many platforms contain an extra piece of hardware called an I/O Memory
 101 Management Unit (IOMMU). An IOMMU is much like a regular MMU, except it
 102 provides virtualized address spaces to peripheral devices (i.e. on the PCI
 103 bus). The MMU knows about virtual to physical mappings per process on the
 104 system, so the IOMMU associates a particular device with one of these mappings
 105 and then allows the user to assign arbitrary *bus addresses* to virtual
 106 addresses in their process. All DMA operations between the PCI device and
 107 system memory are then translated through the IOMMU by converting the bus
 108 address to a virtual address and then the virtual address to the physical
 109 address. This allows the operating system to freely modify the virtual to
 110 physical address mapping without breaking ongoing DMA operations. Linux
 111 provides a device driver, `vfio-pci`, that allows a user to configure the IOMMU
 112 with their current process.
 113
 114 This is a future-proof, hardware-accelerated solution for performing DMA
 115 operations into and out of a user space process and forms the long-term
 116 foundation for SPDK and DPDK's memory management strategy. We highly recommend
 117 that applications are deployed using vfio and the IOMMU enabled, which is fully
 118 supported today.