ceph/src/spdk/CHANGELOG.md

   1 # Changelog
   2
   3 ## v17.03: Blobstore and userspace vhost-scsi target
   4
   5 ### Blobstore and BlobFS
   6
   7 The blobstore is a persistent, power-fail safe block allocator designed to be
   8 used as the local storage system backing a higher-level storage service.
   9 See the [blobstore documentation](http://www.spdk.io/doc/blob.html) for more details.
  10
  11 BlobFS adds basic filesystem functionality like filenames on top of the blobstore.
  12 This release also includes a RocksDB Env implementation using BlobFS in place of the
  13 kernel filesystem.
  14 See the [BlobFS documentation](http://www.spdk.io/doc/blobfs.html) for more details.
  15
  16 ### Userspace vhost-scsi target
  17
  18 A userspace implementation of the QEMU vhost-scsi protocol has been added.
  19 The vhost target is capable of exporting SPDK bdevs to QEMU-based VMs as virtio devices.
  20 See the [vhost documentation](http://www.spdk.io/doc/vhost.html) for more details.
  21
  22 ### Event framework
  23
  24 The overhead of the main reactor event loop was reduced by optimizing the number of
  25 calls to spdk_get_ticks() per iteration.
  26
  27 ### NVMe library
  28
  29 The NVMe library will now automatically split readv/writev requests with scatter-gather
  30 lists that do not map to valid PRP lists when the NVMe controller does not natively
  31 support SGLs.
  32
  33 The `identify` and `perf` NVMe examples were modified to add a consistent format for
  34 specifying remote NVMe over Fabrics devices via the `-r` option.
  35 This is implemented using the new `spdk_nvme_transport_id_parse()` function.
  36
  37 ### iSCSI Target
  38
  39 The [Nvme] section of the configuration file was modified to remove the `BDF` directive
  40 and replace it with a `TransportID` directive. Both local (PCIe) and remote (NVMe-oF)
  41 devices can now be specified as the backing block device. A script to generate an
  42 entire [Nvme] section based on the local NVMe devices attached was added at
  43 `scripts/gen_nvme.sh`.
  44
  45 ### NVMe-oF Target
  46
  47 The [Nvme] section of the configuration file was modified to remove the `BDF` directive
  48 and replace it with a `TransportID` directive. Both local (PCIe) and remote (NVMe-oF)
  49 devices can now be specified as the backing block device. A script to generate an
  50 entire [Nvme] section based on the local NVMe devices attached was added at
  51 `scripts/gen_nvme.sh`.
  52
  53 ## v16.12: NVMe over Fabrics host, hotplug, and multi-process
  54
  55 ### NVMe library
  56
  57 The NVMe library has been changed to create its own request memory pool rather than
  58 requiring the user to initialize the global `request_mempool` variable.  Apps can be
  59 updated by simply removing the initialization of `request_mempool`.  Since the NVMe
  60 library user no longer needs to know the size of the internal NVMe request
  61 structure to create the pool, the `spdk_nvme_request_size()` function was also removed.
  62
  63 The `spdk_nvme_ns_cmd_deallocate()` function was renamed and extended to become
  64 `spdk_nvme_ns_cmd_dataset_management()`, which allows access to all of the NVMe
  65 Dataset Management command's parameters.  Existing callers can be updated to use
  66 `spdk_nvme_ns_cmd_dataset_management()` with `SPDK_NVME_DSM_ATTR_DEALLOCATE` as the
  67 `type` parameter.
  68
  69 The NVMe library SGL callback prototype has been changed to return virtual addresses
  70 rather than physical addresses.  Callers of `spdk_nvme_ns_cmd_readv()` and
  71 `spdk_nvme_ns_cmd_writev()` must update their `next_sge_fn` callbacks to match.
  72
  73 The NVMe library now supports NVMe over Fabrics devices in addition to the existing
  74 support for local PCIe-attached NVMe devices.  For an example of how to enable
  75 NVMe over Fabrics support in an application, see `examples/nvme/identify` and
  76 `examples/nvme/perf`.
  77
  78 Hot insert/remove support for NVMe devices has been added.  To enable NVMe hotplug
  79 support, an application should call the `spdk_nvme_probe()` function on a regular
  80 basis to probe for new devices (reported via the existing `probe_cb` callback) and
  81 removed devices (reported via a new `remove_cb` callback).  Hotplug is currently
  82 only supported on Linux with the `uio_pci_generic` driver, and newly-added NVMe
  83 devices must be bound to `uio_pci_generic` by an external script or tool.
  84
  85 Multiple processes may now coordinate and use a single NVMe device simultaneously
  86 using [DPDK Multi-process Support](http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html).
  87
  88 ### NVMe over Fabrics target (`nvmf_tgt`)
  89
  90 The `nvmf_tgt` configuration file format has been updated significantly to enable
  91 new features.  See the example configuration file `etc/spdk/nvmf.conf.in` for
  92 more details on the new and changed options.
  93
  94 The NVMe over Fabrics target now supports virtual mode subsystems, which allow the
  95 user to export devices from the SPDK block device abstraction layer as NVMe over
  96 Fabrics subsystems.  Direct mode (raw NVMe device access) is also still supported,
  97 and a single `nvmf_tgt` may export both types of subsystems simultaneously.
  98
  99 ### Block device abstraction layer (bdev)
 100
 101 The bdev layer now supports scatter/gather read and write I/O APIs, and the NVMe
 102 blockdev driver has been updated to support scatter/gather.  Apps can use the
 103 new scatter/gather support via the `spdk_bdev_readv()` and `spdk_bdev_writev()`
 104 functions.
 105
 106 The bdev status returned from each I/O has been extended to pass through NVMe
 107 or SCSI status codes directly in cases where the underlying device can provide
 108 a more specific status code.
 109
 110 A Ceph RBD (RADOS Block Device) blockdev driver has been added.  This allows the
 111 `iscsi_tgt` and `nvmf_tgt` apps to export Ceph RBD volumes as iSCSI LUNs or
 112 NVMe namespaces.
 113
 114 ### General changes
 115
 116 `libpciaccess` has been removed as a dependency and DPDK PCI enumeration is
 117 used instead. Prior to DPDK 16.07 enumeration by class code was not supported,
 118 so for earlier DPDK versions, only Intel SSD DC P3x00 devices will be discovered
 119 by the NVMe library.
 120
 121 The `env` environment abstraction library has been introduced, and a default
 122 DPDK-based implementation is provided as part of SPDK.  The goal of the `env`
 123 layer is to enable use of alternate user-mode memory allocation and PCI access
 124 libraries.  See `doc/porting.md` for more details.
 125
 126 The build process has been modified to produce all of the library files in the
 127 `build/lib` directory.  This is intended to simplify the use of SPDK from external
 128 projects, which can now link to SPDK libraries by adding the `build/lib` directory
 129 to the library path via `-L` and linking the SPDK libraries by name (for example,
 130 `-lspdk_nvme -lspdk_log -lspdk_util`).
 131
 132 `nvmf_tgt` and `iscsi_tgt` now have a JSON-RPC interface, which allows the user
 133 to query and modify the configuration at runtime.  The RPC service is disabled by
 134 default, since it currently does not provide any authentication or security
 135 mechanisms; it should only be enabled on systems with controlled user access
 136 behind a firewall. An example RPC client implemented in Python is provided in
 137 `scripts/rpc.py`.
 138
 139 ## v16.08: iSCSI target, NVMe over Fabrics maturity
 140
 141 This release adds a userspace iSCSI target. The iSCSI target is capable of exporting
 142 NVMe devices over a network using the iSCSI protocol. The application is located
 143 in app/iscsi_tgt and a documented configuration file can be found at etc/spdk/spdk.conf.in.
 144
 145 This release also significantly improves the existing NVMe over Fabrics target.
 146   - The configuration file format was changed, which will require updates to
 147     any existing nvmf.conf files (see `etc/spdk/nvmf.conf.in`):
 148     - `SubsystemGroup` was renamed to `Subsystem`.
 149     - `AuthFile` was removed (it was unimplemented).
 150     - `nvmf_tgt` was updated to correctly recognize NQN (NVMe Qualified Names)
 151       when naming subsystems.  The default node name was changed to reflect this;
 152       it is now "nqn.2016-06.io.spdk".
 153     - `Port` and `Host` sections were merged into the `Subsystem` section
 154     - Global options to control max queue depth, number of queues, max I/O
 155       size, and max in-capsule data size were added.
 156     - The Nvme section was removed. Now a list of devices is specified by
 157       bus/device/function directly in the Subsystem section.
 158     - Subsystems now have a Mode, which can be Direct or Virtual. This is an attempt
 159       to future-proof the interface, so the only mode supported by this release
 160       is "Direct".
 161   - Many bug fixes and cleanups were applied to the `nvmf_tgt` app and library.
 162   - The target now supports discovery.
 163
 164 This release also adds one new feature and provides some better examples and tools
 165 for the NVMe driver.
 166   - The Weighted Round Robin arbitration method is now supported. This allows
 167     the user to specify different priorities on a per-I/O-queue basis.  To
 168     enable WRR, set the `arb_mechanism` field during `spdk_nvme_probe()`.
 169   - A simplified "Hello World" example was added to show the proper way to use
 170     the NVMe library API; see `examples/nvme/hello_world/hello_world.c`.
 171   - A test for measuring software overhead was added. See `test/lib/nvme/overhead`.
 172
 173 ## v16.06: NVMf userspace target
 174
 175 This release adds a userspace NVMf (NVMe over Fabrics) target, conforming to the
 176 newly-released NVMf 1.0/NVMe 1.2.1 specification.  The NVMf target exports NVMe
 177 devices from a host machine over the network via RDMA.  Currently, the target is
 178 limited to directly exporting physical NVMe devices, and the discovery subsystem
 179 is not supported.
 180
 181 This release includes a general API cleanup, including renaming all declarations
 182 in public headers to include a `spdk` prefix to prevent namespace clashes with
 183 user code.
 184
 185 - NVMe
 186   - The `nvme_attach()` API was reworked into a new probe/attach model, which
 187   moves device detection into the NVMe library.  The new API also allows
 188   parallel initialization of NVMe controllers, providing a major reduction in
 189   startup time when using multiple controllers.
 190   - I/O queue allocation was changed to be explicit in the API.  Each function
 191   that generates I/O requests now takes a queue pair (`spdk_nvme_qpair *`)
 192   argument, and I/O queues may be allocated using
 193   `spdk_nvme_ctrlr_alloc_io_qpair()`.  This allows more flexible assignment of
 194   queue pairs than the previous model, which only allowed a single queue
 195   per thread and limited the total number of I/O queues to the lowest number
 196   supported on any attached controller.
 197   - Added support for the Write Zeroes command.
 198   - `examples/nvme/perf` can now report I/O command latency from the
 199    the controller's viewpoint using the Intel vendor-specific read/write latency
 200    log page.
 201   - Added namespace reservation command support, which can be used to coordinate
 202   sharing of a namespace between multiple hosts.
 203   - Added hardware SGL support, which enables use of scattered buffers that
 204    don't conform to the PRP list alignment and length requirements on supported
 205    NVMe controllers.
 206   - Added end-to-end data protection support, including the ability to write and
 207   read metadata in extended LBA (metadata appended to each block of data in the
 208   buffer) and separate metadata buffer modes.
 209   See `spdk_nvme_ns_cmd_write_with_md()` and `spdk_nvme_ns_cmd_read_with_md()`
 210   for details.
 211 - IOAT
 212   - The DMA block fill feature is now exposed via the `ioat_submit_fill()`
 213   function.  This is functionally similar to `memset()`, except the memory is
 214   filled with an 8-byte repeating pattern instead of a single byte like memset.
 215 - PCI
 216   - Added support for using DPDK for PCI device mapping in addition to the
 217   existing libpciaccess option.  Using the DPDK PCI support also allows use of
 218   the Linux VFIO driver model, which means that SPDK userspace drivers will work
 219   with the IOMMU enabled.  Additionally, SPDK applications may be run as an
 220   unprivileged user with access restricted to a specific set of PCIe devices.
 221   - The PCI library API was made more generic to abstract away differences
 222   between the underlying PCI access implementations.
 223
 224 ## v1.2.0: IOAT user-space driver
 225
 226 This release adds a user-space driver with support for the Intel I/O Acceleration Technology (I/OAT, also known as "Crystal Beach") DMA offload engine.
 227
 228 - IOAT
 229   - New user-space driver supporting DMA memory copy offload
 230   - Example programs `ioat/perf` and `ioat/verify`
 231   - Kernel-mode DMA engine test driver `kperf` for performance comparison
 232 - NVMe
 233   - Per-I/O flags for Force Unit Access (FUA) and Limited Retry
 234   - Public API for retrieving log pages
 235   - Reservation register/acquire/release/report command support
 236   - Scattered payload support - an alternate API to provide I/O buffers via a sequence of callbacks
 237   - Declarations and `nvme/identify` support for Intel SSD DC P3700 series vendor-specific log pages and features
 238 - Updated to support DPDK 2.2.0
 239
 240
 241 ## v1.0.0: NVMe user-space driver
 242
 243 This is the initial open source release of the Storage Performance Development Kit (SPDK).
 244
 245 Features:
 246 - NVMe user-space driver
 247 - NVMe example programs
 248   - `examples/nvme/perf` tests performance (IOPS) using the NVMe user-space driver
 249   - `examples/nvme/identify` displays NVMe controller information in a human-readable format
 250 - Linux and FreeBSD support