1 =======================
3 =======================
5 See `Block Device`_ for additional details.
10 ``rbd compression hint``
12 :Description: Hint to send to the OSDs on write operations. If set to `compressible` and the OSD `bluestore compression mode` setting is `passive`, the OSD will attempt to compress the data. If set to `incompressible` and the OSD compression setting is `aggressive`, the OSD will not attempt to compress the data.
16 :Values: ``none``, ``compressible``, ``incompressible``
19 =======================
21 .. sidebar:: Kernel Caching
23 The kernel driver for Ceph block devices can use the Linux page cache to
26 The user space implementation of the Ceph block device (i.e., ``librbd``) cannot
27 take advantage of the Linux page cache, so it includes its own in-memory
28 caching, called "RBD caching." RBD caching behaves just like well-behaved hard
29 disk caching. When the OS sends a barrier or a flush request, all dirty data is
30 written to the OSDs. This means that using write-back caching is just as safe as
31 using a well-behaved physical hard disk with a VM that properly sends flushes
32 (i.e. Linux kernel >= 2.6.32). The cache uses a Least Recently Used (LRU)
33 algorithm, and in write-back mode it can coalesce contiguous requests for
36 .. versionadded:: 0.46
38 Ceph supports write-back caching for RBD. To enable it, add ``rbd cache =
39 true`` to the ``[client]`` section of your ``ceph.conf`` file. By default
40 ``librbd`` does not perform any caching. Writes and reads go directly to the
41 storage cluster, and writes return only when the data is on disk on all
42 replicas. With caching enabled, writes return immediately, unless there are more
43 than ``rbd cache max dirty`` unflushed bytes. In this case, the write triggers
44 writeback and blocks until enough bytes are flushed.
46 .. versionadded:: 0.47
48 Ceph supports write-through caching for RBD. You can set the size of
49 the cache, and you can set targets and limits to switch from
50 write-back caching to write through caching. To enable write-through
51 mode, set ``rbd cache max dirty`` to 0. This means writes return only
52 when the data is on disk on all replicas, but reads may come from the
53 cache. The cache is in memory on the client, and each RBD image has
54 its own. Since the cache is local to the client, there's no coherency
55 if there are others accessing the image. Running GFS or OCFS on top of
56 RBD will not work with caching enabled.
58 The ``ceph.conf`` file settings for RBD should be set in the ``[client]``
59 section of your configuration file. The settings include:
64 :Description: Enable caching for RADOS Block Device (RBD).
72 :Description: The RBD cache size in bytes.
78 ``rbd cache max dirty``
80 :Description: The ``dirty`` limit in bytes at which the cache triggers write-back. If ``0``, uses write-through caching.
83 :Constraint: Must be less than ``rbd cache size``.
87 ``rbd cache target dirty``
89 :Description: The ``dirty target`` before the cache begins writing data to the data storage. Does not block writes to the cache.
92 :Constraint: Must be less than ``rbd cache max dirty``.
96 ``rbd cache max dirty age``
98 :Description: The number of seconds dirty data is in the cache before writeback starts.
103 .. versionadded:: 0.60
105 ``rbd cache writethrough until flush``
107 :Description: Start out in write-through mode, and switch to write-back after the first flush request is received. Enabling this is a conservative but safe setting in case VMs running on rbd are too old to send flushes, like the virtio driver in Linux before 2.6.32.
112 .. _Block Device: ../../rbd
116 =======================
118 .. versionadded:: 0.86
120 RBD supports read-ahead/prefetching to optimize small, sequential reads.
121 This should normally be handled by the guest OS in the case of a VM,
122 but boot loaders may not issue efficient reads.
123 Read-ahead is automatically disabled if caching is disabled.
126 ``rbd readahead trigger requests``
128 :Description: Number of sequential read requests necessary to trigger read-ahead.
134 ``rbd readahead max bytes``
136 :Description: Maximum size of a read-ahead request. If zero, read-ahead is disabled.
137 :Type: 64-bit Integer
139 :Default: ``512 KiB``
142 ``rbd readahead disable after bytes``
144 :Description: After this many bytes have been read from an RBD image, read-ahead is disabled for that image until it is closed. This allows the guest OS to take over read-ahead once it is booted. If zero, read-ahead stays enabled.
145 :Type: 64-bit Integer
153 RBD supports advanced features which can be specified via the command line when creating images or the default features can be specified via Ceph config file via 'rbd_default_features = <sum of feature numeric values>' or 'rbd_default_features = <comma-delimited list of CLI values>'
157 :Description: Layering enables you to use cloning.
160 :Added in: v0.70 (Emperor)
161 :KRBD support: since v3.10
166 :Description: Striping spreads data across multiple objects. Striping helps with parallelism for sequential read/write workloads.
169 :Added in: v0.70 (Emperor)
170 :KRBD support: since v3.10
173 ``Exclusive locking``
175 :Description: When enabled, it requires a client to get a lock on an object before making a write. Exclusive lock should only be enabled when a single client is accessing an image at the same time.
177 :CLI value: exclusive-lock
178 :Added in: v0.92 (Hammer)
179 :KRBD support: since v4.9
184 :Description: Object map support depends on exclusive lock support. Block devices are thin provisioned—meaning, they only store data that actually exists. Object map support helps track which objects actually exist (have data stored on a drive). Enabling object map support speeds up I/O operations for cloning; importing and exporting a sparsely populated image; and deleting.
186 :CLI value: object-map
187 :Added in: v0.93 (Hammer)
194 :Description: Fast-diff support depends on object map support and exclusive lock support. It adds another property to the object map, which makes it much faster to generate diffs between snapshots of an image, and the actual data usage of a snapshot much faster.
196 :CLI value: fast-diff
197 :Added in: v9.0.1 (Infernalis)
204 :Description: Deep-flatten makes rbd flatten work on all the snapshots of an image, in addition to the image itself. Without it, snapshots of an image will still rely on the parent, so the parent will not be delete-able until the snapshots are deleted. Deep-flatten makes a parent independent of its clones, even if they have snapshots.
206 :CLI value: deep-flatten
207 :Added in: v9.0.2 (Infernalis)
214 :Description: Journaling support depends on exclusive lock support. Journaling records all modifications to an image in the order they occur. RBD mirroring utilizes the journal to replicate a crash consistent image to a remote cluster.
216 :CLI value: journaling
217 :Added in: v10.0.1 (Jewel)
224 :Description: On erasure-coded pools, the image data block objects need to be stored on a separate pool from the image metadata.
226 :Added in: v11.1.0 (Kraken)
227 :KRBD support: since v4.11
233 :Description: Used to restrict older clients from performing certain maintenance operations against an image (e.g. clone, snap create).
235 :Added in: v13.0.2 (Mimic)
236 :KRBD support: since v4.16
241 :Description: Used to restrict older clients from opening an image when it is in migration state.
243 :Added in: v14.0.1 (Nautilus)
250 RBD supports limiting per image IO, controlled by the following
253 ``rbd qos iops limit``
255 :Description: The desired limit of IO operations per second.
256 :Type: Unsigned Integer
261 ``rbd qos bps limit``
263 :Description: The desired limit of IO bytes per second.
264 :Type: Unsigned Integer
269 ``rbd qos read iops limit``
271 :Description: The desired limit of read operations per second.
272 :Type: Unsigned Integer
277 ``rbd qos write iops limit``
279 :Description: The desired limit of write operations per second.
280 :Type: Unsigned Integer
285 ``rbd qos read bps limit``
287 :Description: The desired limit of read bytes per second.
288 :Type: Unsigned Integer
293 ``rbd qos write bps limit``
295 :Description: The desired limit of write bytes per second.
296 :Type: Unsigned Integer
301 ``rbd qos iops burst``
303 :Description: The desired burst limit of IO operations.
304 :Type: Unsigned Integer
309 ``rbd qos bps burst``
311 :Description: The desired burst limit of IO bytes.
312 :Type: Unsigned Integer
317 ``rbd qos read iops burst``
319 :Description: The desired burst limit of read operations.
320 :Type: Unsigned Integer
325 ``rbd qos write iops burst``
327 :Description: The desired burst limit of write operations.
328 :Type: Unsigned Integer
333 ``rbd qos read bps burst``
335 :Description: The desired burst limit of read bytes.
336 :Type: Unsigned Integer
341 ``rbd qos write bps burst``
343 :Description: The desired burst limit of write bytes.
344 :Type: Unsigned Integer
349 ``rbd qos schedule tick min``
351 :Description: The minimum schedule tick (in milliseconds) for QoS.
352 :Type: Unsigned Integer