ceph/doc/rbd/rbd-persistent-read-only-cache.rst

   1 ===============================
   2  RBD Persistent Read-only Cache
   3 ===============================
   4
   5 .. index:: Ceph Block Device; Persistent Read-only Cache
   6
   7 Shared, Read-only Parent Image Cache
   8 ====================================
   9
  10 `Cloned RBD images`_ usually modify only a small fraction of the parent
  11 image. For example, in a VDI use-case, VMs are cloned from the same
  12 base image and initially differ only by hostname and IP address. During
  13 booting, all of these VMs read portions of the same parent
  14 image data. If we have a local cache of the parent
  15 image, this speeds up reads on the caching host.  We also achieve
  16 reduction of client-to-cluster network traffic.
  17 RBD cache must be explicitly enabled in
  18 ``ceph.conf``. The ``ceph-immutable-object-cache`` daemon is responsible for
  19 caching the parent content on the local disk, and future reads on that data
  20 will be serviced from the local cache.
  21
  22 .. note:: RBD shared read-only parent image cache requires the Ceph Nautilus release or later.
  23
  24 .. ditaa::
  25
  26             +--------------------------------------------------------+
  27             |                         QEMU                           |
  28             +--------------------------------------------------------+
  29             |                librbd (cloned images)                  |
  30             +-------------------+-+----------------------------------+
  31             |      librados     | |  ceph--immutable--object--cache  |
  32             +-------------------+ +----------------------------------+
  33             |      OSDs/Mons    | |     local cached parent image    |
  34             +-------------------+ +----------------------------------+
  35
  36
  37 Enable RBD Shared Read-only Parent Image Cache
  38 ----------------------------------------------
  39
  40 To enable RBD shared read-only parent image cache, the following Ceph settings
  41 need to added in the ``[client]`` `section`_ of your ``ceph.conf`` file::
  42
  43         rbd parent cache enabled = true
  44         rbd plugins = parent_cache
  45
  46 Immutable Object Cache Daemon
  47 =============================
  48
  49 Introduction and Generic Settings
  50 ---------------------------------
  51
  52 The ``ceph-immutable-object-cache`` daemon is responsible for caching parent
  53 image content within its local caching directory. Using SSDs as the underlying
  54 storage is recommended because doing so provides better performance.
  55
  56 The key components of the daemon are:
  57
  58 #. **Domain socket based IPC:** The daemon listens on a local domain socket at
  59    startup and waits for connections from librbd clients.
  60
  61 #. **LRU based promotion/demotion policy:** The daemon maintains in-memory
  62    statistics of cache hits for each cache file. It demotes the cold cache
  63    if capacity reaches the configured threshold.
  64
  65 #. **File-based caching store:** The daemon maintains a simple file-based cache
  66    store. On promotion, the RADOS objects are fetched from RADOS cluster and
  67    stored in the local caching directory.
  68
  69 When each cloned RBD image is opened, ``librbd`` tries to connect to the cache
  70 daemon through its Unix domain socket. After ``librbd`` is successfully
  71 connected, it coordinates with the daemon upon every subsequent read. In the
  72 case of an uncached read, the daemon promotes the RADOS object to the local
  73 caching directory and the next read of the object is serviced from the cache.
  74 The daemon maintains simple LRU statistics, which are used to evict cold cache
  75 files when required (for example, when the cache is at capacity and under
  76 pressure).
  77
  78 Here are some important cache configuration settings:
  79
  80 ``immutable_object_cache_sock``
  81
  82 :Description: The path to the domain socket used for communication between
  83               librbd clients and the ceph-immutable-object-cache daemon.
  84 :Type: String
  85 :Required: No
  86 :Default: ``/var/run/ceph/immutable_object_cache_sock``
  87
  88
  89 ``immutable_object_cache_path``
  90
  91 :Description: The immutable object cache data directory.
  92 :Type: String
  93 :Required: No
  94 :Default: ``/tmp/ceph_immutable_object_cache``
  95
  96
  97 ``immutable_object_cache_max_size``
  98
  99 :Description: The max size for immutable cache.
 100 :Type: Size
 101 :Required: No
 102 :Default: ``1G``
 103
 104
 105 ``immutable_object_cache_watermark``
 106
 107 :Description: The high-water mark for the cache. The value is between (0, 1).
 108               If the cache size reaches this threshold the daemon will start
 109               to delete cold cache based on LRU statistics.
 110 :Type: Float
 111 :Required: No
 112 :Default: ``0.9``
 113
 114 The ``ceph-immutable-object-cache`` daemon is available within the optional
 115 ``ceph-immutable-object-cache`` distribution package.
 116
 117 .. important:: ``ceph-immutable-object-cache`` daemon requires the ability to
 118    connect RADOS clusters.
 119
 120 Running the Immutable Object Cache Daemon
 121 -----------------------------------------
 122
 123 ``ceph-immutable-object-cache`` daemon should use a unique Ceph user ID.
 124 To `create a Ceph user`_, with ``ceph`` specify the ``auth get-or-create``
 125 command, user name, monitor caps, and OSD caps::
 126
 127   ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'allow r' osd 'profile rbd-read-only'
 128
 129 The ``ceph-immutable-object-cache`` daemon can be managed by ``systemd`` by specifying the user
 130 ID as the daemon instance::
 131
 132   systemctl enable ceph-immutable-object-cache@ceph-immutable-object-cache.{unique id}
 133
 134 The ``ceph-immutable-object-cache`` can also be run in foreground by ``ceph-immutable-object-cache`` command::
 135
 136   ceph-immutable-object-cache -f --log-file={log_path}
 137
 138 QOS Settings
 139 ------------
 140
 141 The immutable object cache supports throttling, controlled by the following settings:
 142
 143 ``immutable_object_cache_qos_schedule_tick_min``
 144
 145 :Description: Minimum schedule tick for immutable object cache.
 146 :Type: Milliseconds
 147 :Required: No
 148 :Default: ``50``
 149
 150
 151 ``immutable_object_cache_qos_iops_limit``
 152
 153 :Description: The desired immutable object cache IO operations limit per second.
 154 :Type: Unsigned Integer
 155 :Required: No
 156 :Default: ``0``
 157
 158
 159 ``immutable_object_cache_qos_iops_burst``
 160
 161 :Description: The desired burst limit of immutable object cache IO operations.
 162 :Type: Unsigned Integer
 163 :Required: No
 164 :Default: ``0``
 165
 166
 167 ``immutable_object_cache_qos_iops_burst_seconds``
 168
 169 :Description: The desired burst duration in seconds of immutable object cache IO operations.
 170 :Type: Seconds
 171 :Required: No
 172 :Default: ``1``
 173
 174
 175 ``immutable_object_cache_qos_bps_limit``
 176
 177 :Description: The desired immutable object cache IO bytes limit per second.
 178 :Type: Unsigned Integer
 179 :Required: No
 180 :Default: ``0``
 181
 182
 183 ``immutable_object_cache_qos_bps_burst``
 184
 185 :Description: The desired burst limit of immutable object cache IO bytes.
 186 :Type: Unsigned Integer
 187 :Required: No
 188 :Default: ``0``
 189
 190
 191 ``immutable_object_cache_qos_bps_burst_seconds``
 192
 193 :Description: The desired burst duration in seconds of immutable object cache IO bytes.
 194 :Type: Seconds
 195 :Required: No
 196 :Default: ``1``
 197
 198 .. _Cloned RBD Images: ../rbd-snapshot/#layering
 199 .. _section: ../../rados/configuration/ceph-conf/#configuration-sections
 200 .. _create a Ceph user: ../../rados/operations/user-management#add-a-user
 201