ceph/doc/rbd/rbd-persistent-read-only-cache.rst

   1 ===============================
   2  RBD Persistent Read-only Cache
   3 ===============================
   4
   5 .. index:: Ceph Block Device; Persistent Read-only Cache
   6
   7 Shared, Read-only Parent Image Cache
   8 ====================================
   9
  10 `Cloned RBD images`_ usually modify only a small fraction of the parent
  11 image. For example, in a VDI use-case, VMs are cloned from the same
  12 base image and initially differ only by hostname and IP address. During
  13 booting, all of these VMs read portions of the same parent
  14 image data. If we have a local cache of the parent
  15 image, this speeds up reads on the caching host.  We also achieve
  16 reduction of client-to-cluster network traffic.
  17 RBD cache must be explicitly enabled in
  18 ``ceph.conf``. The ``ceph-immutable-object-cache`` daemon is responsible for
  19 caching the parent content on the local disk, and future reads on that data
  20 will be serviced from the local cache.
  21
  22 .. note:: RBD shared read-only parent image cache requires the Ceph Nautilus release or later.
  23
  24 .. ditaa::
  25
  26             +--------------------------------------------------------+
  27             |                         QEMU                           |
  28             +--------------------------------------------------------+
  29             |                librbd (cloned images)                  |
  30             +-------------------+-+----------------------------------+
  31             |      librados     | |  ceph--immutable--object--cache  |
  32             +-------------------+ +----------------------------------+
  33             |      OSDs/Mons    | |     local cached parent image    |
  34             +-------------------+ +----------------------------------+
  35
  36
  37 Enable RBD Shared Read-only Parent Image Cache
  38 ----------------------------------------------
  39
  40 To enable RBD shared read-only parent image cache, the following Ceph settings
  41 need to added in the ``[client]`` `section`_ of your ``ceph.conf`` file::
  42
  43         rbd parent cache enabled = true
  44         rbd plugins = parent_cache
  45
  46 Immutable Object Cache Daemon
  47 =============================
  48
  49 Introduction and Generic Settings
  50 ---------------------------------
  51
  52 The ``ceph-immutable-object-cache`` daemon is responsible for caching parent
  53 image content within its local caching directory. For better performance it's
  54 recommended to use SSDs as the underlying storage.
  55
  56 The key components of the daemon are:
  57
  58 #. **Domain socket based IPC:** The daemon will listen on a local domain
  59    socket on start up and wait for connections from librbd clients.
  60
  61 #. **LRU based promotion/demotion policy:** The daemon will maintain
  62    in-memory statistics of cache-hits on each cache file. It will demote the
  63    cold cache if capacity reaches to the configured threshold.
  64
  65 #. **File-based caching store:** The daemon will maintain a simple file
  66    based cache store. On promotion the RADOS objects will be fetched from
  67    RADOS cluster and stored in the local caching directory.
  68
  69 On opening each cloned rbd image, ``librbd`` will try to connect to the
  70 cache daemon through its Unix domain socket. Once successfully connected,
  71 ``librbd`` will coordinate with the daemon on the subsequent reads.
  72 If there's a read that's not cached, the daemon will promote the RADOS object
  73 to local caching directory, so the next read on that object will be serviced
  74 from cache. The daemon also maintains simple LRU statistics so that under
  75 capacity pressure it will evict cold cache files as needed.
  76
  77 Here are some important cache configuration settings:
  78
  79 ``immutable_object_cache_sock``
  80
  81 :Description: The path to the domain socket used for communication between
  82               librbd clients and the ceph-immutable-object-cache daemon.
  83 :Type: String
  84 :Required: No
  85 :Default: ``/var/run/ceph/immutable_object_cache_sock``
  86
  87
  88 ``immutable_object_cache_path``
  89
  90 :Description: The immutable object cache data directory.
  91 :Type: String
  92 :Required: No
  93 :Default: ``/tmp/ceph_immutable_object_cache``
  94
  95
  96 ``immutable_object_cache_max_size``
  97
  98 :Description: The max size for immutable cache.
  99 :Type: Size
 100 :Required: No
 101 :Default: ``1G``
 102
 103
 104 ``immutable_object_cache_watermark``
 105
 106 :Description: The high-water mark for the cache. The value is between (0, 1).
 107               If the cache size reaches this threshold the daemon will start
 108               to delete cold cache based on LRU statistics.
 109 :Type: Float
 110 :Required: No
 111 :Default: ``0.9``
 112
 113 The ``ceph-immutable-object-cache`` daemon is available within the optional
 114 ``ceph-immutable-object-cache`` distribution package.
 115
 116 .. important:: ``ceph-immutable-object-cache`` daemon requires the ability to
 117    connect RADOS clusters.
 118
 119 Running the Immutable Object Cache Daemon
 120 -----------------------------------------
 121
 122 ``ceph-immutable-object-cache`` daemon should use a unique Ceph user ID.
 123 To `create a Ceph user`_, with ``ceph`` specify the ``auth get-or-create``
 124 command, user name, monitor caps, and OSD caps::
 125
 126   ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'allow r' osd 'profile rbd-read-only'
 127
 128 The ``ceph-immutable-object-cache`` daemon can be managed by ``systemd`` by specifying the user
 129 ID as the daemon instance::
 130
 131   systemctl enable ceph-immutable-object-cache@ceph-immutable-object-cache.{unique id}
 132
 133 The ``ceph-immutable-object-cache`` can also be run in foreground by ``ceph-immutable-object-cache`` command::
 134
 135   ceph-immutable-object-cache -f --log-file={log_path}
 136
 137 QOS Settings
 138 ------------
 139
 140 The immutable object cache supports throttling, controlled by the following settings:
 141
 142 ``immutable_object_cache_qos_schedule_tick_min``
 143
 144 :Description: Minimum schedule tick for immutable object cache.
 145 :Type: Milliseconds
 146 :Required: No
 147 :Default: ``50``
 148
 149
 150 ``immutable_object_cache_qos_iops_limit``
 151
 152 :Description: The desired immutable object cache IO operations limit per second.
 153 :Type: Unsigned Integer
 154 :Required: No
 155 :Default: ``0``
 156
 157
 158 ``immutable_object_cache_qos_iops_burst``
 159
 160 :Description: The desired burst limit of immutable object cache IO operations.
 161 :Type: Unsigned Integer
 162 :Required: No
 163 :Default: ``0``
 164
 165
 166 ``immutable_object_cache_qos_iops_burst_seconds``
 167
 168 :Description: The desired burst duration in seconds of immutable object cache IO operations.
 169 :Type: Seconds
 170 :Required: No
 171 :Default: ``1``
 172
 173
 174 ``immutable_object_cache_qos_bps_limit``
 175
 176 :Description: The desired immutable object cache IO bytes limit per second.
 177 :Type: Unsigned Integer
 178 :Required: No
 179 :Default: ``0``
 180
 181
 182 ``immutable_object_cache_qos_bps_burst``
 183
 184 :Description: The desired burst limit of immutable object cache IO bytes.
 185 :Type: Unsigned Integer
 186 :Required: No
 187 :Default: ``0``
 188
 189
 190 ``immutable_object_cache_qos_bps_burst_seconds``
 191
 192 :Description: The desired burst duration in seconds of immutable object cache IO bytes.
 193 :Type: Seconds
 194 :Required: No
 195 :Default: ``1``
 196
 197 .. _Cloned RBD Images: ../rbd-snapshot/#layering
 198 .. _section: ../../rados/configuration/ceph-conf/#configuration-sections
 199 .. _create a Ceph user: ../../rados/operations/user-management#add-a-user
 200