ceph/doc/rbd/rbd-persistent-cache.rst

   1 =======================
   2  RBD Persistent Cache
   3 =======================
   4
   5 .. index:: Ceph Block Device; Persistent Cache
   6
   7 Shared, Read-only Parent Image Cache
   8 ====================================
   9
  10 `Cloned RBD images`_ from a parent usually only modify a small portion of
  11 the image. For example, in a VDI workload, the VMs are cloned from the same
  12 base image and initially only differ by hostname and IP address. During the
  13 booting stage, all of these VMs would re-read portions of duplicate parent
  14 image data from the RADOS cluster. If we have a local cache of the parent
  15 image, this will help to speed up the read process on one host, as well as
  16 to save the client to cluster network traffic.
  17 RBD shared read-only parent image cache requires explicitly enabling in
  18 ``ceph.conf``. The ``ceph-immutable-object-cache`` daemon is responsible for
  19 caching the parent content on the local disk, and future reads on that data
  20 will be serviced from the local cache.
  21
  22 .. note:: RBD shared read-only parent image cache requires the Ceph Nautilus release or later.
  23
  24 .. ditaa::
  25
  26             +--------------------------------------------------------+
  27             |                         QEMU                           |
  28             +--------------------------------------------------------+
  29             |                librbd (cloned images)                  |
  30             +-------------------+-+----------------------------------+
  31             |      librados     | |  ceph--immutable--object--cache  |
  32             +-------------------+ +----------------------------------+
  33             |      OSDs/Mons    | |     local cached parent image    |
  34             +-------------------+ +----------------------------------+
  35
  36
  37 Enable RBD Shared Read-only Parent Image Cache
  38 ----------------------------------------------
  39
  40 To enable RBD shared read-only parent image cache, the following Ceph settings
  41 need to added in the ``[client]`` `section`_ of your ``ceph.conf`` file.
  42
  43 ``rbd parent cache enabled = true``
  44
  45
  46 Immutable Object Cache Daemon
  47 =============================
  48
  49 The ``ceph-immutable-object-cache`` daemon is responsible for caching parent
  50 image content within its local caching directory. For better performance it's
  51 recommended to use SSDs as the underlying storage.
  52
  53 The key components of the daemon are:
  54
  55 #. **Domain socket based IPC:** The daemon will listen on a local domain
  56    socket on start up and wait for connections from librbd clients.
  57
  58 #. **LRU based promotion/demotion policy:** The daemon will maintain
  59    in-memory statistics of cache-hits on each cache file. It will demote the
  60    cold cache if capacity reaches to the configured threshold.
  61
  62 #. **File-based caching store:** The daemon will maintain a simple file
  63    based cache store. On promotion the RADOS objects will be fetched from
  64    RADOS cluster and stored in the local caching directory.
  65
  66 On opening each cloned rbd image, ``librbd`` will try to connect to the
  67 cache daemon over its domain socket. If it's successfully connected,
  68 ``librbd`` will automatically check with the daemon on the subsequent reads.
  69 If there's a read that's not cached, the daemon will promote the RADOS object
  70 to local caching directory, so the next read on that object will be serviced
  71 from local file. The daemon also maintains simple LRU statistics so if there's
  72 not enough capacity it will delete some cold cache files.
  73
  74 Here are some important cache options correspond to the following settings:
  75
  76 - ``immutable_object_cache_sock`` The path to the domain socket used for
  77   communication between librbd clients and the ceph-immutable-object-cache
  78   daemon.
  79
  80 - ``immutable_object_cache_path`` The immutable object cache data directory.
  81
  82 - ``immutable_object_cache_max_size`` The max size for immutable cache.
  83
  84 - ``immutable_object_cache_watermark`` The watermark for the cache. If the
  85   capacity reaches to this watermark, the daemon will delete cold cache based
  86   on the LRU statistics.
  87
  88 The ``ceph-immutable-object-cache`` daemon is available within the optional
  89 ``ceph-immutable-object-cache`` distribution package.
  90
  91 .. important:: ``ceph-immutable-object-cache`` daemon requires the ability to
  92    connect RADOS clusters.
  93
  94 ``ceph-immutable-object-cache`` daemon should use a unique Ceph user ID.
  95 To `create a Ceph user`_, with ``ceph`` specify the ``auth get-or-create``
  96 command, user name, monitor caps, and OSD caps::
  97
  98   ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'allow r' osd 'profile rbd-read-only'
  99
 100 The ``ceph-immutable-object-cache`` daemon can be managed by ``systemd`` by specifying the user
 101 ID as the daemon instance::
 102
 103   systemctl enable ceph-immutable-object-cache@immutable-object-cache.{unique id}
 104
 105 The ``ceph-immutable-object-cache`` can also be run in foreground by ``ceph-immutable-object-cache`` command::
 106
 107   ceph-immutable-object-cache -f --log-file={log_path}
 108
 109 .. _Cloned RBD Images: ../rbd-snapshot/#layering
 110 .. _section: ../../rados/configuration/ceph-conf/#configuration-sections
 111 .. _create a Ceph user: ../../rados/operations/user-management#add-a-user
 112