]>
Commit | Line | Data |
---|---|---|
9f95a23c TL |
1 | ======================= |
2 | RBD Persistent Cache | |
3 | ======================= | |
4 | ||
5 | .. index:: Ceph Block Device; Persistent Cache | |
6 | ||
7 | Shared, Read-only Parent Image Cache | |
8 | ==================================== | |
9 | ||
10 | `Cloned RBD images`_ from a parent usually only modify a small portion of | |
11 | the image. For example, in a VDI workload, the VMs are cloned from the same | |
12 | base image and initially only differ by hostname and IP address. During the | |
13 | booting stage, all of these VMs would re-read portions of duplicate parent | |
14 | image data from the RADOS cluster. If we have a local cache of the parent | |
15 | image, this will help to speed up the read process on one host, as well as | |
16 | to save the client to cluster network traffic. | |
17 | RBD shared read-only parent image cache requires explicitly enabling in | |
18 | ``ceph.conf``. The ``ceph-immutable-object-cache`` daemon is responsible for | |
19 | caching the parent content on the local disk, and future reads on that data | |
20 | will be serviced from the local cache. | |
21 | ||
22 | .. note:: RBD shared read-only parent image cache requires the Ceph Nautilus release or later. | |
23 | ||
f91f0fd5 TL |
24 | .. ditaa:: |
25 | ||
26 | +--------------------------------------------------------+ | |
9f95a23c TL |
27 | | QEMU | |
28 | +--------------------------------------------------------+ | |
29 | | librbd (cloned images) | | |
30 | +-------------------+-+----------------------------------+ | |
31 | | librados | | ceph--immutable--object--cache | | |
32 | +-------------------+ +----------------------------------+ | |
33 | | OSDs/Mons | | local cached parent image | | |
34 | +-------------------+ +----------------------------------+ | |
35 | ||
36 | ||
37 | Enable RBD Shared Read-only Parent Image Cache | |
38 | ---------------------------------------------- | |
39 | ||
40 | To enable RBD shared read-only parent image cache, the following Ceph settings | |
41 | need to added in the ``[client]`` `section`_ of your ``ceph.conf`` file. | |
42 | ||
43 | ``rbd parent cache enabled = true`` | |
44 | ||
45 | ||
46 | Immutable Object Cache Daemon | |
47 | ============================= | |
48 | ||
49 | The ``ceph-immutable-object-cache`` daemon is responsible for caching parent | |
50 | image content within its local caching directory. For better performance it's | |
51 | recommended to use SSDs as the underlying storage. | |
52 | ||
53 | The key components of the daemon are: | |
54 | ||
55 | #. **Domain socket based IPC:** The daemon will listen on a local domain | |
56 | socket on start up and wait for connections from librbd clients. | |
57 | ||
58 | #. **LRU based promotion/demotion policy:** The daemon will maintain | |
59 | in-memory statistics of cache-hits on each cache file. It will demote the | |
60 | cold cache if capacity reaches to the configured threshold. | |
61 | ||
62 | #. **File-based caching store:** The daemon will maintain a simple file | |
63 | based cache store. On promotion the RADOS objects will be fetched from | |
64 | RADOS cluster and stored in the local caching directory. | |
65 | ||
66 | On opening each cloned rbd image, ``librbd`` will try to connect to the | |
67 | cache daemon over its domain socket. If it's successfully connected, | |
68 | ``librbd`` will automatically check with the daemon on the subsequent reads. | |
69 | If there's a read that's not cached, the daemon will promote the RADOS object | |
70 | to local caching directory, so the next read on that object will be serviced | |
71 | from local file. The daemon also maintains simple LRU statistics so if there's | |
72 | not enough capacity it will delete some cold cache files. | |
73 | ||
74 | Here are some important cache options correspond to the following settings: | |
75 | ||
f6b5b4d7 TL |
76 | - ``immutable_object_cache_sock`` The path to the domain socket used for |
77 | communication between librbd clients and the ceph-immutable-object-cache | |
78 | daemon. | |
79 | ||
9f95a23c TL |
80 | - ``immutable_object_cache_path`` The immutable object cache data directory. |
81 | ||
82 | - ``immutable_object_cache_max_size`` The max size for immutable cache. | |
83 | ||
84 | - ``immutable_object_cache_watermark`` The watermark for the cache. If the | |
85 | capacity reaches to this watermark, the daemon will delete cold cache based | |
86 | on the LRU statistics. | |
87 | ||
88 | The ``ceph-immutable-object-cache`` daemon is available within the optional | |
89 | ``ceph-immutable-object-cache`` distribution package. | |
90 | ||
91 | .. important:: ``ceph-immutable-object-cache`` daemon requires the ability to | |
92 | connect RADOS clusters. | |
93 | ||
94 | ``ceph-immutable-object-cache`` daemon should use a unique Ceph user ID. | |
95 | To `create a Ceph user`_, with ``ceph`` specify the ``auth get-or-create`` | |
96 | command, user name, monitor caps, and OSD caps:: | |
97 | ||
98 | ceph auth get-or-create client.ceph-immutable-object-cache.{unique id} mon 'allow r' osd 'profile rbd-read-only' | |
99 | ||
100 | The ``ceph-immutable-object-cache`` daemon can be managed by ``systemd`` by specifying the user | |
101 | ID as the daemon instance:: | |
102 | ||
103 | systemctl enable ceph-immutable-object-cache@immutable-object-cache.{unique id} | |
104 | ||
105 | The ``ceph-immutable-object-cache`` can also be run in foreground by ``ceph-immutable-object-cache`` command:: | |
106 | ||
107 | ceph-immutable-object-cache -f --log-file={log_path} | |
108 | ||
109 | .. _Cloned RBD Images: ../rbd-snapshot/#layering | |
110 | .. _section: ../../rados/configuration/ceph-conf/#configuration-sections | |
111 | .. _create a Ceph user: ../../rados/operations/user-management#add-a-user | |
112 |