]>
Commit | Line | Data |
---|---|---|
b5b8bbf5 | 1 | .. _mgr-administrator-guide: |
7c673cae FG |
2 | |
3 | ceph-mgr administrator's guide | |
4 | ============================== | |
5 | ||
31f18b77 FG |
6 | Manual setup |
7 | ------------ | |
7c673cae | 8 | |
31f18b77 FG |
9 | Usually, you would set up a ceph-mgr daemon using a tool such |
10 | as ceph-ansible. These instructions describe how to set up | |
11 | a ceph-mgr daemon manually. | |
7c673cae | 12 | |
31f18b77 FG |
13 | First, create an authentication key for your daemon:: |
14 | ||
15 | ceph auth get-or-create mgr.$name mon 'allow profile mgr' osd 'allow *' mds 'allow *' | |
7c673cae | 16 | |
20effc67 TL |
17 | Place that key as file named ``keyring`` into ``mgr data`` path, which for a cluster "ceph" |
18 | and mgr $name "foo" would be ``/var/lib/ceph/mgr/ceph-foo`` respective ``/var/lib/ceph/mgr/ceph-foo/keyring``. | |
7c673cae FG |
19 | |
20 | Start the ceph-mgr daemon:: | |
21 | ||
22 | ceph-mgr -i $name | |
23 | ||
24 | Check that the mgr has come up by looking at the output | |
25 | of ``ceph status``, which should now include a mgr status line:: | |
26 | ||
27 | mgr active: $name | |
28 | ||
31f18b77 FG |
29 | Client authentication |
30 | --------------------- | |
224ce89b | 31 | |
31f18b77 FG |
32 | The manager is a new daemon which requires new CephX capabilities. If you upgrade |
33 | a cluster from an old version of Ceph, or use the default install/deploy tools, | |
34 | your admin client should get this capability automatically. If you use tooling from | |
35 | elsewhere, you may get EACCES errors when invoking certain ceph cluster commands. | |
224ce89b | 36 | To fix that, add a "mgr allow \*" stanza to your client's cephx capabilities by |
31f18b77 FG |
37 | `Modifying User Capabilities`_. |
38 | ||
7c673cae FG |
39 | High availability |
40 | ----------------- | |
41 | ||
42 | In general, you should set up a ceph-mgr on each of the hosts | |
b5b8bbf5 | 43 | running a ceph-mon daemon to achieve the same level of availability. |
7c673cae FG |
44 | |
45 | By default, whichever ceph-mgr instance comes up first will be made | |
46 | active by the monitors, and the others will be standbys. There is | |
47 | no requirement for quorum among the ceph-mgr daemons. | |
48 | ||
49 | If the active daemon fails to send a beacon to the monitors for | |
20effc67 | 50 | more than :confval:`mon_mgr_beacon_grace`, then it will be replaced |
7c673cae FG |
51 | by a standby. |
52 | ||
20effc67 | 53 | If you want to preempt failover, you can explicitly mark a ceph-mgr |
7c673cae FG |
54 | daemon as failed using ``ceph mgr fail <mgr name>``. |
55 | ||
20effc67 TL |
56 | Performance and Scalability |
57 | --------------------------- | |
58 | ||
59 | All the mgr modules share a cache that can be enabled with | |
60 | ``ceph config set mgr mgr_ttl_cache_expire_seconds <seconds>``, where seconds | |
61 | is the time to live of the cached python objects. | |
62 | ||
63 | It is recommended to enable the cache with a 10 seconds TTL when there are 500+ | |
64 | osds or 10k+ pgs as internal structures might increase in size, and cause latency | |
65 | issues when requesting large structures. As an example, an OSDMap with 1000 osds | |
1e59de90 | 66 | has a approximate size of 4MiB. With heavy load, on a 3000 osd cluster there has |
20effc67 TL |
67 | been a 1.5x improvement enabling the cache. |
68 | ||
69 | Furthermore, you can run ``ceph daemon mgr.${MGRNAME} perf dump`` to retrieve perf | |
70 | counters of a mgr module. In ``mgr.cache_hit`` and ``mgr.cache_miss`` you'll find the | |
71 | hit/miss ratio of the mgr cache. | |
72 | ||
3efd9988 FG |
73 | Using modules |
74 | ------------- | |
75 | ||
76 | Use the command ``ceph mgr module ls`` to see which modules are | |
20effc67 TL |
77 | available, and which are currently enabled. Use ``ceph mgr module ls --format=json-pretty`` |
78 | to view detailed metadata about disabled modules. Enable or disable modules | |
3efd9988 FG |
79 | using the commands ``ceph mgr module enable <module>`` and |
80 | ``ceph mgr module disable <module>`` respectively. | |
81 | ||
82 | If a module is *enabled* then the active ceph-mgr daemon will load | |
83 | and execute it. In the case of modules that provide a service, | |
84 | such as an HTTP server, the module may publish its address when it | |
20effc67 | 85 | is loaded. To see the addresses of such modules, use the command |
3efd9988 FG |
86 | ``ceph mgr services``. |
87 | ||
88 | Some modules may also implement a special standby mode which runs on | |
89 | standby ceph-mgr daemons as well as the active daemon. This enables | |
90 | modules that provide services to redirect their clients to the active | |
91 | daemon, if the client tries to connect to a standby. | |
92 | ||
93 | Consult the documentation pages for individual manager modules for more | |
94 | information about what functionality each module provides. | |
95 | ||
11fdf7f2 | 96 | Here is an example of enabling the :term:`Dashboard` module: |
3efd9988 | 97 | |
20effc67 | 98 | .. code-block:: console |
3efd9988 FG |
99 | |
100 | $ ceph mgr module ls | |
101 | { | |
102 | "enabled_modules": [ | |
103 | "restful", | |
104 | "status" | |
105 | ], | |
106 | "disabled_modules": [ | |
107 | "dashboard" | |
108 | ] | |
109 | } | |
110 | ||
111 | $ ceph mgr module enable dashboard | |
112 | $ ceph mgr module ls | |
113 | { | |
114 | "enabled_modules": [ | |
115 | "restful", | |
116 | "status", | |
117 | "dashboard" | |
118 | ], | |
119 | "disabled_modules": [ | |
120 | ] | |
121 | } | |
122 | ||
123 | $ ceph mgr services | |
124 | { | |
125 | "dashboard": "http://myserver.com:7789/", | |
126 | "restful": "https://myserver.com:8789/" | |
127 | } | |
128 | ||
129 | ||
20effc67 | 130 | The first time the cluster starts, it uses the :confval:`mgr_initial_modules` |
11fdf7f2 TL |
131 | setting to override which modules to enable. However, this setting |
132 | is ignored through the rest of the lifetime of the cluster: only | |
133 | use it for bootstrapping. For example, before starting your | |
134 | monitor daemons for the first time, you might add a section like | |
135 | this to your ``ceph.conf``: | |
136 | ||
20effc67 | 137 | .. code-block:: ini |
11fdf7f2 TL |
138 | |
139 | [mon] | |
f67539c2 | 140 | mgr_initial_modules = dashboard balancer |
11fdf7f2 | 141 | |
20effc67 TL |
142 | Module Pool |
143 | ----------- | |
144 | ||
145 | The manager creates a pool for use by its module to store state. The name of | |
146 | this pool is ``.mgr`` (with the leading ``.`` indicating a reserved pool | |
147 | name). | |
148 | ||
149 | .. note:: | |
150 | ||
151 | Prior to Quincy, the ``devicehealth`` module created a | |
152 | ``device_health_metrics`` pool to store device SMART statistics. With | |
153 | Quincy, this pool is automatically renamed to be the common manager module | |
154 | pool. | |
155 | ||
156 | ||
7c673cae FG |
157 | Calling module commands |
158 | ----------------------- | |
159 | ||
c07f9fc5 | 160 | Where a module implements command line hooks, the commands will |
9f95a23c TL |
161 | be accessible as ordinary Ceph commands. Ceph will automatically incorporate |
162 | module commands into the standard CLI interface and route them appropriately to | |
163 | the module.:: | |
7c673cae | 164 | |
c07f9fc5 FG |
165 | ceph <command | help> |
166 | ||
7c673cae FG |
167 | Configuration |
168 | ------------- | |
169 | ||
20effc67 TL |
170 | .. confval:: mgr_module_path |
171 | .. confval:: mgr_initial_modules | |
172 | .. confval:: mgr_disabled_modules | |
173 | .. confval:: mgr_standby_modules | |
174 | .. confval:: mgr_data | |
175 | .. confval:: mgr_tick_period | |
176 | .. confval:: mon_mgr_beacon_grace | |
7c673cae | 177 | |
224ce89b | 178 | .. _Modifying User Capabilities: ../../rados/operations/user-management/#modify-user-capabilities |