ceph/doc/mgr/administrator.rst

   1 .. _mgr-administrator-guide:
   2
   3 ceph-mgr administrator's guide
   4 ==============================
   5
   6 Manual setup
   7 ------------
   8
   9 Usually, you would set up a ceph-mgr daemon using a tool such
  10 as ceph-ansible.  These instructions describe how to set up
  11 a ceph-mgr daemon manually.
  12
  13 First, create an authentication key for your daemon::
  14
  15     ceph auth get-or-create mgr.$name mon 'allow profile mgr' osd 'allow *' mds 'allow *'
  16
  17 Place that key as file named ``keyring`` into ``mgr data`` path, which for a cluster "ceph"
  18 and mgr $name "foo" would be ``/var/lib/ceph/mgr/ceph-foo`` respective ``/var/lib/ceph/mgr/ceph-foo/keyring``.
  19
  20 Start the ceph-mgr daemon::
  21
  22     ceph-mgr -i $name
  23
  24 Check that the mgr has come up by looking at the output
  25 of ``ceph status``, which should now include a mgr status line::
  26
  27     mgr active: $name
  28
  29 Client authentication
  30 ---------------------
  31
  32 The manager is a new daemon which requires new CephX capabilities. If you upgrade
  33 a cluster from an old version of Ceph, or use the default install/deploy tools,
  34 your admin client should get this capability automatically. If you use tooling from
  35 elsewhere, you may get EACCES errors when invoking certain ceph cluster commands.
  36 To fix that, add a "mgr allow \*" stanza to your client's cephx capabilities by
  37 `Modifying User Capabilities`_.
  38
  39 High availability
  40 -----------------
  41
  42 In general, you should set up a ceph-mgr on each of the hosts
  43 running a ceph-mon daemon to achieve the same level of availability.
  44
  45 By default, whichever ceph-mgr instance comes up first will be made
  46 active by the monitors, and the others will be standbys.  There is
  47 no requirement for quorum among the ceph-mgr daemons.
  48
  49 If the active daemon fails to send a beacon to the monitors for
  50 more than :confval:`mon_mgr_beacon_grace`, then it will be replaced
  51 by a standby.
  52
  53 If you want to preempt failover, you can explicitly mark a ceph-mgr
  54 daemon as failed using ``ceph mgr fail <mgr name>``.
  55
  56 Performance and Scalability
  57 ---------------------------
  58
  59 All the mgr modules share a cache that can be enabled with
  60 ``ceph config set mgr mgr_ttl_cache_expire_seconds <seconds>``, where seconds
  61 is the time to live of the cached python objects.
  62
  63 It is recommended to enable the cache with a 10 seconds TTL when there are 500+
  64 osds or 10k+ pgs as internal structures might increase in size, and cause latency
  65 issues when requesting large structures. As an example, an OSDMap with 1000 osds
  66 has a aproximate size of 4MiB. With heavy load, on a 3000 osd cluster there has
  67 been a 1.5x improvement enabling the cache.
  68
  69 Furthermore, you can run ``ceph daemon mgr.${MGRNAME} perf dump`` to retrieve perf
  70 counters of a mgr module. In ``mgr.cache_hit`` and ``mgr.cache_miss`` you'll find the
  71 hit/miss ratio of the mgr cache.
  72
  73 Using modules
  74 -------------
  75
  76 Use the command ``ceph mgr module ls`` to see which modules are
  77 available, and which are currently enabled. Use ``ceph mgr module ls --format=json-pretty``
  78 to view detailed metadata about disabled modules. Enable or disable modules
  79 using the commands ``ceph mgr module enable <module>`` and
  80 ``ceph mgr module disable <module>`` respectively.
  81
  82 If a module is *enabled* then the active ceph-mgr daemon will load
  83 and execute it.  In the case of modules that provide a service,
  84 such as an HTTP server, the module may publish its address when it
  85 is loaded.  To see the addresses of such modules, use the command
  86 ``ceph mgr services``.
  87
  88 Some modules may also implement a special standby mode which runs on
  89 standby ceph-mgr daemons as well as the active daemon.  This enables
  90 modules that provide services to redirect their clients to the active
  91 daemon, if the client tries to connect to a standby.
  92
  93 Consult the documentation pages for individual manager modules for more
  94 information about what functionality each module provides.
  95
  96 Here is an example of enabling the :term:`Dashboard` module:
  97
  98 .. code-block:: console
  99
 100         $ ceph mgr module ls
 101         {
 102                 "enabled_modules": [
 103                         "restful",
 104                         "status"
 105                 ],
 106                 "disabled_modules": [
 107                         "dashboard"
 108                 ]
 109         }
 110
 111         $ ceph mgr module enable dashboard
 112         $ ceph mgr module ls
 113         {
 114                 "enabled_modules": [
 115                         "restful",
 116                         "status",
 117                         "dashboard"
 118                 ],
 119                 "disabled_modules": [
 120                 ]
 121         }
 122
 123         $ ceph mgr services
 124         {
 125                 "dashboard": "http://myserver.com:7789/",
 126                 "restful": "https://myserver.com:8789/"
 127         }
 128
 129
 130 The first time the cluster starts, it uses the :confval:`mgr_initial_modules`
 131 setting to override which modules to enable.  However, this setting
 132 is ignored through the rest of the lifetime of the cluster: only
 133 use it for bootstrapping.  For example, before starting your
 134 monitor daemons for the first time, you might add a section like
 135 this to your ``ceph.conf``:
 136
 137 .. code-block:: ini
 138
 139     [mon]
 140         mgr_initial_modules = dashboard balancer
 141
 142 Module Pool
 143 -----------
 144
 145 The manager creates a pool for use by its module to store state. The name of
 146 this pool is ``.mgr`` (with the leading ``.`` indicating a reserved pool
 147 name).
 148
 149 .. note::
 150
 151    Prior to Quincy, the ``devicehealth`` module created a
 152    ``device_health_metrics`` pool to store device SMART statistics. With
 153    Quincy, this pool is automatically renamed to be the common manager module
 154    pool.
 155
 156
 157 Calling module commands
 158 -----------------------
 159
 160 Where a module implements command line hooks, the commands will
 161 be accessible as ordinary Ceph commands.  Ceph will automatically incorporate
 162 module commands into the standard CLI interface and route them appropriately to
 163 the module.::
 164
 165     ceph <command | help>
 166
 167 Configuration
 168 -------------
 169
 170 .. confval:: mgr_module_path
 171 .. confval:: mgr_initial_modules
 172 .. confval:: mgr_disabled_modules
 173 .. confval:: mgr_standby_modules
 174 .. confval:: mgr_data
 175 .. confval:: mgr_tick_period
 176 .. confval:: mon_mgr_beacon_grace
 177
 178 .. _Modifying User Capabilities: ../../rados/operations/user-management/#modify-user-capabilities