ceph/doc/cephadm/mon.rst

   1 ===========
   2 MON Service
   3 ===========
   4
   5 .. _deploy_additional_monitors:
   6
   7 Deploying additional monitors
   8 -----------------------------
   9
  10 A typical Ceph cluster has three or five monitor daemons that are spread
  11 across different hosts.  We recommend deploying five monitors if there are
  12 five or more nodes in your cluster.
  13
  14 .. _CIDR: https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#CIDR_notation
  15
  16 Ceph deploys monitor daemons automatically as the cluster grows and Ceph
  17 scales back monitor daemons automatically as the cluster shrinks. The
  18 smooth execution of this automatic growing and shrinking depends upon
  19 proper subnet configuration.
  20
  21 The cephadm bootstrap procedure assigns the first monitor daemon in the
  22 cluster to a particular subnet. ``cephadm`` designates that subnet as the
  23 default subnet of the cluster. New monitor daemons will be assigned by
  24 default to that subnet unless cephadm is instructed to do otherwise.
  25
  26 If all of the ceph monitor daemons in your cluster are in the same subnet,
  27 manual administration of the ceph monitor daemons is not necessary.
  28 ``cephadm`` will automatically add up to five monitors to the subnet, as
  29 needed, as new hosts are added to the cluster.
  30
  31 Designating a Particular Subnet for Monitors
  32 --------------------------------------------
  33
  34 To designate a particular IP subnet for use by ceph monitor daemons, use a
  35 command of the following form, including the subnet's address in `CIDR`_
  36 format (e.g., ``10.1.2.0/24``):
  37
  38   .. prompt:: bash #
  39
  40      ceph config set mon public_network *<mon-cidr-network>*
  41
  42   For example:
  43
  44   .. prompt:: bash #
  45
  46      ceph config set mon public_network 10.1.2.0/24
  47
  48 Cephadm deploys new monitor daemons only on hosts that have IP addresses in
  49 the designated subnet.
  50
  51 Changing the number of monitors from the default
  52 ------------------------------------------------
  53
  54 If you want to adjust the default of 5 monitors, run this command:
  55
  56   .. prompt:: bash #
  57
  58      ceph orch apply mon *<number-of-monitors>*
  59
  60 Deploying monitors only to specific hosts
  61 -----------------------------------------
  62
  63 To deploy monitors on a specific set of hosts, run this command:
  64
  65   .. prompt:: bash #
  66
  67     ceph orch apply mon *<host1,host2,host3,...>*
  68
  69   Be sure to include the first (bootstrap) host in this list.
  70
  71 Using Host Labels
  72 -----------------
  73
  74 You can control which hosts the monitors run on by making use of host labels.
  75 To set the ``mon`` label to the appropriate hosts, run this command:
  76
  77   .. prompt:: bash #
  78
  79     ceph orch host label add *<hostname>* mon
  80
  81   To view the current hosts and labels, run this command:
  82
  83   .. prompt:: bash #
  84
  85     ceph orch host ls
  86
  87   For example:
  88
  89   .. prompt:: bash #
  90
  91     ceph orch host label add host1 mon
  92     ceph orch host label add host2 mon
  93     ceph orch host label add host3 mon
  94     ceph orch host ls
  95
  96   .. code-block:: bash
  97
  98     HOST   ADDR   LABELS  STATUS
  99     host1         mon
 100     host2         mon
 101     host3         mon
 102     host4
 103     host5
 104
 105   Tell cephadm to deploy monitors based on the label by running this command:
 106
 107   .. prompt:: bash #
 108
 109     ceph orch apply mon label:mon
 110
 111 See also :ref:`host labels <orchestrator-host-labels>`.
 112
 113 Deploying Monitors on a Particular Network
 114 ------------------------------------------
 115
 116 You can explicitly specify the IP address or CIDR network for each monitor and
 117 control where each monitor is placed.  To disable automated monitor deployment,
 118 run this command:
 119
 120   .. prompt:: bash #
 121
 122     ceph orch apply mon --unmanaged
 123
 124   To deploy each additional monitor:
 125
 126   .. prompt:: bash #
 127
 128     ceph orch daemon add mon *<host1:ip-or-network1> [<host1:ip-or-network-2>...]*
 129
 130   For example, to deploy a second monitor on ``newhost1`` using an IP
 131   address ``10.1.2.123`` and a third monitor on ``newhost2`` in
 132   network ``10.1.2.0/24``, run the following commands:
 133
 134   .. prompt:: bash #
 135
 136     ceph orch apply mon --unmanaged
 137     ceph orch daemon add mon newhost1:10.1.2.123
 138     ceph orch daemon add mon newhost2:10.1.2.0/24
 139
 140   .. note::
 141      The **apply** command can be confusing. For this reason, we recommend using
 142      YAML specifications.
 143
 144      Each ``ceph orch apply mon`` command supersedes the one before it.
 145      This means that you must use the proper comma-separated list-based
 146      syntax when you want to apply monitors to more than one host.
 147      If you do not use the proper syntax, you will clobber your work
 148      as you go.
 149
 150      For example:
 151
 152      .. prompt:: bash #
 153
 154           ceph orch apply mon host1
 155           ceph orch apply mon host2
 156           ceph orch apply mon host3
 157
 158      This results in only one host having a monitor applied to it: host 3.
 159
 160      (The first command creates a monitor on host1. Then the second command
 161      clobbers the monitor on host1 and creates a monitor on host2. Then the
 162      third command clobbers the monitor on host2 and creates a monitor on
 163      host3. In this scenario, at this point, there is a monitor ONLY on
 164      host3.)
 165
 166      To make certain that a monitor is applied to each of these three hosts,
 167      run a command like this:
 168
 169      .. prompt:: bash #
 170
 171        ceph orch apply mon "host1,host2,host3"
 172
 173      There is another way to apply monitors to multiple hosts: a ``yaml`` file
 174      can be used. Instead of using the "ceph orch apply mon" commands, run a
 175      command of this form:
 176
 177      .. prompt:: bash #
 178
 179         ceph orch apply -i file.yaml
 180
 181      Here is a sample **file.yaml** file::
 182
 183           service_type: mon
 184           placement:
 185             hosts:
 186              - host1
 187              - host2
 188              - host3