Adding/Removing Monitors
==========================
-When you have a cluster up and running, you may add or remove monitors
-from the cluster at runtime. To bootstrap a monitor, see `Manual Deployment`_
-or `Monitor Bootstrap`_.
+It is possible to add monitors to a running cluster as long as redundancy is
+maintained. To bootstrap a monitor, see `Manual Deployment`_ or `Monitor
+Bootstrap`_.
.. _adding-monitors:
Adding Monitors
===============
-Ceph monitors are lightweight processes that are the single source of truth
-for the cluster map. You can run a cluster with 1 monitor but we recommend at least 3
-for a production cluster. Ceph monitors use a variation of the
-`Paxos`_ algorithm to establish consensus about maps and other critical
-information across the cluster. Due to the nature of Paxos, Ceph requires
-a majority of monitors to be active to establish a quorum (thus establishing
-consensus).
-
-It is advisable to run an odd number of monitors. An
-odd number of monitors is more resilient than an
-even number. For instance, with a two monitor deployment, no
-failures can be tolerated and still maintain a quorum; with three monitors,
-one failure can be tolerated; in a four monitor deployment, one failure can
-be tolerated; with five monitors, two failures can be tolerated. This avoids
-the dreaded *split brain* phenomenon, and is why an odd number is best.
-In short, Ceph needs a majority of
-monitors to be active (and able to communicate with each other), but that
-majority can be achieved using a single monitor, or 2 out of 2 monitors,
-2 out of 3, 3 out of 4, etc.
+Ceph monitors serve as the single source of truth for the cluster map. It is
+possible to run a cluster with only one monitor, but for a production cluster
+it is recommended to have at least three monitors provisioned and in quorum.
+Ceph monitors use a variation of the `Paxos`_ algorithm to maintain consensus
+about maps and about other critical information across the cluster. Due to the
+nature of Paxos, Ceph is able to maintain quorum (and thus establish
+consensus) only if a majority of the monitors are ``active``.
+
+It is best to run an odd number of monitors. This is because a cluster that is
+running an odd number of monitors is more resilient than a cluster running an
+even number. For example, in a two-monitor deployment, no failures can be
+tolerated if quorum is to be maintained; in a three-monitor deployment, one
+failure can be tolerated; in a four-monitor deployment, one failure can be
+tolerated; and in a five-monitor deployment, two failures can be tolerated. In
+general, a cluster running an odd number of monitors is best because it avoids
+what is called the *split brain* phenomenon. In short, Ceph is able to operate
+only if a majority of monitors are ``active`` and able to communicate with each
+other, (for example: there must be a single monitor, two out of two monitors,
+two out of three monitors, three out of five monitors, or the like).
For small or non-critical deployments of multi-node Ceph clusters, it is
-advisable to deploy three monitors, and to increase the number of monitors
-to five for larger clusters or to survive a double failure. There is rarely
-justification for seven or more.
-
-Since monitors are lightweight, it is possible to run them on the same
-host as OSDs; however, we recommend running them on separate hosts,
-because `fsync` issues with the kernel may impair performance.
-Dedicated monitor nodes also minimize disruption since monitor and OSD
-daemons are not inactive at the same time when a node crashes or is
-taken down for maintenance.
-
-Dedicated
-monitor nodes also make for cleaner maintenance by avoiding both OSDs and
-a mon going down if a node is rebooted, taken down, or crashes.
+recommended to deploy three monitors. For larger clusters or for clusters that
+are intended to survive a double failure, it is recommended to deploy five
+monitors. Only in rare circumstances is there any justification for deploying
+seven or more monitors.
+
+It is possible to run a monitor on the same host that is running an OSD.
+However, this approach has disadvantages: for example: `fsync` issues with the
+kernel might weaken performance, monitor and OSD daemons might be inactive at
+the same time and cause disruption if the node crashes, is rebooted, or is
+taken down for maintenance. Because of these risks, it is instead
+recommended to run monitors and managers on dedicated hosts.
.. note:: A *majority* of monitors in your cluster must be able to
- reach each other in order to establish a quorum.
+ reach each other in order for quorum to be established.
-Deploy your Hardware
---------------------
+Deploying your Hardware
+-----------------------
-If you are adding a new host when adding a new monitor, see `Hardware
-Recommendations`_ for details on minimum recommendations for monitor hardware.
-To add a monitor host to your cluster, first make sure you have an up-to-date
-version of Linux installed (typically Ubuntu 16.04 or RHEL 7).
+Some operators choose to add a new monitor host at the same time that they add
+a new monitor. For details on the minimum recommendations for monitor hardware,
+see `Hardware Recommendations`_. Before adding a monitor host to the cluster,
+make sure that there is an up-to-date version of Linux installed.
-Add your monitor host to a rack in your cluster, connect it to the network
-and ensure that it has network connectivity.
+Add the newly installed monitor host to a rack in your cluster, connect the
+host to the network, and make sure that the host has network connectivity.
.. _Hardware Recommendations: ../../../start/hardware-recommendations
-Install the Required Software
------------------------------
+Installing the Required Software
+--------------------------------
-For manually deployed clusters, you must install Ceph packages
-manually. See `Installing Packages`_ for details.
-You should configure SSH to a user with password-less authentication
-and root permissions.
+In manually deployed clusters, it is necessary to install Ceph packages
+manually. For details, see `Installing Packages`_. Configure SSH so that it can
+be used by a user that has passwordless authentication and root permissions.
.. _Installing Packages: ../../../install/install-storage-cluster
Adding a Monitor (Manual)
-------------------------
-This procedure creates a ``ceph-mon`` data directory, retrieves the monitor map
-and monitor keyring, and adds a ``ceph-mon`` daemon to your cluster. If
-this results in only two monitor daemons, you may add more monitors by
-repeating this procedure until you have a sufficient number of ``ceph-mon``
-daemons to achieve a quorum.
+The procedure in this section creates a ``ceph-mon`` data directory, retrieves
+both the monitor map and the monitor keyring, and adds a ``ceph-mon`` daemon to
+the cluster. The procedure might result in a Ceph cluster that contains only
+two monitor daemons. To add more monitors until there are enough ``ceph-mon``
+daemons to establish quorum, repeat the procedure.
-At this point you should define your monitor's id. Traditionally, monitors
-have been named with single letters (``a``, ``b``, ``c``, ...), but you are
-free to define the id as you see fit. For the purpose of this document,
-please take into account that ``{mon-id}`` should be the id you chose,
-without the ``mon.`` prefix (i.e., ``{mon-id}`` should be the ``a``
-on ``mon.a``).
+This is a good point at which to define the new monitor's ``id``. Monitors have
+often been named with single letters (``a``, ``b``, ``c``, etc.), but you are
+free to define the ``id`` however you see fit. In this document, ``{mon-id}``
+refers to the ``id`` exclusive of the ``mon.`` prefix: for example, if
+``mon.a`` has been chosen as the ``id`` of a monitor, then ``{mon-id}`` is
+``a``. ???
-#. Create the default directory on the machine that will host your
- new monitor:
+#. Create a data directory on the machine that will host the new monitor:
.. prompt:: bash $
- ssh {new-mon-host}
- sudo mkdir /var/lib/ceph/mon/ceph-{mon-id}
+ ssh {new-mon-host}
+ sudo mkdir /var/lib/ceph/mon/ceph-{mon-id}
-#. Create a temporary directory ``{tmp}`` to keep the files needed during
- this process. This directory should be different from the monitor's default
- directory created in the previous step, and can be removed after all the
- steps are executed:
+#. Create a temporary directory ``{tmp}`` that will contain the files needed
+ during this procedure. This directory should be different from the data
+ directory created in the previous step. Because this is a temporary
+ directory, it can be removed after the procedure is complete:
.. prompt:: bash $
- mkdir {tmp}
+ mkdir {tmp}
-#. Retrieve the keyring for your monitors, where ``{tmp}`` is the path to
- the retrieved keyring, and ``{key-filename}`` is the name of the file
- containing the retrieved monitor key:
+#. Retrieve the keyring for your monitors (``{tmp}`` is the path to the
+ retrieved keyring and ``{key-filename}`` is the name of the file that
+ contains the retrieved monitor key):
.. prompt:: bash $
ceph auth get mon. -o {tmp}/{key-filename}
-#. Retrieve the monitor map, where ``{tmp}`` is the path to
- the retrieved monitor map, and ``{map-filename}`` is the name of the file
- containing the retrieved monitor map:
+#. Retrieve the monitor map (``{tmp}`` is the path to the retrieved monitor map
+ and ``{map-filename}`` is the name of the file that contains the retrieved
+ monitor map):
.. prompt:: bash $
ceph mon getmap -o {tmp}/{map-filename}
-#. Prepare the monitor's data directory created in the first step. You must
- specify the path to the monitor map so that you can retrieve the
- information about a quorum of monitors and their ``fsid``. You must also
- specify a path to the monitor keyring:
-
+#. Prepare the monitor's data directory, which was created in the first step.
+ The following command must specify the path to the monitor map (so that
+ information about a quorum of monitors and their ``fsid``\s can be
+ retrieved) and specify the path to the monitor keyring:
+
.. prompt:: bash $
sudo ceph-mon -i {mon-id} --mkfs --monmap {tmp}/{map-filename} --keyring {tmp}/{key-filename}
-
-#. Start the new monitor and it will automatically join the cluster.
- The daemon needs to know which address to bind to, via either the
- ``--public-addr {ip}`` or ``--public-network {network}`` argument.
+#. Start the new monitor. It will automatically join the cluster. To provide
+ information to the daemon about which address to bind to, use either the
+ ``--public-addr {ip}`` option or the ``--public-network {network}`` option.
For example:
-
+
.. prompt:: bash $
ceph-mon -i {mon-id} --public-addr {ip:port}
Removing Monitors
=================
-When you remove monitors from a cluster, consider that Ceph monitors use
-Paxos to establish consensus about the master cluster map. You must have
-a sufficient number of monitors to establish a quorum for consensus about
-the cluster map.
+When monitors are removed from a cluster, it is important to remember
+that Ceph monitors use Paxos to maintain consensus about the cluster
+map. Such consensus is possible only if the number of monitors is sufficient
+to establish quorum.
+
.. _Removing a Monitor (Manual):
Removing a Monitor (Manual)
---------------------------
-This procedure removes a ``ceph-mon`` daemon from your cluster. If this
-procedure results in only two monitor daemons, you may add or remove another
-monitor until you have a number of ``ceph-mon`` daemons that can achieve a
-quorum.
+The procedure in this section removes a ``ceph-mon`` daemon from the cluster.
+The procedure might result in a Ceph cluster that contains a number of monitors
+insufficient to maintain quorum, so plan carefully. When replacing an old
+monitor with a new monitor, add the new monitor first, wait for quorum to be
+established, and then remove the old monitor. This ensures that quorum is not
+lost.
+
#. Stop the monitor:
.. prompt:: bash $
service ceph -a stop mon.{mon-id}
-
+
#. Remove the monitor from the cluster:
.. prompt:: bash $
ceph mon remove {mon-id}
-
-#. Remove the monitor entry from ``ceph.conf``.
+
+#. Remove the monitor entry from the ``ceph.conf`` file:
.. _rados-mon-remove-from-unhealthy:
+
Removing Monitors from an Unhealthy Cluster
-------------------------------------------
-This procedure removes a ``ceph-mon`` daemon from an unhealthy
-cluster, for example a cluster where the monitors cannot form a
-quorum.
-
+The procedure in this section removes a ``ceph-mon`` daemon from an unhealthy
+cluster (for example, a cluster whose monitors are unable to form a quorum).
#. Stop all ``ceph-mon`` daemons on all monitor hosts:
ssh {mon-host}
systemctl stop ceph-mon.target
- Repeat for all monitor hosts.
+ Repeat this step on every monitor host.
-#. Identify a surviving monitor and log in to that host:
+#. Identify a surviving monitor and log in to the monitor's host:
.. prompt:: bash $
ssh {mon-host}
-#. Extract a copy of the monmap file:
+#. Extract a copy of the ``monmap`` file by running a command of the following
+ form:
.. prompt:: bash $
ceph-mon -i {mon-id} --extract-monmap {map-path}
- In most cases, this command will be:
+ Here is a more concrete example. In this example, ``hostname`` is the
+ ``{mon-id}`` and ``/tmp/monpap`` is the ``{map-path}``:
.. prompt:: bash $
ceph-mon -i `hostname` --extract-monmap /tmp/monmap
-#. Remove the non-surviving or problematic monitors. For example, if
- you have three monitors, ``mon.a``, ``mon.b``, and ``mon.c``, where
- only ``mon.a`` will survive, follow the example below:
+#. Remove the non-surviving or otherwise problematic monitors:
.. prompt:: bash $
monmaptool {map-path} --rm {mon-id}
- For example,
+ For example, suppose that there are three monitors |---| ``mon.a``, ``mon.b``,
+ and ``mon.c`` |---| and that only ``mon.a`` will survive:
.. prompt:: bash $
monmaptool /tmp/monmap --rm b
monmaptool /tmp/monmap --rm c
-
-#. Inject the surviving map with the removed monitors into the
- surviving monitor(s). For example, to inject a map into monitor
- ``mon.a``, follow the example below:
+
+#. Inject the surviving map that includes the removed monitors into the
+ monmap of the surviving monitor(s):
.. prompt:: bash $
ceph-mon -i {mon-id} --inject-monmap {map-path}
- For example:
+ Continuing with the above example, inject a map into monitor ``mon.a`` by
+ running the following command:
.. prompt:: bash $
ceph-mon -i a --inject-monmap /tmp/monmap
+
#. Start only the surviving monitors.
-#. Verify the monitors form a quorum (``ceph -s``).
+#. Verify that the monitors form a quorum by running the command ``ceph -s``.
-#. You may wish to archive the removed monitors' data directory in
- ``/var/lib/ceph/mon`` in a safe location, or delete it if you are
- confident the remaining monitors are healthy and are sufficiently
- redundant.
+#. The data directory of the removed monitors is in ``/var/lib/ceph/mon``:
+ either archive this data directory in a safe location or delete this data
+ directory. However, do not delete it unless you are confident that the
+ remaining monitors are healthy and sufficiently redundant. Make sure that
+ there is enough room for the live DB to expand and compact, and make sure
+ that there is also room for an archived copy of the DB. The archived copy
+ can be compressed.
.. _Changing a Monitor's IP address:
.. important:: Existing monitors are not supposed to change their IP addresses.
-Monitors are critical components of a Ceph cluster, and they need to maintain a
-quorum for the whole system to work properly. To establish a quorum, the
-monitors need to discover each other. Ceph has strict requirements for
-discovering monitors.
+Monitors are critical components of a Ceph cluster. The entire system can work
+properly only if the monitors maintain quorum, and quorum can be established
+only if the monitors have discovered each other by means of their IP addresses.
+Ceph has strict requirements on the discovery of monitors.
-Ceph clients and other Ceph daemons use ``ceph.conf`` to discover monitors.
-However, monitors discover each other using the monitor map, not ``ceph.conf``.
-For example, if you refer to `Adding a Monitor (Manual)`_ you will see that you
-need to obtain the current monmap for the cluster when creating a new monitor,
-as it is one of the required arguments of ``ceph-mon -i {mon-id} --mkfs``. The
-following sections explain the consistency requirements for Ceph monitors, and a
-few safe ways to change a monitor's IP address.
+Although the ``ceph.conf`` file is used by Ceph clients and other Ceph daemons
+to discover monitors, the monitor map is used by monitors to discover each
+other. This is why it is necessary to obtain the current ``monmap`` at the time
+a new monitor is created: as can be seen above in `Adding a Monitor (Manual)`_,
+the ``monmap`` is one of the arguments required by the ``ceph-mon -i {mon-id}
+--mkfs`` command. The following sections explain the consistency requirements
+for Ceph monitors, and also explain a number of safe ways to change a monitor's
+IP address.
Consistency Requirements
------------------------
-A monitor always refers to the local copy of the monmap when discovering other
-monitors in the cluster. Using the monmap instead of ``ceph.conf`` avoids
-errors that could break the cluster (e.g., typos in ``ceph.conf`` when
-specifying a monitor address or port). Since monitors use monmaps for discovery
-and they share monmaps with clients and other Ceph daemons, the monmap provides
-monitors with a strict guarantee that their consensus is valid.
+When a monitor discovers other monitors in the cluster, it always refers to the
+local copy of the monitor map. Using the monitor map instead of using the
+``ceph.conf`` file avoids errors that could break the cluster (for example,
+typos or other slight errors in ``ceph.conf`` when a monitor address or port is
+specified). Because monitors use monitor maps for discovery and because they
+share monitor maps with Ceph clients and other Ceph daemons, the monitor map
+provides monitors with a strict guarantee that their consensus is valid.
Strict consistency also applies to updates to the monmap. As with any other
updates on the monitor, changes to the monmap always run through a distributed
consensus algorithm called `Paxos`_. The monitors must agree on each update to
-the monmap, such as adding or removing a monitor, to ensure that each monitor in
-the quorum has the same version of the monmap. Updates to the monmap are
+the monmap, such as adding or removing a monitor, to ensure that each monitor
+in the quorum has the same version of the monmap. Updates to the monmap are
incremental so that monitors have the latest agreed upon version, and a set of
-previous versions, allowing a monitor that has an older version of the monmap to
-catch up with the current state of the cluster.
-
-If monitors discovered each other through the Ceph configuration file instead of
-through the monmap, it would introduce additional risks because the Ceph
-configuration files are not updated and distributed automatically. Monitors
-might inadvertently use an older ``ceph.conf`` file, fail to recognize a
-monitor, fall out of a quorum, or develop a situation where `Paxos`_ is not able
-to determine the current state of the system accurately. Consequently, making
-changes to an existing monitor's IP address must be done with great care.
-
-
-Changing a Monitor's IP address (The Right Way)
------------------------------------------------
-
-Changing a monitor's IP address in ``ceph.conf`` only is not sufficient to
-ensure that other monitors in the cluster will receive the update. To change a
-monitor's IP address, you must add a new monitor with the IP address you want
-to use (as described in `Adding a Monitor (Manual)`_), ensure that the new
-monitor successfully joins the quorum; then, remove the monitor that uses the
-old IP address. Then, update the ``ceph.conf`` file to ensure that clients and
-other daemons know the IP address of the new monitor.
-
-For example, lets assume there are three monitors in place, such as ::
-
- [mon.a]
- host = host01
- addr = 10.0.0.1:6789
- [mon.b]
- host = host02
- addr = 10.0.0.2:6789
- [mon.c]
- host = host03
- addr = 10.0.0.3:6789
-
-To change ``mon.c`` to ``host04`` with the IP address ``10.0.0.4``, follow the
-steps in `Adding a Monitor (Manual)`_ by adding a new monitor ``mon.d``. Ensure
-that ``mon.d`` is running before removing ``mon.c``, or it will break the
-quorum. Remove ``mon.c`` as described on `Removing a Monitor (Manual)`_. Moving
-all three monitors would thus require repeating this process as many times as
-needed.
-
-
-Changing a Monitor's IP address (The Messy Way)
------------------------------------------------
-
-There may come a time when the monitors must be moved to a different network, a
-different part of the datacenter or a different datacenter altogether. While it
-is possible to do it, the process becomes a bit more hazardous.
-
-In such a case, the solution is to generate a new monmap with updated IP
-addresses for all the monitors in the cluster, and inject the new map on each
-individual monitor. This is not the most user-friendly approach, but we do not
-expect this to be something that needs to be done every other week. As it is
-clearly stated on the top of this section, monitors are not supposed to change
-IP addresses.
-
-Using the previous monitor configuration as an example, assume you want to move
-all the monitors from the ``10.0.0.x`` range to ``10.1.0.x``, and these
-networks are unable to communicate. Use the following procedure:
-
-#. Retrieve the monitor map, where ``{tmp}`` is the path to
- the retrieved monitor map, and ``{filename}`` is the name of the file
- containing the retrieved monitor map:
+previous versions, allowing a monitor that has an older version of the monmap
+to catch up with the current state of the cluster.
+
+There are additional advantages to using the monitor map rather than
+``ceph.conf`` when monitors discover each other. Because ``ceph.conf`` is not
+automatically updated and distributed, its use would bring certain risks:
+monitors might use an outdated ``ceph.conf`` file, might fail to recognize a
+specific monitor, might fall out of quorum, and might develop a situation in
+which `Paxos`_ is unable to accurately ascertain the current state of the
+system. Because of these risks, any changes to an existing monitor's IP address
+must be made with great care.
+
+.. _operations_add_or_rm_mons_changing_mon_ip:
+
+Changing a Monitor's IP address (Preferred Method)
+--------------------------------------------------
+
+If a monitor's IP address is changed only in the ``ceph.conf`` file, there is
+no guarantee that the other monitors in the cluster will receive the update.
+For this reason, the preferred method to change a monitor's IP address is as
+follows: add a new monitor with the desired IP address (as described in `Adding
+a Monitor (Manual)`_), make sure that the new monitor successfully joins the
+quorum, remove the monitor that is using the old IP address, and update the
+``ceph.conf`` file to ensure that clients and other daemons are made aware of
+the new monitor's IP address.
+
+For example, suppose that there are three monitors in place::
+
+ [mon.a]
+ host = host01
+ addr = 10.0.0.1:6789
+ [mon.b]
+ host = host02
+ addr = 10.0.0.2:6789
+ [mon.c]
+ host = host03
+ addr = 10.0.0.3:6789
+
+To change ``mon.c`` so that its name is ``host04`` and its IP address is
+``10.0.0.4``: (1) follow the steps in `Adding a Monitor (Manual)`_ to add a new
+monitor ``mon.d``, (2) make sure that ``mon.d`` is running before removing
+``mon.c`` or else quorum will be broken, and (3) follow the steps in `Removing
+a Monitor (Manual)`_ to remove ``mon.c``. To move all three monitors to new IP
+addresses, repeat this process.
+
+Changing a Monitor's IP address (Advanced Method)
+-------------------------------------------------
+
+There are cases in which the method outlined in :ref"`<Changing a Monitor's IP
+Address (Preferred Method)> operations_add_or_rm_mons_changing_mon_ip` cannot
+be used. For example, it might be necessary to move the cluster's monitors to a
+different network, to a different part of the datacenter, or to a different
+datacenter altogether. It is still possible to change the monitors' IP
+addresses, but a different method must be used.
+
+For such cases, a new monitor map with updated IP addresses for every monitor
+in the cluster must be generated and injected on each monitor. Although this
+method is not particularly easy, such a major migration is unlikely to be a
+routine task. As stated at the beginning of this section, existing monitors are
+not supposed to change their IP addresses.
+
+Continue with the monitor configuration in the example from :ref"`<Changing a
+Monitor's IP Address (Preferred Method)>
+operations_add_or_rm_mons_changing_mon_ip` . Suppose that all of the monitors
+are to be moved from the ``10.0.0.x`` range to the ``10.1.0.x`` range, and that
+these networks are unable to communicate. Carry out the following procedure:
+
+#. Retrieve the monitor map (``{tmp}`` is the path to the retrieved monitor
+ map, and ``{filename}`` is the name of the file that contains the retrieved
+ monitor map):
.. prompt:: bash $
ceph mon getmap -o {tmp}/{filename}
-#. The following example demonstrates the contents of the monmap:
+#. Check the contents of the monitor map:
.. prompt:: bash $
monmaptool --print {tmp}/{filename}
- ::
+ ::
- monmaptool: monmap file {tmp}/{filename}
- epoch 1
- fsid 224e376d-c5fe-4504-96bb-ea6332a19e61
- last_changed 2012-12-17 02:46:41.591248
- created 2012-12-17 02:46:41.591248
- 0: 10.0.0.1:6789/0 mon.a
- 1: 10.0.0.2:6789/0 mon.b
- 2: 10.0.0.3:6789/0 mon.c
+ monmaptool: monmap file {tmp}/{filename}
+ epoch 1
+ fsid 224e376d-c5fe-4504-96bb-ea6332a19e61
+ last_changed 2012-12-17 02:46:41.591248
+ created 2012-12-17 02:46:41.591248
+ 0: 10.0.0.1:6789/0 mon.a
+ 1: 10.0.0.2:6789/0 mon.b
+ 2: 10.0.0.3:6789/0 mon.c
-#. Remove the existing monitors:
+#. Remove the existing monitors from the monitor map:
.. prompt:: bash $
monmaptool --rm a --rm b --rm c {tmp}/{filename}
-
::
- monmaptool: monmap file {tmp}/{filename}
- monmaptool: removing a
- monmaptool: removing b
- monmaptool: removing c
- monmaptool: writing epoch 1 to {tmp}/{filename} (0 monitors)
+ monmaptool: monmap file {tmp}/{filename}
+ monmaptool: removing a
+ monmaptool: removing b
+ monmaptool: removing c
+ monmaptool: writing epoch 1 to {tmp}/{filename} (0 monitors)
-#. Add the new monitor locations:
+#. Add the new monitor locations to the monitor map:
.. prompt:: bash $
monmaptool --add a 10.1.0.1:6789 --add b 10.1.0.2:6789 --add c 10.1.0.3:6789 {tmp}/{filename}
-
::
-
+
monmaptool: monmap file {tmp}/{filename}
monmaptool: writing epoch 1 to {tmp}/{filename} (3 monitors)
-#. Check new contents:
+#. Check the new contents of the monitor map:
.. prompt:: bash $
monmaptool --print {tmp}/{filename}
-
+
::
- monmaptool: monmap file {tmp}/{filename}
- epoch 1
- fsid 224e376d-c5fe-4504-96bb-ea6332a19e61
- last_changed 2012-12-17 02:46:41.591248
- created 2012-12-17 02:46:41.591248
- 0: 10.1.0.1:6789/0 mon.a
- 1: 10.1.0.2:6789/0 mon.b
- 2: 10.1.0.3:6789/0 mon.c
+ monmaptool: monmap file {tmp}/{filename}
+ epoch 1
+ fsid 224e376d-c5fe-4504-96bb-ea6332a19e61
+ last_changed 2012-12-17 02:46:41.591248
+ created 2012-12-17 02:46:41.591248
+ 0: 10.1.0.1:6789/0 mon.a
+ 1: 10.1.0.2:6789/0 mon.b
+ 2: 10.1.0.3:6789/0 mon.c
-At this point, we assume the monitors (and stores) are installed at the new
-location. The next step is to propagate the modified monmap to the new
-monitors, and inject the modified monmap into each new monitor.
+At this point, we assume that the monitors (and stores) have been installed at
+the new location. Next, propagate the modified monitor map to the new monitors,
+and inject the modified monitor map into each new monitor.
-#. First, make sure to stop all your monitors. Injection must be done while
- the daemon is not running.
+#. Make sure all of your monitors have been stopped. Never inject into a
+ monitor while the monitor daemon is running.
-#. Inject the monmap:
+#. Inject the monitor map:
.. prompt:: bash $
ceph-mon -i {mon-id} --inject-monmap {tmp}/{filename}
-#. Restart the monitors.
+#. Restart all of the monitors.
+
+Migration to the new location is now complete. The monitors should operate
+successfully.
-After this step, migration to the new location is complete and
-the monitors should operate successfully.
.. _Manual Deployment: ../../../install/manual-deployment
.. _Monitor Bootstrap: ../../../dev/mon-bootstrap
.. _Paxos: https://en.wikipedia.org/wiki/Paxos_(computer_science)
+
+.. |---| unicode:: U+2014 .. EM DASH
+ :trim:
projects / ceph.git / blobdiff