means that those services cannot currently be managed by cephadm
(e.g., restarted, upgraded, included in `ceph orch ps`).
-You can manage the host(s) by running the following command:
+* You can manage the host(s) by running the following command:
-.. prompt:: bash #
+ .. prompt:: bash #
- ceph orch host add *<hostname>*
+ ceph orch host add *<hostname>*
-.. note::
+ .. note::
- You might need to configure SSH access to the remote host
- before this will work.
+ You might need to configure SSH access to the remote host
+ before this will work.
-Alternatively, you can manually connect to the host and ensure that
-services on that host are removed or migrated to a host that is
-managed by *cephadm*.
+* See :ref:`cephadm-fqdn` for more information about host names and
+ domain names.
-This warning can be disabled entirely by running the following
-command:
+* Alternatively, you can manually connect to the host and ensure that
+ services on that host are removed or migrated to a host that is
+ managed by *cephadm*.
-.. prompt:: bash #
+* This warning can be disabled entirely by running the following
+ command:
- ceph config set mgr mgr/cephadm/warn_on_stray_hosts false
+ .. prompt:: bash #
-See :ref:`cephadm-fqdn` for more information about host names and
-domain names.
+ ceph config set mgr mgr/cephadm/warn_on_stray_hosts false
CEPHADM_STRAY_DAEMON
~~~~~~~~~~~~~~~~~~~~
services cannot currently be managed by cephadm (e.g., restarted,
upgraded, or included in `ceph orch ps`).
-If the daemon is a stateful one (monitor or OSD), it should be adopted
-by cephadm; see :ref:`cephadm-adoption`. For stateless daemons, it is
-usually easiest to provision a new daemon with the ``ceph orch apply``
-command and then stop the unmanaged daemon.
+* If the daemon is a stateful one (monitor or OSD), it should be adopted
+ by cephadm; see :ref:`cephadm-adoption`. For stateless daemons, it is
+ usually easiest to provision a new daemon with the ``ceph orch apply``
+ command and then stop the unmanaged daemon.
-This warning can be disabled entirely by running the following command:
+* If the stray daemon(s) are running on hosts not managed by cephadm, you can manage the host(s) by running the following command:
-.. prompt:: bash #
+ .. prompt:: bash #
+
+ ceph orch host add *<hostname>*
+
+ .. note::
+
+ You might need to configure SSH access to the remote host
+ before this will work.
- ceph config set mgr mgr/cephadm/warn_on_stray_daemons false
+* See :ref:`cephadm-fqdn` for more information about host names and
+ domain names.
+
+* This warning can be disabled entirely by running the following command:
+
+ .. prompt:: bash #
+
+ ceph config set mgr mgr/cephadm/warn_on_stray_daemons false
CEPHADM_HOST_CHECK_FAILED
~~~~~~~~~~~~~~~~~~~~~~~~~
NAME HEALTHCHECK STATUS DESCRIPTION
kernel_security CEPHADM_CHECK_KERNEL_LSM enabled checks SELINUX/Apparmor profiles are consistent across cluster hosts
os_subscription CEPHADM_CHECK_SUBSCRIPTION enabled checks subscription states are consistent for all cluster hosts
- public_network CEPHADM_CHECK_PUBLIC_MEMBERSHIP enabled check that all hosts have a NIC on the Ceph public_netork
+ public_network CEPHADM_CHECK_PUBLIC_MEMBERSHIP enabled check that all hosts have a NIC on the Ceph public_network
osd_mtu_size CEPHADM_CHECK_MTU enabled check that OSD hosts share a common MTU setting
osd_linkspeed CEPHADM_CHECK_LINKSPEED enabled check that OSD hosts share a common linkspeed
network_missing CEPHADM_CHECK_NETWORK_MISSING enabled checks that the cluster/public networks defined exist on the Ceph hosts
CEPHADM_CHECK_KERNEL_VERSION
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The OS kernel version (maj.min) is checked for consistency across the hosts.
-The kernel version of the majority of the hosts is used as the basis for
+The kernel version of the majority of the hosts is used as the basis for
identifying anomalies.
.. _client_keyrings_and_configs:
Client keyrings and configs
===========================
-
Cephadm can distribute copies of the ``ceph.conf`` file and client keyring
-files to hosts. It is usually a good idea to store a copy of the config and
-``client.admin`` keyring on any host used to administer the cluster via the
-CLI. By default, cephadm does this for any nodes that have the ``_admin``
-label (which normally includes the bootstrap host).
+files to hosts. Starting from versions 16.2.10 (Pacific) and 17.2.1 (Quincy),
+in addition to the default location ``/etc/ceph/`` cephadm also stores config
+and keyring files in the ``/var/lib/ceph/<fsid>/config`` directory. It is usually
+a good idea to store a copy of the config and ``client.admin`` keyring on any host
+used to administer the cluster via the CLI. By default, cephadm does this for any
+nodes that have the ``_admin`` label (which normally includes the bootstrap host).
+
+.. note:: Ceph daemons will still use files on ``/etc/ceph/``. The new configuration
+ location ``/var/lib/ceph/<fsid>/config`` is used by cephadm only. Having this config
+ directory under the fsid helps cephadm to load the configuration associated with
+ the cluster.
+
When a client keyring is placed under management, cephadm will:
- build a list of target hosts based on the specified placement spec (see
:ref:`orchestrator-cli-placement-spec`)
- store a copy of the ``/etc/ceph/ceph.conf`` file on the specified host(s)
+ - store a copy of the ``ceph.conf`` file at ``/var/lib/ceph/<fsid>/config/ceph.conf`` on the specified host(s)
+ - store a copy of the ``ceph.client.admin.keyring`` file at ``/var/lib/ceph/<fsid>/config/ceph.client.admin.keyring`` on the specified host(s)
- store a copy of the keyring file on the specified host(s)
- update the ``ceph.conf`` file as needed (e.g., due to a change in the cluster monitors)
- update the keyring file if the entity's key is changed (e.g., via ``ceph
ceph config set mgr mgr/cephadm/manage_etc_ceph_ceph_conf_hosts label:bare_config
(See :ref:`orchestrator-cli-placement-spec` for more information about placement specs.)
+
+Purging a cluster
+=================
+
+.. danger:: THIS OPERATION WILL DESTROY ALL DATA STORED IN THIS CLUSTER
+
+In order to destroy a cluster and delete all data stored in this cluster, disable
+cephadm to stop all orchestration operations (so we avoid deploying new daemons).
+
+.. prompt:: bash #
+
+ ceph mgr module disable cephadm
+
+Then verify the FSID of the cluster:
+
+.. prompt:: bash #
+
+ ceph fsid
+
+Purge ceph daemons from all hosts in the cluster
+
+.. prompt:: bash #
+
+ # For each host:
+ cephadm rm-cluster --force --zap-osds --fsid <fsid>
projects / ceph.git / blobdiff