distribution-specific installations
-----------------------------------
-Some Linux distributions may already include up-to-date Ceph packages. In that
-case, you can install cephadm directly. For example:
+Some Linux distributions may already include up-to-date Ceph packages. In
+that case, you can install cephadm directly. For example:
In Ubuntu:
* First, determine what version of Ceph you will need. You can use the releases
page to find the `latest active releases <https://docs.ceph.com/en/latest/releases/#active-releases>`_.
- For example, we might look at that page and find that ``17.2.6`` is the latest
+ For example, we might look at that page and find that ``18.2.0`` is the latest
active release.
* Use ``curl`` to fetch a build of cephadm for that release.
.. prompt:: bash #
:substitutions:
- CEPH_RELEASE=17.2.6 # replace this with the active release
+ CEPH_RELEASE=18.2.0 # replace this with the active release
curl --silent --remote-name --location https://download.ceph.com/rpm-${CEPH_RELEASE}/el9/noarch/cephadm
Ensure the ``cephadm`` file is executable:
python3.8 ./cephadm <arguments...>
-* Although the standalone cephadm is sufficient to get a cluster started, it is
- convenient to have the ``cephadm`` command installed on the host. To install
- the packages that provide the ``cephadm`` command, run the following
- commands:
+.. _cephadm_update:
- .. prompt:: bash #
- :substitutions:
+update cephadm
+--------------
- ./cephadm add-repo --release |stable-release|
- ./cephadm install
+The cephadm binary can be used to bootstrap a cluster and for a variety
+of other management and debugging tasks. The Ceph team strongly recommends
+using an actively supported version of cephadm. Additionally, although
+the standalone cephadm is sufficient to get a cluster started, it is
+convenient to have the ``cephadm`` command installed on the host. Older or LTS
+distros may also have ``cephadm`` packages that are out-of-date and
+running the commands below can help install a more recent version
+from the Ceph project's repositories.
- Confirm that ``cephadm`` is now in your PATH by running ``which``:
+To install the packages provided by the Ceph project that provide the
+``cephadm`` command, run the following commands:
- .. prompt:: bash #
+.. prompt:: bash #
+ :substitutions:
- which cephadm
+ ./cephadm add-repo --release |stable-release|
+ ./cephadm install
- A successful ``which cephadm`` command will return this:
+Confirm that ``cephadm`` is now in your PATH by running ``which`` or
+``command -v``:
- .. code-block:: bash
+.. prompt:: bash #
- /usr/sbin/cephadm
+ which cephadm
+
+A successful ``which cephadm`` command will return this:
+
+.. code-block:: bash
+
+ /usr/sbin/cephadm
Bootstrap a new cluster
=======================
You must pass the IP address of the Ceph cluster's first host to the ``ceph
bootstrap`` command, so you'll need to know the IP address of that host.
+.. important:: ``ssh`` must be installed and running in order for the
+ bootstrapping procedure to succeed.
+
.. note:: If there are multiple networks and interfaces, be sure to choose one
that will be accessible by any host accessing the Ceph cluster.
with this label will (also) get a copy of ``/etc/ceph/ceph.conf`` and
``/etc/ceph/ceph.client.admin.keyring``.
+.. _cephadm-bootstrap-further-info:
+
Further information about cephadm bootstrap
-------------------------------------------
Adding Hosts
============
-Next, add all hosts to the cluster by following :ref:`cephadm-adding-hosts`.
+Add all hosts to the cluster by following the instructions in
+:ref:`cephadm-adding-hosts`.
-By default, a ``ceph.conf`` file and a copy of the ``client.admin`` keyring
-are maintained in ``/etc/ceph`` on all hosts with the ``_admin`` label, which is initially
-applied only to the bootstrap host. We usually recommend that one or more other hosts be
-given the ``_admin`` label so that the Ceph CLI (e.g., via ``cephadm shell``) is easily
-accessible on multiple hosts. To add the ``_admin`` label to additional host(s):
+By default, a ``ceph.conf`` file and a copy of the ``client.admin`` keyring are
+maintained in ``/etc/ceph`` on all hosts that have the ``_admin`` label. This
+label is initially applied only to the bootstrap host. We usually recommend
+that one or more other hosts be given the ``_admin`` label so that the Ceph CLI
+(for example, via ``cephadm shell``) is easily accessible on multiple hosts. To add
+the ``_admin`` label to additional host(s), run a command of the following form:
.. prompt:: bash #
ceph orch host label add *<host>* _admin
+
Adding additional MONs
======================
cephadm --image *<hostname>*:5000/ceph/ceph bootstrap --mon-ip *<mon-ip>*
.. _cluster network: ../rados/configuration/network-config-ref#cluster-network
+
+.. _cephadm-bootstrap-custom-ssh-keys:
+
+Deployment with custom SSH keys
+-------------------------------
+
+Bootstrap allows users to create their own private/public SSH key pair
+rather than having cephadm generate them automatically.
+
+To use custom SSH keys, pass the ``--ssh-private-key`` and ``--ssh-public-key``
+fields to bootstrap. Both parameters require a path to the file where the
+keys are stored:
+
+.. prompt:: bash #
+
+ cephadm bootstrap --mon-ip <ip-addr> --ssh-private-key <private-key-filepath> --ssh-public-key <public-key-filepath>
+
+This setup allows users to use a key that has already been distributed to hosts
+the user wants in the cluster before bootstrap.
+
+.. note:: In order for cephadm to connect to other hosts you'd like to add
+ to the cluster, make sure the public key of the key pair provided is set up
+ as an authorized key for the ssh user being used, typically root. If you'd
+ like more info on using a non-root user as the ssh user, see :ref:`cephadm-bootstrap-further-info`
+
+.. _cephadm-bootstrap-ca-signed-keys:
+
+Deployment with CA signed SSH keys
+----------------------------------
+
+As an alternative to standard public key authentication, cephadm also supports
+deployment using CA signed keys. Before bootstrapping it's recommended to set up
+the CA public key as a trusted CA key on hosts you'd like to eventually add to
+the cluster. For example:
+
+.. prompt:: bash
+
+ # we will act as our own CA, therefore we'll need to make a CA key
+ [root@host1 ~]# ssh-keygen -t rsa -f ca-key -N ""
+
+ # make the ca key trusted on the host we've generated it on
+ # this requires adding in a line in our /etc/sshd_config
+ # to mark this key as trusted
+ [root@host1 ~]# cp ca-key.pub /etc/ssh
+ [root@host1 ~]# vi /etc/ssh/sshd_config
+ [root@host1 ~]# cat /etc/ssh/sshd_config | grep ca-key
+ TrustedUserCAKeys /etc/ssh/ca-key.pub
+ # now restart sshd so it picks up the config change
+ [root@host1 ~]# systemctl restart sshd
+
+ # now, on all other hosts we want in the cluster, also install the CA key
+ [root@host1 ~]# scp /etc/ssh/ca-key.pub host2:/etc/ssh/
+
+ # on other hosts, make the same changes to the sshd_config
+ [root@host2 ~]# vi /etc/ssh/sshd_config
+ [root@host2 ~]# cat /etc/ssh/sshd_config | grep ca-key
+ TrustedUserCAKeys /etc/ssh/ca-key.pub
+ # and restart sshd so it picks up the config change
+ [root@host2 ~]# systemctl restart sshd
+
+Once the CA key has been installed and marked as a trusted key, you are ready
+to use a private key/CA signed cert combination for SSH. Continuing with our
+current example, we will create a new key-pair for for host access and then
+sign it with our CA key
+
+.. prompt:: bash
+
+ # make a new key pair
+ [root@host1 ~]# ssh-keygen -t rsa -f cephadm-ssh-key -N ""
+ # sign the private key. This will create a new cephadm-ssh-key-cert.pub
+ # note here we're using user "root". If you'd like to use a non-root
+ # user the arguments to the -I and -n params would need to be adjusted
+ # Additionally, note the -V param indicates how long until the cert
+ # this creates will expire
+ [root@host1 ~]# ssh-keygen -s ca-key -I user_root -n root -V +52w cephadm-ssh-key
+ [root@host1 ~]# ls
+ ca-key ca-key.pub cephadm-ssh-key cephadm-ssh-key-cert.pub cephadm-ssh-key.pub
+
+ # verify our signed key is working. To do this, make sure the generated private
+ # key ("cephadm-ssh-key" in our example) and the newly signed cert are stored
+ # in the same directory. Then try to ssh using the private key
+ [root@host1 ~]# ssh -i cephadm-ssh-key host2
+
+Once you have your private key and corresponding CA signed cert and have tested
+SSH authentication using that key works, you can pass those keys to bootstrap
+in order to have cephadm use them for SSHing between cluster hosts
+
+.. prompt:: bash
+
+ [root@host1 ~]# cephadm bootstrap --mon-ip <ip-addr> --ssh-private-key cephadm-ssh-key --ssh-signed-cert cephadm-ssh-key-cert.pub
+
+Note that this setup does not require installing the corresponding public key
+from the private key passed to bootstrap on other nodes. In fact, cephadm will
+reject the ``--ssh-public-key`` argument when passed along with ``--ssh-signed-cert``.
+Not because having the public key breaks anything, but because it is not at all needed
+for this setup and it helps bootstrap differentiate if the user wants the CA signed
+keys setup or standard pubkey encryption. What this means is, SSH key rotation
+would simply be a matter of getting another key signed by the same CA and providing
+cephadm with the new private key and signed cert. No additional distribution of
+keys to cluster nodes is needed after the initial setup of the CA key as a trusted key,
+no matter how many new private key/signed cert pairs are rotated in.
projects / ceph.git / blobdiff