Storage pool type: `cephfs`
-http://ceph.com[Ceph] is a distributed object store and file system designed to
-provide excellent performance, reliability and scalability. CephFS implements a
-POSIX-compliant filesystem storage, with the following advantages:
-
-* thin provisioning
-* distributed and redundant (striped over multiple OSDs)
-* snapshot capabilities
-* self healing
-* no single point of failure
-* scalable to the exabyte level
-* kernel and user space implementation available
-
-NOTE: For smaller deployments, it is also possible to run Ceph
-services directly on your {pve} nodes. Recent hardware has plenty
-of CPU power and RAM, so running storage services and VMs on same node
-is possible.
+CephFS implements a POSIX-compliant filesystem, using a https://ceph.com[Ceph]
+storage cluster to store its data. As CephFS builds upon Ceph, it shares most of
+its properties. This includes redundancy, scalability, self-healing, and high
+availability.
+
+TIP: {pve} can xref:chapter_pveceph[manage Ceph setups], which makes
+configuring a CephFS storage easier. As modern hardware offers a lot of
+processing power and RAM, running storage services and VMs on same node is
+possible without a significant performance impact.
+
+To use the CephFS storage plugin, you must replace the stock Debian Ceph client,
+by adding our xref:sysadmin_package_repositories_ceph[Ceph repository].
+Once added, run `apt update`, followed by `apt dist-upgrade`, in order to get
+the newest packages.
+
+WARNING: Please ensure that there are no other Ceph repositories configured.
+Otherwise the installation will fail or there will be mixed package versions on
+the node, leading to unexpected behavior.
[[storage_cephfs_config]]
Configuration
~~~~~~~~~~~~~
This backend supports the common storage properties `nodes`,
-`disable`, `content`, and the following `cephfs` specific properties:
+`disable`, `content`, as well as the following `cephfs` specific properties:
+
+fs-name::
+
+Name of the Ceph FS.
monhost::
-List of monitor daemon IPs. Optional, only needed if Ceph is not running on the
-PVE cluster.
+List of monitor daemon addresses. Optional, only needed if Ceph is not running
+on the {pve} cluster.
path::
username::
-Ceph user Id. Optional, only needed if Ceph is not running on the PVE cluster.
+Ceph user id. Optional, only needed if Ceph is not running on the {pve} cluster,
+where it defaults to `admin`.
subdir::
Access CephFS through FUSE, instead of the kernel client. Optional, defaults
to `0`.
-.Configuration Example for a external Ceph cluster (`/etc/pve/storage.cfg`)
+.Configuration example for an external Ceph cluster (`/etc/pve/storage.cfg`)
----
cephfs: cephfs-external
monhost 10.1.1.20 10.1.1.21 10.1.1.22
path /mnt/pve/cephfs-external
content backup
username admin
+ fs-name cephfs
----
+NOTE: Don't forget to set up the client's secret key file, if cephx was not
+disabled.
Authentication
~~~~~~~~~~~~~~
-If you use `cephx` authentication, you need to copy the secret from your
-external Ceph cluster to a Proxmox VE host.
+NOTE: If Ceph is installed locally on the {pve} cluster, the following is done
+automatically when adding the storage.
+
+If you use `cephx` authentication, which is enabled by default, you need to
+provide the secret from the external Ceph cluster.
+
+To configure the storage via the CLI, you first need to make the file
+containing the secret available. One way is to copy the file from the external
+Ceph cluster directly to one of the {pve} nodes. The following example will
+copy it to the `/root` directory of the node on which we run it:
+
+----
+# scp <external cephserver>:/etc/ceph/cephfs.secret /root/cephfs.secret
+----
+
+Then use the `pvesm` CLI tool to configure the external RBD storage, use the
+`--keyring` parameter, which needs to be a path to the secret file that you
+copied. For example:
-Create the directory `/etc/pve/priv/ceph` with
+----
+# pvesm add cephfs <name> --monhost "10.1.1.20 10.1.1.21 10.1.1.22" --content backup --keyring /root/cephfs.secret
+----
- mkdir /etc/pve/priv/ceph
+When configuring an external RBD storage via the GUI, you can copy and paste
+the secret into the appropriate field.
-Then copy the secret
+The secret is only the key itself, as opposed to the `rbd` backend which also
+contains a `[client.userid]` section.
- scp <cephserver>:/etc/ceph/cephfs.secret /etc/pve/priv/ceph/<STORAGE_ID>.secret
+The secret will be stored at
-The secret must be named to match your `<STORAGE_ID>`. Copying the
-secret generally requires root privileges. The file must only contain the
-secret itself, opposed to the `rbd` backend.
+----
+# /etc/pve/priv/ceph/<STORAGE_ID>.secret
+----
+
+A secret can be received from the Ceph cluster (as Ceph admin) by issuing the
+command below, where `userid` is the client ID that has been configured to
+access the cluster. For further information on Ceph user management, see the
+Ceph docs.footnoteref:[cephusermgmt]
-If Ceph is installed locally on the PVE cluster, this is done automatically.
+----
+# ceph auth get-key client.userid > cephfs.secret
+----
Storage Features
~~~~~~~~~~~~~~~~
-The `cephfs` backend is a POSIX-compliant filesystem on top of a Ceph cluster.
+The `cephfs` backend is a POSIX-compliant filesystem, on top of a Ceph cluster.
.Storage features for backend `cephfs`
[width="100%",cols="m,m,3*d",options="header"]
|==============================================================================
-|Content types |Image formats |Shared |Snapshots |Clones
-|vztmpl iso backup |none |yes |yes |no
+|Content types |Image formats |Shared |Snapshots |Clones
+|vztmpl iso backup snippets |none |yes |yes^[1]^ |no
|==============================================================================
+^[1]^ While no known bugs exist, snapshots are not yet guaranteed to be stable,
+as they lack sufficient testing.
ifdef::wiki[]
projects / pve-docs.git / blobdiff