pve-storage-cephfs.adoc

   1 [[storage_cephfs]]
   2 Ceph Filesystem (CephFS)
   3 ------------------------
   4 ifdef::wiki[]
   5 :pve-toplevel:
   6 :title: Storage: CephFS
   7 endif::wiki[]
   8
   9 Storage pool type: `cephfs`
  10
  11 CephFS implements a POSIX-compliant filesystem using a http://ceph.com[Ceph]
  12 storage cluster to store its data. As CephFS builds on Ceph it shares most of
  13 its properties, this includes redundancy, scalability, self healing and high
  14 availability.
  15
  16 TIP: {pve} can xref:chapter_pveceph[manage ceph setups], which makes
  17 configuring a CephFS storage easier. As recent hardware has plenty of CPU power
  18 and RAM, running storage services and VMs on same node is possible without a
  19 big performance impact.
  20
  21 To use the CephFS storage plugin you need update the debian stock Ceph client.
  22 Add our Ceph repository xref:sysadmin_package_repositories_ceph[Ceph repository].
  23 Once added, run an `apt update` and `apt dist-upgrade` cycle to get the newest
  24 packages.
  25
  26 You need to make sure that there is no other Ceph repository configured,
  27 otherwise the installation will fail or there will be mixed package
  28 versions on the node, leading to unexpected behavior.
  29
  30 [[storage_cephfs_config]]
  31 Configuration
  32 ~~~~~~~~~~~~~
  33
  34 This backend supports the common storage properties `nodes`,
  35 `disable`, `content`, and the following `cephfs` specific properties:
  36
  37 monhost::
  38
  39 List of monitor daemon addresses. Optional, only needed if Ceph is not running
  40 on the PVE cluster.
  41
  42 path::
  43
  44 The local mount point. Optional, defaults to `/mnt/pve/<STORAGE_ID>/`.
  45
  46 username::
  47
  48 Ceph user id. Optional, only needed if Ceph is not running on the PVE cluster
  49 where it defaults to `admin`.
  50
  51 subdir::
  52
  53 CephFS subdirectory to mount. Optional, defaults to `/`.
  54
  55 fuse::
  56
  57 Access CephFS through FUSE, instead of the kernel client. Optional, defaults
  58 to `0`.
  59
  60 .Configuration Example for a external Ceph cluster (`/etc/pve/storage.cfg`)
  61 ----
  62 cephfs: cephfs-external
  63         monhost 10.1.1.20 10.1.1.21 10.1.1.22
  64         path /mnt/pve/cephfs-external
  65         content backup
  66         username admin
  67 ----
  68 NOTE: Don't forget to setup the client secret key file if cephx was not turned
  69 off.
  70
  71 Authentication
  72 ~~~~~~~~~~~~~~
  73
  74 If you use the, by-default enabled, `cephx` authentication, you need to copy
  75 the secret from your external Ceph cluster to a Proxmox VE host.
  76
  77 Create the directory `/etc/pve/priv/ceph` with
  78
  79  mkdir /etc/pve/priv/ceph
  80
  81 Then copy the secret
  82
  83  scp cephfs.secret <proxmox>:/etc/pve/priv/ceph/<STORAGE_ID>.secret
  84
  85 The secret must be named to match your `<STORAGE_ID>`. Copying the
  86 secret generally requires root privileges. The file must only contain the
  87 secret key itself, opposed to the `rbd` backend which also contains a
  88 `[client.userid]` section.
  89
  90 A secret can be received from the ceph cluster (as ceph admin) by issuing the
  91 following command. Replace the `userid` with the actual client ID configured to
  92 access the cluster. For further ceph user management see the Ceph docs
  93 footnote:[Ceph user management http://docs.ceph.com/docs/luminous/rados/operations/user-management/].
  94
  95  ceph auth get-key client.userid > cephfs.secret
  96
  97 If Ceph is installed locally on the PVE cluster, i.e., setup with `pveceph`,
  98 this is done automatically.
  99
 100 Storage Features
 101 ~~~~~~~~~~~~~~~~
 102
 103 The `cephfs` backend is a POSIX-compliant filesystem on top of a Ceph cluster.
 104
 105 .Storage features for backend `cephfs`
 106 [width="100%",cols="m,m,3*d",options="header"]
 107 |==============================================================================
 108 |Content types              |Image formats  |Shared |Snapshots |Clones
 109 |vztmpl iso backup snippets |none           |yes    |yes^[1]^  |no
 110 |==============================================================================
 111 ^[1]^ Snapshots, while no known bugs, cannot be guaranteed to be stable yet, as
 112 they lack testing.
 113
 114 ifdef::wiki[]
 115
 116 See Also
 117 ~~~~~~~~
 118
 119 * link:/wiki/Storage[Storage]
 120
 121 endif::wiki[]
 122