]>
Commit | Line | Data |
---|---|---|
669bce8b AA |
1 | [[storage_cephfs]] |
2 | Ceph Filesystem (CephFS) | |
3 | ------------------------ | |
4 | ifdef::wiki[] | |
5 | :pve-toplevel: | |
6 | :title: Storage: CephFS | |
7 | endif::wiki[] | |
8 | ||
9 | Storage pool type: `cephfs` | |
10 | ||
a55d30db | 11 | CephFS implements a POSIX-compliant filesystem, using a https://ceph.com[Ceph] |
ddf68e2c DW |
12 | storage cluster to store its data. As CephFS builds upon Ceph, it shares most of |
13 | its properties. This includes redundancy, scalability, self-healing, and high | |
6a8897ca TL |
14 | availability. |
15 | ||
ddf68e2c DW |
16 | TIP: {pve} can xref:chapter_pveceph[manage Ceph setups], which makes |
17 | configuring a CephFS storage easier. As modern hardware offers a lot of | |
18 | processing power and RAM, running storage services and VMs on same node is | |
19 | possible without a significant performance impact. | |
669bce8b | 20 | |
ddf68e2c DW |
21 | To use the CephFS storage plugin, you must replace the stock Debian Ceph client, |
22 | by adding our xref:sysadmin_package_repositories_ceph[Ceph repository]. | |
23 | Once added, run `apt update`, followed by `apt dist-upgrade`, in order to get | |
24 | the newest packages. | |
2ecb5f80 | 25 | |
ddf68e2c DW |
26 | WARNING: Please ensure that there are no other Ceph repositories configured. |
27 | Otherwise the installation will fail or there will be mixed package versions on | |
28 | the node, leading to unexpected behavior. | |
2ecb5f80 | 29 | |
669bce8b AA |
30 | [[storage_cephfs_config]] |
31 | Configuration | |
32 | ~~~~~~~~~~~~~ | |
33 | ||
34 | This backend supports the common storage properties `nodes`, | |
ddf68e2c | 35 | `disable`, `content`, as well as the following `cephfs` specific properties: |
669bce8b | 36 | |
82afba9a AL |
37 | fs-name:: |
38 | ||
39 | Name of the Ceph FS. | |
40 | ||
669bce8b AA |
41 | monhost:: |
42 | ||
6a8897ca | 43 | List of monitor daemon addresses. Optional, only needed if Ceph is not running |
8ebd2453 | 44 | on the {pve} cluster. |
669bce8b AA |
45 | |
46 | path:: | |
47 | ||
48 | The local mount point. Optional, defaults to `/mnt/pve/<STORAGE_ID>/`. | |
49 | ||
50 | username:: | |
51 | ||
8ebd2453 | 52 | Ceph user id. Optional, only needed if Ceph is not running on the {pve} cluster, |
6a8897ca | 53 | where it defaults to `admin`. |
669bce8b AA |
54 | |
55 | subdir:: | |
56 | ||
57 | CephFS subdirectory to mount. Optional, defaults to `/`. | |
58 | ||
59 | fuse:: | |
60 | ||
61 | Access CephFS through FUSE, instead of the kernel client. Optional, defaults | |
62 | to `0`. | |
63 | ||
ddf68e2c | 64 | .Configuration example for an external Ceph cluster (`/etc/pve/storage.cfg`) |
669bce8b AA |
65 | ---- |
66 | cephfs: cephfs-external | |
67 | monhost 10.1.1.20 10.1.1.21 10.1.1.22 | |
68 | path /mnt/pve/cephfs-external | |
69 | content backup | |
70 | username admin | |
82afba9a | 71 | fs-name cephfs |
669bce8b | 72 | ---- |
ddf68e2c DW |
73 | NOTE: Don't forget to set up the client's secret key file, if cephx was not |
74 | disabled. | |
669bce8b AA |
75 | |
76 | Authentication | |
77 | ~~~~~~~~~~~~~~ | |
78 | ||
33ccd432 FE |
79 | NOTE: If Ceph is installed locally on the {pve} cluster, the following is done |
80 | automatically when adding the storage. | |
81 | ||
876f6dce AL |
82 | If you use `cephx` authentication, which is enabled by default, you need to |
83 | provide the secret from the external Ceph cluster. | |
669bce8b | 84 | |
876f6dce AL |
85 | To configure the storage via the CLI, you first need to make the file |
86 | containing the secret available. One way is to copy the file from the external | |
87 | Ceph cluster directly to one of the {pve} nodes. The following example will | |
88 | copy it to the `/root` directory of the node on which we run it: | |
669bce8b | 89 | |
876f6dce AL |
90 | ---- |
91 | # scp <external cephserver>:/etc/ceph/cephfs.secret /root/cephfs.secret | |
92 | ---- | |
93 | ||
94 | Then use the `pvesm` CLI tool to configure the external RBD storage, use the | |
95 | `--keyring` parameter, which needs to be a path to the secret file that you | |
96 | copied. For example: | |
97 | ||
98 | ---- | |
99 | # pvesm add cephfs <name> --monhost "10.1.1.20 10.1.1.21 10.1.1.22" --content backup --keyring /root/cephfs.secret | |
100 | ---- | |
669bce8b | 101 | |
876f6dce AL |
102 | When configuring an external RBD storage via the GUI, you can copy and paste |
103 | the secret into the appropriate field. | |
669bce8b | 104 | |
876f6dce AL |
105 | The secret is only the key itself, as opposed to the `rbd` backend which also |
106 | contains a `[client.userid]` section. | |
669bce8b | 107 | |
876f6dce AL |
108 | The secret will be stored at |
109 | ||
110 | ---- | |
111 | # /etc/pve/priv/ceph/<STORAGE_ID>.secret | |
112 | ---- | |
669bce8b | 113 | |
ddf68e2c DW |
114 | A secret can be received from the Ceph cluster (as Ceph admin) by issuing the |
115 | command below, where `userid` is the client ID that has been configured to | |
116 | access the cluster. For further information on Ceph user management, see the | |
876f6dce | 117 | Ceph docs.footnoteref:[cephusermgmt] |
fdbb2634 | 118 | |
876f6dce AL |
119 | ---- |
120 | # ceph auth get-key client.userid > cephfs.secret | |
121 | ---- | |
fdbb2634 | 122 | |
669bce8b AA |
123 | Storage Features |
124 | ~~~~~~~~~~~~~~~~ | |
125 | ||
ddf68e2c | 126 | The `cephfs` backend is a POSIX-compliant filesystem, on top of a Ceph cluster. |
669bce8b AA |
127 | |
128 | .Storage features for backend `cephfs` | |
129 | [width="100%",cols="m,m,3*d",options="header"] | |
130 | |============================================================================== | |
c2c8eb89 DC |
131 | |Content types |Image formats |Shared |Snapshots |Clones |
132 | |vztmpl iso backup snippets |none |yes |yes^[1]^ |no | |
669bce8b | 133 | |============================================================================== |
ddf68e2c DW |
134 | ^[1]^ While no known bugs exist, snapshots are not yet guaranteed to be stable, |
135 | as they lack sufficient testing. | |
669bce8b AA |
136 | |
137 | ifdef::wiki[] | |
138 | ||
139 | See Also | |
140 | ~~~~~~~~ | |
141 | ||
142 | * link:/wiki/Storage[Storage] | |
143 | ||
144 | endif::wiki[] | |
145 |