]>
Commit | Line | Data |
---|---|---|
11fdf7f2 | 1 | .. _cephfs-administration: |
7c673cae FG |
2 | |
3 | CephFS Administrative commands | |
4 | ============================== | |
5 | ||
9f95a23c TL |
6 | File Systems |
7 | ------------ | |
7c673cae | 8 | |
e306af50 TL |
9 | .. note:: The names of the file systems, metadata pools, and data pools can |
10 | only have characters in the set [a-zA-Z0-9\_-.]. | |
11 | ||
9f95a23c TL |
12 | These commands operate on the CephFS file systems in your Ceph cluster. |
13 | Note that by default only one file system is permitted: to enable | |
14 | creation of multiple file systems use ``ceph fs flag set enable_multiple true``. | |
7c673cae FG |
15 | |
16 | :: | |
17 | ||
9f95a23c | 18 | fs new <file system name> <metadata pool name> <data pool name> |
7c673cae | 19 | |
11fdf7f2 TL |
20 | This command creates a new file system. The file system name and metadata pool |
21 | name are self-explanatory. The specified data pool is the default data pool and | |
22 | cannot be changed once set. Each file system has its own set of MDS daemons | |
23 | assigned to ranks so ensure that you have sufficient standby daemons available | |
24 | to accommodate the new file system. | |
25 | ||
7c673cae FG |
26 | :: |
27 | ||
28 | fs ls | |
29 | ||
11fdf7f2 TL |
30 | List all file systems by name. |
31 | ||
20effc67 TL |
32 | :: |
33 | ||
34 | fs lsflags <file system name> | |
35 | ||
36 | List all the flags set on a file system. | |
37 | ||
7c673cae FG |
38 | :: |
39 | ||
11fdf7f2 TL |
40 | fs dump [epoch] |
41 | ||
42 | This dumps the FSMap at the given epoch (default: current) which includes all | |
43 | file system settings, MDS daemons and the ranks they hold, and the list of | |
44 | standby MDS daemons. | |
45 | ||
7c673cae FG |
46 | |
47 | :: | |
48 | ||
9f95a23c | 49 | fs rm <file system name> [--yes-i-really-mean-it] |
11fdf7f2 TL |
50 | |
51 | Destroy a CephFS file system. This wipes information about the state of the | |
52 | file system from the FSMap. The metadata pool and data pools are untouched and | |
53 | must be destroyed separately. | |
7c673cae FG |
54 | |
55 | :: | |
56 | ||
9f95a23c | 57 | fs get <file system name> |
7c673cae | 58 | |
11fdf7f2 TL |
59 | Get information about the named file system, including settings and ranks. This |
60 | is a subset of the same information from the ``fs dump`` command. | |
61 | ||
7c673cae FG |
62 | :: |
63 | ||
9f95a23c | 64 | fs set <file system name> <var> <val> |
7c673cae | 65 | |
11fdf7f2 TL |
66 | Change a setting on a file system. These settings are specific to the named |
67 | file system and do not affect other file systems. | |
68 | ||
7c673cae FG |
69 | :: |
70 | ||
9f95a23c | 71 | fs add_data_pool <file system name> <pool name/id> |
7c673cae | 72 | |
11fdf7f2 TL |
73 | Add a data pool to the file system. This pool can be used for file layouts |
74 | as an alternate location to store file data. | |
75 | ||
7c673cae FG |
76 | :: |
77 | ||
9f95a23c | 78 | fs rm_data_pool <file system name> <pool name/id> |
7c673cae | 79 | |
11fdf7f2 TL |
80 | This command removes the specified pool from the list of data pools for the |
81 | file system. If any files have layouts for the removed data pool, the file | |
82 | data will become unavailable. The default data pool (when creating the file | |
83 | system) cannot be removed. | |
84 | ||
20effc67 TL |
85 | :: |
86 | ||
87 | fs rename <file system name> <new file system name> [--yes-i-really-mean-it] | |
88 | ||
89 | Rename a Ceph file system. This also changes the application tags on the data | |
90 | pools and metadata pool of the file system to the new file system name. | |
91 | The CephX IDs authorized to the old file system name need to be reauthorized | |
92 | to the new name. Any on-going operations of the clients using these IDs may be | |
93 | disrupted. Mirroring is expected to be disabled on the file system. | |
94 | ||
7c673cae | 95 | |
31f18b77 FG |
96 | Settings |
97 | -------- | |
98 | ||
99 | :: | |
100 | ||
101 | fs set <fs name> max_file_size <size in bytes> | |
102 | ||
103 | CephFS has a configurable maximum file size, and it's 1TB by default. | |
104 | You may wish to set this limit higher if you expect to store large files | |
105 | in CephFS. It is a 64-bit field. | |
106 | ||
107 | Setting ``max_file_size`` to 0 does not disable the limit. It would | |
108 | simply limit clients to only creating empty files. | |
109 | ||
110 | ||
111 | Maximum file sizes and performance | |
112 | ---------------------------------- | |
113 | ||
114 | CephFS enforces the maximum file size limit at the point of appending to | |
115 | files or setting their size. It does not affect how anything is stored. | |
116 | ||
117 | When users create a file of an enormous size (without necessarily | |
118 | writing any data to it), some operations (such as deletes) cause the MDS | |
119 | to have to do a large number of operations to check if any of the RADOS | |
120 | objects within the range that could exist (according to the file size) | |
121 | really existed. | |
122 | ||
123 | The ``max_file_size`` setting prevents users from creating files that | |
124 | appear to be eg. exabytes in size, causing load on the MDS as it tries | |
125 | to enumerate the objects during operations like stats or deletes. | |
126 | ||
127 | ||
28e407b8 AA |
128 | Taking the cluster down |
129 | ----------------------- | |
130 | ||
11fdf7f2 TL |
131 | Taking a CephFS cluster down is done by setting the down flag: |
132 | ||
133 | :: | |
134 | ||
135 | fs set <fs_name> down true | |
136 | ||
137 | To bring the cluster back online: | |
138 | ||
139 | :: | |
140 | ||
141 | fs set <fs_name> down false | |
142 | ||
143 | This will also restore the previous value of max_mds. MDS daemons are brought | |
144 | down in a way such that journals are flushed to the metadata pool and all | |
145 | client I/O is stopped. | |
146 | ||
147 | ||
148 | Taking the cluster down rapidly for deletion or disaster recovery | |
149 | ----------------------------------------------------------------- | |
150 | ||
151 | To allow rapidly deleting a file system (for testing) or to quickly bring the | |
152 | file system and MDS daemons down, use the ``fs fail`` command: | |
153 | ||
154 | :: | |
155 | ||
156 | fs fail <fs_name> | |
157 | ||
158 | This command sets a file system flag to prevent standbys from | |
159 | activating on the file system (the ``joinable`` flag). | |
160 | ||
161 | This process can also be done manually by doing the following: | |
162 | ||
163 | :: | |
164 | ||
165 | fs set <fs_name> joinable false | |
166 | ||
167 | Then the operator can fail all of the ranks which causes the MDS daemons to | |
168 | respawn as standbys. The file system will be left in a degraded state. | |
169 | ||
170 | :: | |
171 | ||
172 | # For all ranks, 0-N: | |
173 | mds fail <fs_name>:<n> | |
174 | ||
175 | Once all ranks are inactive, the file system may also be deleted or left in | |
176 | this state for other purposes (perhaps disaster recovery). | |
177 | ||
178 | To bring the cluster back up, simply set the joinable flag: | |
28e407b8 AA |
179 | |
180 | :: | |
28e407b8 | 181 | |
11fdf7f2 TL |
182 | fs set <fs_name> joinable true |
183 | ||
28e407b8 | 184 | |
7c673cae FG |
185 | Daemons |
186 | ------- | |
187 | ||
11fdf7f2 TL |
188 | Most commands manipulating MDSs take a ``<role>`` argument which can take one |
189 | of three forms: | |
190 | ||
191 | :: | |
192 | ||
193 | <fs_name>:<rank> | |
194 | <fs_id>:<rank> | |
195 | <rank> | |
196 | ||
197 | Commands to manipulate MDS daemons: | |
7c673cae FG |
198 | |
199 | :: | |
200 | ||
224ce89b | 201 | mds fail <gid/name/role> |
7c673cae | 202 | |
224ce89b WB |
203 | Mark an MDS daemon as failed. This is equivalent to what the cluster |
204 | would do if an MDS daemon had failed to send a message to the mon | |
205 | for ``mds_beacon_grace`` second. If the daemon was active and a suitable | |
206 | standby is available, using ``mds fail`` will force a failover to the standby. | |
207 | ||
208 | If the MDS daemon was in reality still running, then using ``mds fail`` | |
209 | will cause the daemon to restart. If it was active and a standby was | |
210 | available, then the "failed" daemon will return as a standby. | |
7c673cae | 211 | |
c07f9fc5 FG |
212 | |
213 | :: | |
214 | ||
11fdf7f2 | 215 | tell mds.<daemon name> command ... |
224ce89b | 216 | |
11fdf7f2 TL |
217 | Send a command to the MDS daemon(s). Use ``mds.*`` to send a command to all |
218 | daemons. Use ``ceph tell mds.* help`` to learn available commands. | |
7c673cae FG |
219 | |
220 | :: | |
221 | ||
222 | mds metadata <gid/name/role> | |
223 | ||
11fdf7f2 TL |
224 | Get metadata about the given MDS known to the Monitors. |
225 | ||
7c673cae FG |
226 | :: |
227 | ||
228 | mds repaired <role> | |
229 | ||
11fdf7f2 TL |
230 | Mark the file system rank as repaired. Unlike the name suggests, this command |
231 | does not change a MDS; it manipulates the file system rank which has been | |
232 | marked damaged. | |
233 | ||
7c673cae | 234 | |
b3b6e05e TL |
235 | Required Client Features |
236 | ------------------------ | |
81eedcae | 237 | |
b3b6e05e TL |
238 | It is sometimes desirable to set features that clients must support to talk to |
239 | CephFS. Clients without those features may disrupt other clients or behave in | |
240 | surprising ways. Or, you may want to require newer features to prevent older | |
241 | and possibly buggy clients from connecting. | |
81eedcae | 242 | |
b3b6e05e | 243 | Commands to manipulate required client features of a file system: |
f67539c2 TL |
244 | |
245 | :: | |
246 | ||
247 | fs required_client_features <fs name> add reply_encoding | |
248 | fs required_client_features <fs name> rm reply_encoding | |
249 | ||
250 | To list all CephFS features | |
251 | ||
252 | :: | |
253 | ||
254 | fs feature ls | |
255 | ||
b3b6e05e | 256 | Clients that are missing newly added features will be evicted automatically. |
f67539c2 | 257 | |
b3b6e05e | 258 | Here are the current CephFS features and first release they came out: |
f67539c2 TL |
259 | |
260 | +------------------+--------------+-----------------+ | |
261 | | Feature | Ceph release | Upstream Kernel | | |
262 | +==================+==============+=================+ | |
263 | | jewel | jewel | 4.5 | | |
264 | +------------------+--------------+-----------------+ | |
265 | | kraken | kraken | 4.13 | | |
266 | +------------------+--------------+-----------------+ | |
267 | | luminous | luminous | 4.13 | | |
268 | +------------------+--------------+-----------------+ | |
269 | | mimic | mimic | 4.19 | | |
270 | +------------------+--------------+-----------------+ | |
271 | | reply_encoding | nautilus | 5.1 | | |
272 | +------------------+--------------+-----------------+ | |
273 | | reclaim_client | nautilus | N/A | | |
274 | +------------------+--------------+-----------------+ | |
275 | | lazy_caps_wanted | nautilus | 5.1 | | |
276 | +------------------+--------------+-----------------+ | |
277 | | multi_reconnect | nautilus | 5.1 | | |
278 | +------------------+--------------+-----------------+ | |
279 | | deleg_ino | octopus | 5.6 | | |
280 | +------------------+--------------+-----------------+ | |
281 | | metric_collect | pacific | N/A | | |
282 | +------------------+--------------+-----------------+ | |
b3b6e05e TL |
283 | | alternate_name | pacific | PLANNED | |
284 | +------------------+--------------+-----------------+ | |
f67539c2 TL |
285 | |
286 | CephFS Feature Descriptions | |
287 | ||
288 | ||
289 | :: | |
290 | ||
291 | reply_encoding | |
292 | ||
293 | MDS encodes request reply in extensible format if client supports this feature. | |
294 | ||
295 | ||
296 | :: | |
297 | ||
298 | reclaim_client | |
299 | ||
300 | MDS allows new client to reclaim another (dead) client's states. This feature | |
301 | is used by NFS-Ganesha. | |
302 | ||
303 | ||
304 | :: | |
305 | ||
306 | lazy_caps_wanted | |
307 | ||
308 | When a stale client resumes, if the client supports this feature, mds only needs | |
309 | to re-issue caps that are explicitly wanted. | |
310 | ||
311 | ||
312 | :: | |
313 | ||
314 | multi_reconnect | |
315 | ||
316 | When mds failover, client sends reconnect messages to mds, to reestablish cache | |
317 | states. If MDS supports this feature, client can split large reconnect message | |
318 | into multiple ones. | |
319 | ||
320 | ||
321 | :: | |
322 | ||
323 | deleg_ino | |
324 | ||
325 | MDS delegate inode numbers to client if client supports this feature. Having | |
326 | delegated inode numbers is a prerequisite for client to do async file creation. | |
327 | ||
328 | ||
329 | :: | |
330 | ||
331 | metric_collect | |
332 | ||
333 | Clients can send performance metric to MDS if MDS support this feature. | |
334 | ||
b3b6e05e TL |
335 | :: |
336 | ||
337 | alternate_name | |
338 | ||
339 | Clients can set and understand "alternate names" for directory entries. This is | |
340 | to be used for encrypted file name support. | |
341 | ||
81eedcae | 342 | |
7c673cae FG |
343 | Global settings |
344 | --------------- | |
345 | ||
7c673cae FG |
346 | |
347 | :: | |
348 | ||
349 | fs flag set <flag name> <flag val> [<confirmation string>] | |
350 | ||
11fdf7f2 TL |
351 | Sets a global CephFS flag (i.e. not specific to a particular file system). |
352 | Currently, the only flag setting is 'enable_multiple' which allows having | |
353 | multiple CephFS file systems. | |
7c673cae FG |
354 | |
355 | Some flags require you to confirm your intentions with "--yes-i-really-mean-it" | |
356 | or a similar string they will prompt you with. Consider these actions carefully | |
357 | before proceeding; they are placed on especially dangerous activities. | |
358 | ||
20effc67 | 359 | .. _advanced-cephfs-admin-settings: |
7c673cae FG |
360 | |
361 | Advanced | |
362 | -------- | |
363 | ||
364 | These commands are not required in normal operation, and exist | |
365 | for use in exceptional circumstances. Incorrect use of these | |
366 | commands may cause serious problems, such as an inaccessible | |
9f95a23c | 367 | file system. |
7c673cae | 368 | |
7c673cae FG |
369 | :: |
370 | ||
371 | mds rmfailed | |
372 | ||
11fdf7f2 | 373 | This removes a rank from the failed set. |
31f18b77 | 374 | |
7c673cae FG |
375 | :: |
376 | ||
9f95a23c | 377 | fs reset <file system name> |
7c673cae | 378 | |
11fdf7f2 TL |
379 | This command resets the file system state to defaults, except for the name and |
380 | pools. Non-zero ranks are saved in the stopped set. | |
522d829b TL |
381 | |
382 | ||
383 | :: | |
384 | ||
385 | fs new <file system name> <metadata pool name> <data pool name> --fscid <fscid> --force | |
386 | ||
387 | This command creates a file system with a specific **fscid** (file system cluster ID). | |
388 | You may want to do this when an application expects the file system's ID to be | |
389 | stable after it has been recovered, e.g., after monitor databases are lost and | |
390 | rebuilt. Consequently, file system IDs don't always keep increasing with newer | |
391 | file systems. |