]>
Commit | Line | Data |
---|---|---|
11fdf7f2 | 1 | .. _cephfs-administration: |
7c673cae FG |
2 | |
3 | CephFS Administrative commands | |
4 | ============================== | |
5 | ||
9f95a23c TL |
6 | File Systems |
7 | ------------ | |
7c673cae | 8 | |
e306af50 TL |
9 | .. note:: The names of the file systems, metadata pools, and data pools can |
10 | only have characters in the set [a-zA-Z0-9\_-.]. | |
11 | ||
9f95a23c TL |
12 | These commands operate on the CephFS file systems in your Ceph cluster. |
13 | Note that by default only one file system is permitted: to enable | |
14 | creation of multiple file systems use ``ceph fs flag set enable_multiple true``. | |
7c673cae FG |
15 | |
16 | :: | |
17 | ||
9f95a23c | 18 | fs new <file system name> <metadata pool name> <data pool name> |
7c673cae | 19 | |
11fdf7f2 TL |
20 | This command creates a new file system. The file system name and metadata pool |
21 | name are self-explanatory. The specified data pool is the default data pool and | |
22 | cannot be changed once set. Each file system has its own set of MDS daemons | |
23 | assigned to ranks so ensure that you have sufficient standby daemons available | |
24 | to accommodate the new file system. | |
25 | ||
7c673cae FG |
26 | :: |
27 | ||
28 | fs ls | |
29 | ||
11fdf7f2 TL |
30 | List all file systems by name. |
31 | ||
7c673cae FG |
32 | :: |
33 | ||
11fdf7f2 TL |
34 | fs dump [epoch] |
35 | ||
36 | This dumps the FSMap at the given epoch (default: current) which includes all | |
37 | file system settings, MDS daemons and the ranks they hold, and the list of | |
38 | standby MDS daemons. | |
39 | ||
7c673cae FG |
40 | |
41 | :: | |
42 | ||
9f95a23c | 43 | fs rm <file system name> [--yes-i-really-mean-it] |
11fdf7f2 TL |
44 | |
45 | Destroy a CephFS file system. This wipes information about the state of the | |
46 | file system from the FSMap. The metadata pool and data pools are untouched and | |
47 | must be destroyed separately. | |
7c673cae FG |
48 | |
49 | :: | |
50 | ||
9f95a23c | 51 | fs get <file system name> |
7c673cae | 52 | |
11fdf7f2 TL |
53 | Get information about the named file system, including settings and ranks. This |
54 | is a subset of the same information from the ``fs dump`` command. | |
55 | ||
7c673cae FG |
56 | :: |
57 | ||
9f95a23c | 58 | fs set <file system name> <var> <val> |
7c673cae | 59 | |
11fdf7f2 TL |
60 | Change a setting on a file system. These settings are specific to the named |
61 | file system and do not affect other file systems. | |
62 | ||
7c673cae FG |
63 | :: |
64 | ||
9f95a23c | 65 | fs add_data_pool <file system name> <pool name/id> |
7c673cae | 66 | |
11fdf7f2 TL |
67 | Add a data pool to the file system. This pool can be used for file layouts |
68 | as an alternate location to store file data. | |
69 | ||
7c673cae FG |
70 | :: |
71 | ||
9f95a23c | 72 | fs rm_data_pool <file system name> <pool name/id> |
7c673cae | 73 | |
11fdf7f2 TL |
74 | This command removes the specified pool from the list of data pools for the |
75 | file system. If any files have layouts for the removed data pool, the file | |
76 | data will become unavailable. The default data pool (when creating the file | |
77 | system) cannot be removed. | |
78 | ||
7c673cae | 79 | |
31f18b77 FG |
80 | Settings |
81 | -------- | |
82 | ||
83 | :: | |
84 | ||
85 | fs set <fs name> max_file_size <size in bytes> | |
86 | ||
87 | CephFS has a configurable maximum file size, and it's 1TB by default. | |
88 | You may wish to set this limit higher if you expect to store large files | |
89 | in CephFS. It is a 64-bit field. | |
90 | ||
91 | Setting ``max_file_size`` to 0 does not disable the limit. It would | |
92 | simply limit clients to only creating empty files. | |
93 | ||
94 | ||
95 | Maximum file sizes and performance | |
96 | ---------------------------------- | |
97 | ||
98 | CephFS enforces the maximum file size limit at the point of appending to | |
99 | files or setting their size. It does not affect how anything is stored. | |
100 | ||
101 | When users create a file of an enormous size (without necessarily | |
102 | writing any data to it), some operations (such as deletes) cause the MDS | |
103 | to have to do a large number of operations to check if any of the RADOS | |
104 | objects within the range that could exist (according to the file size) | |
105 | really existed. | |
106 | ||
107 | The ``max_file_size`` setting prevents users from creating files that | |
108 | appear to be eg. exabytes in size, causing load on the MDS as it tries | |
109 | to enumerate the objects during operations like stats or deletes. | |
110 | ||
111 | ||
28e407b8 AA |
112 | Taking the cluster down |
113 | ----------------------- | |
114 | ||
11fdf7f2 TL |
115 | Taking a CephFS cluster down is done by setting the down flag: |
116 | ||
117 | :: | |
118 | ||
119 | fs set <fs_name> down true | |
120 | ||
121 | To bring the cluster back online: | |
122 | ||
123 | :: | |
124 | ||
125 | fs set <fs_name> down false | |
126 | ||
127 | This will also restore the previous value of max_mds. MDS daemons are brought | |
128 | down in a way such that journals are flushed to the metadata pool and all | |
129 | client I/O is stopped. | |
130 | ||
131 | ||
132 | Taking the cluster down rapidly for deletion or disaster recovery | |
133 | ----------------------------------------------------------------- | |
134 | ||
135 | To allow rapidly deleting a file system (for testing) or to quickly bring the | |
136 | file system and MDS daemons down, use the ``fs fail`` command: | |
137 | ||
138 | :: | |
139 | ||
140 | fs fail <fs_name> | |
141 | ||
142 | This command sets a file system flag to prevent standbys from | |
143 | activating on the file system (the ``joinable`` flag). | |
144 | ||
145 | This process can also be done manually by doing the following: | |
146 | ||
147 | :: | |
148 | ||
149 | fs set <fs_name> joinable false | |
150 | ||
151 | Then the operator can fail all of the ranks which causes the MDS daemons to | |
152 | respawn as standbys. The file system will be left in a degraded state. | |
153 | ||
154 | :: | |
155 | ||
156 | # For all ranks, 0-N: | |
157 | mds fail <fs_name>:<n> | |
158 | ||
159 | Once all ranks are inactive, the file system may also be deleted or left in | |
160 | this state for other purposes (perhaps disaster recovery). | |
161 | ||
162 | To bring the cluster back up, simply set the joinable flag: | |
28e407b8 AA |
163 | |
164 | :: | |
28e407b8 | 165 | |
11fdf7f2 TL |
166 | fs set <fs_name> joinable true |
167 | ||
28e407b8 | 168 | |
7c673cae FG |
169 | Daemons |
170 | ------- | |
171 | ||
11fdf7f2 TL |
172 | Most commands manipulating MDSs take a ``<role>`` argument which can take one |
173 | of three forms: | |
174 | ||
175 | :: | |
176 | ||
177 | <fs_name>:<rank> | |
178 | <fs_id>:<rank> | |
179 | <rank> | |
180 | ||
181 | Commands to manipulate MDS daemons: | |
7c673cae FG |
182 | |
183 | :: | |
184 | ||
224ce89b | 185 | mds fail <gid/name/role> |
7c673cae | 186 | |
224ce89b WB |
187 | Mark an MDS daemon as failed. This is equivalent to what the cluster |
188 | would do if an MDS daemon had failed to send a message to the mon | |
189 | for ``mds_beacon_grace`` second. If the daemon was active and a suitable | |
190 | standby is available, using ``mds fail`` will force a failover to the standby. | |
191 | ||
192 | If the MDS daemon was in reality still running, then using ``mds fail`` | |
193 | will cause the daemon to restart. If it was active and a standby was | |
194 | available, then the "failed" daemon will return as a standby. | |
7c673cae | 195 | |
c07f9fc5 FG |
196 | |
197 | :: | |
198 | ||
11fdf7f2 | 199 | tell mds.<daemon name> command ... |
224ce89b | 200 | |
11fdf7f2 TL |
201 | Send a command to the MDS daemon(s). Use ``mds.*`` to send a command to all |
202 | daemons. Use ``ceph tell mds.* help`` to learn available commands. | |
7c673cae FG |
203 | |
204 | :: | |
205 | ||
206 | mds metadata <gid/name/role> | |
207 | ||
11fdf7f2 TL |
208 | Get metadata about the given MDS known to the Monitors. |
209 | ||
7c673cae FG |
210 | :: |
211 | ||
212 | mds repaired <role> | |
213 | ||
11fdf7f2 TL |
214 | Mark the file system rank as repaired. Unlike the name suggests, this command |
215 | does not change a MDS; it manipulates the file system rank which has been | |
216 | marked damaged. | |
217 | ||
7c673cae | 218 | |
b3b6e05e TL |
219 | Required Client Features |
220 | ------------------------ | |
81eedcae | 221 | |
b3b6e05e TL |
222 | It is sometimes desirable to set features that clients must support to talk to |
223 | CephFS. Clients without those features may disrupt other clients or behave in | |
224 | surprising ways. Or, you may want to require newer features to prevent older | |
225 | and possibly buggy clients from connecting. | |
81eedcae | 226 | |
b3b6e05e | 227 | Commands to manipulate required client features of a file system: |
f67539c2 TL |
228 | |
229 | :: | |
230 | ||
231 | fs required_client_features <fs name> add reply_encoding | |
232 | fs required_client_features <fs name> rm reply_encoding | |
233 | ||
234 | To list all CephFS features | |
235 | ||
236 | :: | |
237 | ||
238 | fs feature ls | |
239 | ||
b3b6e05e | 240 | Clients that are missing newly added features will be evicted automatically. |
f67539c2 | 241 | |
b3b6e05e | 242 | Here are the current CephFS features and first release they came out: |
f67539c2 TL |
243 | |
244 | +------------------+--------------+-----------------+ | |
245 | | Feature | Ceph release | Upstream Kernel | | |
246 | +==================+==============+=================+ | |
247 | | jewel | jewel | 4.5 | | |
248 | +------------------+--------------+-----------------+ | |
249 | | kraken | kraken | 4.13 | | |
250 | +------------------+--------------+-----------------+ | |
251 | | luminous | luminous | 4.13 | | |
252 | +------------------+--------------+-----------------+ | |
253 | | mimic | mimic | 4.19 | | |
254 | +------------------+--------------+-----------------+ | |
255 | | reply_encoding | nautilus | 5.1 | | |
256 | +------------------+--------------+-----------------+ | |
257 | | reclaim_client | nautilus | N/A | | |
258 | +------------------+--------------+-----------------+ | |
259 | | lazy_caps_wanted | nautilus | 5.1 | | |
260 | +------------------+--------------+-----------------+ | |
261 | | multi_reconnect | nautilus | 5.1 | | |
262 | +------------------+--------------+-----------------+ | |
263 | | deleg_ino | octopus | 5.6 | | |
264 | +------------------+--------------+-----------------+ | |
265 | | metric_collect | pacific | N/A | | |
266 | +------------------+--------------+-----------------+ | |
b3b6e05e TL |
267 | | alternate_name | pacific | PLANNED | |
268 | +------------------+--------------+-----------------+ | |
f67539c2 TL |
269 | |
270 | CephFS Feature Descriptions | |
271 | ||
272 | ||
273 | :: | |
274 | ||
275 | reply_encoding | |
276 | ||
277 | MDS encodes request reply in extensible format if client supports this feature. | |
278 | ||
279 | ||
280 | :: | |
281 | ||
282 | reclaim_client | |
283 | ||
284 | MDS allows new client to reclaim another (dead) client's states. This feature | |
285 | is used by NFS-Ganesha. | |
286 | ||
287 | ||
288 | :: | |
289 | ||
290 | lazy_caps_wanted | |
291 | ||
292 | When a stale client resumes, if the client supports this feature, mds only needs | |
293 | to re-issue caps that are explicitly wanted. | |
294 | ||
295 | ||
296 | :: | |
297 | ||
298 | multi_reconnect | |
299 | ||
300 | When mds failover, client sends reconnect messages to mds, to reestablish cache | |
301 | states. If MDS supports this feature, client can split large reconnect message | |
302 | into multiple ones. | |
303 | ||
304 | ||
305 | :: | |
306 | ||
307 | deleg_ino | |
308 | ||
309 | MDS delegate inode numbers to client if client supports this feature. Having | |
310 | delegated inode numbers is a prerequisite for client to do async file creation. | |
311 | ||
312 | ||
313 | :: | |
314 | ||
315 | metric_collect | |
316 | ||
317 | Clients can send performance metric to MDS if MDS support this feature. | |
318 | ||
b3b6e05e TL |
319 | :: |
320 | ||
321 | alternate_name | |
322 | ||
323 | Clients can set and understand "alternate names" for directory entries. This is | |
324 | to be used for encrypted file name support. | |
325 | ||
81eedcae | 326 | |
7c673cae FG |
327 | Global settings |
328 | --------------- | |
329 | ||
7c673cae FG |
330 | |
331 | :: | |
332 | ||
333 | fs flag set <flag name> <flag val> [<confirmation string>] | |
334 | ||
11fdf7f2 TL |
335 | Sets a global CephFS flag (i.e. not specific to a particular file system). |
336 | Currently, the only flag setting is 'enable_multiple' which allows having | |
337 | multiple CephFS file systems. | |
7c673cae FG |
338 | |
339 | Some flags require you to confirm your intentions with "--yes-i-really-mean-it" | |
340 | or a similar string they will prompt you with. Consider these actions carefully | |
341 | before proceeding; they are placed on especially dangerous activities. | |
342 | ||
343 | ||
344 | Advanced | |
345 | -------- | |
346 | ||
347 | These commands are not required in normal operation, and exist | |
348 | for use in exceptional circumstances. Incorrect use of these | |
349 | commands may cause serious problems, such as an inaccessible | |
9f95a23c | 350 | file system. |
7c673cae FG |
351 | |
352 | :: | |
353 | ||
354 | mds compat rm_compat | |
355 | ||
11fdf7f2 TL |
356 | Removes an compatibility feature flag. |
357 | ||
7c673cae FG |
358 | :: |
359 | ||
360 | mds compat rm_incompat | |
361 | ||
11fdf7f2 | 362 | Removes an incompatibility feature flag. |
7c673cae FG |
363 | |
364 | :: | |
365 | ||
11fdf7f2 | 366 | mds compat show |
7c673cae | 367 | |
11fdf7f2 | 368 | Show MDS compatibility flags. |
7c673cae FG |
369 | |
370 | :: | |
371 | ||
372 | mds rmfailed | |
373 | ||
11fdf7f2 | 374 | This removes a rank from the failed set. |
31f18b77 | 375 | |
7c673cae FG |
376 | :: |
377 | ||
9f95a23c | 378 | fs reset <file system name> |
7c673cae | 379 | |
11fdf7f2 TL |
380 | This command resets the file system state to defaults, except for the name and |
381 | pools. Non-zero ranks are saved in the stopped set. |