1 .. _cephfs-administration:
3 CephFS Administrative commands
4 ==============================
9 .. note:: The names of the file systems, metadata pools, and data pools can
10 only have characters in the set [a-zA-Z0-9\_-.].
12 These commands operate on the CephFS file systems in your Ceph cluster.
13 Note that by default only one file system is permitted: to enable
14 creation of multiple file systems use ``ceph fs flag set enable_multiple true``.
18 fs new <file system name> <metadata pool name> <data pool name>
20 This command creates a new file system. The file system name and metadata pool
21 name are self-explanatory. The specified data pool is the default data pool and
22 cannot be changed once set. Each file system has its own set of MDS daemons
23 assigned to ranks so ensure that you have sufficient standby daemons available
24 to accommodate the new file system.
30 List all file systems by name.
34 fs lsflags <file system name>
36 List all the flags set on a file system.
42 This dumps the FSMap at the given epoch (default: current) which includes all
43 file system settings, MDS daemons and the ranks they hold, and the list of
49 fs rm <file system name> [--yes-i-really-mean-it]
51 Destroy a CephFS file system. This wipes information about the state of the
52 file system from the FSMap. The metadata pool and data pools are untouched and
53 must be destroyed separately.
57 fs get <file system name>
59 Get information about the named file system, including settings and ranks. This
60 is a subset of the same information from the ``fs dump`` command.
64 fs set <file system name> <var> <val>
66 Change a setting on a file system. These settings are specific to the named
67 file system and do not affect other file systems.
71 fs add_data_pool <file system name> <pool name/id>
73 Add a data pool to the file system. This pool can be used for file layouts
74 as an alternate location to store file data.
78 fs rm_data_pool <file system name> <pool name/id>
80 This command removes the specified pool from the list of data pools for the
81 file system. If any files have layouts for the removed data pool, the file
82 data will become unavailable. The default data pool (when creating the file
83 system) cannot be removed.
87 fs rename <file system name> <new file system name> [--yes-i-really-mean-it]
89 Rename a Ceph file system. This also changes the application tags on the data
90 pools and metadata pool of the file system to the new file system name.
91 The CephX IDs authorized to the old file system name need to be reauthorized
92 to the new name. Any on-going operations of the clients using these IDs may be
93 disrupted. Mirroring is expected to be disabled on the file system.
101 fs set <fs name> max_file_size <size in bytes>
103 CephFS has a configurable maximum file size, and it's 1TB by default.
104 You may wish to set this limit higher if you expect to store large files
105 in CephFS. It is a 64-bit field.
107 Setting ``max_file_size`` to 0 does not disable the limit. It would
108 simply limit clients to only creating empty files.
111 Maximum file sizes and performance
112 ----------------------------------
114 CephFS enforces the maximum file size limit at the point of appending to
115 files or setting their size. It does not affect how anything is stored.
117 When users create a file of an enormous size (without necessarily
118 writing any data to it), some operations (such as deletes) cause the MDS
119 to have to do a large number of operations to check if any of the RADOS
120 objects within the range that could exist (according to the file size)
123 The ``max_file_size`` setting prevents users from creating files that
124 appear to be eg. exabytes in size, causing load on the MDS as it tries
125 to enumerate the objects during operations like stats or deletes.
128 Taking the cluster down
129 -----------------------
131 Taking a CephFS cluster down is done by setting the down flag:
135 fs set <fs_name> down true
137 To bring the cluster back online:
141 fs set <fs_name> down false
143 This will also restore the previous value of max_mds. MDS daemons are brought
144 down in a way such that journals are flushed to the metadata pool and all
145 client I/O is stopped.
148 Taking the cluster down rapidly for deletion or disaster recovery
149 -----------------------------------------------------------------
151 To allow rapidly deleting a file system (for testing) or to quickly bring the
152 file system and MDS daemons down, use the ``fs fail`` command:
158 This command sets a file system flag to prevent standbys from
159 activating on the file system (the ``joinable`` flag).
161 This process can also be done manually by doing the following:
165 fs set <fs_name> joinable false
167 Then the operator can fail all of the ranks which causes the MDS daemons to
168 respawn as standbys. The file system will be left in a degraded state.
172 # For all ranks, 0-N:
173 mds fail <fs_name>:<n>
175 Once all ranks are inactive, the file system may also be deleted or left in
176 this state for other purposes (perhaps disaster recovery).
178 To bring the cluster back up, simply set the joinable flag:
182 fs set <fs_name> joinable true
188 Most commands manipulating MDSs take a ``<role>`` argument which can take one
197 Commands to manipulate MDS daemons:
201 mds fail <gid/name/role>
203 Mark an MDS daemon as failed. This is equivalent to what the cluster
204 would do if an MDS daemon had failed to send a message to the mon
205 for ``mds_beacon_grace`` second. If the daemon was active and a suitable
206 standby is available, using ``mds fail`` will force a failover to the standby.
208 If the MDS daemon was in reality still running, then using ``mds fail``
209 will cause the daemon to restart. If it was active and a standby was
210 available, then the "failed" daemon will return as a standby.
215 tell mds.<daemon name> command ...
217 Send a command to the MDS daemon(s). Use ``mds.*`` to send a command to all
218 daemons. Use ``ceph tell mds.* help`` to learn available commands.
222 mds metadata <gid/name/role>
224 Get metadata about the given MDS known to the Monitors.
230 Mark the file system rank as repaired. Unlike the name suggests, this command
231 does not change a MDS; it manipulates the file system rank which has been
235 Required Client Features
236 ------------------------
238 It is sometimes desirable to set features that clients must support to talk to
239 CephFS. Clients without those features may disrupt other clients or behave in
240 surprising ways. Or, you may want to require newer features to prevent older
241 and possibly buggy clients from connecting.
243 Commands to manipulate required client features of a file system:
247 fs required_client_features <fs name> add reply_encoding
248 fs required_client_features <fs name> rm reply_encoding
250 To list all CephFS features
256 Clients that are missing newly added features will be evicted automatically.
258 Here are the current CephFS features and first release they came out:
260 +------------------+--------------+-----------------+
261 | Feature | Ceph release | Upstream Kernel |
262 +==================+==============+=================+
263 | jewel | jewel | 4.5 |
264 +------------------+--------------+-----------------+
265 | kraken | kraken | 4.13 |
266 +------------------+--------------+-----------------+
267 | luminous | luminous | 4.13 |
268 +------------------+--------------+-----------------+
269 | mimic | mimic | 4.19 |
270 +------------------+--------------+-----------------+
271 | reply_encoding | nautilus | 5.1 |
272 +------------------+--------------+-----------------+
273 | reclaim_client | nautilus | N/A |
274 +------------------+--------------+-----------------+
275 | lazy_caps_wanted | nautilus | 5.1 |
276 +------------------+--------------+-----------------+
277 | multi_reconnect | nautilus | 5.1 |
278 +------------------+--------------+-----------------+
279 | deleg_ino | octopus | 5.6 |
280 +------------------+--------------+-----------------+
281 | metric_collect | pacific | N/A |
282 +------------------+--------------+-----------------+
283 | alternate_name | pacific | PLANNED |
284 +------------------+--------------+-----------------+
286 CephFS Feature Descriptions
293 MDS encodes request reply in extensible format if client supports this feature.
300 MDS allows new client to reclaim another (dead) client's states. This feature
301 is used by NFS-Ganesha.
308 When a stale client resumes, if the client supports this feature, mds only needs
309 to re-issue caps that are explicitly wanted.
316 When mds failover, client sends reconnect messages to mds, to reestablish cache
317 states. If MDS supports this feature, client can split large reconnect message
325 MDS delegate inode numbers to client if client supports this feature. Having
326 delegated inode numbers is a prerequisite for client to do async file creation.
333 Clients can send performance metric to MDS if MDS support this feature.
339 Clients can set and understand "alternate names" for directory entries. This is
340 to be used for encrypted file name support.
349 fs flag set <flag name> <flag val> [<confirmation string>]
351 Sets a global CephFS flag (i.e. not specific to a particular file system).
352 Currently, the only flag setting is 'enable_multiple' which allows having
353 multiple CephFS file systems.
355 Some flags require you to confirm your intentions with "--yes-i-really-mean-it"
356 or a similar string they will prompt you with. Consider these actions carefully
357 before proceeding; they are placed on especially dangerous activities.
359 .. _advanced-cephfs-admin-settings:
364 These commands are not required in normal operation, and exist
365 for use in exceptional circumstances. Incorrect use of these
366 commands may cause serious problems, such as an inaccessible
373 This removes a rank from the failed set.
377 fs reset <file system name>
379 This command resets the file system state to defaults, except for the name and
380 pools. Non-zero ranks are saved in the stopped set.
385 fs new <file system name> <metadata pool name> <data pool name> --fscid <fscid> --force
387 This command creates a file system with a specific **fscid** (file system cluster ID).
388 You may want to do this when an application expects the file system's ID to be
389 stable after it has been recovered, e.g., after monitor databases are lost and
390 rebuilt. Consequently, file system IDs don't always keep increasing with newer