ceph/doc/cephfs/administration.rst

   1 .. _cephfs-administration:
   2
   3 CephFS Administrative commands
   4 ==============================
   5
   6 File Systems
   7 ------------
   8
   9 .. note:: The names of the file systems, metadata pools, and data pools can
  10           only have characters in the set [a-zA-Z0-9\_-.].
  11
  12 These commands operate on the CephFS file systems in your Ceph cluster.
  13 Note that by default only one file system is permitted: to enable
  14 creation of multiple file systems use ``ceph fs flag set enable_multiple true``.
  15
  16 ::
  17
  18     fs new <file system name> <metadata pool name> <data pool name>
  19
  20 This command creates a new file system. The file system name and metadata pool
  21 name are self-explanatory. The specified data pool is the default data pool and
  22 cannot be changed once set. Each file system has its own set of MDS daemons
  23 assigned to ranks so ensure that you have sufficient standby daemons available
  24 to accommodate the new file system.
  25
  26 ::
  27
  28     fs ls
  29
  30 List all file systems by name.
  31
  32 ::
  33
  34     fs dump [epoch]
  35
  36 This dumps the FSMap at the given epoch (default: current) which includes all
  37 file system settings, MDS daemons and the ranks they hold, and the list of
  38 standby MDS daemons.
  39
  40
  41 ::
  42
  43     fs rm <file system name> [--yes-i-really-mean-it]
  44
  45 Destroy a CephFS file system. This wipes information about the state of the
  46 file system from the FSMap. The metadata pool and data pools are untouched and
  47 must be destroyed separately.
  48
  49 ::
  50
  51     fs get <file system name>
  52
  53 Get information about the named file system, including settings and ranks. This
  54 is a subset of the same information from the ``fs dump`` command.
  55
  56 ::
  57
  58     fs set <file system name> <var> <val>
  59
  60 Change a setting on a file system. These settings are specific to the named
  61 file system and do not affect other file systems.
  62
  63 ::
  64
  65     fs add_data_pool <file system name> <pool name/id>
  66
  67 Add a data pool to the file system. This pool can be used for file layouts
  68 as an alternate location to store file data.
  69
  70 ::
  71
  72     fs rm_data_pool <file system name> <pool name/id>
  73
  74 This command removes the specified pool from the list of data pools for the
  75 file system.  If any files have layouts for the removed data pool, the file
  76 data will become unavailable. The default data pool (when creating the file
  77 system) cannot be removed.
  78
  79
  80 Settings
  81 --------
  82
  83 ::
  84
  85     fs set <fs name> max_file_size <size in bytes>
  86
  87 CephFS has a configurable maximum file size, and it's 1TB by default.
  88 You may wish to set this limit higher if you expect to store large files
  89 in CephFS. It is a 64-bit field.
  90
  91 Setting ``max_file_size`` to 0 does not disable the limit. It would
  92 simply limit clients to only creating empty files.
  93
  94
  95 Maximum file sizes and performance
  96 ----------------------------------
  97
  98 CephFS enforces the maximum file size limit at the point of appending to
  99 files or setting their size. It does not affect how anything is stored.
 100
 101 When users create a file of an enormous size (without necessarily
 102 writing any data to it), some operations (such as deletes) cause the MDS
 103 to have to do a large number of operations to check if any of the RADOS
 104 objects within the range that could exist (according to the file size)
 105 really existed.
 106
 107 The ``max_file_size`` setting prevents users from creating files that
 108 appear to be eg. exabytes in size, causing load on the MDS as it tries
 109 to enumerate the objects during operations like stats or deletes.
 110
 111
 112 Taking the cluster down
 113 -----------------------
 114
 115 Taking a CephFS cluster down is done by setting the down flag:
 116
 117 ::
 118
 119     fs set <fs_name> down true
 120
 121 To bring the cluster back online:
 122
 123 ::
 124
 125     fs set <fs_name> down false
 126
 127 This will also restore the previous value of max_mds. MDS daemons are brought
 128 down in a way such that journals are flushed to the metadata pool and all
 129 client I/O is stopped.
 130
 131
 132 Taking the cluster down rapidly for deletion or disaster recovery
 133 -----------------------------------------------------------------
 134
 135 To allow rapidly deleting a file system (for testing) or to quickly bring the
 136 file system and MDS daemons down, use the ``fs fail`` command:
 137
 138 ::
 139
 140     fs fail <fs_name>
 141
 142 This command sets a file system flag to prevent standbys from
 143 activating on the file system (the ``joinable`` flag).
 144
 145 This process can also be done manually by doing the following:
 146
 147 ::
 148
 149     fs set <fs_name> joinable false
 150
 151 Then the operator can fail all of the ranks which causes the MDS daemons to
 152 respawn as standbys. The file system will be left in a degraded state.
 153
 154 ::
 155
 156     # For all ranks, 0-N:
 157     mds fail <fs_name>:<n>
 158
 159 Once all ranks are inactive, the file system may also be deleted or left in
 160 this state for other purposes (perhaps disaster recovery).
 161
 162 To bring the cluster back up, simply set the joinable flag:
 163
 164 ::
 165
 166     fs set <fs_name> joinable true
 167
 168
 169 Daemons
 170 -------
 171
 172 Most commands manipulating MDSs take a ``<role>`` argument which can take one
 173 of three forms:
 174
 175 ::
 176
 177     <fs_name>:<rank>
 178     <fs_id>:<rank>
 179     <rank>
 180
 181 Commands to manipulate MDS daemons:
 182
 183 ::
 184
 185     mds fail <gid/name/role>
 186
 187 Mark an MDS daemon as failed.  This is equivalent to what the cluster
 188 would do if an MDS daemon had failed to send a message to the mon
 189 for ``mds_beacon_grace`` second.  If the daemon was active and a suitable
 190 standby is available, using ``mds fail`` will force a failover to the standby.
 191
 192 If the MDS daemon was in reality still running, then using ``mds fail``
 193 will cause the daemon to restart.  If it was active and a standby was
 194 available, then the "failed" daemon will return as a standby.
 195
 196
 197 ::
 198
 199     tell mds.<daemon name> command ...
 200
 201 Send a command to the MDS daemon(s). Use ``mds.*`` to send a command to all
 202 daemons. Use ``ceph tell mds.* help`` to learn available commands.
 203
 204 ::
 205
 206     mds metadata <gid/name/role>
 207
 208 Get metadata about the given MDS known to the Monitors.
 209
 210 ::
 211
 212     mds repaired <role>
 213
 214 Mark the file system rank as repaired. Unlike the name suggests, this command
 215 does not change a MDS; it manipulates the file system rank which has been
 216 marked damaged.
 217
 218
 219 Minimum Client Version
 220 ----------------------
 221
 222 It is sometimes desirable to set the minimum version of Ceph that a client must be
 223 running to connect to a CephFS cluster. Older clients may sometimes still be
 224 running with bugs that can cause locking issues between clients (due to
 225 capability release). CephFS provides a mechanism to set the minimum
 226 client version:
 227
 228 ::
 229
 230     fs set <fs name> min_compat_client <release>
 231
 232 For example, to only allow Nautilus clients, use:
 233
 234 ::
 235
 236     fs set cephfs min_compat_client nautilus
 237
 238 Clients running an older version will be automatically evicted.
 239
 240
 241 Global settings
 242 ---------------
 243
 244
 245 ::
 246
 247     fs flag set <flag name> <flag val> [<confirmation string>]
 248
 249 Sets a global CephFS flag (i.e. not specific to a particular file system).
 250 Currently, the only flag setting is 'enable_multiple' which allows having
 251 multiple CephFS file systems.
 252
 253 Some flags require you to confirm your intentions with "--yes-i-really-mean-it"
 254 or a similar string they will prompt you with. Consider these actions carefully
 255 before proceeding; they are placed on especially dangerous activities.
 256
 257
 258 Advanced
 259 --------
 260
 261 These commands are not required in normal operation, and exist
 262 for use in exceptional circumstances.  Incorrect use of these
 263 commands may cause serious problems, such as an inaccessible
 264 file system.
 265
 266 ::
 267
 268     mds compat rm_compat
 269
 270 Removes an compatibility feature flag.
 271
 272 ::
 273
 274     mds compat rm_incompat
 275
 276 Removes an incompatibility feature flag.
 277
 278 ::
 279
 280     mds compat show
 281
 282 Show MDS compatibility flags.
 283
 284 ::
 285
 286     mds rmfailed
 287
 288 This removes a rank from the failed set.
 289
 290 ::
 291
 292     fs reset <file system name>
 293
 294 This command resets the file system state to defaults, except for the name and
 295 pools. Non-zero ranks are saved in the stopped set.