ceph/doc/cephfs/quota.rst

   1 CephFS Quotas
   2 =============
   3
   4 CephFS allows quotas to be set on any directory in the file system.  The
   5 quota can restrict the number of *bytes* or the number of *files*
   6 stored beneath that point in the directory hierarchy.
   7
   8 Like most other things in CephFS, quotas are configured using virtual
   9 extended attributes:
  10
  11  * ``ceph.quota.max_files`` -- file limit
  12  * ``ceph.quota.max_bytes`` -- byte limit
  13
  14 If the extended attributes appear on a directory that means a quota is
  15 configured there. If they are not present then no quota is set on that
  16 directory (although one may still be configured on a parent directory).
  17
  18 To set a quota, set the extended attribute on a CephFS directory with a
  19 value::
  20
  21   setfattr -n ceph.quota.max_bytes -v 100000000 /some/dir     # 100 MB
  22   setfattr -n ceph.quota.max_files -v 10000 /some/dir         # 10,000 files
  23
  24 To view quota limit::
  25
  26   $ getfattr -n ceph.quota.max_bytes /some/dir
  27   # file: dir1/
  28   ceph.quota.max_bytes="100000000"
  29   $
  30   $ getfattr -n ceph.quota.max_files /some/dir
  31   # file: dir1/
  32   ceph.quota.max_files="10000"
  33
  34 .. note:: Running ``getfattr /some/dir -d -m -`` for a CephFS directory will
  35    print none of the CephFS extended attributes. This is because the CephFS
  36    kernel and FUSE clients hide this information from the ``listxattr(2)``
  37    system call. Instead, a specific CephFS extended attribute can be viewed by
  38    running ``getfattr /some/dir -n ceph.<some-xattr>``.
  39
  40 To remove a quota, set the value of extended attribute to ``0``::
  41
  42   $ setfattr -n ceph.quota.max_bytes -v 0 /some/dir
  43   $ getfattr /some/dir -n ceph.quota.max_bytes
  44   dir1/: ceph.quota.max_bytes: No such attribute
  45   $
  46   $ setfattr -n ceph.quota.max_files -v 0 /some/dir
  47   $ getfattr dir1/ -n ceph.quota.max_files
  48   dir1/: ceph.quota.max_files: No such attribute
  49
  50 Space Usage Reporting and CephFS Quotas
  51 ---------------------------------------
  52 When the root directory of the CephFS mount has quota set on it, the available
  53 space on the CephFS reported by space usage report tools (like ``df``) is
  54 based on quota limit. That is, ``available space = quota limit - used space``
  55 instead of ``available space = total space - used space``.
  56
  57 This behaviour can be disabled by setting following option in client section
  58 of ``ceph.conf``::
  59
  60     client quota df = false
  61
  62 Limitations
  63 -----------
  64
  65 #. *Quotas are cooperative and non-adversarial.* CephFS quotas rely on
  66    the cooperation of the client who is mounting the file system to
  67    stop writers when a limit is reached.  A modified or adversarial
  68    client cannot be prevented from writing as much data as it needs.
  69    Quotas should not be relied on to prevent filling the system in
  70    environments where the clients are fully untrusted.
  71
  72 #. *Quotas are imprecise.* Processes that are writing to the file
  73    system will be stopped a short time after the quota limit is
  74    reached.  They will inevitably be allowed to write some amount of
  75    data over the configured limit.  How far over the quota they are
  76    able to go depends primarily on the amount of time, not the amount
  77    of data.  Generally speaking writers will be stopped within 10s of
  78    seconds of crossing the configured limit.
  79
  80 #. *Quotas are implemented in the kernel client 4.17 and higher.*
  81    Quotas are supported by the userspace client (libcephfs, ceph-fuse).
  82    Linux kernel clients >= 4.17 support CephFS quotas but only on
  83    mimic+ clusters.  Kernel clients (even recent versions) will fail
  84    to handle quotas on older clusters, even if they may be able to set
  85    the quotas extended attributes.
  86
  87 #. *Quotas must be configured carefully when used with path-based
  88    mount restrictions.* The client needs to have access to the
  89    directory inode on which quotas are configured in order to enforce
  90    them.  If the client has restricted access to a specific path
  91    (e.g., ``/home/user``) based on the MDS capability, and a quota is
  92    configured on an ancestor directory they do not have access to
  93    (e.g., ``/home``), the client will not enforce it.  When using
  94    path-based access restrictions be sure to configure the quota on
  95    the directory the client is restricted too (e.g., ``/home/user``)
  96    or something nested beneath it.
  97
  98    In case of a kernel client, it needs to have access to the parent
  99    of the directory inode on which quotas are configured in order to
 100    enforce them. If quota is configured on a directory path
 101    (e.g., ``/home/volumes/group``), the kclient needs to have access
 102    to the parent (e.g., ``/home/volumes``).
 103
 104    An example command to create such an user is as below::
 105
 106      $ ceph auth get-or-create client.guest mds 'allow r path=/home/volumes, allow rw path=/home/volumes/group' mgr 'allow rw' osd 'allow rw tag cephfs metadata=*' mon 'allow r'
 107
 108    See also: https://tracker.ceph.com/issues/55090
 109
 110 #. *Snapshot file data which has since been deleted or changed does not count
 111    towards the quota.* See also: http://tracker.ceph.com/issues/24284