ceph/doc/cephfs/file-layouts.rst

   1
   2 File layouts
   3 ============
   4
   5 The layout of a file controls how its contents are mapped to Ceph RADOS objects.  You can
   6 read and write a file's layout using *virtual extended attributes* or xattrs.
   7
   8 The name of the layout xattrs depends on whether a file is a regular file or a directory.  Regular
   9 files' layout xattrs are called ``ceph.file.layout``, whereas directories' layout xattrs are called
  10 ``ceph.dir.layout``.  Where subsequent examples refer to ``ceph.file.layout``, substitute ``dir`` as appropriate
  11 when dealing with directories.
  12
  13 .. tip::
  14
  15     Your linux distribution may not ship with commands for manipulating xattrs by default,
  16     the required package is usually called ``attr``.
  17
  18 Layout fields
  19 -------------
  20
  21 pool
  22     String, giving ID or name.  Which RADOS pool a file's data objects will be stored in.
  23
  24 pool_namespace
  25     String.  Within the data pool, which RADOS namespace the objects will
  26     be written to.  Empty by default (i.e. default namespace).
  27
  28 stripe_unit
  29     Integer in bytes.  The size (in bytes) of a block of data used in the RAID 0 distribution of a file. All stripe units for a file have equal size. The last stripe unit is typically incomplete–i.e. it represents the data at the end of the file as well as unused “space” beyond it up to the end of the fixed stripe unit size.
  30
  31 stripe_count
  32     Integer.  The number of consecutive stripe units that constitute a RAID 0 “stripe” of file data.
  33
  34 object_size
  35     Integer in bytes.  File data is chunked into RADOS objects of this size.
  36
  37 .. tip::
  38
  39     RADOS enforces a configurable limit on object sizes: if you increase CephFS
  40     object sizes beyond that limit then writes may not succeed.  The OSD
  41     setting is ``rados_max_object_size``, which is 128MB by default.
  42     Very large RADOS objects may prevent smooth operation of the cluster,
  43     so increasing the object size limit past the default is not recommended.
  44
  45 Reading layouts with ``getfattr``
  46 ---------------------------------
  47
  48 Read the layout information as a single string:
  49
  50 .. code-block:: bash
  51
  52     $ touch file
  53     $ getfattr -n ceph.file.layout file
  54     # file: file
  55     ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data"
  56
  57 Read individual layout fields:
  58
  59 .. code-block:: bash
  60
  61     $ getfattr -n ceph.file.layout.pool file
  62     # file: file
  63     ceph.file.layout.pool="cephfs_data"
  64     $ getfattr -n ceph.file.layout.stripe_unit file
  65     # file: file
  66     ceph.file.layout.stripe_unit="4194304"
  67     $ getfattr -n ceph.file.layout.stripe_count file
  68     # file: file
  69     ceph.file.layout.stripe_count="1"
  70     $ getfattr -n ceph.file.layout.object_size file
  71     # file: file
  72     ceph.file.layout.object_size="4194304"
  73
  74 .. note::
  75
  76     When reading layouts, the pool will usually be indicated by name.  However, in
  77     rare cases when pools have only just been created, the ID may be output instead.
  78
  79 Directories do not have an explicit layout until it is customized.  Attempts to read
  80 the layout will fail if it has never been modified: this indicates that layout of the
  81 next ancestor directory with an explicit layout will be used.
  82
  83 .. code-block:: bash
  84
  85     $ mkdir dir
  86     $ getfattr -n ceph.dir.layout dir
  87     dir: ceph.dir.layout: No such attribute
  88     $ setfattr -n ceph.dir.layout.stripe_count -v 2 dir
  89     $ getfattr -n ceph.dir.layout dir
  90     # file: dir
  91     ceph.dir.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
  92
  93
  94 Writing layouts with ``setfattr``
  95 ---------------------------------
  96
  97 Layout fields are modified using ``setfattr``:
  98
  99 .. code-block:: bash
 100
 101     $ ceph osd lspools
 102     0 rbd,1 cephfs_data,2 cephfs_metadata,
 103
 104     $ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2
 105     $ setfattr -n ceph.file.layout.stripe_count -v 8 file2
 106     $ setfattr -n ceph.file.layout.object_size -v 10485760 file2
 107     $ setfattr -n ceph.file.layout.pool -v 1 file2  # Setting pool by ID
 108     $ setfattr -n ceph.file.layout.pool -v cephfs_data file2  # Setting pool by name
 109
 110 .. note::
 111
 112     When the layout fields of a file are modified using ``setfattr``, this file must be empty, otherwise an error will occur.
 113
 114 .. code-block:: bash
 115
 116     # touch an empty file
 117     $ touch file1
 118     # modify layout field successfully
 119     $ setfattr -n ceph.file.layout.stripe_count -v 3 file1
 120
 121     # write something to file1
 122     $ echo "hello world" > file1
 123     $ setfattr -n ceph.file.layout.stripe_count -v 4 file1
 124     setfattr: file1: Directory not empty
 125
 126 Clearing layouts
 127 ----------------
 128
 129 If you wish to remove an explicit layout from a directory, to revert to
 130 inherting the layout of its ancestor, you can do so:
 131
 132 .. code-block:: bash
 133
 134     setfattr -x ceph.dir.layout mydir
 135
 136 Similarly, if you have set the ``pool_namespace`` attribute and wish
 137 to modify the layout to use the default namespace instead:
 138
 139 .. code-block:: bash
 140
 141     # Create a dir and set a namespace on it
 142     mkdir mydir
 143     setfattr -n ceph.dir.layout.pool_namespace -v foons mydir
 144     getfattr -n ceph.dir.layout mydir
 145     ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_a pool_namespace=foons"
 146
 147     # Clear the namespace from the directory's layout
 148     setfattr -x ceph.dir.layout.pool_namespace mydir
 149     getfattr -n ceph.dir.layout mydir
 150     ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_a"
 151
 152
 153 Inheritance of layouts
 154 ----------------------
 155
 156 Files inherit the layout of their parent directory at creation time.  However, subsequent
 157 changes to the parent directory's layout do not affect children.
 158
 159 .. code-block:: bash
 160
 161     $ getfattr -n ceph.dir.layout dir
 162     # file: dir
 163     ceph.dir.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
 164
 165     # Demonstrate file1 inheriting its parent's layout
 166     $ touch dir/file1
 167     $ getfattr -n ceph.file.layout dir/file1
 168     # file: dir/file1
 169     ceph.file.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
 170
 171     # Now update the layout of the directory before creating a second file
 172     $ setfattr -n ceph.dir.layout.stripe_count -v 4 dir
 173     $ touch dir/file2
 174
 175     # Demonstrate that file1's layout is unchanged
 176     $ getfattr -n ceph.file.layout dir/file1
 177     # file: dir/file1
 178     ceph.file.layout="stripe_unit=4194304 stripe_count=2 object_size=4194304 pool=cephfs_data"
 179
 180     # ...while file2 has the parent directory's new layout
 181     $ getfattr -n ceph.file.layout dir/file2
 182     # file: dir/file2
 183     ceph.file.layout="stripe_unit=4194304 stripe_count=4 object_size=4194304 pool=cephfs_data"
 184
 185
 186 Files created as descendents of the directory also inherit the layout, if the intermediate
 187 directories do not have layouts set:
 188
 189 .. code-block:: bash
 190
 191     $ getfattr -n ceph.dir.layout dir
 192     # file: dir
 193     ceph.dir.layout="stripe_unit=4194304 stripe_count=4 object_size=4194304 pool=cephfs_data"
 194     $ mkdir dir/childdir
 195     $ getfattr -n ceph.dir.layout dir/childdir
 196     dir/childdir: ceph.dir.layout: No such attribute
 197     $ touch dir/childdir/grandchild
 198     $ getfattr -n ceph.file.layout dir/childdir/grandchild
 199     # file: dir/childdir/grandchild
 200     ceph.file.layout="stripe_unit=4194304 stripe_count=4 object_size=4194304 pool=cephfs_data"
 201
 202
 203 Adding a data pool to the MDS
 204 ---------------------------------
 205
 206 Before you can use a pool with CephFS you have to add it to the Metadata Servers.
 207
 208 .. code-block:: bash
 209
 210     $ ceph fs add_data_pool cephfs cephfs_data_ssd
 211     # Pool should now show up
 212     $ ceph fs ls
 213     .... data pools: [cephfs_data cephfs_data_ssd ]
 214
 215 Make sure that your cephx keys allows the client to access this new pool.