ceph/doc/radosgw/placement.rst

   1 ==================================
   2 Pool Placement and Storage Classes
   3 ==================================
   4
   5 .. contents::
   6
   7 Placement Targets
   8 =================
   9
  10 .. versionadded:: Jewel
  11
  12 Placement targets control which `Pools`_ are associated with a particular
  13 bucket. A bucket's placement target is selected on creation, and cannot be
  14 modified. The ``radosgw-admin bucket stats`` command will display its
  15 ``placement_rule``.
  16
  17 The zonegroup configuration contains a list of placement targets with an
  18 initial target named ``default-placement``. The zone configuration then maps
  19 each zonegroup placement target name onto its local storage. This zone
  20 placement information includes the ``index_pool`` name for the bucket index,
  21 the ``data_extra_pool`` name for metadata about incomplete multipart uploads,
  22 and a ``data_pool`` name for each storage class.
  23
  24 .. _storage_classes:
  25
  26 Storage Classes
  27 ===============
  28
  29 .. versionadded:: Nautilus
  30
  31 Storage classes are used to customize the placement of object data. S3 Bucket
  32 Lifecycle rules can automate the transition of objects between storage classes.
  33
  34 Storage classes are defined in terms of placement targets. Each zonegroup
  35 placement target lists its available storage classes with an initial class
  36 named ``STANDARD``. The zone configuration is responsible for providing a
  37 ``data_pool`` pool name for each of the zonegroup's storage classes.
  38
  39 Zonegroup/Zone Configuration
  40 ============================
  41
  42 Placement configuration is performed with ``radosgw-admin`` commands on
  43 the zonegroups and zones.
  44
  45 The zonegroup placement configuration can be queried with:
  46
  47 ::
  48
  49   $ radosgw-admin zonegroup get
  50   {
  51       "id": "ab01123f-e0df-4f29-9d71-b44888d67cd5",
  52       "name": "default",
  53       "api_name": "default",
  54       ...
  55       "placement_targets": [
  56           {
  57               "name": "default-placement",
  58               "tags": [],
  59               "storage_classes": [
  60                   "STANDARD"
  61               ]
  62           }
  63       ],
  64       "default_placement": "default-placement",
  65       ...
  66   }
  67
  68 The zone placement configuration can be queried with:
  69
  70 ::
  71
  72   $ radosgw-admin zone get
  73   {
  74       "id": "557cdcee-3aae-4e9e-85c7-2f86f5eddb1f",
  75       "name": "default",
  76       "domain_root": "default.rgw.meta:root",
  77       ...
  78       "placement_pools": [
  79           {
  80               "key": "default-placement",
  81               "val": {
  82                   "index_pool": "default.rgw.buckets.index",
  83                   "storage_classes": {
  84                       "STANDARD": {
  85                           "data_pool": "default.rgw.buckets.data"
  86                       }
  87                   },
  88                   "data_extra_pool": "default.rgw.buckets.non-ec",
  89                   "index_type": 0
  90               }
  91           }
  92       ],
  93       ...
  94   }
  95
  96 .. note:: If you have not done any previous `Multisite Configuration`_,
  97           a ``default`` zone and zonegroup are created for you, and changes
  98           to the zone/zonegroup will not take effect until the Ceph Object
  99           Gateways are restarted. If you have created a realm for multisite,
 100           the zone/zonegroup changes will take effect once the changes are
 101           committed with ``radosgw-admin period update --commit``.
 102
 103 Adding a Placement Target
 104 -------------------------
 105
 106 To create a new placement target named ``temporary``, start by adding it to
 107 the zonegroup:
 108
 109 ::
 110
 111   $ radosgw-admin zonegroup placement add \
 112         --rgw-zonegroup default \
 113         --placement-id temporary
 114
 115 Then provide the zone placement info for that target:
 116
 117 ::
 118
 119   $ radosgw-admin zone placement add \
 120         --rgw-zone default \
 121         --placement-id temporary \
 122         --data-pool default.rgw.temporary.data \
 123         --index-pool default.rgw.temporary.index \
 124         --data-extra-pool default.rgw.temporary.non-ec
 125
 126 .. _adding_a_storage_class:
 127
 128 Adding a Storage Class
 129 ----------------------
 130
 131 To add a new storage class named ``GLACIER`` to the ``default-placement`` target,
 132 start by adding it to the zonegroup:
 133
 134 ::
 135
 136   $ radosgw-admin zonegroup placement add \
 137         --rgw-zonegroup default \
 138         --placement-id default-placement \
 139         --storage-class GLACIER
 140
 141 Then provide the zone placement info for that storage class:
 142
 143 ::
 144
 145   $ radosgw-admin zone placement add \
 146         --rgw-zone default \
 147         --placement-id default-placement \
 148         --storage-class GLACIER \
 149         --data-pool default.rgw.glacier.data \
 150         --compression lz4
 151
 152 Customizing Placement
 153 =====================
 154
 155 Default Placement
 156 -----------------
 157
 158 By default, new buckets will use the zonegroup's ``default_placement`` target.
 159 This zonegroup setting can be changed with:
 160
 161 ::
 162
 163   $ radosgw-admin zonegroup placement default \
 164         --rgw-zonegroup default \
 165         --placement-id new-placement
 166
 167 User Placement
 168 --------------
 169
 170 A Ceph Object Gateway user can override the zonegroup's default placement
 171 target by setting a non-empty ``default_placement`` field in the user info.
 172 Similarly, the ``default_storage_class`` can override the ``STANDARD``
 173 storage class applied to objects by default.
 174
 175 ::
 176
 177   $ radosgw-admin user info --uid testid
 178   {
 179       ...
 180       "default_placement": "",
 181       "default_storage_class": "",
 182       "placement_tags": [],
 183       ...
 184   }
 185
 186 If a zonegroup's placement target contains any ``tags``, users will be unable
 187 to create buckets with that placement target unless their user info contains
 188 at least one matching tag in its ``placement_tags`` field. This can be useful
 189 to restrict access to certain types of storage.
 190
 191 The ``radosgw-admin`` command can modify these fields directly with:
 192
 193 ::
 194
 195   $ radosgw-admin user modify \
 196         --uid <user-id> \
 197         --placement-id <default-placement-id> \
 198         --storage-class <default-storage-class> \
 199         --tags <tag1,tag2>
 200
 201 .. _s3_bucket_placement:
 202
 203 S3 Bucket Placement
 204 -------------------
 205
 206 When creating a bucket with the S3 protocol, a placement target can be
 207 provided as part of the LocationConstraint to override the default placement
 208 targets from the user and zonegroup.
 209
 210 Normally, the LocationConstraint must match the zonegroup's ``api_name``:
 211
 212 ::
 213
 214   <LocationConstraint>default</LocationConstraint>
 215
 216 A custom placement target can be added to the ``api_name`` following a colon:
 217
 218 ::
 219
 220   <LocationConstraint>default:new-placement</LocationConstraint>
 221
 222 Swift Bucket Placement
 223 ----------------------
 224
 225 When creating a bucket with the Swift protocol, a placement target can be
 226 provided in the HTTP header ``X-Storage-Policy``:
 227
 228 ::
 229
 230   X-Storage-Policy: new-placement
 231
 232 Using Storage Classes
 233 =====================
 234
 235 All placement targets have a ``STANDARD`` storage class which is applied to
 236 new objects by default. The user can override this default with its
 237 ``default_storage_class``.
 238
 239 To create an object in a non-default storage class, provide that storage class
 240 name in an HTTP header with the request. The S3 protocol uses the
 241 ``X-Amz-Storage-Class`` header, while the Swift protocol uses the
 242 ``X-Object-Storage-Class`` header.
 243
 244 When using AWS S3 SDKs such as ``boto3``, it is important that non-default
 245 storage class names match those provided by AWS S3, or else the SDK
 246 will drop the request and raise an exception.
 247
 248 S3 Object Lifecycle Management can then be used to move object data between
 249 storage classes using ``Transition`` actions.
 250
 251 .. _`Pools`: ../pools
 252 .. _`Multisite Configuration`: ../multisite