ceph/doc/radosgw/multisite.rst

   1 .. _multisite:
   2
   3 ==========
   4 Multi-Site
   5 ==========
   6
   7 .. versionadded:: Jewel
   8
   9 A single zone configuration typically consists of one zone group containing one
  10 zone and one or more `ceph-radosgw` instances where you may load-balance gateway
  11 client requests between the instances. In a single zone configuration, typically
  12 multiple gateway instances point to a single Ceph storage cluster. However, Kraken
  13 supports several multi-site configuration options for the Ceph Object Gateway:
  14
  15 - **Multi-zone:** A more advanced configuration consists of one zone group and
  16   multiple zones, each zone with one or more `ceph-radosgw` instances. Each zone
  17   is backed by its own Ceph Storage Cluster. Multiple zones in a zone group
  18   provides disaster recovery for the zone group should one of the zones experience
  19   a significant failure. In Kraken, each zone is active and may receive write
  20   operations. In addition to disaster recovery, multiple active zones may also
  21   serve as a foundation for content delivery networks.
  22
  23 - **Multi-zone-group:** Formerly called 'regions', Ceph Object Gateway can also
  24   support multiple zone groups, each zone group with one or more zones. Objects
  25   stored to zones in one zone group within the same realm as another zone
  26   group will share a global object namespace, ensuring unique object IDs across
  27   zone groups and zones.
  28
  29 - **Multiple Realms:** In Kraken, the Ceph Object Gateway supports the notion
  30   of realms, which can be a single zone group or multiple zone groups and
  31   a globally unique namespace for the realm. Multiple realms provide the ability
  32   to support numerous configurations and namespaces.
  33
  34 Replicating object data between zones within a zone group looks something
  35 like this:
  36
  37 .. image:: ../images/zone-sync2.png
  38    :align: center
  39
  40 For additional details on setting up a cluster, see `Ceph Object Gateway for
  41 Production <https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html/ceph_object_gateway_for_production/>`__.
  42
  43 Functional Changes from Infernalis
  44 ==================================
  45
  46 In Kraken, you can configure each Ceph Object Gateway to
  47 work in an active-active zone configuration, allowing for writes to
  48 non-master zones.
  49
  50 The multi-site configuration is stored within a container called a
  51 "realm." The realm stores zone groups, zones, and a time "period" with
  52 multiple epochs for tracking changes to the configuration. In Kraken,
  53 the ``ceph-radosgw`` daemons handle the synchronization,
  54 eliminating the need for a separate synchronization agent. Additionally,
  55 the new approach to synchronization allows the Ceph Object Gateway to
  56 operate with an "active-active" configuration instead of
  57 "active-passive".
  58
  59 Requirements and Assumptions
  60 ============================
  61
  62 A multi-site configuration requires at least two Ceph storage clusters,
  63 preferably given a distinct cluster name. At least two Ceph object
  64 gateway instances, one for each Ceph storage cluster.
  65
  66 This guide assumes at least two Ceph storage clusters are in geographically
  67 separate locations; however, the configuration can work on the same
  68 site. This guide also assumes two Ceph object gateway servers named
  69 ``rgw1`` and ``rgw2``.
  70
  71 .. important:: Running a single Ceph storage cluster is NOT recommended unless you have
  72                low latency WAN connections.
  73
  74 A multi-site configuration requires a master zone group and a master
  75 zone. Additionally, each zone group requires a master zone. Zone groups
  76 may have one or more secondary or non-master zones.
  77
  78 In this guide, the ``rgw1`` host will serve as the master zone of the
  79 master zone group; and, the ``rgw2`` host will serve as the secondary zone
  80 of the master zone group.
  81
  82 See `Pools`_ for instructions on creating and tuning pools for Ceph
  83 Object Storage.
  84
  85
  86 Configuring a Master Zone
  87 =========================
  88
  89 All gateways in a multi-site configuration will retrieve their
  90 configuration from a ``ceph-radosgw`` daemon on a host within the master
  91 zone group and master zone. To configure your gateways in a multi-site
  92 configuration, choose a ``ceph-radosgw`` instance to configure the
  93 master zone group and master zone.
  94
  95 Create a Realm
  96 --------------
  97
  98 A realm contains the multi-site configuration of zone groups and zones
  99 and also serves to enforce a globally unique namespace within the realm.
 100
 101 Create a new realm for the multi-site configuration by opening a command
 102 line interface on a host identified to serve in the master zone group
 103 and zone. Then, execute the following:
 104
 105 ::
 106
 107     # radosgw-admin realm create --rgw-realm={realm-name} [--default]
 108
 109 For example:
 110
 111 ::
 112
 113     # radosgw-admin realm create --rgw-realm=movies --default
 114
 115 If the cluster will have a single realm, specify the ``--default`` flag.
 116 If ``--default`` is specified, ``radosgw-admin`` will use this realm by
 117 default. If ``--default`` is not specified, adding zone-groups and zones
 118 requires specifying either the ``--rgw-realm`` flag or the
 119 ``--realm-id`` flag to identify the realm when adding zone groups and
 120 zones.
 121
 122 After creating the realm, ``radosgw-admin`` will echo back the realm
 123 configuration. For example:
 124
 125 ::
 126
 127     {
 128         "id": "0956b174-fe14-4f97-8b50-bb7ec5e1cf62",
 129         "name": "movies",
 130         "current_period": "1950b710-3e63-4c41-a19e-46a715000980",
 131         "epoch": 1
 132     }
 133
 134 .. note:: Ceph generates a unique ID for the realm, which allows the renaming
 135           of a realm if the need arises.
 136
 137 Create a Master Zone Group
 138 --------------------------
 139
 140 A realm must have at least one zone group, which will serve as the
 141 master zone group for the realm.
 142
 143 Create a new master zone group for the multi-site configuration by
 144 opening a command line interface on a host identified to serve in the
 145 master zone group and zone. Then, execute the following:
 146
 147 ::
 148
 149     # radosgw-admin zonegroup create --rgw-zonegroup={name} --endpoints={url} [--rgw-realm={realm-name}|--realm-id={realm-id}] --master --default
 150
 151 For example:
 152
 153 ::
 154
 155     # radosgw-admin zonegroup create --rgw-zonegroup=us --endpoints=http://rgw1:80 --rgw-realm=movies --master --default
 156
 157 If the realm will only have a single zone group, specify the
 158 ``--default`` flag. If ``--default`` is specified, ``radosgw-admin``
 159 will use this zone group by default when adding new zones. If
 160 ``--default`` is not specified, adding zones will require either the
 161 ``--rgw-zonegroup`` flag or the ``--zonegroup-id`` flag to identify the
 162 zone group when adding or modifying zones.
 163
 164 After creating the master zone group, ``radosgw-admin`` will echo back
 165 the zone group configuration. For example:
 166
 167 ::
 168
 169     {
 170         "id": "f1a233f5-c354-4107-b36c-df66126475a6",
 171         "name": "us",
 172         "api_name": "us",
 173         "is_master": "true",
 174         "endpoints": [
 175             "http:\/\/rgw1:80"
 176         ],
 177         "hostnames": [],
 178         "hostnames_s3webzone": [],
 179         "master_zone": "",
 180         "zones": [],
 181         "placement_targets": [],
 182         "default_placement": "",
 183         "realm_id": "0956b174-fe14-4f97-8b50-bb7ec5e1cf62"
 184     }
 185
 186 Create a Master Zone
 187 --------------------
 188
 189 .. important:: Zones must be created on a Ceph Object Gateway node that will be
 190                within the zone.
 191
 192 Create a new master zone for the multi-site configuration by opening a
 193 command line interface on a host identified to serve in the master zone
 194 group and zone. Then, execute the following:
 195
 196 ::
 197
 198     # radosgw-admin zone create --rgw-zonegroup={zone-group-name} \
 199                                 --rgw-zone={zone-name} \
 200                                 --master --default \
 201                                 --endpoints={http://fqdn}[,{http://fqdn}]
 202
 203
 204 For example:
 205
 206 ::
 207
 208     # radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-east \
 209                                 --master --default \
 210                                 --endpoints={http://fqdn}[,{http://fqdn}]
 211
 212
 213 .. note:: The ``--access-key`` and ``--secret`` aren’t specified. These
 214           settings will be added to the zone once the user is created in the
 215           next section.
 216
 217 .. important:: The following steps assume a multi-site configuration using newly
 218                installed systems that aren’t storing data yet. DO NOT DELETE the
 219                ``default`` zone and its pools if you are already using it to store
 220                data, or the data will be deleted and unrecoverable.
 221
 222 Delete Default Zone Group and Zone
 223 ----------------------------------
 224
 225 Delete the ``default`` zone if it exists. Make sure to remove it from
 226 the default zone group first.
 227
 228 ::
 229
 230     # radosgw-admin zonegroup remove --rgw-zonegroup=default --rgw-zone=default
 231     # radosgw-admin period update --commit
 232     # radosgw-admin zone rm --rgw-zone=default
 233     # radosgw-admin period update --commit
 234     # radosgw-admin zonegroup delete --rgw-zonegroup=default
 235     # radosgw-admin period update --commit
 236
 237 Finally, delete the ``default`` pools in your Ceph storage cluster if
 238 they exist.
 239
 240 .. important:: The following step assumes a multi-site configuration using newly
 241                installed systems that aren’t currently storing data. DO NOT DELETE
 242                the ``default`` zone group if you are already using it to store
 243                data.
 244
 245 ::
 246
 247     # ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it
 248     # ceph osd pool rm default.rgw.data.root default.rgw.data.root --yes-i-really-really-mean-it
 249     # ceph osd pool rm default.rgw.gc default.rgw.gc --yes-i-really-really-mean-it
 250     # ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it
 251     # ceph osd pool rm default.rgw.users.uid default.rgw.users.uid --yes-i-really-really-mean-it
 252
 253 Create a System User
 254 --------------------
 255
 256 The ``ceph-radosgw`` daemons must authenticate before pulling realm and
 257 period information. In the master zone, create a system user to
 258 facilitate authentication between daemons.
 259
 260 ::
 261
 262     # radosgw-admin user create --uid="{user-name}" --display-name="{Display Name}" --system
 263
 264 For example:
 265
 266 ::
 267
 268     # radosgw-admin user create --uid="synchronization-user" --display-name="Synchronization User" --system
 269
 270 Make a note of the ``access_key`` and ``secret_key``, as the secondary
 271 zones will require them to authenticate with the master zone.
 272
 273 Finally, add the system user to the master zone.
 274
 275 ::
 276
 277     # radosgw-admin zone modify --rgw-zone=us-east --access-key={access-key} --secret={secret}
 278     # radosgw-admin period update --commit
 279
 280 Update the Period
 281 -----------------
 282
 283 After updating the master zone configuration, update the period.
 284
 285 ::
 286
 287     # radosgw-admin period update --commit
 288
 289 .. note:: Updating the period changes the epoch, and ensures that other zones
 290           will receive the updated configuration.
 291
 292 Update the Ceph Configuration File
 293 ----------------------------------
 294
 295 Update the Ceph configuration file on master zone hosts by adding the
 296 ``rgw_zone`` configuration option and the name of the master zone to the
 297 instance entry.
 298
 299 ::
 300
 301     [client.rgw.{instance-name}]
 302     ...
 303     rgw_zone={zone-name}
 304
 305 For example:
 306
 307 ::
 308
 309     [client.rgw.rgw1]
 310     host = rgw1
 311     rgw frontends = "civetweb port=80"
 312     rgw_zone=us-east
 313
 314 Start the Gateway
 315 -----------------
 316
 317 On the object gateway host, start and enable the Ceph Object Gateway
 318 service:
 319
 320 ::
 321
 322     # systemctl start ceph-radosgw@rgw.`hostname -s`
 323     # systemctl enable ceph-radosgw@rgw.`hostname -s`
 324
 325 Configure Secondary Zones
 326 =========================
 327
 328 Zones within a zone group replicate all data to ensure that each zone
 329 has the same data. When creating the secondary zone, execute all of the
 330 following operations on a host identified to serve the secondary zone.
 331
 332 .. note:: To add a third zone, follow the same procedures as for adding the
 333           secondary zone. Use different zone name.
 334
 335 .. important:: You must execute metadata operations, such as user creation, on a
 336                host within the master zone. The master zone and the secondary zone
 337                can receive bucket operations, but the secondary zone redirects
 338                bucket operations to the master zone. If the master zone is down,
 339                bucket operations will fail.
 340
 341 Pull the Realm
 342 --------------
 343
 344 Using the URL path, access key and secret of the master zone in the
 345 master zone group, pull the realm configuration to the host. To pull a
 346 non-default realm, specify the realm using the ``--rgw-realm`` or
 347 ``--realm-id`` configuration options.
 348
 349 ::
 350
 351     # radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key} --secret={secret}
 352
 353 .. note:: Pulling the realm also retrieves the remote's current period
 354           configuration, and makes it the current period on this host as well.
 355
 356 If this realm is the default realm or the only realm, make the realm the
 357 default realm.
 358
 359 ::
 360
 361     # radosgw-admin realm default --rgw-realm={realm-name}
 362
 363 Create a Secondary Zone
 364 -----------------------
 365
 366 .. important:: Zones must be created on a Ceph Object Gateway node that will be
 367                within the zone.
 368
 369 Create a secondary zone for the multi-site configuration by opening a
 370 command line interface on a host identified to serve the secondary zone.
 371 Specify the zone group ID, the new zone name and an endpoint for the
 372 zone. **DO NOT** use the ``--master`` or ``--default`` flags. In Kraken,
 373 all zones run in an active-active configuration by
 374 default; that is, a gateway client may write data to any zone and the
 375 zone will replicate the data to all other zones within the zone group.
 376 If the secondary zone should not accept write operations, specify the
 377 ``--read-only`` flag to create an active-passive configuration between
 378 the master zone and the secondary zone. Additionally, provide the
 379 ``access_key`` and ``secret_key`` of the generated system user stored in
 380 the master zone of the master zone group. Execute the following:
 381
 382 ::
 383
 384     # radosgw-admin zone create --rgw-zonegroup={zone-group-name}\
 385                                 --rgw-zone={zone-name} --endpoints={url} \
 386                                 --access-key={system-key} --secret={secret}\
 387                                 --endpoints=http://{fqdn}:80 \
 388                                 [--read-only]
 389
 390 For example:
 391
 392 ::
 393
 394     # radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-west \
 395                                 --access-key={system-key} --secret={secret} \
 396                                 --endpoints=http://rgw2:80
 397
 398 .. important:: The following steps assume a multi-site configuration using newly
 399                installed systems that aren’t storing data. **DO NOT DELETE** the
 400                ``default`` zone and its pools if you are already using it to store
 401                data, or the data will be lost and unrecoverable.
 402
 403 Delete the default zone if needed.
 404
 405 ::
 406
 407     # radosgw-admin zone rm --rgw-zone=default
 408
 409 Finally, delete the default pools in your Ceph storage cluster if
 410 needed.
 411
 412 ::
 413
 414     # ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it
 415     # ceph osd pool rm default.rgw.data.root default.rgw.data.root --yes-i-really-really-mean-it
 416     # ceph osd pool rm default.rgw.gc default.rgw.gc --yes-i-really-really-mean-it
 417     # ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it
 418     # ceph osd pool rm default.rgw.users.uid default.rgw.users.uid --yes-i-really-really-mean-it
 419
 420 Update the Ceph Configuration File
 421 ----------------------------------
 422
 423 Update the Ceph configuration file on the secondary zone hosts by adding
 424 the ``rgw_zone`` configuration option and the name of the secondary zone
 425 to the instance entry.
 426
 427 ::
 428
 429     [client.rgw.{instance-name}]
 430     ...
 431     rgw_zone={zone-name}
 432
 433 For example:
 434
 435 ::
 436
 437     [client.rgw.rgw2]
 438     host = rgw2
 439     rgw frontends = "civetweb port=80"
 440     rgw_zone=us-west
 441
 442 Update the Period
 443 -----------------
 444
 445 After updating the master zone configuration, update the period.
 446
 447 ::
 448
 449     # radosgw-admin period update --commit
 450
 451 .. note:: Updating the period changes the epoch, and ensures that other zones
 452           will receive the updated configuration.
 453
 454 Start the Gateway
 455 -----------------
 456
 457 On the object gateway host, start and enable the Ceph Object Gateway
 458 service:
 459
 460 ::
 461
 462     # systemctl start ceph-radosgw@rgw.`hostname -s`
 463     # systemctl enable ceph-radosgw@rgw.`hostname -s`
 464
 465 Check Synchronization Status
 466 ----------------------------
 467
 468 Once the secondary zone is up and running, check the synchronization
 469 status. Synchronization copies users and buckets created in the master
 470 zone to the secondary zone.
 471
 472 ::
 473
 474     # radosgw-admin sync status
 475
 476 The output will provide the status of synchronization operations. For
 477 example:
 478
 479 ::
 480
 481     realm f3239bc5-e1a8-4206-a81d-e1576480804d (earth)
 482         zonegroup c50dbb7e-d9ce-47cc-a8bb-97d9b399d388 (us)
 483              zone 4c453b70-4a16-4ce8-8185-1893b05d346e (us-west)
 484     metadata sync syncing
 485                   full sync: 0/64 shards
 486                   metadata is caught up with master
 487                   incremental sync: 64/64 shards
 488         data sync source: 1ee9da3e-114d-4ae3-a8a4-056e8a17f532 (us-east)
 489                           syncing
 490                           full sync: 0/128 shards
 491                           incremental sync: 128/128 shards
 492                           data is caught up with source
 493
 494 .. note:: Secondary zones accept bucket operations; however, secondary zones
 495           redirect bucket operations to the master zone and then synchronize
 496           with the master zone to receive the result of the bucket operations.
 497           If the master zone is down, bucket operations executed on the
 498           secondary zone will fail, but object operations should succeed.
 499
 500
 501 Maintenance
 502 ===========
 503
 504 Checking the Sync Status
 505 ------------------------
 506
 507 Information about the replication status of a zone can be queried with::
 508
 509     $ radosgw-admin sync status
 510             realm b3bc1c37-9c44-4b89-a03b-04c269bea5da (earth)
 511         zonegroup f54f9b22-b4b6-4a0e-9211-fa6ac1693f49 (us)
 512              zone adce11c9-b8ed-4a90-8bc5-3fc029ff0816 (us-2)
 513             metadata sync syncing
 514                   full sync: 0/64 shards
 515                   incremental sync: 64/64 shards
 516                   metadata is behind on 1 shards
 517                   oldest incremental change not applied: 2017-03-22 10:20:00.0.881361s
 518         data sync source: 341c2d81-4574-4d08-ab0f-5a2a7b168028 (us-1)
 519                           syncing
 520                           full sync: 0/128 shards
 521                           incremental sync: 128/128 shards
 522                           data is caught up with source
 523                   source: 3b5d1a3f-3f27-4e4a-8f34-6072d4bb1275 (us-3)
 524                           syncing
 525                           full sync: 0/128 shards
 526                           incremental sync: 128/128 shards
 527                           data is caught up with source
 528
 529 Changing the Metadata Master Zone
 530 ---------------------------------
 531
 532 .. important:: Care must be taken when changing which zone is the metadata
 533                master. If a zone has not finished syncing metadata from the current master
 534                zone, it will be unable to serve any remaining entries when promoted to
 535                master and those changes will be lost. For this reason, waiting for a
 536                zone's ``radosgw-admin sync status`` to catch up on metadata sync before
 537                promoting it to master is recommended.
 538
 539                Similarly, if changes to metadata are being processed by the current master
 540                zone while another zone is being promoted to master, those changes are
 541                likely to be lost. To avoid this, shutting down any ``radosgw`` instances
 542                on the previous master zone is recommended. After promoting another zone,
 543                its new period can be fetched with ``radosgw-admin period pull`` and the
 544                gateway(s) can be restarted.
 545
 546 To promote a zone (for example, zone ``us-2`` in zonegroup ``us``) to metadata
 547 master, run the following commands on that zone::
 548
 549   $ radosgw-admin zone modify --rgw-zone=us-2 --master
 550   $ radosgw-admin zonegroup modify --rgw-zonegroup=us --master
 551   $ radosgw-admin period update --commit
 552
 553 This will generate a new period, and the radosgw instance(s) in zone ``us-2``
 554 will send this period to other zones.
 555
 556 Failover and Disaster Recovery
 557 ==============================
 558
 559 If the master zone should fail, failover to the secondary zone for
 560 disaster recovery.
 561
 562 1. Make the secondary zone the master and default zone. For example:
 563
 564    ::
 565
 566        # radosgw-admin zone modify --rgw-zone={zone-name} --master --default
 567
 568    By default, Ceph Object Gateway will run in an active-active
 569    configuration. If the cluster was configured to run in an
 570    active-passive configuration, the secondary zone is a read-only zone.
 571    Remove the ``--read-only`` status to allow the zone to receive write
 572    operations. For example:
 573
 574    ::
 575
 576        # radosgw-admin zone modify --rgw-zone={zone-name} --master --default \
 577                                    --read-only=false
 578
 579 2. Update the period to make the changes take effect.
 580
 581    ::
 582
 583        # radosgw-admin period update --commit
 584
 585 3. Finally, restart the Ceph Object Gateway.
 586
 587    ::
 588
 589        # systemctl restart ceph-radosgw@rgw.`hostname -s`
 590
 591 If the former master zone recovers, revert the operation.
 592
 593 1. From the recovered zone, pull the latest realm configuration
 594    from the current master zone.
 595
 596    ::
 597
 598        # radosgw-admin realm pull --url={url-to-master-zone-gateway} \
 599                                   --access-key={access-key} --secret={secret}
 600
 601 2. Make the recovered zone the master and default zone.
 602
 603    ::
 604
 605        # radosgw-admin zone modify --rgw-zone={zone-name} --master --default
 606
 607 3. Update the period to make the changes take effect.
 608
 609    ::
 610
 611        # radosgw-admin period update --commit
 612
 613 4. Then, restart the Ceph Object Gateway in the recovered zone.
 614
 615    ::
 616
 617        # systemctl restart ceph-radosgw@rgw.`hostname -s`
 618
 619 5. If the secondary zone needs to be a read-only configuration, update
 620    the secondary zone.
 621
 622    ::
 623
 624        # radosgw-admin zone modify --rgw-zone={zone-name} --read-only
 625
 626 6. Update the period to make the changes take effect.
 627
 628    ::
 629
 630        # radosgw-admin period update --commit
 631
 632 7. Finally, restart the Ceph Object Gateway in the secondary zone.
 633
 634    ::
 635
 636        # systemctl restart ceph-radosgw@rgw.`hostname -s`
 637
 638 Migrating a Single Site System to Multi-Site
 639 ============================================
 640
 641 To migrate from a single site system with a ``default`` zone group and
 642 zone to a multi site system, use the following steps:
 643
 644 1. Create a realm. Replace ``<name>`` with the realm name.
 645
 646    ::
 647
 648        # radosgw-admin realm create --rgw-realm=<name> --default
 649
 650 2. Rename the default zone and zonegroup. Replace ``<name>`` with the
 651    zonegroup or zone name.
 652
 653    ::
 654
 655        # radosgw-admin zonegroup rename --rgw-zonegroup default --zonegroup-new-name=<name>
 656        # radosgw-admin zone rename --rgw-zone default --zone-new-name us-east-1 --rgw-zonegroup=<name>
 657
 658 3. Configure the master zonegroup. Replace ``<name>`` with the realm or
 659    zonegroup name. Replace ``<fqdn>`` with the fully qualified domain
 660    name(s) in the zonegroup.
 661
 662    ::
 663
 664        # radosgw-admin zonegroup modify --rgw-realm=<name> --rgw-zonegroup=<name> --endpoints http://<fqdn>:80 --master --default
 665
 666 4. Configure the master zone. Replace ``<name>`` with the realm,
 667    zonegroup or zone name. Replace ``<fqdn>`` with the fully qualified
 668    domain name(s) in the zonegroup.
 669
 670    ::
 671
 672        # radosgw-admin zone modify --rgw-realm=<name> --rgw-zonegroup=<name> \
 673                                    --rgw-zone=<name> --endpoints http://<fqdn>:80 \
 674                                    --access-key=<access-key> --secret=<secret-key> \
 675                                    --master --default
 676
 677 5. Create a system user. Replace ``<user-id>`` with the username.
 678    Replace ``<display-name>`` with a display name. It may contain
 679    spaces.
 680
 681    ::
 682
 683        # radosgw-admin user create --uid=<user-id> --display-name="<display-name>"\
 684                                    --access-key=<access-key> --secret=<secret-key> --system
 685
 686 6. Commit the updated configuration.
 687
 688    ::
 689
 690        # radosgw-admin period update --commit
 691
 692 7. Finally, restart the Ceph Object Gateway.
 693
 694    ::
 695
 696        # systemctl restart ceph-radosgw@rgw.`hostname -s`
 697
 698 After completing this procedure, proceed to `Configure a Secondary
 699 Zone <#configure-secondary-zones>`__ to create a secondary zone
 700 in the master zone group.
 701
 702
 703 Multi-Site Configuration Reference
 704 ==================================
 705
 706 The following sections provide additional details and command-line
 707 usage for realms, periods, zone groups and zones.
 708
 709 Realms
 710 ------
 711
 712 A realm represents a globally unique namespace consisting of one or more
 713 zonegroups containing one or more zones, and zones containing buckets,
 714 which in turn contain objects. A realm enables the Ceph Object Gateway
 715 to support multiple namespaces and their configuration on the same
 716 hardware.
 717
 718 A realm contains the notion of periods. Each period represents the state
 719 of the zone group and zone configuration in time. Each time you make a
 720 change to a zonegroup or zone, update the period and commit it.
 721
 722 By default, the Ceph Object Gateway does not create a realm
 723 for backward compatibility with Infernalis and earlier releases.
 724 However, as a best practice, we recommend creating realms for new
 725 clusters.
 726
 727 Create a Realm
 728 ~~~~~~~~~~~~~~
 729
 730 To create a realm, execute ``realm create`` and specify the realm name.
 731 If the realm is the default, specify ``--default``.
 732
 733 ::
 734
 735     # radosgw-admin realm create --rgw-realm={realm-name} [--default]
 736
 737 For example:
 738
 739 ::
 740
 741     # radosgw-admin realm create --rgw-realm=movies --default
 742
 743 By specifying ``--default``, the realm will be called implicitly with
 744 each ``radosgw-admin`` call unless ``--rgw-realm`` and the realm name
 745 are explicitly provided.
 746
 747 Make a Realm the Default
 748 ~~~~~~~~~~~~~~~~~~~~~~~~
 749
 750 One realm in the list of realms should be the default realm. There may
 751 be only one default realm. If there is only one realm and it wasn’t
 752 specified as the default realm when it was created, make it the default
 753 realm. Alternatively, to change which realm is the default, execute:
 754
 755 ::
 756
 757     # radosgw-admin realm default --rgw-realm=movies
 758
 759 .. note:: When the realm is default, the command line assumes
 760    ``--rgw-realm=<realm-name>`` as an argument.
 761
 762 Delete a Realm
 763 ~~~~~~~~~~~~~~
 764
 765 To delete a realm, execute ``realm delete`` and specify the realm name.
 766
 767 ::
 768
 769     # radosgw-admin realm delete --rgw-realm={realm-name}
 770
 771 For example:
 772
 773 ::
 774
 775     # radosgw-admin realm delete --rgw-realm=movies
 776
 777 Get a Realm
 778 ~~~~~~~~~~~
 779
 780 To get a realm, execute ``realm get`` and specify the realm name.
 781
 782 ::
 783
 784     #radosgw-admin realm get --rgw-realm=<name>
 785
 786 For example:
 787
 788 ::
 789
 790     # radosgw-admin realm get --rgw-realm=movies [> filename.json]
 791
 792 The CLI will echo a JSON object with the realm properties.
 793
 794 ::
 795
 796     {
 797         "id": "0a68d52e-a19c-4e8e-b012-a8f831cb3ebc",
 798         "name": "movies",
 799         "current_period": "b0c5bbef-4337-4edd-8184-5aeab2ec413b",
 800         "epoch": 1
 801     }
 802
 803 Use ``>`` and an output file name to output the JSON object to a file.
 804
 805 Set a Realm
 806 ~~~~~~~~~~~
 807
 808 To set a realm, execute ``realm set``, specify the realm name, and
 809 ``--infile=`` with an input file name.
 810
 811 ::
 812
 813     #radosgw-admin realm set --rgw-realm=<name> --infile=<infilename>
 814
 815 For example:
 816
 817 ::
 818
 819     # radosgw-admin realm set --rgw-realm=movies --infile=filename.json
 820
 821 List Realms
 822 ~~~~~~~~~~~
 823
 824 To list realms, execute ``realm list``.
 825
 826 ::
 827
 828     # radosgw-admin realm list
 829
 830 List Realm Periods
 831 ~~~~~~~~~~~~~~~~~~
 832
 833 To list realm periods, execute ``realm list-periods``.
 834
 835 ::
 836
 837     # radosgw-admin realm list-periods
 838
 839 Pull a Realm
 840 ~~~~~~~~~~~~
 841
 842 To pull a realm from the node containing the master zone group and
 843 master zone to a node containing a secondary zone group or zone, execute
 844 ``realm pull`` on the node that will receive the realm configuration.
 845
 846 ::
 847
 848     # radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key} --secret={secret}
 849
 850 Rename a Realm
 851 ~~~~~~~~~~~~~~
 852
 853 A realm is not part of the period. Consequently, renaming the realm is
 854 only applied locally, and will not get pulled with ``realm pull``. When
 855 renaming a realm with multiple zones, run the command on each zone. To
 856 rename a realm, execute the following:
 857
 858 ::
 859
 860     # radosgw-admin realm rename --rgw-realm=<current-name> --realm-new-name=<new-realm-name>
 861
 862 .. note:: DO NOT use ``realm set`` to change the ``name`` parameter. That
 863           changes the internal name only. Specifying ``--rgw-realm`` would
 864           still use the old realm name.
 865
 866 Zone Groups
 867 -----------
 868
 869 The Ceph Object Gateway supports multi-site deployments and a global
 870 namespace by using the notion of zone groups. Formerly called a region
 871 in Infernalis, a zone group defines the geographic location of one or more Ceph
 872 Object Gateway instances within one or more zones.
 873
 874 Configuring zone groups differs from typical configuration procedures,
 875 because not all of the settings end up in a Ceph configuration file. You
 876 can list zone groups, get a zone group configuration, and set a zone
 877 group configuration.
 878
 879 Create a Zone Group
 880 ~~~~~~~~~~~~~~~~~~~
 881
 882 Creating a zone group consists of specifying the zone group name.
 883 Creating a zone assumes it will live in the default realm unless
 884 ``--rgw-realm=<realm-name>`` is specified. If the zonegroup is the
 885 default zonegroup, specify the ``--default`` flag. If the zonegroup is
 886 the master zonegroup, specify the ``--master`` flag. For example:
 887
 888 ::
 889
 890     # radosgw-admin zonegroup create --rgw-zonegroup=<name> [--rgw-realm=<name>][--master] [--default]
 891
 892
 893 .. note:: Use ``zonegroup modify --rgw-zonegroup=<zonegroup-name>`` to modify
 894           an existing zone group’s settings.
 895
 896 Make a Zone Group the Default
 897 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 898
 899 One zonegroup in the list of zonegroups should be the default zonegroup.
 900 There may be only one default zonegroup. If there is only one zonegroup
 901 and it wasn’t specified as the default zonegroup when it was created,
 902 make it the default zonegroup. Alternatively, to change which zonegroup
 903 is the default, execute:
 904
 905 ::
 906
 907     # radosgw-admin zonegroup default --rgw-zonegroup=comedy
 908
 909 .. note:: When the zonegroup is default, the command line assumes
 910           ``--rgw-zonegroup=<zonegroup-name>`` as an argument.
 911
 912 Then, update the period:
 913
 914 ::
 915
 916     # radosgw-admin period update --commit
 917
 918 Add a Zone to a Zone Group
 919 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 920
 921 To add a zone to a zonegroup, execute the following:
 922
 923 ::
 924
 925     # radosgw-admin zonegroup add --rgw-zonegroup=<name> --rgw-zone=<name>
 926
 927 Then, update the period:
 928
 929 ::
 930
 931     # radosgw-admin period update --commit
 932
 933 Remove a Zone from a Zone Group
 934 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 935
 936 To remove a zone from a zonegroup, execute the following:
 937
 938 ::
 939
 940     # radosgw-admin zonegroup remove --rgw-zonegroup=<name> --rgw-zone=<name>
 941
 942 Then, update the period:
 943
 944 ::
 945
 946     # radosgw-admin period update --commit
 947
 948 Rename a Zone Group
 949 ~~~~~~~~~~~~~~~~~~~
 950
 951 To rename a zonegroup, execute the following:
 952
 953 ::
 954
 955     # radosgw-admin zonegroup rename --rgw-zonegroup=<name> --zonegroup-new-name=<name>
 956
 957 Then, update the period:
 958
 959 ::
 960
 961     # radosgw-admin period update --commit
 962
 963 Delete a Zone Group
 964 ~~~~~~~~~~~~~~~~~~~
 965
 966 To delete a zonegroup, execute the following:
 967
 968 ::
 969
 970     # radosgw-admin zonegroup delete --rgw-zonegroup=<name>
 971
 972 Then, update the period:
 973
 974 ::
 975
 976     # radosgw-admin period update --commit
 977
 978 List Zone Groups
 979 ~~~~~~~~~~~~~~~~
 980
 981 A Ceph cluster contains a list of zone groups. To list the zone groups,
 982 execute:
 983
 984 ::
 985
 986     # radosgw-admin zonegroup list
 987
 988 The ``radosgw-admin`` returns a JSON formatted list of zone groups.
 989
 990 ::
 991
 992     {
 993         "default_info": "90b28698-e7c3-462c-a42d-4aa780d24eda",
 994         "zonegroups": [
 995             "us"
 996         ]
 997     }
 998
 999 Get a Zone Group Map
1000 ~~~~~~~~~~~~~~~~~~~~
1001
1002 To list the details of each zone group, execute:
1003
1004 ::
1005
1006     # radosgw-admin zonegroup-map get
1007
1008 .. note:: If you receive a ``failed to read zonegroup map`` error, run
1009           ``radosgw-admin zonegroup-map update`` as ``root`` first.
1010
1011 Get a Zone Group
1012 ~~~~~~~~~~~~~~~~
1013
1014 To view the configuration of a zone group, execute:
1015
1016 ::
1017
1018     radosgw-admin zonegroup get [--rgw-zonegroup=<zonegroup>]
1019
1020 The zone group configuration looks like this:
1021
1022 ::
1023
1024     {
1025         "id": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1026         "name": "us",
1027         "api_name": "us",
1028         "is_master": "true",
1029         "endpoints": [
1030             "http:\/\/rgw1:80"
1031         ],
1032         "hostnames": [],
1033         "hostnames_s3website": [],
1034         "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e",
1035         "zones": [
1036             {
1037                 "id": "9248cab2-afe7-43d8-a661-a40bf316665e",
1038                 "name": "us-east",
1039                 "endpoints": [
1040                     "http:\/\/rgw1"
1041                 ],
1042                 "log_meta": "true",
1043                 "log_data": "true",
1044                 "bucket_index_max_shards": 0,
1045                 "read_only": "false"
1046             },
1047             {
1048                 "id": "d1024e59-7d28-49d1-8222-af101965a939",
1049                 "name": "us-west",
1050                 "endpoints": [
1051                     "http:\/\/rgw2:80"
1052                 ],
1053                 "log_meta": "false",
1054                 "log_data": "true",
1055                 "bucket_index_max_shards": 0,
1056                 "read_only": "false"
1057             }
1058         ],
1059         "placement_targets": [
1060             {
1061                 "name": "default-placement",
1062                 "tags": []
1063             }
1064         ],
1065         "default_placement": "default-placement",
1066         "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe"
1067     }
1068
1069 Set a Zone Group
1070 ~~~~~~~~~~~~~~~~
1071
1072 Defining a zone group consists of creating a JSON object, specifying at
1073 least the required settings:
1074
1075 1. ``name``: The name of the zone group. Required.
1076
1077 2. ``api_name``: The API name for the zone group. Optional.
1078
1079 3. ``is_master``: Determines if the zone group is the master zone group.
1080    Required. **note:** You can only have one master zone group.
1081
1082 4. ``endpoints``: A list of all the endpoints in the zone group. For
1083    example, you may use multiple domain names to refer to the same zone
1084    group. Remember to escape the forward slashes (``\/``). You may also
1085    specify a port (``fqdn:port``) for each endpoint. Optional.
1086
1087 5. ``hostnames``: A list of all the hostnames in the zone group. For
1088    example, you may use multiple domain names to refer to the same zone
1089    group. Optional. The ``rgw dns name`` setting will automatically be
1090    included in this list. You should restart the gateway daemon(s) after
1091    changing this setting.
1092
1093 6. ``master_zone``: The master zone for the zone group. Optional. Uses
1094    the default zone if not specified. **note:** You can only have one
1095    master zone per zone group.
1096
1097 7. ``zones``: A list of all zones within the zone group. Each zone has a
1098    name (required), a list of endpoints (optional), and whether or not
1099    the gateway will log metadata and data operations (false by default).
1100
1101 8. ``placement_targets``: A list of placement targets (optional). Each
1102    placement target contains a name (required) for the placement target
1103    and a list of tags (optional) so that only users with the tag can use
1104    the placement target (i.e., the user’s ``placement_tags`` field in
1105    the user info).
1106
1107 9. ``default_placement``: The default placement target for the object
1108    index and object data. Set to ``default-placement`` by default. You
1109    may also set a per-user default placement in the user info for each
1110    user.
1111
1112 To set a zone group, create a JSON object consisting of the required
1113 fields, save the object to a file (e.g., ``zonegroup.json``); then,
1114 execute the following command:
1115
1116 ::
1117
1118     # radosgw-admin zonegroup set --infile zonegroup.json
1119
1120 Where ``zonegroup.json`` is the JSON file you created.
1121
1122 .. important:: The ``default`` zone group ``is_master`` setting is ``true`` by
1123                default. If you create a new zone group and want to make it the
1124                master zone group, you must either set the ``default`` zone group
1125                ``is_master`` setting to ``false``, or delete the ``default`` zone
1126                group.
1127
1128 Finally, update the period:
1129
1130 ::
1131
1132     # radosgw-admin period update --commit
1133
1134 Set a Zone Group Map
1135 ~~~~~~~~~~~~~~~~~~~~
1136
1137 Setting a zone group map consists of creating a JSON object consisting
1138 of one or more zone groups, and setting the ``master_zonegroup`` for the
1139 cluster. Each zone group in the zone group map consists of a key/value
1140 pair, where the ``key`` setting is equivalent to the ``name`` setting
1141 for an individual zone group configuration, and the ``val`` is a JSON
1142 object consisting of an individual zone group configuration.
1143
1144 You may only have one zone group with ``is_master`` equal to ``true``,
1145 and it must be specified as the ``master_zonegroup`` at the end of the
1146 zone group map. The following JSON object is an example of a default
1147 zone group map.
1148
1149 ::
1150
1151     {
1152         "zonegroups": [
1153             {
1154                 "key": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1155                 "val": {
1156                     "id": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1157                     "name": "us",
1158                     "api_name": "us",
1159                     "is_master": "true",
1160                     "endpoints": [
1161                         "http:\/\/rgw1:80"
1162                     ],
1163                     "hostnames": [],
1164                     "hostnames_s3website": [],
1165                     "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e",
1166                     "zones": [
1167                         {
1168                             "id": "9248cab2-afe7-43d8-a661-a40bf316665e",
1169                             "name": "us-east",
1170                             "endpoints": [
1171                                 "http:\/\/rgw1"
1172                             ],
1173                             "log_meta": "true",
1174                             "log_data": "true",
1175                             "bucket_index_max_shards": 0,
1176                             "read_only": "false"
1177                         },
1178                         {
1179                             "id": "d1024e59-7d28-49d1-8222-af101965a939",
1180                             "name": "us-west",
1181                             "endpoints": [
1182                                 "http:\/\/rgw2:80"
1183                             ],
1184                             "log_meta": "false",
1185                             "log_data": "true",
1186                             "bucket_index_max_shards": 0,
1187                             "read_only": "false"
1188                         }
1189                     ],
1190                     "placement_targets": [
1191                         {
1192                             "name": "default-placement",
1193                             "tags": []
1194                         }
1195                     ],
1196                     "default_placement": "default-placement",
1197                     "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe"
1198                 }
1199             }
1200         ],
1201         "master_zonegroup": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1202         "bucket_quota": {
1203             "enabled": false,
1204             "max_size_kb": -1,
1205             "max_objects": -1
1206         },
1207         "user_quota": {
1208             "enabled": false,
1209             "max_size_kb": -1,
1210             "max_objects": -1
1211         }
1212     }
1213
1214 To set a zone group map, execute the following:
1215
1216 ::
1217
1218     # radosgw-admin zonegroup-map set --infile zonegroupmap.json
1219
1220 Where ``zonegroupmap.json`` is the JSON file you created. Ensure that
1221 you have zones created for the ones specified in the zone group map.
1222 Finally, update the period.
1223
1224 ::
1225
1226     # radosgw-admin period update --commit
1227
1228 Zones
1229 -----
1230
1231 Ceph Object Gateway supports the notion of zones. A zone defines a
1232 logical group consisting of one or more Ceph Object Gateway instances.
1233
1234 Configuring zones differs from typical configuration procedures, because
1235 not all of the settings end up in a Ceph configuration file. You can
1236 list zones, get a zone configuration and set a zone configuration.
1237
1238 Create a Zone
1239 ~~~~~~~~~~~~~
1240
1241 To create a zone, specify a zone name. If it is a master zone, specify
1242 the ``--master`` option. Only one zone in a zone group may be a master
1243 zone. To add the zone to a zonegroup, specify the ``--rgw-zonegroup``
1244 option with the zonegroup name.
1245
1246 ::
1247
1248     # radosgw-admin zone create --rgw-zone=<name> \
1249                     [--zonegroup=<zonegroup-name]\
1250                     [--endpoints=<endpoint>[,<endpoint>] \
1251                     [--master] [--default] \
1252                     --access-key $SYSTEM_ACCESS_KEY --secret $SYSTEM_SECRET_KEY
1253
1254 Then, update the period:
1255
1256 ::
1257
1258     # radosgw-admin period update --commit
1259
1260 Delete a Zone
1261 ~~~~~~~~~~~~~
1262
1263 To delete zone, first remove it from the zonegroup.
1264
1265 ::
1266
1267     # radosgw-admin zonegroup remove --zonegroup=<name>\
1268                                      --zone=<name>
1269
1270 Then, update the period:
1271
1272 ::
1273
1274     # radosgw-admin period update --commit
1275
1276 Next, delete the zone. Execute the following:
1277
1278 ::
1279
1280     # radosgw-admin zone rm --rgw-zone<name>
1281
1282 Finally, update the period:
1283
1284 ::
1285
1286     # radosgw-admin period update --commit
1287
1288 .. important:: Do not delete a zone without removing it from a zone group first.
1289                Otherwise, updating the period will fail.
1290
1291 If the pools for the deleted zone will not be used anywhere else,
1292 consider deleting the pools. Replace ``<del-zone>`` in the example below
1293 with the deleted zone’s name.
1294
1295 .. important:: Only delete the pools with prepended zone names. Deleting the root
1296                pool, such as, ``.rgw.root`` will remove all of the system’s
1297                configuration.
1298
1299 .. important:: Once the pools are deleted, all of the data within them are deleted
1300                in an unrecoverable manner. Only delete the pools if the pool
1301                contents are no longer needed.
1302
1303 ::
1304
1305     # ceph osd pool rm <del-zone>.rgw.control <del-zone>.rgw.control --yes-i-really-really-mean-it
1306     # ceph osd pool rm <del-zone>.rgw.data.root <del-zone>.rgw.data.root --yes-i-really-really-mean-it
1307     # ceph osd pool rm <del-zone>.rgw.gc <del-zone>.rgw.gc --yes-i-really-really-mean-it
1308     # ceph osd pool rm <del-zone>.rgw.log <del-zone>.rgw.log --yes-i-really-really-mean-it
1309     # ceph osd pool rm <del-zone>.rgw.users.uid <del-zone>.rgw.users.uid --yes-i-really-really-mean-it
1310
1311 Modify a Zone
1312 ~~~~~~~~~~~~~
1313
1314 To modify a zone, specify the zone name and the parameters you wish to
1315 modify.
1316
1317 ::
1318
1319     # radosgw-admin zone modify [options]
1320
1321 Where ``[options]``:
1322
1323 - ``--access-key=<key>``
1324 - ``--secret/--secret-key=<key>``
1325 - ``--master``
1326 - ``--default``
1327 - ``--endpoints=<list>``
1328
1329 Then, update the period:
1330
1331 ::
1332
1333     # radosgw-admin period update --commit
1334
1335 List Zones
1336 ~~~~~~~~~~
1337
1338 As ``root``, to list the zones in a cluster, execute:
1339
1340 ::
1341
1342     # radosgw-admin zone list
1343
1344 Get a Zone
1345 ~~~~~~~~~~
1346
1347 As ``root``, to get the configuration of a zone, execute:
1348
1349 ::
1350
1351     # radosgw-admin zone get [--rgw-zone=<zone>]
1352
1353 The ``default`` zone looks like this:
1354
1355 ::
1356
1357     { "domain_root": ".rgw",
1358       "control_pool": ".rgw.control",
1359       "gc_pool": ".rgw.gc",
1360       "log_pool": ".log",
1361       "intent_log_pool": ".intent-log",
1362       "usage_log_pool": ".usage",
1363       "user_keys_pool": ".users",
1364       "user_email_pool": ".users.email",
1365       "user_swift_pool": ".users.swift",
1366       "user_uid_pool": ".users.uid",
1367       "system_key": { "access_key": "", "secret_key": ""},
1368       "placement_pools": [
1369           {  "key": "default-placement",
1370              "val": { "index_pool": ".rgw.buckets.index",
1371                       "data_pool": ".rgw.buckets"}
1372           }
1373         ]
1374       }
1375
1376 Set a Zone
1377 ~~~~~~~~~~
1378
1379 Configuring a zone involves specifying a series of Ceph Object Gateway
1380 pools. For consistency, we recommend using a pool prefix that is the
1381 same as the zone name. See
1382 `Pools <http://docs.ceph.com/docs/master/rados/operations/pools/#pools>`__
1383 for details of configuring pools.
1384
1385 To set a zone, create a JSON object consisting of the pools, save the
1386 object to a file (e.g., ``zone.json``); then, execute the following
1387 command, replacing ``{zone-name}`` with the name of the zone:
1388
1389 ::
1390
1391     # radosgw-admin zone set --rgw-zone={zone-name} --infile zone.json
1392
1393 Where ``zone.json`` is the JSON file you created.
1394
1395 Then, as ``root``, update the period:
1396
1397 ::
1398
1399     # radosgw-admin period update --commit
1400
1401 Rename a Zone
1402 ~~~~~~~~~~~~~
1403
1404 To rename a zone, specify the zone name and the new zone name.
1405
1406 ::
1407
1408     # radosgw-admin zone rename --rgw-zone=<name> --zone-new-name=<name>
1409
1410 Then, update the period:
1411
1412 ::
1413
1414     # radosgw-admin period update --commit
1415
1416 Zone Group and Zone Settings
1417 ----------------------------
1418
1419 When configuring a default zone group and zone, the pool name includes
1420 the zone name. For example:
1421
1422 -  ``default.rgw.control``
1423
1424 To change the defaults, include the following settings in your Ceph
1425 configuration file under each ``[client.radosgw.{instance-name}]``
1426 instance.
1427
1428 +-------------------------------------+-----------------------------------+---------+-----------------------+
1429 | Name                                | Description                       | Type    | Default               |
1430 +=====================================+===================================+=========+=======================+
1431 | ``rgw_zone``                        | The name of the zone for the      | String  | None                  |
1432 |                                     | gateway instance.                 |         |                       |
1433 +-------------------------------------+-----------------------------------+---------+-----------------------+
1434 | ``rgw_zonegroup``                   | The name of the zone group for    | String  | None                  |
1435 |                                     | the gateway instance.             |         |                       |
1436 +-------------------------------------+-----------------------------------+---------+-----------------------+
1437 | ``rgw_zonegroup_root_pool``         | The root pool for the zone group. | String  | ``.rgw.root``         |
1438 +-------------------------------------+-----------------------------------+---------+-----------------------+
1439 | ``rgw_zone_root_pool``              | The root pool for the zone.       | String  | ``.rgw.root``         |
1440 +-------------------------------------+-----------------------------------+---------+-----------------------+
1441 | ``rgw_default_zone_group_info_oid`` | The OID for storing the default   | String  | ``default.zonegroup`` |
1442 |                                     | zone group. We do not recommend   |         |                       |
1443 |                                     | changing this setting.            |         |                       |
1444 +-------------------------------------+-----------------------------------+---------+-----------------------+
1445
1446
1447 .. _`Pools`: ../pools