7 .. versionadded:: Jewel
9 A single zone configuration typically consists of one zone group containing one
10 zone and one or more `ceph-radosgw` instances where you may load-balance gateway
11 client requests between the instances. In a single zone configuration, typically
12 multiple gateway instances point to a single Ceph storage cluster. However, Kraken
13 supports several multi-site configuration options for the Ceph Object Gateway:
15 - **Multi-zone:** A more advanced configuration consists of one zone group and
16 multiple zones, each zone with one or more `ceph-radosgw` instances. Each zone
17 is backed by its own Ceph Storage Cluster. Multiple zones in a zone group
18 provides disaster recovery for the zone group should one of the zones experience
19 a significant failure. In Kraken, each zone is active and may receive write
20 operations. In addition to disaster recovery, multiple active zones may also
21 serve as a foundation for content delivery networks.
23 - **Multi-zone-group:** Formerly called 'regions', Ceph Object Gateway can also
24 support multiple zone groups, each zone group with one or more zones. Objects
25 stored to zones in one zone group within the same realm as another zone
26 group will share a global object namespace, ensuring unique object IDs across
27 zone groups and zones.
29 - **Multiple Realms:** In Kraken, the Ceph Object Gateway supports the notion
30 of realms, which can be a single zone group or multiple zone groups and
31 a globally unique namespace for the realm. Multiple realms provide the ability
32 to support numerous configurations and namespaces.
34 Replicating object data between zones within a zone group looks something
37 .. image:: ../images/zone-sync2.png
40 For additional details on setting up a cluster, see `Ceph Object Gateway for
41 Production <https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/3/html/ceph_object_gateway_for_production/index/>`__.
43 Functional Changes from Infernalis
44 ==================================
46 In Kraken, you can configure each Ceph Object Gateway to
47 work in an active-active zone configuration, allowing for writes to
50 The multi-site configuration is stored within a container called a
51 "realm." The realm stores zone groups, zones, and a time "period" with
52 multiple epochs for tracking changes to the configuration. In Kraken,
53 the ``ceph-radosgw`` daemons handle the synchronization,
54 eliminating the need for a separate synchronization agent. Additionally,
55 the new approach to synchronization allows the Ceph Object Gateway to
56 operate with an "active-active" configuration instead of
59 Requirements and Assumptions
60 ============================
62 A multi-site configuration requires at least two Ceph storage clusters,
63 preferably given a distinct cluster name. At least two Ceph object
64 gateway instances, one for each Ceph storage cluster.
66 This guide assumes at least two Ceph storage clusters are in geographically
67 separate locations; however, the configuration can work on the same
68 site. This guide also assumes two Ceph object gateway servers named
69 ``rgw1`` and ``rgw2``.
71 .. important:: Running a single Ceph storage cluster is NOT recommended unless you have
72 low latency WAN connections.
74 A multi-site configuration requires a master zone group and a master
75 zone. Additionally, each zone group requires a master zone. Zone groups
76 may have one or more secondary or non-master zones.
78 In this guide, the ``rgw1`` host will serve as the master zone of the
79 master zone group; and, the ``rgw2`` host will serve as the secondary zone
80 of the master zone group.
82 See `Pools`_ for instructions on creating and tuning pools for Ceph
85 See `Sync Policy Config`_ for instructions on defining fine grained bucket sync
88 .. _master-zone-label:
90 Configuring a Master Zone
91 =========================
93 All gateways in a multi-site configuration will retrieve their
94 configuration from a ``ceph-radosgw`` daemon on a host within the master
95 zone group and master zone. To configure your gateways in a multi-site
96 configuration, choose a ``ceph-radosgw`` instance to configure the
97 master zone group and master zone.
102 A realm contains the multi-site configuration of zone groups and zones
103 and also serves to enforce a globally unique namespace within the realm.
105 Create a new realm for the multi-site configuration by opening a command
106 line interface on a host identified to serve in the master zone group
107 and zone. Then, execute the following:
111 # radosgw-admin realm create --rgw-realm={realm-name} [--default]
117 # radosgw-admin realm create --rgw-realm=movies --default
119 If the cluster will have a single realm, specify the ``--default`` flag.
120 If ``--default`` is specified, ``radosgw-admin`` will use this realm by
121 default. If ``--default`` is not specified, adding zone-groups and zones
122 requires specifying either the ``--rgw-realm`` flag or the
123 ``--realm-id`` flag to identify the realm when adding zone groups and
126 After creating the realm, ``radosgw-admin`` will echo back the realm
127 configuration. For example:
132 "id": "0956b174-fe14-4f97-8b50-bb7ec5e1cf62",
134 "current_period": "1950b710-3e63-4c41-a19e-46a715000980",
138 .. note:: Ceph generates a unique ID for the realm, which allows the renaming
139 of a realm if the need arises.
141 Create a Master Zone Group
142 --------------------------
144 A realm must have at least one zone group, which will serve as the
145 master zone group for the realm.
147 Create a new master zone group for the multi-site configuration by
148 opening a command line interface on a host identified to serve in the
149 master zone group and zone. Then, execute the following:
153 # radosgw-admin zonegroup create --rgw-zonegroup={name} --endpoints={url} [--rgw-realm={realm-name}|--realm-id={realm-id}] --master --default
159 # radosgw-admin zonegroup create --rgw-zonegroup=us --endpoints=http://rgw1:80 --rgw-realm=movies --master --default
161 If the realm will only have a single zone group, specify the
162 ``--default`` flag. If ``--default`` is specified, ``radosgw-admin``
163 will use this zone group by default when adding new zones. If
164 ``--default`` is not specified, adding zones will require either the
165 ``--rgw-zonegroup`` flag or the ``--zonegroup-id`` flag to identify the
166 zone group when adding or modifying zones.
168 After creating the master zone group, ``radosgw-admin`` will echo back
169 the zone group configuration. For example:
174 "id": "f1a233f5-c354-4107-b36c-df66126475a6",
182 "hostnames_s3website": [],
185 "placement_targets": [],
186 "default_placement": "",
187 "realm_id": "0956b174-fe14-4f97-8b50-bb7ec5e1cf62"
193 .. important:: Zones must be created on a Ceph Object Gateway node that will be
196 Create a new master zone for the multi-site configuration by opening a
197 command line interface on a host identified to serve in the master zone
198 group and zone. Then, execute the following:
202 # radosgw-admin zone create --rgw-zonegroup={zone-group-name} \
203 --rgw-zone={zone-name} \
205 --endpoints={http://fqdn}[,{http://fqdn}]
212 # radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-east \
214 --endpoints={http://fqdn}[,{http://fqdn}]
217 .. note:: The ``--access-key`` and ``--secret`` aren’t specified. These
218 settings will be added to the zone once the user is created in the
221 .. important:: The following steps assume a multi-site configuration using newly
222 installed systems that aren’t storing data yet. DO NOT DELETE the
223 ``default`` zone and its pools if you are already using it to store
224 data, or the data will be deleted and unrecoverable.
226 Delete Default Zone Group and Zone
227 ----------------------------------
229 Delete the ``default`` zone if it exists. Make sure to remove it from
230 the default zone group first.
234 # radosgw-admin zonegroup remove --rgw-zonegroup=default --rgw-zone=default
235 # radosgw-admin period update --commit
236 # radosgw-admin zone rm --rgw-zone=default
237 # radosgw-admin period update --commit
238 # radosgw-admin zonegroup delete --rgw-zonegroup=default
239 # radosgw-admin period update --commit
241 Finally, delete the ``default`` pools in your Ceph storage cluster if
244 .. important:: The following step assumes a multi-site configuration using newly
245 installed systems that aren’t currently storing data. DO NOT DELETE
246 the ``default`` zone group if you are already using it to store
251 # ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it
252 # ceph osd pool rm default.rgw.data.root default.rgw.data.root --yes-i-really-really-mean-it
253 # ceph osd pool rm default.rgw.gc default.rgw.gc --yes-i-really-really-mean-it
254 # ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it
255 # ceph osd pool rm default.rgw.users.uid default.rgw.users.uid --yes-i-really-really-mean-it
260 The ``ceph-radosgw`` daemons must authenticate before pulling realm and
261 period information. In the master zone, create a system user to
262 facilitate authentication between daemons.
266 # radosgw-admin user create --uid="{user-name}" --display-name="{Display Name}" --system
272 # radosgw-admin user create --uid="synchronization-user" --display-name="Synchronization User" --system
274 Make a note of the ``access_key`` and ``secret_key``, as the secondary
275 zones will require them to authenticate with the master zone.
277 Finally, add the system user to the master zone.
281 # radosgw-admin zone modify --rgw-zone=us-east --access-key={access-key} --secret={secret}
282 # radosgw-admin period update --commit
287 After updating the master zone configuration, update the period.
291 # radosgw-admin period update --commit
293 .. note:: Updating the period changes the epoch, and ensures that other zones
294 will receive the updated configuration.
296 Update the Ceph Configuration File
297 ----------------------------------
299 Update the Ceph configuration file on master zone hosts by adding the
300 ``rgw_zone`` configuration option and the name of the master zone to the
305 [client.rgw.{instance-name}]
315 rgw frontends = "civetweb port=80"
321 On the object gateway host, start and enable the Ceph Object Gateway
326 # systemctl start ceph-radosgw@rgw.`hostname -s`
327 # systemctl enable ceph-radosgw@rgw.`hostname -s`
329 .. _secondary-zone-label:
331 Configure Secondary Zones
332 =========================
334 Zones within a zone group replicate all data to ensure that each zone
335 has the same data. When creating the secondary zone, execute all of the
336 following operations on a host identified to serve the secondary zone.
338 .. note:: To add a third zone, follow the same procedures as for adding the
339 secondary zone. Use different zone name.
341 .. important:: You must execute metadata operations, such as user creation, on a
342 host within the master zone. The master zone and the secondary zone
343 can receive bucket operations, but the secondary zone redirects
344 bucket operations to the master zone. If the master zone is down,
345 bucket operations will fail.
350 Using the URL path, access key and secret of the master zone in the
351 master zone group, pull the realm configuration to the host. To pull a
352 non-default realm, specify the realm using the ``--rgw-realm`` or
353 ``--realm-id`` configuration options.
357 # radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key} --secret={secret}
359 .. note:: Pulling the realm also retrieves the remote's current period
360 configuration, and makes it the current period on this host as well.
362 If this realm is the default realm or the only realm, make the realm the
367 # radosgw-admin realm default --rgw-realm={realm-name}
369 Create a Secondary Zone
370 -----------------------
372 .. important:: Zones must be created on a Ceph Object Gateway node that will be
375 Create a secondary zone for the multi-site configuration by opening a
376 command line interface on a host identified to serve the secondary zone.
377 Specify the zone group ID, the new zone name and an endpoint for the
378 zone. **DO NOT** use the ``--master`` or ``--default`` flags. In Kraken,
379 all zones run in an active-active configuration by
380 default; that is, a gateway client may write data to any zone and the
381 zone will replicate the data to all other zones within the zone group.
382 If the secondary zone should not accept write operations, specify the
383 ``--read-only`` flag to create an active-passive configuration between
384 the master zone and the secondary zone. Additionally, provide the
385 ``access_key`` and ``secret_key`` of the generated system user stored in
386 the master zone of the master zone group. Execute the following:
390 # radosgw-admin zone create --rgw-zonegroup={zone-group-name}\
391 --rgw-zone={zone-name} --endpoints={url} \
392 --access-key={system-key} --secret={secret}\
393 --endpoints=http://{fqdn}:80 \
400 # radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-west \
401 --access-key={system-key} --secret={secret} \
402 --endpoints=http://rgw2:80
404 .. important:: The following steps assume a multi-site configuration using newly
405 installed systems that aren’t storing data. **DO NOT DELETE** the
406 ``default`` zone and its pools if you are already using it to store
407 data, or the data will be lost and unrecoverable.
409 Delete the default zone if needed.
413 # radosgw-admin zone rm --rgw-zone=default
415 Finally, delete the default pools in your Ceph storage cluster if
420 # ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it
421 # ceph osd pool rm default.rgw.data.root default.rgw.data.root --yes-i-really-really-mean-it
422 # ceph osd pool rm default.rgw.gc default.rgw.gc --yes-i-really-really-mean-it
423 # ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it
424 # ceph osd pool rm default.rgw.users.uid default.rgw.users.uid --yes-i-really-really-mean-it
426 Update the Ceph Configuration File
427 ----------------------------------
429 Update the Ceph configuration file on the secondary zone hosts by adding
430 the ``rgw_zone`` configuration option and the name of the secondary zone
431 to the instance entry.
435 [client.rgw.{instance-name}]
445 rgw frontends = "civetweb port=80"
451 After updating the master zone configuration, update the period.
455 # radosgw-admin period update --commit
457 .. note:: Updating the period changes the epoch, and ensures that other zones
458 will receive the updated configuration.
463 On the object gateway host, start and enable the Ceph Object Gateway
468 # systemctl start ceph-radosgw@rgw.`hostname -s`
469 # systemctl enable ceph-radosgw@rgw.`hostname -s`
471 Check Synchronization Status
472 ----------------------------
474 Once the secondary zone is up and running, check the synchronization
475 status. Synchronization copies users and buckets created in the master
476 zone to the secondary zone.
480 # radosgw-admin sync status
482 The output will provide the status of synchronization operations. For
487 realm f3239bc5-e1a8-4206-a81d-e1576480804d (earth)
488 zonegroup c50dbb7e-d9ce-47cc-a8bb-97d9b399d388 (us)
489 zone 4c453b70-4a16-4ce8-8185-1893b05d346e (us-west)
490 metadata sync syncing
491 full sync: 0/64 shards
492 metadata is caught up with master
493 incremental sync: 64/64 shards
494 data sync source: 1ee9da3e-114d-4ae3-a8a4-056e8a17f532 (us-east)
496 full sync: 0/128 shards
497 incremental sync: 128/128 shards
498 data is caught up with source
500 .. note:: Secondary zones accept bucket operations; however, secondary zones
501 redirect bucket operations to the master zone and then synchronize
502 with the master zone to receive the result of the bucket operations.
503 If the master zone is down, bucket operations executed on the
504 secondary zone will fail, but object operations should succeed.
506 Verification of an Object
507 -------------------------
509 By default, the objects are not verified again after the synchronization of an
510 object was successful. To enable that, you can set :confval:`rgw_sync_obj_etag_verify`
511 to ``true``. After enabling the optional objects that will be synchronized
512 going forward, an additional MD5 checksum will verify that it is computed on
513 the source and the destination. This is to ensure the integrity of the objects
514 fetched from a remote server over HTTP including multisite sync. This option
515 can decrease the performance of your RGW as more computation is needed.
521 Checking the Sync Status
522 ------------------------
524 Information about the replication status of a zone can be queried with::
526 $ radosgw-admin sync status
527 realm b3bc1c37-9c44-4b89-a03b-04c269bea5da (earth)
528 zonegroup f54f9b22-b4b6-4a0e-9211-fa6ac1693f49 (us)
529 zone adce11c9-b8ed-4a90-8bc5-3fc029ff0816 (us-2)
530 metadata sync syncing
531 full sync: 0/64 shards
532 incremental sync: 64/64 shards
533 metadata is behind on 1 shards
534 oldest incremental change not applied: 2017-03-22 10:20:00.0.881361s
535 data sync source: 341c2d81-4574-4d08-ab0f-5a2a7b168028 (us-1)
537 full sync: 0/128 shards
538 incremental sync: 128/128 shards
539 data is caught up with source
540 source: 3b5d1a3f-3f27-4e4a-8f34-6072d4bb1275 (us-3)
542 full sync: 0/128 shards
543 incremental sync: 128/128 shards
544 data is caught up with source
546 The output can differ depending on the sync status. The shards are described
547 as two different types during sync:
549 - **Behind shards** are shards that need a full data sync and shards needing
550 an incremental data sync because they are not up-to-date.
552 - **Recovery shards** are shards that encountered an error during sync and marked
553 for retry. The error mostly occurs on minor issues like acquiring a lock on
554 a bucket. This will typically resolve itself.
559 For multi-site only, you can check out the metadata log (``mdlog``),
560 the bucket index log (``bilog``) and the data log (``datalog``).
561 You can list them and also trim them which is not needed in most cases as
562 :confval:`rgw_sync_log_trim_interval` is set to 20 minutes as default. If it isn't manually
563 set to 0, you shouldn't have to trim it at any time as it could cause side effects otherwise.
565 Changing the Metadata Master Zone
566 ---------------------------------
568 .. important:: Care must be taken when changing which zone is the metadata
569 master. If a zone has not finished syncing metadata from the current master
570 zone, it will be unable to serve any remaining entries when promoted to
571 master and those changes will be lost. For this reason, waiting for a
572 zone's ``radosgw-admin sync status`` to catch up on metadata sync before
573 promoting it to master is recommended.
575 Similarly, if changes to metadata are being processed by the current master
576 zone while another zone is being promoted to master, those changes are
577 likely to be lost. To avoid this, shutting down any ``radosgw`` instances
578 on the previous master zone is recommended. After promoting another zone,
579 its new period can be fetched with ``radosgw-admin period pull`` and the
580 gateway(s) can be restarted.
582 To promote a zone (for example, zone ``us-2`` in zonegroup ``us``) to metadata
583 master, run the following commands on that zone::
585 $ radosgw-admin zone modify --rgw-zone=us-2 --master
586 $ radosgw-admin zonegroup modify --rgw-zonegroup=us --master
587 $ radosgw-admin period update --commit
589 This will generate a new period, and the radosgw instance(s) in zone ``us-2``
590 will send this period to other zones.
592 Failover and Disaster Recovery
593 ==============================
595 If the master zone should fail, failover to the secondary zone for
598 1. Make the secondary zone the master and default zone. For example:
602 # radosgw-admin zone modify --rgw-zone={zone-name} --master --default
604 By default, Ceph Object Gateway will run in an active-active
605 configuration. If the cluster was configured to run in an
606 active-passive configuration, the secondary zone is a read-only zone.
607 Remove the ``--read-only`` status to allow the zone to receive write
608 operations. For example:
612 # radosgw-admin zone modify --rgw-zone={zone-name} --master --default \
615 2. Update the period to make the changes take effect.
619 # radosgw-admin period update --commit
621 3. Finally, restart the Ceph Object Gateway.
625 # systemctl restart ceph-radosgw@rgw.`hostname -s`
627 If the former master zone recovers, revert the operation.
629 1. From the recovered zone, pull the latest realm configuration
630 from the current master zone.
634 # radosgw-admin realm pull --url={url-to-master-zone-gateway} \
635 --access-key={access-key} --secret={secret}
637 2. Make the recovered zone the master and default zone.
641 # radosgw-admin zone modify --rgw-zone={zone-name} --master --default
643 3. Update the period to make the changes take effect.
647 # radosgw-admin period update --commit
649 4. Then, restart the Ceph Object Gateway in the recovered zone.
653 # systemctl restart ceph-radosgw@rgw.`hostname -s`
655 5. If the secondary zone needs to be a read-only configuration, update
660 # radosgw-admin zone modify --rgw-zone={zone-name} --read-only
662 6. Update the period to make the changes take effect.
666 # radosgw-admin period update --commit
668 7. Finally, restart the Ceph Object Gateway in the secondary zone.
672 # systemctl restart ceph-radosgw@rgw.`hostname -s`
674 .. _rgw-multisite-migrate-from-single-site:
676 Migrating a Single Site System to Multi-Site
677 ============================================
679 To migrate from a single site system with a ``default`` zone group and
680 zone to a multi site system, use the following steps:
682 1. Create a realm. Replace ``<name>`` with the realm name.
686 # radosgw-admin realm create --rgw-realm=<name> --default
688 2. Rename the default zone and zonegroup. Replace ``<name>`` with the
689 zonegroup or zone name.
693 # radosgw-admin zonegroup rename --rgw-zonegroup default --zonegroup-new-name=<name>
694 # radosgw-admin zone rename --rgw-zone default --zone-new-name us-east-1 --rgw-zonegroup=<name>
696 3. Configure the master zonegroup. Replace ``<name>`` with the realm or
697 zonegroup name. Replace ``<fqdn>`` with the fully qualified domain
698 name(s) in the zonegroup.
702 # radosgw-admin zonegroup modify --rgw-realm=<name> --rgw-zonegroup=<name> --endpoints http://<fqdn>:80 --master --default
704 4. Configure the master zone. Replace ``<name>`` with the realm,
705 zonegroup or zone name. Replace ``<fqdn>`` with the fully qualified
706 domain name(s) in the zonegroup.
710 # radosgw-admin zone modify --rgw-realm=<name> --rgw-zonegroup=<name> \
711 --rgw-zone=<name> --endpoints http://<fqdn>:80 \
712 --access-key=<access-key> --secret=<secret-key> \
715 5. Create a system user. Replace ``<user-id>`` with the username.
716 Replace ``<display-name>`` with a display name. It may contain
721 # radosgw-admin user create --uid=<user-id> --display-name="<display-name>"\
722 --access-key=<access-key> --secret=<secret-key> --system
724 6. Commit the updated configuration.
728 # radosgw-admin period update --commit
730 7. Finally, restart the Ceph Object Gateway.
734 # systemctl restart ceph-radosgw@rgw.`hostname -s`
736 After completing this procedure, proceed to `Configure a Secondary
737 Zone <#configure-secondary-zones>`__ to create a secondary zone
738 in the master zone group.
741 Multi-Site Configuration Reference
742 ==================================
744 The following sections provide additional details and command-line
745 usage for realms, periods, zone groups and zones.
747 For more details on every available configuration option, please check out
748 ``src/common/options/rgw.yaml.in`` or go to the more comfortable :ref:`mgr-dashboard`
749 configuration page (found under `Cluster`) where you can view and set all
750 options easily. On the page, set the level to ``advanced`` and search for RGW,
751 to see all basic and advanced configuration options with a short description.
752 Expand the details of an option to reveal a longer description.
757 A realm represents a globally unique namespace consisting of one or more
758 zonegroups containing one or more zones, and zones containing buckets,
759 which in turn contain objects. A realm enables the Ceph Object Gateway
760 to support multiple namespaces and their configuration on the same
763 A realm contains the notion of periods. Each period represents the state
764 of the zone group and zone configuration in time. Each time you make a
765 change to a zonegroup or zone, update the period and commit it.
767 By default, the Ceph Object Gateway does not create a realm
768 for backward compatibility with Infernalis and earlier releases.
769 However, as a best practice, we recommend creating realms for new
775 To create a realm, execute ``realm create`` and specify the realm name.
776 If the realm is the default, specify ``--default``.
780 # radosgw-admin realm create --rgw-realm={realm-name} [--default]
786 # radosgw-admin realm create --rgw-realm=movies --default
788 By specifying ``--default``, the realm will be called implicitly with
789 each ``radosgw-admin`` call unless ``--rgw-realm`` and the realm name
790 are explicitly provided.
792 Make a Realm the Default
793 ~~~~~~~~~~~~~~~~~~~~~~~~
795 One realm in the list of realms should be the default realm. There may
796 be only one default realm. If there is only one realm and it wasn’t
797 specified as the default realm when it was created, make it the default
798 realm. Alternatively, to change which realm is the default, execute:
802 # radosgw-admin realm default --rgw-realm=movies
804 .. note:: When the realm is default, the command line assumes
805 ``--rgw-realm=<realm-name>`` as an argument.
810 To delete a realm, execute ``realm rm`` and specify the realm name.
814 # radosgw-admin realm rm --rgw-realm={realm-name}
820 # radosgw-admin realm rm --rgw-realm=movies
825 To get a realm, execute ``realm get`` and specify the realm name.
829 #radosgw-admin realm get --rgw-realm=<name>
835 # radosgw-admin realm get --rgw-realm=movies [> filename.json]
837 The CLI will echo a JSON object with the realm properties.
842 "id": "0a68d52e-a19c-4e8e-b012-a8f831cb3ebc",
844 "current_period": "b0c5bbef-4337-4edd-8184-5aeab2ec413b",
848 Use ``>`` and an output file name to output the JSON object to a file.
853 To set a realm, execute ``realm set``, specify the realm name, and
854 ``--infile=`` with an input file name.
858 #radosgw-admin realm set --rgw-realm=<name> --infile=<infilename>
864 # radosgw-admin realm set --rgw-realm=movies --infile=filename.json
869 To list realms, execute ``realm list``.
873 # radosgw-admin realm list
878 To list realm periods, execute ``realm list-periods``.
882 # radosgw-admin realm list-periods
887 To pull a realm from the node containing the master zone group and
888 master zone to a node containing a secondary zone group or zone, execute
889 ``realm pull`` on the node that will receive the realm configuration.
893 # radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key} --secret={secret}
898 A realm is not part of the period. Consequently, renaming the realm is
899 only applied locally, and will not get pulled with ``realm pull``. When
900 renaming a realm with multiple zones, run the command on each zone. To
901 rename a realm, execute the following:
905 # radosgw-admin realm rename --rgw-realm=<current-name> --realm-new-name=<new-realm-name>
907 .. note:: DO NOT use ``realm set`` to change the ``name`` parameter. That
908 changes the internal name only. Specifying ``--rgw-realm`` would
909 still use the old realm name.
914 The Ceph Object Gateway supports multi-site deployments and a global
915 namespace by using the notion of zone groups. Formerly called a region
916 in Infernalis, a zone group defines the geographic location of one or more Ceph
917 Object Gateway instances within one or more zones.
919 Configuring zone groups differs from typical configuration procedures,
920 because not all of the settings end up in a Ceph configuration file. You
921 can list zone groups, get a zone group configuration, and set a zone
927 Creating a zone group consists of specifying the zone group name.
928 Creating a zone assumes it will live in the default realm unless
929 ``--rgw-realm=<realm-name>`` is specified. If the zonegroup is the
930 default zonegroup, specify the ``--default`` flag. If the zonegroup is
931 the master zonegroup, specify the ``--master`` flag. For example:
935 # radosgw-admin zonegroup create --rgw-zonegroup=<name> [--rgw-realm=<name>][--master] [--default]
938 .. note:: Use ``zonegroup modify --rgw-zonegroup=<zonegroup-name>`` to modify
939 an existing zone group’s settings.
941 Make a Zone Group the Default
942 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
944 One zonegroup in the list of zonegroups should be the default zonegroup.
945 There may be only one default zonegroup. If there is only one zonegroup
946 and it wasn’t specified as the default zonegroup when it was created,
947 make it the default zonegroup. Alternatively, to change which zonegroup
948 is the default, execute:
952 # radosgw-admin zonegroup default --rgw-zonegroup=comedy
954 .. note:: When the zonegroup is default, the command line assumes
955 ``--rgw-zonegroup=<zonegroup-name>`` as an argument.
957 Then, update the period:
961 # radosgw-admin period update --commit
963 Add a Zone to a Zone Group
964 ~~~~~~~~~~~~~~~~~~~~~~~~~~
966 To add a zone to a zonegroup, execute the following:
970 # radosgw-admin zonegroup add --rgw-zonegroup=<name> --rgw-zone=<name>
972 Then, update the period:
976 # radosgw-admin period update --commit
978 Remove a Zone from a Zone Group
979 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
981 To remove a zone from a zonegroup, execute the following:
985 # radosgw-admin zonegroup remove --rgw-zonegroup=<name> --rgw-zone=<name>
987 Then, update the period:
991 # radosgw-admin period update --commit
996 To rename a zonegroup, execute the following:
1000 # radosgw-admin zonegroup rename --rgw-zonegroup=<name> --zonegroup-new-name=<name>
1002 Then, update the period:
1006 # radosgw-admin period update --commit
1011 To delete a zonegroup, execute the following:
1015 # radosgw-admin zonegroup delete --rgw-zonegroup=<name>
1017 Then, update the period:
1021 # radosgw-admin period update --commit
1026 A Ceph cluster contains a list of zone groups. To list the zone groups,
1031 # radosgw-admin zonegroup list
1033 The ``radosgw-admin`` returns a JSON formatted list of zone groups.
1038 "default_info": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1044 Get a Zone Group Map
1045 ~~~~~~~~~~~~~~~~~~~~
1047 To list the details of each zone group, execute:
1051 # radosgw-admin zonegroup-map get
1053 .. note:: If you receive a ``failed to read zonegroup map`` error, run
1054 ``radosgw-admin zonegroup-map update`` as ``root`` first.
1059 To view the configuration of a zone group, execute:
1063 radosgw-admin zonegroup get [--rgw-zonegroup=<zonegroup>]
1065 The zone group configuration looks like this:
1070 "id": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1073 "is_master": "true",
1078 "hostnames_s3website": [],
1079 "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e",
1082 "id": "9248cab2-afe7-43d8-a661-a40bf316665e",
1089 "bucket_index_max_shards": 0,
1090 "read_only": "false"
1093 "id": "d1024e59-7d28-49d1-8222-af101965a939",
1098 "log_meta": "false",
1100 "bucket_index_max_shards": 0,
1101 "read_only": "false"
1104 "placement_targets": [
1106 "name": "default-placement",
1110 "default_placement": "default-placement",
1111 "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe"
1117 Defining a zone group consists of creating a JSON object, specifying at
1118 least the required settings:
1120 1. ``name``: The name of the zone group. Required.
1122 2. ``api_name``: The API name for the zone group. Optional.
1124 3. ``is_master``: Determines if the zone group is the master zone group.
1125 Required. **note:** You can only have one master zone group.
1127 4. ``endpoints``: A list of all the endpoints in the zone group. For
1128 example, you may use multiple domain names to refer to the same zone
1129 group. Remember to escape the forward slashes (``\/``). You may also
1130 specify a port (``fqdn:port``) for each endpoint. Optional.
1132 5. ``hostnames``: A list of all the hostnames in the zone group. For
1133 example, you may use multiple domain names to refer to the same zone
1134 group. Optional. The ``rgw dns name`` setting will automatically be
1135 included in this list. You should restart the gateway daemon(s) after
1136 changing this setting.
1138 6. ``master_zone``: The master zone for the zone group. Optional. Uses
1139 the default zone if not specified. **note:** You can only have one
1140 master zone per zone group.
1142 7. ``zones``: A list of all zones within the zone group. Each zone has a
1143 name (required), a list of endpoints (optional), and whether or not
1144 the gateway will log metadata and data operations (false by default).
1146 8. ``placement_targets``: A list of placement targets (optional). Each
1147 placement target contains a name (required) for the placement target
1148 and a list of tags (optional) so that only users with the tag can use
1149 the placement target (i.e., the user’s ``placement_tags`` field in
1152 9. ``default_placement``: The default placement target for the object
1153 index and object data. Set to ``default-placement`` by default. You
1154 may also set a per-user default placement in the user info for each
1157 To set a zone group, create a JSON object consisting of the required
1158 fields, save the object to a file (e.g., ``zonegroup.json``); then,
1159 execute the following command:
1163 # radosgw-admin zonegroup set --infile zonegroup.json
1165 Where ``zonegroup.json`` is the JSON file you created.
1167 .. important:: The ``default`` zone group ``is_master`` setting is ``true`` by
1168 default. If you create a new zone group and want to make it the
1169 master zone group, you must either set the ``default`` zone group
1170 ``is_master`` setting to ``false``, or delete the ``default`` zone
1173 Finally, update the period:
1177 # radosgw-admin period update --commit
1179 Set a Zone Group Map
1180 ~~~~~~~~~~~~~~~~~~~~
1182 Setting a zone group map consists of creating a JSON object consisting
1183 of one or more zone groups, and setting the ``master_zonegroup`` for the
1184 cluster. Each zone group in the zone group map consists of a key/value
1185 pair, where the ``key`` setting is equivalent to the ``name`` setting
1186 for an individual zone group configuration, and the ``val`` is a JSON
1187 object consisting of an individual zone group configuration.
1189 You may only have one zone group with ``is_master`` equal to ``true``,
1190 and it must be specified as the ``master_zonegroup`` at the end of the
1191 zone group map. The following JSON object is an example of a default
1199 "key": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1201 "id": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1204 "is_master": "true",
1209 "hostnames_s3website": [],
1210 "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e",
1213 "id": "9248cab2-afe7-43d8-a661-a40bf316665e",
1220 "bucket_index_max_shards": 0,
1221 "read_only": "false"
1224 "id": "d1024e59-7d28-49d1-8222-af101965a939",
1229 "log_meta": "false",
1231 "bucket_index_max_shards": 0,
1232 "read_only": "false"
1235 "placement_targets": [
1237 "name": "default-placement",
1241 "default_placement": "default-placement",
1242 "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe"
1246 "master_zonegroup": "90b28698-e7c3-462c-a42d-4aa780d24eda",
1259 To set a zone group map, execute the following:
1263 # radosgw-admin zonegroup-map set --infile zonegroupmap.json
1265 Where ``zonegroupmap.json`` is the JSON file you created. Ensure that
1266 you have zones created for the ones specified in the zone group map.
1267 Finally, update the period.
1271 # radosgw-admin period update --commit
1276 Ceph Object Gateway supports the notion of zones. A zone defines a
1277 logical group consisting of one or more Ceph Object Gateway instances.
1279 Configuring zones differs from typical configuration procedures, because
1280 not all of the settings end up in a Ceph configuration file. You can
1281 list zones, get a zone configuration and set a zone configuration.
1286 To create a zone, specify a zone name. If it is a master zone, specify
1287 the ``--master`` option. Only one zone in a zone group may be a master
1288 zone. To add the zone to a zonegroup, specify the ``--rgw-zonegroup``
1289 option with the zonegroup name.
1293 # radosgw-admin zone create --rgw-zone=<name> \
1294 [--zonegroup=<zonegroup-name]\
1295 [--endpoints=<endpoint>[,<endpoint>] \
1296 [--master] [--default] \
1297 --access-key $SYSTEM_ACCESS_KEY --secret $SYSTEM_SECRET_KEY
1299 Then, update the period:
1303 # radosgw-admin period update --commit
1308 To delete zone, first remove it from the zonegroup.
1312 # radosgw-admin zonegroup remove --zonegroup=<name>\
1315 Then, update the period:
1319 # radosgw-admin period update --commit
1321 Next, delete the zone. Execute the following:
1325 # radosgw-admin zone rm --rgw-zone<name>
1327 Finally, update the period:
1331 # radosgw-admin period update --commit
1333 .. important:: Do not delete a zone without removing it from a zone group first.
1334 Otherwise, updating the period will fail.
1336 If the pools for the deleted zone will not be used anywhere else,
1337 consider deleting the pools. Replace ``<del-zone>`` in the example below
1338 with the deleted zone’s name.
1340 .. important:: Only delete the pools with prepended zone names. Deleting the root
1341 pool, such as, ``.rgw.root`` will remove all of the system’s
1344 .. important:: Once the pools are deleted, all of the data within them are deleted
1345 in an unrecoverable manner. Only delete the pools if the pool
1346 contents are no longer needed.
1350 # ceph osd pool rm <del-zone>.rgw.control <del-zone>.rgw.control --yes-i-really-really-mean-it
1351 # ceph osd pool rm <del-zone>.rgw.data.root <del-zone>.rgw.data.root --yes-i-really-really-mean-it
1352 # ceph osd pool rm <del-zone>.rgw.gc <del-zone>.rgw.gc --yes-i-really-really-mean-it
1353 # ceph osd pool rm <del-zone>.rgw.log <del-zone>.rgw.log --yes-i-really-really-mean-it
1354 # ceph osd pool rm <del-zone>.rgw.users.uid <del-zone>.rgw.users.uid --yes-i-really-really-mean-it
1359 To modify a zone, specify the zone name and the parameters you wish to
1364 # radosgw-admin zone modify [options]
1366 Where ``[options]``:
1368 - ``--access-key=<key>``
1369 - ``--secret/--secret-key=<key>``
1372 - ``--endpoints=<list>``
1374 Then, update the period:
1378 # radosgw-admin period update --commit
1383 As ``root``, to list the zones in a cluster, execute:
1387 # radosgw-admin zone list
1392 As ``root``, to get the configuration of a zone, execute:
1396 # radosgw-admin zone get [--rgw-zone=<zone>]
1398 The ``default`` zone looks like this:
1402 { "domain_root": ".rgw",
1403 "control_pool": ".rgw.control",
1404 "gc_pool": ".rgw.gc",
1406 "intent_log_pool": ".intent-log",
1407 "usage_log_pool": ".usage",
1408 "user_keys_pool": ".users",
1409 "user_email_pool": ".users.email",
1410 "user_swift_pool": ".users.swift",
1411 "user_uid_pool": ".users.uid",
1412 "system_key": { "access_key": "", "secret_key": ""},
1413 "placement_pools": [
1414 { "key": "default-placement",
1415 "val": { "index_pool": ".rgw.buckets.index",
1416 "data_pool": ".rgw.buckets"}
1424 Configuring a zone involves specifying a series of Ceph Object Gateway
1425 pools. For consistency, we recommend using a pool prefix that is the
1426 same as the zone name. See
1427 `Pools <http://docs.ceph.com/en/latest/rados/operations/pools/#pools>`__
1428 for details of configuring pools.
1430 To set a zone, create a JSON object consisting of the pools, save the
1431 object to a file (e.g., ``zone.json``); then, execute the following
1432 command, replacing ``{zone-name}`` with the name of the zone:
1436 # radosgw-admin zone set --rgw-zone={zone-name} --infile zone.json
1438 Where ``zone.json`` is the JSON file you created.
1440 Then, as ``root``, update the period:
1444 # radosgw-admin period update --commit
1449 To rename a zone, specify the zone name and the new zone name.
1453 # radosgw-admin zone rename --rgw-zone=<name> --zone-new-name=<name>
1455 Then, update the period:
1459 # radosgw-admin period update --commit
1461 Zone Group and Zone Settings
1462 ----------------------------
1464 When configuring a default zone group and zone, the pool name includes
1465 the zone name. For example:
1467 - ``default.rgw.control``
1469 To change the defaults, include the following settings in your Ceph
1470 configuration file under each ``[client.radosgw.{instance-name}]``
1473 +-------------------------------------+-----------------------------------+---------+-----------------------+
1474 | Name | Description | Type | Default |
1475 +=====================================+===================================+=========+=======================+
1476 | ``rgw_zone`` | The name of the zone for the | String | None |
1477 | | gateway instance. | | |
1478 +-------------------------------------+-----------------------------------+---------+-----------------------+
1479 | ``rgw_zonegroup`` | The name of the zone group for | String | None |
1480 | | the gateway instance. | | |
1481 +-------------------------------------+-----------------------------------+---------+-----------------------+
1482 | ``rgw_zonegroup_root_pool`` | The root pool for the zone group. | String | ``.rgw.root`` |
1483 +-------------------------------------+-----------------------------------+---------+-----------------------+
1484 | ``rgw_zone_root_pool`` | The root pool for the zone. | String | ``.rgw.root`` |
1485 +-------------------------------------+-----------------------------------+---------+-----------------------+
1486 | ``rgw_default_zone_group_info_oid`` | The OID for storing the default | String | ``default.zonegroup`` |
1487 | | zone group. We do not recommend | | |
1488 | | changing this setting. | | |
1489 +-------------------------------------+-----------------------------------+---------+-----------------------+
1492 .. _`Pools`: ../pools
1493 .. _`Sync Policy Config`: ../multisite-sync-policy