]>
Commit | Line | Data |
---|---|---|
11fdf7f2 TL |
1 | .. _multisite: |
2 | ||
7c673cae FG |
3 | ========== |
4 | Multi-Site | |
5 | ========== | |
6 | ||
7 | .. versionadded:: Jewel | |
8 | ||
9 | A single zone configuration typically consists of one zone group containing one | |
10 | zone and one or more `ceph-radosgw` instances where you may load-balance gateway | |
11 | client requests between the instances. In a single zone configuration, typically | |
12 | multiple gateway instances point to a single Ceph storage cluster. However, Kraken | |
13 | supports several multi-site configuration options for the Ceph Object Gateway: | |
14 | ||
15 | - **Multi-zone:** A more advanced configuration consists of one zone group and | |
16 | multiple zones, each zone with one or more `ceph-radosgw` instances. Each zone | |
17 | is backed by its own Ceph Storage Cluster. Multiple zones in a zone group | |
18 | provides disaster recovery for the zone group should one of the zones experience | |
19 | a significant failure. In Kraken, each zone is active and may receive write | |
20 | operations. In addition to disaster recovery, multiple active zones may also | |
21 | serve as a foundation for content delivery networks. | |
22 | ||
23 | - **Multi-zone-group:** Formerly called 'regions', Ceph Object Gateway can also | |
24 | support multiple zone groups, each zone group with one or more zones. Objects | |
25 | stored to zones in one zone group within the same realm as another zone | |
26 | group will share a global object namespace, ensuring unique object IDs across | |
27 | zone groups and zones. | |
28 | ||
29 | - **Multiple Realms:** In Kraken, the Ceph Object Gateway supports the notion | |
30 | of realms, which can be a single zone group or multiple zone groups and | |
31 | a globally unique namespace for the realm. Multiple realms provide the ability | |
32 | to support numerous configurations and namespaces. | |
33 | ||
34 | Replicating object data between zones within a zone group looks something | |
35 | like this: | |
36 | ||
37 | .. image:: ../images/zone-sync2.png | |
38 | :align: center | |
39 | ||
40 | For additional details on setting up a cluster, see `Ceph Object Gateway for | |
41 | Production <https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html/ceph_object_gateway_for_production/>`__. | |
42 | ||
43 | Functional Changes from Infernalis | |
44 | ================================== | |
45 | ||
46 | In Kraken, you can configure each Ceph Object Gateway to | |
47 | work in an active-active zone configuration, allowing for writes to | |
48 | non-master zones. | |
49 | ||
50 | The multi-site configuration is stored within a container called a | |
51 | "realm." The realm stores zone groups, zones, and a time "period" with | |
52 | multiple epochs for tracking changes to the configuration. In Kraken, | |
53 | the ``ceph-radosgw`` daemons handle the synchronization, | |
54 | eliminating the need for a separate synchronization agent. Additionally, | |
55 | the new approach to synchronization allows the Ceph Object Gateway to | |
56 | operate with an "active-active" configuration instead of | |
57 | "active-passive". | |
58 | ||
59 | Requirements and Assumptions | |
60 | ============================ | |
61 | ||
62 | A multi-site configuration requires at least two Ceph storage clusters, | |
63 | preferably given a distinct cluster name. At least two Ceph object | |
64 | gateway instances, one for each Ceph storage cluster. | |
65 | ||
11fdf7f2 | 66 | This guide assumes at least two Ceph storage clusters are in geographically |
7c673cae | 67 | separate locations; however, the configuration can work on the same |
31f18b77 | 68 | site. This guide also assumes two Ceph object gateway servers named |
7c673cae FG |
69 | ``rgw1`` and ``rgw2``. |
70 | ||
11fdf7f2 TL |
71 | .. important:: Running a single Ceph storage cluster is NOT recommended unless you have |
72 | low latency WAN connections. | |
73 | ||
7c673cae FG |
74 | A multi-site configuration requires a master zone group and a master |
75 | zone. Additionally, each zone group requires a master zone. Zone groups | |
76 | may have one or more secondary or non-master zones. | |
77 | ||
78 | In this guide, the ``rgw1`` host will serve as the master zone of the | |
79 | master zone group; and, the ``rgw2`` host will serve as the secondary zone | |
80 | of the master zone group. | |
81 | ||
31f18b77 FG |
82 | See `Pools`_ for instructions on creating and tuning pools for Ceph |
83 | Object Storage. | |
7c673cae FG |
84 | |
85 | ||
86 | Configuring a Master Zone | |
87 | ========================= | |
88 | ||
89 | All gateways in a multi-site configuration will retrieve their | |
90 | configuration from a ``ceph-radosgw`` daemon on a host within the master | |
91 | zone group and master zone. To configure your gateways in a multi-site | |
92 | configuration, choose a ``ceph-radosgw`` instance to configure the | |
93 | master zone group and master zone. | |
94 | ||
95 | Create a Realm | |
96 | -------------- | |
97 | ||
98 | A realm contains the multi-site configuration of zone groups and zones | |
99 | and also serves to enforce a globally unique namespace within the realm. | |
100 | ||
101 | Create a new realm for the multi-site configuration by opening a command | |
102 | line interface on a host identified to serve in the master zone group | |
103 | and zone. Then, execute the following: | |
104 | ||
105 | :: | |
106 | ||
107 | # radosgw-admin realm create --rgw-realm={realm-name} [--default] | |
108 | ||
109 | For example: | |
110 | ||
111 | :: | |
112 | ||
113 | # radosgw-admin realm create --rgw-realm=movies --default | |
114 | ||
115 | If the cluster will have a single realm, specify the ``--default`` flag. | |
116 | If ``--default`` is specified, ``radosgw-admin`` will use this realm by | |
117 | default. If ``--default`` is not specified, adding zone-groups and zones | |
118 | requires specifying either the ``--rgw-realm`` flag or the | |
119 | ``--realm-id`` flag to identify the realm when adding zone groups and | |
120 | zones. | |
121 | ||
122 | After creating the realm, ``radosgw-admin`` will echo back the realm | |
123 | configuration. For example: | |
124 | ||
125 | :: | |
126 | ||
127 | { | |
128 | "id": "0956b174-fe14-4f97-8b50-bb7ec5e1cf62", | |
129 | "name": "movies", | |
130 | "current_period": "1950b710-3e63-4c41-a19e-46a715000980", | |
131 | "epoch": 1 | |
132 | } | |
133 | ||
134 | .. note:: Ceph generates a unique ID for the realm, which allows the renaming | |
135 | of a realm if the need arises. | |
136 | ||
137 | Create a Master Zone Group | |
138 | -------------------------- | |
139 | ||
140 | A realm must have at least one zone group, which will serve as the | |
141 | master zone group for the realm. | |
142 | ||
143 | Create a new master zone group for the multi-site configuration by | |
144 | opening a command line interface on a host identified to serve in the | |
145 | master zone group and zone. Then, execute the following: | |
146 | ||
147 | :: | |
148 | ||
149 | # radosgw-admin zonegroup create --rgw-zonegroup={name} --endpoints={url} [--rgw-realm={realm-name}|--realm-id={realm-id}] --master --default | |
150 | ||
151 | For example: | |
152 | ||
153 | :: | |
154 | ||
155 | # radosgw-admin zonegroup create --rgw-zonegroup=us --endpoints=http://rgw1:80 --rgw-realm=movies --master --default | |
156 | ||
157 | If the realm will only have a single zone group, specify the | |
158 | ``--default`` flag. If ``--default`` is specified, ``radosgw-admin`` | |
159 | will use this zone group by default when adding new zones. If | |
160 | ``--default`` is not specified, adding zones will require either the | |
161 | ``--rgw-zonegroup`` flag or the ``--zonegroup-id`` flag to identify the | |
162 | zone group when adding or modifying zones. | |
163 | ||
164 | After creating the master zone group, ``radosgw-admin`` will echo back | |
165 | the zone group configuration. For example: | |
166 | ||
167 | :: | |
168 | ||
169 | { | |
170 | "id": "f1a233f5-c354-4107-b36c-df66126475a6", | |
171 | "name": "us", | |
172 | "api_name": "us", | |
173 | "is_master": "true", | |
174 | "endpoints": [ | |
175 | "http:\/\/rgw1:80" | |
176 | ], | |
177 | "hostnames": [], | |
178 | "hostnames_s3webzone": [], | |
179 | "master_zone": "", | |
180 | "zones": [], | |
181 | "placement_targets": [], | |
182 | "default_placement": "", | |
183 | "realm_id": "0956b174-fe14-4f97-8b50-bb7ec5e1cf62" | |
184 | } | |
185 | ||
186 | Create a Master Zone | |
187 | -------------------- | |
188 | ||
189 | .. important:: Zones must be created on a Ceph Object Gateway node that will be | |
190 | within the zone. | |
191 | ||
192 | Create a new master zone for the multi-site configuration by opening a | |
193 | command line interface on a host identified to serve in the master zone | |
194 | group and zone. Then, execute the following: | |
195 | ||
196 | :: | |
197 | ||
198 | # radosgw-admin zone create --rgw-zonegroup={zone-group-name} \ | |
199 | --rgw-zone={zone-name} \ | |
200 | --master --default \ | |
201 | --endpoints={http://fqdn}[,{http://fqdn}] | |
202 | ||
203 | ||
204 | For example: | |
205 | ||
206 | :: | |
207 | ||
208 | # radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-east \ | |
209 | --master --default \ | |
210 | --endpoints={http://fqdn}[,{http://fqdn}] | |
211 | ||
212 | ||
213 | .. note:: The ``--access-key`` and ``--secret`` aren’t specified. These | |
214 | settings will be added to the zone once the user is created in the | |
215 | next section. | |
216 | ||
217 | .. important:: The following steps assume a multi-site configuration using newly | |
218 | installed systems that aren’t storing data yet. DO NOT DELETE the | |
219 | ``default`` zone and its pools if you are already using it to store | |
220 | data, or the data will be deleted and unrecoverable. | |
221 | ||
222 | Delete Default Zone Group and Zone | |
223 | ---------------------------------- | |
224 | ||
225 | Delete the ``default`` zone if it exists. Make sure to remove it from | |
226 | the default zone group first. | |
227 | ||
228 | :: | |
229 | ||
230 | # radosgw-admin zonegroup remove --rgw-zonegroup=default --rgw-zone=default | |
231 | # radosgw-admin period update --commit | |
eafe8130 | 232 | # radosgw-admin zone rm --rgw-zone=default |
7c673cae FG |
233 | # radosgw-admin period update --commit |
234 | # radosgw-admin zonegroup delete --rgw-zonegroup=default | |
235 | # radosgw-admin period update --commit | |
236 | ||
237 | Finally, delete the ``default`` pools in your Ceph storage cluster if | |
238 | they exist. | |
239 | ||
240 | .. important:: The following step assumes a multi-site configuration using newly | |
241 | installed systems that aren’t currently storing data. DO NOT DELETE | |
242 | the ``default`` zone group if you are already using it to store | |
243 | data. | |
244 | ||
245 | :: | |
246 | ||
11fdf7f2 TL |
247 | # ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it |
248 | # ceph osd pool rm default.rgw.data.root default.rgw.data.root --yes-i-really-really-mean-it | |
249 | # ceph osd pool rm default.rgw.gc default.rgw.gc --yes-i-really-really-mean-it | |
250 | # ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it | |
251 | # ceph osd pool rm default.rgw.users.uid default.rgw.users.uid --yes-i-really-really-mean-it | |
7c673cae FG |
252 | |
253 | Create a System User | |
254 | -------------------- | |
255 | ||
256 | The ``ceph-radosgw`` daemons must authenticate before pulling realm and | |
257 | period information. In the master zone, create a system user to | |
258 | facilitate authentication between daemons. | |
259 | ||
260 | :: | |
261 | ||
262 | # radosgw-admin user create --uid="{user-name}" --display-name="{Display Name}" --system | |
263 | ||
264 | For example: | |
265 | ||
266 | :: | |
267 | ||
268 | # radosgw-admin user create --uid="synchronization-user" --display-name="Synchronization User" --system | |
269 | ||
270 | Make a note of the ``access_key`` and ``secret_key``, as the secondary | |
271 | zones will require them to authenticate with the master zone. | |
272 | ||
273 | Finally, add the system user to the master zone. | |
274 | ||
275 | :: | |
276 | ||
277 | # radosgw-admin zone modify --rgw-zone=us-east --access-key={access-key} --secret={secret} | |
278 | # radosgw-admin period update --commit | |
279 | ||
280 | Update the Period | |
281 | ----------------- | |
282 | ||
283 | After updating the master zone configuration, update the period. | |
284 | ||
285 | :: | |
286 | ||
287 | # radosgw-admin period update --commit | |
288 | ||
289 | .. note:: Updating the period changes the epoch, and ensures that other zones | |
290 | will receive the updated configuration. | |
291 | ||
292 | Update the Ceph Configuration File | |
293 | ---------------------------------- | |
294 | ||
295 | Update the Ceph configuration file on master zone hosts by adding the | |
296 | ``rgw_zone`` configuration option and the name of the master zone to the | |
297 | instance entry. | |
298 | ||
299 | :: | |
300 | ||
301 | [client.rgw.{instance-name}] | |
302 | ... | |
303 | rgw_zone={zone-name} | |
304 | ||
305 | For example: | |
306 | ||
307 | :: | |
308 | ||
309 | [client.rgw.rgw1] | |
310 | host = rgw1 | |
311 | rgw frontends = "civetweb port=80" | |
312 | rgw_zone=us-east | |
313 | ||
314 | Start the Gateway | |
315 | ----------------- | |
316 | ||
317 | On the object gateway host, start and enable the Ceph Object Gateway | |
318 | service: | |
319 | ||
320 | :: | |
321 | ||
322 | # systemctl start ceph-radosgw@rgw.`hostname -s` | |
323 | # systemctl enable ceph-radosgw@rgw.`hostname -s` | |
324 | ||
325 | Configure Secondary Zones | |
326 | ========================= | |
327 | ||
328 | Zones within a zone group replicate all data to ensure that each zone | |
329 | has the same data. When creating the secondary zone, execute all of the | |
330 | following operations on a host identified to serve the secondary zone. | |
331 | ||
332 | .. note:: To add a third zone, follow the same procedures as for adding the | |
333 | secondary zone. Use different zone name. | |
334 | ||
335 | .. important:: You must execute metadata operations, such as user creation, on a | |
336 | host within the master zone. The master zone and the secondary zone | |
337 | can receive bucket operations, but the secondary zone redirects | |
338 | bucket operations to the master zone. If the master zone is down, | |
339 | bucket operations will fail. | |
340 | ||
341 | Pull the Realm | |
342 | -------------- | |
343 | ||
344 | Using the URL path, access key and secret of the master zone in the | |
494da23a TL |
345 | master zone group, pull the realm configuration to the host. To pull a |
346 | non-default realm, specify the realm using the ``--rgw-realm`` or | |
347 | ``--realm-id`` configuration options. | |
7c673cae FG |
348 | |
349 | :: | |
350 | ||
351 | # radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key} --secret={secret} | |
352 | ||
494da23a TL |
353 | .. note:: Pulling the realm also retrieves the remote's current period |
354 | configuration, and makes it the current period on this host as well. | |
355 | ||
7c673cae FG |
356 | If this realm is the default realm or the only realm, make the realm the |
357 | default realm. | |
358 | ||
359 | :: | |
360 | ||
361 | # radosgw-admin realm default --rgw-realm={realm-name} | |
362 | ||
7c673cae FG |
363 | Create a Secondary Zone |
364 | ----------------------- | |
365 | ||
366 | .. important:: Zones must be created on a Ceph Object Gateway node that will be | |
367 | within the zone. | |
368 | ||
369 | Create a secondary zone for the multi-site configuration by opening a | |
370 | command line interface on a host identified to serve the secondary zone. | |
371 | Specify the zone group ID, the new zone name and an endpoint for the | |
372 | zone. **DO NOT** use the ``--master`` or ``--default`` flags. In Kraken, | |
373 | all zones run in an active-active configuration by | |
374 | default; that is, a gateway client may write data to any zone and the | |
375 | zone will replicate the data to all other zones within the zone group. | |
376 | If the secondary zone should not accept write operations, specify the | |
377 | ``--read-only`` flag to create an active-passive configuration between | |
378 | the master zone and the secondary zone. Additionally, provide the | |
379 | ``access_key`` and ``secret_key`` of the generated system user stored in | |
380 | the master zone of the master zone group. Execute the following: | |
381 | ||
382 | :: | |
383 | ||
384 | # radosgw-admin zone create --rgw-zonegroup={zone-group-name}\ | |
385 | --rgw-zone={zone-name} --endpoints={url} \ | |
386 | --access-key={system-key} --secret={secret}\ | |
387 | --endpoints=http://{fqdn}:80 \ | |
388 | [--read-only] | |
389 | ||
390 | For example: | |
391 | ||
392 | :: | |
393 | ||
394 | # radosgw-admin zone create --rgw-zonegroup=us --rgw-zone=us-west \ | |
395 | --access-key={system-key} --secret={secret} \ | |
396 | --endpoints=http://rgw2:80 | |
397 | ||
398 | .. important:: The following steps assume a multi-site configuration using newly | |
399 | installed systems that aren’t storing data. **DO NOT DELETE** the | |
400 | ``default`` zone and its pools if you are already using it to store | |
401 | data, or the data will be lost and unrecoverable. | |
402 | ||
403 | Delete the default zone if needed. | |
404 | ||
405 | :: | |
406 | ||
eafe8130 | 407 | # radosgw-admin zone rm --rgw-zone=default |
7c673cae FG |
408 | |
409 | Finally, delete the default pools in your Ceph storage cluster if | |
410 | needed. | |
411 | ||
412 | :: | |
413 | ||
11fdf7f2 TL |
414 | # ceph osd pool rm default.rgw.control default.rgw.control --yes-i-really-really-mean-it |
415 | # ceph osd pool rm default.rgw.data.root default.rgw.data.root --yes-i-really-really-mean-it | |
416 | # ceph osd pool rm default.rgw.gc default.rgw.gc --yes-i-really-really-mean-it | |
417 | # ceph osd pool rm default.rgw.log default.rgw.log --yes-i-really-really-mean-it | |
418 | # ceph osd pool rm default.rgw.users.uid default.rgw.users.uid --yes-i-really-really-mean-it | |
7c673cae FG |
419 | |
420 | Update the Ceph Configuration File | |
421 | ---------------------------------- | |
422 | ||
423 | Update the Ceph configuration file on the secondary zone hosts by adding | |
424 | the ``rgw_zone`` configuration option and the name of the secondary zone | |
425 | to the instance entry. | |
426 | ||
427 | :: | |
428 | ||
429 | [client.rgw.{instance-name}] | |
430 | ... | |
431 | rgw_zone={zone-name} | |
432 | ||
433 | For example: | |
434 | ||
435 | :: | |
436 | ||
437 | [client.rgw.rgw2] | |
438 | host = rgw2 | |
439 | rgw frontends = "civetweb port=80" | |
440 | rgw_zone=us-west | |
441 | ||
442 | Update the Period | |
443 | ----------------- | |
444 | ||
445 | After updating the master zone configuration, update the period. | |
446 | ||
447 | :: | |
448 | ||
449 | # radosgw-admin period update --commit | |
450 | ||
451 | .. note:: Updating the period changes the epoch, and ensures that other zones | |
452 | will receive the updated configuration. | |
453 | ||
454 | Start the Gateway | |
455 | ----------------- | |
456 | ||
457 | On the object gateway host, start and enable the Ceph Object Gateway | |
458 | service: | |
459 | ||
460 | :: | |
461 | ||
462 | # systemctl start ceph-radosgw@rgw.`hostname -s` | |
463 | # systemctl enable ceph-radosgw@rgw.`hostname -s` | |
464 | ||
465 | Check Synchronization Status | |
466 | ---------------------------- | |
467 | ||
468 | Once the secondary zone is up and running, check the synchronization | |
469 | status. Synchronization copies users and buckets created in the master | |
470 | zone to the secondary zone. | |
471 | ||
472 | :: | |
473 | ||
474 | # radosgw-admin sync status | |
475 | ||
476 | The output will provide the status of synchronization operations. For | |
477 | example: | |
478 | ||
479 | :: | |
480 | ||
481 | realm f3239bc5-e1a8-4206-a81d-e1576480804d (earth) | |
482 | zonegroup c50dbb7e-d9ce-47cc-a8bb-97d9b399d388 (us) | |
483 | zone 4c453b70-4a16-4ce8-8185-1893b05d346e (us-west) | |
484 | metadata sync syncing | |
485 | full sync: 0/64 shards | |
486 | metadata is caught up with master | |
487 | incremental sync: 64/64 shards | |
488 | data sync source: 1ee9da3e-114d-4ae3-a8a4-056e8a17f532 (us-east) | |
489 | syncing | |
490 | full sync: 0/128 shards | |
491 | incremental sync: 128/128 shards | |
492 | data is caught up with source | |
493 | ||
494 | .. note:: Secondary zones accept bucket operations; however, secondary zones | |
495 | redirect bucket operations to the master zone and then synchronize | |
496 | with the master zone to receive the result of the bucket operations. | |
497 | If the master zone is down, bucket operations executed on the | |
498 | secondary zone will fail, but object operations should succeed. | |
499 | ||
500 | ||
501 | Maintenance | |
502 | =========== | |
503 | ||
504 | Checking the Sync Status | |
505 | ------------------------ | |
506 | ||
507 | Information about the replication status of a zone can be queried with:: | |
508 | ||
509 | $ radosgw-admin sync status | |
510 | realm b3bc1c37-9c44-4b89-a03b-04c269bea5da (earth) | |
511 | zonegroup f54f9b22-b4b6-4a0e-9211-fa6ac1693f49 (us) | |
512 | zone adce11c9-b8ed-4a90-8bc5-3fc029ff0816 (us-2) | |
513 | metadata sync syncing | |
514 | full sync: 0/64 shards | |
515 | incremental sync: 64/64 shards | |
516 | metadata is behind on 1 shards | |
517 | oldest incremental change not applied: 2017-03-22 10:20:00.0.881361s | |
518 | data sync source: 341c2d81-4574-4d08-ab0f-5a2a7b168028 (us-1) | |
519 | syncing | |
520 | full sync: 0/128 shards | |
521 | incremental sync: 128/128 shards | |
522 | data is caught up with source | |
523 | source: 3b5d1a3f-3f27-4e4a-8f34-6072d4bb1275 (us-3) | |
524 | syncing | |
525 | full sync: 0/128 shards | |
526 | incremental sync: 128/128 shards | |
527 | data is caught up with source | |
528 | ||
529 | Changing the Metadata Master Zone | |
530 | --------------------------------- | |
531 | ||
532 | .. important:: Care must be taken when changing which zone is the metadata | |
533 | master. If a zone has not finished syncing metadata from the current master | |
534 | zone, it will be unable to serve any remaining entries when promoted to | |
535 | master and those changes will be lost. For this reason, waiting for a | |
536 | zone's ``radosgw-admin sync status`` to catch up on metadata sync before | |
537 | promoting it to master is recommended. | |
538 | ||
539 | Similarly, if changes to metadata are being processed by the current master | |
540 | zone while another zone is being promoted to master, those changes are | |
541 | likely to be lost. To avoid this, shutting down any ``radosgw`` instances | |
542 | on the previous master zone is recommended. After promoting another zone, | |
543 | its new period can be fetched with ``radosgw-admin period pull`` and the | |
544 | gateway(s) can be restarted. | |
545 | ||
546 | To promote a zone (for example, zone ``us-2`` in zonegroup ``us``) to metadata | |
547 | master, run the following commands on that zone:: | |
548 | ||
549 | $ radosgw-admin zone modify --rgw-zone=us-2 --master | |
550 | $ radosgw-admin zonegroup modify --rgw-zonegroup=us --master | |
551 | $ radosgw-admin period update --commit | |
552 | ||
553 | This will generate a new period, and the radosgw instance(s) in zone ``us-2`` | |
554 | will send this period to other zones. | |
555 | ||
556 | Failover and Disaster Recovery | |
557 | ============================== | |
558 | ||
559 | If the master zone should fail, failover to the secondary zone for | |
560 | disaster recovery. | |
561 | ||
562 | 1. Make the secondary zone the master and default zone. For example: | |
563 | ||
564 | :: | |
565 | ||
566 | # radosgw-admin zone modify --rgw-zone={zone-name} --master --default | |
567 | ||
568 | By default, Ceph Object Gateway will run in an active-active | |
569 | configuration. If the cluster was configured to run in an | |
570 | active-passive configuration, the secondary zone is a read-only zone. | |
571 | Remove the ``--read-only`` status to allow the zone to receive write | |
572 | operations. For example: | |
573 | ||
574 | :: | |
575 | ||
576 | # radosgw-admin zone modify --rgw-zone={zone-name} --master --default \ | |
494da23a | 577 | --read-only=false |
7c673cae FG |
578 | |
579 | 2. Update the period to make the changes take effect. | |
580 | ||
581 | :: | |
582 | ||
583 | # radosgw-admin period update --commit | |
584 | ||
585 | 3. Finally, restart the Ceph Object Gateway. | |
586 | ||
587 | :: | |
588 | ||
589 | # systemctl restart ceph-radosgw@rgw.`hostname -s` | |
590 | ||
591 | If the former master zone recovers, revert the operation. | |
592 | ||
494da23a TL |
593 | 1. From the recovered zone, pull the latest realm configuration |
594 | from the current master zone. | |
7c673cae FG |
595 | |
596 | :: | |
597 | ||
494da23a TL |
598 | # radosgw-admin realm pull --url={url-to-master-zone-gateway} \ |
599 | --access-key={access-key} --secret={secret} | |
7c673cae FG |
600 | |
601 | 2. Make the recovered zone the master and default zone. | |
602 | ||
603 | :: | |
604 | ||
605 | # radosgw-admin zone modify --rgw-zone={zone-name} --master --default | |
606 | ||
607 | 3. Update the period to make the changes take effect. | |
608 | ||
609 | :: | |
610 | ||
611 | # radosgw-admin period update --commit | |
612 | ||
613 | 4. Then, restart the Ceph Object Gateway in the recovered zone. | |
614 | ||
615 | :: | |
616 | ||
617 | # systemctl restart ceph-radosgw@rgw.`hostname -s` | |
618 | ||
619 | 5. If the secondary zone needs to be a read-only configuration, update | |
620 | the secondary zone. | |
621 | ||
622 | :: | |
623 | ||
624 | # radosgw-admin zone modify --rgw-zone={zone-name} --read-only | |
625 | ||
626 | 6. Update the period to make the changes take effect. | |
627 | ||
628 | :: | |
629 | ||
630 | # radosgw-admin period update --commit | |
631 | ||
632 | 7. Finally, restart the Ceph Object Gateway in the secondary zone. | |
633 | ||
634 | :: | |
635 | ||
636 | # systemctl restart ceph-radosgw@rgw.`hostname -s` | |
637 | ||
638 | Migrating a Single Site System to Multi-Site | |
639 | ============================================ | |
640 | ||
641 | To migrate from a single site system with a ``default`` zone group and | |
642 | zone to a multi site system, use the following steps: | |
643 | ||
644 | 1. Create a realm. Replace ``<name>`` with the realm name. | |
645 | ||
646 | :: | |
647 | ||
648 | # radosgw-admin realm create --rgw-realm=<name> --default | |
649 | ||
650 | 2. Rename the default zone and zonegroup. Replace ``<name>`` with the | |
651 | zonegroup or zone name. | |
652 | ||
653 | :: | |
654 | ||
655 | # radosgw-admin zonegroup rename --rgw-zonegroup default --zonegroup-new-name=<name> | |
656 | # radosgw-admin zone rename --rgw-zone default --zone-new-name us-east-1 --rgw-zonegroup=<name> | |
657 | ||
658 | 3. Configure the master zonegroup. Replace ``<name>`` with the realm or | |
659 | zonegroup name. Replace ``<fqdn>`` with the fully qualified domain | |
660 | name(s) in the zonegroup. | |
661 | ||
662 | :: | |
663 | ||
664 | # radosgw-admin zonegroup modify --rgw-realm=<name> --rgw-zonegroup=<name> --endpoints http://<fqdn>:80 --master --default | |
665 | ||
666 | 4. Configure the master zone. Replace ``<name>`` with the realm, | |
667 | zonegroup or zone name. Replace ``<fqdn>`` with the fully qualified | |
668 | domain name(s) in the zonegroup. | |
669 | ||
670 | :: | |
671 | ||
672 | # radosgw-admin zone modify --rgw-realm=<name> --rgw-zonegroup=<name> \ | |
673 | --rgw-zone=<name> --endpoints http://<fqdn>:80 \ | |
674 | --access-key=<access-key> --secret=<secret-key> \ | |
675 | --master --default | |
676 | ||
677 | 5. Create a system user. Replace ``<user-id>`` with the username. | |
678 | Replace ``<display-name>`` with a display name. It may contain | |
679 | spaces. | |
680 | ||
681 | :: | |
682 | ||
683 | # radosgw-admin user create --uid=<user-id> --display-name="<display-name>"\ | |
684 | --access-key=<access-key> --secret=<secret-key> --system | |
685 | ||
686 | 6. Commit the updated configuration. | |
687 | ||
688 | :: | |
689 | ||
690 | # radosgw-admin period update --commit | |
691 | ||
692 | 7. Finally, restart the Ceph Object Gateway. | |
693 | ||
694 | :: | |
695 | ||
696 | # systemctl restart ceph-radosgw@rgw.`hostname -s` | |
697 | ||
698 | After completing this procedure, proceed to `Configure a Secondary | |
699 | Zone <#configure-secondary-zones>`__ to create a secondary zone | |
700 | in the master zone group. | |
701 | ||
702 | ||
703 | Multi-Site Configuration Reference | |
704 | ================================== | |
705 | ||
706 | The following sections provide additional details and command-line | |
707 | usage for realms, periods, zone groups and zones. | |
708 | ||
709 | Realms | |
710 | ------ | |
711 | ||
712 | A realm represents a globally unique namespace consisting of one or more | |
713 | zonegroups containing one or more zones, and zones containing buckets, | |
714 | which in turn contain objects. A realm enables the Ceph Object Gateway | |
715 | to support multiple namespaces and their configuration on the same | |
716 | hardware. | |
717 | ||
718 | A realm contains the notion of periods. Each period represents the state | |
719 | of the zone group and zone configuration in time. Each time you make a | |
720 | change to a zonegroup or zone, update the period and commit it. | |
721 | ||
722 | By default, the Ceph Object Gateway does not create a realm | |
723 | for backward compatibility with Infernalis and earlier releases. | |
724 | However, as a best practice, we recommend creating realms for new | |
725 | clusters. | |
726 | ||
727 | Create a Realm | |
728 | ~~~~~~~~~~~~~~ | |
729 | ||
730 | To create a realm, execute ``realm create`` and specify the realm name. | |
731 | If the realm is the default, specify ``--default``. | |
732 | ||
733 | :: | |
734 | ||
735 | # radosgw-admin realm create --rgw-realm={realm-name} [--default] | |
736 | ||
737 | For example: | |
738 | ||
739 | :: | |
740 | ||
741 | # radosgw-admin realm create --rgw-realm=movies --default | |
742 | ||
743 | By specifying ``--default``, the realm will be called implicitly with | |
744 | each ``radosgw-admin`` call unless ``--rgw-realm`` and the realm name | |
745 | are explicitly provided. | |
746 | ||
747 | Make a Realm the Default | |
748 | ~~~~~~~~~~~~~~~~~~~~~~~~ | |
749 | ||
750 | One realm in the list of realms should be the default realm. There may | |
751 | be only one default realm. If there is only one realm and it wasn’t | |
752 | specified as the default realm when it was created, make it the default | |
753 | realm. Alternatively, to change which realm is the default, execute: | |
754 | ||
755 | :: | |
756 | ||
757 | # radosgw-admin realm default --rgw-realm=movies | |
758 | ||
31f18b77 FG |
759 | .. note:: When the realm is default, the command line assumes |
760 | ``--rgw-realm=<realm-name>`` as an argument. | |
7c673cae FG |
761 | |
762 | Delete a Realm | |
763 | ~~~~~~~~~~~~~~ | |
764 | ||
765 | To delete a realm, execute ``realm delete`` and specify the realm name. | |
766 | ||
767 | :: | |
768 | ||
769 | # radosgw-admin realm delete --rgw-realm={realm-name} | |
770 | ||
771 | For example: | |
772 | ||
773 | :: | |
774 | ||
775 | # radosgw-admin realm delete --rgw-realm=movies | |
776 | ||
777 | Get a Realm | |
778 | ~~~~~~~~~~~ | |
779 | ||
780 | To get a realm, execute ``realm get`` and specify the realm name. | |
781 | ||
782 | :: | |
783 | ||
784 | #radosgw-admin realm get --rgw-realm=<name> | |
785 | ||
786 | For example: | |
787 | ||
788 | :: | |
789 | ||
790 | # radosgw-admin realm get --rgw-realm=movies [> filename.json] | |
791 | ||
792 | The CLI will echo a JSON object with the realm properties. | |
793 | ||
794 | :: | |
795 | ||
796 | { | |
797 | "id": "0a68d52e-a19c-4e8e-b012-a8f831cb3ebc", | |
798 | "name": "movies", | |
799 | "current_period": "b0c5bbef-4337-4edd-8184-5aeab2ec413b", | |
800 | "epoch": 1 | |
801 | } | |
802 | ||
803 | Use ``>`` and an output file name to output the JSON object to a file. | |
804 | ||
805 | Set a Realm | |
806 | ~~~~~~~~~~~ | |
807 | ||
808 | To set a realm, execute ``realm set``, specify the realm name, and | |
809 | ``--infile=`` with an input file name. | |
810 | ||
811 | :: | |
812 | ||
813 | #radosgw-admin realm set --rgw-realm=<name> --infile=<infilename> | |
814 | ||
815 | For example: | |
816 | ||
817 | :: | |
818 | ||
819 | # radosgw-admin realm set --rgw-realm=movies --infile=filename.json | |
820 | ||
821 | List Realms | |
822 | ~~~~~~~~~~~ | |
823 | ||
824 | To list realms, execute ``realm list``. | |
825 | ||
826 | :: | |
827 | ||
828 | # radosgw-admin realm list | |
829 | ||
830 | List Realm Periods | |
831 | ~~~~~~~~~~~~~~~~~~ | |
832 | ||
833 | To list realm periods, execute ``realm list-periods``. | |
834 | ||
835 | :: | |
836 | ||
837 | # radosgw-admin realm list-periods | |
838 | ||
839 | Pull a Realm | |
840 | ~~~~~~~~~~~~ | |
841 | ||
842 | To pull a realm from the node containing the master zone group and | |
843 | master zone to a node containing a secondary zone group or zone, execute | |
844 | ``realm pull`` on the node that will receive the realm configuration. | |
845 | ||
846 | :: | |
847 | ||
848 | # radosgw-admin realm pull --url={url-to-master-zone-gateway} --access-key={access-key} --secret={secret} | |
849 | ||
850 | Rename a Realm | |
851 | ~~~~~~~~~~~~~~ | |
852 | ||
853 | A realm is not part of the period. Consequently, renaming the realm is | |
854 | only applied locally, and will not get pulled with ``realm pull``. When | |
855 | renaming a realm with multiple zones, run the command on each zone. To | |
856 | rename a realm, execute the following: | |
857 | ||
858 | :: | |
859 | ||
860 | # radosgw-admin realm rename --rgw-realm=<current-name> --realm-new-name=<new-realm-name> | |
861 | ||
862 | .. note:: DO NOT use ``realm set`` to change the ``name`` parameter. That | |
863 | changes the internal name only. Specifying ``--rgw-realm`` would | |
864 | still use the old realm name. | |
865 | ||
866 | Zone Groups | |
867 | ----------- | |
868 | ||
869 | The Ceph Object Gateway supports multi-site deployments and a global | |
870 | namespace by using the notion of zone groups. Formerly called a region | |
871 | in Infernalis, a zone group defines the geographic location of one or more Ceph | |
872 | Object Gateway instances within one or more zones. | |
873 | ||
874 | Configuring zone groups differs from typical configuration procedures, | |
875 | because not all of the settings end up in a Ceph configuration file. You | |
876 | can list zone groups, get a zone group configuration, and set a zone | |
877 | group configuration. | |
878 | ||
879 | Create a Zone Group | |
880 | ~~~~~~~~~~~~~~~~~~~ | |
881 | ||
882 | Creating a zone group consists of specifying the zone group name. | |
883 | Creating a zone assumes it will live in the default realm unless | |
884 | ``--rgw-realm=<realm-name>`` is specified. If the zonegroup is the | |
885 | default zonegroup, specify the ``--default`` flag. If the zonegroup is | |
886 | the master zonegroup, specify the ``--master`` flag. For example: | |
887 | ||
888 | :: | |
889 | ||
890 | # radosgw-admin zonegroup create --rgw-zonegroup=<name> [--rgw-realm=<name>][--master] [--default] | |
891 | ||
892 | ||
893 | .. note:: Use ``zonegroup modify --rgw-zonegroup=<zonegroup-name>`` to modify | |
894 | an existing zone group’s settings. | |
895 | ||
896 | Make a Zone Group the Default | |
897 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
898 | ||
899 | One zonegroup in the list of zonegroups should be the default zonegroup. | |
900 | There may be only one default zonegroup. If there is only one zonegroup | |
901 | and it wasn’t specified as the default zonegroup when it was created, | |
902 | make it the default zonegroup. Alternatively, to change which zonegroup | |
903 | is the default, execute: | |
904 | ||
905 | :: | |
906 | ||
907 | # radosgw-admin zonegroup default --rgw-zonegroup=comedy | |
908 | ||
909 | .. note:: When the zonegroup is default, the command line assumes | |
910 | ``--rgw-zonegroup=<zonegroup-name>`` as an argument. | |
911 | ||
912 | Then, update the period: | |
913 | ||
914 | :: | |
915 | ||
916 | # radosgw-admin period update --commit | |
917 | ||
918 | Add a Zone to a Zone Group | |
919 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
920 | ||
921 | To add a zone to a zonegroup, execute the following: | |
922 | ||
923 | :: | |
924 | ||
925 | # radosgw-admin zonegroup add --rgw-zonegroup=<name> --rgw-zone=<name> | |
926 | ||
927 | Then, update the period: | |
928 | ||
929 | :: | |
930 | ||
931 | # radosgw-admin period update --commit | |
932 | ||
933 | Remove a Zone from a Zone Group | |
934 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
935 | ||
936 | To remove a zone from a zonegroup, execute the following: | |
937 | ||
938 | :: | |
939 | ||
940 | # radosgw-admin zonegroup remove --rgw-zonegroup=<name> --rgw-zone=<name> | |
941 | ||
942 | Then, update the period: | |
943 | ||
944 | :: | |
945 | ||
946 | # radosgw-admin period update --commit | |
947 | ||
948 | Rename a Zone Group | |
949 | ~~~~~~~~~~~~~~~~~~~ | |
950 | ||
951 | To rename a zonegroup, execute the following: | |
952 | ||
953 | :: | |
954 | ||
955 | # radosgw-admin zonegroup rename --rgw-zonegroup=<name> --zonegroup-new-name=<name> | |
956 | ||
957 | Then, update the period: | |
958 | ||
959 | :: | |
960 | ||
961 | # radosgw-admin period update --commit | |
962 | ||
963 | Delete a Zone Group | |
964 | ~~~~~~~~~~~~~~~~~~~ | |
965 | ||
966 | To delete a zonegroup, execute the following: | |
967 | ||
968 | :: | |
969 | ||
970 | # radosgw-admin zonegroup delete --rgw-zonegroup=<name> | |
971 | ||
972 | Then, update the period: | |
973 | ||
974 | :: | |
975 | ||
976 | # radosgw-admin period update --commit | |
977 | ||
978 | List Zone Groups | |
979 | ~~~~~~~~~~~~~~~~ | |
980 | ||
981 | A Ceph cluster contains a list of zone groups. To list the zone groups, | |
982 | execute: | |
983 | ||
984 | :: | |
985 | ||
986 | # radosgw-admin zonegroup list | |
987 | ||
988 | The ``radosgw-admin`` returns a JSON formatted list of zone groups. | |
989 | ||
990 | :: | |
991 | ||
992 | { | |
993 | "default_info": "90b28698-e7c3-462c-a42d-4aa780d24eda", | |
994 | "zonegroups": [ | |
995 | "us" | |
996 | ] | |
997 | } | |
998 | ||
999 | Get a Zone Group Map | |
1000 | ~~~~~~~~~~~~~~~~~~~~ | |
1001 | ||
1002 | To list the details of each zone group, execute: | |
1003 | ||
1004 | :: | |
1005 | ||
1006 | # radosgw-admin zonegroup-map get | |
1007 | ||
1008 | .. note:: If you receive a ``failed to read zonegroup map`` error, run | |
1009 | ``radosgw-admin zonegroup-map update`` as ``root`` first. | |
1010 | ||
1011 | Get a Zone Group | |
1012 | ~~~~~~~~~~~~~~~~ | |
1013 | ||
1014 | To view the configuration of a zone group, execute: | |
1015 | ||
1016 | :: | |
1017 | ||
1018 | radosgw-admin zonegroup get [--rgw-zonegroup=<zonegroup>] | |
1019 | ||
1020 | The zone group configuration looks like this: | |
1021 | ||
1022 | :: | |
1023 | ||
1024 | { | |
1025 | "id": "90b28698-e7c3-462c-a42d-4aa780d24eda", | |
1026 | "name": "us", | |
1027 | "api_name": "us", | |
1028 | "is_master": "true", | |
1029 | "endpoints": [ | |
1030 | "http:\/\/rgw1:80" | |
1031 | ], | |
1032 | "hostnames": [], | |
1033 | "hostnames_s3website": [], | |
1034 | "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e", | |
1035 | "zones": [ | |
1036 | { | |
1037 | "id": "9248cab2-afe7-43d8-a661-a40bf316665e", | |
1038 | "name": "us-east", | |
1039 | "endpoints": [ | |
1040 | "http:\/\/rgw1" | |
1041 | ], | |
1042 | "log_meta": "true", | |
1043 | "log_data": "true", | |
1044 | "bucket_index_max_shards": 0, | |
1045 | "read_only": "false" | |
1046 | }, | |
1047 | { | |
1048 | "id": "d1024e59-7d28-49d1-8222-af101965a939", | |
1049 | "name": "us-west", | |
1050 | "endpoints": [ | |
1051 | "http:\/\/rgw2:80" | |
1052 | ], | |
1053 | "log_meta": "false", | |
1054 | "log_data": "true", | |
1055 | "bucket_index_max_shards": 0, | |
1056 | "read_only": "false" | |
1057 | } | |
1058 | ], | |
1059 | "placement_targets": [ | |
1060 | { | |
1061 | "name": "default-placement", | |
1062 | "tags": [] | |
1063 | } | |
1064 | ], | |
1065 | "default_placement": "default-placement", | |
1066 | "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe" | |
1067 | } | |
1068 | ||
1069 | Set a Zone Group | |
1070 | ~~~~~~~~~~~~~~~~ | |
1071 | ||
1072 | Defining a zone group consists of creating a JSON object, specifying at | |
1073 | least the required settings: | |
1074 | ||
1075 | 1. ``name``: The name of the zone group. Required. | |
1076 | ||
1077 | 2. ``api_name``: The API name for the zone group. Optional. | |
1078 | ||
1079 | 3. ``is_master``: Determines if the zone group is the master zone group. | |
1080 | Required. **note:** You can only have one master zone group. | |
1081 | ||
1082 | 4. ``endpoints``: A list of all the endpoints in the zone group. For | |
1083 | example, you may use multiple domain names to refer to the same zone | |
1084 | group. Remember to escape the forward slashes (``\/``). You may also | |
1085 | specify a port (``fqdn:port``) for each endpoint. Optional. | |
1086 | ||
1087 | 5. ``hostnames``: A list of all the hostnames in the zone group. For | |
1088 | example, you may use multiple domain names to refer to the same zone | |
1089 | group. Optional. The ``rgw dns name`` setting will automatically be | |
1090 | included in this list. You should restart the gateway daemon(s) after | |
1091 | changing this setting. | |
1092 | ||
1093 | 6. ``master_zone``: The master zone for the zone group. Optional. Uses | |
1094 | the default zone if not specified. **note:** You can only have one | |
1095 | master zone per zone group. | |
1096 | ||
1097 | 7. ``zones``: A list of all zones within the zone group. Each zone has a | |
1098 | name (required), a list of endpoints (optional), and whether or not | |
1099 | the gateway will log metadata and data operations (false by default). | |
1100 | ||
1101 | 8. ``placement_targets``: A list of placement targets (optional). Each | |
1102 | placement target contains a name (required) for the placement target | |
1103 | and a list of tags (optional) so that only users with the tag can use | |
1104 | the placement target (i.e., the user’s ``placement_tags`` field in | |
1105 | the user info). | |
1106 | ||
1107 | 9. ``default_placement``: The default placement target for the object | |
1108 | index and object data. Set to ``default-placement`` by default. You | |
1109 | may also set a per-user default placement in the user info for each | |
1110 | user. | |
1111 | ||
1112 | To set a zone group, create a JSON object consisting of the required | |
1113 | fields, save the object to a file (e.g., ``zonegroup.json``); then, | |
1114 | execute the following command: | |
1115 | ||
1116 | :: | |
1117 | ||
1118 | # radosgw-admin zonegroup set --infile zonegroup.json | |
1119 | ||
1120 | Where ``zonegroup.json`` is the JSON file you created. | |
1121 | ||
1122 | .. important:: The ``default`` zone group ``is_master`` setting is ``true`` by | |
1123 | default. If you create a new zone group and want to make it the | |
1124 | master zone group, you must either set the ``default`` zone group | |
1125 | ``is_master`` setting to ``false``, or delete the ``default`` zone | |
1126 | group. | |
1127 | ||
1128 | Finally, update the period: | |
1129 | ||
1130 | :: | |
1131 | ||
1132 | # radosgw-admin period update --commit | |
1133 | ||
1134 | Set a Zone Group Map | |
1135 | ~~~~~~~~~~~~~~~~~~~~ | |
1136 | ||
1137 | Setting a zone group map consists of creating a JSON object consisting | |
1138 | of one or more zone groups, and setting the ``master_zonegroup`` for the | |
1139 | cluster. Each zone group in the zone group map consists of a key/value | |
1140 | pair, where the ``key`` setting is equivalent to the ``name`` setting | |
1141 | for an individual zone group configuration, and the ``val`` is a JSON | |
1142 | object consisting of an individual zone group configuration. | |
1143 | ||
1144 | You may only have one zone group with ``is_master`` equal to ``true``, | |
1145 | and it must be specified as the ``master_zonegroup`` at the end of the | |
1146 | zone group map. The following JSON object is an example of a default | |
1147 | zone group map. | |
1148 | ||
1149 | :: | |
1150 | ||
1151 | { | |
1152 | "zonegroups": [ | |
1153 | { | |
1154 | "key": "90b28698-e7c3-462c-a42d-4aa780d24eda", | |
1155 | "val": { | |
1156 | "id": "90b28698-e7c3-462c-a42d-4aa780d24eda", | |
1157 | "name": "us", | |
1158 | "api_name": "us", | |
1159 | "is_master": "true", | |
1160 | "endpoints": [ | |
1161 | "http:\/\/rgw1:80" | |
1162 | ], | |
1163 | "hostnames": [], | |
1164 | "hostnames_s3website": [], | |
1165 | "master_zone": "9248cab2-afe7-43d8-a661-a40bf316665e", | |
1166 | "zones": [ | |
1167 | { | |
1168 | "id": "9248cab2-afe7-43d8-a661-a40bf316665e", | |
1169 | "name": "us-east", | |
1170 | "endpoints": [ | |
1171 | "http:\/\/rgw1" | |
1172 | ], | |
1173 | "log_meta": "true", | |
1174 | "log_data": "true", | |
1175 | "bucket_index_max_shards": 0, | |
1176 | "read_only": "false" | |
1177 | }, | |
1178 | { | |
1179 | "id": "d1024e59-7d28-49d1-8222-af101965a939", | |
1180 | "name": "us-west", | |
1181 | "endpoints": [ | |
1182 | "http:\/\/rgw2:80" | |
1183 | ], | |
1184 | "log_meta": "false", | |
1185 | "log_data": "true", | |
1186 | "bucket_index_max_shards": 0, | |
1187 | "read_only": "false" | |
1188 | } | |
1189 | ], | |
1190 | "placement_targets": [ | |
1191 | { | |
1192 | "name": "default-placement", | |
1193 | "tags": [] | |
1194 | } | |
1195 | ], | |
1196 | "default_placement": "default-placement", | |
1197 | "realm_id": "ae031368-8715-4e27-9a99-0c9468852cfe" | |
1198 | } | |
1199 | } | |
1200 | ], | |
1201 | "master_zonegroup": "90b28698-e7c3-462c-a42d-4aa780d24eda", | |
1202 | "bucket_quota": { | |
1203 | "enabled": false, | |
1204 | "max_size_kb": -1, | |
1205 | "max_objects": -1 | |
1206 | }, | |
1207 | "user_quota": { | |
1208 | "enabled": false, | |
1209 | "max_size_kb": -1, | |
1210 | "max_objects": -1 | |
1211 | } | |
1212 | } | |
1213 | ||
1214 | To set a zone group map, execute the following: | |
1215 | ||
1216 | :: | |
1217 | ||
1218 | # radosgw-admin zonegroup-map set --infile zonegroupmap.json | |
1219 | ||
1220 | Where ``zonegroupmap.json`` is the JSON file you created. Ensure that | |
1221 | you have zones created for the ones specified in the zone group map. | |
1222 | Finally, update the period. | |
1223 | ||
1224 | :: | |
1225 | ||
1226 | # radosgw-admin period update --commit | |
1227 | ||
1228 | Zones | |
1229 | ----- | |
1230 | ||
1231 | Ceph Object Gateway supports the notion of zones. A zone defines a | |
1232 | logical group consisting of one or more Ceph Object Gateway instances. | |
1233 | ||
1234 | Configuring zones differs from typical configuration procedures, because | |
1235 | not all of the settings end up in a Ceph configuration file. You can | |
1236 | list zones, get a zone configuration and set a zone configuration. | |
1237 | ||
1238 | Create a Zone | |
1239 | ~~~~~~~~~~~~~ | |
1240 | ||
1241 | To create a zone, specify a zone name. If it is a master zone, specify | |
1242 | the ``--master`` option. Only one zone in a zone group may be a master | |
1243 | zone. To add the zone to a zonegroup, specify the ``--rgw-zonegroup`` | |
1244 | option with the zonegroup name. | |
1245 | ||
1246 | :: | |
1247 | ||
1248 | # radosgw-admin zone create --rgw-zone=<name> \ | |
1249 | [--zonegroup=<zonegroup-name]\ | |
1250 | [--endpoints=<endpoint>[,<endpoint>] \ | |
1251 | [--master] [--default] \ | |
1252 | --access-key $SYSTEM_ACCESS_KEY --secret $SYSTEM_SECRET_KEY | |
1253 | ||
1254 | Then, update the period: | |
1255 | ||
1256 | :: | |
1257 | ||
1258 | # radosgw-admin period update --commit | |
1259 | ||
1260 | Delete a Zone | |
1261 | ~~~~~~~~~~~~~ | |
1262 | ||
1263 | To delete zone, first remove it from the zonegroup. | |
1264 | ||
1265 | :: | |
1266 | ||
1267 | # radosgw-admin zonegroup remove --zonegroup=<name>\ | |
1268 | --zone=<name> | |
1269 | ||
1270 | Then, update the period: | |
1271 | ||
1272 | :: | |
1273 | ||
1274 | # radosgw-admin period update --commit | |
1275 | ||
1276 | Next, delete the zone. Execute the following: | |
1277 | ||
1278 | :: | |
1279 | ||
eafe8130 | 1280 | # radosgw-admin zone rm --rgw-zone<name> |
7c673cae FG |
1281 | |
1282 | Finally, update the period: | |
1283 | ||
1284 | :: | |
1285 | ||
1286 | # radosgw-admin period update --commit | |
1287 | ||
1288 | .. important:: Do not delete a zone without removing it from a zone group first. | |
1289 | Otherwise, updating the period will fail. | |
1290 | ||
1291 | If the pools for the deleted zone will not be used anywhere else, | |
1292 | consider deleting the pools. Replace ``<del-zone>`` in the example below | |
1293 | with the deleted zone’s name. | |
1294 | ||
1295 | .. important:: Only delete the pools with prepended zone names. Deleting the root | |
1296 | pool, such as, ``.rgw.root`` will remove all of the system’s | |
1297 | configuration. | |
1298 | ||
1299 | .. important:: Once the pools are deleted, all of the data within them are deleted | |
1300 | in an unrecoverable manner. Only delete the pools if the pool | |
1301 | contents are no longer needed. | |
1302 | ||
1303 | :: | |
1304 | ||
11fdf7f2 TL |
1305 | # ceph osd pool rm <del-zone>.rgw.control <del-zone>.rgw.control --yes-i-really-really-mean-it |
1306 | # ceph osd pool rm <del-zone>.rgw.data.root <del-zone>.rgw.data.root --yes-i-really-really-mean-it | |
1307 | # ceph osd pool rm <del-zone>.rgw.gc <del-zone>.rgw.gc --yes-i-really-really-mean-it | |
1308 | # ceph osd pool rm <del-zone>.rgw.log <del-zone>.rgw.log --yes-i-really-really-mean-it | |
1309 | # ceph osd pool rm <del-zone>.rgw.users.uid <del-zone>.rgw.users.uid --yes-i-really-really-mean-it | |
7c673cae FG |
1310 | |
1311 | Modify a Zone | |
1312 | ~~~~~~~~~~~~~ | |
1313 | ||
1314 | To modify a zone, specify the zone name and the parameters you wish to | |
1315 | modify. | |
1316 | ||
1317 | :: | |
1318 | ||
1319 | # radosgw-admin zone modify [options] | |
1320 | ||
1321 | Where ``[options]``: | |
1322 | ||
1323 | - ``--access-key=<key>`` | |
1324 | - ``--secret/--secret-key=<key>`` | |
1325 | - ``--master`` | |
1326 | - ``--default`` | |
1327 | - ``--endpoints=<list>`` | |
1328 | ||
1329 | Then, update the period: | |
1330 | ||
1331 | :: | |
1332 | ||
1333 | # radosgw-admin period update --commit | |
1334 | ||
1335 | List Zones | |
1336 | ~~~~~~~~~~ | |
1337 | ||
1338 | As ``root``, to list the zones in a cluster, execute: | |
1339 | ||
1340 | :: | |
1341 | ||
1342 | # radosgw-admin zone list | |
1343 | ||
1344 | Get a Zone | |
1345 | ~~~~~~~~~~ | |
1346 | ||
1347 | As ``root``, to get the configuration of a zone, execute: | |
1348 | ||
1349 | :: | |
1350 | ||
1351 | # radosgw-admin zone get [--rgw-zone=<zone>] | |
1352 | ||
1353 | The ``default`` zone looks like this: | |
1354 | ||
1355 | :: | |
1356 | ||
1357 | { "domain_root": ".rgw", | |
1358 | "control_pool": ".rgw.control", | |
1359 | "gc_pool": ".rgw.gc", | |
1360 | "log_pool": ".log", | |
1361 | "intent_log_pool": ".intent-log", | |
1362 | "usage_log_pool": ".usage", | |
1363 | "user_keys_pool": ".users", | |
1364 | "user_email_pool": ".users.email", | |
1365 | "user_swift_pool": ".users.swift", | |
1366 | "user_uid_pool": ".users.uid", | |
1367 | "system_key": { "access_key": "", "secret_key": ""}, | |
1368 | "placement_pools": [ | |
1369 | { "key": "default-placement", | |
1370 | "val": { "index_pool": ".rgw.buckets.index", | |
1371 | "data_pool": ".rgw.buckets"} | |
1372 | } | |
1373 | ] | |
1374 | } | |
1375 | ||
1376 | Set a Zone | |
1377 | ~~~~~~~~~~ | |
1378 | ||
1379 | Configuring a zone involves specifying a series of Ceph Object Gateway | |
1380 | pools. For consistency, we recommend using a pool prefix that is the | |
1381 | same as the zone name. See | |
1382 | `Pools <http://docs.ceph.com/docs/master/rados/operations/pools/#pools>`__ | |
1383 | for details of configuring pools. | |
1384 | ||
1385 | To set a zone, create a JSON object consisting of the pools, save the | |
1386 | object to a file (e.g., ``zone.json``); then, execute the following | |
1387 | command, replacing ``{zone-name}`` with the name of the zone: | |
1388 | ||
1389 | :: | |
1390 | ||
1391 | # radosgw-admin zone set --rgw-zone={zone-name} --infile zone.json | |
1392 | ||
1393 | Where ``zone.json`` is the JSON file you created. | |
1394 | ||
1395 | Then, as ``root``, update the period: | |
1396 | ||
1397 | :: | |
1398 | ||
1399 | # radosgw-admin period update --commit | |
1400 | ||
1401 | Rename a Zone | |
1402 | ~~~~~~~~~~~~~ | |
1403 | ||
1404 | To rename a zone, specify the zone name and the new zone name. | |
1405 | ||
1406 | :: | |
1407 | ||
1408 | # radosgw-admin zone rename --rgw-zone=<name> --zone-new-name=<name> | |
1409 | ||
1410 | Then, update the period: | |
1411 | ||
1412 | :: | |
1413 | ||
1414 | # radosgw-admin period update --commit | |
1415 | ||
1416 | Zone Group and Zone Settings | |
1417 | ---------------------------- | |
1418 | ||
1419 | When configuring a default zone group and zone, the pool name includes | |
1420 | the zone name. For example: | |
1421 | ||
1422 | - ``default.rgw.control`` | |
1423 | ||
1424 | To change the defaults, include the following settings in your Ceph | |
1425 | configuration file under each ``[client.radosgw.{instance-name}]`` | |
1426 | instance. | |
1427 | ||
1428 | +-------------------------------------+-----------------------------------+---------+-----------------------+ | |
1429 | | Name | Description | Type | Default | | |
1430 | +=====================================+===================================+=========+=======================+ | |
1431 | | ``rgw_zone`` | The name of the zone for the | String | None | | |
1432 | | | gateway instance. | | | | |
1433 | +-------------------------------------+-----------------------------------+---------+-----------------------+ | |
1434 | | ``rgw_zonegroup`` | The name of the zone group for | String | None | | |
1435 | | | the gateway instance. | | | | |
1436 | +-------------------------------------+-----------------------------------+---------+-----------------------+ | |
1437 | | ``rgw_zonegroup_root_pool`` | The root pool for the zone group. | String | ``.rgw.root`` | | |
1438 | +-------------------------------------+-----------------------------------+---------+-----------------------+ | |
1439 | | ``rgw_zone_root_pool`` | The root pool for the zone. | String | ``.rgw.root`` | | |
1440 | +-------------------------------------+-----------------------------------+---------+-----------------------+ | |
1441 | | ``rgw_default_zone_group_info_oid`` | The OID for storing the default | String | ``default.zonegroup`` | | |
1442 | | | zone group. We do not recommend | | | | |
1443 | | | changing this setting. | | | | |
1444 | +-------------------------------------+-----------------------------------+---------+-----------------------+ | |
31f18b77 FG |
1445 | |
1446 | ||
1447 | .. _`Pools`: ../pools |