ceph/doc/cephadm/services/rgw.rst

   1 ===========
   2 RGW Service
   3 ===========
   4
   5 .. _cephadm-deploy-rgw:
   6
   7 Deploy RGWs
   8 ===========
   9
  10 Cephadm deploys radosgw as a collection of daemons that manage a
  11 single-cluster deployment or a particular *realm* and *zone* in a
  12 multisite deployment.  (For more information about realms and zones,
  13 see :ref:`multisite`.)
  14
  15 Note that with cephadm, radosgw daemons are configured via the monitor
  16 configuration database instead of via a `ceph.conf` or the command line.  If
  17 that configuration isn't already in place (usually in the
  18 ``client.rgw.<something>`` section), then the radosgw
  19 daemons will start up with default settings (e.g., binding to port
  20 80).
  21
  22 To deploy a set of radosgw daemons, with an arbitrary service name
  23 *name*, run the following command:
  24
  25 .. prompt:: bash #
  26
  27   ceph orch apply rgw *<name>* [--realm=*<realm-name>*] [--zone=*<zone-name>*] --placement="*<num-daemons>* [*<host1>* ...]"
  28
  29 Trivial setup
  30 -------------
  31
  32 For example, to deploy 2 RGW daemons (the default) for a single-cluster RGW deployment
  33 under the arbitrary service id *foo*:
  34
  35 .. prompt:: bash #
  36
  37    ceph orch apply rgw foo
  38
  39 .. _cephadm-rgw-designated_gateways:
  40
  41 Designated gateways
  42 -------------------
  43
  44 A common scenario is to have a labeled set of hosts that will act
  45 as gateways, with multiple instances of radosgw running on consecutive
  46 ports 8000 and 8001:
  47
  48 .. prompt:: bash #
  49
  50    ceph orch host label add gwhost1 rgw  # the 'rgw' label can be anything
  51    ceph orch host label add gwhost2 rgw
  52    ceph orch apply rgw foo '--placement=label:rgw count-per-host:2' --port=8000
  53
  54 See also: :ref:`cephadm_co_location`.
  55
  56 .. _cephadm-rgw-networks:
  57
  58 Specifying Networks
  59 -------------------
  60
  61 The RGW service can have the network they bind to configured with a yaml service specification.
  62
  63 example spec file:
  64
  65 .. code-block:: yaml
  66
  67     service_type: rgw
  68     service_id: foo
  69     placement:
  70       label: rgw
  71       count_per_host: 2
  72     networks:
  73     - 192.169.142.0/24
  74     spec:
  75       rgw_frontend_port: 8080
  76
  77
  78 Multisite zones
  79 ---------------
  80
  81 To deploy RGWs serving the multisite *myorg* realm and the *us-east-1* zone on
  82 *myhost1* and *myhost2*:
  83
  84 .. prompt:: bash #
  85
  86    ceph orch apply rgw east --realm=myorg --zone=us-east-1 --placement="2 myhost1 myhost2"
  87
  88 Note that in a multisite situation, cephadm only deploys the daemons.  It does not create
  89 or update the realm or zone configurations.  To create a new realm and zone, you need to do
  90 something like:
  91
  92 .. prompt:: bash #
  93
  94   radosgw-admin realm create --rgw-realm=<realm-name> --default
  95
  96 .. prompt:: bash #
  97
  98   radosgw-admin zonegroup create --rgw-zonegroup=<zonegroup-name>  --master --default
  99
 100 .. prompt:: bash #
 101
 102   radosgw-admin zone create --rgw-zonegroup=<zonegroup-name> --rgw-zone=<zone-name> --master --default
 103
 104 .. prompt:: bash #
 105
 106   radosgw-admin period update --rgw-realm=<realm-name> --commit
 107
 108 See :ref:`orchestrator-cli-placement-spec` for details of the placement
 109 specification.  See :ref:`multisite` for more information of setting up multisite RGW.
 110
 111 See also :ref:`multisite`.
 112
 113 Setting up HTTPS
 114 ----------------
 115
 116 In order to enable HTTPS for RGW services, apply a spec file following this scheme:
 117
 118 .. code-block:: yaml
 119
 120   service_type: rgw
 121   service_id: myrgw
 122   spec:
 123     rgw_frontend_ssl_certificate: |
 124       -----BEGIN PRIVATE KEY-----
 125       V2VyIGRhcyBsaWVzdCBpc3QgZG9vZi4gTG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFt
 126       ZXQsIGNvbnNldGV0dXIgc2FkaXBzY2luZyBlbGl0ciwgc2VkIGRpYW0gbm9udW15
 127       IGVpcm1vZCB0ZW1wb3IgaW52aWR1bnQgdXQgbGFib3JlIGV0IGRvbG9yZSBtYWdu
 128       YSBhbGlxdXlhbSBlcmF0LCBzZWQgZGlhbSB2b2x1cHR1YS4gQXQgdmVybyBlb3Mg
 129       ZXQgYWNjdXNhbSBldCBqdXN0byBkdW8=
 130       -----END PRIVATE KEY-----
 131       -----BEGIN CERTIFICATE-----
 132       V2VyIGRhcyBsaWVzdCBpc3QgZG9vZi4gTG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFt
 133       ZXQsIGNvbnNldGV0dXIgc2FkaXBzY2luZyBlbGl0ciwgc2VkIGRpYW0gbm9udW15
 134       IGVpcm1vZCB0ZW1wb3IgaW52aWR1bnQgdXQgbGFib3JlIGV0IGRvbG9yZSBtYWdu
 135       YSBhbGlxdXlhbSBlcmF0LCBzZWQgZGlhbSB2b2x1cHR1YS4gQXQgdmVybyBlb3Mg
 136       ZXQgYWNjdXNhbSBldCBqdXN0byBkdW8=
 137       -----END CERTIFICATE-----
 138     ssl: true
 139
 140 Then apply this yaml document:
 141
 142 .. prompt:: bash #
 143
 144   ceph orch apply -i myrgw.yaml
 145
 146 Note the value of ``rgw_frontend_ssl_certificate`` is a literal string as
 147 indicated by a ``|`` character preserving newline characters.
 148
 149 Service specification
 150 ---------------------
 151
 152 .. py:currentmodule:: ceph.deployment.service_spec
 153
 154 .. autoclass:: RGWSpec
 155    :members:
 156
 157 .. _orchestrator-haproxy-service-spec:
 158
 159 High availability service for RGW
 160 =================================
 161
 162 The *ingress* service allows you to create a high availability endpoint
 163 for RGW with a minimum set of configuration options.  The orchestrator will
 164 deploy and manage a combination of haproxy and keepalived to provide load
 165 balancing on a floating virtual IP.
 166
 167 If the RGW service is configured with SSL enabled, then the ingress service
 168 will use the `ssl` and `verify none` options in the backend configuration.
 169 Trust verification is disabled because the backends are accessed by IP
 170 address instead of FQDN.
 171
 172 .. image:: ../../images/HAProxy_for_RGW.svg
 173
 174 There are N hosts where the ingress service is deployed.  Each host
 175 has a haproxy daemon and a keepalived daemon.  A virtual IP is
 176 automatically configured on only one of these hosts at a time.
 177
 178 Each keepalived daemon checks every few seconds whether the haproxy
 179 daemon on the same host is responding.  Keepalived will also check
 180 that the master keepalived daemon is running without problems.  If the
 181 "master" keepalived daemon or the active haproxy is not responding,
 182 one of the remaining keepalived daemons running in backup mode will be
 183 elected as master, and the virtual IP will be moved to that node.
 184
 185 The active haproxy acts like a load balancer, distributing all RGW requests
 186 between all the RGW daemons available.
 187
 188 Prerequisites
 189 -------------
 190
 191 * An existing RGW service.
 192
 193 Deploying
 194 ---------
 195
 196 Use the command::
 197
 198     ceph orch apply -i <ingress_spec_file>
 199
 200 Service specification
 201 ---------------------
 202
 203 It is a yaml format file with the following properties:
 204
 205 .. code-block:: yaml
 206
 207     service_type: ingress
 208     service_id: rgw.something    # adjust to match your existing RGW service
 209     placement:
 210       hosts:
 211         - host1
 212         - host2
 213         - host3
 214     spec:
 215       backend_service: rgw.something      # adjust to match your existing RGW service
 216       virtual_ip: <string>/<string>       # ex: 192.168.20.1/24
 217       frontend_port: <integer>            # ex: 8080
 218       monitor_port: <integer>             # ex: 1967, used by haproxy for load balancer status
 219       virtual_interface_networks: [ ... ] # optional: list of CIDR networks
 220       ssl_cert: |                         # optional: SSL certificate and key
 221         -----BEGIN CERTIFICATE-----
 222         ...
 223         -----END CERTIFICATE-----
 224         -----BEGIN PRIVATE KEY-----
 225         ...
 226         -----END PRIVATE KEY-----
 227
 228 .. code-block:: yaml
 229
 230     service_type: ingress
 231     service_id: rgw.something    # adjust to match your existing RGW service
 232     placement:
 233       hosts:
 234         - host1
 235         - host2
 236         - host3
 237     spec:
 238       backend_service: rgw.something      # adjust to match your existing RGW service
 239       virtual_ips_list:
 240       - <string>/<string>                 # ex: 192.168.20.1/24
 241       - <string>/<string>                 # ex: 192.168.20.2/24
 242       - <string>/<string>                 # ex: 192.168.20.3/24
 243       frontend_port: <integer>            # ex: 8080
 244       monitor_port: <integer>             # ex: 1967, used by haproxy for load balancer status
 245       virtual_interface_networks: [ ... ] # optional: list of CIDR networks
 246       ssl_cert: |                         # optional: SSL certificate and key
 247         -----BEGIN CERTIFICATE-----
 248         ...
 249         -----END CERTIFICATE-----
 250         -----BEGIN PRIVATE KEY-----
 251         ...
 252         -----END PRIVATE KEY-----
 253
 254
 255 where the properties of this service specification are:
 256
 257 * ``service_type``
 258     Mandatory and set to "ingress"
 259 * ``service_id``
 260     The name of the service.  We suggest naming this after the service you are
 261     controlling ingress for (e.g., ``rgw.foo``).
 262 * ``placement hosts``
 263     The hosts where it is desired to run the HA daemons. An haproxy and a
 264     keepalived container will be deployed on these hosts.  These hosts do not need
 265     to match the nodes where RGW is deployed.
 266 * ``virtual_ip``
 267     The virtual IP (and network) in CIDR format where the ingress service will be available.
 268 * ``virtual_ips_list``
 269     The virtual IP address in CIDR format where the ingress service will be available.
 270     Each virtual IP address will be primary on one node running the ingress service. The number
 271     of virtual IP addresses must be less than or equal to the number of ingress nodes.
 272 * ``virtual_interface_networks``
 273     A list of networks to identify which ethernet interface to use for the virtual IP.
 274 * ``frontend_port``
 275     The port used to access the ingress service.
 276 * ``ssl_cert``:
 277     SSL certificate, if SSL is to be enabled. This must contain the both the certificate and
 278     private key blocks in .pem format.
 279
 280 .. _ingress-virtual-ip:
 281
 282 Selecting ethernet interfaces for the virtual IP
 283 ------------------------------------------------
 284
 285 You cannot simply provide the name of the network interface on which
 286 to configure the virtual IP because interface names tend to vary
 287 across hosts (and/or reboots).  Instead, cephadm will select
 288 interfaces based on other existing IP addresses that are already
 289 configured.
 290
 291 Normally, the virtual IP will be configured on the first network
 292 interface that has an existing IP in the same subnet.  For example, if
 293 the virtual IP is 192.168.0.80/24 and eth2 has the static IP
 294 192.168.0.40/24, cephadm will use eth2.
 295
 296 In some cases, the virtual IP may not belong to the same subnet as an existing static
 297 IP.  In such cases, you can provide a list of subnets to match against existing IPs,
 298 and cephadm will put the virtual IP on the first network interface to match.  For example,
 299 if the virtual IP is 192.168.0.80/24 and we want it on the same interface as the machine's
 300 static IP in 10.10.0.0/16, you can use a spec like::
 301
 302   service_type: ingress
 303   service_id: rgw.something
 304   spec:
 305     virtual_ip: 192.168.0.80/24
 306     virtual_interface_networks:
 307       - 10.10.0.0/16
 308     ...
 309
 310 A consequence of this strategy is that you cannot currently configure the virtual IP
 311 on an interface that has no existing IP address.  In this situation, we suggest
 312 configuring a "dummy" IP address is an unroutable network on the correct interface
 313 and reference that dummy network in the networks list (see above).
 314
 315
 316 Useful hints for ingress
 317 ------------------------
 318
 319 * It is good to have at least 3 RGW daemons.
 320 * We recommend at least 3 hosts for the ingress service.
 321
 322 Further Reading
 323 ===============
 324
 325 * :ref:`object-gateway`
 326 * :ref:`mgr-rgw-module`