ceph/doc/radosgw/vault.rst

   1 ===========================
   2 HashiCorp Vault Integration
   3 ===========================
   4
   5 HashiCorp `Vault`_ can be used as a secure key management service for
   6 `Server-Side Encryption`_ (SSE-KMS).
   7
   8 .. ditaa::
   9
  10            +---------+       +---------+        +-------+     +-------+
  11            |  Client |       | RadosGW |        | Vault |     |  OSD  |
  12            +---------+       +---------+        +-------+     +-------+
  13                 | create secret   |                 |             |
  14                 | key for key ID  |                 |             |
  15                 |-----------------+---------------->|             |
  16                 |                 |                 |             |
  17                 | upload object   |                 |             |
  18                 | with key ID     |                 |             |
  19                 |---------------->| request secret  |             |
  20                 |                 | key for key ID  |             |
  21                 |                 |---------------->|             |
  22                 |                 |<----------------|             |
  23                 |                 | return secret   |             |
  24                 |                 | key             |             |
  25                 |                 |                 |             |
  26                 |                 | encrypt object  |             |
  27                 |                 | with secret key |             |
  28                 |                 |--------------+  |             |
  29                 |                 |              |  |             |
  30                 |                 |<-------------+  |             |
  31                 |                 |                 |             |
  32                 |                 | store encrypted |             |
  33                 |                 | object          |             |
  34                 |                 |------------------------------>|
  35
  36 #. `Vault secrets engines`_
  37 #. `Vault authentication`_
  38 #. `Vault namespaces`_
  39 #. `Create a key in Vault`_
  40 #. `Configure the Ceph Object Gateway`_
  41 #. `Upload object`_
  42
  43 Some examples below use the Vault command line utility to interact with
  44 Vault. You may need to set the following environment variable with the correct
  45 address of your Vault server to use this utility::
  46
  47   export VAULT_ADDR='http://vault-server:8200'
  48
  49 Vault secrets engines
  50 =====================
  51
  52 Vault provides several secrets engines, which can store, generate, and encrypt
  53 data. Currently, the Object Gateway supports:
  54
  55 - `KV secrets engine`_ version 2
  56 - `Transit engine`_
  57
  58 KV secrets engine
  59 -----------------
  60
  61 The KV secrets engine is used to store arbitrary key/value secrets in Vault. To
  62 enable the KV engine version 2 in Vault, use the following command::
  63
  64   vault secrets enable kv-v2
  65
  66 The Object Gateway can be configured to use the KV engine version 2 with the
  67 following setting::
  68
  69   rgw crypt vault secret engine = kv
  70
  71 Transit secrets engine
  72 ----------------------
  73
  74 The transit engine handles cryptographic functions on data in-transit. To enable
  75 it in Vault, use the following command::
  76
  77   vault secrets enable transit
  78
  79 The Object Gateway can be configured to use the transit engine with the
  80 following setting::
  81
  82   rgw crypt vault secret engine = transit
  83
  84 Vault authentication
  85 ====================
  86
  87 Vault supports several authentication mechanisms. Currently, the Object
  88 Gateway can be configured to authenticate to Vault using the
  89 `Token authentication method`_ or a `Vault agent`_.
  90
  91 Token authentication
  92 --------------------
  93
  94 .. note:: Token authentication is not recommended for production environments.
  95
  96 The token authentication method expects a Vault token to be present in a
  97 plaintext file. The Object Gateway can be configured to use token authentication
  98 with the following settings::
  99
 100   rgw crypt vault auth = token
 101   rgw crypt vault token file = /etc/ceph/vault.token
 102   rgw crypt vault addr = http://vault-server:8200
 103
 104 For security reasons, the token file must be readable by the Object Gateway
 105 only. Also, the Object Gateway should be given a Vault token with a restricted
 106 policy that allows it to fetch keyrings from a specific path only. Such a policy
 107 can be created in Vault using the command line utility as in the following
 108 examples::
 109
 110   vault policy write rgw-kv-policy -<<EOF
 111     path "secret/data/*" {
 112       capabilities = ["read"]
 113     }
 114   EOF
 115
 116   vault policy write rgw-transit-policy -<<EOF
 117     path "transit/export/encryption-key/*" {
 118       capabilities = ["read"]
 119     }
 120   EOF
 121
 122 Once the policy is created, a token can be generated by a Vault administrator::
 123
 124   vault token create -policy=rgw-kv-policy
 125
 126 Sample output::
 127
 128   Key                  Value
 129   ---                  -----
 130   token                s.72KuPujbc065OdWB71poOmIq
 131   token_accessor       jv95ZYBUFv6Ss84x7SCSy6lZ
 132   token_duration       768h
 133   token_renewable      true
 134   token_policies       ["default" "rgw-kv-policy"]
 135   identity_policies    []
 136   policies             ["default" "rgw-kv-policy"]
 137
 138 The actual token, displayed in the ``Value`` column of the first line of the
 139 output, must be saved in a file as plaintext.
 140
 141 Vault agent
 142 -----------
 143
 144 The Vault agent is a client daemon that provides authentication to Vault and
 145 manages token renewal and caching. It typically runs on the same host as the
 146 Object Gateway. With a Vault agent, it is possible to use other Vault
 147 authentication mechanism such as AppRole, AWS, Certs, JWT, and Azure.
 148
 149 The Object Gateway can be configured to use a Vault agent with the following
 150 settings::
 151
 152   rgw crypt vault auth = agent
 153   rgw crypt vault addr = http://localhost:8100
 154
 155 Vault namespaces
 156 ================
 157
 158 In the Enterprise version, Vault supports the concept of `namespaces`_, which
 159 allows centralized management for teams within an organization while ensuring
 160 that those teams operate within isolated environments known as tenants.
 161
 162 The Object Gateway can be configured to access Vault within a particular
 163 namespace using the following configuration setting::
 164
 165   rgw crypt vault namespace = tenant1
 166
 167 Create a key in Vault
 168 =====================
 169
 170 .. note:: Keys for server-side encryption must be 256-bit long and base-64
 171    encoded.
 172
 173 Using the KV engine
 174 -------------------
 175
 176 A key for server-side encryption can be created in the KV version 2 engine using
 177 the command line utility, as in the following example::
 178
 179   vault kv put secret/myproject/mybucketkey key=$(openssl rand -base64 32)
 180
 181 Sample output::
 182
 183   ====== Metadata ======
 184   Key              Value
 185   ---              -----
 186   created_time     2019-08-29T17:01:09.095824999Z
 187   deletion_time    n/a
 188   destroyed        false
 189   version          1
 190
 191 Note that in the KV secrets engine, secrets are stored as key-value pairs, and
 192 the Gateway expects the key name to be ``key``, i.e. the secret must be in the
 193 form ``key=<secret key>``.
 194
 195 Using the Transit engine
 196 ------------------------
 197
 198 Keys created with the Transit engine must be exportable in order to be used for
 199 server-side encryption with the Object Gateway. An exportable key can be created
 200 with the command line utility as follows::
 201
 202   vault write -f transit/keys/mybucketkey exportable=true
 203
 204 The command above creates a keyring, which contains a key of type
 205 ``aes256-gcm96`` by default. To verify that the key was correctly created, use
 206 the following command::
 207
 208   vault read transit/export/encryption-key/mybucketkey/1
 209
 210 Sample output::
 211
 212   Key     Value
 213   ---     -----
 214   keys    map[1:-gbTI9lNpqv/V/2lDcmH2Nq1xKn6FPDWarCmFM2aNsQ=]
 215   name    mybucketkey
 216   type    aes256-gcm96
 217
 218 Note that in order to read the key created with the Transit engine, the full
 219 path must be provided including the key version.
 220
 221 Configure the Ceph Object Gateway
 222 =================================
 223
 224 Edit the Ceph configuration file to enable Vault as a KMS backend for
 225 server-side encryption::
 226
 227   rgw crypt s3 kms backend = vault
 228
 229 Choose the Vault authentication method, e.g.::
 230
 231   rgw crypt vault auth = token
 232   rgw crypt vault token file = /etc/ceph/vault.token
 233   rgw crypt vault addr = http://vault-server:8200
 234
 235 Or::
 236
 237   rgw crypt vault auth = agent
 238   rgw crypt vault addr = http://localhost:8100
 239
 240 Choose the secrets engine::
 241
 242   rgw crypt vault secret engine = kv
 243
 244 Or::
 245
 246   rgw crypt vault secret engine = transit
 247
 248 Optionally, set the Vault namespace where encryption keys will be fetched from::
 249
 250   rgw crypt vault namespace = tenant1
 251
 252 Finally, the URLs where the Gateway will retrieve encryption keys from Vault can
 253 be restricted by setting a path prefix. For instance, the Gateway can be
 254 restricted to fetch KV keys as follows::
 255
 256   rgw crypt vault prefix = /v1/secret/data
 257
 258 Or, in the case of exportable transit keys::
 259
 260   rgw crypt vault prefix = /v1/transit/export/encryption-key
 261
 262 In the example above, the Gateway would only fetch transit encryption keys under
 263 ``http://vault-server:8200/v1/transit/export/encryption-key``.
 264
 265 Upload object
 266 =============
 267
 268 When uploading an object to the Gateway, provide the SSE key ID in the request.
 269 As an example, using the AWS command-line client::
 270
 271   aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey
 272
 273 The Object Gateway will fetch the key from Vault, encrypt the object and store
 274 it in the bucket. Any request to download the object will make the Gateway
 275 automatically retrieve the correspondent key from Vault and decrypt the object.
 276
 277 Note that the secret will be fetched from Vault using a URL constructed by
 278 concatenating the base address (``rgw crypt vault addr``), the (optional)
 279 URL prefix (``rgw crypt vault prefix``), and finally the key ID. In the example
 280 above, the Gateway would fetch the secret from::
 281
 282   http://vaultserver:8200/v1/secret/data/myproject/mybucketkey
 283
 284 .. _Server-Side Encryption: ../encryption
 285 .. _Vault: https://www.vaultproject.io/docs/
 286 .. _Token authentication method: https://www.vaultproject.io/docs/auth/token.html
 287 .. _Vault agent: https://www.vaultproject.io/docs/agent/index.html
 288 .. _KV Secrets engine: https://www.vaultproject.io/docs/secrets/kv/
 289 .. _Transit engine: https://www.vaultproject.io/docs/secrets/transit
 290 .. _namespaces: https://www.vaultproject.io/docs/enterprise/namespaces/index.html