ceph/doc/radosgw/vault.rst

   1 ===========================
   2 HashiCorp Vault Integration
   3 ===========================
   4
   5 HashiCorp `Vault`_ can be used as a secure key management service for
   6 `Server-Side Encryption`_ (SSE-KMS).
   7
   8 .. ditaa:: +---------+       +---------+        +-------+     +-------+
   9            |  Client |       | RadosGW |        | Vault |     |  OSD  |
  10            +---------+       +---------+        +-------+     +-------+
  11                 | create secret   |                 |             |
  12                 | key for key ID  |                 |             |
  13                 |-----------------+---------------->|             |
  14                 |                 |                 |             |
  15                 | upload object   |                 |             |
  16                 | with key ID     |                 |             |
  17                 |---------------->| request secret  |             |
  18                 |                 | key for key ID  |             |
  19                 |                 |---------------->|             |
  20                 |                 |<----------------|             |
  21                 |                 | return secret   |             |
  22                 |                 | key             |             |
  23                 |                 |                 |             |
  24                 |                 | encrypt object  |             |
  25                 |                 | with secret key |             |
  26                 |                 |--------------+  |             |
  27                 |                 |              |  |             |
  28                 |                 |<-------------+  |             |
  29                 |                 |                 |             |
  30                 |                 | store encrypted |             |
  31                 |                 | object          |             |
  32                 |                 |------------------------------>|
  33
  34 #. `Vault secrets engines`_
  35 #. `Vault authentication`_
  36 #. `Vault namespaces`_
  37 #. `Create a key in Vault`_
  38 #. `Configure the Ceph Object Gateway`_
  39 #. `Upload object`_
  40
  41 Some examples below use the Vault command line utility to interact with
  42 Vault. You may need to set the following environment variable with the correct
  43 address of your Vault server to use this utility::
  44
  45   export VAULT_ADDR='http://vault-server:8200'
  46
  47 Vault secrets engines
  48 =====================
  49
  50 Vault provides several secrets engines, which can store, generate, and encrypt
  51 data. Currently, the Object Gateway supports:
  52
  53 - `KV secrets engine`_ version 2
  54 - `Transit engine`_
  55
  56 KV secrets engine
  57 -----------------
  58
  59 The KV secrets engine is used to store arbitrary key/value secrets in Vault. To
  60 enable the KV engine version 2 in Vault, use the following command::
  61
  62   vault secrets enable kv-v2
  63
  64 The Object Gateway can be configured to use the KV engine version 2 with the
  65 following setting::
  66
  67   rgw crypt vault secret engine = kv
  68
  69 Transit secrets engine
  70 ----------------------
  71
  72 The transit engine handles cryptographic functions on data in-transit. To enable
  73 it in Vault, use the following command::
  74
  75   vault secrets enable transit
  76
  77 The Object Gateway can be configured to use the transit engine with the
  78 following setting::
  79
  80   rgw crypt vault secret engine = transit
  81
  82 Vault authentication
  83 ====================
  84
  85 Vault supports several authentication mechanisms. Currently, the Object
  86 Gateway can be configured to authenticate to Vault using the
  87 `Token authentication method`_ or a `Vault agent`_.
  88
  89 Token authentication
  90 --------------------
  91
  92 .. note:: Token authentication is not recommended for production environments.
  93
  94 The token authentication method expects a Vault token to be present in a
  95 plaintext file. The Object Gateway can be configured to use token authentication
  96 with the following settings::
  97
  98   rgw crypt vault auth = token
  99   rgw crypt vault token file = /etc/ceph/vault.token
 100   rgw crypt vault addr = http://vault-server:8200
 101
 102 For security reasons, the token file must be readable by the Object Gateway
 103 only. Also, the Object Gateway should be given a Vault token with a restricted
 104 policy that allows it to fetch keyrings from a specific path only. Such a policy
 105 can be created in Vault using the command line utility as in the following
 106 examples::
 107
 108   vault policy write rgw-kv-policy -<<EOF
 109     path "secret/data/*" {
 110       capabilities = ["read"]
 111     }
 112   EOF
 113
 114   vault policy write rgw-transit-policy -<<EOF
 115     path "transit/export/encryption-key/*" {
 116       capabilities = ["read"]
 117     }
 118   EOF
 119
 120 Once the policy is created, a token can be generated by a Vault administrator::
 121
 122   vault token create -policy=rgw-kv-policy
 123
 124 Sample output::
 125
 126   Key                  Value
 127   ---                  -----
 128   token                s.72KuPujbc065OdWB71poOmIq
 129   token_accessor       jv95ZYBUFv6Ss84x7SCSy6lZ
 130   token_duration       768h
 131   token_renewable      true
 132   token_policies       ["default" "rgw-kv-policy"]
 133   identity_policies    []
 134   policies             ["default" "rgw-kv-policy"]
 135
 136 The actual token, displayed in the ``Value`` column of the first line of the
 137 output, must be saved in a file as plaintext.
 138
 139 Vault agent
 140 -----------
 141
 142 The Vault agent is a client daemon that provides authentication to Vault and
 143 manages token renewal and caching. It typically runs on the same host as the
 144 Object Gateway. With a Vault agent, it is possible to use other Vault
 145 authentication mechanism such as AppRole, AWS, Certs, JWT, and Azure.
 146
 147 The Object Gateway can be configured to use a Vault agent with the following
 148 settings::
 149
 150   rgw crypt vault auth = agent
 151   rgw crypt vault addr = http://localhost:8100
 152
 153 Vault namespaces
 154 ================
 155
 156 In the Enterprise version, Vault supports the concept of `namespaces`_, which
 157 allows centralized management for teams within an organization while ensuring
 158 that those teams operate within isolated environments known as tenants.
 159
 160 The Object Gateway can be configured to access Vault within a particular
 161 namespace using the following configuration setting::
 162
 163   rgw crypt vault namespace = tenant1
 164
 165 Create a key in Vault
 166 =====================
 167
 168 .. note:: Keys for server-side encryption must be 256-bit long and base-64
 169    encoded.
 170
 171 Using the KV engine
 172 -------------------
 173
 174 A key for server-side encryption can be created in the KV version 2 engine using
 175 the command line utility, as in the following example::
 176
 177   vault kv put secret/myproject/mybucketkey key=$(openssl rand -base64 32)
 178
 179 Sample output::
 180
 181   ====== Metadata ======
 182   Key              Value
 183   ---              -----
 184   created_time     2019-08-29T17:01:09.095824999Z
 185   deletion_time    n/a
 186   destroyed        false
 187   version          1
 188
 189 Note that in the KV secrets engine, secrets are stored as key-value pairs, and
 190 the Gateway expects the key name to be ``key``, i.e. the secret must be in the
 191 form ``key=<secret key>``.
 192
 193 Using the Transit engine
 194 ------------------------
 195
 196 Keys created with the Transit engine must be exportable in order to be used for
 197 server-side encryption with the Object Gateway. An exportable key can be created
 198 with the command line utility as follows::
 199
 200   vault write -f transit/keys/mybucketkey exportable=true
 201
 202 The command above creates a keyring, which contains a key of type
 203 ``aes256-gcm96`` by default. To verify that the key was correctly created, use
 204 the following command::
 205
 206   vault read transit/export/encryption-key/mybucketkey/1
 207
 208 Sample output::
 209
 210   Key     Value
 211   ---     -----
 212   keys    map[1:-gbTI9lNpqv/V/2lDcmH2Nq1xKn6FPDWarCmFM2aNsQ=]
 213   name    mybucketkey
 214   type    aes256-gcm96
 215
 216 Note that in order to read the key created with the Transit engine, the full
 217 path must be provided including the key version.
 218
 219 Configure the Ceph Object Gateway
 220 =================================
 221
 222 Edit the Ceph configuration file to enable Vault as a KMS backend for
 223 server-side encryption::
 224
 225   rgw crypt s3 kms backend = vault
 226
 227 Choose the Vault authentication method, e.g.::
 228
 229   rgw crypt vault auth = token
 230   rgw crypt vault token file = /etc/ceph/vault.token
 231   rgw crypt vault addr = http://vault-server:8200
 232
 233 Or::
 234
 235   rgw crypt vault auth = agent
 236   rgw crypt vault addr = http://localhost:8100
 237
 238 Choose the secrets engine::
 239
 240   rgw crypt vault secret engine = kv
 241
 242 Or::
 243
 244   rgw crypt vault secret engine = transit
 245
 246 Optionally, set the Vault namespace where encryption keys will be fetched from::
 247
 248   rgw crypt vault namespace = tenant1
 249
 250 Finally, the URLs where the Gateway will retrieve encryption keys from Vault can
 251 be restricted by setting a path prefix. For instance, the Gateway can be
 252 restricted to fetch KV keys as follows::
 253
 254   rgw crypt vault prefix = /v1/secret/data
 255
 256 Or, in the case of exportable transit keys::
 257
 258   rgw crypt vault prefix = /v1/transit/export/encryption-key
 259
 260 In the example above, the Gateway would only fetch transit encryption keys under
 261 ``http://vault-server:8200/v1/transit/export/encryption-key``.
 262
 263 Upload object
 264 =============
 265
 266 When uploading an object to the Gateway, provide the SSE key ID in the request.
 267 As an example, using the AWS command-line client::
 268
 269   aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey
 270
 271 The Object Gateway will fetch the key from Vault, encrypt the object and store
 272 it in the bucket. Any request to download the object will make the Gateway
 273 automatically retrieve the correspondent key from Vault and decrypt the object.
 274
 275 Note that the secret will be fetched from Vault using a URL constructed by
 276 concatenating the base address (``rgw crypt vault addr``), the (optional)
 277 URL prefix (``rgw crypt vault prefix``), and finally the key ID. In the example
 278 above, the Gateway would fetch the secret from::
 279
 280   http://vaultserver:8200/v1/secret/data/myproject/mybucketkey
 281
 282 .. _Server-Side Encryption: ../encryption
 283 .. _Vault: https://www.vaultproject.io/docs/
 284 .. _Token authentication method: https://www.vaultproject.io/docs/auth/token.html
 285 .. _Vault agent: https://www.vaultproject.io/docs/agent/index.html
 286 .. _KV Secrets engine: https://www.vaultproject.io/docs/secrets/kv/
 287 .. _Transit engine: https://www.vaultproject.io/docs/secrets/transit
 288 .. _namespaces: https://www.vaultproject.io/docs/enterprise/namespaces/index.html