]>
Commit | Line | Data |
---|---|---|
9f95a23c TL |
1 | =========================== |
2 | HashiCorp Vault Integration | |
3 | =========================== | |
4 | ||
5 | HashiCorp `Vault`_ can be used as a secure key management service for | |
6 | `Server-Side Encryption`_ (SSE-KMS). | |
7 | ||
f91f0fd5 TL |
8 | .. ditaa:: |
9 | ||
10 | +---------+ +---------+ +-------+ +-------+ | |
9f95a23c TL |
11 | | Client | | RadosGW | | Vault | | OSD | |
12 | +---------+ +---------+ +-------+ +-------+ | |
13 | | create secret | | | | |
14 | | key for key ID | | | | |
15 | |-----------------+---------------->| | | |
16 | | | | | | |
17 | | upload object | | | | |
18 | | with key ID | | | | |
19 | |---------------->| request secret | | | |
20 | | | key for key ID | | | |
21 | | |---------------->| | | |
22 | | |<----------------| | | |
23 | | | return secret | | | |
24 | | | key | | | |
25 | | | | | | |
26 | | | encrypt object | | | |
27 | | | with secret key | | | |
28 | | |--------------+ | | | |
29 | | | | | | | |
30 | | |<-------------+ | | | |
31 | | | | | | |
32 | | | store encrypted | | | |
33 | | | object | | | |
34 | | |------------------------------>| | |
35 | ||
36 | #. `Vault secrets engines`_ | |
37 | #. `Vault authentication`_ | |
38 | #. `Vault namespaces`_ | |
39 | #. `Create a key in Vault`_ | |
40 | #. `Configure the Ceph Object Gateway`_ | |
41 | #. `Upload object`_ | |
42 | ||
43 | Some examples below use the Vault command line utility to interact with | |
44 | Vault. You may need to set the following environment variable with the correct | |
45 | address of your Vault server to use this utility:: | |
46 | ||
f67539c2 | 47 | export VAULT_ADDR='https://vault-server-fqdn:8200' |
9f95a23c TL |
48 | |
49 | Vault secrets engines | |
50 | ===================== | |
51 | ||
52 | Vault provides several secrets engines, which can store, generate, and encrypt | |
53 | data. Currently, the Object Gateway supports: | |
54 | ||
55 | - `KV secrets engine`_ version 2 | |
56 | - `Transit engine`_ | |
57 | ||
58 | KV secrets engine | |
59 | ----------------- | |
60 | ||
61 | The KV secrets engine is used to store arbitrary key/value secrets in Vault. To | |
62 | enable the KV engine version 2 in Vault, use the following command:: | |
63 | ||
f67539c2 | 64 | vault secrets enable -path secret kv-v2 |
9f95a23c TL |
65 | |
66 | The Object Gateway can be configured to use the KV engine version 2 with the | |
67 | following setting:: | |
68 | ||
69 | rgw crypt vault secret engine = kv | |
70 | ||
71 | Transit secrets engine | |
72 | ---------------------- | |
73 | ||
74 | The transit engine handles cryptographic functions on data in-transit. To enable | |
75 | it in Vault, use the following command:: | |
76 | ||
77 | vault secrets enable transit | |
78 | ||
79 | The Object Gateway can be configured to use the transit engine with the | |
80 | following setting:: | |
81 | ||
82 | rgw crypt vault secret engine = transit | |
83 | ||
84 | Vault authentication | |
85 | ==================== | |
86 | ||
87 | Vault supports several authentication mechanisms. Currently, the Object | |
88 | Gateway can be configured to authenticate to Vault using the | |
89 | `Token authentication method`_ or a `Vault agent`_. | |
90 | ||
f67539c2 TL |
91 | Most tokens in Vault have limited lifetimes and powers. The only |
92 | sort of Vault token that does not have a lifetime are root tokens. | |
93 | For all other tokens, it is necesary to periodically refresh them, | |
94 | either by performing initial authentication, or by renewing the token. | |
95 | Ceph does not have any logic to perform either operation. | |
96 | The simplest best way to use Vault tokens with ceph is to | |
97 | also run the Vault agent and have it refresh the token file. | |
98 | When the Vault agent is used in this mode, file system permissions | |
99 | can be used to restrict who has the use of tokens. | |
100 | ||
101 | Instead of having Vault agent refresh a token file, it can be told | |
102 | to act as a proxy server. In this mode, Vault will add a token when | |
103 | necessary and add it to requests passed to it before forwarding them on | |
104 | to the real server. Vault agent will still handle token renewal just | |
105 | as it would when storing a token in the filesystem. In this mode, it | |
106 | is necessary to properly secure the network path rgw uses to reach the | |
107 | Vault agent, such as having the Vault agent listen only to localhost. | |
108 | ||
109 | Token policies for the object gateway | |
110 | ------------------------------------- | |
111 | ||
112 | All Vault tokens have powers as specified by the polices attached | |
113 | to that token. Multiple policies may be associated with one | |
114 | token. You should only use the policy necessary for your | |
115 | configuration. | |
116 | ||
117 | When using the kv secret engine with the object gateway:: | |
118 | ||
119 | vault policy write rgw-kv-policy -<<EOF | |
120 | path "secret/data/*" { | |
121 | capabilities = ["read"] | |
122 | } | |
123 | EOF | |
124 | ||
125 | When using the transit secret engine with the object gateway:: | |
126 | ||
127 | vault policy write rgw-transit-policy -<<EOF | |
128 | path "transit/keys/*" { | |
129 | capabilities = [ "create", "update" ] | |
130 | denied_parameters = {"exportable" = [], "allow_plaintext_backup" = [] } | |
131 | } | |
132 | ||
133 | path "transit/keys/*" { | |
134 | capabilities = ["read", "delete"] | |
135 | } | |
136 | ||
137 | path "transit/keys/" { | |
138 | capabilities = ["list"] | |
139 | } | |
140 | ||
141 | path "transit/keys/+/rotate" { | |
142 | capabilities = [ "update" ] | |
143 | } | |
144 | ||
145 | path "transit/*" { | |
146 | capabilities = [ "update" ] | |
147 | } | |
148 | EOF | |
149 | ||
150 | If you had previously used an older version of ceph with the | |
151 | transit secret engine, you might need the following policy:: | |
152 | ||
153 | vault policy write old-rgw-transit-policy -<<EOF | |
154 | path "transit/export/encryption-key/*" { | |
155 | capabilities = ["read"] | |
156 | } | |
157 | EOF | |
158 | ||
159 | ||
9f95a23c TL |
160 | Token authentication |
161 | -------------------- | |
162 | ||
f67539c2 | 163 | .. note: Never use root tokens with ceph in production environments. |
9f95a23c TL |
164 | |
165 | The token authentication method expects a Vault token to be present in a | |
166 | plaintext file. The Object Gateway can be configured to use token authentication | |
167 | with the following settings:: | |
168 | ||
169 | rgw crypt vault auth = token | |
f67539c2 TL |
170 | rgw crypt vault token file = /run/.rgw-vault-token |
171 | rgw crypt vault addr = https://vault-server-fqdn:8200 | |
9f95a23c | 172 | |
f67539c2 | 173 | Adjust these settinsg to match your configuration. |
9f95a23c | 174 | For security reasons, the token file must be readable by the Object Gateway |
f67539c2 | 175 | only. |
9f95a23c | 176 | |
f67539c2 | 177 | You might set up vault agent as follows:: |
9f95a23c | 178 | |
f67539c2 TL |
179 | vault write auth/approle/role/rgw-ap \ |
180 | token_policies=rgw-transit-policy,default \ | |
181 | token_max_ttl=60m | |
9f95a23c | 182 | |
f67539c2 | 183 | Change the policy here to match your configuration. |
9f95a23c | 184 | |
f67539c2 | 185 | Get the role-id:: |
9f95a23c | 186 | |
f67539c2 TL |
187 | vault read auth/approle/role/rgw-ap/role-id -format=json | \ |
188 | jq -r .data.role_id | |
189 | ||
190 | Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-role-id | |
191 | ||
192 | Get the secret-id:: | |
193 | ||
194 | vault read auth/approle/role/rgw-ap/role-id -format=json | \ | |
195 | jq -r .data.role_id | |
196 | ||
197 | Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-secret-id | |
198 | ||
199 | Create configuration for the Vault agent, such as:: | |
200 | ||
201 | pid_file = "/run/rgw-vault-agent-pid" | |
202 | auto_auth { | |
203 | method "AppRole" { | |
204 | mount_path = "auth/approle" | |
205 | config = { | |
206 | role_id_file_path ="/usr/local/etc/vault/.rgw-ap-role-id" | |
207 | secret_id_file_path ="/usr/local/etc/vault/.rgw-ap-secret-id" | |
208 | remove_secret_id_file_after_reading ="false" | |
209 | } | |
210 | } | |
211 | sink "file" { | |
212 | config = { | |
213 | path = "/run/.rgw-vault-token" | |
214 | } | |
215 | } | |
216 | } | |
217 | vault { | |
218 | address = "https://vault-server-fqdn:8200" | |
219 | } | |
220 | ||
221 | Then use systemctl or another method of your choice to run | |
222 | a persistent daemon with the following arguments:: | |
9f95a23c | 223 | |
f67539c2 | 224 | /usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl |
9f95a23c | 225 | |
f67539c2 TL |
226 | Once the vault agent is running, the token file should be populated |
227 | with a valid token. | |
9f95a23c TL |
228 | |
229 | Vault agent | |
230 | ----------- | |
231 | ||
232 | The Vault agent is a client daemon that provides authentication to Vault and | |
233 | manages token renewal and caching. It typically runs on the same host as the | |
234 | Object Gateway. With a Vault agent, it is possible to use other Vault | |
235 | authentication mechanism such as AppRole, AWS, Certs, JWT, and Azure. | |
236 | ||
237 | The Object Gateway can be configured to use a Vault agent with the following | |
238 | settings:: | |
239 | ||
240 | rgw crypt vault auth = agent | |
f67539c2 TL |
241 | rgw crypt vault addr = http://127.0.0.1:8100 |
242 | ||
243 | You might set up vault agent as follows:: | |
244 | ||
245 | vault write auth/approle/role/rgw-ap \ | |
246 | token_policies=rgw-transit-policy,default \ | |
247 | token_max_ttl=60m | |
248 | ||
249 | Change the policy here to match your configuration. | |
250 | ||
251 | Get the role-id: | |
252 | vault read auth/approle/role/rgw-ap/role-id -format=json | \ | |
253 | jq -r .data.role_id | |
254 | ||
255 | Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-role-id | |
256 | ||
257 | Get the secret-id: | |
258 | vault read auth/approle/role/rgw-ap/role-id -format=json | \ | |
259 | jq -r .data.role_id | |
260 | ||
261 | Store the output in some file, such as /usr/local/etc/vault/.rgw-ap-secret-id | |
262 | ||
263 | Create configuration for the Vault agent, such as:: | |
264 | ||
265 | pid_file = "/run/rgw-vault-agent-pid" | |
266 | auto_auth { | |
267 | method "AppRole" { | |
268 | mount_path = "auth/approle" | |
269 | config = { | |
270 | role_id_file_path ="/usr/local/etc/vault/.rgw-ap-role-id" | |
271 | secret_id_file_path ="/usr/local/etc/vault/.rgw-ap-secret-id" | |
272 | remove_secret_id_file_after_reading ="false" | |
273 | } | |
274 | } | |
275 | } | |
276 | cache { | |
277 | use_auto_auth_token = true | |
278 | } | |
279 | listener "tcp" { | |
280 | address = "127.0.0.1:8100" | |
281 | tls_disable = true | |
282 | } | |
283 | vault { | |
284 | address = "https://vault-server-fqdn:8200" | |
285 | } | |
286 | ||
287 | Then use systemctl or another method of your choice to run | |
288 | a persistent daemon with the following arguments:: | |
289 | ||
290 | /usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl | |
291 | ||
292 | Once the vault agent is running, you should find it listening | |
293 | to port 8100 on localhost, and you should be able to interact | |
294 | with it using the vault command. | |
9f95a23c TL |
295 | |
296 | Vault namespaces | |
297 | ================ | |
298 | ||
299 | In the Enterprise version, Vault supports the concept of `namespaces`_, which | |
300 | allows centralized management for teams within an organization while ensuring | |
301 | that those teams operate within isolated environments known as tenants. | |
302 | ||
303 | The Object Gateway can be configured to access Vault within a particular | |
304 | namespace using the following configuration setting:: | |
305 | ||
306 | rgw crypt vault namespace = tenant1 | |
307 | ||
308 | Create a key in Vault | |
309 | ===================== | |
310 | ||
311 | .. note:: Keys for server-side encryption must be 256-bit long and base-64 | |
312 | encoded. | |
313 | ||
314 | Using the KV engine | |
315 | ------------------- | |
316 | ||
317 | A key for server-side encryption can be created in the KV version 2 engine using | |
318 | the command line utility, as in the following example:: | |
319 | ||
320 | vault kv put secret/myproject/mybucketkey key=$(openssl rand -base64 32) | |
321 | ||
322 | Sample output:: | |
323 | ||
324 | ====== Metadata ====== | |
325 | Key Value | |
326 | --- ----- | |
327 | created_time 2019-08-29T17:01:09.095824999Z | |
328 | deletion_time n/a | |
329 | destroyed false | |
330 | version 1 | |
331 | ||
332 | Note that in the KV secrets engine, secrets are stored as key-value pairs, and | |
333 | the Gateway expects the key name to be ``key``, i.e. the secret must be in the | |
334 | form ``key=<secret key>``. | |
335 | ||
336 | Using the Transit engine | |
337 | ------------------------ | |
338 | ||
f67539c2 TL |
339 | Keys created for use with the Transit engine should no longer be marked |
340 | exportable. They can be created with:: | |
9f95a23c | 341 | |
f67539c2 | 342 | vault write -f transit/keys/mybucketkey |
9f95a23c TL |
343 | |
344 | The command above creates a keyring, which contains a key of type | |
345 | ``aes256-gcm96`` by default. To verify that the key was correctly created, use | |
346 | the following command:: | |
347 | ||
f67539c2 | 348 | vault read transit/mybucketkey |
9f95a23c TL |
349 | |
350 | Sample output:: | |
351 | ||
352 | Key Value | |
353 | --- ----- | |
f67539c2 TL |
354 | derived false |
355 | exportable false | |
356 | name mybucketkey | |
357 | type aes256-gcm96 | |
9f95a23c TL |
358 | |
359 | Configure the Ceph Object Gateway | |
360 | ================================= | |
361 | ||
362 | Edit the Ceph configuration file to enable Vault as a KMS backend for | |
363 | server-side encryption:: | |
364 | ||
365 | rgw crypt s3 kms backend = vault | |
366 | ||
367 | Choose the Vault authentication method, e.g.:: | |
368 | ||
369 | rgw crypt vault auth = token | |
f67539c2 TL |
370 | rgw crypt vault token file = /run/.rgw-vault-token |
371 | rgw crypt vault addr = https://vault-server-fqdn:8200 | |
9f95a23c TL |
372 | |
373 | Or:: | |
374 | ||
375 | rgw crypt vault auth = agent | |
376 | rgw crypt vault addr = http://localhost:8100 | |
377 | ||
378 | Choose the secrets engine:: | |
379 | ||
380 | rgw crypt vault secret engine = kv | |
381 | ||
382 | Or:: | |
383 | ||
384 | rgw crypt vault secret engine = transit | |
385 | ||
386 | Optionally, set the Vault namespace where encryption keys will be fetched from:: | |
387 | ||
388 | rgw crypt vault namespace = tenant1 | |
389 | ||
390 | Finally, the URLs where the Gateway will retrieve encryption keys from Vault can | |
391 | be restricted by setting a path prefix. For instance, the Gateway can be | |
392 | restricted to fetch KV keys as follows:: | |
393 | ||
394 | rgw crypt vault prefix = /v1/secret/data | |
395 | ||
f67539c2 | 396 | Or, when using the transit secret engine:: |
9f95a23c | 397 | |
f67539c2 | 398 | rgw crypt vault prefix = /v1/transit |
9f95a23c TL |
399 | |
400 | In the example above, the Gateway would only fetch transit encryption keys under | |
f67539c2 TL |
401 | ``https://vault-server:8200/v1/transit``. |
402 | ||
403 | Transit engine compatibility support | |
404 | ------------------------------------ | |
405 | The transit engine has compatibility support for previous | |
406 | versions of ceph, which used the transit engine as a simple key store. | |
407 | ||
408 | There is a a "compat" option which can be given to the transit | |
409 | engine to configure the compatibility support, | |
410 | ||
411 | To entirely disable backwards support, use:: | |
412 | ||
413 | rgw crypt vault secret engine = transit compat=0 | |
414 | ||
415 | This will be the default in future verisons. and is safe to use | |
416 | for new installs using the current version. | |
417 | ||
418 | This is the normal default with the current version:: | |
419 | ||
420 | rgw crypt vault secret engine = transit compat=1 | |
421 | ||
422 | This enables the new engine for newly created objects, | |
423 | but still allows the old engine to be used for old objects. | |
424 | In order to access old and new objects, the vault token given | |
425 | to ceph must have both the old and new transit policies. | |
426 | ||
427 | To force use of only the old engine, use:: | |
428 | ||
429 | rgw crypt vault secret engine = transit compat=2 | |
430 | ||
431 | This mode is automatically selected if the vault prefix | |
432 | ends in export/encryption-key, which was the previously | |
433 | documented setting. | |
9f95a23c TL |
434 | |
435 | Upload object | |
436 | ============= | |
437 | ||
438 | When uploading an object to the Gateway, provide the SSE key ID in the request. | |
f67539c2 | 439 | As an example, for the kv engine, using the AWS command-line client:: |
9f95a23c TL |
440 | |
441 | aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey | |
f67539c2 TL |
442 | |
443 | As an example, for the transit engine (new flavor), using the AWS command-line client:: | |
444 | ||
445 | aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id mybucketkey | |
9f95a23c TL |
446 | |
447 | The Object Gateway will fetch the key from Vault, encrypt the object and store | |
448 | it in the bucket. Any request to download the object will make the Gateway | |
449 | automatically retrieve the correspondent key from Vault and decrypt the object. | |
450 | ||
451 | Note that the secret will be fetched from Vault using a URL constructed by | |
452 | concatenating the base address (``rgw crypt vault addr``), the (optional) | |
f67539c2 TL |
453 | URL prefix (``rgw crypt vault prefix``), and finally the key ID. |
454 | ||
455 | In the kv engine example above, the Gateway would fetch the secret from:: | |
9f95a23c TL |
456 | |
457 | http://vaultserver:8200/v1/secret/data/myproject/mybucketkey | |
458 | ||
f67539c2 TL |
459 | In the transit engine example above, the Gateway would encrypt the secret using this key:: |
460 | ||
461 | http://vaultserver:8200/v1/transit/mybucketkey | |
462 | ||
9f95a23c TL |
463 | .. _Server-Side Encryption: ../encryption |
464 | .. _Vault: https://www.vaultproject.io/docs/ | |
465 | .. _Token authentication method: https://www.vaultproject.io/docs/auth/token.html | |
466 | .. _Vault agent: https://www.vaultproject.io/docs/agent/index.html | |
467 | .. _KV Secrets engine: https://www.vaultproject.io/docs/secrets/kv/ | |
468 | .. _Transit engine: https://www.vaultproject.io/docs/secrets/transit | |
469 | .. _namespaces: https://www.vaultproject.io/docs/enterprise/namespaces/index.html |