certificate-management.adoc

   1 [[sysadmin_certificate_management]]
   2 Certificate Management
   3 ----------------------
   4 ifdef::wiki[]
   5 :pve-toplevel:
   6 endif::wiki[]
   7
   8
   9 Certificates for Intra-Cluster Communication
  10 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  11
  12 Each {PVE} cluster creates by default its own (self-signed) Certificate
  13 Authority (CA) and generates a certificate for each node which gets signed by
  14 the aforementioned CA. These certificates are used for encrypted communication
  15 with the cluster's `pveproxy` service and the Shell/Console feature if SPICE is
  16 used.
  17
  18 The CA certificate and key are stored in the xref:chapter_pmxcfs[Proxmox Cluster File System (pmxcfs)].
  19
  20
  21 [[sysadmin_certs_api_gui]]
  22 Certificates for API and Web GUI
  23 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  24
  25 The REST API and web GUI are provided by the `pveproxy` service, which runs on
  26 each node.
  27
  28 You have the following options for the certificate used by `pveproxy`:
  29
  30 1. By default the node-specific certificate in
  31 `/etc/pve/nodes/NODENAME/pve-ssl.pem` is used. This certificate is signed by
  32 the cluster CA and therefore not automatically trusted by browsers and
  33 operating systems.
  34 2. use an externally provided certificate (e.g. signed by a commercial CA).
  35 3. use ACME (Let's Encrypt) to get a trusted certificate with automatic
  36 renewal, this is also integrated in the {pve} API and Webinterface.
  37
  38 For options 2 and 3 the file `/etc/pve/local/pveproxy-ssl.pem` (and
  39 `/etc/pve/local/pveproxy-ssl.key`, which needs to be without password) is used.
  40
  41 NOTE: Keep in mind that `/etc/pve/local` is a node specific symlink to
  42 `/etc/pve/nodes/NODENAME`.
  43
  44 Certificates are managed with the {PVE} Node management command
  45 (see the `pvenode(1)` manpage).
  46
  47 WARNING: Do not replace or manually modify the automatically generated node
  48 certificate files in `/etc/pve/local/pve-ssl.pem` and
  49 `/etc/pve/local/pve-ssl.key` or the cluster CA files in
  50 `/etc/pve/pve-root-ca.pem` and `/etc/pve/priv/pve-root-ca.key`.
  51
  52
  53 [[sysadmin_certs_get_trusted_acme_cert]]
  54 Trusted certificates via Let's Encrypt (ACME)
  55 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  56
  57 {PVE} includes an implementation of the **A**utomatic **C**ertificate
  58 **M**anagement **E**nvironment **ACME** protocol, allowing {pve} admins to
  59 interface with Let's Encrypt for easy setup of trusted TLS certificates which
  60 are accepted out of the box on most modern operating systems and browsers.
  61
  62 Currently the two ACME endpoints implemented are the
  63 https://letsencrypt.org[Let's Encrypt (LE)] production and its staging
  64 environment. Our ACME client supports validation of `http-01` challenges using
  65 a built-in webserver and validation of `dns-01` challenges using a DNS plugin
  66 supporting all the DNS API endpoints https://acme.sh[acme.sh] does.
  67
  68 [[sysadmin_certs_acme_account]]
  69 ACME Account
  70 ^^^^^^^^^^^^
  71 You need to register an ACME account per cluster with the endpoint you want to
  72 use. The email address used for that account will server as contact point for
  73 renewal-due or similar notifications from the ACME endpoint.
  74
  75 // TODO: screenshot of account register here
  76
  77 You can register and deactivate ACME accounts over the web interface
  78 `Datacenter -> ACME` or using the `pvenode` command line tool.
  79 ----
  80  pvenode acme account register account-name mail@example.com
  81 ----
  82
  83 TIP: Because of https://letsencrypt.org/docs/rate-limits/[rate-limits] you
  84 should use LE `staging` for experiments or if you use ACME for the first time.
  85
  86 [[sysadmin_certs_acme_plugins]]
  87 ACME Plugins
  88 ^^^^^^^^^^^^
  89
  90 The ACME plugins task is to provide automatic verification that you, and thus
  91 the {pve} cluster under your operation, are the real owner of a domain. This is
  92 the basis building block for automatic certificate management.
  93
  94 The ACME protocol specifies different types of challenges, for example the
  95 `http-01` where a webserver provides a file with a certain value to proof that
  96 it controls a domain. Sometimes this isn't possible, either because of
  97 technical limitations or if the address a domain points to is not reachable
  98 from the public internet. For such cases one could use the `dns-01` challenge.
  99 That challenge provides also a certain value, but not over a text file, but
 100 through a DNS record on the authority name server of the domain.
 101
 102 {pve} supports both of those challenge types out of the box, you can configure
 103 plugins either over the web interface under `Datacenter -> ACME`, or using the
 104 `pvenode acme plugin add` command.
 105
 106 ACME Plugin configurations are stored in `/etc/pve/priv/acme/plugins.cfg`.
 107
 108 [[sysadmin_certs_acme_http_challenge]]
 109 ACME HTTP Challenge Plugin
 110 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 111
 112 There is always an implicitly configured `standalone` plugin for validating
 113 `http-01` challenges via the built-in webserver spawned on port 80.
 114
 115 NOTE: The name `standalone` means that it can provide the validation on it's
 116 own, without any third party service. So, this plugin works also for cluster
 117 nodes.
 118
 119 There are a few prerequisites to use it for certificate management with Let's
 120 Encrypts ACME.
 121
 122 * You have to accept the ToS of Let's Encrypt to register an account.
 123 * **Port 80** of the node needs to be reachable from the internet.
 124 * There **must** be no other listener on port 80.
 125 * The requested (sub)domain needs to resolve to a public IP of the Node.
 126
 127
 128 [[sysadmin_certs_acme_dns_challenge]]
 129 ACME DNS API Challenge Plugin
 130 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 131
 132 On systems where external access for validation via the `http-01` method is
 133 not possible or desired, it is possible to use the `dns-01` validation method.
 134 This validation method requires a DNS server that allows provisioning of `TXT`
 135 records via an API.
 136
 137 [[sysadmin_certs_acme_dns_api_config]]
 138 Configuring ACME DNS APIs for validation
 139 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 140
 141 {PVE} re-uses the DNS plugins developed for the `acme.sh`
 142 footnote:[acme.sh https://github.com/acmesh-official/acme.sh]
 143 project, please refer to its documentation for details on configuration of
 144 specific APIs.
 145
 146 The easiest way to configure a new plugin with the DNS API is using the web
 147 interface (`Datacenter -> ACME`).
 148
 149 Choose `DNS` as challenge type. Then you can select your API provider, enter
 150 the credential data to access your account over their API.
 151
 152 TIP: See the acme.sh
 153 https://github.com/acmesh-official/acme.sh/wiki/dnsapi#how-to-use-dns-api[How to use DNS API]
 154 wiki for more detailed information about getting API credentials for your
 155 provider.
 156
 157 As there are so many API endpoints {pve} autogenerates the formular for the
 158 credentials, but not all providers are annotated yet. For those you will see a
 159 bigger text area, simply copy all the credentials `KEY`=`VALUE` pairs in there.
 160
 161 DNS Validation through CNAME Alias
 162 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 163
 164 A special `alias` mode can be used to handle the validation on a different
 165 domain/DNS server, in case your primary/real DNS does not support provisioning
 166 via an API. Manually set up a permanent `CNAME` record for
 167 `_acme-challenge.domain1.example` pointing to `_acme-challenge.domain2.example`
 168 and set the `alias` property in the {PVE} node configuration file to
 169 `domain2.example` to allow the DNS server of `domain2.example` to validate all
 170 challenges for `domain1.example`.
 171
 172
 173 Combination of Plugins
 174 ^^^^^^^^^^^^^^^^^^^^^^
 175
 176 Combining `http-01` and `dns-01` validation is possible in case your node is
 177 reachable via multiple domains with different requirements / DNS provisioning
 178 capabilities. Mixing DNS APIs from multiple providers or instances is also
 179 possible by specifying different plugin instances per domain.
 180
 181 TIP: Accessing the same service over multiple domains increases complexity and
 182 should be avoided if possible.
 183
 184 [[sysadmin_certs_acme_automatic_renewal]]
 185 Automatic renewal of ACME certificates
 186 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 187
 188 If a node has been successfully configured with an ACME-provided certificate
 189 (either via pvenode or via the GUI), the certificate will be automatically
 190 renewed by the pve-daily-update.service. Currently, renewal will be attempted
 191 if the certificate has expired already, or will expire in the next 30 days.
 192
 193
 194 ACME Examples with `pvenode`
 195 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 196
 197 Example: Sample `pvenode` invocation for using Let's Encrypt certificates
 198 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 199
 200 ----
 201 root@proxmox:~# pvenode acme account register default mail@example.invalid
 202 Directory endpoints:
 203 0) Let's Encrypt V2 (https://acme-v02.api.letsencrypt.org/directory)
 204 1) Let's Encrypt V2 Staging (https://acme-staging-v02.api.letsencrypt.org/directory)
 205 2) Custom
 206 Enter selection: 1
 207
 208 Terms of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
 209 Do you agree to the above terms? [y|N]y
 210 ...
 211 Task OK
 212 root@proxmox:~# pvenode config set --acme domains=example.invalid
 213 root@proxmox:~# pvenode acme cert order
 214 Loading ACME account details
 215 Placing ACME order
 216 ...
 217 Status is 'valid'!
 218
 219 All domains validated!
 220 ...
 221 Downloading certificate
 222 Setting pveproxy certificate and key
 223 Restarting pveproxy
 224 Task OK
 225 ----
 226
 227 Example: Setting up the OVH API for validating a domain
 228 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 229
 230 NOTE: the account registration steps are the same no matter which plugins are
 231 used, and are not repeated here.
 232
 233 NOTE: `OVH_AK` and `OVH_AS` need to be obtained from OVH according to the OVH
 234 API documentation
 235
 236
 237 First you need to get all information so you and {pve} can access the API.
 238
 239 ----
 240 root@proxmox:~# cat /path/to/api-token
 241 OVH_AK=XXXXXXXXXXXXXXXX
 242 OVH_AS=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
 243 root@proxmox:~# source /path/to/api-token
 244 root@proxmox:~# curl -XPOST -H"X-Ovh-Application: $OVH_AK" -H "Content-type: application/json" \
 245 https://eu.api.ovh.com/1.0/auth/credential  -d '{
 246   "accessRules": [
 247     {"method": "GET","path": "/auth/time"},
 248     {"method": "GET","path": "/domain"},
 249     {"method": "GET","path": "/domain/zone/*"},
 250     {"method": "GET","path": "/domain/zone/*/record"},
 251     {"method": "POST","path": "/domain/zone/*/record"},
 252     {"method": "POST","path": "/domain/zone/*/refresh"},
 253     {"method": "PUT","path": "/domain/zone/*/record/"},
 254     {"method": "DELETE","path": "/domain/zone/*/record/*"}
 255 ]
 256 }'
 257 {"consumerKey":"ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ","state":"pendingValidation","validationUrl":"https://eu.api.ovh.com/auth/?credentialToken=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}
 258
 259 (open validation URL and follow instructions to link Application Key with account/Consumer Key)
 260
 261 root@proxmox:~# echo "OVH_CK=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ" >> /path/to/api-token
 262 ----
 263
 264 Now you can setup the the ACME plugin:
 265
 266 ----
 267 root@proxmox:~# pvenode acme plugin add dns example_plugin --api ovh --data /path/to/api_token
 268 root@proxmox:~# pvenode acme plugin config example_plugin
 269 ┌────────┬──────────────────────────────────────────┐
 270 │ key    │ value                                    │
 271 ╞════════╪══════════════════════════════════════════╡
 272 │ api    │ ovh                                      │
 273 ├────────┼──────────────────────────────────────────┤
 274 │ data   │ OVH_AK=XXXXXXXXXXXXXXXX                  │
 275 │        │ OVH_AS=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY  │
 276 │        │ OVH_CK=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ  │
 277 ├────────┼──────────────────────────────────────────┤
 278 │ digest │ 867fcf556363ca1bea866863093fcab83edf47a1 │
 279 ├────────┼──────────────────────────────────────────┤
 280 │ plugin │ example_plugin                           │
 281 ├────────┼──────────────────────────────────────────┤
 282 │ type   │ dns                                      │
 283 └────────┴──────────────────────────────────────────┘
 284 ----
 285
 286 At last you can configure the domain you want to get certitficates for and
 287 place the certificate order for it:
 288
 289 ----
 290 root@proxmox:~# pvenode config set -acmedomain0 example.proxmox.com,plugin=example_plugin
 291 root@proxmox:~# pvenode acme cert order
 292 Loading ACME account details
 293 Placing ACME order
 294 Order URL: https://acme-staging-v02.api.letsencrypt.org/acme/order/11111111/22222222
 295
 296 Getting authorization details from 'https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/33333333'
 297 The validation for example.proxmox.com is pending!
 298 [Wed Apr 22 09:25:30 CEST 2020] Using OVH endpoint: ovh-eu
 299 [Wed Apr 22 09:25:30 CEST 2020] Checking authentication
 300 [Wed Apr 22 09:25:30 CEST 2020] Consumer key is ok.
 301 [Wed Apr 22 09:25:31 CEST 2020] Adding record
 302 [Wed Apr 22 09:25:32 CEST 2020] Added, sleep 10 seconds.
 303 Add TXT record: _acme-challenge.example.proxmox.com
 304 Triggering validation
 305 Sleeping for 5 seconds
 306 Status is 'valid'!
 307 [Wed Apr 22 09:25:48 CEST 2020] Using OVH endpoint: ovh-eu
 308 [Wed Apr 22 09:25:48 CEST 2020] Checking authentication
 309 [Wed Apr 22 09:25:48 CEST 2020] Consumer key is ok.
 310 Remove TXT record: _acme-challenge.example.proxmox.com
 311
 312 All domains validated!
 313
 314 Creating CSR
 315 Checking order status
 316 Order is ready, finalizing order
 317 valid!
 318
 319 Downloading certificate
 320 Setting pveproxy certificate and key
 321 Restarting pveproxy
 322 Task OK
 323 ----
 324
 325 [[sysadmin_certs_acme_switch_from_staging]]
 326 Example: Switching from the `staging` to the regular ACME directory
 327 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 328
 329 Changing the ACME directory for an account is unsupported, but as {pve}
 330 supports more than one account you can just create a new one with the
 331 production (trusted) ACME directory as endpoint.  You can also deactivate the
 332 staging account and recreate it.
 333
 334 // TODO: add example with account screenshot here
 335
 336 .Example: Changing the `default` ACME account from `staging` to directory using `pvenode`
 337 ----
 338 root@proxmox:~# pvenode acme account deactivate default
 339 Renaming account file from '/etc/pve/priv/acme/default' to '/etc/pve/priv/acme/_deactivated_default_4'
 340 Task OK
 341
 342 root@proxmox:~# pvenode acme account register default example@proxmox.com
 343 Directory endpoints:
 344 0) Let's Encrypt V2 (https://acme-v02.api.letsencrypt.org/directory)
 345 1) Let's Encrypt V2 Staging (https://acme-staging-v02.api.letsencrypt.org/directory)
 346 2) Custom
 347 Enter selection: 0
 348
 349 Terms of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
 350 Do you agree to the above terms? [y|N]y
 351 ...
 352 Task OK
 353 ----