certificate-management.adoc

   1 [[sysadmin_certificate_management]]
   2 Certificate Management
   3 ----------------------
   4 ifdef::wiki[]
   5 :pve-toplevel:
   6 endif::wiki[]
   7
   8
   9 Certificates for Intra-Cluster Communication
  10 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  11
  12 Each {PVE} cluster creates by default its own (self-signed) Certificate
  13 Authority (CA) and generates a certificate for each node which gets signed by
  14 the aforementioned CA. These certificates are used for encrypted communication
  15 with the cluster's `pveproxy` service and the Shell/Console feature if SPICE is
  16 used.
  17
  18 The CA certificate and key are stored in the xref:chapter_pmxcfs[Proxmox Cluster File System (pmxcfs)].
  19
  20
  21 [[sysadmin_certs_api_gui]]
  22 Certificates for API and Web GUI
  23 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  24
  25 The REST API and web GUI are provided by the `pveproxy` service, which runs on
  26 each node.
  27
  28 You have the following options for the certificate used by `pveproxy`:
  29
  30 1. By default the node-specific certificate in
  31 `/etc/pve/nodes/NODENAME/pve-ssl.pem` is used. This certificate is signed by
  32 the cluster CA and therefore not automatically trusted by browsers and
  33 operating systems.
  34 2. use an externally provided certificate (e.g. signed by a commercial CA).
  35 3. use ACME (Let's Encrypt) to get a trusted certificate with automatic
  36 renewal, this is also integrated in the {pve} API and web interface.
  37
  38 For options 2 and 3 the file `/etc/pve/local/pveproxy-ssl.pem` (and
  39 `/etc/pve/local/pveproxy-ssl.key`, which needs to be without password) is used.
  40
  41 NOTE: Keep in mind that `/etc/pve/local` is a node specific symlink to
  42 `/etc/pve/nodes/NODENAME`.
  43
  44 Certificates are managed with the {PVE} Node management command
  45 (see the `pvenode(1)` manpage).
  46
  47 WARNING: Do not replace or manually modify the automatically generated node
  48 certificate files in `/etc/pve/local/pve-ssl.pem` and
  49 `/etc/pve/local/pve-ssl.key` or the cluster CA files in
  50 `/etc/pve/pve-root-ca.pem` and `/etc/pve/priv/pve-root-ca.key`.
  51
  52 [[sysadmin_certs_upload_custom]]
  53 Upload Custom Certificate
  54 ~~~~~~~~~~~~~~~~~~~~~~~~~
  55
  56 If you already have a certificate which you want to use for a {pve} node you
  57 can upload that certificate simply over the web interface.
  58
  59 [thumbnail="screenshot/gui-node-certs-upload-custom.png"]
  60
  61 Note that the certificates key file, if provided, mustn't be password
  62 protected.
  63
  64 [[sysadmin_certs_get_trusted_acme_cert]]
  65 Trusted certificates via Let's Encrypt (ACME)
  66 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  67
  68 {PVE} includes an implementation of the **A**utomatic **C**ertificate
  69 **M**anagement **E**nvironment **ACME** protocol, allowing {pve} admins to
  70 use an ACME provider like Let's Encrypt for easy setup of TLS certificates
  71 which are accepted and trusted on modern operating systems and web browsers
  72 out of the box.
  73
  74 Currently, the two ACME endpoints implemented are the
  75 https://letsencrypt.org[Let's Encrypt (LE)] production and its staging
  76 environment. Our ACME client supports validation of `http-01` challenges using
  77 a built-in web server and validation of `dns-01` challenges using a DNS plugin
  78 supporting all the DNS API endpoints https://acme.sh[acme.sh] does.
  79
  80 [[sysadmin_certs_acme_account]]
  81 ACME Account
  82 ^^^^^^^^^^^^
  83
  84 [thumbnail="screenshot/gui-datacenter-acme-register-account.png"]
  85
  86 You need to register an ACME account per cluster with the endpoint you want to
  87 use. The email address used for that account will serve as contact point for
  88 renewal-due or similar notifications from the ACME endpoint.
  89
  90 You can register and deactivate ACME accounts over the web interface
  91 `Datacenter -> ACME` or using the `pvenode` command line tool.
  92 ----
  93  pvenode acme account register account-name mail@example.com
  94 ----
  95
  96 TIP: Because of https://letsencrypt.org/docs/rate-limits/[rate-limits] you
  97 should use LE `staging` for experiments or if you use ACME for the first time.
  98
  99 [[sysadmin_certs_acme_plugins]]
 100 ACME Plugins
 101 ^^^^^^^^^^^^
 102
 103 The ACME plugins task is to provide automatic verification that you, and thus
 104 the {pve} cluster under your operation, are the real owner of a domain. This is
 105 the basis building block for automatic certificate management.
 106
 107 The ACME protocol specifies different types of challenges, for example the
 108 `http-01` where a web server provides a file with a certain content to prove
 109 that it controls a domain. Sometimes this isn't possible, either because of
 110 technical limitations or if the address of a record to is not reachable from
 111 the public internet. The `dns-01` challenge can be used in these cases.  This
 112 challenge is fulfilled by creating a certain DNS record in the domain's zone.
 113
 114 [thumbnail="screenshot/gui-datacenter-acme-overview.png"]
 115
 116 {pve} supports both of those challenge types out of the box, you can configure
 117 plugins either over the web interface under `Datacenter -> ACME`, or using the
 118 `pvenode acme plugin add` command.
 119
 120 ACME Plugin configurations are stored in `/etc/pve/priv/acme/plugins.cfg`.
 121 A plugin is available for all nodes in the cluster.
 122
 123 Node Domains
 124 ^^^^^^^^^^^^
 125
 126 Each domain is node specific. You can add new or manage existing domain entries
 127 under `Node -> Certificates`, or using the `pvenode config` command.
 128
 129 [thumbnail="screenshot/gui-node-certs-add-domain.png"]
 130
 131 After configuring the desired domain(s) for a node and ensuring that the
 132 desired ACME account is selected, you can order your new certificate over the
 133 web-interface. On success the interface will reload after 10 seconds.
 134
 135 Renewal will happen xref:sysadmin_certs_acme_automatic_renewal[automatically].
 136
 137 [[sysadmin_certs_acme_http_challenge]]
 138 ACME HTTP Challenge Plugin
 139 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 140
 141 There is always an implicitly configured `standalone` plugin for validating
 142 `http-01` challenges via the built-in webserver spawned on port 80.
 143
 144 NOTE: The name `standalone` means that it can provide the validation on it's
 145 own, without any third party service. So, this plugin works also for cluster
 146 nodes.
 147
 148 There are a few prerequisites to use it for certificate management with Let's
 149 Encrypts ACME.
 150
 151 * You have to accept the ToS of Let's Encrypt to register an account.
 152 * **Port 80** of the node needs to be reachable from the internet.
 153 * There **must** be no other listener on port 80.
 154 * The requested (sub)domain needs to resolve to a public IP of the Node.
 155
 156
 157 [[sysadmin_certs_acme_dns_challenge]]
 158 ACME DNS API Challenge Plugin
 159 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 160
 161 On systems where external access for validation via the `http-01` method is
 162 not possible or desired, it is possible to use the `dns-01` validation method.
 163 This validation method requires a DNS server that allows provisioning of `TXT`
 164 records via an API.
 165
 166 [[sysadmin_certs_acme_dns_api_config]]
 167 Configuring ACME DNS APIs for validation
 168 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 169
 170 {PVE} re-uses the DNS plugins developed for the `acme.sh`
 171 footnote:[acme.sh https://github.com/acmesh-official/acme.sh] project, please
 172 refer to its documentation for details on configuration of specific APIs.
 173
 174 The easiest way to configure a new plugin with the DNS API is using the web
 175 interface (`Datacenter -> ACME`).
 176
 177 [thumbnail="screenshot/gui-datacenter-acme-add-dns-plugin.png"]
 178
 179 Choose `DNS` as challenge type. Then you can select your API provider, enter
 180 the credential data to access your account over their API.
 181
 182 TIP: See the acme.sh
 183 https://github.com/acmesh-official/acme.sh/wiki/dnsapi#how-to-use-dns-api[How to use DNS API]
 184 wiki for more detailed information about getting API credentials for your
 185 provider.
 186
 187 As there are many DNS providers and API endpoints {pve} automatically generates
 188 the form for the credentials for some providers. For the others you will see a
 189 bigger text area, simply copy all the credentials `KEY`=`VALUE` pairs in there.
 190
 191 DNS Validation through CNAME Alias
 192 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 193
 194 A special `alias` mode can be used to handle the validation on a different
 195 domain/DNS server, in case your primary/real DNS does not support provisioning
 196 via an API. Manually set up a permanent `CNAME` record for
 197 `_acme-challenge.domain1.example` pointing to `_acme-challenge.domain2.example`
 198 and set the `alias` property in the {PVE} node configuration file to
 199 `domain2.example` to allow the DNS server of `domain2.example` to validate all
 200 challenges for `domain1.example`.
 201
 202
 203 Combination of Plugins
 204 ^^^^^^^^^^^^^^^^^^^^^^
 205
 206 Combining `http-01` and `dns-01` validation is possible in case your node is
 207 reachable via multiple domains with different requirements / DNS provisioning
 208 capabilities. Mixing DNS APIs from multiple providers or instances is also
 209 possible by specifying different plugin instances per domain.
 210
 211 TIP: Accessing the same service over multiple domains increases complexity and
 212 should be avoided if possible.
 213
 214 [[sysadmin_certs_acme_automatic_renewal]]
 215 Automatic renewal of ACME certificates
 216 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 217
 218 If a node has been successfully configured with an ACME-provided certificate
 219 (either via pvenode or via the GUI), the certificate will be automatically
 220 renewed by the `pve-daily-update.service`. Currently, renewal will be attempted
 221 if the certificate has expired already, or will expire in the next 30 days.
 222
 223
 224 ACME Examples with `pvenode`
 225 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 226
 227 Example: Sample `pvenode` invocation for using Let's Encrypt certificates
 228 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 229
 230 ----
 231 root@proxmox:~# pvenode acme account register default mail@example.invalid
 232 Directory endpoints:
 233 0) Let's Encrypt V2 (https://acme-v02.api.letsencrypt.org/directory)
 234 1) Let's Encrypt V2 Staging (https://acme-staging-v02.api.letsencrypt.org/directory)
 235 2) Custom
 236 Enter selection: 1
 237
 238 Terms of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
 239 Do you agree to the above terms? [y|N]y
 240 ...
 241 Task OK
 242 root@proxmox:~# pvenode config set --acme domains=example.invalid
 243 root@proxmox:~# pvenode acme cert order
 244 Loading ACME account details
 245 Placing ACME order
 246 ...
 247 Status is 'valid'!
 248
 249 All domains validated!
 250 ...
 251 Downloading certificate
 252 Setting pveproxy certificate and key
 253 Restarting pveproxy
 254 Task OK
 255 ----
 256
 257 Example: Setting up the OVH API for validating a domain
 258 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 259
 260 NOTE: the account registration steps are the same no matter which plugins are
 261 used, and are not repeated here.
 262
 263 NOTE: `OVH_AK` and `OVH_AS` need to be obtained from OVH according to the OVH
 264 API documentation
 265
 266
 267 First you need to get all information so you and {pve} can access the API.
 268
 269 ----
 270 root@proxmox:~# cat /path/to/api-token
 271 OVH_AK=XXXXXXXXXXXXXXXX
 272 OVH_AS=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
 273 root@proxmox:~# source /path/to/api-token
 274 root@proxmox:~# curl -XPOST -H"X-Ovh-Application: $OVH_AK" -H "Content-type: application/json" \
 275 https://eu.api.ovh.com/1.0/auth/credential  -d '{
 276   "accessRules": [
 277     {"method": "GET","path": "/auth/time"},
 278     {"method": "GET","path": "/domain"},
 279     {"method": "GET","path": "/domain/zone/*"},
 280     {"method": "GET","path": "/domain/zone/*/record"},
 281     {"method": "POST","path": "/domain/zone/*/record"},
 282     {"method": "POST","path": "/domain/zone/*/refresh"},
 283     {"method": "PUT","path": "/domain/zone/*/record/"},
 284     {"method": "DELETE","path": "/domain/zone/*/record/*"}
 285 ]
 286 }'
 287 {"consumerKey":"ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ","state":"pendingValidation","validationUrl":"https://eu.api.ovh.com/auth/?credentialToken=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"}
 288
 289 (open validation URL and follow instructions to link Application Key with account/Consumer Key)
 290
 291 root@proxmox:~# echo "OVH_CK=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ" >> /path/to/api-token
 292 ----
 293
 294 Now you can setup the the ACME plugin:
 295
 296 ----
 297 root@proxmox:~# pvenode acme plugin add dns example_plugin --api ovh --data /path/to/api_token
 298 root@proxmox:~# pvenode acme plugin config example_plugin
 299 ┌────────┬──────────────────────────────────────────┐
 300 │ key    │ value                                    │
 301 ╞════════╪══════════════════════════════════════════╡
 302 │ api    │ ovh                                      │
 303 ├────────┼──────────────────────────────────────────┤
 304 │ data   │ OVH_AK=XXXXXXXXXXXXXXXX                  │
 305 │        │ OVH_AS=YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY  │
 306 │        │ OVH_CK=ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ  │
 307 ├────────┼──────────────────────────────────────────┤
 308 │ digest │ 867fcf556363ca1bea866863093fcab83edf47a1 │
 309 ├────────┼──────────────────────────────────────────┤
 310 │ plugin │ example_plugin                           │
 311 ├────────┼──────────────────────────────────────────┤
 312 │ type   │ dns                                      │
 313 └────────┴──────────────────────────────────────────┘
 314 ----
 315
 316 At last you can configure the domain you want to get certificates for and
 317 place the certificate order for it:
 318
 319 ----
 320 root@proxmox:~# pvenode config set -acmedomain0 example.proxmox.com,plugin=example_plugin
 321 root@proxmox:~# pvenode acme cert order
 322 Loading ACME account details
 323 Placing ACME order
 324 Order URL: https://acme-staging-v02.api.letsencrypt.org/acme/order/11111111/22222222
 325
 326 Getting authorization details from 'https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/33333333'
 327 The validation for example.proxmox.com is pending!
 328 [Wed Apr 22 09:25:30 CEST 2020] Using OVH endpoint: ovh-eu
 329 [Wed Apr 22 09:25:30 CEST 2020] Checking authentication
 330 [Wed Apr 22 09:25:30 CEST 2020] Consumer key is ok.
 331 [Wed Apr 22 09:25:31 CEST 2020] Adding record
 332 [Wed Apr 22 09:25:32 CEST 2020] Added, sleep 10 seconds.
 333 Add TXT record: _acme-challenge.example.proxmox.com
 334 Triggering validation
 335 Sleeping for 5 seconds
 336 Status is 'valid'!
 337 [Wed Apr 22 09:25:48 CEST 2020] Using OVH endpoint: ovh-eu
 338 [Wed Apr 22 09:25:48 CEST 2020] Checking authentication
 339 [Wed Apr 22 09:25:48 CEST 2020] Consumer key is ok.
 340 Remove TXT record: _acme-challenge.example.proxmox.com
 341
 342 All domains validated!
 343
 344 Creating CSR
 345 Checking order status
 346 Order is ready, finalizing order
 347 valid!
 348
 349 Downloading certificate
 350 Setting pveproxy certificate and key
 351 Restarting pveproxy
 352 Task OK
 353 ----
 354
 355 [[sysadmin_certs_acme_switch_from_staging]]
 356 Example: Switching from the `staging` to the regular ACME directory
 357 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 358
 359 Changing the ACME directory for an account is unsupported, but as {pve}
 360 supports more than one account you can just create a new one with the
 361 production (trusted) ACME directory as endpoint.  You can also deactivate the
 362 staging account and recreate it.
 363
 364 // TODO: add example with account screenshot here
 365
 366 .Example: Changing the `default` ACME account from `staging` to directory using `pvenode`
 367 ----
 368 root@proxmox:~# pvenode acme account deactivate default
 369 Renaming account file from '/etc/pve/priv/acme/default' to '/etc/pve/priv/acme/_deactivated_default_4'
 370 Task OK
 371
 372 root@proxmox:~# pvenode acme account register default example@proxmox.com
 373 Directory endpoints:
 374 0) Let's Encrypt V2 (https://acme-v02.api.letsencrypt.org/directory)
 375 1) Let's Encrypt V2 Staging (https://acme-staging-v02.api.letsencrypt.org/directory)
 376 2) Custom
 377 Enter selection: 0
 378
 379 Terms of Service: https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf
 380 Do you agree to the above terms? [y|N]y
 381 ...
 382 Task OK
 383 ----