]> git.proxmox.com Git - pmg-api.git/log
pmg-api.git
2 years agocluster: add '=' to ssh pubkey pattern
Stoiko Ivanov [Wed, 14 Jul 2021 14:44:30 +0000 (16:44 +0200)]
cluster: add '=' to ssh pubkey pattern

ssh public keys are base64 encoded, thus can potentially contain =.
until now the RSA keys generated by Debian were 2048 bits long and did
not need padding

with bullseye (openssh (1:8.0p1-1)) the RSA keysize got increased to
3072 bits, and now does contain a =

noticed while trying to join a PMG container from a bullseye template
to my existing cluster (the error happens on the new node).

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agocluster: refactor ssh pubkey verification
Stoiko Ivanov [Wed, 14 Jul 2021 14:44:29 +0000 (16:44 +0200)]
cluster: refactor ssh pubkey verification

to only have the regex in one place.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agobump version to 7.0-5
Thomas Lamprecht [Wed, 14 Jul 2021 09:58:53 +0000 (11:58 +0200)]
bump version to 7.0-5

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agouse bullseye for enterprise repository
Thomas Lamprecht [Wed, 14 Jul 2021 09:58:21 +0000 (11:58 +0200)]
use bullseye for enterprise repository

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agobump version to 7.0-4
Thomas Lamprecht [Wed, 14 Jul 2021 08:32:38 +0000 (10:32 +0200)]
bump version to 7.0-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agod/control: recommend ifupdown2 and suggest zfsutils-linux
Thomas Lamprecht [Wed, 14 Jul 2021 08:02:15 +0000 (10:02 +0200)]
d/control: recommend ifupdown2 and suggest zfsutils-linux

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agobump version to 7.0-3
Thomas Lamprecht [Tue, 13 Jul 2021 16:42:14 +0000 (18:42 +0200)]
bump version to 7.0-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoutils: fix typo in error message
Thomas Lamprecht [Tue, 13 Jul 2021 16:40:50 +0000 (18:40 +0200)]
utils: fix typo in error message

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: services: track chrony
Thomas Lamprecht [Tue, 13 Jul 2021 15:15:42 +0000 (17:15 +0200)]
api: services: track chrony

2 years agoapi: services: check if unit is even found
Thomas Lamprecht [Tue, 13 Jul 2021 15:15:32 +0000 (17:15 +0200)]
api: services: check if unit is even found

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: services: code cleanup
Thomas Lamprecht [Tue, 13 Jul 2021 15:14:52 +0000 (17:14 +0200)]
api: services: code cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: services: return active- and unit-state infos
Thomas Lamprecht [Tue, 13 Jul 2021 15:06:43 +0000 (17:06 +0200)]
api: services: return active- and unit-state infos

same is PVE nowadays

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoutils: service command: improve refusal to stop error message
Thomas Lamprecht [Tue, 13 Jul 2021 15:05:09 +0000 (17:05 +0200)]
utils: service command: improve refusal to stop error message

As "ERROR" is probably the worst reason to tell an user ;)

use the same as PVE has

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agod/control: bump versioned dependency for libpmg-rs-perl
Thomas Lamprecht [Tue, 13 Jul 2021 12:16:19 +0000 (14:16 +0200)]
d/control: bump versioned dependency for libpmg-rs-perl

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: apt: add calls for repositories
Fabian Ebner [Tue, 13 Jul 2021 08:04:12 +0000 (10:04 +0200)]
api: apt: add calls for repositories

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 years agoapi: termproxy: sort hash-keys when setting enum
Thomas Lamprecht [Tue, 13 Jul 2021 08:36:45 +0000 (10:36 +0200)]
api: termproxy: sort hash-keys when setting enum

else the api-viewer's dumper may get a false-positive change every
time we update the schema there

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agonodeconfig: parse acme config before writing
Stoiko Ivanov [Thu, 15 Apr 2021 19:46:20 +0000 (21:46 +0200)]
nodeconfig: parse acme config before writing

we use `get_acme_conf` as higher level sanity checker (e.g. to ensure
that wildcard certificates have a configured DNS plugin)

(adapted from pve-manger (where this is done in the corresponding API
call)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoacme: check plugin for wildcard certificates
Stoiko Ivanov [Thu, 15 Apr 2021 19:46:19 +0000 (21:46 +0200)]
acme: check plugin for wildcard certificates

Let's Encrypt currently only issues wildcard certificates if the
domain ownership is validated via a dns-01 type plugin.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoacme: handle wildcard dns validation
Stoiko Ivanov [Thu, 15 Apr 2021 19:46:18 +0000 (21:46 +0200)]
acme: handle wildcard dns validation

Wildcard DNS names (*.domain.example) are validated through their
base-domain (domain.example) according to the ACME RFC [0].

We store the indirection while parsing the acme config, and check for
an extra validation target during ordering.

This makes it possible to order wildcard certificates which are not
valid for the base-domain.

[0] https://tools.ietf.org/html/rfc8555#section-7.1.3

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agobump version to 7.0-2
Thomas Lamprecht [Sat, 3 Jul 2021 20:51:18 +0000 (22:51 +0200)]
bump version to 7.0-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoconfig: freshclam: default to incremental downloads
Stoiko Ivanov [Wed, 30 Jun 2021 16:39:55 +0000 (18:39 +0200)]
config: freshclam: default to incremental downloads

clamav recently started yielding 429 (too many requests) response
codes on even comparatively low attempts to download the complete
signature files (cvd)(see [0]), instead of the incremental changes
(cdiff) (see [1] for some background)

changing the default to scriptedupdates (a.k.a. cdiff download) seems
sensible for most situations.

[0] https://docs.clamav.net/faq/faq-freshclam.html
[1] https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoutils: fix service_aliases usage
Stoiko Ivanov [Wed, 30 Jun 2021 16:39:54 +0000 (18:39 +0200)]
utils: fix service_aliases usage

2 errors were introduced in 4f06ff8ac21baa5736060221191fb109e20d8e37:

* a typo in the postgresql service name
* it missed the other uses of the service_name hash, apart from the
  lookup_real_service_name sub.

both fixed here

Reported-by: Martin Maurer <martin@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agocluster: fix missing import
Stoiko Ivanov [Wed, 30 Jun 2021 15:42:57 +0000 (17:42 +0200)]
cluster: fix missing import

The missing use PMG::Ticket import is problematic during ACME cert
renewal from pmg-daily->PMG::API2::Certificates->renew_acme_cert,
since pmg-daily does not import it.

Reported-by: Martin Maurer <martin@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agod/postinst: handle static machine-id from ISO 5.0 <= x <= 6.0
Thomas Lamprecht [Tue, 29 Jun 2021 08:41:56 +0000 (10:41 +0200)]
d/postinst: handle static machine-id from ISO 5.0 <= x <= 6.0

Ported over from commit 8797cc74cd94583130ab4c2f541d1a75b518cfa6
of pve-manager, but with the list of machine-id extracted from PMG
ISOs.

The full map:

PMG 5.0-5  -> "e378bde63ac54872a85af23a8e4dac73"
PMG 5.1-1  -> "932b668d1fad4709b4976d54152d223c"
PMG 5.1-2  -> "931a8410cd034202a26b0e19d56e157a"
PMG 5.2-1  -> "9a1f3c5284e1423c9b0e0ee5819db6c9"
PMG 6.0-1  -> "5472a49c6436426fbebd7881f7b7f13b"

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: acme: indentation cleanup
Thomas Lamprecht [Tue, 29 Jun 2021 07:38:03 +0000 (09:38 +0200)]
api: acme: indentation cleanup

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agobump version to 7.0-1
Thomas Lamprecht [Mon, 28 Jun 2021 14:04:30 +0000 (16:04 +0200)]
bump version to 7.0-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agod/control: bump pve-common and pve-http-server dependency versions
Thomas Lamprecht [Mon, 28 Jun 2021 13:36:18 +0000 (15:36 +0200)]
d/control: bump pve-common and pve-http-server dependency versions

To ensure we have the new `upid_normalize_status_type` and the
new http-server listening behavior available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: tasks: add 'status' filter
Dominik Csapak [Thu, 24 Jun 2021 07:10:16 +0000 (09:10 +0200)]
api: tasks: add 'status' filter

like in PVE/PBS

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ Thomas: adapt to renamed PVE::Tools helper method ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: tasks: add 'since' and 'until' filters
Dominik Csapak [Thu, 24 Jun 2021 07:10:15 +0000 (09:10 +0200)]
api: tasks: add 'since' and 'until' filters

like in PVE/PBS

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2 years agoapi: tasks: add filter for task-type
Dominik Csapak [Thu, 24 Jun 2021 07:10:14 +0000 (09:10 +0200)]
api: tasks: add filter for task-type

Like we have in PVE/PBS

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2 years agofix #2013 spamreport: remove ticket if authmode is ldap
Stoiko Ivanov [Mon, 17 May 2021 14:02:57 +0000 (16:02 +0200)]
fix #2013 spamreport: remove ticket if authmode is ldap

Currently the 'authmode' setting for the spamquarantine is not used
anywhere. According to documentation setting it to 'ldap' should allow
access to the quarantine only with ldap credentials.

This patch addresses the issue by not generating a quarantineticket,
and adapting all links accordingly if the authmode is 'ldap'.

tested by changing the authmode and running
`pmgqm send -receiver <email-address> -debug 1`

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2 years agoadpi: node config: fix style nits/issues
Thomas Lamprecht [Mon, 28 Jun 2021 13:27:02 +0000 (15:27 +0200)]
adpi: node config: fix style nits/issues

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: nodeconfig: validate acme config before writing
Stoiko Ivanov [Wed, 16 Jun 2021 18:36:40 +0000 (20:36 +0200)]
api: nodeconfig: validate acme config before writing

Currently it is possible to add the same domains as different
acmedomainX keys to the node config, which prevents the user from
ordering certificates later.

This patch adds a call to get_acme_conf, which does the semantic
validation (and is also used in all other sites, which read the
config).

Reported in our community forum:
https://forum.proxmox.com/threads/lets-encrypt-cert-on-gui-not-working.91014/

quickly tested in my setup, by successfully adding the same domain
twice without the patch, and failing to do so with it applied.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agod/control: drop transitional apt-transport-https, provided by apt
Thomas Lamprecht [Mon, 28 Jun 2021 12:15:28 +0000 (14:15 +0200)]
d/control: drop transitional apt-transport-https, provided by apt

It was actually integrated into apt quite a bit before version 2.0
but it does not really hurts and version 2 is available since Q1 2020
on sid, bullseye will have 2.2.x so using (>= 2~) is just fine.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agod/control: bump standards version and use https in homepage
Thomas Lamprecht [Mon, 28 Jun 2021 12:15:08 +0000 (14:15 +0200)]
d/control: bump standards version and use https in homepage

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agobuildsys: upload to bullseye
Thomas Lamprecht [Mon, 28 Jun 2021 12:11:54 +0000 (14:11 +0200)]
buildsys: upload to bullseye

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agotests: greylist: retry connecting 3 times
Stoiko Ivanov [Fri, 11 Jun 2021 15:54:47 +0000 (17:54 +0200)]
tests: greylist: retry connecting 3 times

Sometimes pmgpolicy is not done starting up when we try connecting.
Sadly strace on test_greylist.pl makes the problem disappear.

Looping 3 times should work robustly.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2 years agogreylisting: drop unneeded Host column form cgreylist table
Stoiko Ivanov [Fri, 11 Jun 2021 15:54:46 +0000 (17:54 +0200)]
greylisting: drop unneeded Host column form cgreylist table

With the changes added in f61d54891d4820b21ef9e53f7ce0ebb1d5be1f73
greylisting does the matches based on a configurable netmask, and
does not use the 'Host' column in the cgreylist table anymore.

Drop it now with PMG 7.0

Quickly tested the following scenarios (all successfully):
* Upgrading from a previous version
* Restoring a pmg-backup taken with PMG 5.2 (the greylist table is
  excluded from the backup)
* Adding a node with the changes to an existing cluster without the
  change
* Adding a node without the changes to a master-node having them

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2 years agoaccess control: style: drop some extra newlines
Thomas Lamprecht [Wed, 16 Jun 2021 12:19:51 +0000 (14:19 +0200)]
access control: style: drop some extra newlines

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 years agoapi: nodes: add definedness check to avoid perl warning
Fabian Ebner [Tue, 15 Jun 2021 13:27:52 +0000 (15:27 +0200)]
api: nodes: add definedness check to avoid perl warning

also add the missing import for 'raise_perm_exc' to avoid having the error about
the undefined subroutine instead of the actual error.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 years agounpack: adapt to new libarchive methods
Stoiko Ivanov [Mon, 7 Jun 2021 18:55:22 +0000 (18:55 +0000)]
unpack: adapt to new libarchive methods

This patch changes the deprecated method used from libarchive-perl.
It needs a versioned dependency bump on libarchive-perl (>= 3.4.0).

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoupdate postgresql dependency to 13
Stoiko Ivanov [Mon, 31 May 2021 14:03:15 +0000 (14:03 +0000)]
update postgresql dependency to 13

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoutils: do not hardcode postgres version
Stoiko Ivanov [Mon, 31 May 2021 13:53:05 +0000 (13:53 +0000)]
utils: do not hardcode postgres version

PMG::Utils::lookup_real_service_name is only called
for translating the service names provided as arguments
to PMG::API2::Nodes::syslog (for fetching the journal
for specific units). Instead of hardcoding the
version getting it with a call to `psql` seems justified.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoapi: statistics: drop deprecated detail statistic methods
Stoiko Ivanov [Tue, 8 Jun 2021 17:25:29 +0000 (17:25 +0000)]
api: statistics: drop deprecated detail statistic methods

in e89b61c5190e3e374c2c3bcb3dce444c64c718cf we introduced a method
taking the address as explicit parameter instead of path component
(local-parts can contain '/'). now we can drop the old paths.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoapi: quarantine: drop deprecated b/w-list methods
Stoiko Ivanov [Tue, 8 Jun 2021 17:22:17 +0000 (17:22 +0000)]
api: quarantine: drop deprecated b/w-list methods

in e8d909c11faeb5a4f84f39ef50e0eaf8ea65046d we introduced methods
which take the address to be deleted as parameter instead of path
component (local-parts can contain '/') - now we can drop the old
paths as indicated in 53e5e5da24ec13c49e9fdeffa9560833b28ad107

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoTLSPolicy: drop deprecated 'domain' parameter
Stoiko Ivanov [Tue, 8 Jun 2021 16:26:49 +0000 (16:26 +0000)]
TLSPolicy: drop deprecated 'domain' parameter

the domain parameter was a misnomer and was replaced by destination in
cce8e372aa9e902dfa7b8b17770e82450dd319b9

With a major version change upcoming we can now drop the old
parameter name.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoRevert "improve csrf backward compatibility"
Stoiko Ivanov [Tue, 8 Jun 2021 16:11:55 +0000 (16:11 +0000)]
Revert "improve csrf backward compatibility"

This commit was added in PMG 5.2 for backwards compatibility and can
now be dropped.

This reverts commit 7aff29f38091462a13431ec80f6cbdd8e93e2c4b.

2 years agoapi: nodes: drop deprecated 'upgrade' option of termproxy
Stoiko Ivanov [Tue, 8 Jun 2021 16:06:50 +0000 (16:06 +0000)]
api: nodes: drop deprecated 'upgrade' option of termproxy

The termproxy api was adapted to the changes from PVE and PBS
in d9e79ff4b7f0f9b2c49f06484091546353980c5e
We can now drop the 'upgrade' option kept for backwards compatibility

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoclamav: remove deprecated SafeBrowsing
Oguz Bektas [Wed, 19 May 2021 12:32:43 +0000 (14:32 +0200)]
clamav: remove deprecated SafeBrowsing

this database was discontinued in 2019 due to legal reasons.

see:
https://blog.clamav.net/2021/04/are-you-still-attempting-to-download.html

archive in case it goes 404:
https://web.archive.org/web/20210408140341/https://blog.clamav.net/2021/04/are-you-still-attempting-to-download.html

* remove from freshclam.conf.in template
* remove from description
* default to 0 just to be sure
* if 'safebrowsing' set in pmg.conf, this is now ignored

note about removing the option in PMG 7.0

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
changed the removal notice form PMG 7.0 to 8.0
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agod/control: bump versioned dependency on pve-common and pve-http-server
Stoiko Ivanov [Fri, 11 Jun 2021 14:48:35 +0000 (16:48 +0200)]
d/control: bump versioned dependency on pve-common and pve-http-server

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agopmgproxy: allow setting LISTEN_IP parameter
Oguz Bektas [Wed, 24 Mar 2021 12:59:31 +0000 (13:59 +0100)]
pmgproxy: allow setting LISTEN_IP parameter

LISTEN_IP is defined in /etc/default/pmgproxy.

this depends on the changes in pve-common and pve-http-server (#2997)

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
added changes ported from pve-manager commit
8da3ad7ce8ca9d609f0e5be3860f63f3d1a58889
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agobuildsys: update lintian-overrides
Stoiko Ivanov [Tue, 8 Jun 2021 13:47:28 +0000 (13:47 +0000)]
buildsys: update lintian-overrides

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agobuildsys: bump d/compat to 12 and adapt d/rules
Stoiko Ivanov [Mon, 31 May 2021 12:55:05 +0000 (12:55 +0000)]
buildsys: bump d/compat to 12 and adapt d/rules

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agobuildsys: wrap-and-sort
Stoiko Ivanov [Mon, 31 May 2021 10:52:37 +0000 (10:52 +0000)]
buildsys: wrap-and-sort

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2 years agoapi: upload costum certificate: actually return info
Fabian Ebner [Wed, 28 Apr 2021 08:22:50 +0000 (10:22 +0200)]
api: upload costum certificate: actually return info

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2 years agoaccess control: add newline when dying on auth
Thomas Lamprecht [Wed, 28 Apr 2021 05:59:43 +0000 (07:59 +0200)]
access control: add newline when dying on auth

to avoid the module-path + line suffixed in the error..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.4-4
Thomas Lamprecht [Thu, 15 Apr 2021 13:19:04 +0000 (15:19 +0200)]
bump version to 6.4-4

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoacme: allow wildcard domain entries
Stoiko Ivanov [Mon, 12 Apr 2021 19:28:31 +0000 (21:28 +0200)]
acme: allow wildcard domain entries

Reported in our community forum [0], support for wildcard certificates
via ACME sounds like a good enhancement (especially for PMG).

In order for this to work you need to configure the wild-card
sub-entry (*.domain.example) as ACME domains and be able to verify
that via a DNS Plugin.
This is best described in the announcement by Let's Encrypt announcing
wildcard certificate support [1], or the dns challenge type
documentation[2].

Quickly tested with a domain of mine (and the powerdns plugin)

[0]: https://forum.proxmox.com/threads/feature-request-add-wildcard-support-for-acme.87495/
[1]: https://community.letsencrypt.org/t/acme-v2-production-environment-wildcards/55578
[2]: https://letsencrypt.org/docs/challenge-types/#dns-01-challenge

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agofix #2228: automatically deliver/delete on whitelist/blacklist
Thomas Lamprecht [Tue, 30 Mar 2021 15:05:47 +0000 (17:05 +0200)]
fix #2228: automatically deliver/delete on whitelist/blacklist

If a user chooses to blacklist a specific sender of a mail, the mail
is highly probably undesired too, so delete it in that case.

The reverse should hold for white listing a mail's sender, deliver it
in that case.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-By: Stoiko Ivanov <s.ivanov@proxmox.com>
Reviewed-By: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agoquarantine/content: better error message for internal error
Thomas Lamprecht [Tue, 30 Mar 2021 09:09:14 +0000 (11:09 +0200)]
quarantine/content: better error message for internal error

should really not happen, but if (e.g., enum gets adaped) it is good
to know the actual value triggering it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.4-3
Thomas Lamprecht [Fri, 26 Mar 2021 18:09:49 +0000 (19:09 +0100)]
bump version to 6.4-3

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopbs: rephrase notification mail
Stoiko Ivanov [Tue, 23 Mar 2021 16:45:47 +0000 (17:45 +0100)]
pbs: rephrase notification mail

currently the provided logs are not too detailed.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agobump version to 6.4-3
Stoiko Ivanov [Fri, 26 Mar 2021 17:53:18 +0000 (18:53 +0100)]
bump version to 6.4-3

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agomailqueue: fixup rmtree/mkpath usage
Stoiko Ivanov [Fri, 26 Mar 2021 17:43:56 +0000 (18:43 +0100)]
mailqueue: fixup rmtree/mkpath usage

the mkpath and rmtree functions [0] seem to interpret their
second and third argument as 'verbose' and 'mode', leading to
wrongly created spooldirs.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agobump version to 6.4-2
Thomas Lamprecht [Tue, 23 Mar 2021 07:30:51 +0000 (08:30 +0100)]
bump version to 6.4-2

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agofixup check for statement execute
Thomas Lamprecht [Tue, 23 Mar 2021 07:29:43 +0000 (08:29 +0100)]
fixup check for statement execute

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 6.4-1
Thomas Lamprecht [Mon, 22 Mar 2021 16:30:41 +0000 (17:30 +0100)]
bump version to 6.4-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agofix #3164: api: quarantine: allow to return spam from all users
Thomas Lamprecht [Mon, 22 Mar 2021 07:49:55 +0000 (08:49 +0100)]
fix #3164: api: quarantine: allow to return spam from all users

The pmail was only checked for the spam quarantine call, and there
mainly to ensure that the quarantine user only can check their own
mails. Make the pmail parameter also optional for this quarantine
related endpoint as long as one has a role other than quser.
This allows to query all spam quarantine entries from all pmails at
once, providing the backend side to address #3164.

The main argument against this was performance, but postgres can
handle even hundreds of thousands of rows rather fine, it's a high
performant database after all and this is quite the simple query
(single join, but no functions on columns, nested queries or other
performance hogs).

Some data, 45k records on a read limited disk, gathered with EXPLAIN
ANALYZE commands:

All caches dropped and fresh start: 440ms
Running for a bit with caches warm:  55ms

A simple extrapolation would mean that for half a million rows we
would spent about 5s in the DB, which is not too bad considering our
hard limit of 30s per requests, and the overhead of perl/https seems
to put the limit on my not so beefy VM at at least ~1.5 million rows
from a *cold* cache, which seems plenty (default 7 days keep window
and an avg. of 10 spam mails per day means >21k qusers). And with
warm caches and a beefier machine one can probably gain one or even
two order of magnitudes here.

And at the end, no mail admin is forced to use this and if they run a
setup with tens of millions of spam in their spam-keep time window,
well, they really should not be surprised that querying all has a
certain cost.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-By: Stoiko Ivanov <s.ivanov@proxmox.com>
Reviewed-By: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agotypo fixes all over the place
Thomas Lamprecht [Sun, 21 Mar 2021 18:18:07 +0000 (19:18 +0100)]
typo fixes all over the place

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmgproxy: server widget toolkit image directory
Thomas Lamprecht [Fri, 19 Mar 2021 18:03:32 +0000 (19:03 +0100)]
pmgproxy: server widget toolkit image directory

so that the pmx-clear-trigger CSS class from the widget toolkit can
be used

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: certs: drop unused variable
Thomas Lamprecht [Thu, 18 Mar 2021 16:02:02 +0000 (17:02 +0100)]
api: certs: drop unused variable

commit 1ecf138e1729dee908431673b73ac5baf8fd35a9 forgot to remove the
variable it changes made useless

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agounify File::path use
Thomas Lamprecht [Thu, 18 Mar 2021 15:58:53 +0000 (16:58 +0100)]
unify File::path use

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocerts: reload postfix to activate new certificate
Stoiko Ivanov [Thu, 18 Mar 2021 15:14:49 +0000 (16:14 +0100)]
certs: reload postfix to activate new certificate

the current logic for reloading postfix only does so if the tls config
parameter changes (after rewriting the config files).
this does not cover the case where a certificate is replaced in a
setup, which already has tls enabled (config stays the same, so
postfix does not get reloaded)

the issue is mostly cosmetic, since postfix does eventually fork off
new smtpd instances, which read the files from disk, but it's
inconvenient, when trying out the new acme integration, and then
running a ssl-check on your PMG from external just to see that the
certificate was not updated.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agocluster: use old and new fingerprint on master
Stoiko Ivanov [Thu, 18 Mar 2021 15:14:48 +0000 (16:14 +0100)]
cluster: use old and new fingerprint on master

when triggering a fingerprint update on master right after reloading
pmgproxy as we do for ACME certificates it can happen that the
connection is made against the old pmgproxy process (with the old
fingerprint). Simply trusting both fingerprints in that case seems
acceptable from a security perspective and makes the fingerprint
update more robust

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agoacme: recursively create account directory
Stoiko Ivanov [Thu, 18 Mar 2021 15:14:47 +0000 (16:14 +0100)]
acme: recursively create account directory

to account for the new layout

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agopackage: ship /etc/pmg/acme/accounts in deb
Stoiko Ivanov [Thu, 18 Mar 2021 15:14:46 +0000 (16:14 +0100)]
package: ship /etc/pmg/acme/accounts in deb

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
3 years agobump version to 6.3-7
Thomas Lamprecht [Thu, 18 Mar 2021 10:05:24 +0000 (11:05 +0100)]
bump version to 6.3-7

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agonode config: add format_description for 'usage' property
Thomas Lamprecht [Thu, 18 Mar 2021 10:15:04 +0000 (11:15 +0100)]
node config: add format_description for 'usage' property

the docs API schema dumper requires it

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agorearrange acme files
Wolfgang Bumiller [Thu, 18 Mar 2021 08:51:58 +0000 (09:51 +0100)]
rearrange acme files

Old layout:
  /etc/pmg/acme-plugins.conf
  /etc/pmg/acme/<account_name>

New layout:
  /etc/pmg/acme/plugins.conf
  /etc/pmg/acme/accounts/<account_name>

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agofix revocation of old certificate on renewal
Wolfgang Bumiller [Wed, 17 Mar 2021 10:12:18 +0000 (11:12 +0100)]
fix revocation of old certificate on renewal

in PMG key and certificate are in the same file so we need
to extract the certificate (like in the revoke api call)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocheck acme cert expiration in pmg-daily
Wolfgang Bumiller [Wed, 17 Mar 2021 10:02:16 +0000 (11:02 +0100)]
check acme cert expiration in pmg-daily

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd PMG::NodeConfig::filter_domains_by_type helper
Wolfgang Bumiller [Wed, 17 Mar 2021 10:02:15 +0000 (11:02 +0100)]
add PMG::NodeConfig::filter_domains_by_type helper

for reuse

The private $filter_domains is still there to do the
in-place modification it did before.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agosupport forced account deactivation
Wolfgang Bumiller [Wed, 17 Mar 2021 10:02:14 +0000 (11:02 +0100)]
support forced account deactivation

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd missing use statement
Wolfgang Bumiller [Wed, 17 Mar 2021 10:02:13 +0000 (11:02 +0100)]
add missing use statement

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmgcm: use minus for join-command, alias for compat
Thomas Lamprecht [Tue, 16 Mar 2021 18:12:45 +0000 (19:12 +0100)]
pmgcm: use minus for join-command, alias for compat

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmgcm: do not error out if not a cluster
Thomas Lamprecht [Tue, 16 Mar 2021 18:03:32 +0000 (19:03 +0100)]
pmgcm: do not error out if not a cluster

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmgcm: drop 'trigger' from 'update-fingerprints' name
Thomas Lamprecht [Tue, 16 Mar 2021 18:03:06 +0000 (19:03 +0100)]
pmgcm: drop 'trigger' from 'update-fingerprints' name

one verb is enough

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocluster: whitespace/indentation fixes
Thomas Lamprecht [Tue, 16 Mar 2021 18:02:17 +0000 (19:02 +0100)]
cluster: whitespace/indentation fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: cluster/update-fingerprint: code cleanups and iterate sorted
Thomas Lamprecht [Tue, 16 Mar 2021 18:01:03 +0000 (19:01 +0100)]
api: cluster/update-fingerprint: code cleanups and iterate sorted

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: certificates: trigger fingerprint update
Stoiko Ivanov [Mon, 15 Mar 2021 22:01:35 +0000 (23:01 +0100)]
api: certificates: trigger fingerprint update

in clustered systems, so that the cluster-sync remains possible with
the new certificate.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agopmgcm: add trigger-update-fingerprint
Stoiko Ivanov [Mon, 15 Mar 2021 22:01:34 +0000 (23:01 +0100)]
pmgcm: add trigger-update-fingerprint

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocluster: add trigger_update_fingerprints
Stoiko Ivanov [Mon, 15 Mar 2021 22:01:33 +0000 (23:01 +0100)]
cluster: add trigger_update_fingerprints

this commit adds a method that sends a POST request to
'/config/cluster/update-fingerprints' on the master node in a cluster.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: cluster: add update-fingerprints call
Stoiko Ivanov [Mon, 15 Mar 2021 22:01:32 +0000 (23:01 +0100)]
api: cluster: add update-fingerprints call

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocluster: add helper to get remote cert fingerprint
Stoiko Ivanov [Mon, 15 Mar 2021 22:01:31 +0000 (23:01 +0100)]
cluster: add helper to get remote cert fingerprint

via ssh executing 'openssl x509'

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agocluster: refactor rsync_command
Stoiko Ivanov [Mon, 15 Mar 2021 22:01:30 +0000 (23:01 +0100)]
cluster: refactor rsync_command

pull out the ssh part for later reusal

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: cert order: print details for failed challenge if available
Thomas Lamprecht [Tue, 16 Mar 2021 17:02:18 +0000 (18:02 +0100)]
api: cert order: print details for failed challenge if available

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: certs: sort hash keys, just to be sure
Thomas Lamprecht [Tue, 16 Mar 2021 16:43:52 +0000 (17:43 +0100)]
api: certs: sort hash keys, just to be sure

knowing perl this is just nicer and should never be an issue as
without it they will be scrambled around on each run anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi: acme order: fix check for configured domains
Thomas Lamprecht [Tue, 16 Mar 2021 13:43:01 +0000 (14:43 +0100)]
api: acme order: fix check for configured domains

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoadd acme and cert subcommands to pmgconfig
Wolfgang Bumiller [Tue, 16 Mar 2021 10:24:15 +0000 (11:24 +0100)]
add acme and cert subcommands to pmgconfig

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoadd node-config api entry points
Wolfgang Bumiller [Tue, 16 Mar 2021 10:24:14 +0000 (11:24 +0100)]
add node-config api entry points

adds /nodes/{nodename}/config to access node config

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>