Stoiko Ivanov [Tue, 17 May 2022 10:19:50 +0000 (12:19 +0200)]
rulesystem: matchfield: match all headers not only the first
currently the match field uses $entity->head->get in scalar context,
which only returns the first matching header (see [0])
switch over to using get_all in list context and iterating over all
headers makes it possible to match subsequent headers.
while it is uncommon in general - the Received headers are usually not
restricted to one - reported in our community forum:
https://forum.proxmox.com/threads/.109629/
Thomas Lamprecht [Sat, 14 May 2022 15:21:56 +0000 (17:21 +0200)]
d/control: bump versioned dependencies
for namespace support, but note that proxmox-backup-client 2.1.10-1
is still missing some changes only in git yet, i.e., making the CLI
prune command NS aware.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Markus Frank [Wed, 30 Mar 2022 12:32:15 +0000 (14:32 +0200)]
fix #3924: ldap: accept only valid email-address
If a mail attribute contains special characters in ldap at the first
line, it will be set as primary email and results in a
"400 invalid format - value does not look like a valid email address"
Error-Statement in the webconsole. This mostly can happen if SIP
Addresses are in Active-Directory's proxyAddresses which begin with "SIP:".
To make the validation more strict I changed the api to use
pmg-email-address and added a regex which looks for protocolnames (sip:)
that could be in proxyAddresses but are not compatible and skips these
addresses.
avoid a overly long line and a useless overwriting a scalar only to
extend another one with its value, really no biggie especially in the
context that's used, but its so easy to avoid that it still has some
merit.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stoiko Ivanov [Thu, 25 Nov 2021 17:48:13 +0000 (18:48 +0100)]
fix #2795: add support for DSN
store the esmtp parameters for the MAIL and RCPT command needed to
support Delivery status notifications (DSN - RFC 3464 [0]) and pass
them to the outbound postfix instance (port 10025) used for sending
the mail further (see also [1]).
Postfix does syntax-checking before passing the mail to the proxy
also in before-queue filtering mode.
Since the handling is done by postfix we don't need to generate any
DSN in the regular case.
For mail put into quarantine I decided to skip sending a delivery
notification (on the expectation, that few people are using quarantine
outbound, and that I would not consider a mail put in quarantine as
delivered successfully)
We only store a whitelist of parameters, instead of passing all,
because some parameters might not be valid anymore after processing
(e.g. SIZE)
The DSN EHLO keyword was added for the after-queue filtering case -
else the inbound postfix is the system that sends out the
notification.
tested with various combinations of the -V, -N and -R parameters to
sendmail (e.g.):
```
/usr/sbin/sendmail -N success,delay,failure \
-V '<xxxxxxxx@test.proxmox.com>'\
-R hdrs test@test.domain.example
```
tested the following scenarios in before and after-queue filter mode:
* successful delivery
* successful delivery with set DSN
* failed delivery (recipient rejects with 544)
* failed delivery with DSN
* delivering a mail with empty envelope sender (bounce)
some tests with invalid combinations were also done with netcat.
Stoiko Ivanov [Thu, 25 Nov 2021 17:44:11 +0000 (18:44 +0100)]
partially fix #2795: allow for '>' in smtp parameters
The regular expressions parsing the MAIL and RCPT commands do not
cover the case where a esmtp parameter may contain angle brackets
(e.g. the ENVID parameter for the delivery status notification
extension - RFC3464 [0]).
following section 4.1.2 of RFC5321 [1] the regex is changed to:
* consider everything up to the first '>' the mailbox
* consider everything afterwards (if it starts with a ' ') as
parameters
* since the parameter group might not match (in case no parameters are
set - e.g. after-queue filtering) - default to '' if it's not
defined
This is fairly robusts, only not parsing correctly if the local part
contains '>' (as quoted text) - but this did not work before anyways
(and causes problems in other places as well).
Dominik Csapak [Thu, 25 Nov 2021 14:14:41 +0000 (15:14 +0100)]
fix #3734: scrub 'url' from style tags/attributes
if 'view images' for the quarantine is disabled, it is expected that
*no* images will be loaded. but in addition to img (src/href/etc.)
also css can load external images via the 'url' directive
since html scrubber does not parse/iterate over css, we simply remove
the url+protocol part of those tags/attributes. this technically leaves behind
invalid css, but the browsers should cope with that.
(we cannot 'cleanly' remove without much more effort because of quoting)
also we have to scrub the style tags in 'dump_html' since HTML::Scrubber
does not have a way to modify the *content* of a tag, only the
attributes...
Stoiko Ivanov [Wed, 24 Nov 2021 21:00:48 +0000 (22:00 +0100)]
rulesystem: limit linelength of disclaimer to 998 bytes
As described in
http://www.postfix.org/postconf.5.html#smtp_line_length_limit
postfix splits lines which are longer by inserting <cr><lf><space> to
adhere with RFC 5322 (section 2.1.1):
https://datatracker.ietf.org/doc/html/rfc5322#section-2.1.1
(or actually section 4.5.3.1.6. where characters are translated to
octets)
If a longer line is part of the disclaimer pmg-smtp-filter adds it
without this modification, which breaks DKIM signatures (since the
body is modified by postfix after the body hash is computed)
regular-expression matching is used instead of length(), because the
limit is on line-length (and a disclaimer can contain multiple lines)
reported in our community forum:
https://forum.proxmox.com/threads/.97919/
Stoiko Ivanov [Wed, 24 Nov 2021 16:04:09 +0000 (17:04 +0100)]
api-daemons: set oom-policy to continue
OOMPolicy [0] defaults to stop - resulting in the complete daemon to
be killed.
Our Daemon class does start new workers automatically if it detects
that fewer than configured are running.
Dominik Csapak [Wed, 24 Nov 2021 14:48:52 +0000 (15:48 +0100)]
api: journal: stream the journal data to the client
instead of accumulating the whole output of 'mini-journalreader' in
the api call (this can be quite big), use the download mechanic of the
http-server to stream the output to the client.
we lose some error handling possibilities, but we do not have
to allocate anything here, and since perl does not free memory after
allocating[0] this is our desired behaviour.
to keep api compatiblitiy, we need to give the journalreader the '-j'
flag to let it output json.
also tell the http server that the encoding is gzip and pipe
the output through it.
Stoiko Ivanov [Mon, 22 Nov 2021 19:49:39 +0000 (20:49 +0100)]
fix #3712: strip trailing dot from searchdomain
having a trailing '.' in the search domain is perfectly legal syntax
(for domain names in general). postfix refuses to use a fqdn with
trailing dot as hostname[0].
The restriction might be due to section 2.3.5 (Domain Names) of
RFC5321 (a top-level domain is a single string without any dots) [1]
[0] src/util/valid_hostname.c in the postfix source
[1] https://datatracker.ietf.org/doc/html/rfc5321#section-2.3.5
Thomas Lamprecht [Mon, 20 Sep 2021 06:52:28 +0000 (08:52 +0200)]
services: add drop weird binary-exists condition
The package that ships the service is the same as the one that ships
the binaries, so quite the useless check and a remainder from initial
switch from sysv to systemd in ~2015 (when it was not 100% clear
what/how systemd features should be integrated or units encoded).
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Mon, 20 Sep 2021 06:50:46 +0000 (08:50 +0200)]
services: add After=network-online.target and update
while we indirectly got that by the remote-fs ordering constraint its
better to encode it explicitly, especially as the remote-fs does not
make much sense and may get removed soon
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Mon, 20 Sep 2021 06:38:08 +0000 (08:38 +0200)]
config: fix "var declared in conditional statement"
This is actually buggy and can lead to unexpected issues as in the
case the check on the declared variable did not evaluates to true it
gets (or better keeps) the value from the previous time when it was
actually assigned. Found with perlcritic, which reports the highest
severity for this mistake.
Refactor out the "is current file equal to generated config" check
which fixes three instances of that on its own and reduces code bloat
a bit.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sat, 18 Sep 2021 13:17:21 +0000 (15:17 +0200)]
pmgbanner: retry getting local ip for a bit
basically only useful for setups using (hopefully static) DHCP for
the PMG host, but we can have that in evaluation, especially when
using CTs or installing on top of a plain Debian.
This was favored over adding an After=network-online.target order
constraint for the pmgbanner service, as it'd delay the console-getty
service needlessly in most setups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sat, 18 Sep 2021 13:06:57 +0000 (15:06 +0200)]
prefer more flexible get_local_ip where possible
get_ip_from_hostname does only check getaddrinfo, which can fail for
the local node in some environments, especially container ones.
Rather, use the new get_local_ip helper, that still tries to do a gai
call first, but falls back to configured (/etc/network/interfaces)
IPs and also on the currently, from kernel POV active ones.
A big bonus is that the new helper is much less likely to die, so it
won't break service startup in restricted (CTs) envs after initial
setup as often anymore.
While yes, if no addr is resolved, configured or active the PMG won't
work, but killing pmg proxy/daemon won't better that situation either
;)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Oguz Bektas [Thu, 2 Sep 2021 08:47:12 +0000 (10:47 +0200)]
api: apt: use actual pmg-style permission for endpoint schema
it wrongly uses the permission model from PVE, which caused the
endpoints to be root-only as a side effect, since PMG API doesn't
recognize the PVE-specific permissions.
fix those to allow PMG users with administrator role to add/delete
repositories, and auditor role to view the repositories.
Thomas Lamprecht [Fri, 16 Jul 2021 10:27:01 +0000 (12:27 +0200)]
api: implement live network reload with ifupdown2
Like most of the other call here, copied over from PVE, with the SDN
stuff dropped and some task-log feedback if we actually moved a
pending change in. Also adding error handling for the rename, both
should be added to PVE too.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
ssh public keys are base64 encoded, thus can potentially contain =.
until now the RSA keys generated by Debian were 2048 bits long and did
not need padding
with bullseye (openssh (1:8.0p1-1)) the RSA keysize got increased to
3072 bits, and now does contain a =
noticed while trying to join a PMG container from a bullseye template
to my existing cluster (the error happens on the new node).
projects / pmg-api.git / log