Mira Limbeck [Tue, 20 Feb 2024 10:06:47 +0000 (11:06 +0100)]
cleanup: remove unused strftime function
since we switched to proxmox_time::strftime_local for printing the start
and end times using local time, the strftime functions in src/time.rs is
unused and can be removed
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 20 Feb 2024 10:06:46 +0000 (11:06 +0100)]
tests: update for log tracker time handling changes
since we now calculate the time in UTC rather than local time, the time
values of each test had to be touched. those should differ by `E10`
(3600 seconds) in most cases since we set the TZ to Europe/Vienna for
the tests.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 20 Feb 2024 10:06:45 +0000 (11:06 +0100)]
tests: improve test output consistency
`expected` and `command` are more helpful than `new` and `old`.
the order of `expected` and `command` should now always be the same:
expected before command
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 20 Feb 2024 10:06:44 +0000 (11:06 +0100)]
rfc3339: move timezone offset compatibility code to old time parser
The compatibility code was added to the new rfc3339 code path temporarily so
that the old code path would not be changed before the PMG 8 release.
Now move it to the old time format code to make sure the rfc3339 code path
works as expected. Since we have all the information we need (year, month,
day, hours, minutes, seconds, timezone), there's no need for a workaround in
this code path.
As a side effect of parsing the time format `YYYY-MM-DD HH:MM:SS` in
localtime, it's now possible to parse an rfc3339 compliant format
(except for fractional seconds).
The change needs to be accompanied by one in pmg-api MailTracker.pmg to
keep the time displayed in the GUI the same for the old time format, and
correct for the new rfc3339 format.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Dominik Csapak [Wed, 25 Oct 2023 11:20:26 +0000 (13:20 +0200)]
use pico-args instead of clap
Instead of upgrading from clap3 to clap4 (which seems to change their
interface every year or so), switch to the much smaller pico-args. (Same
as we did for termproxy recently, see [0])
It has almost all features we need (except producing help output) and
supports OsString, but wihout any dependencies. This decreases compile
time and reduces the size of the resulting binary. It also reduces the
lines of code.
The only difference is now the different output for errors, e.g. for
missing values of options.
Mira Limbeck [Wed, 28 Jun 2023 08:54:28 +0000 (10:54 +0200)]
add compatibility with API/Tracking Center
The API assumes the timestamps to be in the local timezone rather than
UTC. It then subtracts the timezone offset leading to wrong values when
timestamps are in UTC, but timezone is not.
For compatibility, add the local timezone to those timestamps.
Stoiko Ivanov [Mon, 19 Jun 2023 18:40:48 +0000 (20:40 +0200)]
tests: explicitly set timezone for testcases with unix timestamps
noticed when building with sbuild, which uses UTC as timezone:
Due to the parsing of traditional syslog time information (which does
not contain timezone information) the testcases passing
unix-timestamps as arguments failed.
Explicitly setting TZ to Europe/Vienna fixes the tests (and makes the
assumption visible)
changed the date handling to have current_month and current_year
represent the month and year at time of invocation.
This change was not carried over to the logfile collecting, which set
the current month to january.
Stoiko Ivanov [Mon, 3 Jan 2022 16:09:54 +0000 (17:09 +0100)]
do not assume constant year based on file index
rather start with the current time at invocation and if the month in
the log is larger assume a year-wrap happened between logwriting and
invocation.
the old logic is flawed (but that can be said about parsing
traditional syslog timestamps (w/o year) in general) - it got worse
with the change in bullseye of rotating syslog weekly by default -
resulting in users losing one week of logs per day in the new year as
reported in https://forum.proxmox.com/threads/.102322/
Reviewed-by: Wolfgang Bumiller <w.bumiller@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Mira Limbeck [Wed, 1 Dec 2021 15:41:57 +0000 (16:41 +0100)]
fix #3758: fix `to` filter with empty NoqueueEntry `to` field
We only compared the filter `to` with the Noqueue `to` when both were
not empty, and in turn set the nq.dstatus to DStatus::Invalid when they
didn't match.
But in the case of an empty Noqueue `to`, we have to skip it as well if
the filter `to` is not empty.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Thu, 29 Apr 2021 13:25:08 +0000 (15:25 +0200)]
fix mutable borrow panic on duplicate msgid
Because of the relay before lmtp issue we now add and remove QEntrys
based on their message-id. But if the cleanup line containing the
message-id appears twice in the log for the same QEntry it keeps a weak
reference to itself which leads to a panic because of a mutable borrow
while it is already mutably borrowed in the 'finalize' function.
To circumvent this we check after the lookup if it is the same QEntry
and if so, insert it again with that message-id in the lookup table and
don't add the weak reference.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Tested-By: Aaron Lauterer <a.lauterer@proxmox.com>
Mira Limbeck [Tue, 23 Mar 2021 14:28:22 +0000 (15:28 +0100)]
after-queue: fix wrong DStatus if relay is 'removed' before 'lmtp' line
We only match a QEntry with the relay Qentry in the after-queue case
when we get to the 'lmtp' line. This is because it contains a reference
from the first QEntry to the filter, and the filter contains a reference
to the relay QEntry.
If the relay log entries are finished before the 'lmtp' line, there's no
way to match the relay to the first QEntry and we still assume it is
before-queue filtered.
To fix this we try to match the QEntries via message-id and add weak
references to each other. Then we wait with finalizing either until the
other is also 'removed' (finished).
The message-id matching might break if the same message-id is used for
different entries, so this is rather a 'hack' than a nice solution, but
there's no other info in the logs we could use to match both QEntries.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 23 Mar 2021 11:05:31 +0000 (12:05 +0100)]
close #2106: show outgoing TLS connection in tracking center
This is a best effort try to add the outgoing TLS connection information
to the output of pmg-log-tracker. The only thing we can match on is the
PID of the 'smtp' process. In the code we asumme that the TLS log entry
always happens before the actual smtp send entry that has a QID. This means
we save the TLS log entry in a map with the PID as key and then, once the
send entry happens, we look it up and add the log entry to the QEntry's
logs.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Mon, 22 Mar 2021 13:06:18 +0000 (14:06 +0100)]
change tests to use faketime
We only have the month, day, hours, minutes and seconds available to us,
but not the year. Because of this we assume the logs are always relative
to the current year. This means when running the log tracker the CTIME
is calculated based on the current year and once a new year starts, all
old tests have to be updated to match the new year.
To work around this issue, use 'faketime' to set the time to 2020-12-31
23:59:59 as all tests assume 2020 as the year.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Mira Limbeck [Mon, 9 Nov 2020 14:18:46 +0000 (15:18 +0100)]
change case sensitive string match to case insensitive
With the rewrite from C to Rust the search string match was changed to
be case sensitive by accident. Fix this by comparing the lowercase values
of both the input and the search string.
Mira Limbeck [Fri, 28 Aug 2020 10:19:51 +0000 (12:19 +0200)]
add output for proxy-reject with 'Syntax:' message
Messages like the following
'proxy-reject: END-OF-MESSAGE: 501 5.5.2 Syntax: MAIL FROM: <address>;'
can happen if an EHLO keyword is announced which is not handled by
pmg-smtp-filter (see #2795). This patch adds output to the log tracker
so this mail shows up as 'rejected' in the GUI instead of silently
ignoring it.
Mira Limbeck [Tue, 19 May 2020 13:31:06 +0000 (15:31 +0200)]
fix ToEntrys in FEntrys not being considered for 'to' filtering
When a 'to' filter is specified and a mail has multiple 'to's with at
least one being accepted, the others blocked or quarantined and the
'to' for the accepted mail does not match, the entries in the filter
will be ignored even if one of them would match.
To fix it the filter ToEntrys are now considered in the 'to' check.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 19 May 2020 13:31:05 +0000 (15:31 +0200)]
fix before-queue filter string_match check
In before-queue filtering a filter can be set on the SEntry. When a
search-string is set, we have to check both the SEntry and the attached
filter for a match, not just the SEntry. Otherwise some entries are not shown.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Fri, 15 May 2020 14:49:05 +0000 (16:49 +0200)]
print blocked/quarantined entries if accepted mail exists
In the case of before-queue filtering we have the separation 'QEntry'
for accepted mails, as those are the only ones with a queue. And
'SEntry' for blocked and quarantined mails. But if a mail is accepted
for one address, but at the same time blocked or quarantined for another
address we previously discarded those entries. To fix it we now print it
in the 'QEntry' if the attached filter contains blocked and/or
quarantined mails.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Fri, 15 May 2020 14:49:04 +0000 (16:49 +0200)]
make QID parsing more robust
As a user had the problem that 'fa' of 'fatal' matched for a QID,
increase the number of characters that have to match. The QID always has
at least 5 characters for the microseconds, so increase it to 5.
See http://www.postfix.org/postconf.5.html#enable_long_queue_ids
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Currently the status of a before queue filtered mail is the status of the
downstream delivery in case the downstream rejects or defers a mail.
This information is too little to distinguish between a mail which went
through the rule-system, and a locally generated one
(e.g. bounce or spamreport).
By introducing 3 new distinct dstatus values we can use the information to
correctly display the status of a mail in the GUI.
Reviewed-By: Mira Limbeck <m.limbeck@proxmox.com> Tested-By: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Mira Limbeck [Fri, 24 Apr 2020 15:21:49 +0000 (17:21 +0200)]
fix from entries being shown even with a from filter
When a 'from' and 'to' filter were set, we used a 'from || to' test to
see if we should print it or not. This patch changes this to a 'from &&
to' test. This gets rid of entries from unrelated senders ('from') when
a 'from filter did not match but the 'to' filter matched.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Thu, 2 Apr 2020 14:57:14 +0000 (16:57 +0200)]
add a test case for recipient filtering
Add a before queue test case to test recipient filtering. Previously
all SEntry filter entries were printed because filtering failed. Now
test that it only prints the specified ones.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Fri, 20 Mar 2020 12:08:28 +0000 (13:08 +0100)]
fix SEntry filter in before-queue case
With before-queue filtering if we don't accept the mail (e.g.
quarantine), we don't have a queue which means we don't have a QEntry,
so the SEntry has to handle the filter entries (ToEntrys).
This means we can't just return from print() when either a 'from' or 'to'
filter is set or we exclude greylist entries or NDRs and no Noqueue entries
exist or no entry matches any of the filters.
So continue printing if there is no filter parameter set, but an FEntry
reference in the SEntry. If there's an FEntry reference, compare all ToEntrys
to the filter parameter and return if there is no match at all.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
projects / pmg-log-tracker.git / log