changed the date handling to have current_month and current_year
represent the month and year at time of invocation.
This change was not carried over to the logfile collecting, which set
the current month to january.
Stoiko Ivanov [Mon, 3 Jan 2022 16:09:54 +0000 (17:09 +0100)]
do not assume constant year based on file index
rather start with the current time at invocation and if the month in
the log is larger assume a year-wrap happened between logwriting and
invocation.
the old logic is flawed (but that can be said about parsing
traditional syslog timestamps (w/o year) in general) - it got worse
with the change in bullseye of rotating syslog weekly by default -
resulting in users losing one week of logs per day in the new year as
reported in https://forum.proxmox.com/threads/.102322/
Reviewed-by: Wolfgang Bumiller <w.bumiller@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Mira Limbeck [Wed, 1 Dec 2021 15:41:57 +0000 (16:41 +0100)]
fix #3758: fix `to` filter with empty NoqueueEntry `to` field
We only compared the filter `to` with the Noqueue `to` when both were
not empty, and in turn set the nq.dstatus to DStatus::Invalid when they
didn't match.
But in the case of an empty Noqueue `to`, we have to skip it as well if
the filter `to` is not empty.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Thu, 29 Apr 2021 13:25:08 +0000 (15:25 +0200)]
fix mutable borrow panic on duplicate msgid
Because of the relay before lmtp issue we now add and remove QEntrys
based on their message-id. But if the cleanup line containing the
message-id appears twice in the log for the same QEntry it keeps a weak
reference to itself which leads to a panic because of a mutable borrow
while it is already mutably borrowed in the 'finalize' function.
To circumvent this we check after the lookup if it is the same QEntry
and if so, insert it again with that message-id in the lookup table and
don't add the weak reference.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Tested-By: Aaron Lauterer <a.lauterer@proxmox.com>
Mira Limbeck [Tue, 23 Mar 2021 14:28:22 +0000 (15:28 +0100)]
after-queue: fix wrong DStatus if relay is 'removed' before 'lmtp' line
We only match a QEntry with the relay Qentry in the after-queue case
when we get to the 'lmtp' line. This is because it contains a reference
from the first QEntry to the filter, and the filter contains a reference
to the relay QEntry.
If the relay log entries are finished before the 'lmtp' line, there's no
way to match the relay to the first QEntry and we still assume it is
before-queue filtered.
To fix this we try to match the QEntries via message-id and add weak
references to each other. Then we wait with finalizing either until the
other is also 'removed' (finished).
The message-id matching might break if the same message-id is used for
different entries, so this is rather a 'hack' than a nice solution, but
there's no other info in the logs we could use to match both QEntries.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 23 Mar 2021 11:05:31 +0000 (12:05 +0100)]
close #2106: show outgoing TLS connection in tracking center
This is a best effort try to add the outgoing TLS connection information
to the output of pmg-log-tracker. The only thing we can match on is the
PID of the 'smtp' process. In the code we asumme that the TLS log entry
always happens before the actual smtp send entry that has a QID. This means
we save the TLS log entry in a map with the PID as key and then, once the
send entry happens, we look it up and add the log entry to the QEntry's
logs.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Mon, 22 Mar 2021 13:06:18 +0000 (14:06 +0100)]
change tests to use faketime
We only have the month, day, hours, minutes and seconds available to us,
but not the year. Because of this we assume the logs are always relative
to the current year. This means when running the log tracker the CTIME
is calculated based on the current year and once a new year starts, all
old tests have to be updated to match the new year.
To work around this issue, use 'faketime' to set the time to 2020-12-31
23:59:59 as all tests assume 2020 as the year.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Mira Limbeck [Mon, 9 Nov 2020 14:18:46 +0000 (15:18 +0100)]
change case sensitive string match to case insensitive
With the rewrite from C to Rust the search string match was changed to
be case sensitive by accident. Fix this by comparing the lowercase values
of both the input and the search string.
Mira Limbeck [Fri, 28 Aug 2020 10:19:51 +0000 (12:19 +0200)]
add output for proxy-reject with 'Syntax:' message
Messages like the following
'proxy-reject: END-OF-MESSAGE: 501 5.5.2 Syntax: MAIL FROM: <address>;'
can happen if an EHLO keyword is announced which is not handled by
pmg-smtp-filter (see #2795). This patch adds output to the log tracker
so this mail shows up as 'rejected' in the GUI instead of silently
ignoring it.
Mira Limbeck [Tue, 19 May 2020 13:31:06 +0000 (15:31 +0200)]
fix ToEntrys in FEntrys not being considered for 'to' filtering
When a 'to' filter is specified and a mail has multiple 'to's with at
least one being accepted, the others blocked or quarantined and the
'to' for the accepted mail does not match, the entries in the filter
will be ignored even if one of them would match.
To fix it the filter ToEntrys are now considered in the 'to' check.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 19 May 2020 13:31:05 +0000 (15:31 +0200)]
fix before-queue filter string_match check
In before-queue filtering a filter can be set on the SEntry. When a
search-string is set, we have to check both the SEntry and the attached
filter for a match, not just the SEntry. Otherwise some entries are not shown.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Fri, 15 May 2020 14:49:05 +0000 (16:49 +0200)]
print blocked/quarantined entries if accepted mail exists
In the case of before-queue filtering we have the separation 'QEntry'
for accepted mails, as those are the only ones with a queue. And
'SEntry' for blocked and quarantined mails. But if a mail is accepted
for one address, but at the same time blocked or quarantined for another
address we previously discarded those entries. To fix it we now print it
in the 'QEntry' if the attached filter contains blocked and/or
quarantined mails.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Fri, 15 May 2020 14:49:04 +0000 (16:49 +0200)]
make QID parsing more robust
As a user had the problem that 'fa' of 'fatal' matched for a QID,
increase the number of characters that have to match. The QID always has
at least 5 characters for the microseconds, so increase it to 5.
See http://www.postfix.org/postconf.5.html#enable_long_queue_ids
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Currently the status of a before queue filtered mail is the status of the
downstream delivery in case the downstream rejects or defers a mail.
This information is too little to distinguish between a mail which went
through the rule-system, and a locally generated one
(e.g. bounce or spamreport).
By introducing 3 new distinct dstatus values we can use the information to
correctly display the status of a mail in the GUI.
Reviewed-By: Mira Limbeck <m.limbeck@proxmox.com> Tested-By: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Mira Limbeck [Fri, 24 Apr 2020 15:21:49 +0000 (17:21 +0200)]
fix from entries being shown even with a from filter
When a 'from' and 'to' filter were set, we used a 'from || to' test to
see if we should print it or not. This patch changes this to a 'from &&
to' test. This gets rid of entries from unrelated senders ('from') when
a 'from filter did not match but the 'to' filter matched.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Thu, 2 Apr 2020 14:57:14 +0000 (16:57 +0200)]
add a test case for recipient filtering
Add a before queue test case to test recipient filtering. Previously
all SEntry filter entries were printed because filtering failed. Now
test that it only prints the specified ones.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Fri, 20 Mar 2020 12:08:28 +0000 (13:08 +0100)]
fix SEntry filter in before-queue case
With before-queue filtering if we don't accept the mail (e.g.
quarantine), we don't have a queue which means we don't have a QEntry,
so the SEntry has to handle the filter entries (ToEntrys).
This means we can't just return from print() when either a 'from' or 'to'
filter is set or we exclude greylist entries or NDRs and no Noqueue entries
exist or no entry matches any of the filters.
So continue printing if there is no filter parameter set, but an FEntry
reference in the SEntry. If there's an FEntry reference, compare all ToEntrys
to the filter parameter and return if there is no match at all.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
Mira Limbeck [Tue, 18 Feb 2020 15:36:39 +0000 (16:36 +0100)]
add tests
Add tests for some command line options. Not all have a test yet, but at
least most of the ones used by the GUI (-s, -e, -q, -x). '-g' and '-n' are
currently still missing.
The tests will only be valid until end of January 31st 2021 because of
missing year info in the syslog.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Mira Limbeck [Tue, 18 Feb 2020 15:36:38 +0000 (16:36 +0100)]
add before queue filter support
Initial before queue filter support. Requires a patch to pmg-api to add
the pmg-smtp-filter ID to the replies on a reject to correctly match the
pmg-smtp-filter to the smtpd.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Mira Limbeck [Tue, 18 Feb 2020 15:36:36 +0000 (16:36 +0100)]
rewrite in rust
pmg-log-tracker has been rewritten in Rust. Functionality is the same.
Output sometimes has a different order than the pmg-log-tracker in C.
This only happens when the time of the entries match.
There's one change regarding the interface. In addition to the short
versions of arguments also long versions exist.
The implementation uses Rc<>, Weak<> and RefCell<> to make holding mutable
cross-references possible, without having to change the original logic
completely. This allowed for easier translation from C to Rust.
The file debian/cargo-checksum.json is required by dh-cargo, otherwise
it won't compile. The cargo-checksum.json should contain the upstream
.crate file which does not exist in this case, so we just create an
empty one with the required keys. (see 'Library package structure' in
https://wiki.debian.org/Teams/RustPackaging/Policy)
The change to the minimum version of debhelper required was done
according to other rust packages (rust-clap, rust-bindgen, rust-ripgrep).
Adds a README that provides an overview of the stages a mail passes
through and what we can use to match those together for a single mail.
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dominik Csapak [Thu, 21 Feb 2019 08:42:02 +0000 (09:42 +0100)]
fix csum calculation and update checksums
it was intended to be a rol instead of a simple left shift which changes the
checksums for the various programs (like 'proxprox', 'postfix/postscreen',
etc.) so we update those and use defines instead of the hardcoded hex values.
this makes it much more readable and less error prone
projects / pmg-log-tracker.git / log