add disclaimer position/separator docs

[pmg-docs.git] / pmg-mail-filter.adoc

[[chapter_mailfilter]]
Rule-Based Mail Filter
======================

{pmg} ships with a highly configurable mail filter. This provides an
easy but powerful way to define filter rules by user, domain, time
frame, content type, and resulting action.

[thumbnail="pmg-gui-mail-filter-rules.png", big=1]

Every rule has 5 categories ('FROM', 'TO', 'WHEN', 'WHAT', and
'ACTION'), and each category may contain several objects to match
certain criteria:

'Who' - objects::

Who is the sender or recipient of the email? Those objects can be used
for the 'TO' and/or 'FROM' category.
+
====
Example: EMail-object - Who is the sender or recipient of the email?
====

'What' - objects::

What is in the email?
+
====
Example: Does the email contain spam?
====

'When' - objects::

When is the email received by {pmg}?
+
====
Example: Office Hours - Mail is received between 8:00 and 16:00.
====

'Action' - objects::

Defines the final actions.
+
====
Example: Mark email with “SPAM:” in the subject.
====

Rules are ordered by priority, so rules with higher priority are
executed first. It is also possible to set a processing direction:

'In'::  Rule applies to all incoming emails

'Out':: Rule applies to all outgoing emails

'In & Out':: Rule applies to both directions

You can also disable a rule completely, which is mostly useful for
testing and debugging. The 'Factory Defaults' button allows you to
reset the filter rules.

Application of Rules
--------------------

When there is more than one object category or multiple objects configured
within a single rule, the following logic is used to determine if the rule
should be applied by default:

* Within one category (WHAT/FROM/TO/WHEN), all objects are logical-or linked,
  meaning that only one object of any one object group from the same category
  has to match for the whole category to match.

* FROM/TO/WHAT/WHEN category match results are logical-and linked, so all
  categories that have at least one object in them must match for the rule to
  match.

When these conditions are met, all configured actions are executed.

Alternatively, one can configure the 'mode' to 'any' (the default) or 'all' and
set 'invert' (default off) per object group and per object category for each
rule.

When the mode is 'all' for a group, all objects within must match for the
object group to count as a match. This can be helpful when one wants to match
multiple conditions at the same time (e.g. file content-type and filename).

When 'all' is set for a category of a rule, all object groups for that
type must match for the type to match.

When 'invert' is active on a group, the original result of the group will
simply be inverted, so a match becomes a non-match and vice versa.

The same is true for the object group types for rules.

Special handling is done for WHAT matches that mark mail parts (e.g. filename)
since that is not a simple yes/no match for the complete mail, but could be a
match for each part of the e-mail (e.g. attachments, or parts of a multi-part
e-mail).

So for WHAT match object groups, the 'mode' and 'invert' is applied to
the single parts of the e-mail, not the message as a whole.

This means one has to be very careful with the 'invert' option, as previously
not matching parts, will match when using 'invert' (e.g. an inverted filename
matching will also mark non attachment parts of the mail).

On the rule level, these marks of the parts will always be logical-or linked,
this way even more scenarios can be represented.

To make it a bit easier to understand, the options are combined to a single
selection in the web ui:

* Any must match => mode: 'any', invert: 'off'
* All must match => mode: 'all', invert: 'off'
* At least one must not match => mode: 'all', invert: 'on'
* None must match => mode: 'any', invert: 'on'

[[pmg_mailfilter_action]]
'Action' - objects
------------------

[thumbnail="pmg-gui-mail-filter-actions.png", big=1]

Please note that some actions stop further rule processing. We call
such actions 'final'.

Accept
~~~~~~

Accept mail for Delivery. This is a 'final' action.


Block
~~~~~

Block mail. This is a 'final' action.


Quarantine
~~~~~~~~~~

Move to quarantine (virus mails are moved to the “virus quarantine”;
other mails are moved to “spam quarantine”). This is also a 'final' action.


Notification
~~~~~~~~~~~~

Send notifications. Please note that object configuration can use
xref:rule_system_macros[macros], so it is easy to include additional
information. For example, the default 'Notify Admin' object sends the
following information:

.Sample notification action body:
----
Proxmox Notification:
Sender:   __SENDER__
Receiver: __RECEIVERS__
Targets:  __TARGETS__
Subject: __SUBJECT__
Matching Rule: __RULE__

__RULE_INFO__

__VIRUS_INFO__
__SPAM_INFO__
----

Notification can also include a copy of the original mail.


Blind Carbon Copy (BCC)
~~~~~~~~~~~~~~~~~~~~~~~

The BCC object simply sends a copy to another target. It is possible to
send the original unmodified mail, or the processed result. Please
note that this can be quite different, for instance, when a previous rule
removed attachments.


Header Attributes
~~~~~~~~~~~~~~~~~

This object is able to add or modify mail header attributes. As with
Notifications above, you can use xref:rule_system_macros[macros],
making this a very powerful object. For example, the 'Modify Spam
Level' actions add detailed information about detected Spam
characteristics to the `X-SPAM-LEVEL` header.

.'Modify Spam Level' Header Attribute
----
Field: X-SPAM-LEVEL
Value: __SPAM_INFO__
----

Another prominent example is the 'Modify Spam Subject' action. This
simply adds the 'SPAM:' prefix to the original mail subject:

.'Modify Spam Subject' Header Attribute
----
Field: subject
Value: SPAM: __SUBJECT__
----


Remove attachments
~~~~~~~~~~~~~~~~~~

Remove attachments can either remove all attachments, or only those
matched by the rule's 'What' - object. You can also specify the
replacement text, if you want.

You can optionally move these mails into the attachment quarantine, where
the original mail with all attachments will be stored. The mail with the
attachments removed will continue through the rule system.

NOTE: The Attachment Quarantine lifetime is the same as for the Spam Quarantine.


Disclaimer
~~~~~~~~~~

Add a Disclaimer.

The disclaimer can contain HTML markup. It will be added to the first
`text/html` and `text/plain` part of an email. A disclaimer only gets added if
its text can be encoded in the mail's character encoding.

By default it will be appended at the end of the selected part of the mail with
`--` as a separator. The position (start or end of the selected part) and the
existence of the separator can be configured with the `position` and
`add-separator` options respectively.


[[pmg_mailfilter_who]]
'Who' objects
-------------

[thumbnail="pmg-gui-mail-filter-who-objects.png", big=1]

These types of objects can be used for the 'TO' and/or 'FROM' category,
and match the sender or recipient of the email. A single object can
combine multiple items, and the following item types are available:

EMail::

Allows you to match a single mail address.

Domain::

Only match the domain part of the mail address.

Regular Expression::

This one uses a regular expression to match the whole mail address.

IP Address or Network::

This can be used to match the senders IP address.

LDAP User or Group::

Test if the mail address belongs to a specific LDAP user or group.

We have two important 'Who' objects called 'Blacklist' and
'Whitelist'. These are used in the default ruleset to globally block
or allow specific senders.


[[pmg_mailfilter_what]]
'What' objects
--------------

[thumbnail="pmg-gui-mail-filter-what-objects.png", big=1]

'What' objects are used to classify the mail's content. A single
object can combine multiple items, and the following item types are
available:

Spam Filter::

Matches if the detected spam level is greater than or equal to the
configured value.

Virus Filter::

Matches on infected mails.

Match Field::

Match specified mail header fields (for example, `Subject:`, `From:`, ...)

Content Type Filter::

Can be used to match specific content types.

Match Filename::

Uses regular expressions to match attachment filenames.

Archive Filter::

Can be used to match specific content types inside archives.
This also matches the content-types of all regular (non-archived) attachments.

Match Archive Filename::

Uses regular expressions to match attachment filenames inside archives.
This also matches the filenames for all regular (non-archived) attachments.


[[pmg_mailfilter_when]]
'When' objects
--------------

[thumbnail="pmg-gui-mail-filter-when-objects.png", big=1]

'When' objects are used to activate rules at specific times of the
day. You can compose them from one or more time frame items.

The default ruleset defines 'Office Hours', but this is not used by
the default rules.


[[pmg_mailfilter_regex]]
Using regular expressions
-------------------------

A regular expression is a string of characters which represents a list
of text patterns which you would like to match. The following is a
short introduction to the syntax of regular expressions used by some
objects. If you are familiar with Perl, you will already know the
syntax.

Simple regular expressions
~~~~~~~~~~~~~~~~~~~~~~~~~~

In its simplest form, a regular expression is just a word or phrase to
search for. `Mail` would match the string "Mail". The search is case
sensitive so "MAIL", "Mail", "mail" would not be matched.

Metacharacters
~~~~~~~~~~~~~~

Some characters have a special meaning. These characters are called
metacharacters.  The Period (`.`) is a commonly used metacharacter. It
matches exactly one character, regardless of what the character is.
`e.mail` would match either "e-mail" or "e2mail" but not
"e-some-mail" or "email".

The question mark (`?`) indicates that the character immediately
preceding it shows up either zero or one time. `e?mail` would match
either "email" or "mail" but not "e-mail".

Another metacharacter is the asterisk (`*`). This indicates that the
character immediately preceding it may be repeated any number of times,
including zero. `e*mail` would match "email", "mail", and
"eeemail".

The plus (`+`) metacharacter indicates that the character immediately
preceding it appears one or more times. So `e+mail` does not match
"mail".

Metacharacters can also be combined. A common combination includes the
period and asterisk metacharacters (`.*`), with the asterisk
immediately following the period. This is used to match an arbitrary
string of any length, including the null string. For example:
`.*company.*` matches "company@domain.com" or "company@domain.co.uk"
or "department.company@domain.com".

The book xref:Friedl97[] provides a more comprehensive introduction.