`/etc/network/interfaces`::
-Network setup. We never modify this files directly. Instead, we write
+Network setup. We never modify this file directly. Instead, we write
changes to `/etc/network/interfaces.new`. When you reboot, we rename
-the file to `/etc/network/interfaces`, so any changes gets activated
+the file to `/etc/network/interfaces`, so the changes are applied
on the next reboot.
`/etc/resolv.conf`::
-DNS search domain and nameserver setup.
+DNS search domain and nameserver setup. {pmg} uses the search domain setting
+to create the FQDN and domain name used in the postfix configuration.
`/etc/hostname`::
-The system's host name.
+The system's host name. {pmg} uses the hostname to create the FQDN used
+in the postfix configuration.
`/etc/hosts`::
The list of relay domains.
+`/etc/pmg/dkim/domains`::
+
+The list of domains for outbound DKIM signing.
+
`/etc/pmg/fetchmailrc`::
Fetchmail configuration (POP3 and IMAP setup).
Custom {spamassassin} setup.
+`/etc/mail/spamassassin/pmg-scores.cf`::
+
+Custom {spamassassin} rule scores.
Keys and Certificates
---------------------
Key and certificate (combined) to encrypt mail traffic (TLS).
+`/etc/pmg/dkim/<selector>.private`::
+
+Key for DKIM signing mails with selector '<selector>'.
+
+[[pmgconfig_template_engine]]
Service Configuration Templates
-------------------------------
{pmg} uses various services to implement mail filtering, for example
the {postfix} Mail Transport Agent (MTA), the {clamav} antivirus
-engine and the Apache {spamassassin} project. Those services use
+engine and the Apache {spamassassin} project. These services use
separate configuration files, so we need to rewrite those files when
configuration is changed.
NOTE: Modified templates from `/etc/pmg/templates/` are automatically
synced from the master node to all cluster members.
+[[pmgconfig_whitelist_overview]]
+White- and Blacklists
+---------------------
+
+{pmg} has multiple white- and blacklists. It differentiates between the
+xref:pmgconfig_mailproxy_options[SMTP Whitelist]. The rule-based whitelist
+and the user whitelist.
+In addition to the whitelists there are 2 separate blacklists. The rule-based
+blacklist and the user blacklist.
+
+SMTP Whitelist
+~~~~~~~~~~~~~~
+
+The xref:pmgconfig_mailproxy_options[SMTP Whitelist] is responsible for disabling
+greylisting as well as SPF and DNSBL checks. These are done during the SMTP
+dialogue.
+
+Rule-based White-/Blacklist
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The xref:chapter_mailfilter[rule-based white- and blacklists] are predefined
+rules. They work by checking the attached 'Who' objects, containing e.g. a
+domain or a mail address, for a match. If it matches, the assigned action is
+used which by default is 'Accept' for the whitelist rule and 'Block' for the
+blacklist rule. In the default setup the blacklist rule has priority over the
+whitelist rule and spam checks.
+
+User White-/Blacklist
+~~~~~~~~~~~~~~~~~~~~~
+
+The user white- and blacklist are user specific. Every user can add mail addresses
+to their white- and blacklist. When a user adds a mail address to the whitelist,
+the result of the spam analysis will be discarded for that recipient. This can
+help the mail being accepted, but it still depends on the other rules what
+happens next. In the default setup this results in the mail being accepted for
+this recipient.
+
+For mail addresses on a user's blacklist the spam score will be increased by 100.
+It still depends on the rule system what happens when a spam score that high is
+encountered. In the default setup it will be recognized as spam and quarantined
+(spam score of 3 or higher).
[[pmgconfig_systemconfig]]
System Configuration
~~~~~~~~~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-network-config.png[]
+[thumbnail="pmg-gui-network-config.png", big=1]
endif::manvolnum[]
Normally the network and time is already configured when you visit the
.DNS recommendations
Many tests to detect SPAM mails use DNS queries, so it is important to
-have a fast and reliable DNS server. We also query some public
+have a fast and reliable DNS server. We also query some publicly
available DNS Blacklists. Most of them apply rate limits for clients,
so they simply will not work if you use a public DNS server (because
they are usually blocked). We recommend to use your own DNS server,
-which need to be configured in 'recursive' mode.
+which needs to be configured in 'recursive' mode.
Options
~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-system-options.png[]
+[thumbnail="pmg-gui-system-options.png", big=1]
endif::manvolnum[]
~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-relaying.png[]
+[thumbnail="pmg-gui-mailproxy-relaying.png", big=1]
endif::manvolnum[]
Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
~~~~~~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-relaydomains.png[]
+[thumbnail="pmg-gui-mailproxy-relaydomains.png", big=1]
endif::manvolnum[]
List of relayed mail domains, i.e. what destination domains this
~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-ports.png[]
+[thumbnail="pmg-gui-mailproxy-ports.png", big=1]
endif::manvolnum[]
Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-options.png[]
+[thumbnail="pmg-gui-mailproxy-options.png", big=1]
endif::manvolnum[]
Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
include::pmg.mail-options-conf-opts.adoc[]
+[[pmgconfig_mailproxy_before_after_queue]]
+Before and After Queue scanning
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Scanning email can happen at two different stages of mail-processing:
+
+* Before-queue filtering: During the SMTP Session, after the complete message
+ has been received (after the 'DATA' command).
+
+* After-queue filtering: After initially accepting the mail and putting it on
+ a queue for further processing.
+
+Before-queue filtering has the advantage that the system can reject a mail (by
+sending a permanent reject code '554'), and leave the task of notifying the
+original sender to the other mailserver. This is of particular advantage if
+the processed mail is a spam message or contains a virus and has a forged
+sender-address. Sending out a notification in this situation leads so-called
+'backscatter' mail, which might cause your server to get listed as spamming on
+RBLs (Real-time Blackhole List).
+
+After-queue filtering has the advantage of providing faster delivery of
+mails for the sending servers, since queueing mails is much faster than
+analyzing it for spam and viruses.
+
+If a mail is addressed to multiple recipients (e.g. when multiple addresses are
+subscribed to the same mailing list) the situation is more complicated: Your
+mailserver can only reject or accept the mail for all recipients, after having
+received the complete message, while your rule setup might accept the mail for
+part of the recipients and reject it for others. This can be due to a
+complicated rule setup, or if your users use the 'User White- and Blacklist'
+feature.
+
+If the resulting action of the rule system is the same for all recipients {pmg}
+responds accordingly if configured for before queue filtering (sending '554'
+for a blocked mail and '250' for an accepted or quarantined mail). If some
+mailboxes accept the mail and some reject it, the system has to accept the mail.
+
+Whether {pmg} notifies the sender that delivery failed for some recipients by
+sending a non-delivery report, depends on the 'ndr_on_block' setting in
+'/etc/pmg/pmg.conf'. If enabled an NDR is sent. Keeping it disabled prevents
+NDRs being sent to the (possibly forged) sender and thus minimizes the chance
+of getting your IP listed on a RBL. However in certain environments it can be
+unacceptable not to inform the sender about a rejected mail.
+
+The setting has the same effect if after queue filtering is configured, with
+the exception that an NDR is always sent out, even if all recipients block the
+mail, since the mail already got accepted before being analyzed.
+
+The details of integrating the mail proxy with {postfix} in both setups are
+explained in {postfix_beforequeue} and {postfix_afterqueue} respectively.
+
+
+[[pmgconfig_mailproxy_greylisting]]
+Greylisting
+~~~~~~~~~~~
+
+Greylisting is a technique for preventing unwanted messages from reaching the
+resource intensive stages of content analysis (virus detection and spam
+detection): By initially replying with a temporary failure code ('450') to
+each new email, the {pmg} tells the sending server that it should queue the
+mail and retry delivery at a later moment. Since certain kinds of spam get
+sent out by software, which has no provisioning for queueing, these mails are
+dropped without reaching {pmg} or your mailbox.
+
+The downside of greylisting is the delay introduced by the initial deferral of
+the email, which usually amounts to less than 30 minutes.
+
+In order to prevent unnecessary delays in delivery from known sources, emails
+coming from a source for a recipient, which have passed greylisting in the
+past are directly passed on: For each email the triple '<sender network,
+sender email, recipient email>' is stored in a list, along with the time when
+delivery was attempted. If an email fits an already existing triple, the
+timestamp for that triple is updated and the email is accepted for further
+processing.
+
+As long as a sender and recipient do communicate frequently there is no delay
+introduced by enabling greylisting. A triple is removed after a longer period
+of time, when no mail fitting that triple has been seen. The timeouts in {pmg}
+are:
+
+* 2 days for the retry of the first delivery
+
+* 36 days for known triples
+
+Mails with an empty envelope-sender are always delayed.
+
+Some email service providers send out emails for one domain from multiple
+servers. To prevent delays due to an email coming in from 2 separate IPs of
+the same provider the triples store a network ('cidr') instead of a single IP.
+For certain large providers the default network size might be too small. You
+can configure the netmask applied to an IP for the greylist lookup in
+'/etc/pmg/pmg.conf' or in the GUI with the settings 'greylistmask' for IPv4
+and 'greylistmask6' for IPv6 respectively.
+
+
[[pmgconfig_mailproxy_transports]]
Transports
~~~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-transports.png[]
+[thumbnail="pmg-gui-mailproxy-transports.png", big=1]
endif::manvolnum[]
-You can use {pmg} to send e-mails to different internal
-e-mail servers. For example you can send e-mails addressed to
-domain.com to your first e-mail server, and e-mails addressed to
+You can use {pmg} to send emails to different internal
+email servers. For example you can send emails addressed to
+domain.com to your first email server, and emails addressed to
subdomain.domain.com to a second one.
-You can add the IP addresses, hostname and SMTP ports and mail domains (or
-just single email addresses) of your additional e-mail servers.
+You can add the IP addresses, hostname, transport protocol (smtp/lmtp),
+transport ports and mail domains (or just single email addresses)
+of your additional email servers. When transport protocol is set to `lmtp`,
+the option 'Use MX' is useless and will be automatically set to 'No'.
[[pmgconfig_mailproxy_networks]]
~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-networks.png[]
+[thumbnail="pmg-gui-mailproxy-networks.png", big=1]
endif::manvolnum[]
You can add additional internal (trusted) IP networks or hosts.
~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-tls.png[]
+[thumbnail="pmg-gui-mailproxy-tls.png", big=1]
endif::manvolnum[]
Transport Layer Security (TLS) provides certificate-based
{pmg} uses opportunistic TLS encryption by default. The SMTP transaction is
encrypted if the 'STARTTLS' ESMTP feature is supported by the remote
-server. Otherwise, messages are sent in the clear.
-You can set a different TLS policy per desitination domain, should you for
-example need to prevent e-mail delivery without encryption, or to work around
-a broken 'STARTTLS' ESMTP implementation. See {postfix_tls_readme} for details
-on the supported policies.
+server. Otherwise, messages are sent in the clear.
+
+You can set a different TLS policy per destination. A destination is either a
+remote domain or a next-hop destination as specified in `/etc/pmg/transport`.
+This can be used if you need to prevent email delivery without
+encryption, or to work around a broken 'STARTTLS' ESMTP implementation. See
+{postfix_tls_readme} for details on the supported policies.
Enable TLS logging::
To get additional information about SMTP TLS activity you can enable
TLS logging. That way information about TLS sessions and used
-certificate’s is logged via syslog.
+certificates is logged via syslog.
Add TLS received header::
DKIM Signing
~~~~~~~~~~~~
+ifndef::manvolnum[]
+[thumbnail="pmg-gui-mailproxy-dkim.png", big=1]
+endif::manvolnum[]
+
DomainKeys Identified Mail (DKIM) Signatures (see {dkim_rfc}) is a method to
cryptographically authenticate a mail as originating from a particular domain.
Before sending the mail a hash over certain header fields and the body is
The verification is done by the receiver: The public key is fetched
via DNS TXT lookup for `yourselector._domainkey.yourdomain.example` and used
for verifying the hash. You can publish multiple selectors for your domain,
-each use by a system which sends e-mail from your domain, without the need to
+each used by a system which sends email from your domain, without the need to
share the private key.
{pmg} verifies DKIM Signatures for inbound mail in the Spam Filter by default.
`CC`, `Reply-To` and `Subject` get oversigned.
You can either sign all mails received on the internal port using the domain of
-the envelope sender address or create a list of domains, for which e-mails
+the envelope sender address or create a list of domains, for which emails
should be signed, defaulting to the list of relay domains.
Selector::
The selector used for signing the mail. The private key used for signing is
-saved under `/etc/pmg/yourselector.private`. You can display the DNS TXT
+saved under `/etc/pmg/dkim/yourselector.private`. You can display the DNS TXT
record which you need to add to all domains signed by {pmg} by clicking on the
'View DNS Record' Button.
~~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-mailproxy-whitelist.png[]
+[thumbnail="pmg-gui-mailproxy-whitelist.png", big=1]
endif::manvolnum[]
-All SMTP checks are disabled for those entries (e. g. Greylisting,
-SPF, RBL, ...)
+All SMTP checks are disabled for those entries (e.g. Greylisting,
+SPF, DNSBL, ...)
+
+DNSBL checks are done by `postscreen` which works on IP addresses and networks.
+This means it can only make use of the `IP Address` and `IP Network` entries.
NOTE: If you use a backup MX server (e.g. your ISP offers this service
for you) you should always add those servers here.
+NOTE: To disable DNSBL checks entirely, remove any `DNSBL Sites` entries in
+xref:pmgconfig_mailproxy_options[Mail Proxy Options].
[[pmgconfig_spamdetector]]
Spam Detector Configuration
~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-spam-options.png[]
+[thumbnail="pmg-gui-spam-options.png", big=1]
endif::manvolnum[]
{pmg} uses a wide variety of local and network tests to identify spam
signatures. This makes it harder for spammers to identify one aspect
which they can craft their messages to work around the spam filter.
-Every single e-mail will be analyzed and gets a spam score
+Every single email will be analyzed and gets a spam score
assigned. The system attempts to optimize the efficiency of the rules
that are run in terms of minimizing the number of false positives and
false negatives.
~~~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-spamquar-options.png[]
+[thumbnail="pmg-gui-spamquar-options.png", big=1]
endif::manvolnum[]
-Proxmox analyses all incoming e-mail messages and decides for each
-e-mail if its ham or spam (or virus). Good e-mails are delivered to
-the inbox and spam messages can be moved into the spam quarantine.
+{pmg} analyses all incoming email messages and decides for each
+email if it is ham or spam (or virus). Good emails are delivered to
+the inbox and spam messages are moved into the spam quarantine.
The system can be configured to send daily reports to inform users
-about the personal spam messages received the last day. That report is
+about the personal spam messages received the last day. The report is
only sent if there are new messages in the quarantine.
Some options are only available in the config file `/etc/pmg/pmg.conf`,
-and not in the webinterface.
+and not in the web interface.
include::pmg.spamquar-conf-opts.adoc[]
+[[pmgconfig_spamdetector_customscores]]
+Customization of Rulescores
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ifndef::manvolnum[]
+[thumbnail="pmg-gui-spam-custom-scores.png", big=1]
+endif::manvolnum[]
+
+While the default scoring of {spamassassin}'s ruleset provides very good
+detection rates, sometimes your particular environment can benefit from
+slightly adjusting the score of a particular rule. Two examples:
+
+* Your system receives spam mails which are scored at 4.9 and you have
+ a rule which puts all mails above 5 in the quarantine. The one thing the
+ spam mails have in common is that they all hit 'URIBL_BLACK'. By increasing
+ the score of this rule by 0.2 points the spam mails would all be quarantined
+ instead of being sent to your users
+
+* Your system tags many legitimate mails from a partner organization as spam,
+ because the organization has a policy that each mail has to start with
+ 'Dear madam or sir' (generating 1.9 points through the rule
+ 'DEAR_SOMETHING'). By setting the score of this rule to 0 you can disable
+ it completely.
+
+The system logs all the rules which a particular mail hits. Analyzing the logs can
+lead to finding such a pattern in your environment.
+
+You can adjust the score of a rule by creating a new 'Custom Rule Score' entry
+in the GUI.
+
+NOTE: In general it is strongly recommended to not make large changes to the
+default scores.
+
+
[[pmgconfig_clamav]]
Virus Detector Configuration
----------------------------
~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-virus-options.png[]
+[thumbnail="pmg-gui-virus-options.png", big=1]
endif::manvolnum[]
All mails are automatically passed to the included virus detector
-({clamav}). The default setting are considered safe, so it is usually
+({clamav}). The default settings are considered safe, so it is usually
not required to change them.
{clamav} related settings are saved to subsection 'clamav' in `/etc/pmg/pmg.conf`,
include::pmg.clamav-conf-opts.adoc[]
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-clamav-database.png[]
+[thumbnail="pmg-gui-clamav-database.png", big=1]
endif::manvolnum[]
-Please note that the virus signature database it automatically
-updated. But you can see the database status on the GUI, and you can
+Please note that the virus signature database is automatically
+updated. You can see the database status in the GUI, and also
trigger manual updates there.
~~~~~~~~~~
ifndef::manvolnum[]
-image::images/screenshot/pmg-gui-virusquar-options.png[]
+[thumbnail="pmg-gui-virusquar-options.png", big=1]
endif::manvolnum[]
Indentified virus mails are automatically moved to the virus
-quarantine. The administartor can view those mails using the GUI, or
-deliver them in case of false positives. {pmg} does not notify
+quarantine. The administrator can view these mails using the GUI, and
+choose to deliver them in case of false positives. {pmg} does not notify
individual users about received virus mails.
Virus quarantine related settings are saved to subsection 'virusquar'
Custom SpamAssassin configuration
---------------------------------
-This is only for advanced users. To add or change the Proxmox
-{spamassassin} configuration please login to the console via SSH. Go
-to directory `/etc/mail/spamassassin/`. In this directory there are several
-files (`init.pre`, `local.cf`, ...) – do not change them.
-
-To add your special configuration, you have to create a new file and
-name it `custom.cf` (in this directory), then add your
-configuration there. Be aware to use the {spamassassin}
-syntax, and test with
+This is only for advanced users. {spamassassin}'s rules and their associated
+scores get updated regularly and are trained on a huge corpus, which gets
+classified by experts. In most cases adding a rule for matching a particular
+keyword is the wrong approach, leading to many false positives. Usually bad
+detection rates are better addressed by properly setting up DNS than by adding
+a custom rule - watch out for matches to 'URIBL_BLOCKED' in the logs or
+spam-headers - see the {spamassassin_dnsbl}.
+
+To add or change the Proxmox {spamassassin} configuration please login to the
+console via SSH. Change to the `/etc/mail/spamassassin/` directory. In this
+directory there are several files (`init.pre`, `local.cf`, ...) - do not change
+them, as `init.pre`, `v310.pre`, `v320.pre`, `local.cf` will be overwritten by
+the xref:pmgconfig_template_engine[template engine], while the others can
+get updated by any {spamassassin} package upgrade.
+
+To add your custom configuration, you have to create a new file and name it
+`custom.cf` (in this directory), then add your configuration there. Make sure
+to use the correct {spamassassin} syntax, and test it with:
----
# spamassassin -D --lint
----
If you run a cluster, the `custom.cf` file is synchronized from the
-master node to all cluster members.
+master node to all cluster members automatically.
+
+To adjust the score assigned to a particular rule you
+can also use the xref:pmgconfig_spamdetector_customscores[Custom Rule Score]
+settings in the GUI.
[[pmgconfig_custom_check]]
Custom Check Interface
----------------------
-For use cases which are not handled by the {pmg} Virus Detector and
+For use-cases which are not handled by the {pmg} Virus Detector and
{spamassassin} configuration, advanced users can create a custom check
executable which, if enabled will be called before the Virus Detector and before
-passing an e-mail through the Rule System. The custom check API is kept as
+passing an email through the Rule System. The custom check API is kept as
simple as possible, while still providing a great deal of control over the
-treatment of an e-mail. Its input is passed via two CLI arguments:
+treatment of an email. Its input is passed via two CLI arguments:
* the 'api-version' (currently `v1`) - for potential future change of the
invocation
-* the 'queue-file-name' - a filename, which contains the complete e-mail as
+* the 'queue-file-name' - a filename, which contains the complete email as
rfc822/eml file
The expected output need to be printed on STDOUT and consists of two lines:
* the 'api-version' (currently 'v1') - see above
* one of the following 3 results:
-** 'OK' - e-mail is ok
-** 'VIRUS: <virusdescription>' - e-mail is treated as if it contained a virus
- (the virusdescription is logged and added to the e-mail's headers)
+** 'OK' - email is ok
+** 'VIRUS: <virusdescription>' - email is treated as if it contained a virus
+ (the virus description is logged and added to the email's headers)
** 'SCORE: <number>' - <number> is added (negative numbers are also possible)
- to the e-mail's spamscore
+ to the email's spamscore
The check is run with a 5 minute timeout - if it is exceeded the check
-executable is killed and the e-mail is treated as OK.
+executable is killed and the email is treated as OK.
All output written to STDERR by the check is written with priority 'err' to the
journal/mail.log.
Local Users
~~~~~~~~~~~
-image::images/screenshot/pmg-gui-local-user-config.png[]
+[thumbnail="pmg-gui-local-user-config.png", big=1]
-Local users are used to manage and audit {pmg}. Those users can login on the
+Local users can manage and audit {pmg}. They can login on the
management web interface.
There are three roles:
not to edit it.
In addition there is always the 'root' user, which is used to perform special
-system administrator tasks, such as updgrading a host or changing the
+system administrator tasks, such as upgrading a host or changing the
network configuration.
NOTE: Only pam users are able to login via the webconsole and ssh, which the
LDAP/Active Directory
~~~~~~~~~~~~~~~~~~~~~
-image::images/screenshot/pmg-gui-ldap-user-config.png[]
+[thumbnail="pmg-gui-ldap-user-config.png", big=1]
You can specify multiple LDAP/Active Directory profiles, so that you can
create rules matching those users and groups.
^^^^
{pmg} synchronizes the relevant user and group info periodically, so that
-that information is available in a fast manner, even when the LDAP/AD server
+the information is available in a fast manner, even when the LDAP/AD server
is temporarily not accessible.
-After a successfull sync, the groups and users should be visible on the web
+After a successful sync, the groups and users should be visible on the web
interface. After that, you can create rules targeting LDAP users and groups.
Fetchmail
~~~~~~~~~
-image::images/screenshot/pmg-gui-fetchmail-config.png[]
+[thumbnail="pmg-gui-fetchmail-config.png", big=1]
-Fetchmail is utility for polling and forwarding e-mails. You can define
-e-mail accounts, which will then be fetched and forwarded to the e-mail
+Fetchmail is utility for polling and forwarding emails. You can define
+email accounts, which will then be fetched and forwarded to the email
address you defined.
You have to add an entry for each account/target combination you want to
projects / pmg-docs.git / blobdiff