]> git.proxmox.com Git - pmg-docs.git/blame - pmgconfig.adoc
Add initial documentation for custom checks
[pmg-docs.git] / pmgconfig.adoc
CommitLineData
e62ceaf0
DM
1[[chapter_pmgconfig]]
2ifdef::manvolnum[]
3pmgconfig(1)
4============
5:pmg-toplevel:
6
7NAME
8----
9
10pmgconfig - Proxmox Mail Gateway Configuration Management Toolkit
11
12
13SYNOPSIS
14--------
15
16include::pmgconfig.1-synopsis.adoc[]
17
18
19DESCRIPTION
20-----------
21endif::manvolnum[]
22ifndef::manvolnum[]
66e9c719
DM
23Configuration Management
24========================
e62ceaf0
DM
25:pmg-toplevel:
26endif::manvolnum[]
27
685576c2
DM
28{pmg} is usually configured using the web-based Graphical User
29Interface (GUI), but it is also possible to directly edit the
30configuration files, use the REST API over 'https'
66e9c719 31or the command line tool `pmgsh`.
685576c2 32
66e9c719 33The command line tool `pmgconfig` is used to simplify some common
685576c2
DM
34configuration tasks, i.e. to generate cerificates and to rewrite
35service configuration files.
36
66e9c719
DM
37NOTE: We use a Postgres database to store mail filter rules and
38statistic data. See chapter xref:chapter_pmgdb[Database Management]
39for more information.
40
41
42Configuration files overview
43----------------------------
44
45`/etc/network/interfaces`::
46
47Network setup. We never modify this files directly. Instead, we write
48changes to `/etc/network/interfaces.new`. When you reboot, we rename
49the file to `/etc/network/interfaces`, so any changes gets activated
50on the next reboot.
51
9bfe27f3
DM
52`/etc/resolv.conf`::
53
54DNS search domain and nameserver setup.
55
56`/etc/hostname`::
57
58The system's host name.
59
60`/etc/hosts`::
61
62Static table lookup for hostnames.
63
66e9c719
DM
64`/etc/pmg/pmg.conf`::
65
66Stores common administration options, i.e. the spam and mail proxy setup.
67
68`/etc/pmg/cluster.conf`::
69
70The cluster setup.
71
72`/etc/pmg/domains`::
73
74The list of relay domains.
75
76`/etc/pmg/fetchmailrc`::
77
78Fetchmail configuration (POP3 and IMAP setup).
79
80`/etc/pmg/ldap.conf`::
81
82LDAP configuration.
83
84`/etc/pmg/mynetworks`::
85
86List of local (trusted) networks.
87
88`/etc/pmg/subscription`::
89
90Stores your subscription key and status.
91
37b2b051
SI
92`/etc/pmg/tls_policy`::
93
94TLS policy for outbound connections.
95
66e9c719
DM
96`/etc/pmg/transports`::
97
98Message delivery transport setup.
99
100`/etc/pmg/user.conf`::
101
102GUI user configuration.
103
797db11d
DM
104`/etc/mail/spamassassin/custom.cf`::
105
106Custom {spamassassin} setup.
107
66e9c719
DM
108
109Keys and Certificates
110---------------------
111
112`/etc/pmg/pmg-api.pem`::
113
114Key and certificate (combined) used be the HTTPs server (API).
115
116`/etc/pmg/pmg-authkey.key`::
117
118Privat key use to generate authentication tickets.
119
120`/etc/pmg/pmg-authkey.pub`::
121
122Public key use to verify authentication tickets.
123
124`/etc/pmg/pmg-csrf.key`::
125
126Internally used to generate CSRF tokens.
127
128`/etc/pmg/pmg-tls.pem`::
129
130Key and certificate (combined) to encrypt mail traffic (TLS).
131
132
133Service Configuration Templates
134-------------------------------
135
9c85cc80
DM
136{pmg} uses various services to implement mail filtering, for example
137the {postfix} Mail Transport Agent (MTA), the {clamav} antivirus
138engine and the Apache {spamassassin} project. Those services use
139separate configuration files, so we need to rewrite those files when
140configuration is changed.
141
142We use a template based approach to generate those files. The {tts} is
143a well known, fast and flexible template processing system. You can
144find the default templates in `/var/lib/pmg/templates/`. Please do not
145modify them directly, because your modification would get lost on the
9dd45bd7
SI
146next update. Instead, copy the template you wish to change to
147`/etc/pmg/templates/`, then apply your changes there.
9c85cc80
DM
148
149Templates can access any configuration setting, and you can use the
150`pmgconfig dump` command to get a list of all variable names:
151
152----
153# pmgconfig dump
154...
155dns.domain = yourdomain.tld
156dns.hostname = pmg
157ipconfig.int_ip = 192.168.2.127
158pmg.admin.advfilter = 1
159...
160----
161
162The same tool is used to force regeneration of all template based
163configuration files. You need to run that after modifying a template,
164or when you directly edit configuration files
165
166----
167# pmgconfig sync --restart 1
168----
169
9dd45bd7 170The above command also restarts services if the underlying configuration
9c85cc80
DM
171files are changed. Please note that this is automatically done when
172you change the configuration using the GUI or API.
173
174NOTE: Modified templates from `/etc/pmg/templates/` are automatically
175synced from the master node to all cluster members.
66e9c719
DM
176
177
4a08dffe 178[[pmgconfig_systemconfig]]
685576c2
DM
179System Configuration
180--------------------
181
182Network and Time
183~~~~~~~~~~~~~~~~
184
185ifndef::manvolnum[]
186image::images/screenshot/pmg-gui-network-config.png[]
187endif::manvolnum[]
188
45de5bf5 189Normally the network and time is already configured when you visit the
c6e27848 190GUI. The installer asks for those settings and sets up the correct
45de5bf5
DM
191values.
192
193The default setup uses a single Ethernet adapter and static IP
194assignment. The configuration is stored at '/etc/network/interfaces',
195and the actual network setup is done the standard Debian way using
196package 'ifupdown'.
197
198.Example network setup '/etc/network/interfaces'
199----
200source /etc/network/interfaces.d/*
201
202auto lo
203iface lo inet loopback
204
205auto ens18
206iface ens18 inet static
207 address 192.168.2.127
208 netmask 255.255.240.0
209 gateway 192.168.2.1
210----
211
212.DNS recommendations
213
214Many tests to detect SPAM mails use DNS queries, so it is important to
215have a fast and reliable DNS server. We also query some public
216available DNS Blacklists. Most of them apply rate limits for clients,
217so they simply will not work if you use a public DNS server (because
218they are usually blocked). We recommend to use your own DNS server,
219which need to be configured in 'recursive' mode.
685576c2
DM
220
221
222Options
223~~~~~~~
224
225ifndef::manvolnum[]
226image::images/screenshot/pmg-gui-system-options.png[]
227endif::manvolnum[]
228
e09057ab
DM
229
230Those settings are saved to subsection 'admin' in `/etc/pmg/pmg.conf`,
231using the following configuration keys:
232
685576c2
DM
233include::pmg.admin-conf-opts.adoc[]
234
c331641e
DM
235
236Mail Proxy Configuration
237------------------------
238
4a08dffe 239[[pmgconfig_mailproxy_relaying]]
c331641e
DM
240Relaying
241~~~~~~~~
242
c331641e
DM
243ifndef::manvolnum[]
244image::images/screenshot/pmg-gui-mailproxy-relaying.png[]
245endif::manvolnum[]
246
e09057ab
DM
247Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
248using the following configuration keys:
249
250include::pmg.mail-relaying-conf-opts.adoc[]
c331641e 251
4a08dffe 252[[pmgconfig_mailproxy_relay_domains]]
c331641e
DM
253Relay Domains
254~~~~~~~~~~~~~
255
c331641e
DM
256ifndef::manvolnum[]
257image::images/screenshot/pmg-gui-mailproxy-relaydomains.png[]
258endif::manvolnum[]
259
6822b369
DM
260List of relayed mail domains, i.e. what destination domains this
261system will relay mail to. The system will reject incoming mails to
262other domains.
c331641e 263
d9c56b22 264
4a08dffe 265[[pmgconfig_mailproxy_ports]]
c331641e
DM
266Ports
267~~~~~
268
c331641e
DM
269ifndef::manvolnum[]
270image::images/screenshot/pmg-gui-mailproxy-ports.png[]
271endif::manvolnum[]
272
d9c56b22
DM
273Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
274using the following configuration keys:
275
276include::pmg.mail-ports-conf-opts.adoc[]
277
c331641e 278
4a08dffe 279[[pmgconfig_mailproxy_options]]
c331641e
DM
280Options
281~~~~~~~
282
c331641e
DM
283ifndef::manvolnum[]
284image::images/screenshot/pmg-gui-mailproxy-options.png[]
285endif::manvolnum[]
286
e3d778e0
DM
287Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
288using the following configuration keys:
289
290include::pmg.mail-options-conf-opts.adoc[]
c331641e
DM
291
292
4a08dffe 293[[pmgconfig_mailproxy_transports]]
c331641e
DM
294Transports
295~~~~~~~~~~
296
297ifndef::manvolnum[]
298image::images/screenshot/pmg-gui-mailproxy-transports.png[]
299endif::manvolnum[]
300
b335e06b
DM
301You can use {pmg} to send e-mails to different internal
302e-mail servers. For example you can send e-mails addressed to
303domain.com to your first e-mail server, and e-mails addressed to
304subdomain.domain.com to a second one.
305
306You can add the IP addresses, hostname and SMTP ports and mail domains (or
307just single email addresses) of your additional e-mail servers.
c331641e
DM
308
309
4a08dffe 310[[pmgconfig_mailproxy_networks]]
c331641e
DM
311Networks
312~~~~~~~~
313
314ifndef::manvolnum[]
315image::images/screenshot/pmg-gui-mailproxy-networks.png[]
316endif::manvolnum[]
317
20e879ad
DM
318You can add additional internal (trusted) IP networks or hosts.
319All hosts in this list are allowed to relay.
320
321NOTE: Hosts in the same subnet with Proxmox can relay by default and
322it’s not needed to add them in this list.
c331641e
DM
323
324
4a08dffe 325[[pmgconfig_mailproxy_tls]]
c331641e
DM
326TLS
327~~~
328
329ifndef::manvolnum[]
330image::images/screenshot/pmg-gui-mailproxy-tls.png[]
331endif::manvolnum[]
332
20e879ad
DM
333Transport Layer Security (TLS) provides certificate-based
334authentication and encrypted sessions. An encrypted session protects
335the information that is transmitted with SMTP mail. When you activate
336TLS, {pmg} automatically generates a new self signed
337certificate for you (`/etc/pmg/pmg-tls.pem`).
338
37b2b051 339{pmg} uses opportunistic TLS encryption by default. The SMTP transaction is
20e879ad 340encrypted if the 'STARTTLS' ESMTP feature is supported by the remote
37b2b051
SI
341server. Otherwise, messages are sent in the clear.
342You can set a different TLS policy per desitination domain, should you for
343example need to prevent e-mail delivery without encryption, or to work around
344a broken 'STARTTLS' ESMTP implementation. See {postfix_tls_readme} for details
345on the supported policies.
20e879ad
DM
346
347Enable TLS logging::
348
349To get additional information about SMTP TLS activity you can enable
350TLS logging. That way information about TLS sessions and used
351certificate’s is logged via syslog.
352
353Add TLS received header::
354
355Set this option to include information about the protocol and cipher
356used as well as the client and issuer CommonName into the "Received:"
357message header.
358
a649b38f
DM
359Those settings are saved to subsection 'mail' in `/etc/pmg/pmg.conf`,
360using the following configuration keys:
361
362include::pmg.mail-tls-conf-opts.adoc[]
363
c331641e
DM
364
365Whitelist
366~~~~~~~~~
367
368ifndef::manvolnum[]
369image::images/screenshot/pmg-gui-mailproxy-whitelist.png[]
370endif::manvolnum[]
371
6822b369
DM
372All SMTP checks are disabled for those entries (e. g. Greylisting,
373SPF, RBL, ...)
374
375NOTE: If you use a backup MX server (e.g. your ISP offers this service
376for you) you should always add those servers here.
c331641e
DM
377
378
4a08dffe 379[[pmgconfig_spamdetector]]
c331641e
DM
380Spam Detector Configuration
381---------------------------
382
2d672352
DM
383Options
384~~~~~~~
385
74bfe8ba
DM
386ifndef::manvolnum[]
387image::images/screenshot/pmg-gui-spam-options.png[]
388endif::manvolnum[]
389
3371c521
DM
390{pmg} uses a wide variety of local and network tests to identify spam
391signatures. This makes it harder for spammers to identify one aspect
392which they can craft their messages to work around the spam filter.
393
394Every single e-mail will be analyzed and gets a spam score
395assigned. The system attempts to optimize the efficiency of the rules
396that are run in terms of minimizing the number of false positives and
397false negatives.
398
399include::pmg.spam-conf-opts.adoc[]
400
401
4a08dffe 402[[pmgconfig_spamdetector_quarantine]]
2d672352
DM
403Quarantine
404~~~~~~~~~~
3371c521 405
74bfe8ba
DM
406ifndef::manvolnum[]
407image::images/screenshot/pmg-gui-spamquar-options.png[]
408endif::manvolnum[]
409
3371c521
DM
410Proxmox analyses all incoming e-mail messages and decides for each
411e-mail if its ham or spam (or virus). Good e-mails are delivered to
412the inbox and spam messages can be moved into the spam quarantine.
413
414The system can be configured to send daily reports to inform users
415about the personal spam messages received the last day. That report is
416only sent if there are new messages in the quarantine.
417
ee34edb0
DC
418Some options are only available in the config file `/etc/pmg/pmg.conf`,
419and not in the webinterface.
420
3371c521 421include::pmg.spamquar-conf-opts.adoc[]
c331641e
DM
422
423
4a08dffe 424[[pmgconfig_clamav]]
c331641e
DM
425Virus Detector Configuration
426----------------------------
427
4a08dffe 428[[pmgconfig_clamav_options]]
2d672352
DM
429Options
430~~~~~~~
431
e7c18c7c
DM
432ifndef::manvolnum[]
433image::images/screenshot/pmg-gui-virus-options.png[]
434endif::manvolnum[]
435
0bfbbf88
DM
436All mails are automatically passed to the included virus detector
437({clamav}). The default setting are considered safe, so it is usually
438not required to change them.
439
440{clamav} related settings are saved to subsection 'clamav' in `/etc/pmg/pmg.conf`,
441using the following configuration keys:
442
443include::pmg.clamav-conf-opts.adoc[]
444
e7c18c7c
DM
445ifndef::manvolnum[]
446image::images/screenshot/pmg-gui-clamav-database.png[]
447endif::manvolnum[]
448
449Please note that the virus signature database it automatically
450updated. But you can see the database status on the GUI, and you can
451trigger manual updates there.
452
0bfbbf88 453
4a08dffe 454[[pmgconfig_clamav_quarantine]]
2d672352
DM
455Quarantine
456~~~~~~~~~~
0bfbbf88 457
e7c18c7c
DM
458ifndef::manvolnum[]
459image::images/screenshot/pmg-gui-virusquar-options.png[]
460endif::manvolnum[]
461
0bfbbf88
DM
462Indentified virus mails are automatically moved to the virus
463quarantine. The administartor can view those mails using the GUI, or
464deliver them in case of false positives. {pmg} does not notify
465individual users about received virus mails.
466
467Virus quarantine related settings are saved to subsection 'virusquar'
468in `/etc/pmg/pmg.conf`, using the following configuration keys:
469
470include::pmg.virusquar-conf-opts.adoc[]
c331641e
DM
471
472
7eff8815
DM
473Custom SpamAssassin configuration
474---------------------------------
475
476This is only for advanced users. To add or change the Proxmox
477{spamassassin} configuration please login to the console via SSH. Go
5de0ffd7 478to directory `/etc/mail/spamassassin/`. In this directory there are several
7eff8815
DM
479files (`init.pre`, `local.cf`, ...) – do not change them.
480
481To add your special configuration, you have to create a new file and
482name it `custom.cf` (in this directory), then add your
483configuration there. Be aware to use the {spamassassin}
484syntax, and test with
485
486----
487# spamassassin -D --lint
488----
489
490If you run a cluster, the `custom.cf` file is synchronized from the
491master node to all cluster members.
492
493
ed7970d8
SI
494[[pmgconfig_custom_check]]
495Custom Check Interface
496----------------------
497
498For use cases which are not handled by the {pmg} Virus Detector and
499{spamassassin} configuration, advanced users can create a custom check
500executable which, if enabled will be called before the Virus Detector and before
501passing an e-mail through the Rule System. The custom check API is kept as
502simple as possible, while still providing a great deal of control over the
503treatment of an e-mail. Its input is passed via two CLI arguments:
504
505* the 'api-version' (currently `v1`) - for potential future change of the
506 invocation
507
508* the 'queue-file-name' - a filename, which contains the complete e-mail as
509 rfc822/eml file
510
511The expected output need to be printed on STDOUT and consists of two lines:
512
513* the 'api-version' (currently 'v1') - see above
514
515* one of the following 3 results:
516** 'OK' - e-mail is ok
517** 'VIRUS: <virusdescription>' - e-mail is treated as if it contained a virus
518 (the virusdescription is logged and added to the e-mail's headers)
519** 'SCORE: <number>' - <number> is added (negative numbers are also possible)
520 to the e-mail's spamscore
521
522The check is run with a 5 minute timeout - if it is exceeded the check
523executable is killed and the e-mail is treated as OK.
524
525All output written to STDERR by the check is written with priority 'err' to the
526journal/mail.log.
527
528A simple sample script following the API (and yielding a random result) for
529reference:
530
531----
532#!/bin/sh
533
534echo "called with $*" 1>&2
535
536if [ "$#" -ne 2 ]; then
537 echo "usage: $0 APIVERSION QUEUEFILENAME" 1>&2
538 exit 1
539fi
540
541apiver="$1"
542shift
543
544if [ "$apiver" != "v1" ]; then
545 echo "wrong APIVERSION: $apiver" 1>&2
546 exit 2
547fi
548
549queue_file="$1"
550
551echo "v1"
552
553choice=$(shuf -i 0-3 -n1)
554
555case "$choice" in
556 0)
557 echo OK
558 ;;
559 1)
560 echo SCORE: 4
561 ;;
562 2)
563 echo VIRUS: Random Virus
564 ;;
565 3) #timeout-test
566 for i in $(seq 1 7); do
567 echo "custom checking mail: $queue_file - minute $i" 1>&2
568 sleep 60
569 done
570 ;;
571esac
572
573exit 0
574----
575
576The custom check needs to be enabled in the admin section of `/etc/pmg/pmg.conf`
577
578----
579section: admin
580 custom_check 1
581----
582
583The location of the custom check executable can also be set there with the key
584`custom_check_path` and defaults to `/usr/local/bin/pmg-custom-check`.
585
586
c331641e
DM
587User Management
588---------------
589
05336835
DC
590User management in {pmg} consists of three types of users/accounts:
591
592
4a08dffe 593[[pmgconfig_localuser]]
05336835
DC
594Local Users
595~~~~~~~~~~~
596
f02d2b90
DM
597image::images/screenshot/pmg-gui-local-user-config.png[]
598
05336835
DC
599Local users are used to manage and audit {pmg}. Those users can login on the
600management web interface.
601
602There are three roles:
603
604* Administrator
605+
606Is allowed to manage settings of {pmg}, except some tasks like
607network configuration and upgrading.
608
609* Quarantine manager
610+
611Is allowed to manage quarantines, blacklists and whitelists, but not other
612settings. Has no right to view any other data.
613
614* Auditor
615+
616With this role, the user is only allowed to view data and configuration, but
617not to edit it.
618
619In addition there is always the 'root' user, which is used to perform special
620system administrator tasks, such as updgrading a host or changing the
621network configuration.
622
623NOTE: Only pam users are able to login via the webconsole and ssh, which the
624users created with the web interface are not. Those users are created for
625{pmg} administration only.
626
627Local user related settings are saved in `/etc/pmg/user.conf`.
628
629For details of the fields see xref:pmg_user_configuration_file[user.conf]
630
4a08dffe 631[[pmgconfig_ldap]]
05336835
DC
632LDAP/Active Directory
633~~~~~~~~~~~~~~~~~~~~~
634
f02d2b90
DM
635image::images/screenshot/pmg-gui-ldap-user-config.png[]
636
05336835
DC
637You can specify multiple LDAP/Active Directory profiles, so that you can
638create rules matching those users and groups.
639
640Creating a profile requires (at least) the following:
641
642* profile name
643* protocol (LDAP or LDAPS; LDAPS is recommended)
644* at least one server
645* a user and password (if your server does not support anonymous binds)
646
647All other fields should work with the defaults for most setups, but can be
648used to customize the queries.
649
650The settings are saved to `/etc/pmg/ldap.conf`. Details for the options
651can be found here: xref:pmg_ldap_configuration_file[ldap.conf]
652
653Bind user
654^^^^^^^^^
655
656It is highly recommended that the user which you use for connecting to the
657LDAP server only has the permission to query the server. For LDAP servers
658(for example OpenLDAP or FreeIPA), the username has to be of a format like
659'uid=username,cn=users,cn=accounts,dc=domain' , where the specific fields are
660depending on your setup. For Active Directory servers, the format should be
661like 'username@domain' or 'domain\username'.
662
663Sync
664^^^^
665
666{pmg} synchronizes the relevant user and group info periodically, so that
667that information is available in a fast manner, even when the LDAP/AD server
668is temporarily not accessible.
669
670After a successfull sync, the groups and users should be visible on the web
671interface. After that, you can create rules targeting LDAP users and groups.
c331641e
DM
672
673
4a08dffe 674[[pmgconfig_fetchmail]]
8538d9a2 675Fetchmail
05336835
DC
676~~~~~~~~~
677
f02d2b90
DM
678image::images/screenshot/pmg-gui-fetchmail-config.png[]
679
05336835
DC
680Fetchmail is utility for polling and forwarding e-mails. You can define
681e-mail accounts, which will then be fetched and forwarded to the e-mail
682address you defined.
683
684You have to add an entry for each account/target combination you want to
685fetch and forward. Those will then be regularly polled and forwarded,
686according to your configuration.
687
688The API and web interface offer following configuration options:
8538d9a2
DM
689
690include::fetchmail.conf.5-opts.adoc[]
691
692
e62ceaf0
DM
693ifdef::manvolnum[]
694include::pmg-copyright.adoc[]
695endif::manvolnum[]
696