1 [[chapter_notifications]]
10 [thumbnail="screenshot/gui-datacenter-notification-overview.png"]
12 {pve} will send notifications if case of noteworthy events in the system.
14 There are a number of different xref:notification_events[notification events],
15 each with their own set of metadata fields that can be used in
16 notification matchers.
18 A xref:notification_matchers[notification matcher] determines
19 _which_ notifications shall be sent _where_.
20 A matcher has _match rules_, that can be used to
21 match on certain notification properties (e.g. timestamp, severity,
23 If a matcher matches a notification, the notification will be routed
24 to a configurable set of notification targets.
26 A xref:notification_targets[notification target] is an abstraction for a
27 destination where a notification should be sent to - for instance,
28 a Gotify server instance, or a set of email addresses.
29 There are multiple types of notification targets, including
30 `sendmail`, which uses the system's sendmail command to send emails,
31 or `gotify`, which sends a notification to a Gotify instance.
33 The notification system can be configured in the GUI under
34 Datacenter -> Notifications. The configuration is stored in
35 `/etc/pve/notifications.cfg` and `/etc/pve/priv/notifications.cfg` -
36 the latter contains sensitive configuration options such as
37 passwords or authentication tokens for notification targets.
39 [[notification_targets]]
43 [[notification_targets_sendmail]]
46 [thumbnail="screenshot/gui-datacenter-notification-sendmail.png"]
48 The sendmail binary is a program commonly found on Unix-like operating systems
49 that handles the sending of email messages.
50 It is a command-line utility that allows users and applications to send emails
51 directly from the command line or from within scripts.
53 The sendmail notification target uses the `sendmail` binary to send emails.
56 NOTE: In standard {pve} installations, the `sendmail` binary is provided by
57 Postfix. For this type of target to work correctly, it might be necessary to
58 change Postfix's configuration so that it can correctly deliver emails.
59 For cluster setups it is necessary to have a working Postfix configuration on
60 every single cluster node.
62 The configuration for Sendmail target plugins has the following options:
64 * `mailto`: E-Mail address to which the notification shall be sent to. Can be
65 set multiple times to accomodate multiple recipients.
66 * `mailto-user`: Users to which emails shall be sent to. The user's email
67 address will be looked up in `users.cfg`. Can be set multiple times to
68 accomodate multiple recipients.
69 * `author`: Sets the author of the E-Mail. Defaults to `Proxmox VE`.
70 * `from-address`: Sets the from address of the E-Mail. If the parameter is not
71 set, the plugin will fall back to the `email_from` setting from
72 `datacenter.cfg`. If that is also not set, the plugin will default to
73 `root@$hostname`, where `$hostname` is the hostname of the node.
74 * `comment`: Comment for this target
75 The `From` header in the email will be set to `$author <$from-address>`.
77 Example configuration (`/etc/pve/notifications.cfg`):
82 mailto max@example.com
83 from-address pve1@example.com
84 comment Send to multiple users/addresses
87 [[notification_targets_smtp]]
90 [thumbnail="screenshot/gui-datacenter-notification-smtp.png"]
92 SMTP notification targets can send emails directly to an SMTP mail relay.
94 The configuration for SMTP target plugins has the following options:
96 * `mailto`: E-Mail address to which the notification shall be sent to. Can be
97 set multiple times to accomodate multiple recipients.
98 * `mailto-user`: Users to which emails shall be sent to. The user's email
99 address will be looked up in `users.cfg`. Can be set multiple times to
100 accomodate multiple recipients.
101 * `author`: Sets the author of the E-Mail. Defaults to `Proxmox VE`.
102 * `from-address`: Sets the From-addresss of the email. SMTP relays might require
103 that this address is owned by the user in order to avoid spoofing.
104 The `From` header in the email will be set to `$author <$from-address>`.
105 * `username`: Username to use during authentication. If no username is set,
106 no authentication will be performed. The PLAIN and LOGIN authentication methods
108 * `password`: Password to use when authenticating.
109 * `mode`: Sets the encryption mode (`insecure`, `starttls` or `tls`). Defaults
111 * `server`: Address/IP of the SMTP relay
112 * `port`: The port to connect to. If not set, the used port
113 defaults to 25 (`insecure`), 465 (`tls`) or 587 (`starttls`), depending on the
115 * `comment`: Comment for this target
117 Example configuration (`/etc/pve/notifications.cfg`):
121 mailto-user admin@pve
122 mailto max@example.com
123 from-address pve1@example.com
125 server mail.example.com
128 The matching entry in `/etc/pve/priv/notifications.cfg`, containing the
132 password somepassword
135 [[notification_targets_gotify]]
138 [thumbnail="screenshot/gui-datacenter-notification-gotify.png"]
140 http://gotify.net[Gotify] is an open-source self-hosted notification server that
141 allows you to send and receive push notifications to various devices and
142 applications. It provides a simple API and web interface, making it easy to
143 integrate with different platforms and services.
145 The configuration for Gotify target plugins has the following options:
147 * `server`: The base URL of the Gotify server, e.g. `http://<ip>:8888`
148 * `token`: The authentication token. Tokens can be generated within the Gotify
150 * `comment`: Comment for this target
152 NOTE: The Gotify target plugin will respect the HTTP proxy settings from the
153 xref:datacenter_configuration_file[datacenter configuration]
155 Example configuration (`/etc/pve/notifications.cfg`):
158 server http://gotify.example.com:8888
159 comment Send to multiple users/addresses
162 The matching entry in `/etc/pve/priv/notifications.cfg`, containing the
166 token somesecrettoken
169 [[notification_matchers]]
170 Notification Matchers
171 ---------------------
172 [thumbnail="screenshot/gui-datacenter-notification-matcher.png"]
174 Notification matchers route notifications to notification targets based
175 on their matching rules. These rules can match certain properties of a
176 notification, such as the timestamp (`match-calendar`), the severity of
177 the notification (`match-severity`) or metadata fields (`match-field`).
178 If a notification is matched by a matcher, all targets configured for the
179 matcher will receive the notification.
181 An arbitrary number of matchers can be created, each with with their own
182 matching rules and targets to notify.
183 Every target is notified at most once for every notification, even if
184 the target is used in multiple matchers.
186 A matcher without any matching rules is always true; the configured targets
187 will always be notified.
189 matcher: always-matches
191 comment This matcher always matches
197 * `target`: Determine which target should be notified if the matcher matches.
198 can be used multiple times to notify multiple targets.
199 * `invert-match`: Inverts the result of the whole matcher
200 * `mode`: Determines how the individual match rules are evaluated to compute
201 the result for the whole matcher. If set to `all`, all matching rules must
202 match. If set to `any`, at least one rule must match.
203 a matcher must be true. Defaults to `all`.
204 * `match-calendar`: Match the notification's timestamp against a schedule
205 * `match-field`: Match the notification's metadata fields
206 * `match-severity`: Match the notification's severity
207 * `comment`: Comment for this matcher
209 [[notification_matchers_calendar]]
210 Calendar Matching Rules
211 ~~~~~~~~~~~~~~~~~~~~~~~
212 A calendar matcher matches the time when a notification is sent agaist a
213 configurable schedule.
215 * `match-calendar 8-12`
216 * `match-calendar 8:00-15:30`
217 * `match-calendar mon-fri 9:00-17:00`
218 * `match-calendar sun,tue-wed,fri 9-17`
220 [[notification_matchers_field]]
223 Notifications have a selection of metadata fields that can be matched.
225 * `match-field exact:type=vzdump` Only match notifications about backups.
226 * `match-field regex:hostname=^.+\.example\.com$` Match the hostname of
229 If a matched metadata field does not exist, the notification will not be
231 For instance, a `match-field regex:hostname=.*` directive will only match
232 notifications that have an arbitraty `hostname` metadata field, but will
233 not match if the field does not exist.
235 [[notification_matchers_severity]]
236 Severity Matching Rules
237 ~~~~~~~~~~~~~~~~~~~~~~~
238 A notification has a associated severity that can be matched.
240 * `match-severity error`: Only match errors
241 * `match-severity warning,error`: Match warnings and error
243 The following severities are in use:
244 `info`, `notice`, `warning`, `error`, `unknown`.
251 match-calendar mon-fri 9-17
253 comment Notify admins during working hours
255 matcher: night-and-weekend
256 match-calendar mon-fri 9-17
258 target on-call-admins
259 comment Separate target for non-working hours
263 matcher: backup-failures
264 match-field exact:type=vzdump
267 comment Send notifications about backup failures to one group of admins
269 matcher: cluster-failures
270 match-field exact:type=replication
271 match-field exact:type=fencing
273 target cluster-admins
274 comment Send cluster-related notifications to other group of admins
277 The last matcher could also be rewritten using a field matcher with a regular
280 matcher: cluster-failures
281 match-field regex:type=^(replication|fencing)$
282 target cluster-admins
283 comment Send cluster-related notifications to other group of admins
286 [[notification_events]]
290 [width="100%",options="header"]
291 |===========================================================================
292 | Event | `type` | Severity | Metadata fields (in addition to `type`)
293 | System updates available |`package-updates` | `info` | `hostname`
294 | Cluster node fenced |`fencing` | `error` | `hostname`
295 | Storage replication failed |`replication` | `error` | -
296 | Backup finished |`vzdump` | `info` (`error` on failure) | `hostname`
297 | Mail for root |`system-mail` | `unknown`| -
298 |===========================================================================
300 [width="100%",options="header"]
301 |=======================================================================
302 | Field name | Description
303 | `type` | Type of the notifcation
304 | `hostname` | Hostname, including domain (e.g. `pve1.example.com`)
305 |=======================================================================
307 System Mail Forwarding
308 ---------------------
310 Certain local system daemons, such as `smartd`, generate notification emails
311 that are initially directed to the local `root` user. {pve} will
312 feed these mails into the notification system as a notification of
313 type `system-mail` and with severity `unknown`.
315 When the forwarding process involves an email-based target
316 (like `sendmail` or `smtp`), the email is forwarded exactly as received, with all
317 original mail headers remaining intact. For all other targets,
318 the system tries to extract both a subject line and the main text body
319 from the email content. In instances where emails solely consist of HTML
320 content, they will be transformed into plain text format during this process.
325 In order to modify/view the configuration for notification targets,
326 the `Mapping.Modify/Mapping.Audit` permissions are required for the
327 `/mapping/notifications` ACL node.
329 Testing a target requires `Mapping.Use`, `Mapping.Audit` or `Mapping.Modify`
330 permissions on `/mapping/notifications`