]>
Commit | Line | Data |
---|---|---|
1ba1e952 LW |
1 | [[chapter_notifications]] |
2 | Notifications | |
3 | ============= | |
4 | ifndef::manvolnum[] | |
5 | :pve-toplevel: | |
6 | endif::manvolnum[] | |
7 | ||
c877f0aa LW |
8 | Overview |
9 | -------- | |
617fd56e | 10 | [thumbnail="screenshot/gui-datacenter-notification-overview.png"] |
c877f0aa LW |
11 | |
12 | {pve} will send notifications if case of noteworthy events in the system. | |
13 | ||
14 | There are a number of different xref:notification_events[notification events], | |
15 | each with their own set of metadata fields that can be used in | |
16 | notification matchers. | |
17 | ||
18 | A xref:notification_matchers[notification matcher] determines | |
19 | _which_ notifications shall be sent _where_. | |
20 | A matcher has _match rules_, that can be used to | |
21 | match on certain notification properties (e.g. timestamp, severity, | |
22 | metadata fields). | |
23 | If a matcher matches a notification, the notification will be routed | |
24 | to a configurable set of notification targets. | |
25 | ||
26 | A xref:notification_targets[notification target] is an abstraction for a | |
27 | destination where a notification should be sent to - for instance, | |
28 | a Gotify server instance, or a set of email addresses. | |
29 | There are multiple types of notification targets, including | |
30 | `sendmail`, which uses the system's sendmail command to send emails, | |
31 | or `gotify`, which sends a notification to a Gotify instance. | |
32 | ||
33 | The notification system can be configured in the GUI under | |
34 | Datacenter -> Notifications. The configuration is stored in | |
35 | `/etc/pve/notifications.cfg` and `/etc/pve/priv/notifications.cfg` - | |
36 | the latter contains sensitive configuration options such as | |
37 | passwords or authentication tokens for notification targets. | |
1ba1e952 LW |
38 | |
39 | [[notification_targets]] | |
40 | Notification Targets | |
41 | -------------------- | |
42 | ||
f5f316bd | 43 | [[notification_targets_sendmail]] |
1ba1e952 LW |
44 | Sendmail |
45 | ~~~~~~~~ | |
617fd56e LW |
46 | [thumbnail="screenshot/gui-datacenter-notification-sendmail.png"] |
47 | ||
1ba1e952 LW |
48 | The sendmail binary is a program commonly found on Unix-like operating systems |
49 | that handles the sending of email messages. | |
50 | It is a command-line utility that allows users and applications to send emails | |
51 | directly from the command line or from within scripts. | |
52 | ||
53 | The sendmail notification target uses the `sendmail` binary to send emails. | |
54 | ||
55 | ||
56 | NOTE: In standard {pve} installations, the `sendmail` binary is provided by | |
57 | Postfix. For this type of target to work correctly, it might be necessary to | |
58 | change Postfix's configuration so that it can correctly deliver emails. | |
59 | For cluster setups it is necessary to have a working Postfix configuration on | |
60 | every single cluster node. | |
61 | ||
62 | The configuration for Sendmail target plugins has the following options: | |
63 | ||
64 | * `mailto`: E-Mail address to which the notification shall be sent to. Can be | |
65 | set multiple times to accomodate multiple recipients. | |
66 | * `mailto-user`: Users to which emails shall be sent to. The user's email | |
67 | address will be looked up in `users.cfg`. Can be set multiple times to | |
68 | accomodate multiple recipients. | |
69 | * `author`: Sets the author of the E-Mail. Defaults to `Proxmox VE`. | |
70 | * `from-address`: Sets the from address of the E-Mail. If the parameter is not | |
71 | set, the plugin will fall back to the `email_from` setting from | |
72 | `datacenter.cfg`. If that is also not set, the plugin will default to | |
73 | `root@$hostname`, where `$hostname` is the hostname of the node. | |
e39dfad9 | 74 | * `comment`: Comment for this target |
7a20522e | 75 | The `From` header in the email will be set to `$author <$from-address>`. |
1ba1e952 | 76 | |
c877f0aa LW |
77 | Example configuration (`/etc/pve/notifications.cfg`): |
78 | ---- | |
79 | sendmail: example | |
80 | mailto-user root@pam | |
81 | mailto-user admin@pve | |
82 | mailto max@example.com | |
83 | from-address pve1@example.com | |
84 | comment Send to multiple users/addresses | |
85 | ---- | |
1ba1e952 | 86 | |
f5f316bd | 87 | [[notification_targets_smtp]] |
7a20522e LW |
88 | SMTP |
89 | ~~~~ | |
617fd56e | 90 | [thumbnail="screenshot/gui-datacenter-notification-smtp.png"] |
7a20522e LW |
91 | |
92 | SMTP notification targets can send emails directly to an SMTP mail relay. | |
93 | ||
94 | The configuration for SMTP target plugins has the following options: | |
95 | ||
96 | * `mailto`: E-Mail address to which the notification shall be sent to. Can be | |
97 | set multiple times to accomodate multiple recipients. | |
98 | * `mailto-user`: Users to which emails shall be sent to. The user's email | |
99 | address will be looked up in `users.cfg`. Can be set multiple times to | |
100 | accomodate multiple recipients. | |
101 | * `author`: Sets the author of the E-Mail. Defaults to `Proxmox VE`. | |
102 | * `from-address`: Sets the From-addresss of the email. SMTP relays might require | |
103 | that this address is owned by the user in order to avoid spoofing. | |
104 | The `From` header in the email will be set to `$author <$from-address>`. | |
105 | * `username`: Username to use during authentication. If no username is set, | |
106 | no authentication will be performed. The PLAIN and LOGIN authentication methods | |
107 | are supported. | |
108 | * `password`: Password to use when authenticating. | |
109 | * `mode`: Sets the encryption mode (`insecure`, `starttls` or `tls`). Defaults | |
110 | to `tls`. | |
111 | * `server`: Address/IP of the SMTP relay | |
112 | * `port`: The port to connect to. If not set, the used port | |
113 | defaults to 25 (`insecure`), 465 (`tls`) or 587 (`starttls`), depending on the | |
114 | value of `mode`. | |
115 | * `comment`: Comment for this target | |
116 | ||
117 | Example configuration (`/etc/pve/notifications.cfg`): | |
118 | ---- | |
119 | smtp: example | |
120 | mailto-user root@pam | |
121 | mailto-user admin@pve | |
122 | mailto max@example.com | |
123 | from-address pve1@example.com | |
124 | username pve1 | |
125 | server mail.example.com | |
126 | mode starttls | |
127 | ---- | |
128 | The matching entry in `/etc/pve/priv/notifications.cfg`, containing the | |
129 | secret token: | |
130 | ---- | |
131 | smtp: example | |
132 | password somepassword | |
133 | ---- | |
134 | ||
f5f316bd | 135 | [[notification_targets_gotify]] |
1ba1e952 LW |
136 | Gotify |
137 | ~~~~~~ | |
617fd56e | 138 | [thumbnail="screenshot/gui-datacenter-notification-gotify.png"] |
1ba1e952 LW |
139 | |
140 | http://gotify.net[Gotify] is an open-source self-hosted notification server that | |
141 | allows you to send and receive push notifications to various devices and | |
142 | applications. It provides a simple API and web interface, making it easy to | |
143 | integrate with different platforms and services. | |
144 | ||
145 | The configuration for Gotify target plugins has the following options: | |
146 | ||
147 | * `server`: The base URL of the Gotify server, e.g. `http://<ip>:8888` | |
148 | * `token`: The authentication token. Tokens can be generated within the Gotify | |
149 | web interface. | |
e39dfad9 | 150 | * `comment`: Comment for this target |
1ba1e952 LW |
151 | |
152 | NOTE: The Gotify target plugin will respect the HTTP proxy settings from the | |
153 | xref:datacenter_configuration_file[datacenter configuration] | |
154 | ||
c877f0aa LW |
155 | Example configuration (`/etc/pve/notifications.cfg`): |
156 | ---- | |
157 | gotify: example | |
158 | server http://gotify.example.com:8888 | |
159 | comment Send to multiple users/addresses | |
160 | ---- | |
1ba1e952 | 161 | |
c877f0aa LW |
162 | The matching entry in `/etc/pve/priv/notifications.cfg`, containing the |
163 | secret token: | |
164 | ---- | |
165 | gotify: example | |
166 | token somesecrettoken | |
167 | ---- | |
1ba1e952 | 168 | |
c877f0aa LW |
169 | [[notification_matchers]] |
170 | Notification Matchers | |
171 | --------------------- | |
617fd56e LW |
172 | [thumbnail="screenshot/gui-datacenter-notification-matcher.png"] |
173 | ||
c877f0aa | 174 | Notification matchers route notifications to notification targets based |
4a6bd57a CE |
175 | on their matching rules. These rules can match certain properties of a |
176 | notification, such as the timestamp (`match-calendar`), the severity of | |
177 | the notification (`match-severity`) or metadata fields (`match-field`). | |
178 | If a notification is matched by a matcher, all targets configured for the | |
179 | matcher will receive the notification. | |
c877f0aa LW |
180 | |
181 | An arbitrary number of matchers can be created, each with with their own | |
182 | matching rules and targets to notify. | |
183 | Every target is notified at most once for every notification, even if | |
184 | the target is used in multiple matchers. | |
185 | ||
186 | A matcher without any matching rules is always true; the configured targets | |
187 | will always be notified. | |
188 | ---- | |
189 | matcher: always-matches | |
190 | target admin | |
191 | comment This matcher always matches | |
192 | ---- | |
1ba1e952 | 193 | |
c877f0aa LW |
194 | Matcher Options |
195 | ~~~~~~~~~~~~~~~ | |
1ba1e952 | 196 | |
c877f0aa LW |
197 | * `target`: Determine which target should be notified if the matcher matches. |
198 | can be used multiple times to notify multiple targets. | |
199 | * `invert-match`: Inverts the result of the whole matcher | |
200 | * `mode`: Determines how the individual match rules are evaluated to compute | |
201 | the result for the whole matcher. If set to `all`, all matching rules must | |
202 | match. If set to `any`, at least one rule must match. | |
203 | a matcher must be true. Defaults to `all`. | |
204 | * `match-calendar`: Match the notification's timestamp against a schedule | |
205 | * `match-field`: Match the notification's metadata fields | |
206 | * `match-severity`: Match the notification's severity | |
e39dfad9 | 207 | * `comment`: Comment for this matcher |
1ba1e952 | 208 | |
f5f316bd | 209 | [[notification_matchers_calendar]] |
c877f0aa LW |
210 | Calendar Matching Rules |
211 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
212 | A calendar matcher matches the time when a notification is sent agaist a | |
213 | configurable schedule. | |
1ba1e952 | 214 | |
c877f0aa LW |
215 | * `match-calendar 8-12` |
216 | * `match-calendar 8:00-15:30` | |
217 | * `match-calendar mon-fri 9:00-17:00` | |
218 | * `match-calendar sun,tue-wed,fri 9-17` | |
1ba1e952 | 219 | |
f5f316bd | 220 | [[notification_matchers_field]] |
c877f0aa LW |
221 | Field Matching Rules |
222 | ~~~~~~~~~~~~~~~~~~~~ | |
223 | Notifications have a selection of metadata fields that can be matched. | |
1ba1e952 | 224 | |
c877f0aa LW |
225 | * `match-field exact:type=vzdump` Only match notifications about backups. |
226 | * `match-field regex:hostname=^.+\.example\.com$` Match the hostname of | |
227 | the node. | |
1ba1e952 | 228 | |
c877f0aa LW |
229 | If a matched metadata field does not exist, the notification will not be |
230 | matched. | |
231 | For instance, a `match-field regex:hostname=.*` directive will only match | |
232 | notifications that have an arbitraty `hostname` metadata field, but will | |
233 | not match if the field does not exist. | |
1ba1e952 | 234 | |
f5f316bd | 235 | [[notification_matchers_severity]] |
c877f0aa LW |
236 | Severity Matching Rules |
237 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
238 | A notification has a associated severity that can be matched. | |
1ba1e952 | 239 | |
c877f0aa LW |
240 | * `match-severity error`: Only match errors |
241 | * `match-severity warning,error`: Match warnings and error | |
1ba1e952 | 242 | |
c877f0aa | 243 | The following severities are in use: |
24451e5d | 244 | `info`, `notice`, `warning`, `error`, `unknown`. |
1ba1e952 | 245 | |
1ba1e952 | 246 | |
c877f0aa LW |
247 | Examples |
248 | ~~~~~~~~ | |
249 | ---- | |
250 | matcher: workday | |
251 | match-calendar mon-fri 9-17 | |
252 | target admin | |
253 | comment Notify admins during working hours | |
254 | ||
255 | matcher: night-and-weekend | |
256 | match-calendar mon-fri 9-17 | |
257 | invert-match true | |
258 | target on-call-admins | |
259 | comment Separate target for non-working hours | |
260 | ---- | |
1ba1e952 | 261 | |
c877f0aa LW |
262 | ---- |
263 | matcher: backup-failures | |
264 | match-field exact:type=vzdump | |
265 | match-severity error | |
266 | target backup-admins | |
267 | comment Send notifications about backup failures to one group of admins | |
268 | ||
269 | matcher: cluster-failures | |
270 | match-field exact:type=replication | |
271 | match-field exact:type=fencing | |
272 | mode any | |
273 | target cluster-admins | |
274 | comment Send cluster-related notifications to other group of admins | |
275 | ---- | |
1ba1e952 | 276 | |
c877f0aa LW |
277 | The last matcher could also be rewritten using a field matcher with a regular |
278 | expression: | |
279 | ---- | |
280 | matcher: cluster-failures | |
281 | match-field regex:type=^(replication|fencing)$ | |
282 | target cluster-admins | |
283 | comment Send cluster-related notifications to other group of admins | |
284 | ---- | |
1ba1e952 | 285 | |
c877f0aa LW |
286 | [[notification_events]] |
287 | Notification Events | |
288 | ------------------- | |
1ba1e952 | 289 | |
c877f0aa LW |
290 | [width="100%",options="header"] |
291 | |=========================================================================== | |
292 | | Event | `type` | Severity | Metadata fields (in addition to `type`) | |
293 | | System updates available |`package-updates` | `info` | `hostname` | |
294 | | Cluster node fenced |`fencing` | `error` | `hostname` | |
295 | | Storage replication failed |`replication` | `error` | - | |
296 | | Backup finished |`vzdump` | `info` (`error` on failure) | `hostname` | |
7b293b5d | 297 | | Mail for root |`system-mail` | `unknown`| - |
c877f0aa | 298 | |=========================================================================== |
1ba1e952 | 299 | |
c877f0aa LW |
300 | [width="100%",options="header"] |
301 | |======================================================================= | |
302 | | Field name | Description | |
303 | | `type` | Type of the notifcation | |
304 | | `hostname` | Hostname, including domain (e.g. `pve1.example.com`) | |
305 | |======================================================================= | |
1ba1e952 | 306 | |
7b293b5d LW |
307 | System Mail Forwarding |
308 | --------------------- | |
309 | ||
310 | Certain local system daemons, such as `smartd`, generate notification emails | |
311 | that are initially directed to the local `root` user. {pve} will | |
312 | feed these mails into the notification system as a notification of | |
313 | type `system-mail` and with severity `unknown`. | |
314 | ||
315 | When the forwarding process involves an email-based target | |
316 | (like `sendmail` or `smtp`), the email is forwarded exactly as received, with all | |
317 | original mail headers remaining intact. For all other targets, | |
318 | the system tries to extract both a subject line and the main text body | |
319 | from the email content. In instances where emails solely consist of HTML | |
320 | content, they will be transformed into plain text format during this process. | |
321 | ||
c877f0aa LW |
322 | Permissions |
323 | ----------- | |
324 | ||
b4c816f4 LW |
325 | In order to modify/view the configuration for notification targets, |
326 | the `Mapping.Modify/Mapping.Audit` permissions are required for the | |
327 | `/mapping/notifications` ACL node. | |
328 | ||
329 | Testing a target requires `Mapping.Use`, `Mapping.Audit` or `Mapping.Modify` | |
330 | permissions on `/mapping/notifications` |