Max Carrara [Tue, 21 Nov 2023 10:08:44 +0000 (11:08 +0100)]
rest-server: Refactor `AcceptBuilder`, provide support for optional TLS
The new public function `accept_tls_optional()` is added, which
accepts both plain TCP streams and TCP streams running TLS. Plain TCP
streams are sent along via a separate channel in order to clearly
distinguish between "secure" and "insecure" connections.
Furthermore, instead of `AcceptBuilder` itself holding a reference to
an `SslAcceptor`, its public functions now take the acceptor as an
argument. The public functions' names are changed to distinguish
between their functionality in a more explicit manner:
* `accept()` --> `accept_tls()`
* NEW --> `accept_tls_optional()`
Signed-off-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Lukas Wagner <l.wagner@proxmox.com> Reviewed-by: Lukas Wagner <l.wagner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Stefan Sterz [Thu, 9 Nov 2023 15:33:59 +0000 (16:33 +0100)]
subscription: expose the `next_due_date` as an `i64`
internally `SubscriptionInfo` already uses the `parse_next_due` helper
to parse the next due date to an epoch. this exposes a function that
allows us to use the epoch outside of this crate too. for example, a
user of pom may have multiple subscription for the same system. in
that case we want to apply the one with the due date that is furthest
in the future.
Stefan Sterz [Thu, 9 Nov 2023 15:33:58 +0000 (16:33 +0100)]
type: move `ProductType` type to `proxmox-subscription` from pom
previously this type lived inside of pom. this made it harder to
access the product type from a `SubscriptionInfo` trait in other
products. move the type here so we can check product types more
consistently across products (e. g. in pom and pbs)
Lukas Wagner [Tue, 14 Nov 2023 12:59:23 +0000 (13:59 +0100)]
notify: add built-in config and 'origin' parameter
This allows us to define a (modifiable) builtin-config, which is
at the moment hardcoded in PVEContext
The 'origin' parameter indicates whether a config entry was created by
a user, builtin or a modified builtin.
These changes require context to be set for tests, so we set
PVEContext by default if in a test context. There might be a nicer
solution for that, but for now this should work.
Lukas Wagner [Tue, 14 Nov 2023 12:59:20 +0000 (13:59 +0100)]
notify: add 'smtp' endpoint
This commit adds a new endpoint type, namely 'smtp'. This endpoint
uses the `lettre` crate to directly send emails to SMTP relays.
The `lettre` crate was chosen since it is by far the most popular SMTP
implementation for Rust that looks like it is well maintained.
Also, it includes async support (for when we want to extend
proxmox-notify to be async).
For this new endpoint type, a new section-config type was introduced
(smtp). It has the same fields as the type for `sendmail`, with the
addition of some new options (smtp server, authentication, tls mode,
etc.).
Some of the behavior that is shared between sendmail and smtp
endpoints has been moved to a new `endpoints::common::mail` module.
Lukas Wagner [Tue, 14 Nov 2023 12:59:19 +0000 (13:59 +0100)]
notify: add PVE/PBS context
This commit moves PVEContext from `proxmox-perl-rs` into the
`proxmox-notify` crate, since we now also need to access it from
`promxox-mail-forward`. The context is now hidden behind a feature
flag `pve-context`, ensuring that we only compile it when needed.
This commit adds PBSContext, since we now require it for
`proxmox-mail-forward`. Some of the code for PBSContext comes
from `proxmox-mail-forward`.
This commit also changes the global context from being stored in a
`once_cell` to a regular `Mutex`, since we now need to set/reset
the context in `proxmox-mail-forward`.
Lukas Wagner [Tue, 14 Nov 2023 12:59:18 +0000 (13:59 +0100)]
notify: add mechanisms for email message forwarding
As preparation for the integration of `proxmox-mail-foward` into the
notification system, this commit makes a few changes that allow us to
forward raw email messages (as passed from postfix).
For mail-based notification targets, the email will be forwarded
as-is, including all headers. The only thing that changes is the
message envelope.
For other notification targets, the mail is parsed using the
`mail-parser` crate, which allows us to extract a subject and a body.
As a body we use the plain-text version of the mail. If an email is
HTML-only, the `mail-parser` crate will automatically attempt to
transform the HTML into readable plain text.
Lukas Wagner [Tue, 14 Nov 2023 12:59:16 +0000 (13:59 +0100)]
notify: let a matcher always match if it has no matching directives
This should be a bit more intuitive to users than the current
behavior, which is 'always match' for mode==all and 'never match' for
mode==any. The current behavior originates in the neutral element of
the underlying logical operation (and, or).
Lukas Wagner [Tue, 14 Nov 2023 12:59:13 +0000 (13:59 +0100)]
notify: replace filters and groups with matcher-based system
This shifts notification routing into the matcher-system. Every
notification has associated metadata (key-value fields, severity -
to be extended) that can be match with match directives in
notification matchers. Right now, there are 2 matching directives,
match-field and match-severity. The first one allows one to do a
regex match/exact match on a metadata field, the other one allows one
to match one or more severites.
Every matcher also allows 'target' directives, these decide which
target(s) will be notified if a matcher matches a notification.
Since routing now happens in matchers, the API for sending is
simplified, since we do not need to specify a target any more.
The API routes for filters and groups have been removed completely.
The parser for the configuration file will still accept filter/group
entries, but will delete them once the config is saved again. This is
needed to allow a smooth transition from the old system to the new
system, since the old system was already available on pvetest.
Lukas Wagner [Tue, 14 Nov 2023 12:59:12 +0000 (13:59 +0100)]
notify: factor out notification content into its own type
This will be useful later for system mail forwarding, where
the content of the mail should be forwarded unchanged.
This moves notification properties into this new type and calls them
'data'. They will exclusively used for template rendering.
`Notification` will receive a separate field for metadata, which
will be useful for notification filtering. This decouples
template rendering and filtering, which enables us to be very precise
about which metadata fields we allow to be used in filters.
Thomas Lamprecht [Tue, 24 Oct 2023 11:52:15 +0000 (13:52 +0200)]
sys: depreacate PTY module, moves to termproxy directly
As termproxy only used this module from the huge proxmox-sys crate,
and this module was also only used here, it makes sense to move it
over there (and dropping proxmox-sys halved build-time from 8.5 s to
4.2 s).
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Gabriel Goller [Mon, 23 Oct 2023 07:57:24 +0000 (09:57 +0200)]
schema: beautify parameter error output
If there is only one error, output in a single line, without any
excessive newlines at the end. If there are multiple errors, show them
in a bulleted list.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Thomas Lamprecht [Wed, 18 Oct 2023 17:36:32 +0000 (19:36 +0200)]
rest-server: factor out task-log directory and creation
We had two call sites deriving the directory "shard" where the task
log file is actually saved to, this can lead to ugly bugs and is
better done in a central single-source-of-truth way.
While at it factor out the creation of the log file (and it's shard
directory) to avoid crowding the WorkerTask new fn to much.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
This mostly affected attribute parsing (due to the syn::Meta changes).
Also creating `DelimSpan`s for custom-built `syn::Attribute`s is a
bit... ugly.
Upshot: turns out we can drop some helpers in util.rs with the new
`syn::Meta` changes.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
previously we changed the internal type of the 'data' property
from Option<T> to T in the assumption the api always returns
'data:null'.
this is actually only the case when the api call succeeds. in an error
case there is no data property at all.
to fix this issue while behaving the same for 'data:null' we have to
revert to Option<T> for RawApiResponse but instead of always throwing an
error for 'data:null' in 'check' we now try there to deserialize from
Value::Null for T if there was no data. This will succeed for the Type
'()' which was the motivation for the original change.
The only downside is that the RawApiResponse now has a trait bound that
T is deserializeable, but was a requirement for using it anyway
(as there was no other way of constructing it)
Gabriel Goller [Fri, 8 Sep 2023 13:06:57 +0000 (15:06 +0200)]
fix #4162: sys: added `auto-submitted` header to email body
`Auto-Submitted` is defined in the rfc 5436 [1] and describes how
an automatic response (f.e. ooo replies, etc.) should behave on the
emails. When using `Auto-Submitted: auto-generated` (or any value
other than `none`) automatic replies won't be triggered.
[1]: https://www.rfc-editor.org/rfc/rfc3834.html
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Dominik Csapak [Thu, 24 Aug 2023 10:22:31 +0000 (12:22 +0200)]
rest-server: accept empty body as valid parameters
technically an empty string is not valid json, but when sending an api
request without any parameters, treating the empty body as an empty
parameter hash instead of an error, makes the the api more robust for
clients
This moves the clean-up to happen up-front. That way one can still
inspect the test data after, e.g., a failed test.
Originally done almost like this in a patch from Fiona [0] that I just
overlooked, but now also factored out, avoid crowding the test code to
much with duplicate code.
the "main" repo only exists for Quincy to allow an easier transition
from Proxmox VE 7 to Proxmox VE 8, for when the enterprise repo got
added for ceph too.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
For now just duplicate the Ceph Quincy entries, as I want to avoid
using macros and we do not yet have support for enums inside enums
with the api macro.
Adapt and expand the tests slightly to have at least some simple
coverage there too.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Tue, 29 Aug 2023 12:04:40 +0000 (14:04 +0200)]
client: remove option from inner RawApiResponse
when using the client for an api call that does not return any data
(it returns '{"data":null}'), we would always get an error 'api returned
no data'. The message is technically correct, but it should not be an
error when we expect no data (e.g. most of our CRUD PUT/POST calls)
instead of having the Option<T> in the RawApiResponse type itself, move
it into to the 'nodata' function intended for api calls where we don't
expect any data.
Lukas Wagner [Fri, 25 Aug 2023 11:35:57 +0000 (13:35 +0200)]
notify: make template rendering helpers more robust
This commit has the aim of making template rendering a bit more
robust. It does so by a.) Accepting also strings for helpers that
expect a number, parsing the number if needed, and b.) Ignoring errors
if a template helper fails to render a value and showing an error in
the logs, instead of failing to render the whole template (leading
to no notification being sent).
Max Carrara [Mon, 21 Aug 2023 11:37:45 +0000 (13:37 +0200)]
async: runtime: Modernise module and update docs
This commit updates all helper functions, taking into account recent
developments regarding `tokio`.
In particular, the `block_in_place()` and `block_on()` functions now
don't panic anymore if used within the single-threaded `tokio` runtime
and instead behave as expected in both runtime flavours.
Furthermore, because `tokio` may add more runtime flavours in the
future, all helpers will now panic if used within an unsupported
runtime. This is to prevent unforeseen behavioural quirks and
interactions with `tokio` internals.
The above changes make `BlockingGuard` redundant; it is consequently
removed.
The documentation is also updated, describing the behaviour of the
helper functions and the purpose of the `runtime.rs` module in more
detail.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Since the WASM client cannot actually use a `http::Request` the way we
expect it to, that is, it cannot manually along cookies, we turn the
client bit inside out:
This crate mainly defines the `HttpApiClient` trait which expects the
http client to perform *authenticated* API calls, that is, the
handling of API tokens and tickets should happen at the *implementor*
side.
The product clients will require *this* trait to be implemented, and
will not themselves offer a way to login.
As for the `Client` struct, this will now instead *implement* this
trait and will *not* be used in the `wasm` ecosystem. Rather, this is
the ticket handling http client that already exists in the PWT based
ui code.
The PVE client in `pve-api-types` will not *contain* a `Client`
anymore, but rather, it will provide PVE api call implementations for
something implementing `HttpApiClient`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
The environment trait was useful on the CLI, but does not really
translate well to eg. the wasm ui (or pdm for that matter), so drop it
and instead have `.login` and `.login_tfa` just take the
`proxmox_login` type and handle the updating of authentication data.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Max Carrara [Mon, 17 Jul 2023 14:19:40 +0000 (16:19 +0200)]
proxmox-io: fix `sparse_copy` not copying sparsely on irregular read operations
In the uncommon circumstance that calls to `read()` end up reading any number of
bytes other than 4096, the subsequently read bytes become misaligned, causing
blocks of zeroes to be written unnecessarily.
The first and last block are filled with some data, whereas the middle block is
empty and will therefore result in only zeroes being read.
In order for the empty block to be skipped with `seek()`, the entire buffer has
to be filled with zeroes.
If, for example, the first `read()` ends up putting only 3KiB into the buffer,
the empty block in the middle won't be detected properly, as the buffer will
now always contain some data. What results are four misaligned reads:
This is fixed by ensuring chunks of 4KiB are always read into the buffer,
except when the last block is truncated. In order to prevent frequent small
reads, the incoming reader is also buffered via `io::BufReader`.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Stefan Sterz [Fri, 21 Jul 2023 14:34:03 +0000 (16:34 +0200)]
ldap: only search base of base_dn when checking connection
this should avoid most common size limitations. the search should also
complete quicker as fewer results need to be computed. note that this
way a configuration may be accepted, but the related sync job can
fail due to and exceeded size limit warning for some ldap servers
(such as 2.5.14+dfsg-0ubuntu0.22.04.2).
projects / proxmox.git / log