Thomas Lamprecht [Thu, 14 Jul 2022 14:41:43 +0000 (16:41 +0200)]
docs: let sphinx build man pages for better integration and less cruft
rst2man is only good for use in projects that don't use sphinx
already, as there it can help to avoid bringing in the full sphinx
dependencies and be easier to manage (as long as it stay small).
But we already use sphinx, so there's no point in managing the manual
pages in a separate, semi-related way that is quite restricted as we
have no access to sphinx infrastructure like conf.py defined
variables and helpers for things like the current version.
Besides that, we're rather big, so the complexity of sphinx can
shine, e.g., see the diffstat report from switching out rst2man for
sphinx:
datastore: factor type out of ListGroups into ListGroupsType
In the API we want to iterate over all backup groups
belonging to a particular type at least once, and iterating
through *everything* and simply "skipping" over every single
entry from another type makes no sense given that the groups
are organized into subdirectories based on their type.
Let's have an `.iter_backup_type()` method which returns an
iterator over all the groups of a specific type named
ListGroupsType and factorize the type level iterator out of
ListGroups for reuse.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dominik Csapak [Mon, 20 Jun 2022 07:51:13 +0000 (09:51 +0200)]
restore-daemon: make file listing 'streaming'
this prevents an oom kill when listing large directories.
Without this, i'd get an oom kill in the restore vm when
i tried to list a directory with ~60000 entries, but with this,
i'd get the response for even 250000 entries
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Stefan Sterz [Fri, 10 Jun 2022 08:13:25 +0000 (10:13 +0200)]
config: remove duplicate privilege lookup in cached_user_info
`lookup_privs` just uses `lookup_privs_details` but ignores the
propagated privileges it returns. thus, the lookup here is redundant
as it is immediately followed by a call to `lookup_privs_details` with
the same parameters.
The previous implementation had one issue with not handling API
tokens correctly.
In general, AclTree(Node) operates on the role level, not the priv
level - the latter is handled by cached_user_info.rs
Accordingly, the ACL tree helpers now return a list of paths where *any*
role is defined for the given AuthId, and any_priv_below then maps those
paths to privs via the regular helpers for priv lookup/checking. this
approach should also be robust if groups and group ACLs are ever
introduced.
api: tape restore: code cleanup to reduce indentation level
No semantic change intended. IMO the interface of "both a datastore
and NS mapping must be present" is still a bit weird, at least in how
its used here to decide what to skip and what not, maybe we can
implement this in a more clear way (or maybe I'm just overlooking
something that makes it clearer as is).
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Thu, 2 Jun 2022 12:18:04 +0000 (14:18 +0200)]
replace 'disk_usage' with 'fs_info' from proxmox-sys
Use the moved 'fs_info' helpers from the proxmox-sys crate (available
from there since proxmox-sys 0.3.0) as replacement for 'disk_usage'
in the workspace local tools crate and remove the latter as we do not
need it anymore.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ T: squashed in removal of now unused import and reworded commit
message to include version availability info, among other things ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
api: list datastore: avoid iterating over NS for priv check, use AclTree
Make the assumption that if a user has any privilege that would make
an NS and (parts) of its content visible they also should be able to
know about the datastore and very basic errors on lookup (path
existence and maintenance mode) even if that NS doesn't even exists
(yet), as they could, e.g., make or view a backup and find out
anyway.
This avoids iterating over parts of the whole datastore folder tree
on disk, doing a priv check on each, swapping IO to virtual in memory
checks on info we got available already anyway, is always a good idea
after all
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
config: any_priv_below: plural name & switch to slice of &str for path
s/any_priv_below/any_privs_below/ for consistency and switch from a
single &str for the path param to the slice-ref string variant, as
that allows to use it more often without allocation.
Also allow passing the whole path as single &str element in the slice
by splitting each component on '/' like we do in other parts
nowadays. Note though that we need to omit the leading slash then.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Sterz [Fri, 3 Jun 2022 15:32:24 +0000 (17:32 +0200)]
pbs-config: acl-tree: add any_priv_below
`any_priv_below()` checks if a given AuthId has any given privileges
on a sub-tree of the AclTree. to do so, it first takes into account
propagating privileges on the path itself and then uses a depth-first
search to check if any of the provided privileges are set on any
node of the sub-tree pointed to by the path.
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
datastore: swap ConfigVersionCache with digest for change detection
We got the digest available anyway, and it's only 16 bytes more to
save (compared to last_generation and the recently removed last_time,
both being 64 bit = 8 bytes each)
Side benefit, we detect config changes made manually (e.g., `vim
datacenter.cfg`) immediately.
Note that we could restructure the maintenance mode checking to only
be done after checking if there's a cached datastore, in which case
using the generation could make sense to decide if we need to re-load
it again before blindly loading the config anyway. As that's not only
some (not exactly hard but not really trivial like a typo fix either)
restructuring work but also means we'd lose the "detect manual
changes" again I'd rather keep using the digest.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
to avoid the problematic open fresh datastore with fresh chunkstore
with, and that's the actual problematic part, fresh process locker.
As the latter uses posix record locks which are pretty dangreous as
they operate on a path level (not FD level) and thus closing any file
opened (even if it wasn't opened for locking at all) drops all active
locks on the same file on completely unrelated file descriptors -.-
Also, no operation wasn't exactly correct for this thing in the first
place, but we cannot use Operation::Lookup either, as we're currently
indeed using a rather stupid-simple way and *are* reading.
So until we optimize this to allow querying the AclTree if there's
any priv XYZ below a path, use the Operation::Read.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
projects / proxmox-backup.git / log