Thomas Lamprecht [Sun, 15 May 2022 14:47:42 +0000 (16:47 +0200)]
ui: datastore: use safe destroy as base for dialog
only ask the name of the current NS, not the full NS path to avoid
too long input requirements on deep levels.
needs a few smaller hacks, ideally we would pull out the basic stuff
from Edit window in some EditBase window and let both, SafeDestroy
and Edit window derive from that, for better common, in sync
features.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Sterz [Thu, 5 May 2022 13:52:51 +0000 (15:52 +0200)]
fix #4001: datastore/catalog: add number of files to directory entry
When listing the content of a catalog, add the number of files
contained in the directory as its size. Also removes redundant code,
the `mtime` and the `size` of a file is already set when creating the
archive entry, but we naturally need to override the size now for
directories.
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sat, 14 May 2022 16:29:09 +0000 (18:29 +0200)]
ui: remote target ns selector: fix clearing value on edit
never makes sense to clear the value due to remote or remoteStore
change as we weren't enabled then in the first place.
This fixes clearing the currently set namespace on editing an
existing job, which always made it seem like the Root namespace was
selected, even if the originalValue was correct (thus the dirty-form
reset/ok behaviour still worked, making it even more confusing)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sat, 14 May 2022 12:48:31 +0000 (14:48 +0200)]
ui: namespace selector: set queryMode to local
to avoid that the comobox triggers automatic API request with the
queryParam default `query` GET param on manual typing (e.g., for
filtering) from the user, we have all data already loaded and locally
available.
Thomas Lamprecht [Sat, 14 May 2022 12:38:58 +0000 (14:38 +0200)]
ui: trigger datastore update after maintenance mode edit
This provides immediate feedback for adding the respective icon in
the navigation tree entry most of the time, and we can then increase
the query period of the datastore list store to the original 15
again, as it was lowered to 5 seconds for just this reason in commit fbd6f54f39b243e5
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Sat, 14 May 2022 10:28:01 +0000 (12:28 +0200)]
ui: fix storeId casing to register store correctly
we query that store to add the datastore specific ACL paths to
improve UX there, this failed a while due the StoreManager lookup
always failing as the store wasn't registered in the StoreManager due
to using storeid vs. correct storeId
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Fri, 13 May 2022 13:39:37 +0000 (15:39 +0200)]
docs: tape: remove note about global content namespace
that is actually not true, we save the datastore in the chunk archives
as well as the snapshot archives, otherwise we could not backup
multiple datastores to a single media-set.
Dominik Csapak [Fri, 13 May 2022 11:44:30 +0000 (13:44 +0200)]
ui: tape/Restore: allow simple namespace mapping
add a default namespace selector (of the current default store)
and a namespace selector per target datastore (for media-sets with
multiple datastores).
to achieve that we have to change the way we handle the mapping field a bit:
* don't use it as field directly (otherwise the value gets stringified),
but use the 'getValue' method in 'onGetValues'.
* set the defaultStore there, not only that we have one
(with this we can now easily show it as emptytext for each store)
* add a reference to the widgets to the record so that we can access
them in the respective change handler (also clean those references up,
else we have a cyclic reference between record <-> widget)
in onGetValues, if we have multiple datastores, the mapping grid does
all the work for us, otherwise, we have to create the ns mapping
ourselves there.
the snapshot string format is not backwards compatible since it now has
an in-line namespace prefix. it's possible to select which magic to use
at the start of the backup, since a tape backup job knows whether it
operates on non-root namespaces up-front.
the MediaCatalog itself also has a similar incompatible change, but
there
- updating existing catalogs in-place
- not knowing what the catalog will contain in the future when initially
creating/opening it
makes bumping the magic there harder. since the tape contents are
sufficiently guarded by the other two bumps, ignoring the
backwards-incomaptible change of the on-disk catalogs seems like an okay
tradeoff.
Dominik Csapak [Thu, 5 May 2022 13:59:36 +0000 (15:59 +0200)]
api: tape/restore: add namespace mapping
by adding a new parameter 'namespaces', which contains a mapping
for a namespace like this:
store=datastore,source=foo,target=bar,max-depth=2
if source or target are omitted the root namespace is used for its value
this mapping can be given several times (on the cli) or as an array (via
api) to have mappings for multiple datastores
if a specific snapshot list is given simultaneously, the given snapshots
will be restored according to this mapping, or to the source namespace
if no mapping was found.
to do this, we reutilize the restore_list_worker, but change it so that
it does not hold a lock for the duration of the restore, but fails
if the snapshot does exist at the end. also the snapshot will now
be temporarily restored into the target datastore into the
'.tmp/<media-set-uuid>' folder.
Dominik Csapak [Tue, 10 May 2022 14:09:42 +0000 (16:09 +0200)]
tape: fix snapshot path in catalog and snapshot_archive
both used the 'Display' trait of pbs_datastore::BackupDir, which is not
intended to be serialized anywhere. Instead, manually format the path
using the print_ns_and_snapshot helper, and conversely, parse with
'parse_ns_and_snapshot'. to be a bit safer, change the register_snapshot
signature to take a BackupNamespace and BackupDir instead of a string.
Dominik Csapak [Tue, 10 May 2022 14:06:41 +0000 (16:06 +0200)]
pbs-api-types: add parse and print ns_and_snapshot
these are helpers for the few cases where we want to print and parse
from a format that has the namespace and snapshot combined, like for
the on-tape catalog and snapshot archive.
Dominik Csapak [Tue, 3 May 2022 06:07:08 +0000 (08:07 +0200)]
tape: notify when arriving at end of media
when continuing a media set, we first move to the end of the tape and
start with the next (chunk) archive. If that takes long, the task logs
last line is 'moving to end of media' even if we already startet
writing. To make this less confusing, log that we arrived at the
end of the media.
to handle the unlikely case of `ns` being deeper than `remote-ns`,
`max-depth` being set to `None` and a too-deep sub-ns of `ns` existing.
such a sub-ns cannot have been created by a previous run of this sync
job, so avoid unexpectedly removing it.
into the regular one (with default == MAX) and the one used for
pull/sync, where the default is 'None' which actually means the remote
end reduces the scope of sync automatically (or, if needed,
backwards-compat mode without any remote namespaces at all).
pull/sync: detect remote lack of namespace support
and fall back to only syncing the root namespace, if possible. the sync
job will still be marked as failed to prompt the admin to resolve the
situation:
- explicitly mark the job as syncing *only* the root namespace
- or upgrade remote end to support namespaces
Thomas Lamprecht [Thu, 12 May 2022 13:40:55 +0000 (15:40 +0200)]
datastore: drop bogus chunk size check, can cause trouble
other sizes can happen in legitimate and illegitimate ways:
- illegitimate: encryped chunks and bad actor client
- legitimate: same chunk but newer zstd version (or compression
level) can compress it better (or worse) so the
Ideally we could take the actual smaller chunk so that improved zstd
tech gets leveraged, but we could only allow to do that for
un-encrypted ones.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Thu, 12 May 2022 12:23:52 +0000 (14:23 +0200)]
ui: navigation tree: fix losing datastore selection on store load
instead of using 'replaceChild', simply set the appropriate
properties. When using the 'nodeUpdate' (protected function of extjs,
intended to be overwritten) instead of the private 'updateNode', it
will be called when the properties change
This way, the treenode stays the same and it can keep the selection
Thomas Lamprecht [Thu, 12 May 2022 10:55:37 +0000 (12:55 +0200)]
ui: content: fix various tree-checks from action handlers
they all still used some odd side effects of the tree structure to
decided what record type they operated on, just move them over to the
new `ty` record.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Fabian Ebner [Wed, 4 May 2022 11:33:24 +0000 (13:33 +0200)]
rest server: daemon: update PID file before sending MAINPID notification
There is a race upon reload, where it can happen that:
1. systemd forks off /bin/kill -HUP $MAINPID
2. Current instance forks off new one and notifies systemd with the
new MAINPID.
3. systemd sets new MAINPID.
4. systemd receives SIGCHLD for the kill process (which is the current
control process for the service) and reads the PID of the old
instance from the PID file, resetting MAINPID to the PID of the old
instance.
5. Old instance exits.
6. systemd receives SIGCHLD for the old instance, reads the PID of the
old instance from the PID file once more. systemd sees that the
MAINPID matches the child PID and considers the service exited.
7. systemd receivese notification from the new PID and is confused.
The service won't get active, because the notification wasn't
handled.
To fix it, update the PID file before sending the MAINPID
notification, similar to what a comment in systemd's
src/core/service.c suggests:
> /* Forking services may occasionally move to a new PID.
> * As long as they update the PID file before exiting the old
> * PID, they're fine. */
but for our Type=notify "before sending the notification" rather than
"before exiting", because otherwise, the mix-up in 4. could still
happen (although it might not actually be problematic without the
mix-up in 6., it still seems better to avoid).
Thomas Lamprecht [Thu, 12 May 2022 09:34:48 +0000 (11:34 +0200)]
GC scheduling: avoid triggering operation tracking error for upfront checks
without that one gets a "failed to lookup datastore X" in the log for
every datastore that is in read-only or offline maintenance mode,
even if they aren't scheduled for GC anyway.
Avoid that by first opening the datastore through a Lookup operation,
and only re-open it as Write op once we know that GC needs to get
scheduled for it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Thu, 12 May 2022 09:31:07 +0000 (11:31 +0200)]
datastore: add new Lookup for operations tracking
We sometimes need to do some in-memory only stuff, e.g., to check if
GC is already running for a datastore, which is a try_lock on a mutex
that is in-memory.
Actually the whole thing would be nicer if we could guarantee to hold
the correct contract statically, e.g., like
https://docs.rust-embedded.org/book/static-guarantees/design-contracts.html
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
projects / proxmox-backup.git / log