Dominik Csapak [Mon, 12 Feb 2024 12:27:34 +0000 (13:27 +0100)]
fix #5229: tape: remove max sequence number limit
The idea was to limit the number of tapes in a media set, but this was
not enforced when adding a medium to a media set, only on read/parsing
the inventory. With that, it is possible to create media sets greater
than the limit which in turn blocks access to most functions via
api/cli/gui due to the check.
Instead of enforcing an arbitrary limit, simply warn on creation when
the media-set is very large (20).
To restore the whole media set, the time taken would still be at least 38
hours for LTO-4 and 250 hours for LTO-9.
We already have a section in the docs where we tell about the
disadvantages of large media sets.
Gabriel Goller [Tue, 6 Feb 2024 10:09:07 +0000 (11:09 +0100)]
fix #5190: api: OIDC: accept generic URIs for the ACR value
Allow more complex strings for the acr-value when using openid. The
openid documentation only specifies the acr-value *should* be an URI
[0]. Implemented a regex that loosely disallows some of the reserved
URI characters specified in the RFC [1].
Currently values like:
- "urn:mace:incommon:iap:silver"
- "urn:comsolve.nl:idp:contract:rba:location"
do NOT work, although they are correct URI's and common acr tokens.
For Proxmox VE we had to actually make this more strict to align with
each other, as there we accepted any string.
Since we additonally also support delcaring a "type" property for
`oneOf` schemas (to use with serde's *internally* tagged enum
representation, this contains an additional `typeProperty` and
`typeSchema` value.
It dumps as follows:
{
"type": "object",
"description": ...,
"typeProperty": "name-of-type-property",
"typeSchema": {
"type": "string",
"enum": [ ... ], // technically not enforced by the code
},
"oneOf": [
{
"title": "<value from the above 'enum' array>",
<schema>,
},
{
"title": "<value from the above 'enum' array>",
<schema>,
},
... <one for each 'enum' above>
// ^ exact match is not technically enforced by code
}
}
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
tape: factor getting encryption fingerprint tuple out
makes it a bit more readable as there's less "noise" in the read_label
function and as the separate new fn allows us to nicely use ? to early
return as it has an option in the return signature avoiding 5 lines of
code while not really getting more terse.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Wed, 31 Jan 2024 13:42:33 +0000 (14:42 +0100)]
tape: fix regression in restoring key from medium
Since commit 1343dcaf we automatically try to load the key into the
drive after reading the media-set label, this cannot work for the case
where we actually restore the key from the tape itself.
To address this special case while preserving the automatic key
loading, everything except the setup of the key has been separated
from the 'read_label' method into a new function named
'read_label_without_loading_key'. Consequently, the 'restore-key' API
endpoint can be switched to utilize this new method, thereby avoiding
the issue.
Fixes: 1343dcaf ("tape: move 'set_encryption' calls to the TapeDriver") Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: reword and shorten commit message ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Nightly rustc now warns about unused private fields in the case of a
non-pub newtype struct, so use an underscore-prefixed dummy field name
to get rid of the warning.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dominik Csapak [Mon, 22 Jan 2024 11:50:34 +0000 (12:50 +0100)]
tape: assert encryption mode when using the PoolWriter
by introducing an 'assert_encryption_mode' that checks the desired
state, and bails out if it's different, called directly where we
previously set the encryption mode (which is now done automatically)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: add drive_ prefix and fleece in comment ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Mon, 22 Jan 2024 11:50:33 +0000 (12:50 +0100)]
tape: move 'set_encryption' calls to the TapeDriver (and implementation)
namely everytime we know what the key for the tape has to be:
* after we write the MediaSetLabel
* after reading the MediaSetLabel
When handling data on tape, we always have to have the MediaSetLabel, so
we should always trigger one of these. Because of that, we should not be
able to forget to set the encryption mode.
Dominik Csapak [Mon, 22 Jan 2024 11:50:32 +0000 (12:50 +0100)]
tape: fix wrongly unloading encryption key
For security, we want to automatically unload the encryption key from
the drive when we're done, so there was a Drop handler for SgTape that
handles that. Sadly, our tool we use to set it in the first place, also
invoked the Drop handler, thus unloading the keys again immediately
To fix that, move the Drop handler one logical level higher to the
LtoTapeHandle, which is not used by the 'sg-tape-cmd'.
Dominik Csapak [Mon, 22 Jan 2024 11:50:30 +0000 (12:50 +0100)]
tape: use SgTape in sg-tape-cmd
instead of LtoTapeHandle. This way, we can simply always call the binary
from LtoTapeHandle, and don't have to concern ourselves with the sg_tape
calling.
Thomas Lamprecht [Thu, 18 Jan 2024 14:13:32 +0000 (15:13 +0100)]
report: change output contract of functions
let them manage it completely themselves, as we cannot really say if a
code-block fits for the whole output, like it was the case for the
function that returned a limited output of a 'top' process status
command.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Thu, 11 Jan 2024 10:40:34 +0000 (11:40 +0100)]
api: tape: don't allow duplicate media label-texts
quite a few parts of our code assumes that the label-text is unique in
the inventory, which leads to rather unexpected behaviour when having
more than one tape with the same label-text, e.g. a
`proxmox-tape media destroy <LABEL>`
destroys the first one in the config
(same with moving to vault, etc.)
since having multiple tapes with the same human readable name is always
confusing, simply disallow that here
Dominik Csapak [Thu, 11 Jan 2024 10:40:32 +0000 (11:40 +0100)]
tape: handle duplicate label-texts in inventory
find_media_by_label_text assumes that the label-texts are unique, but
currently this is not necessarily the case. To properly handle that,
change the signature to return a result, and in case there are duplicate
ones, return an error.
Philipp Hufnagl [Tue, 2 Jan 2024 11:06:53 +0000 (12:06 +0100)]
ui: Show if Filter includes or excludes
To make the UI compatible, the Group Filter dialogue has been extended
by a second list, so it now features a list for all include filter and
one for all exclude filters.
Internally, all include as well as exclude filter are managed into one
list. The 2 list view is just for a cleaner representation in the UI.
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
Philipp Hufnagl [Tue, 2 Jan 2024 11:06:52 +0000 (12:06 +0100)]
fix #4315: jobs: modify GroupFilter so include/exclude is tracked
After some discussion I canged the include/exclude behavior to first run
all include filter and after that all exclude filter (rather then
allowing to alternate inbetween). This is done by splitting them into 2
lists, running include first.
A lot of discussion happened how edge cases should be handled and we
came to following conclusion:
no include filter + no exclude filter => include all
some include filter + no exclude filter => filter as always
no include filter + some exclude filter => include all then exclude
Since a GroupFilter now also features an behavior, the Struct has been
renamed To GroupType (since simply type is a keyword). The new
GroupFilter now has a behaviour as a flag 'is_exclude'.
I considered calling it 'is_include' but a reader later then might not
know what the opposite of 'include' is (do not include? deactivate?). I
also considered making a new enum 'behaviour' but since there are only 2
values I considered it over engeneered.
Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
Dominik Csapak [Wed, 13 Dec 2023 09:00:26 +0000 (10:00 +0100)]
tape: work around buggy changer implementations
allocation length for read element status is a 3 byte field, but it
seems some changers only look at the bottom two bytes. Since we used
0x010000 for it, those changers did not return any data and the calls
failed.
To work around it, request one byte less (0xFFFF) which should still be
enough for the data, but should now work with those buggy
implementations.
Reported by a user in the forum: https://forum.proxmox.com/threads/137391/
Dominik Csapak [Thu, 14 Dec 2023 09:05:19 +0000 (10:05 +0100)]
tape: move 'eject-before-unload' to a plain changer config option
instead of having it in a property string. For now this should be fine,
and if we need many more such options, we can still move them into a
property string if we want.
Also update the cli command in the docs on how to set it now.
Dominik Csapak [Wed, 13 Dec 2023 10:11:12 +0000 (11:11 +0100)]
tape: fix 'eject-before-unload' api type
by converting the bool into an option, otherwise having the options not
set at all will fail the unload while deserializing with
'eject-before-unload is not optional'
Also if we can automatically decide this in the future, we can now
detect if the option was explicitely set or not.
Fixes: 66402cdc ("fix #4904: tape changer: add option to eject before unload") Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Dominik Csapak [Thu, 7 Dec 2023 12:51:02 +0000 (13:51 +0100)]
fix #4904: tape changer: add option to eject before unload
some tape libraries need the tape being ejected from the drive before
doing an unload. Since we cannot easily detect if that's the case,
introduce an 'eject_before_unload' option.
Instead of just adding a bool flag to the config, add a new 'options'
property string where we can put such niche options similar to how we
handle the datastore tuning options.
Extend the LtoTapeHandle with 'medium_present' which just uses a
TEST UNIT READY command to check for present medium, so we don't
try to eject an already ejected tape.
Dominik Csapak [Wed, 6 Dec 2023 08:17:54 +0000 (09:17 +0100)]
tape: improve error on decode element status page
instead of wrapping the function body in a 'try_block', simply move the
map_err to the only call site, where we can even add more context than
in the function itself.
aside from better error output, no functional change intended
this could help in debugging cases like this issue reported in the forum:
https://forum.proxmox.com/threads/137391/
Dominik Csapak [Tue, 12 Dec 2023 11:32:47 +0000 (12:32 +0100)]
tape: adapt format_media for LTO9+
starting with LTO9, a FORMAT(04h) command also reinitializates the tape,
which can take up to tw hours. Since we don't actually want to do that
every time we format, use 'erase_media' when we want a fast erase.
(On a slow erase, we let it run and wait until the drive is ready
again).
The users have to pre-initializate the tapes before using it for them to
work properly though.
Dominik Csapak [Tue, 12 Dec 2023 11:32:45 +0000 (12:32 +0100)]
tape: add optional timeout to wait_until_ready
instead of hardcodign the default timeout as only option. This will come
in handy when we need to wait for LTO9+ initialization that can take up
to two hours.
Gabriel Goller [Mon, 11 Dec 2023 08:59:01 +0000 (09:59 +0100)]
ui: datastore summary handle non-existent values
Correctly display missing 'avail' and 'used' attributes in the
datatstore summary. This simply sets it to 0, so that we don't get any
errors in the console.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Folke Gleumes [Tue, 14 Nov 2023 14:14:04 +0000 (15:14 +0100)]
cli: acme: add possibility to set eab via the cli
If the ca demands external account binding credentials, the user will be
asked for them. If a custom directory is used, the user will be asked if
eab should be used.
The ID_PART_ENTRY_* values describe what kind of partition this is and
thus can be used to implement the `.is_partition()` method which we
use in the next patch to avoid calling out to `lsblk`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
previously this would always refer to the "top" namespace of the source,
instead of properly iterating over the namespace tree. adapt the trait
accordingly, since this was the only call site.
Dominik Csapak [Wed, 29 Nov 2023 15:49:50 +0000 (16:49 +0100)]
ui: add 'show connection information' button for datastores
this has a similar functionality as the 'show fingerprint' button,
but for repository strings that are needed e.g. for the cli
included with and without the current user for convenience
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: squash in window title rename and iconCls fix for light-mode ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Gabriel Goller [Wed, 29 Nov 2023 13:29:00 +0000 (14:29 +0100)]
node: status: declutter kernel-version
Return a struct with all the components of the kernel version like it
has been done in pve. Also return the legacy `kversion` to keep
backwards compat.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com> Tested-by: Lukas Wagner <l.wagner@proxmox.com>