Thomas Lamprecht [Mon, 14 Nov 2022 15:13:29 +0000 (16:13 +0100)]
file-restore: split out internal QMP logic from unrelated fn
Capsule it in a small QMPSock struct impl, make the usage nicer as
the caller should not have to care & keep track of the initial socket
state+details.
A send_raw and send Value method should cover most needs.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Mon, 14 Nov 2022 14:59:49 +0000 (15:59 +0100)]
file restore: limit max dimm size to 512 MB
this is on top of the normal memory, and over 1.3 GB required is just
huge, sadly the commit adding this has zero details about what setups
fail and what work again with the change, so hard to tell, but any
setup that needs that much sounds like a bug in ZFS or remaining code
here.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Mon, 14 Nov 2022 14:41:07 +0000 (15:41 +0100)]
file restore: move allow-memory-hotplug param from CLI to environment
avoid the need to loop a parameter through a dozen function which all
don't care about it at all; iff this should be a global oncecell or
lock guarded param.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
which registers a binary in /root/.forward and handles mail forwarding
to the mail addresss configured for root@pam in PBS. Similar to how it
is done in PVE currently.
Dominik Csapak [Mon, 31 Oct 2022 11:39:53 +0000 (12:39 +0100)]
file-restore: dynamically increase memory of vm for zpools
when a backup contains a drive with zfs on it, the default memory
size (up to 384 MiB) is often not enough to hold the zfs metadata
to improve that situation, add memory dynamically (1GiB) when a path is
requested that is on zfs. Note that the image must be started with a
kernel capable of memory hotplug.
to achieve that, we also have to add a qmp socket to the vm, so that
we can later connect and add the memory backend and dimm
Stefan Hanreich [Thu, 20 Oct 2022 13:36:30 +0000 (15:36 +0200)]
fix #4301: correctly pass rate limit parameters to API
With the old code the rate limit parameters got passed in their own
dictionary under the limit key, but the API expects the rate-limit
settings as top-level keys. This commit correctly sets the rate-limit
parameters so the API actually uses them.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Dominik Csapak [Thu, 20 Oct 2022 07:40:56 +0000 (09:40 +0200)]
datastore: implement sync-level tuning for datastores
currently, we don't (f)sync on chunk insertion (or at any point after
that), which can lead to broken chunks in case of e.g. an unexpected
powerloss. To fix that, offer a tuning option for datastores that
controls the level of syncs it does:
* None (default): same as current state, no (f)syncs done at any point
* Filesystem: at the end of a backup, the datastore issues
a syncfs(2) to the filesystem of the datastore
* File: issues an fsync on each chunk as they get inserted
(using our 'replace_file' helper) and a fsync on the directory handle
a small benchmark showed the following (times in mm:ss):
setup: virtual pbs, 4 cores, 8GiB memory, ext4 on spinner
so if the backup fits in memory, there is a large difference between all
of the modes (expected), but as soon as it exceeds the memory size,
the difference between not syncing and syncing the fs at the end becomes
much smaller.
i also tested on an nvme, but there the syncs basically made no difference
Dominik Csapak [Wed, 19 Oct 2022 11:13:32 +0000 (13:13 +0200)]
api/tape/inventory: optionally try to restore catalogs
in a disaster recovery case, it is useful to not only re-invetorize
the labels + media-sets, but also to try to recover the catalogs
from the tape (to know whats on there). This adds an option to
the inventory api call that tries to do a fast catalog restore
from each tape to be inventorized.
also sets the correct default for 'read-all-labels' in the api and
converts to a bool
Dominik Csapak [Wed, 19 Oct 2022 11:13:31 +0000 (13:13 +0200)]
tape: replace '&Path' with 'AsRef<Path>' in function parameters
this way we can omit the pattern
```
let status_path = Path::new(TAPE_STATUS_DIR);
some_function(status_path);
```
and give the TAPE_STATUS_DIR directly. In some instances we now have to
give TAPE_STATUS_DIR more often, but most often we save a few
intermediary Paths.
Lukas Wagner [Thu, 20 Oct 2022 09:22:50 +0000 (11:22 +0200)]
pbs-client: fix bash autocompletion for archive-files
Previously, autocompletion of archive names, for instance
in the case of
$ proxmox-backup-client restore <snapshot> <TAB>
did not work if no namespace was provided via the --ns option.
The fix is to fall back to the root namespace if the option is
not provided by the user.
Dominik Csapak [Tue, 11 Oct 2022 09:20:33 +0000 (11:20 +0200)]
ui: node certificate: make parent panel scrollable
The `ACME` panel may be rendered out of sight (overflow) when there
are enough SANs on the upper `Certificates` panel to push it down
enough and the browser/display height is too small.
Enable the `scrollable` config for the parent certificate view panel
in the y-axis (vertical) to make ExtJS automatically add a scrollbar
if the content overflows. Avoid enabling it for the x-axis too as
that breaks reflow of the layout if the browser window gets resized.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[T: fix commit message to one from manager ] Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Dominik Csapak [Thu, 6 Oct 2022 09:08:12 +0000 (11:08 +0200)]
fix #4274: implement prune notifications
we converted the prune settings of datastores to prune-jobs, but did
not actually implement the notifications for them, even though
we had the notification options in the gui (they did not work).
implement the basic ok/error notification for prune jobs
Remove BackupFileDownloader.js file and Makefile entry
While finishing up my previous patch series I stumbled upon this file
which is not referenced anywhere in the proxmox-backup repository nor
in any of the other repositories as far as I can tell (except in the
Makefile).
I searched far and wide for any occurences and tested the backup
explorer GUI without it, which still seems to work normally.
Dominik Csapak [Fri, 27 May 2022 08:22:03 +0000 (10:22 +0200)]
file-restore: remove 'json-error' parameter from list_files
we can reuse the 'output_format' here
also remove the 'error: true' here. we can determine it was an error,
by checking if it's an object with a 'message' property
move the two columns to a field set each, one for the backup job
config, containing weekdays and schedule, and one for the simulation
time, which contains the end date/time and the duration, which got
moved over, as it better fits along the new date/time.
Note that I used end instead of start as its the later point in time,
so hopefully less confusing, but one could argue either.
Flex and width of the fields got adapted such that they now better
work on smaller resolutions (< 1500px total width) and look better
on both, low and high resolutions/width.
The button got moved into the docked area mostly due to me being to
lazy to add yet another container indirection for having the hbox
nested inside a vbox, but also as it makes sense to have it big and
centered available.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
With this, the prune simulator will be more flexible, which might clear
up questions such as these:
https://forum.proxmox.com/threads/pbs-prune-simulator-monthly-backups.115081
In order to have the configuration window not take up too much space,
I added another column and moved the border to the left.
Stefan Hanreich [Fri, 16 Sep 2022 08:48:38 +0000 (10:48 +0200)]
fix #4095: make http client use proxy config from 'ALL_PROXY' env var
In order to be able to use a proxy with the proxmox-backup-client, use
ProxyConfig for parsing proxy server config from the environment. Also
added a section in the documentation that describes how to configure the
environment if a proxy server should be used.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Sterz [Wed, 24 Aug 2022 12:48:25 +0000 (14:48 +0200)]
fix: datastore: make relative_group_path() return relative path
previously the BackGroup trait used the datastore's
namespace_path() method to construct a base path. this would result in
it returning an absolute path equivalent to full_group_path(). use
the namspace's path() method instead to get a relative path, in-line
with backup_dir's relative_path().
when we set a default language server side, we want to display that on
the login page. For that we have to use the 'language' variable in the
index template.
Also set the fallback to '__default__' instead of 'en' so that we show
'Default (English)'.
no real change for PBS usage - the ApiHandler enum is marked
non_exhaustive now because it has extra values if the new (enabled by
default) "server" feature is enabled.
Read should send exactly the number of bytes requested except
on EOF or error, otherwise the rest of the data will be
substituted with zeroes.
but we simply forwarded the bytes we got from 'read_at'. The result was
that files were corrupt as soon as read_at returned not the exact number
of bytes requested. such short reads are easy to trigger with large
files (where reading a file has to cross many chunk boundaries).
To fix that, loop over 'read_at' until our buffer is full, or we read
0 bytes, indicating EOF.
reported in the forum:
https://forum.proxmox.com/threads/proxmox-backup-client-mounting-a-pxar-archive-gives-truncated-files.114447/
else building pbs-config directly (not from the workspace root with `-p
pbs-config`) fails (and so do similar dep chains that don't pull in the
feature via another way, like `cd proxmox-backup-client; cargo build`).
Markus Frank [Fri, 19 Aug 2022 10:48:47 +0000 (12:48 +0200)]
pbs-client: added options to skip acls/xattrs/ownership/permissions
Also added WITH_OWNER and WITH_PERMISSION to Default-Flags,
because otherwise it would be needed to activly set these flags and most
filesystems that support XATTR and ACL also support
POSIX-Permissions & Ownership.
projects / proxmox-backup.git / log