Stefan Reiter [Mon, 26 Apr 2021 13:04:16 +0000 (15:04 +0200)]
file-restore: use less memory for VM and reboot on panic
With the vsock-pkt-buffer fix in proxmox-backup-restore-image, we can
use way less memory for the VM without risking any crashes. 128 MiB
seems to be the lowest it will go and still be fully reliable.
While at it, add the "panic=1" argument to the kernel command line, so
in case the kernel *does* run out of memory, it will at least restart
automatically.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
when a user updates a job schedule, we want to save that point in time
to calculate future runs, otherwise when a user updates a schedule to
a time that would have been between the last run and 'now' the
schedule is triggered instantly
for example:
schedule 08:00
last run today 08:00
now it is 12:00
before this patch:
update schedule to 11:00
-> triggered instantly since we calculate from 08:00
after this patch:
update schedule to 11:00
-> triggered tomorrow 11:00 since we calculate from today 12:00
the change in the enum type is ok, since by default serde does not
error on unknown fields and the new field is optional
Thomas Lamprecht [Thu, 22 Apr 2021 19:24:40 +0000 (21:24 +0200)]
server/rest: fix new type ambiguity
basically the same as commit eeff085d9d09028aba722d4106634e2f9c9ce931
Will be required once we get to use a newer rustc, at least the
client build for archlinux was broken due to this.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Reiter [Thu, 22 Apr 2021 15:34:46 +0000 (17:34 +0200)]
client-tools: add crypto_parameters_keep_fd
same functionality as crypto_parameters, except it keeps the file
descriptor passed as "keyfd" open (and seeks to the beginning after
reading), if one is given.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
in some configurations, samba stores NTFS-ACLs in this xattr[0], so
we should backup (if we can)
altough the 'security' namespace is special (e.g. in use by
selinux, etc.) this value is normally only used by samba and we
should be able to back it up.
to restore it, the user needs at least 'CAP_SYS_ADMIN' rights, otherwise
it cannot be set
tape/changer/sg_pt_changer: read whole descriptor size for each entry
Some changer seem to append more data than we expect, but correctly
annotates that size in the subheader.
For each descriptor entry, read as much as the size given in the
subheader (or until the end of the reader), else our position in
the reader is wrong for the next entry, and we will parse
incorrect data.
While we skip known-verified chunks in the stat-and-inode-sort loop,
those are only the ones from previous indexes. If there's a repeated
chunk in one index they would get re-verified more often as required.
So, add the check again explicitly to the read+verify loop.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
backup/verify: improve speed by sorting chunks by inode
before reading the chunks from disk in the order of the index file,
stat them first and sort them by inode number.
this can have a very positive impact on read speed on spinning disks,
even with the additional stat'ing of the chunks.
memory footprint should be tolerable, for 1_000_000 chunks
we need about ~16MiB of memory (Vec of 64bit position + 64bit inode)
(assuming 4MiB Chunks, such an index would reference 4TiB of data)
two small benchmarks (single spinner, ext4) here showed an improvement from
~430 seconds to ~330 seconds for a 32GiB fixed index
and from
~160 seconds to ~120 seconds for a 10GiB dynamic index
Thomas Lamprecht [Wed, 14 Apr 2021 10:29:18 +0000 (12:29 +0200)]
docs: reorder maintenance & network after client usage/tools
The idea is that people first need to make actual backups before they
need to do maintenance tasks.
Network is already setup when installing with the ISO or on-top of
Debian, so that is not a priority either.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
api2/tape/restore: restore_chunk_archive: only ignore tape related errors
when we get an error from the tape, we possibly want to ignore it,
i.e. when the file was incomplete, but we still want to error
out if the error came from e.g, the datastore, so we have to move
the error checking code to the 'next_chunk' call
Stefan Reiter [Mon, 12 Apr 2021 15:32:52 +0000 (17:32 +0200)]
api/datastore: allow pxar file download of entire archive
Treat filepaths like "/root.pxar.didx" without a trailing slash as
wanting to download the entire archive content instead of erroring. The
zip-creation code already works fine for this scenario.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Thomas Lamprecht [Mon, 12 Apr 2021 12:09:29 +0000 (14:09 +0200)]
ui: nav tree: code cleanup and unification between datastore and tapes
datastore and tape entries are very similar but differ in some points
in such a way that a nice unification is not really that helpful, but
making similar key parts the same is still nice when reading the code
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
tools/async_io: do not error on Accept for StaticIncoming
in proxmox-backup-proxy, we log and discard any errors on 'accept',
so that we can continue to server requests
in proxmox-backup-api, we just have the StaticIncoming that accepts,
which will forward any errors from the underlying TcpListener
this patch also logs and discards the errors, like in the proxy.
Otherwise it could happen that if the api-daemon has more files open
than the proxy, it will shut itself down because of a
'too many open files' error if there are many open connections
(the service should also restart on exit i think, but this is
a separate issue)
if a datastore or root is not used directly on the pool dir
(e.g. the installer creates 2 sub datasets ROOT/pbs-1), info in
/proc/self/mountinfo returns not the pool, but the path to the
dataset, which has no iostats itself in /proc/spl/kstat/zfs/
but only the pool itself
so instead of not gathering data at all, gather the info from the
underlying pool instead. if one has multiple datastores on the same
pool those rrd stats will be the same for all those datastores now
(instead of empty) similar to 'normal' directories
projects / proxmox-backup.git / log