]> git.proxmox.com Git - proxmox-backup.git/log
proxmox-backup.git
3 years agodatastore content: search: set emptytext to searched columns
Thomas Lamprecht [Tue, 1 Sep 2020 16:30:20 +0000 (18:30 +0200)]
datastore content: search: set emptytext to searched columns

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agodatastore content: reduce count column width
Thomas Lamprecht [Tue, 1 Sep 2020 16:28:12 +0000 (18:28 +0200)]
datastore content: reduce count column width

Using 75 as width we can display up to 9999999 which would allow
displaying over 19 years of snapshots done each minute, so quite
enough for the common cases.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agodatastore content: reload after verify
Thomas Lamprecht [Tue, 1 Sep 2020 16:27:30 +0000 (18:27 +0200)]
datastore content: reload after verify

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agod/postinst: always fixup termproxy user id and for all users
Stefan Reiter [Tue, 1 Sep 2020 14:50:18 +0000 (16:50 +0200)]
d/postinst: always fixup termproxy user id and for all users

Anyone with a PAM account and Sys.Console access could have started a
termproxy session, adapt the regex.

Always test for broken entries and run the sed expression to make sure
eventually all occurences of the broken syntax are fixed.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agoui: DataStoreContent.js: fix wrong comma
Dominik Csapak [Tue, 1 Sep 2020 09:05:21 +0000 (11:05 +0200)]
ui: DataStoreContent.js: fix wrong comma

should be semicolon

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoui: DataStoreContent: improve encrypted column
Dominik Csapak [Tue, 1 Sep 2020 09:05:20 +0000 (11:05 +0200)]
ui: DataStoreContent: improve encrypted column

do not count files where we do not have any information

such files exist in the backup dir, but are not in the manifest
so we cannot use those files for determining if the backups are
encrypted or not

this marks encrypted/signed backups with unencrypted client.log.blob files as
encrypted/signed (respectively) instead of 'Mixed'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agosrc/backup/verify.rs: use global hashes (instead of per group)
Dietmar Maurer [Tue, 1 Sep 2020 11:33:04 +0000 (13:33 +0200)]
src/backup/verify.rs: use global hashes (instead of per group)

This makes verify more predictable.

3 years agosrc/backup/data_blob.rs: avoid useless &, data is already a reference
Dietmar Maurer [Tue, 1 Sep 2020 10:47:17 +0000 (12:47 +0200)]
src/backup/data_blob.rs: avoid useless &, data is already a reference

3 years agosrc/backup/verify.rs: use separate thread to load data
Dietmar Maurer [Tue, 1 Sep 2020 09:17:13 +0000 (11:17 +0200)]
src/backup/verify.rs: use separate thread to load data

3 years agodebian/control: fix versions
Dietmar Maurer [Tue, 1 Sep 2020 08:19:40 +0000 (10:19 +0200)]
debian/control: fix versions

3 years agodepend on proxmox 0.3.4
Dietmar Maurer [Fri, 28 Aug 2020 04:32:33 +0000 (06:32 +0200)]
depend on proxmox 0.3.4

3 years agobump version to 0.8.13-1 v0.8.13
Thomas Lamprecht [Thu, 27 Aug 2020 14:15:31 +0000 (16:15 +0200)]
bump version to 0.8.13-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agogc: use human readable units for summary
Thomas Lamprecht [Thu, 27 Aug 2020 13:55:57 +0000 (15:55 +0200)]
gc: use human readable units for summary

and avoid the "percentage done: X %" phrase

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump pxar dep to 0.6.0
Wolfgang Bumiller [Thu, 27 Aug 2020 10:16:03 +0000 (12:16 +0200)]
bump pxar dep to 0.6.0

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agobin/backup-proxy: assert that daemon runs as backup user/group
Thomas Lamprecht [Wed, 26 Aug 2020 08:26:20 +0000 (10:26 +0200)]
bin/backup-proxy: assert that daemon runs as backup user/group

Because if not, the backups it creates have bogus permissions and may
seem like they got broken once the daemon is started again with the
correct user/group.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobackup: add BACKUP_GROUP_NAME const and backup_group helper
Thomas Lamprecht [Wed, 26 Aug 2020 08:02:18 +0000 (10:02 +0200)]
backup: add BACKUP_GROUP_NAME const and backup_group helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobin/backup-proxy: remove outdated perl comments
Thomas Lamprecht [Wed, 26 Aug 2020 08:25:42 +0000 (10:25 +0200)]
bin/backup-proxy: remove outdated perl comments

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobin/backup-proxy: return error directly in main
Thomas Lamprecht [Wed, 26 Aug 2020 08:24:32 +0000 (10:24 +0200)]
bin/backup-proxy: return error directly in main

anyhow makes this a nice error message, similar to the manual
wrapping used.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoui: datastore content: show last verify result from a snapshot
Thomas Lamprecht [Tue, 25 Aug 2020 15:30:28 +0000 (17:30 +0200)]
ui: datastore content: show last verify result from a snapshot

Double-click on the verify grid-cell of a specific snapshot (not the
group) opens the relevant task log.

The date of the last verify is shown as tool-tip.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agosave last verify result in snapshot manifest
Thomas Lamprecht [Tue, 25 Aug 2020 15:30:27 +0000 (17:30 +0200)]
save last verify result in snapshot manifest

Save the state ("ok" or "failed") and the UPID of the respective
verify task. With this we can easily allow to open the relevant task
log and show when the last verify happened.

As we already load the manifest when listing the snapshots, just add
it there directly.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agovarious typo fixes
Thomas Lamprecht [Tue, 25 Aug 2020 16:52:31 +0000 (18:52 +0200)]
various typo fixes

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoSnapshotListItem: add comment field also to schema
Thomas Lamprecht [Tue, 25 Aug 2020 13:24:04 +0000 (15:24 +0200)]
SnapshotListItem: add comment field also to schema

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agotools/format: implement from u64 for HumanByte helper type
Thomas Lamprecht [Tue, 25 Aug 2020 12:18:48 +0000 (14:18 +0200)]
tools/format: implement from u64 for HumanByte helper type

Could be problematic for systems where usize is 32 bit, but we do not
really support those.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoapi datastore: fix typo in error message
Thomas Lamprecht [Tue, 25 Aug 2020 12:16:40 +0000 (14:16 +0200)]
api datastore: fix typo in error message

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoupdate d/control for pxar 0.5.0
Wolfgang Bumiller [Tue, 25 Aug 2020 10:37:06 +0000 (12:37 +0200)]
update d/control for pxar 0.5.0

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agobump pxar dep to 0.5.0
Wolfgang Bumiller [Tue, 25 Aug 2020 10:36:54 +0000 (12:36 +0200)]
bump pxar dep to 0.5.0

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoAdd section "Garbage Collection"
Dylan Whyte [Mon, 24 Aug 2020 10:14:50 +0000 (12:14 +0200)]
Add section "Garbage Collection"

Add the section "Garbage Collection" to section "Backup Server
Management". This briefly explains the "garbage-collection"
subcommand of "proxmox-backup-manager"

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agoAdd section "Network Management"
Dylan Whyte [Mon, 24 Aug 2020 10:13:50 +0000 (12:13 +0200)]
Add section "Network Management"

Add the section "Network Management", which explains the
"network" subcommand of "proxmox-backup-manager"

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agoadd note about TLS benchmark test.
Dietmar Maurer [Tue, 25 Aug 2020 07:36:12 +0000 (09:36 +0200)]
add note about TLS benchmark test.

3 years agoAdd further explanation to benchmarking
Dylan Whyte [Mon, 24 Aug 2020 08:18:01 +0000 (10:18 +0200)]
Add further explanation to benchmarking

Adds a note, explaing the percentages shown in the output
of the benchmark

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agoadministration-guide.rst: remove debug output from code examples
Dietmar Maurer [Tue, 25 Aug 2020 07:29:52 +0000 (09:29 +0200)]
administration-guide.rst: remove debug output from code examples

3 years agoAdd section "Disk Management"
Dylan Whyte [Mon, 24 Aug 2020 08:17:42 +0000 (10:17 +0200)]
Add section "Disk Management"

Add the section "Disk Management" to the admin guide, explaining
the use of the "disk" subcommand of "proxmox-backup-manager"

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agodebian/control: update for new pxar 0.4 dependency
Dietmar Maurer [Tue, 25 Aug 2020 07:09:00 +0000 (09:09 +0200)]
debian/control: update for new pxar 0.4 dependency

3 years agobump version to 0.8.12-1 v0.8.12
Dietmar Maurer [Tue, 25 Aug 2020 06:57:12 +0000 (08:57 +0200)]
bump version to 0.8.12-1

3 years agoverify: speedup - only verify chunks once
Dietmar Maurer [Tue, 25 Aug 2020 06:52:24 +0000 (08:52 +0200)]
verify: speedup - only verify chunks once

We need to do the check before we load the chunk.

3 years agoverify: sort backup groups
Dietmar Maurer [Tue, 25 Aug 2020 06:38:47 +0000 (08:38 +0200)]
verify: sort backup groups

3 years agobump pxar dep to 0.4.0
Wolfgang Bumiller [Mon, 24 Aug 2020 09:55:57 +0000 (11:55 +0200)]
bump pxar dep to 0.4.0

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agodocs: fix PBS wiki link
Thomas Lamprecht [Fri, 21 Aug 2020 09:09:39 +0000 (11:09 +0200)]
docs: fix PBS wiki link

rst/sphinx and comments are a PITA...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agobump version to 0.8.11-1 v0.8.11
Thomas Lamprecht [Wed, 19 Aug 2020 17:20:22 +0000 (19:20 +0200)]
bump version to 0.8.11-1

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoFix typo in program output
Dylan Whyte [Tue, 18 Aug 2020 11:46:50 +0000 (13:46 +0200)]
Fix typo in program output

Change "comptation" -> "computation"

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agod/postinst: fixup userid for older termproxy tasks
Thomas Lamprecht [Tue, 18 Aug 2020 16:41:30 +0000 (18:41 +0200)]
d/postinst: fixup userid for older termproxy tasks

At the time when we can fix this up the new (and possibly an old)
server daemon process is running, so use the flock CLI tool from
util-linux to ensure we do the same locking as the server and thus we
avoid a race condition.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 years agoAdd "Benchmarking" section
Dylan Whyte [Tue, 18 Aug 2020 10:52:24 +0000 (12:52 +0200)]
Add "Benchmarking" section

This adds the "Benchmarking" section which discusses
the proxmox-backup-client benchmark subcommand.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agoAdd section describing acl tool
Dylan Whyte [Tue, 18 Aug 2020 10:51:18 +0000 (12:51 +0200)]
Add section describing acl tool

This adds a section how to use the acl subcommand
to manage user access control

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agoCorrect erroneous instructions and add clarity
Dylan Whyte [Tue, 18 Aug 2020 10:50:07 +0000 (12:50 +0200)]
Correct erroneous instructions and add clarity

This patch changes the following:
- Provide extra clarity to instruction and information where
  appropriate.
- Fix examples and content that would lead to erroneous behavior
  in a command.
- Insert section about installing on Debian into a caution block

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agominor language and formatting fixup
Dylan Whyte [Tue, 18 Aug 2020 10:48:49 +0000 (12:48 +0200)]
minor language and formatting fixup

this fixes minor grammatical errors throughout the pbs docs
and rewords certain sections for improved readability.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agoturn UPID into an API type
Wolfgang Bumiller [Tue, 18 Aug 2020 07:04:21 +0000 (09:04 +0200)]
turn UPID into an API type

It's a string-type.
Implement Serialize via Display, Deserialize via FromStr and
add an API_SCHEMA so that it can be used as a type within
the #[api] macro.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agostyle fixups
Wolfgang Bumiller [Tue, 18 Aug 2020 06:50:14 +0000 (08:50 +0200)]
style fixups

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoremove map_struct helper
Wolfgang Bumiller [Mon, 17 Aug 2020 09:53:01 +0000 (11:53 +0200)]
remove map_struct helper

if we ever need this it should be marked as unsafe!

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoadd methods to allocate a DynamicIndexHeader
Wolfgang Bumiller [Mon, 17 Aug 2020 09:50:32 +0000 (11:50 +0200)]
add methods to allocate a DynamicIndexHeader

to avoid `map_struct` which is actually unsafe because it
does not verify alignment constraints at all

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoapi2/node/services: turn service api calls into workers
Dominik Csapak [Fri, 14 Aug 2020 10:03:08 +0000 (12:03 +0200)]
api2/node/services: turn service api calls into workers

to be in line with pve/pmg and be able to show the progress in the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agofix #2904: zpool status: parse vdevs with state but without statistics
Dominik Csapak [Thu, 30 Jul 2020 08:02:16 +0000 (10:02 +0200)]
fix #2904: zpool status: parse vdevs with state but without statistics

some vdevs (e.g. spares) have a 'state' (e.g. AVAIL), but
not statistics like READ/WRITE/etc.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agozfs status: add test with spares
Dominik Csapak [Thu, 30 Jul 2020 08:02:15 +0000 (10:02 +0200)]
zfs status: add test with spares

this will fail for now, fixed in the next commit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agocleanup last patch
Dietmar Maurer [Fri, 14 Aug 2020 05:30:05 +0000 (07:30 +0200)]
cleanup last patch

3 years agoapi2/node/../disks/directory: added DELETE endpoint for removal of mount-units
Hannes Laimer [Thu, 13 Aug 2020 10:58:52 +0000 (12:58 +0200)]
api2/node/../disks/directory: added DELETE endpoint for removal of mount-units

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
3 years agoFix #2926: parse_iface_attributes: always break on non-{attribitue, comment} token
Fabian Ebner [Thu, 13 Aug 2020 13:04:12 +0000 (15:04 +0200)]
Fix #2926: parse_iface_attributes: always break on non-{attribitue, comment} token

There is no requirement to have at least
a blank line, attribute or comment in between two
interface definitions, e.g.
iface lo inet loopback
iface lo inet6 loopback

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
3 years agoconfig/jobstate: replace Job:load with create_state_file
Dominik Csapak [Thu, 13 Aug 2020 12:30:19 +0000 (14:30 +0200)]
config/jobstate: replace Job:load with create_state_file

it really is not necessary, since the only time we are interested in
loading the state from the file is when we list it, and there
we use JobState::load directly to avoid the lock

we still need to create the file on syncjob creation though, so
that we have the correct time for the schedule

to do this we add a new create_state_file that overwrites it on creation
of a syncjob

for safety, we subtract 30 seconds from the in-memory state in case
the statefile is missing

since we call create_state_file from  proxmox-backup-api,
we have to chown the lock file after creating to the backup user,
else the sync job scheduling cannot aquire the lock

also we remove the lock file on statefile removal

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agocleanup: replace id from do_sync_job with info from job
Dominik Csapak [Thu, 13 Aug 2020 12:30:18 +0000 (14:30 +0200)]
cleanup: replace id from do_sync_job with info from job

we already have it inside the job itself

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agocleanup: merge endtime into TaskState
Dominik Csapak [Thu, 13 Aug 2020 12:30:17 +0000 (14:30 +0200)]
cleanup: merge endtime into TaskState

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoui: syncjob: make some columns smaller
Dominik Csapak [Thu, 13 Aug 2020 08:29:21 +0000 (10:29 +0200)]
ui: syncjob: make some columns smaller

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoui: syncjob: improve task text rendering
Dominik Csapak [Thu, 13 Aug 2020 08:29:20 +0000 (10:29 +0200)]
ui: syncjob: improve task text rendering

to also have the correct icons for warnings and unknown tasks

the text is here "ERROR: ..." now, so leave the 'Error' out

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agosyncjob: use do_sync_job also for scheduled sync jobs
Dominik Csapak [Thu, 13 Aug 2020 08:29:19 +0000 (10:29 +0200)]
syncjob: use do_sync_job also for scheduled sync jobs

and determine the last runtime with the jobstate

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoapi2/pull: extend do_sync_job to also handle schedule and jobstate
Dominik Csapak [Thu, 13 Aug 2020 08:29:18 +0000 (10:29 +0200)]
api2/pull: extend do_sync_job to also handle schedule and jobstate

so that we can log if triggered by a schedule, and writing to a jobstatefile
also correctly polls now the abort_future of the worker, so that
users can stop a sync

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoapi/{pull, sync}: refactor to do_sync_job
Dominik Csapak [Thu, 13 Aug 2020 08:29:17 +0000 (10:29 +0200)]
api/{pull, sync}: refactor to do_sync_job

and move the pull parameters into the worker, so that the task log
contains the error if there is one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoapi2/admin/sync: use JobState for faster access to state info
Dominik Csapak [Thu, 13 Aug 2020 08:29:16 +0000 (10:29 +0200)]
api2/admin/sync: use JobState for faster access to state info

and delete the statefile again on syncjob removal

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoconfig: add JobState helper
Dominik Csapak [Thu, 13 Aug 2020 08:29:15 +0000 (10:29 +0200)]
config: add JobState helper

this is intended to be a generic helper to (de)serialize job states
(e.g., sync, verify, and so on)

writes a json file into '/var/lib/proxmox-backup/jobstates/TYPE-ID.json'

the api creates the directory with the correct permissions, like
the rrd directory

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoserver/worker_task: let upid_read_status also return the endtime
Dominik Csapak [Thu, 13 Aug 2020 08:29:14 +0000 (10:29 +0200)]
server/worker_task: let upid_read_status also return the endtime

the endtime should be the timestamp of the last log line
or if there is no log at all, the starttime

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoserver: change status of a task from a string to an enum
Dominik Csapak [Thu, 13 Aug 2020 08:29:13 +0000 (10:29 +0200)]
server: change status of a task from a string to an enum

representing a state via an enum makes more sense in this case
we also implement FromStr and Display to make it easy to convet from/to
a string

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
3 years agoremove unused import
Wolfgang Bumiller [Thu, 13 Aug 2020 07:05:14 +0000 (09:05 +0200)]
remove unused import

rustc doesn't warn about this kind of import, however,
clippy does

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoformatting fixups
Wolfgang Bumiller [Wed, 12 Aug 2020 12:30:03 +0000 (14:30 +0200)]
formatting fixups

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoreplace and remove old ticket functions
Wolfgang Bumiller [Wed, 12 Aug 2020 10:05:52 +0000 (12:05 +0200)]
replace and remove old ticket functions

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agointroduce Ticket struct
Wolfgang Bumiller [Wed, 12 Aug 2020 08:44:54 +0000 (10:44 +0200)]
introduce Ticket struct

and add tests and compatibility tests

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoworker_task: new_thread() - remove unused tokio channel
Dietmar Maurer [Wed, 12 Aug 2020 06:41:43 +0000 (08:41 +0200)]
worker_task: new_thread() - remove unused tokio channel

3 years agobump version to 0.8.10-1 v0.8.10
Fabian Grünbichler [Tue, 11 Aug 2020 13:47:24 +0000 (15:47 +0200)]
bump version to 0.8.10-1

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agobackup: lock base snapshot and ensure existance on finish
Stefan Reiter [Tue, 11 Aug 2020 08:50:41 +0000 (10:50 +0200)]
backup: lock base snapshot and ensure existance on finish

To prevent forgetting the base snapshot of a running backup, and catch
the case when it still happens (e.g. via manual rm) to at least error
out instead of storing a potentially invalid backup.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agoRevert "backup: ensure base snapshots are still available after backup"
Stefan Reiter [Tue, 11 Aug 2020 08:50:40 +0000 (10:50 +0200)]
Revert "backup: ensure base snapshots are still available after backup"

This reverts commit d53fbe24741ed3bd920ee1c61bc163b1874b7c82.

The HashSet and "register" function are unnecessary, as we already know
which backup is the one we need to check: the last one, stored as
'last_backup'.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agobackup: flock snapshot on backup start
Stefan Reiter [Tue, 11 Aug 2020 08:50:39 +0000 (10:50 +0200)]
backup: flock snapshot on backup start

An flock on the snapshot dir itself is used in addition to the group dir
lock. The lock is used to avoid races with forget and prune, while
having more granularity than the group lock (i.e. the group lock is
necessary to prevent more than one backup per group, but the snapshot
lock still allows backups unrelated to the currently running to be
forgotten/pruned).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agodatastore: prevent in-use deletion with locks instead of heuristic
Stefan Reiter [Tue, 11 Aug 2020 08:50:38 +0000 (10:50 +0200)]
datastore: prevent in-use deletion with locks instead of heuristic

Attempt to lock the backup directory to be deleted, if it works keep the
lock until the deletion is complete. This way we ensure that no other
locking operation (e.g. using a snapshot as base for another backup) can
happen concurrently.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agosrc/backup/backup_info.rs: remove BackupGroup lock()
Dietmar Maurer [Tue, 11 Aug 2020 08:50:37 +0000 (10:50 +0200)]
src/backup/backup_info.rs: remove BackupGroup lock()

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agosrc/tools/fs.rs: new helper lock_dir_noblock
Dietmar Maurer [Tue, 11 Aug 2020 08:50:36 +0000 (10:50 +0200)]
src/tools/fs.rs: new helper lock_dir_noblock

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agoMakefile: build target - do not fail if control file does not exist
Dietmar Maurer [Tue, 11 Aug 2020 08:47:23 +0000 (10:47 +0200)]
Makefile: build target - do not fail if control file does not exist

This can happen if a previous build failed ...

3 years agomark signed manifests as such
Fabian Grünbichler [Mon, 10 Aug 2020 11:25:09 +0000 (13:25 +0200)]
mark signed manifests as such

for less-confusing display in the web interface

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoverify: also check chunk CryptMode
Fabian Grünbichler [Mon, 10 Aug 2020 11:25:08 +0000 (13:25 +0200)]
verify: also check chunk CryptMode

and in-line verify_stored_chunk to avoid double-loading each chunk.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agochunk readers: ensure chunk/index CryptMode matches
Fabian Grünbichler [Mon, 10 Aug 2020 11:25:07 +0000 (13:25 +0200)]
chunk readers: ensure chunk/index CryptMode matches

an encrypted Index should never reference a plain-text chunk, and an
unencrypted Index should never reference an encrypted chunk.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agodatastore api: verify blob/index csum from manifest
Fabian Grünbichler [Mon, 10 Aug 2020 11:25:06 +0000 (13:25 +0200)]
datastore api: verify blob/index csum from manifest

when dowloading decoded files.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agodatastore api: only decode unencrypted indices
Fabian Grünbichler [Mon, 10 Aug 2020 11:25:05 +0000 (13:25 +0200)]
datastore api: only decode unencrypted indices

these checks were already in place for regular downloading of backed up
files, also do them when attempting to decode a catalog, or when
downloading decoded files referenced by a pxar index.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 years agoimpl PartialEq between Realm and RealmRef
Wolfgang Bumiller [Mon, 10 Aug 2020 10:22:33 +0000 (12:22 +0200)]
impl PartialEq between Realm and RealmRef

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoassert that Username does not impl PartialEq
Wolfgang Bumiller [Mon, 10 Aug 2020 10:21:12 +0000 (12:21 +0200)]
assert that Username does not impl PartialEq

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agointroduce Username, Realm and Userid api types
Wolfgang Bumiller [Thu, 6 Aug 2020 13:46:01 +0000 (15:46 +0200)]
introduce Username, Realm and Userid api types

and begin splitting up types.rs as it has grown quite large
already

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agod/control: proxmox 0.3.3
Wolfgang Bumiller [Mon, 10 Aug 2020 10:04:43 +0000 (12:04 +0200)]
d/control: proxmox 0.3.3

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agobump proxmox dependency to 0.3.3 for serde helpers
Wolfgang Bumiller [Mon, 10 Aug 2020 09:31:45 +0000 (11:31 +0200)]
bump proxmox dependency to 0.3.3 for serde helpers

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agomove types.rs to types/mod.rs
Wolfgang Bumiller [Mon, 10 Aug 2020 08:32:31 +0000 (10:32 +0200)]
move types.rs to types/mod.rs

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agoconfig.rs: sort modules
Wolfgang Bumiller [Thu, 6 Aug 2020 10:06:18 +0000 (12:06 +0200)]
config.rs: sort modules

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
3 years agobackup: only allow finished backups as base snapshot
Stefan Reiter [Tue, 4 Aug 2020 10:42:00 +0000 (12:42 +0200)]
backup: only allow finished backups as base snapshot

If the datastore holds broken backups for some reason, do not attempt to
base following snapshots on those. This would lead to an error on
/previous, leaving the client no choice but to upload all chunks, even
though there might be potential for incremental savings.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agoadministration-guide.rst: move Encryption headline up one level
Dietmar Maurer [Fri, 7 Aug 2020 05:10:12 +0000 (07:10 +0200)]
administration-guide.rst: move Encryption headline up one level

3 years agoadministration-guide.rst: fix headline (avoid compile error)
Dietmar Maurer [Fri, 7 Aug 2020 04:56:58 +0000 (06:56 +0200)]
administration-guide.rst: fix headline (avoid compile error)

3 years agoadmin-guide: add section explaining master keys
Dylan Whyte [Thu, 6 Aug 2020 10:03:31 +0000 (12:03 +0200)]
admin-guide: add section explaining master keys

Adds a section under encryption which goes into detail on how to
use a master key to store and recover backup encryption keys.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agofix: master-key: upload RSA encoded key with backup
Dylan Whyte [Thu, 6 Aug 2020 07:13:40 +0000 (09:13 +0200)]
fix: master-key: upload RSA encoded key with backup

When uploading an RSA encoded key alongside the backup,
the backup would fail with the error message: "wrong blob
file extension".
Adding the '.blob' extension to rsa-encrypted.key before the
the call to upload_blob_from_data(), rather than after, fixes
the issue.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
3 years agofinish_backup: mark backup as finished only after checks have passed
Stefan Reiter [Tue, 4 Aug 2020 10:41:59 +0000 (12:41 +0200)]
finish_backup: mark backup as finished only after checks have passed

Commit 9fa55e09 "finish_backup: test/verify manifest at server side"
moved the finished-marking above some checks, which means if those fail
the backup would still be marked as successful on the server.

Revert that part and comment the line for the future.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 years agofix #2909: handle missing chunks gracefully in garbage collection
Oguz Bektas [Wed, 5 Aug 2020 10:22:15 +0000 (12:22 +0200)]
fix #2909: handle missing chunks gracefully in garbage collection

instead of bailing and stopping the entire GC process, warn about the
missing chunks and continue.

this results in "TASK WARNINGS: X" as the status.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
3 years agod/control: bump proxmox to 0.3.2
Wolfgang Bumiller [Tue, 4 Aug 2020 09:34:58 +0000 (11:34 +0200)]
d/control: bump proxmox to 0.3.2

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>