]>
git.proxmox.com Git - proxmox-backup.git/log
Dietmar Maurer [Tue, 18 Jun 2019 04:23:25 +0000 (06:23 +0200)]
Cargo.toml: use serde feature derive
Dietmar Maurer [Mon, 17 Jun 2019 11:10:00 +0000 (13:10 +0200)]
src/bin/proxmox-backup-client.rs: implement "key change-passphrase"
Dietmar Maurer [Mon, 17 Jun 2019 08:33:24 +0000 (10:33 +0200)]
src/bin/proxmox-backup-client.rs: start key management
Dietmar Maurer [Fri, 14 Jun 2019 12:58:37 +0000 (14:58 +0200)]
src/backup/*_index.rs: used generated magic numbers
Dietmar Maurer [Fri, 14 Jun 2019 09:40:04 +0000 (11:40 +0200)]
src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools
Dietmar Maurer [Fri, 14 Jun 2019 08:36:20 +0000 (10:36 +0200)]
src/api2/backup/environment.rs: log index checksums
Dietmar Maurer [Fri, 14 Jun 2019 08:34:29 +0000 (10:34 +0200)]
src/backup/fixed_index.rs: compute checksum over the index
Dietmar Maurer [Fri, 14 Jun 2019 06:57:52 +0000 (08:57 +0200)]
src/backup/dynamic_index.rs: compute checksum over the index
Dietmar Maurer [Fri, 14 Jun 2019 05:12:30 +0000 (07:12 +0200)]
src/api2/backup/environment.rs: log duplicate chunks
Dietmar Maurer [Fri, 14 Jun 2019 04:23:46 +0000 (06:23 +0200)]
src/api2/backup/upload_chunk.rs: verify chunk data if possible
Dietmar Maurer [Thu, 13 Jun 2019 15:24:57 +0000 (17:24 +0200)]
src/bin/cipherbench.rs: add test for zstd compression
Dietmar Maurer [Thu, 13 Jun 2019 15:16:43 +0000 (17:16 +0200)]
src/bin/cipherbench.rs: cipher speed test with large blocks
Dietmar Maurer [Thu, 13 Jun 2019 14:42:38 +0000 (16:42 +0200)]
src/backup/data_chunk.rs: fix typo
Dietmar Maurer [Thu, 13 Jun 2019 10:56:50 +0000 (12:56 +0200)]
src/backup/data_chunk.rs: only use compressed data if it is shorter than uncompressed
Dietmar Maurer [Thu, 13 Jun 2019 10:47:54 +0000 (12:47 +0200)]
src/backup/crypt_setup.rs: only use compressed data if it is shorter than uncompressed
Dietmar Maurer [Thu, 13 Jun 2019 09:47:23 +0000 (11:47 +0200)]
backup: Add support for client side encryption
first try ...
Dietmar Maurer [Thu, 13 Jun 2019 09:44:38 +0000 (11:44 +0200)]
src/backup/data_chunk.rs - decode: make crypt_config optional
Dietmar Maurer [Thu, 13 Jun 2019 05:35:59 +0000 (07:35 +0200)]
src/backup/crypt_setup.rs: add comment for possible improvements
Dietmar Maurer [Thu, 13 Jun 2019 05:35:02 +0000 (07:35 +0200)]
src/backup/data_chunk.rs: Moved ChunkInfo from merge_known_chunks.rs
Dietmar Maurer [Thu, 13 Jun 2019 05:30:42 +0000 (07:30 +0200)]
src/api2/types.rs: define CHUNK_DIGEST_SCHEMA
Dietmar Maurer [Wed, 12 Jun 2019 04:30:03 +0000 (06:30 +0200)]
src/backup/data_chunk.rs: Data Chunk builder
Make handling encoded chunks easier.
Wolfgang Bumiller [Wed, 12 Jun 2019 11:16:36 +0000 (13:16 +0200)]
use ssh+git for the proxmox repository
This works for me, note the `ssh://` protocol, and using `/`
to separate the path instead of `:`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dietmar Maurer [Mon, 10 Jun 2019 07:56:06 +0000 (09:56 +0200)]
src/backup/crypt_setup.rs: fix encoder for uncompressed chunks
Dietmar Maurer [Mon, 10 Jun 2019 06:27:35 +0000 (08:27 +0200)]
src/backup/crypt_setup.rs: allow compressed and uncompressed chunks
Dietmar Maurer [Mon, 10 Jun 2019 05:55:36 +0000 (07:55 +0200)]
src/backup/crypt_setup.rs: remove unused CryptData struct
Dietmar Maurer [Sun, 9 Jun 2019 09:44:17 +0000 (11:44 +0200)]
src/backup/crypt_setup.rs: avoid useless allocations
Dietmar Maurer [Sat, 8 Jun 2019 08:34:20 +0000 (10:34 +0200)]
src/backup.rs: define magic numbers for chunk store file format
Dietmar Maurer [Sat, 8 Jun 2019 07:51:49 +0000 (09:51 +0200)]
src/backup/crypt_setup.rs: crypto helpers
Dietmar Maurer [Fri, 7 Jun 2019 15:34:53 +0000 (17:34 +0200)]
depend on ../proxmox/proxmox
I was unable to use a git url ...
Wolfgang Bumiller [Fri, 7 Jun 2019 11:14:03 +0000 (13:14 +0200)]
use nightly for now
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Fri, 7 Jun 2019 11:10:56 +0000 (13:10 +0200)]
tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Christian Ebner [Thu, 6 Jun 2019 11:37:07 +0000 (13:37 +0200)]
pxar: introduce fs_feature_flags obtained from filesystem magic in Encoder
Not all filesystems support features such as xattrs,acl,... and trying to get
them is rather expensive.
By getting the supported features based on the filesystem magic and masking the
user set feature flags, unsupported features are excluded rather inexpensively
while encoding the archive.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 6 Jun 2019 10:28:25 +0000 (12:28 +0200)]
src/pxar/encoder.rs: refactoring: use ioctl calls from src/tools/fs.rs instead of local
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Wed, 5 Jun 2019 12:37:25 +0000 (14:37 +0200)]
src/pxar/encoder.rs: minor refactoring
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Wed, 5 Jun 2019 12:26:16 +0000 (14:26 +0200)]
src/pxar/encoder.rs: check if ioctl supported by filesystem
Reading the quota project id relies on a ioctl call to get fsxattr.
On FUSE filesystems, ioctl calls might not be supported and will fail with
an errno indicating no support.
For these cases, the error is ignored and the default project id is used
(indicated by returning Ok(None)).
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Fri, 7 Jun 2019 05:52:11 +0000 (07:52 +0200)]
src/api2/admin/datastore.rs: remove test-upload api
Dietmar Maurer [Fri, 7 Jun 2019 05:39:41 +0000 (07:39 +0200)]
src/api2/admin/datastore/upload.rs: remove unused upload api
Dietmar Maurer [Wed, 5 Jun 2019 09:41:50 +0000 (11:41 +0200)]
src/client/http_client.rs: automatically close connection in finish
Wolfgang Bumiller [Wed, 5 Jun 2019 07:45:59 +0000 (09:45 +0200)]
make h2 client connection cancellable
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 5 Jun 2019 07:44:43 +0000 (09:44 +0200)]
bump zstd-sys version
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dietmar Maurer [Wed, 5 Jun 2019 06:41:20 +0000 (08:41 +0200)]
src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
So that we can include it in static doc strings.
Dietmar Maurer [Wed, 5 Jun 2019 06:12:13 +0000 (08:12 +0200)]
src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1
Dietmar Maurer [Wed, 5 Jun 2019 05:57:42 +0000 (07:57 +0200)]
src/tools/xattr.rs: fix test when run as root
Dietmar Maurer [Wed, 5 Jun 2019 05:23:21 +0000 (07:23 +0200)]
src/api2.rs: move backup api to /backup
Christian Ebner [Wed, 29 May 2019 13:43:34 +0000 (15:43 +0200)]
src/bin/proxmox-backup-proxy.rs: fix typo in pkcs12 error message
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Wed, 5 Jun 2019 04:27:39 +0000 (06:27 +0200)]
src/pxar/encoder.rs: style fix
Christian Ebner [Wed, 29 May 2019 12:34:05 +0000 (14:34 +0200)]
pxar: impl storing/dumping/restoring of quota project ids
Allows to store/dump/restore the quota project id associated with an inode in
order to correctly restore project quotas.
The project id is obtained/set via ioctl calls getting/setting the fsxattr
associated with the given file descriptor.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Wed, 29 May 2019 12:34:04 +0000 (14:34 +0200)]
src/pxar/format_definition.rs: add CA_FORMAT_QUOTA_PROJID and CaFormatQuotaProjID
Defines the types needed to store/restore quota project ids in pxar
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Wed, 29 May 2019 12:34:03 +0000 (14:34 +0200)]
src/tools/fs.rs: impl ioctl calls to get/set fsxattr
This implements fs_ioc_fsgetxattr/fs_ioc_fssetxattr calls in order to read or
write fsxattr for a given file descriptor.
This is needed in order to read or write the quota project id for filesystems
which support project quotas (EXT4/XFS/FUSE).
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Tue, 4 Jun 2019 15:33:12 +0000 (17:33 +0200)]
docs/backup-protocol.rst: add backup protocol docs
Dietmar Maurer [Tue, 4 Jun 2019 11:12:42 +0000 (13:12 +0200)]
src/bin/dump-backup-api.rs: helper to generate backup API docs
Dietmar Maurer [Tue, 4 Jun 2019 10:42:02 +0000 (12:42 +0200)]
src/api_schema/format.rs: depend on textwrap, new wrap_text helper
Dietmar Maurer [Tue, 4 Jun 2019 06:24:50 +0000 (08:24 +0200)]
src/cli/command.rs: move doc generator code to src/api_schema/format.rs
Dietmar Maurer [Mon, 3 Jun 2019 08:53:09 +0000 (10:53 +0200)]
src/client/http_client.rs - upload_config: improve error message
Dietmar Maurer [Mon, 3 Jun 2019 08:39:44 +0000 (10:39 +0200)]
src/bin/proxmox-backup-client.rs: implement upload_config
Dietmar Maurer [Mon, 3 Jun 2019 05:46:49 +0000 (07:46 +0200)]
src/api2/admin/datastore/backup.rs: implement config file upload
Dietmar Maurer [Mon, 3 Jun 2019 04:43:47 +0000 (06:43 +0200)]
avoid compiler warnings
Dietmar Maurer [Thu, 30 May 2019 11:28:24 +0000 (13:28 +0200)]
src/backup/chunk_stream.rs: add optional chunk_size parameter
Dietmar Maurer [Thu, 30 May 2019 10:52:12 +0000 (12:52 +0200)]
src/bin/proxmox-backup-client.rs: backup_image() - pass correct chunk size
Dietmar Maurer [Thu, 30 May 2019 10:47:35 +0000 (12:47 +0200)]
src/bin/proxmox-backup-client.rs: remove unused code
Dietmar Maurer [Thu, 30 May 2019 10:46:01 +0000 (12:46 +0200)]
src/bin/proxmox-backup-client.rs: implement image/block-device upload
Dietmar Maurer [Thu, 30 May 2019 10:19:23 +0000 (12:19 +0200)]
src/bin/proxmox-backup-client.rs: use std lib instead of nix
Dietmar Maurer [Thu, 30 May 2019 10:08:34 +0000 (12:08 +0200)]
src/bin/proxmox-backup-client.rs: fix file type test
Dietmar Maurer [Thu, 30 May 2019 07:20:32 +0000 (09:20 +0200)]
src/api2/admin/datastore/backup/environment.rs: log upload statistics
Dietmar Maurer [Thu, 30 May 2019 06:10:06 +0000 (08:10 +0200)]
src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload
So that we can provide better statistics (patches follows).
Wolfgang Bumiller [Wed, 29 May 2019 09:56:52 +0000 (11:56 +0200)]
typo fix and doc improvement
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 29 May 2019 09:49:54 +0000 (11:49 +0200)]
doc fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 29 May 2019 08:27:53 +0000 (10:27 +0200)]
tools: add Cancellable, start a futures submodule
To make a future cancellable, use:
let (future, canceller) =
crate::tools::futures::cancellable(future);
Proceed with using `future` as usual, `canceller` is
clonable and can cancel the future via the `.cancel()`
method.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 29 May 2019 08:27:24 +0000 (10:27 +0200)]
tools: add AsyncMutex::new_locked
Allows creating a pre-locked mutex, returning the mutex and
a guard.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dietmar Maurer [Wed, 29 May 2019 08:38:57 +0000 (10:38 +0200)]
src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files)
Dietmar Maurer [Wed, 29 May 2019 08:17:38 +0000 (10:17 +0200)]
src/api2/admin/datastore/backup.rs: improve debug logging
Dietmar Maurer [Wed, 29 May 2019 07:42:16 +0000 (09:42 +0200)]
src/api_schema/router.rs: disable debug println
Dietmar Maurer [Wed, 29 May 2019 07:35:21 +0000 (09:35 +0200)]
src/api2/admin/datastore/backup.rs: add debug flag
Dietmar Maurer [Wed, 29 May 2019 06:49:57 +0000 (08:49 +0200)]
src/backup/dynamic_index.rs: split class DynamicIndexWriter
Dietmar Maurer [Wed, 29 May 2019 05:08:34 +0000 (07:08 +0200)]
src/backup/fixed_index.rs: remove ChunkStat from struct
Dietmar Maurer [Wed, 29 May 2019 05:02:45 +0000 (07:02 +0200)]
src/backup/fixed_index.rs: improve consistency checks
Christian Ebner [Tue, 28 May 2019 09:16:21 +0000 (11:16 +0200)]
pxar: change logic and impl all command line flags for xattrs/fcaps/acls
Allows to individually set the flags for storing/dumping/restoring of
xattrs/fcaps/acls in the cli of pxar.
Changes logic so that each of them can be threated individually.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Tue, 28 May 2019 10:55:22 +0000 (12:55 +0200)]
src/api2/admin/datastore/backup.rs: log sucessful finish call
Dietmar Maurer [Tue, 28 May 2019 08:12:44 +0000 (10:12 +0200)]
src/bin/proxmox-backup-client.rs: use new BackupClient to upload data
Dietmar Maurer [Tue, 28 May 2019 07:35:08 +0000 (09:35 +0200)]
src/tools.rs: implement image_size helper
Dietmar Maurer [Tue, 28 May 2019 07:12:38 +0000 (09:12 +0200)]
src/api2/admin/datastore/backup/environment.rs: add more consistency checks
Dietmar Maurer [Tue, 28 May 2019 07:01:01 +0000 (09:01 +0200)]
src/client/http_client.rs: use start of chunk as offset
Dietmar Maurer [Tue, 28 May 2019 06:53:27 +0000 (08:53 +0200)]
src/client/http_client.rs: code cleanup
Dietmar Maurer [Tue, 28 May 2019 06:47:03 +0000 (08:47 +0200)]
src/backup/fixed_index.rs: improve bound checks inside add_digest
Dietmar Maurer [Tue, 28 May 2019 06:27:57 +0000 (08:27 +0200)]
src/backup/fixed_index.rs: store index_length to avoid code duplication
Dietmar Maurer [Tue, 28 May 2019 05:13:19 +0000 (07:13 +0200)]
src/api2/admin/datastore/backup.rs: fix typo
Christian Ebner [Mon, 27 May 2019 12:16:13 +0000 (14:16 +0200)]
src/bin/pxar.rs: set archive permissions to 0o640
As the archive can contain potentially sensitive data such as key files, it makes
sense to restrict the permissions.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 27 May 2019 11:59:17 +0000 (13:59 +0200)]
src/pxar/encoder.rs: Refactor file stat
Introduce helper functions to check file stats
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 27 May 2019 11:01:38 +0000 (13:01 +0200)]
src/pxar/encoder.rs: Don't bail if endpoint does not support xattrs.
The encoder bailed if a endpoint which did not support xattrs was encountered.
Instead of bailing, we ignore these errors and simply do not store xattrs for
such endpoints.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Tue, 28 May 2019 04:18:55 +0000 (06:18 +0200)]
src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api
Dietmar Maurer [Tue, 28 May 2019 04:18:24 +0000 (06:18 +0200)]
src/backup/fixed_index.rs: make chunk_size public
Dietmar Maurer [Mon, 27 May 2019 07:35:30 +0000 (09:35 +0200)]
src/client/http_client.rs: correctly parse error response in download_chunk_list
Dietmar Maurer [Mon, 27 May 2019 05:57:43 +0000 (07:57 +0200)]
src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk
Dietmar Maurer [Mon, 27 May 2019 05:40:50 +0000 (07:40 +0200)]
src/api2/admin/datastore/backup.rs: use smaller window sizes
Dietmar Maurer [Mon, 27 May 2019 05:24:32 +0000 (07:24 +0200)]
src/client/http_client.rs: fix request pipelining
Dietmar Maurer [Sun, 26 May 2019 08:52:56 +0000 (10:52 +0200)]
src/client/merge_known_chunks.rs: use MergedChunkInfo as input
Dietmar Maurer [Sun, 26 May 2019 07:57:34 +0000 (09:57 +0200)]
src/client/http_client.rs: append chunks in correct order
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
Dietmar Maurer [Fri, 24 May 2019 08:05:22 +0000 (10:05 +0200)]
src/api2/admin/datastore/backup.rs: verify chunk offset
This clearly shows that the current approach does not work - seems we get
chunks out of order.
Dietmar Maurer [Fri, 24 May 2019 07:10:59 +0000 (09:10 +0200)]
src/server/rest.rs: correctly verify json parameters
Dietmar Maurer [Fri, 24 May 2019 06:32:55 +0000 (08:32 +0200)]
src/client/http_client.rs: encode parameters inside body
To avoid problems with long uris.