]>
git.proxmox.com Git - proxmox-backup.git/log
Christian Ebner [Tue, 5 Nov 2019 12:45:04 +0000 (13:45 +0100)]
src/pxar/decoder.rs: allow to pass match pattern to restore
In order to partially restore the archive starting from the directory provided as
parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 5 Nov 2019 12:45:03 +0000 (13:45 +0100)]
src/pxar/decoder.rs: fix issue with restore
`Decoder::restore()` calls the `SequentialDecoder::restore()` which expects to
encounter a `PxarEntry` at first. But the start of `DirectoryEntry` points to the
filename (except for the root dir), so skip over it.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 4 Nov 2019 09:48:09 +0000 (10:48 +0100)]
src/pxar/fuse.rs: refactor Iterator: use find() instead of position()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Wed, 30 Oct 2019 13:38:05 +0000 (14:38 +0100)]
catalog: store current writer position in CatalogBlobWriter
Counts the bytes written by the CatalogBlobWriter in order to obtain the
stream position, needed to get offset to reference catalog items.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Sat, 26 Oct 2019 09:36:01 +0000 (11:36 +0200)]
avoid some clippy warnings
Dietmar Maurer [Fri, 25 Oct 2019 16:44:51 +0000 (18:44 +0200)]
avoid some clippy warnings
Dietmar Maurer [Fri, 25 Oct 2019 16:22:19 +0000 (18:22 +0200)]
src/pxar/binary_search_tree.rs: allow(clippy::many_single_char_names)
Dietmar Maurer [Fri, 25 Oct 2019 16:04:37 +0000 (18:04 +0200)]
avoid some clippy warnings
Dietmar Maurer [Fri, 25 Oct 2019 16:03:55 +0000 (18:03 +0200)]
allow(clippy::cast_ptr_alignment)
Dietmar Maurer [Fri, 25 Oct 2019 15:22:41 +0000 (17:22 +0200)]
src/backup/dynamic_index.rs: use write_all()
Christian Ebner [Thu, 17 Oct 2019 15:12:22 +0000 (17:12 +0200)]
pxar: decoder: cleanup un-needed Decoder::get_dir()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 17 Oct 2019 15:04:26 +0000 (17:04 +0200)]
pxar: fuse: optimize fuse readdir callback
The expensive call to Decoder::read_directory_entry() can be omitted as
Decoder::attributes() returns all the information the fuse response needs.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Wed, 23 Oct 2019 05:55:56 +0000 (07:55 +0200)]
src/pxar/encoder.rs: avoid duplicate check for MAX_DIRECTORY_ENTRIES
Simply by reordering code.
Christian Ebner [Tue, 22 Oct 2019 13:07:51 +0000 (15:07 +0200)]
pxar: add functionality to pass exclude MatchPatterns on create
This exposes the option to pass a list of exclude MatchPattern via the
'--exclude' option.
The list is encoded as file '.pxarexclude-cli' in the archives root directory.
If such a file is present in the filesystem, it is skipped and not included in
the archive in order to avoid conflicting information.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 22 Oct 2019 13:07:50 +0000 (15:07 +0200)]
pxar: encoder: add encode_pxar_exclude_cli() function.
This provides the functionality needed to encode MatchPatterns passed on the cli
in the root directory.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Wed, 23 Oct 2019 05:29:02 +0000 (07:29 +0200)]
pxar: simplify MatchPattern::to_bytes
Christian Ebner [Tue, 22 Oct 2019 13:07:49 +0000 (15:07 +0200)]
pxar: Implement to_bytes() for MatchPattern in order to write them to file.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Thu, 17 Oct 2019 08:28:21 +0000 (10:28 +0200)]
src/bin/proxmox-backup-client.rs: fix parameters for get_encryption_key_password
Christian Ebner [Wed, 16 Oct 2019 16:06:05 +0000 (18:06 +0200)]
src/pxar/encoder.rs: include .pxarexclude files into the catalog
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Wed, 16 Oct 2019 15:13:58 +0000 (17:13 +0200)]
pxar: fuse: include nsec information in stat
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Wed, 16 Oct 2019 11:23:52 +0000 (13:23 +0200)]
load_and_decrtypt_key: allow function closures to readf passphrase
Dietmar Maurer [Mon, 14 Oct 2019 10:28:27 +0000 (12:28 +0200)]
src/backup/manifest.rs: fix property names (use '-' instead of '_')
Dietmar Maurer [Mon, 14 Oct 2019 08:58:26 +0000 (10:58 +0200)]
src/backup/data_blob.rs: define create_zero_chunk as member function
and rename it to build_zero_chunk()
Dietmar Maurer [Mon, 14 Oct 2019 08:44:46 +0000 (10:44 +0200)]
src/backup/data_blob.rs: add create_zero_chunk() helper
Dietmar Maurer [Sun, 13 Oct 2019 08:09:12 +0000 (10:09 +0200)]
src/backup/manifest.rs: add verify_file
Dietmar Maurer [Sun, 13 Oct 2019 07:39:21 +0000 (09:39 +0200)]
cleanup
Dietmar Maurer [Sun, 13 Oct 2019 06:39:49 +0000 (08:39 +0200)]
src/backup/manifest.rs: improve parser (try_from)
Dietmar Maurer [Sat, 12 Oct 2019 15:58:08 +0000 (17:58 +0200)]
src/backup/manifest.rs: new class to generate/parse index.json
Dietmar Maurer [Sat, 12 Oct 2019 13:53:27 +0000 (15:53 +0200)]
rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME
Dietmar Maurer [Sat, 12 Oct 2019 13:50:26 +0000 (15:50 +0200)]
src/client/backup_reader.rs: add download_manifest
Also add crypt_config as member variable.
Dietmar Maurer [Sat, 12 Oct 2019 12:08:33 +0000 (14:08 +0200)]
set toolchain to beta
Dietmar Maurer [Sat, 12 Oct 2019 11:53:11 +0000 (13:53 +0200)]
src/client/backup_writer.rs: move BackupClient code into extra file
And rename it to BackupWriter.
Dietmar Maurer [Sat, 12 Oct 2019 11:26:47 +0000 (13:26 +0200)]
src/client/backup_reader.rs: add documentation
Dietmar Maurer [Sat, 12 Oct 2019 10:57:08 +0000 (12:57 +0200)]
src/client/backup_reader.rs: split BackupReader code into separate file
Dietmar Maurer [Tue, 8 Oct 2019 11:04:10 +0000 (13:04 +0200)]
src/bin/proxmox-backup-client.rs: pass verbose flag to dump_image
And print some useful status to stderr.
Dietmar Maurer [Mon, 7 Oct 2019 10:48:56 +0000 (12:48 +0200)]
fix typo in error message
Dietmar Maurer [Mon, 7 Oct 2019 10:24:06 +0000 (12:24 +0200)]
src/api2/reader.rs: use tokio::fs::read() to read file async
Dietmar Maurer [Mon, 7 Oct 2019 09:36:39 +0000 (11:36 +0200)]
src/backup/data_blob.rs: avoid Arc<CryptConfig>
use simple reference instead.
Dietmar Maurer [Sun, 6 Oct 2019 08:31:06 +0000 (10:31 +0200)]
remove DataChunk file format - use DataBlob instead
Dietmar Maurer [Sat, 5 Oct 2019 14:10:41 +0000 (16:10 +0200)]
cleanup: remove dead code
Dietmar Maurer [Sat, 5 Oct 2019 09:48:51 +0000 (11:48 +0200)]
src/bin/proxmox-backup-client.rs: cleanup restore
- write directly to /dev/stdout (instead of using stdout line writer)
- avoid BufferedFixedReader
Dietmar Maurer [Sat, 5 Oct 2019 09:45:20 +0000 (11:45 +0200)]
src/client/remote_chunk_reader.rs: pass reference to download_chunk
Make sure we do not move data around.
Dietmar Maurer [Sat, 5 Oct 2019 09:41:19 +0000 (11:41 +0200)]
src/api2/reader.rs - download_chunk: use blocking io
Turns out to be twice as fast as tokio async io...
Dietmar Maurer [Wed, 2 Oct 2019 08:47:20 +0000 (10:47 +0200)]
src/client/http_client.rs: store raw data/blob size in index.json
Dietmar Maurer [Sat, 28 Sep 2019 16:22:48 +0000 (18:22 +0200)]
src/client/http_client.rs: new helper send_upload_request()
Dietmar Maurer [Mon, 23 Sep 2019 09:30:20 +0000 (11:30 +0200)]
src/backup/fixed_index.rs: do not use offset for checksum
Dietmar Maurer [Mon, 23 Sep 2019 08:56:53 +0000 (10:56 +0200)]
backup writer api: verify checksum inside close
Dietmar Maurer [Fri, 20 Sep 2019 11:19:55 +0000 (13:19 +0200)]
src/client/http_client.rs - H2Client::upload: fix parameter order
Dietmar Maurer [Fri, 20 Sep 2019 10:40:23 +0000 (12:40 +0200)]
src/client/http_client.rs: add method parameter to H2 upload
Dietmar Maurer [Fri, 20 Sep 2019 10:23:06 +0000 (12:23 +0200)]
src/client/http_client.rs: add content type parameter to H2 upload/requets_builder
Dietmar Maurer [Fri, 20 Sep 2019 09:57:03 +0000 (11:57 +0200)]
src/client/http_client.rs: cleanup (rename parameter)
Dietmar Maurer [Fri, 20 Sep 2019 06:12:31 +0000 (08:12 +0200)]
remove trailing whitespace
Christian Ebner [Thu, 19 Sep 2019 14:33:09 +0000 (16:33 +0200)]
src/bin/proxmox-backup-client.rs: add functionality to mount pxar archive.
This adds the basic functionality to mount an archive stored in a repository
locally via fuse.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 19 Sep 2019 14:33:08 +0000 (16:33 +0200)]
src/bin/pxar.rs: adapt to new deamonize flag
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 19 Sep 2019 14:33:07 +0000 (16:33 +0200)]
src/pxar/fuse.rs: make verbose flag independent from deamonize by adding this as new flag
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 19 Sep 2019 14:33:06 +0000 (16:33 +0200)]
src/pxar/fuse.rs: refactor Session::new()
and add Session::from_decoder() in order to be able to create a fuse session
with a `Decoder` given as argument.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 19 Sep 2019 12:22:42 +0000 (14:22 +0200)]
src/pxar/fuse.rs: introduce `Context` in order to add caching.
This patch introduces `Context` to hold the decoder, ino_offset and caches for
the attributes and the goodbye table.
By caching, certain callbacks can be handled without the need to read additional
data via the decoder, which improves performance.
The searching of the goodbye table is refactored as well, avoiding recursive
function calls in case of a hash collision.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 16 Sep 2019 14:59:31 +0000 (16:59 +0200)]
src/pxar/decoder.rs: fix bug in decoder read
`offset` points to the `PXAR_FILENAME`, therefore read the filename before
the `PXAR_ENTRY`.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Thu, 19 Sep 2019 12:24:17 +0000 (14:24 +0200)]
src/backup/data_chunk.rs: add into_raw()
Dietmar Maurer [Thu, 19 Sep 2019 11:59:24 +0000 (13:59 +0200)]
BackupClient: add upload() function
Dietmar Maurer [Thu, 19 Sep 2019 11:17:23 +0000 (13:17 +0200)]
make download_chunk_list() public
Dietmar Maurer [Mon, 16 Sep 2019 10:35:23 +0000 (12:35 +0200)]
src/client/http_client.rs: add password parameter to HttpClient::new()
Christian Ebner [Wed, 11 Sep 2019 15:25:03 +0000 (17:25 +0200)]
pxar: fuse: avoid possible hash collision in lookup by additional checking against filename
The hash of the filename in the goodbye table items allows to quickly compare to
a hashed filename.
Unfortunately, a matching hash is no garantee for matching filenames as hash
collisions are possible.
This patch fixes such possible collisions by further checking the filenames once
a matching hash has been found.
This introduces no significant extra cost (except for the filename comparison)
for cases with matching hashes, as the lookup call has to seek and read the file
attributes (including the filename) anyway.
In cases with hash collision, the next matching item is read and treaded
analogously (what means we need at least one extra seek).
As collisions should be not that frequent, this should be an acceptable penalty.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 12:50:45 +0000 (14:50 +0200)]
clippy: fix access to unaligned buffer in dynamic reader
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 12:46:46 +0000 (14:46 +0200)]
backup: cleanup unused imports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 12:45:43 +0000 (14:45 +0200)]
clippy: fix access to unaligned buffer
via the newly introduced ReadExt::read_host_value_boxed()
from proxmox::tools::io.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 12:43:25 +0000 (14:43 +0200)]
backup: use static assertion in fixed index reader
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 12:02:30 +0000 (14:02 +0200)]
clippy: don't use clone a double reference
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 11:59:56 +0000 (13:59 +0200)]
buildsys: add 'make lint'
for now we only activate lints for code that is "outright
wrong"
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 11:57:58 +0000 (13:57 +0200)]
clippy: collapse identical if branches
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 11:57:21 +0000 (13:57 +0200)]
fixup variable naming
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 11:56:09 +0000 (13:56 +0200)]
clippy: use write_all in file logger
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 11:55:56 +0000 (13:55 +0200)]
clippy: fix pointer to temporary value
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 10:31:39 +0000 (12:31 +0200)]
clippy: remove a loop{} which never actually loops
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 10:23:11 +0000 (12:23 +0200)]
more clippy fixups
mostly indentation changes, view with `-w`
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 10:06:59 +0000 (12:06 +0200)]
some clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Wed, 11 Sep 2019 10:07:33 +0000 (12:07 +0200)]
remove Cargo.lock
we currently have no git dependencies we need to lock into
specific revisions anymore
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Christian Ebner [Tue, 10 Sep 2019 15:33:15 +0000 (17:33 +0200)]
src/pxar/fuse.rs: s/Buf/ReplyBuf/ and s/BufState/ReplyBufState/
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 10 Sep 2019 15:14:27 +0000 (17:14 +0200)]
src/pxar/fuse.rs: refactor stat and fix i-node mapping
The functionality of stat is split into smaller sub-functions, which allows
to reuse them more flexible, as the code flow is similar but not always the same.
By this, the ugly and incorrect re-setting of the i-node in the lookup callback
function is avoided.
The correct i-node is now calculated beforehand and stat simply creates a
`libc::stat` struct from the provided parameters.
Also, this fixes incorrect i-node assignments in the readdir callback function.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:32 +0000 (18:27 +0200)]
src/pxar/fuse.rs: impl readdir callback for fuse
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:31 +0000 (18:27 +0200)]
src/pxar/fuse.rs: impl readlink callback for fuse
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:30 +0000 (18:27 +0200)]
src/pxar/decoder.rs: read_link has to read filename before entry.
The offset points to the start of the item, which is the filename
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:29 +0000 (18:27 +0200)]
src/pxar/fuse.rs: impl opendir callback for fuse
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:28 +0000 (18:27 +0200)]
src/pxar/fuse.rs: impl read callback for fuse
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:27 +0000 (18:27 +0200)]
src/pxar/fuse.rs: impl open callback for fuse
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:26 +0000 (18:27 +0200)]
src/pxar/fuse.rs: impl lookup callback for fuse
The lookup call checks if the given filename is found in the directory referenced
by the i-node by calclulating the filenames hash and looking it up in the
directories goodbye table.
If found, the entries parameters are returned.
In order to be able to lookup the parent offset by a given file offset in the
readdir callback, this also stores the corresponding values in a HashMap.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:25 +0000 (18:27 +0200)]
src/pxar/decoder.rs: correctly handle hardlinks in `Decoder`s attribute function
If the attributes are then returned by seeking to the new offset.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:24 +0000 (18:27 +0200)]
src/pxar/decoder.rs: split functionality of list_dir into list_dir and goodbye_table
In order to read the contents of the goodbye table while keeping the
functionality of list_dir in place as is.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:23 +0000 (18:27 +0200)]
src/pxar/decoder.rs: make read_directory_entry usable by fuse
... and thereby allow it to read a single directory entry based on the
start and end archive offsets.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:22 +0000 (18:27 +0200)]
src/pxar/decoder.rs: remove `DirectoryEntry` start_offset and end_offset functions
... as they are not needed with the latest iteration of the fuse callback
function impl (which never made it into the repos).
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 9 Sep 2019 16:27:21 +0000 (18:27 +0200)]
src/pxar/decoder.rs: correctly read, check and skip filename
The previous implementation simply skipped over `size` bytes, which is not
correct as size includes also the header.
By relying on `SequentailDecoder`s read_filename function, this is correctly
taken care of plus some more integrity checks.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Mon, 9 Sep 2019 08:51:08 +0000 (10:51 +0200)]
avoid injecting ENV vars from Makefile
So that we can run "cargo build" without setting vars manually.
Christian Ebner [Fri, 6 Sep 2019 09:38:28 +0000 (11:38 +0200)]
src/pxar/fuse.rs: impl getattr callback for fuse
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Fri, 6 Sep 2019 09:38:27 +0000 (11:38 +0200)]
src/pxar/fuse.rs: Refactor run_in_context and remove inode_to_offset and offset_to_inode
as their functionality is used only once so it makes more sense to keep them
inline.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Fri, 6 Sep 2019 09:38:26 +0000 (11:38 +0200)]
src/pxar/fuse.rs: Improve comments, add rustfmt skip tag and reafctor
Also, removes an unused println statement in the decoder callback function and
fixes a typo.
Further, use ABI compatible Option<&T> for FFI to avoid use of raw pointers.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Wolfgang Bumiller [Fri, 6 Sep 2019 08:12:24 +0000 (10:12 +0200)]
update hyper & h2 to alpha releases on crates.io
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dietmar Maurer [Fri, 6 Sep 2019 06:55:47 +0000 (08:55 +0200)]
src/client/http_client.rs: use async for upload_speedtest()
Christian Ebner [Thu, 5 Sep 2019 15:27:58 +0000 (17:27 +0200)]
src/pxar/decoder.rs: impl functionality needed for fuse implementation
Implements functions attributes, open, read, read_link and get_dir
to be used by the fuse implementation which uses file offsets within the archive
as inodes to reference the archives items.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 5 Sep 2019 11:17:40 +0000 (13:17 +0200)]
src/pxar/decoder.rs: Add comment for unsolved issue and refactor if statement.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Thu, 5 Sep 2019 13:07:37 +0000 (15:07 +0200)]
src/client/http_client.rs: simplify send_request