]>
git.proxmox.com Git - proxmox-backup.git/log
Dietmar Maurer [Thu, 25 Jul 2019 11:44:01 +0000 (13:44 +0200)]
src/api2/backup.rs: new required backup-time parameter
The client should pass the time as parameter.
Dietmar Maurer [Thu, 25 Jul 2019 10:23:33 +0000 (12:23 +0200)]
src/api2/admin/datastore.rs: backup logs may not be written twice
Dietmar Maurer [Thu, 25 Jul 2019 10:22:50 +0000 (12:22 +0200)]
src/api2/admin/datastore.rs: use correct .blob extension for log
Dietmar Maurer [Thu, 25 Jul 2019 10:18:15 +0000 (12:18 +0200)]
src/bin/proxmox-backup-client.rs: implement upload-log
Dietmar Maurer [Thu, 25 Jul 2019 10:17:35 +0000 (12:17 +0200)]
src/client/http_client.rs: allow upload with parameters
Dietmar Maurer [Thu, 25 Jul 2019 08:12:45 +0000 (10:12 +0200)]
src/api2/admin/datastore.rs: add api to upload backup client log file
Dietmar Maurer [Wed, 24 Jul 2019 11:07:02 +0000 (13:07 +0200)]
src/bin/proxmox-backup-client.rs: allow to uploag .log files (as binary blobs)
Dietmar Maurer [Wed, 24 Jul 2019 10:21:25 +0000 (12:21 +0200)]
src/pxar/encoder.rs: add new skip_lost_and_found parameter
Dietmar Maurer [Wed, 24 Jul 2019 09:30:43 +0000 (11:30 +0200)]
pxar_backup_stream.rs: limit lock scope to avoid blocking forever
Dietmar Maurer [Wed, 24 Jul 2019 07:33:52 +0000 (09:33 +0200)]
remove debug output
Dietmar Maurer [Wed, 24 Jul 2019 07:24:35 +0000 (09:24 +0200)]
src/client/pxar_backup_stream.rs: correctly pass errors to stream
Dietmar Maurer [Wed, 24 Jul 2019 05:48:59 +0000 (07:48 +0200)]
src/pxar/encoder.rs: allow to pass list of devices
For better mount point include control...
Dietmar Maurer [Mon, 22 Jul 2019 08:12:51 +0000 (10:12 +0200)]
Revert previous, commit, use UTC RFC3339 without timezone (Z)
We now have human readable data/time, and names are still sortable.
Dietmar Maurer [Mon, 22 Jul 2019 05:57:03 +0000 (07:57 +0200)]
src/backup/backup_info.rs: use unix epoch timestamps instead of RFC3339
RFC3339 contains additional timezone information, but IMHO this just adds
more confusion. The API also uses timestamps.
Christian Ebner [Fri, 19 Jul 2019 08:55:28 +0000 (10:55 +0200)]
src/pxar/sequential_decoder.rs: Call callback only on restored paths.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Fri, 19 Jul 2019 05:01:57 +0000 (07:01 +0200)]
src/bin/pxar.rs - extract: print archive name with --verbose flag
Dietmar Maurer [Fri, 19 Jul 2019 05:00:17 +0000 (07:00 +0200)]
src/bin/pxar.rs: remove wrong debug message
Christian Ebner [Thu, 18 Jul 2019 09:35:51 +0000 (11:35 +0200)]
src/bin/pxar.rs: allow to pass paths and match patterns as args to pxar extract
To improve usability it is now possible to directly pass paths or match patterns
as arguments to pxar extract to partially restore an archive.
The patterns provided via CLI are appended to the ones read from file by the
--files-from option in order to have priority over those.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Thu, 18 Jul 2019 11:16:32 +0000 (13:16 +0200)]
src/bin/proxmox-backup-client.rs: code cleanup
Dietmar Maurer [Thu, 18 Jul 2019 07:52:11 +0000 (09:52 +0200)]
src/bin/proxmox-backup-client.rs - status: only print result[data]
Dietmar Maurer [Thu, 18 Jul 2019 07:11:43 +0000 (09:11 +0200)]
src/api2/admin/datastore.rs: allow to list all snapshots
Dietmar Maurer [Wed, 17 Jul 2019 11:40:10 +0000 (13:40 +0200)]
src/cli/command.rs: set exit status on error
Christian Ebner [Tue, 16 Jul 2019 16:19:45 +0000 (18:19 +0200)]
src/bin/pxar.rs: add --files-from parameter to pxar extract
Allows to pass a list of match pattern to the pxar extract call by reading them
from file.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 16:19:44 +0000 (18:19 +0200)]
pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 16:19:43 +0000 (18:19 +0200)]
src/pxar/sequential_decoder.rs: adapt code for partial restore by match pattern
Allows to partially restore an archive by passing match patterns to the restore
function.
The whole restore is performed in sequential, therefore the whole archive has to
be read.
By wrapping the RawFd into an Option it can be controlled if the corresponding
part is restored (in case of Some(fd)) or if the Reader reads over it
without restore (in case of None).
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 16:19:42 +0000 (18:19 +0200)]
src/pxar/sequential_decoder.rs: implement match_filename to match include pattern for partial restores
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 16:19:41 +0000 (18:19 +0200)]
src/pxar/sequential_decoder.rs: Also check for nul bytes when reading the filename
Check if the filename does not contain invalid nul byes when reading it from the
archive.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 11:19:51 +0000 (13:19 +0200)]
tests/pxar.rs: Change rsync options to --archive
By this the pxar integrity test checks for even more attributes such as a e.g.
correctly restored timestamp. See `man rsync` for details.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 11:19:50 +0000 (13:19 +0200)]
src/pxar/sequential_decoder.rs: remove old comments and refactoring
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 11:19:49 +0000 (13:19 +0200)]
src/pxar/encoder.rs: refactoring of match_exclude_pattern()
match_exclude_pattern() does not need a '&mut self' reference to the encoder,
move it therefore out of the impl.
Further, this patch contains some naming and formatting cosmetics.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 11:19:48 +0000 (13:19 +0200)]
src/pxar/encoder.rs: remove unneeded dir_count
dir_count was used to track the number of directory entries to store in the
archive and bail if the maximum is exceeded.
As the number of entries is equally obtained from the list of the filenames to
include, use that one instead.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 11:19:47 +0000 (13:19 +0200)]
src/pxar/exclude_pattern.rs: do not return partial matches for files
Partial matches make only sense for directories, files are always leafs of the
tree. Take this into account in order to avoid restoring of files which only
matched the front of a match pattern.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Tue, 16 Jul 2019 13:45:17 +0000 (15:45 +0200)]
src/bin/pxar.rs: Make pxar extract target optional
In order to improve usablity, the target on archive extraction will be the
current working directory by default.
A different target can be provided via the optional --target <PATH> parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Tue, 16 Jul 2019 11:35:25 +0000 (13:35 +0200)]
src/bin/proxmox-backup-client.rs: add output-fromat options, implement status api
Dietmar Maurer [Tue, 16 Jul 2019 11:34:38 +0000 (13:34 +0200)]
src/api2/admin/datastore.rs: add status api call
Dietmar Maurer [Tue, 16 Jul 2019 09:27:45 +0000 (11:27 +0200)]
src/cli/command.rs: add helpers/schema to format output
Dietmar Maurer [Tue, 16 Jul 2019 06:38:02 +0000 (08:38 +0200)]
src/bin/pxar.rs: only print archive name in verbose mode
Dietmar Maurer [Tue, 16 Jul 2019 06:31:40 +0000 (08:31 +0200)]
src/bin/pxar.rs: add --verbose parameter to list (replace dump)
Dietmar Maurer [Tue, 16 Jul 2019 05:30:04 +0000 (07:30 +0200)]
src/bin/proxmox-backup-client.rs: make repository parameter optional
And allow to read it from $ENV{PBS_REPOSITORY}
Dietmar Maurer [Mon, 15 Jul 2019 11:37:27 +0000 (13:37 +0200)]
Cargo.toml: update to latest hyper version
Which already includes our path.
Christian Ebner [Wed, 10 Jul 2019 14:41:01 +0000 (16:41 +0200)]
src/pxar/format_definition.rs: fix typo
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Fri, 5 Jul 2019 10:39:25 +0000 (12:39 +0200)]
src/bin/pxar.rs - extract: allow to read from stdin
Dietmar Maurer [Fri, 5 Jul 2019 10:32:15 +0000 (12:32 +0200)]
src/bin/pxar.rs - dump: allow to read from stdin
Dietmar Maurer [Fri, 5 Jul 2019 10:14:50 +0000 (12:14 +0200)]
src/bin/proxmox-backup-client.rs - restore: allow to pipe output to stdout
Dietmar Maurer [Fri, 5 Jul 2019 09:36:45 +0000 (11:36 +0200)]
src/bin/proxmox-backup-client.rs: remove download cli, do everything in restore
Dietmar Maurer [Fri, 5 Jul 2019 08:42:46 +0000 (10:42 +0200)]
src/client/remote_chunk_reader.rs: implement simple caching
Dietmar Maurer [Fri, 5 Jul 2019 07:19:56 +0000 (09:19 +0200)]
src/backup/fixed_index.rs: implement BufferedFixedReader
Christian Ebner [Thu, 4 Jul 2019 14:15:54 +0000 (16:15 +0200)]
src/pxar/sequential_decoder.rs: major refactoring of restore_sequential()
This splits the functionality of restore_sequential() into several smaller
functions in order to allow to reuse them when restoring by seeking based on
the goodbye table offsets.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 4 Jul 2019 12:24:13 +0000 (14:24 +0200)]
src/pxar/encoder.rs: fix bug with incorrect offset stored in goodbye table
Commit
cd7dc87903a3c45e3d1521c3e7c1ea1500617a23 introduced the special treatment
for .pxarexclude files when stored in the archive.
The incorrect placement of a code snipplet from this path leads to an incorrect
offset and size stored in the goodbye table.
This fix places the start to the correct position, restoring the previously
correct behaviour.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Thu, 4 Jul 2019 12:03:20 +0000 (14:03 +0200)]
pxar: Refactor SequentialDecoder to store the callback function within the struct
Reduces the number of arguments for the function calls within the decoder.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Thu, 4 Jul 2019 13:13:22 +0000 (15:13 +0200)]
src/backup/fixed_index.rs: use correct size
We need to consider that the last chunk may have smaller size.
Dietmar Maurer [Thu, 4 Jul 2019 13:11:23 +0000 (15:11 +0200)]
src/client/http_client.rs - finish: hold reference to avoid early connection close
Dietmar Maurer [Thu, 4 Jul 2019 11:40:43 +0000 (13:40 +0200)]
src/api2/backup/environment.rs: improve chunk alignment/size checks
Dietmar Maurer [Thu, 4 Jul 2019 09:39:10 +0000 (11:39 +0200)]
src/backup/chunk_store.rs: coding style fixes
Dietmar Maurer [Thu, 4 Jul 2019 09:27:11 +0000 (11:27 +0200)]
src/backup/chunk_store.rs: additionally log chunk count
Dietmar Maurer [Thu, 4 Jul 2019 09:21:54 +0000 (11:21 +0200)]
src/backup/chunk_store.rs: improve error reporting
Wolfgang Bumiller [Thu, 4 Jul 2019 08:51:52 +0000 (10:51 +0200)]
backup/chunk_store: rework chunk iterator
We can now use iter::from_fn() which makes for a much nicer
logic. The only thing better is going to be when we can use
generators with `yield`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dietmar Maurer [Thu, 4 Jul 2019 08:51:18 +0000 (10:51 +0200)]
src/client/http_client.rs: s/set_recv_buf_size/set_recv_buffer_size/
Dietmar Maurer [Thu, 4 Jul 2019 08:47:16 +0000 (10:47 +0200)]
src/tools/fs.rs - read_dir: return nix::Result
Dietmar Maurer [Thu, 4 Jul 2019 07:26:44 +0000 (09:26 +0200)]
src/backup/chunk_store.rs - get_chunk_iterator: return percentage inside iterator item
Dietmar Maurer [Thu, 4 Jul 2019 06:17:30 +0000 (08:17 +0200)]
src/backup/dynamic_index.rs: lock file inside new, code cleanup
Dietmar Maurer [Thu, 4 Jul 2019 06:09:48 +0000 (08:09 +0200)]
src/backup/fixed_index.rs - FixedIndexReader: remove reference to chunk store
Dietmar Maurer [Thu, 4 Jul 2019 05:57:43 +0000 (07:57 +0200)]
src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats
Dietmar Maurer [Wed, 3 Jul 2019 13:45:12 +0000 (15:45 +0200)]
src/bin/proxmox-backup-client.rs: implement restore using BackupReader
Dietmar Maurer [Wed, 3 Jul 2019 12:56:24 +0000 (14:56 +0200)]
src/client/remote_chunk_reader.rs: remove dead code
Dietmar Maurer [Wed, 3 Jul 2019 12:39:13 +0000 (14:39 +0200)]
src/client/remote_chunk_reader.rs: implement remote chunk reader
Dietmar Maurer [Wed, 3 Jul 2019 12:36:02 +0000 (14:36 +0200)]
src/client/http_client.rs: avoid compiler warning
Dietmar Maurer [Wed, 3 Jul 2019 12:26:07 +0000 (14:26 +0200)]
src/client/http_client.rs: return Arc<Self>
Avoid to call canceller for each clone on drop ...
Dietmar Maurer [Wed, 3 Jul 2019 12:25:18 +0000 (14:25 +0200)]
src/backup/read_chunk.rs: use &mut self
Christian Ebner [Mon, 1 Jul 2019 15:36:08 +0000 (17:36 +0200)]
src/pxar/encoder.rs: refactor simple match statements to map_err()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 1 Jul 2019 15:24:23 +0000 (17:24 +0200)]
src/pxar/sequential_decoder.rs: refactoring of simple match statements to map_err()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Christian Ebner [Mon, 1 Jul 2019 15:03:49 +0000 (17:03 +0200)]
src/pxar/sequential_decoder.rs: Cleanup and fix incorrect error message
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Dietmar Maurer [Wed, 3 Jul 2019 10:00:43 +0000 (12:00 +0200)]
src/server/rest.rs: avoid unwrap
Dietmar Maurer [Wed, 3 Jul 2019 09:54:35 +0000 (11:54 +0200)]
src/server/rest.rs: log peer address, use hyper MakeService
Dietmar Maurer [Tue, 2 Jul 2019 11:33:58 +0000 (13:33 +0200)]
use hyper/tokio-openssl instead of hyper/tokio-tls
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
Dietmar Maurer [Tue, 2 Jul 2019 08:48:58 +0000 (10:48 +0200)]
src/bin/proxmox-backup-proxy.rs: set socket options for max. performance
Dietmar Maurer [Tue, 2 Jul 2019 06:59:33 +0000 (08:59 +0200)]
src/bin/h2s-client.rs: avoid compiler warnings
Dietmar Maurer [Tue, 2 Jul 2019 06:56:56 +0000 (08:56 +0200)]
src/backup/{fixed, dynamic}_index.rs: improve error messages
Dietmar Maurer [Tue, 2 Jul 2019 06:49:16 +0000 (08:49 +0200)]
src/backup/read_chunk.rs: use Arc for DataStore
Dietmar Maurer [Tue, 2 Jul 2019 06:22:29 +0000 (08:22 +0200)]
src/backup/read_chunk.rs: move read chunk trait into extra file
And implement LocalChunkReader.
Dietmar Maurer [Tue, 2 Jul 2019 05:09:46 +0000 (07:09 +0200)]
use latest hyper master branch to include socket buffer size fix
Wolfgang Bumiller [Mon, 1 Jul 2019 09:03:25 +0000 (11:03 +0200)]
replace tools::vec with proxmox::tools::vec
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Mon, 1 Jul 2019 08:44:11 +0000 (10:44 +0200)]
tools: remove io module
now completely replaced by proxmox::tools::io.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Mon, 1 Jul 2019 08:44:03 +0000 (10:44 +0200)]
update remaining users of tools::io::ops
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Mon, 1 Jul 2019 08:39:41 +0000 (10:39 +0200)]
tools: remove read/write
now completely replaced by
proxmox::tools::io::{ReadExt, WriteExt}
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Mon, 1 Jul 2019 08:39:13 +0000 (10:39 +0200)]
daemon: remove last use of tools::read/write
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Wolfgang Bumiller [Mon, 1 Jul 2019 08:36:59 +0000 (10:36 +0200)]
backup: use new proxmox::tools::io traits
This replaces all `write_value` calls with `write_le_value`
calls!
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Dietmar Maurer [Sat, 29 Jun 2019 16:09:50 +0000 (18:09 +0200)]
src/bin/h2s-client.rs: increase tokio tcp stream read buffer size
This improves download speed by factor 10.
Dietmar Maurer [Sat, 29 Jun 2019 13:58:18 +0000 (15:58 +0200)]
src/bin/h2s-client.rs: avoid hyper, use h2 directly
But performance is still bad.
Dietmar Maurer [Sat, 29 Jun 2019 12:57:54 +0000 (14:57 +0200)]
src/bin/h2s-server.rs: test to reproduce slow h2 download
Simplified code, use with h2s-client.rs.
Dietmar Maurer [Sat, 29 Jun 2019 11:43:10 +0000 (13:43 +0200)]
src/bin/download-speed.rs: h2 download speedtest for our server
I get about 30MB/s (much too slow)
Dietmar Maurer [Sat, 29 Jun 2019 09:05:36 +0000 (11:05 +0200)]
src/client/http_client.rs: use maximal h2 window sizes
Dietmar Maurer [Sat, 29 Jun 2019 08:41:24 +0000 (10:41 +0200)]
src/bin/h2client.rs: tune h2 client parameters to get reasonable speed
We now get about 2.5GB/s (not really good, but better)
Dietmar Maurer [Sat, 29 Jun 2019 08:40:43 +0000 (10:40 +0200)]
src/client.rs: make module put_to_stream public
Dietmar Maurer [Sat, 29 Jun 2019 08:08:12 +0000 (10:08 +0200)]
add code to test H2 speed
I currently get about 1MB/s, which is ways too slow (must be a bug)?
Dietmar Maurer [Fri, 28 Jun 2019 14:35:00 +0000 (16:35 +0200)]
src/backup/dynamic_index.rs: introduce ReadChunk trait
Dietmar Maurer [Fri, 28 Jun 2019 14:27:01 +0000 (16:27 +0200)]
src/api2/reader.rs: add download chunk api
Dietmar Maurer [Fri, 28 Jun 2019 14:00:58 +0000 (16:00 +0200)]
src/client/http_client.rs: increase h2 window size, set tcp nodelay option
To increase h2 download speed.
Dietmar Maurer [Fri, 28 Jun 2019 13:48:09 +0000 (15:48 +0200)]
src/backup/chunk_store.rs: new method chunk_path()
Returns the absolute path.
Dietmar Maurer [Fri, 28 Jun 2019 05:07:52 +0000 (07:07 +0200)]
file download: avoid unnecessary copy