Backup Client Usage
===================
-The command line client is called :command:`proxmox-backup-client`.
+The command line client for Proxmox Backup Server is called
+:command:`proxmox-backup-client`.
+.. _client_repository:
-Repository Locations
---------------------
+Backup Repository Locations
+---------------------------
-The client uses the following notation to specify a datastore repository
-on the backup server.
+The client uses the following format to specify a datastore repository
+on the backup server (where username is specified in the form of user@realm):
[[username@]server[:port]:]datastore
The default value for ``username`` is ``root@pam``. If no server is specified,
the default is the local host (``localhost``).
-You can specify a port if your backup server is only reachable on a different
-port (e.g. with NAT and port forwarding).
+You can specify a port if your backup server is only reachable on a non-default
+port (for example, with NAT and port forwarding configurations).
-Note that if the server is an IPv6 address, you have to write it with square
+Note that if the server uses an IPv6 address, you have to write it with square
brackets (for example, `[fe80::01]`).
You can pass the repository with the ``--repository`` command line option, or
by setting the ``PBS_REPOSITORY`` environment variable.
-Here some examples of valid repositories and the real values
+Below are some examples of valid repositories and their corresponding real
+values:
================================ ================== ================== ===========
Example User Host:Port Datastore
mydatastore ``root@pam`` localhost:8007 mydatastore
myhostname:mydatastore ``root@pam`` myhostname:8007 mydatastore
user@pbs@myhostname:mydatastore ``user@pbs`` myhostname:8007 mydatastore
-user@pbs!token@host:store ``user@pbs!token`` myhostname:8007 mydatastore
+user\@pbs!token@host:store ``user@pbs!token`` myhostname:8007 mydatastore
192.168.55.55:1234:mydatastore ``root@pam`` 192.168.55.55:1234 mydatastore
[ff80::51]:mydatastore ``root@pam`` [ff80::51]:8007 mydatastore
[ff80::51]:1234:mydatastore ``root@pam`` [ff80::51]:1234 mydatastore
The default backup repository.
``PBS_PASSWORD``
- When set, this value is used for the password required for the backup server.
- You can also set this to a API token secret.
+ When set, this value is used as the password for the backup server.
+ You can also set this to an API token secret.
+
+``PBS_PASSWORD_FD``, ``PBS_PASSWORD_FILE``, ``PBS_PASSWORD_CMD``
+ Like ``PBS_PASSWORD``, but read data from an open file descriptor, a file
+ name or from the `stdout` of a command, respectively. The first defined
+ environment variable from the order above is preferred.
``PBS_ENCRYPTION_PASSWORD``
When set, this value is used to access the secret encryption key (if
protected by password).
-``PBS_FINGERPRINT`` When set, this value is used to verify the server
- certificate (only used if the system CA certificates cannot validate the
- certificate).
-
-
-Output Format
--------------
-
-Most commands support the ``--output-format`` parameter. It accepts
-the following values:
-
-:``text``: Text format (default). Structured data is rendered as a table.
-
-:``json``: JSON (single line).
+``PBS_ENCRYPTION_PASSWORD_FD``, ``PBS_ENCRYPTION_PASSWORD_FILE``, ``PBS_ENCRYPTION_PASSWORD_CMD``
+ Like ``PBS_ENCRYPTION_PASSWORD``, but read data from an open file descriptor,
+ a file name or from the `stdout` of a command, respectively. The first
+ defined environment variable from the order above is preferred.
-:``json-pretty``: JSON (multiple lines, nicely formatted).
+``PBS_FINGERPRINT``
+ When set, this value is used to verify the server certificate (only used if
+ the system CA certificates cannot validate the certificate).
-Please use the following environment variables to modify output behavior:
+.. Note:: Passwords must be valid UTF-8 and may not contain newlines. For your
+ convenience, Proxmox Backup Server only uses the first line as password, so
+ you can add arbitrary comments after the first newline.
-``PROXMOX_OUTPUT_FORMAT``
- Defines the default output format.
-``PROXMOX_OUTPUT_NO_BORDER``
- If set (to any value), do not render table borders.
-
-``PROXMOX_OUTPUT_NO_HEADER``
- If set (to any value), do not render table headers.
+Output Format
+-------------
-.. note:: The ``text`` format is designed to be human readable, and
- not meant to be parsed by automation tools. Please use the ``json``
- format if you need to process the output.
+.. include:: output-format.rst
-.. _creating-backups:
+.. _client_creating_backups:
Creating Backups
----------------
This section explains how to create a backup from within the machine. This can
-be a physical host, a virtual machine, or a container. Such backups may contain file
-and image archives. There are no restrictions in this case.
+be a physical host, a virtual machine, or a container. Such backups may contain
+file and image archives. There are no restrictions in this case.
-.. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`.
+.. Note:: If you want to backup virtual machines or containers on Proxmox VE,
+ see :ref:`pve-integration`.
-For the following example you need to have a backup server set up, working
-credentials and need to know the repository name.
-In the following examples we use ``backup-server:store1``.
+For the following example, you need to have a backup server set up, have working
+credentials, and know the repository name.
+In the following examples, we use ``backup-server:store1``.
.. code-block:: console
Uploaded 12129 chunks in 87 seconds (564 MB/s).
End Time: 2019-12-03T10:36:29+01:00
-This will prompt you for a password and then uploads a file archive named
+This will prompt you for a password, then upload a file archive named
``root.pxar`` containing all the files in the ``/`` directory.
-.. Caution:: Please note that the proxmox-backup-client does not
+.. Caution:: Please note that proxmox-backup-client does not
automatically include mount points. Instead, you will see a short
- ``skip mount point`` notice for each of them. The idea is to
+ ``skip mount point`` message for each of them. The idea is to
create a separate file archive for each mounted disk. You can
explicitly include them using the ``--include-dev`` option
(i.e. ``--include-dev /boot/efi``). You can use this option
The ``--repository`` option can get quite long and is used by all
commands. You can avoid having to enter this value by setting the
-environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set
-over multiple sessions, you should instead add the below line to your
+environment variable ``PBS_REPOSITORY``. Note that if you would like this to
+remain set over multiple sessions, you should instead add the below line to your
``.bashrc`` file.
.. code-block:: console
# export PBS_REPOSITORY=backup-server:store1
-After this you can execute all commands without specifying the ``--repository``
-option.
+After this, you can execute all commands without having to specify the
+``--repository`` option.
-One single backup is allowed to contain more than one archive. For example, if
-you want to backup two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``:
+A single backup is allowed to contain more than one archive. For example, if
+you want to back up two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``:
.. code-block:: console
<archive-name>.<type>:<source-path>
-Common types are ``.pxar`` for file archives, and ``.img`` for block
-device images. To create a backup of a block device run the following command:
+Common types are ``.pxar`` for file archives and ``.img`` for block
+device images. To create a backup of a block device, run the following command:
.. code-block:: console
# proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
-Excluding files/folders from a backup
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Excluding Files/Directories from a Backup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Sometimes it is desired to exclude certain files or folders from a backup archive.
+Sometimes it is desired to exclude certain files or directories from a backup archive.
To tell the Proxmox Backup client when and how to ignore files and directories,
-place a text file called ``.pxarexclude`` in the filesystem hierarchy.
+place a text file named ``.pxarexclude`` in the filesystem hierarchy.
Whenever the backup client encounters such a file in a directory, it interprets
-each line as glob match patterns for files and directories that are to be excluded
+each line as a glob match pattern for files and directories that are to be excluded
from the backup.
-The file must contain a single glob pattern per line. Empty lines are ignored.
-The same is true for lines starting with ``#``, which indicates a comment.
+The file must contain a single glob pattern per line. Empty lines and lines
+starting with ``#`` (indicating a comment) are ignored.
A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
to an explicit inclusion. This makes it possible to exclude all entries in a
directory except for a few single files/subdirectories.
``\`` is used to escape special glob characters.
``?`` matches any single character.
``*`` matches any character, including an empty string.
-``**`` is used to match subdirectories. It can be used to, for example, exclude
-all files ending in ``.tmp`` within the directory or subdirectories with the
-following pattern ``**/*.tmp``.
+``**`` is used to match current directory and subdirectories. For example, with
+the pattern ``**/*.tmp``, it would exclude all files ending in ``.tmp`` within
+a directory and its subdirectories.
``[...]`` matches a single character from any of the provided characters within
the brackets. ``[!...]`` does the complementary and matches any single character
not contained within the brackets. It is also possible to specify ranges with two
characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
-alphabetic character and ``[0-9]`` matches any one single digit.
+alphabetic character, and ``[0-9]`` matches any single digit.
The order of the glob match patterns defines whether a file is included or
-excluded, that is to say later entries override previous ones.
+excluded, that is to say, later entries override earlier ones.
This is also true for match patterns encountered deeper down the directory tree,
which can override a previous exclusion.
-Be aware that excluded directories will **not** be read by the backup client.
-Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect.
-``.pxarexclude`` files are treated as regular files and will be included in the
-backup archive.
+
+.. Note:: Excluded directories will **not** be read by the backup client. Thus,
+ a ``.pxarexclude`` file in an excluded subdirectory will have no effect.
+ ``.pxarexclude`` files are treated as regular files and will be included in
+ the backup archive.
For example, consider the following directory structure:
. .. file2
-.. _encryption:
+.. _client_encryption:
Encryption
----------
variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
-Using a master key to store and recover encryption keys
+Using a Master Key to Store and Recover Encryption Keys
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can also use ``proxmox-backup-client key`` to create an RSA public/private
.. code-block:: console
- # openssl rsautl -decrypt -inkey master-private.pem -in rsa-encrypted.key -out /path/to/target
- Enter pass phrase for ./master-private.pem: *********
+ # proxmox-backup-client key import-with-master-key /path/to/target --master-keyfile /path/to/master-private.pem --encrypted-keyfile /path/to/rsa-encrypted.key
+ Master Key Password: ******
+ New Password: ******
+ Verify Password: ******
7. The target file will now contain the encryption key information in plain
text. The success of this can be confirmed by passing the resulting ``json``
keep keys ordered and in a place that is separate from the contents being
backed up. It can happen, for example, that you back up an entire system, using
a key on that system. If the system then becomes inaccessible for any reason
- and needs to be restored, this will not be possible as the encryption key will be
- lost along with the broken system. In preparation for the worst case scenario,
- you should consider keeping a paper copy of this key locked away in
- a safe place.
+ and needs to be restored, this will not be possible, as the encryption key will be
+ lost along with the broken system.
+
+It is recommended that you keep your master key safe, but easily accessible, in
+order for quick disaster recovery. For this reason, the best place to store it
+is in your password manager, where it is immediately recoverable. As a backup to
+this, you should also save the key to a USB drive and store that in a secure
+place. This way, it is detached from any system, but is still easy to recover
+from, in case of emergency. Finally, in preparation for the worst case scenario,
+you should also consider keeping a paper copy of your master key locked away in
+a safe place. The ``paperkey`` subcommand can be used to create a QR encoded
+version of your master key. The following command sends the output of the
+``paperkey`` command to a text file, for easy printing.
+
+.. code-block:: console
+
+ proxmox-backup-client key paperkey --output-format text > qrkey.txt
Restoring Data
--------------
-The regular creation of backups is a necessary step to avoiding data
-loss. More importantly, however, is the restoration. It is good practice to perform
-periodic recovery tests to ensure that you can access the data in
-case of problems.
+The regular creation of backups is a necessary step in avoiding data loss. More
+importantly, however, is the restoration. It is good practice to perform
+periodic recovery tests to ensure that you can access the data in case of
+disaster.
First, you need to find the snapshot which you want to restore. The snapshot
-command provides a list of all the snapshots on the server:
+list command provides a list of all the snapshots on the server:
.. code-block:: console
- # proxmox-backup-client snapshots
+ # proxmox-backup-client snapshot list
┌────────────────────────────────┬─────────────┬────────────────────────────────────┐
│ snapshot │ size │ files │
╞════════════════════════════════╪═════════════╪════════════════════════════════════╡
The interactive recovery shell is a minimal command line interface that
utilizes the metadata stored in the catalog to quickly list, navigate and
-search files in a file archive.
+search for files in a file archive.
To restore files, you can select them individually or match them with a glob
pattern.
Using the catalog for navigation reduces the overhead considerably because only
the catalog needs to be downloaded and, optionally, decrypted.
-The actual chunks are only accessed if the metadata in the catalog is not enough
-or for the actual restore.
+The actual chunks are only accessed if the metadata in the catalog is
+insufficient or for the actual restore.
-Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
+Similar to common UNIX shells, ``cd`` and ``ls`` are the commands used to change
working directory and list directory contents in the archive.
``pwd`` shows the full path of the current working directory with respect to the
archive root.
-Being able to quickly search the contents of the archive is a commonly needed feature.
-That's where the catalog is most valuable.
-For example:
+The ability to quickly search the contents of the archive is a commonly required
+feature. That's where the catalog is most valuable. For example:
.. code-block:: console
pxar:/ > restore-selected /target/path
...
-This will find and print all files ending in ``.txt`` located in ``etc/`` or a
-subdirectory and add the corresponding pattern to the list for subsequent restores.
+This will find and print all files ending in ``.txt`` located in ``etc/`` or its
+subdirectories, and add the corresponding pattern to the list for subsequent restores.
``list-selected`` shows these patterns and ``restore-selected`` finally restores
all files in the archive matching the patterns to ``/target/path`` on the local
host. This will scan the whole archive.
-With ``restore /target/path`` you can restore the sub-archive given by the current
-working directory to the local target path ``/target/path`` on your host.
-By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
-further limited to files matching the pattern.
-For example:
+The ``restore`` command can be used to restore all the files contained within
+the backup archive. This is most helpful when paired with the ``--pattern
+<glob>`` option, as it allows you to restore all files matching a specific
+pattern. For example, if you wanted to restore configuration files
+located in ``/etc``, you could do the following:
.. code-block:: console
- pxar:/ > cd /etc/
- pxar:/etc/ > restore /target/ --pattern **/*.conf
+ pxar:/ > restore target/ --pattern etc/**/*.conf
...
-The above will scan trough all the directories below ``/etc`` and restore all
+The above will scan through all the directories below ``/etc`` and restore all
files ending in ``.conf``.
.. todo:: Explain interactive restore in more detail
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The :term:`FUSE` implementation for the pxar archive allows you to mount a
-file archive as a read-only filesystem to a mountpoint on your host.
+file archive as a read-only filesystem to a mount point on your host.
.. code-block:: console
load on your host, depending on the operations you perform on the mounted
filesystem.
-To unmount the filesystem use the ``umount`` command on the mountpoint:
+To unmount the filesystem, use the ``umount`` command on the mount point:
.. code-block:: console
Login and Logout
----------------
-The client tool prompts you to enter the logon password as soon as you
+The client tool prompts you to enter the login password as soon as you
want to access the backup server. The server checks your credentials
and responds with a ticket that is valid for two hours. The client
tool automatically stores that ticket and uses it for further requests
# proxmox-backup-client logout
+.. _changing-backup-owner:
+
+Changing the Owner of a Backup Group
+------------------------------------
+
+By default, the owner of a backup group is the user which was used to originally
+create that backup group (or in the case of sync jobs, ``root@pam``). This
+means that if a user ``mike@pbs`` created a backup, another user ``john@pbs``
+can not be used to create backups in that same backup group. In case you want
+to change the owner of a backup, you can do so with the below command, using a
+user that has ``Datastore.Modify`` privileges on the datastore.
+
+.. code-block:: console
+
+ # proxmox-backup-client change-owner vm/103 john@pbs
+
+This can also be done from within the web interface, by navigating to the
+`Content` section of the datastore that contains the backup group and
+selecting the user icon under the `Actions` column. Common cases for this could
+be to change the owner of a sync job from ``root@pam``, or to repurpose a
+backup group.
+
+
.. _backup-pruning:
Pruning and Removing Backups
.. code-block:: console
- # proxmox-backup-client forget <snapshot>
+ # proxmox-backup-client snapshot forget <snapshot>
.. caution:: This command removes all archives in this backup
.. note:: Neither the ``prune`` command nor the ``forget`` command free space
in the chunk-store. The chunk-store still contains the data blocks. To free
- space you need to perform :ref:`garbage-collection`.
+ space you need to perform :ref:`client_garbage-collection`.
+
+It is also possible to protect single snapshots from being pruned or deleted:
+
+.. code-block:: console
+
+ # proxmox-backup-client snapshot protected update <snapshot> true
+
+This will set the protected flag on the snapshot and prevent pruning or manual
+deletion of this snapshot untilt he flag is removed again with:
+
+.. code-block:: console
+ # proxmox-backup-client snapshot protected update <snapshot> false
-.. _garbage-collection:
+When a group is with a protected snapshot is deleted, only the non-protected
+ones are removed and the group will remain.
+
+.. note:: This flag will not be synced when using pull or sync jobs. If you
+ want to protect a synced snapshot, you have to manually to this again on
+ the target backup server.
+
+.. _client_garbage-collection:
Garbage Collection
------------------
(access time) property. Filesystems are mounted with the ``relatime`` option
by default. This results in a better performance by only updating the
``atime`` property if the last access has been at least 24 hours ago. The
- downside is, that touching a chunk within these 24 hours will not always
+ downside is that touching a chunk within these 24 hours will not always
update its ``atime`` property.
Chunks in the grace period will be logged at the end of the garbage
Average chunk size: 2486565
TASK OK
-
-.. todo:: howto run garbage-collection at regular intervals (cron)
+Garbage collection can also be scheduled using ``promxox-backup-manager`` or
+from the Proxmox Backup Server's web interface.
Benchmarking
------------
The backup client also comes with a benchmarking tool. This tool measures
-various metrics relating to compression and encryption speeds. You can run a
-benchmark using the ``benchmark`` subcommand of ``proxmox-backup-client``:
+various metrics relating to compression and encryption speeds. If a Proxmox
+Backup repository (remote or local) is specified, the TLS upload speed will get
+measured too.
+
+You can run a benchmark using the ``benchmark`` subcommand of
+``proxmox-backup-client``:
+
+.. note:: The TLS speed test is only included if a :ref:`backup server
+ repository is specified <client_repository>`.
.. code-block:: console
# proxmox-backup-client benchmark
- Uploaded 656 chunks in 5 seconds.
- Time per request: 7659 microseconds.
- TLS speed: 547.60 MB/s
- SHA256 speed: 585.76 MB/s
- Compression speed: 1923.96 MB/s
- Decompress speed: 7885.24 MB/s
- AES256/GCM speed: 3974.03 MB/s
+ Uploaded 1517 chunks in 5 seconds.
+ Time per request: 3309 microseconds.
+ TLS speed: 1267.41 MB/s
+ SHA256 speed: 2066.73 MB/s
+ Compression speed: 775.11 MB/s
+ Decompress speed: 1233.35 MB/s
+ AES256/GCM speed: 3688.27 MB/s
+ Verify speed: 783.43 MB/s
┌───────────────────────────────────┬─────────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪═════════════════════╡
- │ TLS (maximal backup upload speed) │ 547.60 MB/s (93%) │
+ │ TLS (maximal backup upload speed) │ 1267.41 MB/s (103%) │
├───────────────────────────────────┼─────────────────────┤
- │ SHA256 checksum computation speed │ 585.76 MB/s (28%) │
+ │ SHA256 checksum computation speed │ 2066.73 MB/s (102%) │
├───────────────────────────────────┼─────────────────────┤
- │ ZStd level 1 compression speed │ 1923.96 MB/s (89%) │
+ │ ZStd level 1 compression speed │ 775.11 MB/s (103%) │
├───────────────────────────────────┼─────────────────────┤
- │ ZStd level 1 decompression speed │ 7885.24 MB/s (98%) │
+ │ ZStd level 1 decompression speed │ 1233.35 MB/s (103%) │
├───────────────────────────────────┼─────────────────────┤
- │ AES256 GCM encryption speed │ 3974.03 MB/s (104%) │
+ │ Chunk verification speed │ 783.43 MB/s (103%) │
+ ├───────────────────────────────────┼─────────────────────┤
+ │ AES256 GCM encryption speed │ 3688.27 MB/s (101%) │
└───────────────────────────────────┴─────────────────────┘
+
.. note:: The percentages given in the output table correspond to a
- comparison against a Ryzen 7 2700X. The TLS test connects to the
- local host, so there is no network involved.
+ comparison against a Ryzen 7 2700X.
You can also pass the ``--output-format`` parameter to output stats in ``json``,
rather than the default table format.
-
-
projects / proxmox-backup.git / blobdiff