4 .. The administration guide.
5 .. todo:: either add a bit more explanation or remove the previous sentence
13 When doing deduplication, there are different strategies to get
14 optimal results in terms of performance and/or deduplication rates.
15 Depending on the type of data, it can be split into *fixed* or *variable*
18 Fixed sized chunking requires minimal CPU power, and is used to
19 backup virtual machine images.
21 Variable sized chunking needs more CPU power, but is essential to get
22 good deduplication rates for file archives.
24 The Proxmox Backup Server supports both strategies.
27 File Archives: ``<name>.pxar``
28 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
30 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
32 A file archive stores a full directory tree. Content is stored using
33 the :ref:`pxar-format`, split into variable-sized chunks. The format
34 is optimized to achieve good deduplication rates.
37 Image Archives: ``<name>.img``
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
40 This is used for virtual machine images and other large binary
41 data. Content is split into fixed-sized chunks.
47 This type is used to store smaller (< 16MB) binary data such as
48 configuration files. Larger files should be stored as image archive.
50 .. caution:: Please do not store all files as BLOBs. Instead, use the
51 file archive to store whole directory trees.
54 Catalog File: ``catalog.pcat1``
55 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
57 The catalog file is an index for file archives. It contains
58 the list of files and is used to speed up search operations.
61 The Manifest: ``index.json``
62 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64 The manifest contains the list of all backup files, their
65 sizes and checksums. It is used to verify the consistency of a
72 The backup server groups backups by *type*, where *type* is one of:
75 This type is used for :term:`virtual machine`\ s. Typically
76 consists of the virtual machine's configuration file and an image archive
80 This type is used for :term:`container`\ s. Consists of the container's
81 configuration and a single file archive for the filesystem content.
84 This type is used for backups created from within the backed up machine.
85 Typically this would be a physical host but could also be a virtual machine
86 or container. Such backups may contain file and image archives, there are no restrictions in this regard.
92 A unique ID. Usually the virtual machine or container ID. ``host``
93 type backups normally use the hostname.
99 The time when the backup was made.
105 The tuple ``<type>/<ID>`` is called a backup group. Such a group
106 may contain one or more backup snapshots.
112 The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It
113 uniquely identifies a specific backup within a datastore.
115 .. code-block:: console
116 :caption: Backup Snapshot Examples
118 vm/104/2019-10-09T08:01:06Z
119 host/elsa/2019-11-08T09:48:14Z
121 As you can see, the time format is RFC3399_ with Coordinated
122 Universal Time (UTC_, identified by the trailing *Z*).
124 Backup Server Management
125 ------------------------
127 The command line tool to configure and manage the backup server is called
128 :command:`proxmox-backup-manager`.
135 A datastore is a place where backups are stored. The current implementation
136 uses a directory inside a standard unix file system (``ext4``, ``xfs``
137 or ``zfs``) to store the backup data.
139 Datastores are identified by a simple *ID*. You can configure it
140 when setting up the backup server.
144 Datastore Configuration
145 ~~~~~~~~~~~~~~~~~~~~~~~
147 You can configure multiple datastores. Minimum one datastore needs to be
148 configured. The datastore is identified by a simple `name` and points to a
149 directory on the filesystem. Each datastore also has associated retention
150 settings of how many backup snapshots for each interval of ``hourly``,
151 ``daily``, ``weekly``, ``monthly``, ``yearly`` as well as an time independent
152 number of backups to keep in that store. :ref:`Pruning <pruning>` and
153 :ref:`garbage collection <garbage-collection>` can also be configured to run
154 periodically based on a configured :term:`schedule` per datastore.
156 The following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
158 .. code-block:: console
160 # proxmox-backup-manager datastore create store1 /backup/disk1/store1
162 To list existing datastores run:
164 .. code-block:: console
166 # proxmox-backup-manager datastore list
167 ┌────────┬──────────────────────┬─────────────────────────────┐
168 │ name │ path │ comment │
169 ╞════════╪══════════════════════╪═════════════════════════════╡
170 │ store1 │ /backup/disk1/store1 │ This is my default storage. │
171 └────────┴──────────────────────┴─────────────────────────────┘
173 You can change settings of a datastore, for example to set a prune and garbage
174 collection schedule or retention settings using ``update`` subcommand and view
175 a datastore with the ``show`` subcommand:
177 .. code-block:: console
179 # proxmox-backup-manager datastore update store1 --keep-last 7 --prune-schedule daily --gc-schedule 'Tue 04:27'
180 # proxmox-backup-manager datastore show store1
181 ┌────────────────┬─────────────────────────────┐
183 ╞════════════════╪═════════════════════════════╡
185 ├────────────────┼─────────────────────────────┤
186 │ path │ /backup/disk1/store1 │
187 ├────────────────┼─────────────────────────────┤
188 │ comment │ This is my default storage. │
189 ├────────────────┼─────────────────────────────┤
190 │ gc-schedule │ Tue 04:27 │
191 ├────────────────┼─────────────────────────────┤
193 ├────────────────┼─────────────────────────────┤
194 │ prune-schedule │ daily │
195 └────────────────┴─────────────────────────────┘
197 Finally, it is possible to remove the datastore configuration:
199 .. code-block:: console
201 # proxmox-backup-manager datastore remove store1
203 .. note:: The above command removes only the datastore configuration. It does
204 not delete any data from the underlying directory.
210 After creating a datastore, the following default layout will appear:
212 .. code-block:: console
214 # ls -arilh /backup/disk1/store1
215 276493 -rw-r--r-- 1 backup backup 0 Jul 8 12:35 .lock
216 276490 drwxr-x--- 1 backup backup 1064960 Jul 8 12:35 .chunks
218 `.lock` is an empty file used for process locking.
220 The `.chunks` directory contains folders, starting from `0000` and taking hexadecimal values until `ffff`. These
221 directories will store the chunked data after a backup operation has been executed.
223 .. code-block:: console
225 # ls -arilh /backup/disk1/store1/.chunks
226 545824 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 ffff
227 545823 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffe
228 415621 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffd
229 415620 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffc
230 353187 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffb
231 344995 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffa
232 144079 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff9
233 144078 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff8
234 144077 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff7
236 403180 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000c
237 403179 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000b
238 403177 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000a
239 402530 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0009
240 402513 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0008
241 402509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0007
242 276509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0006
243 276508 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0005
244 276507 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0004
245 276501 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0003
246 276499 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0002
247 276498 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0001
248 276494 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0000
249 276489 drwxr-xr-x 3 backup backup 4.0K Jul 8 12:35 ..
250 276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 .
257 Proxmox Backup Server supports several authentication realms, and you need to
258 choose the realm when you add a new user. Possible realms are:
260 :pam: Linux PAM standard authentication. Use this if you want to
261 authenticate as Linux system user (Users need to exist on the
264 :pbs: Proxmox Backup Server realm. This type stores hashed passwords in
265 ``/etc/proxmox-backup/shadow.json``.
267 After installation, there is a single user ``root@pam``, which
268 corresponds to the Unix superuser. You can use the
269 ``proxmox-backup-manager`` command line tool to list or manipulate
272 .. code-block:: console
274 # proxmox-backup-manager user list
275 ┌─────────────┬────────┬────────┬───────────┬──────────┬────────────────┬────────────────────┐
276 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
277 ╞═════════════╪════════╪════════╪═══════════╪══════════╪════════════════╪════════════════════╡
278 │ root@pam │ 1 │ │ │ │ │ Superuser │
279 └─────────────┴────────┴────────┴───────────┴──────────┴────────────────┴────────────────────┘
281 The superuser has full administration rights on everything, so you
282 normally want to add other users with less privileges:
284 .. code-block:: console
286 # proxmox-backup-manager user create john@pbs --email john@example.com
288 The create command lets you specify many options like ``--email`` or
289 ``--password``. You can update or change any of them using the
290 update command later:
292 .. code-block:: console
294 # proxmox-backup-manager user update john@pbs --firstname John --lastname Smith
295 # proxmox-backup-manager user update john@pbs --comment "An example user."
297 .. todo:: Mention how to set password without passing plaintext password as cli argument.
300 The resulting user list looks like this:
302 .. code-block:: console
304 # proxmox-backup-manager user list
305 ┌──────────┬────────┬────────┬───────────┬──────────┬──────────────────┬──────────────────┐
306 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
307 ╞══════════╪════════╪════════╪═══════════╪══════════╪══════════════════╪══════════════════╡
308 │ john@pbs │ 1 │ │ John │ Smith │ john@example.com │ An example user. │
309 ├──────────┼────────┼────────┼───────────┼──────────┼──────────────────┼──────────────────┤
310 │ root@pam │ 1 │ │ │ │ │ Superuser │
311 └──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
313 Newly created users do not have any permissions. Please read the next
314 section to learn how to set access permissions.
316 If you want to disable a user account, you can do that by setting ``--enable`` to ``0``
318 .. code-block:: console
320 # proxmox-backup-manager user update john@pbs --enable 0
322 Or completely remove the user with:
324 .. code-block:: console
326 # proxmox-backup-manager user remove john@pbs
332 By default new users do not have any permission. Instead you need to
333 specify what is allowed and what is not. You can do this by assigning
334 roles to users on specific objects like datastores or remotes. The
335 following roles exist:
338 Disable Access - nothing is allowed.
341 The Administrator can do anything.
344 An Auditor can view things, but is not allowed to change settings.
347 Can do anything on datastores.
350 Can view datastore settings and list content. But
351 is not allowed to read the actual data.
354 Can Inspect datastore content and can do restores.
357 Can backup and restore owned backups.
359 **DatastorePowerUser**
360 Can backup, restore, and prune owned backups.
363 Can do anything on remotes.
366 Can view remote settings.
368 **RemoteSyncOperator**
369 Is allowed to read data from a remote.
375 A remote is a different Proxmox Backup Server installation and a user on that
376 installation, from which you can `sync` datastores to a local datastore with a
379 For adding a remote you need its hostname or ip, a userid and password on the
380 remote and its certificate fingerprint to add it. To get the fingerprint use
381 the ``proxmox-backup-manager cert info`` command on the remote.
383 .. code-block:: console
385 # proxmox-backup-manager cert info |grep Fingerprint
386 Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
388 With the needed information add the remote with:
390 .. code-block:: console
392 # proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
394 Use the ``list``, ``show``, ``update``, ``remove`` subcommands of
395 ``proxmox-backup-manager remote`` to manage your remotes:
397 .. code-block:: console
399 # proxmox-backup-manager remote update pbs2 --host pbs2.example
400 # proxmox-backup-manager remote list
401 ┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐
402 │ name │ host │ userid │ fingerprint │ comment │
403 ╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡
404 │ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │ │
405 └──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘
406 # proxmox-backup-manager remote remove pbs2
412 Sync jobs are configured to pull the contents of a datastore on a `Remote` to a
413 local datastore. You can either start the sync job manually on the GUI or
414 provide it with a :term:`schedule` to run regularly. The
415 ``proxmox-backup-manager sync-job`` command is used to manage sync jobs:
417 .. code-block:: console
419 # proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30'
420 # proxmox-backup-manager sync-job update pbs2-local --comment 'offsite'
421 # proxmox-backup-manager sync-job list
422 ┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐
423 │ id │ store │ remote │ remote-store │ schedule │ comment │
424 ╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡
425 │ pbs2-local │ local │ pbs2 │ local │ Wed 02:30 │ offsite │
426 └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
427 # proxmox-backup-manager sync-job remove pbs2-local
433 The command line client is called :command:`proxmox-backup-client`.
436 Respository Locations
437 ~~~~~~~~~~~~~~~~~~~~~
439 The client uses the following notation to specify a datastore repository
440 on the backup server.
442 [[username@]server:]datastore
444 The default value for ``username`` ist ``root``. If no server is specified,
445 the default is the local host (``localhost``).
447 You can pass the repository with the ``--repository`` command
448 line option, or by setting the ``PBS_REPOSITORY`` environment
452 Environment Variables
453 ~~~~~~~~~~~~~~~~~~~~~
456 The default backup repository.
459 When set, this value is used for the password required for the
462 ``PBS_ENCRYPTION_PASSWORD``
463 When set, this value is used to access the secret encryption key (if
464 protected by password).
466 ``PBS_FINGERPRINT`` When set, this value is used to verify the server
467 certificate (only used if the system CA certificates cannot
468 validate the certificate).
474 Most commands support the ``--output-format`` parameter. It accepts
475 the following values:
477 :``text``: Text format (default). Structured data is rendered as a table.
479 :``json``: JSON (single line).
481 :``json-pretty``: JSON (multiple lines, nicely formatted).
484 Please use the following environment variables to modify output behavior:
486 ``PROXMOX_OUTPUT_FORMAT``
487 Defines the default output format.
489 ``PROXMOX_OUTPUT_NO_BORDER``
490 If set (to any value), do not render table borders.
492 ``PROXMOX_OUTPUT_NO_HEADER``
493 If set (to any value), do not render table headers.
495 .. note:: The ``text`` format is designed to be human readable, and
496 not meant to be parsed by automation tools. Please use the ``json``
497 format if you need to process the output.
500 .. _creating-backups:
505 This section explains how to create a backup from within the machine. This can
506 be a physical host, a virtual machine, or a container. Such backups may contain file
507 and image archives. There are no restrictions in this case.
509 .. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`.
511 For the following example you need to have a backup server set up, working
512 credentials and need to know the repository name.
513 In the following examples we use ``backup-server:store1``.
515 .. code-block:: console
517 # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
518 Starting backup: host/elsa/2019-12-03T09:35:01Z
520 skip mount point: "/boot/efi"
521 skip mount point: "/dev"
522 skip mount point: "/run"
523 skip mount point: "/sys"
524 Uploaded 12129 chunks in 87 seconds (564 MB/s).
525 End Time: 2019-12-03T10:36:29+01:00
527 This will prompt you for a password and then uploads a file archive named
528 ``root.pxar`` containing all the files in the ``/`` directory.
530 .. Caution:: Please note that the proxmox-backup-client does not
531 automatically include mount points. Instead, you will see a short
532 ``skip mount point`` notice for each of them. The idea is to
533 create a separate file archive for each mounted disk. You can
534 explicitly include them using the ``--include-dev`` option
535 (i.e. ``--include-dev /boot/efi``). You can use this option
536 multiple times for each mount point that should be included.
538 The ``--repository`` option can get quite long and is used by all
539 commands. You can avoid having to enter this value by setting the
540 environment variable ``PBS_REPOSITORY``.
542 .. code-block:: console
544 # export PBS_REPOSTORY=backup-server:store1
546 After this you can execute all commands without specifying the ``--repository``
549 One single backup is allowed to contain more than one archive. For example, if
550 you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
552 .. code-block:: console
554 # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
556 This creates a backup of both disks.
558 The backup command takes a list of backup specifications, which
559 include the archive name on the server, the type of the archive, and the
560 archive source at the client. The format is:
562 <archive-name>.<type>:<source-path>
564 Common types are ``.pxar`` for file archives, and ``.img`` for block
565 device images. To create a backup of a block device run the following command:
567 .. code-block:: console
569 # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
571 Excluding files/folders from a backup
572 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
574 Sometimes it is desired to exclude certain files or folders from a backup archive.
575 To tell the Proxmox backup client when and how to ignore files and directories,
576 place a text file called ``.pxarexclude`` in the filesystem hierarchy.
577 Whenever the backup client encounters such a file in a directory, it interprets
578 each line as glob match patterns for files and directories that are to be excluded
581 The file must contain a single glob pattern per line. Empty lines are ignored.
582 The same is true for lines starting with ``#``, which indicates a comment.
583 A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
584 to an explicit inclusion. This makes it possible to exclude all entries in a
585 directory except for a few single files/subdirectories.
586 Lines ending in ``/`` match only on directories.
587 The directory containing the ``.pxarexclude`` file is considered to be the root of
588 the given patterns. It is only possible to match files in this directory and its subdirectories.
590 ``\`` is used to escape special glob characters.
591 ``?`` matches any single character.
592 ``*`` matches any character, including an empty string.
593 ``**`` is used to match subdirectories. It can be used to, for example, exclude
594 all files ending in ``.tmp`` within the directory or subdirectories with the
595 following pattern ``**/*.tmp``.
596 ``[...]`` matches a single character from any of the provided characters within
597 the brackets. ``[!...]`` does the complementary and matches any singe character
598 not contained within the brackets. It is also possible to specify ranges with two
599 characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
600 alphabetic character and ``[0-9]`` matches any one single digit.
602 The order of the glob match patterns defines if a file is included or
603 excluded, later entries win over previous ones.
604 This is also true for match patterns encountered deeper down the directory tree,
605 which can override a previous exclusion.
606 Be aware that excluded directories will **not** be read by the backup client.
607 A ``.pxarexclude`` file in a subdirectory will have no effect.
608 ``.pxarexclude`` files are treated as regular files and will be included in the
611 For example, consider the following directory structure:
613 .. code-block:: console
617 . .. .pxarexclude subfolder0 subfolder1
620 . .. file0 file1 file2 file3 .pxarexclude
623 . .. file0 file1 file2 file3
625 The different ``.pxarexclude`` files contain the following:
627 .. code-block:: console
629 # cat folder/.pxarexclude
634 .. code-block:: console
636 # cat folder/subfolder0/.pxarexclude
639 This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of
640 ``subfolder1`` except ``file2``.
642 Restoring this backup will result in:
644 .. code-block:: console
648 . .. .pxarexclude subfolder0 subfolder1
651 . .. file0 file2 .pxarexclude
659 Proxmox backup supports client side encryption with AES-256 in GCM_
660 mode. First you need to create an encryption key:
662 .. code-block:: console
664 # proxmox-backup-client key create my-backup.key
665 Encryption Key Password: **************
667 The key is password protected by default. If you do not need this
668 extra protection, you can also create it without a password:
670 .. code-block:: console
672 # proxmox-backup-client key create /path/to/my-backup.key --kdf none
675 .. code-block:: console
677 # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
679 Encryption Key Password: **************
683 You can avoid entering the passwords by setting the environment
684 variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
686 .. todo:: Explain master-key
692 The regular creation of backups is a necessary step to avoid data
693 loss. More important, however, is the restoration. It is good practice to perform
694 periodic recovery tests to ensure that you can access the data in
697 First, you need to find the snapshot which you want to restore. The snapshot
698 command gives a list of all snapshots on the server:
700 .. code-block:: console
702 # proxmox-backup-client snapshots
703 ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐
704 │ snapshot │ size │ files │
705 ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡
706 │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │
707 ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
708 │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │
709 ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
712 You can inspect the catalog to find specific files.
714 .. code-block:: console
716 # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z
718 d "./root.pxar.didx/etc/cifs-utils"
719 l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
720 d "./root.pxar.didx/etc/console-setup"
723 The restore command lets you restore a single archive from the
726 .. code-block:: console
728 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
730 To get the contents of any archive you can restore the ``ìndex.json`` file in the
731 repository and restore it to '-'. This will dump the content to the standard output.
733 .. code-block:: console
735 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
741 If you only want to restore a few individual files, it is often easier
742 to use the interactive recovery shell.
744 .. code-block:: console
746 # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar
747 Starting interactive shell
749 bin boot dev etc home lib lib32
752 The interactive recovery shell is a minimalistic command line interface that
753 utilizes the metadata stored in the catalog to quickly list, navigate and
754 search files in a file archive.
755 To restore files, you can select them individually or match them with a glob
758 Using the catalog for navigation reduces the overhead considerably because only
759 the catalog needs to be downloaded and, optionally, decrypted.
760 The actual chunks are only accessed if the metadata in the catalog is not enough
761 or for the actual restore.
763 Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
764 working directory and list directory contents in the archive.
765 ``pwd`` shows the full path of the current working directory with respect to the
768 Being able to quickly search the contents of the archive is a often needed feature.
769 That's where the catalog is most valuable.
772 .. code-block:: console
774 pxar:/ > find etc/ **/*.txt --select
776 pxar:/ > list-selected
778 pxar:/ > restore-selected /target/path
781 This will find and print all files ending in ``.txt`` located in ``etc/`` or a
782 subdirectory and add the corresponding pattern to the list for subsequent restores.
783 ``list-selected`` shows these patterns and ``restore-selected`` finally restores
784 all files in the archive matching the patterns to ``/target/path`` on the local
785 host. This will scan the whole archive.
787 With ``restore /target/path`` you can restore the sub-archive given by the current
788 working directory to the local target path ``/target/path`` on your host.
789 By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
790 further limited to files matching the pattern.
793 .. code-block:: console
796 pxar:/etc/ > restore /target/ --pattern **/*.conf
799 The above will scan trough all the directories below ``/etc`` and restore all
800 files ending in ``.conf``.
802 .. todo:: Explain interactive restore in more detail
804 Mounting of Archives via FUSE
805 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
807 The :term:`FUSE` implementation for the pxar archive allows you to mount a
808 file archive as a read-only filesystem to a mountpoint on your host.
810 .. code-block:: console
812 # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt
814 bin dev home lib32 libx32 media opt root sbin sys usr
815 boot etc lib lib64 lost+found mnt proc run srv tmp var
817 This allows you to access the full content of the archive in a seamless manner.
819 .. note:: As the FUSE connection needs to fetch and decrypt chunks from the
820 backup servers datastore, this can cause some additional network and CPU
821 load on your host, depending on the operations you perform on the mounted
824 To unmount the filesystem use the ``umount`` command on the mountpoint:
826 .. code-block:: console
833 The client tool prompts you to enter the logon password as soon as you
834 want to access the backup server. The server checks your credentials
835 and responds with a ticket that is valid for two hours. The client
836 tool automatically stores that ticket and uses it for further requests
839 You can also manually trigger this login/logout using the login and
842 .. code-block:: console
844 # proxmox-backup-client login
847 To remove the ticket, issue a logout:
849 .. code-block:: console
851 # proxmox-backup-client logout
856 Pruning and Removing Backups
857 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
859 You can manually delete a backup snapshot using the ``forget``
862 .. code-block:: console
864 # proxmox-backup-client forget <snapshot>
867 .. caution:: This command removes all archives in this backup
868 snapshot. They will be inaccessible and unrecoverable.
871 The manual removal is sometimes required, but normally the prune
872 command is used to systematically delete older backups. Prune lets
873 you specify which backup snapshots you want to keep. The
874 following retention options are available:
877 Keep the last ``<N>`` backup snapshots.
879 ``--keep-hourly <N>``
880 Keep backups for the last ``<N>`` hours. If there is more than one
881 backup for a single hour, only the latest is kept.
884 Keep backups for the last ``<N>`` days. If there is more than one
885 backup for a single day, only the latest is kept.
887 ``--keep-weekly <N>``
888 Keep backups for the last ``<N>`` weeks. If there is more than one
889 backup for a single week, only the latest is kept.
891 .. note:: Weeks start on Monday and end on Sunday. The software
892 uses the `ISO week date`_ system and handles weeks at
893 the end of the year correctly.
895 ``--keep-monthly <N>``
896 Keep backups for the last ``<N>`` months. If there is more than one
897 backup for a single month, only the latest is kept.
899 ``--keep-yearly <N>``
900 Keep backups for the last ``<N>`` years. If there is more than one
901 backup for a single year, only the latest is kept.
903 The retention options are processed in the order given above. Each option
904 only covers backups within its time period. The next option does not take care
905 of already covered backups. It will only consider older backups.
907 Unfinished and incomplete backups will be removed by the prune command unless
908 they are newer than the last successful backup. In this case, the last failed
911 .. code-block:: console
913 # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3
916 You can use the ``--dry-run`` option to test your settings. This only
917 shows the list of existing snapshots and which action prune would take.
919 .. code-block:: console
921 # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
922 ┌────────────────────────────────┬──────┐
924 ╞════════════════════════════════╪══════╡
925 │ host/elsa/2019-12-04T13:20:37Z │ 1 │
926 ├────────────────────────────────┼──────┤
927 │ host/elsa/2019-12-03T09:35:01Z │ 0 │
928 ├────────────────────────────────┼──────┤
929 │ host/elsa/2019-11-22T11:54:47Z │ 1 │
930 ├────────────────────────────────┼──────┤
931 │ host/elsa/2019-11-21T12:36:25Z │ 0 │
932 ├────────────────────────────────┼──────┤
933 │ host/elsa/2019-11-10T10:42:20Z │ 1 │
934 └────────────────────────────────┴──────┘
936 .. note:: Neither the ``prune`` command nor the ``forget`` command free space
937 in the chunk-store. The chunk-store still contains the data blocks. To free
938 space you need to perform :ref:`garbage-collection`.
941 .. _garbage-collection:
946 The ``prune`` command removes only the backup index files, not the data
947 from the data store. This task is left to the garbage collection
948 command. It is recommended to carry out garbage collection on a regular basis.
950 The garbage collection works in two phases. In the first phase, all
951 data blocks that are still in use are marked. In the second phase,
952 unused data blocks are removed.
954 .. note:: This command needs to read all existing backup index files
955 and touches the complete chunk-store. This can take a long time
956 depending on the number of chunks and the speed of the underlying
960 .. code-block:: console
962 # proxmox-backup-client garbage-collect
963 starting garbage collection on store store2
964 Start GC phase1 (mark used chunks)
965 Start GC phase2 (sweep unused chunks)
966 percentage done: 1, chunk count: 219
967 percentage done: 2, chunk count: 453
969 percentage done: 99, chunk count: 21188
970 Removed bytes: 411368505
972 Original data bytes: 327160886391
973 Disk bytes: 52767414743 (16 %)
975 Average chunk size: 2486565
979 .. todo:: howto run garbage-collection at regular intervalls (cron)
984 `Proxmox VE`_ integration
985 -------------------------
987 You need to define a new storage with type 'pbs' on your `Proxmox VE`_
988 node. The following example uses ``store2`` as storage name, and
989 assumes the server address is ``localhost``, and you want to connect
992 .. code-block:: console
994 # pvesm add pbs store2 --server localhost --datastore store2
995 # pvesm set store2 --username user1@pbs --password <secret>
997 If your backup server uses a self signed certificate, you need to add
998 the certificate fingerprint to the configuration. You can get the
999 fingerprint by running the following command on the backup server:
1001 .. code-block:: console
1003 # proxmox-backup-manager cert info |grep Fingerprint
1004 Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
1006 Please add that fingerprint to your configuration to establish a trust
1009 .. code-block:: console
1011 # pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
1013 After that you should be able to see storage status with:
1015 .. code-block:: console
1017 # pvesm status --storage store2
1018 Name Type Status Total Used Available %
1019 store2 pbs active 3905109820 1336687816 2568422004 34.23%
1023 .. include:: command-line-tools.rst
1025 .. include:: services.rst