4 The administration guide.
6 .. todo:: either add a bit more explanation or remove the previous sentence
14 When doing deduplication, there are different strategies to get
15 optimal results in terms of performance and/or deduplication rates.
16 Depending on the type of data, one can split data into *fixed* or *variable*
19 Fixed sized chunking needs almost no CPU performance, and is used to
20 backup virtual machine images.
22 Variable sized chunking needs more CPU power, but is essential to get
23 good deduplication rates for file archives.
25 The backup server supports both strategies.
28 File Archives: ``<name>.pxar``
29 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
31 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
33 A file archive stores a full directory tree. Content is stored using
34 the :ref:`pxar-format`, split into variable sized chunks. The format
35 is optimized to achieve good deduplication rates.
38 Image Archives: ``<name>.img``
39 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
41 This is used for virtual machine images and other large binary
42 data. Content is split into fixed sized chunks.
48 This type is used to store smaller (< 16MB) binary data such as
49 configuration files. Larger files should be stored as image archive.
51 .. caution:: Please do not store all files as BLOBs. Instead, use the
52 file archive to store whole directory trees.
55 Catalog File: ``catalog.pcat1``
56 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
58 The catalog file is an index for file archives. It contains
59 the list of files and is used to speed-up search operations.
62 The Manifest: ``index.json``
63 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
65 The manifest contains the list of all backup files, their
66 sizes and checksums. It is used to verify the consistency of a
73 The backup server groups backups by *type*, where *type* is one of:
76 This type is used for :term:`virtual machine`\ s. Typically
77 contains the virtual machine's configuration and an image archive
81 This type is used for :term:`container`\ s. Contains the container's
82 configuration and a single file archive for the container content.
85 This type is used for backups created from within the backed up machine.
86 Typically this would be a physical host but could also be a virtual machine
87 or container. Such backups may contain file and image archives, there are no restrictions in this regard.
93 An unique ID. Usually the virtual machine or container ID. ``host``
94 type backups normally use the hostname.
100 The time when the backup was made.
106 The tuple ``<type>/<ID>`` is called a backup group. Such a group
107 may contain one or more backup snapshots.
113 The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It
114 uniquely identifies a specific backup within a datastore.
116 .. code-block:: console
117 :caption: Backup Snapshot Examples
119 vm/104/2019-10-09T08:01:06Z
120 host/elsa/2019-11-08T09:48:14Z
122 As you can see, the time format is RFC3399_ with Coordinated
123 Universal Time (UTC_, identified by the trailing *Z*).
129 A datastore is a place where backups are stored. The current implementation
130 uses a directory inside a standard unix file system (``ext4``, ``xfs``
131 or ``zfs``) to store the backup data.
133 Datastores are identified by a simple *ID*. You can configure it
134 when setting up the backup server.
137 Backup Server Management
138 ------------------------
140 The command line tool to configure and manage the backup server is called
141 :command:`proxmox-backup-manager`.
144 Datastore Configuration
145 ~~~~~~~~~~~~~~~~~~~~~~~
147 A :term:`datastore` is a place to store backups. You can configure
148 multiple datastores. At least one datastore needs to be
149 configured. The datastore is identified by a simple `name` and points
152 The following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
154 .. code-block:: console
156 # proxmox-backup-manager datastore create store1 /backup/disk1/store1
158 To list existing datastores run:
160 .. code-block:: console
162 # proxmox-backup-manager datastore list
163 ┌────────┬──────────────────────┬─────────────────────────────┐
164 │ name │ path │ comment │
165 ╞════════╪══════════════════════╪═════════════════════════════╡
166 │ store1 │ /backup/disk1/store1 │ This is my default storage. │
167 └────────┴──────────────────────┴─────────────────────────────┘
169 Finally, it is possible to remove the datastore configuration:
171 .. code-block:: console
173 # proxmox-backup-manager datastore remove store1
175 .. note:: The above command removes only the datastore configuration. It does
176 not delete any data from the underlying directory.
182 .. todo:: Add datastore file layout example
188 Proxmox Backup support several authentication realms, and you need to
189 choose the realm when you add a new user. Possible realms are:
191 :pam: Linux PAM standard authentication. Use this if you want to
192 authenticate as Linux system user (Users needs to exist on the
195 :pbs: Proxmox Backup Server realm. This type stores hashed passwords in
196 ``/etc/proxmox-backup/shadow.json``.
198 After installation, there is a single user ``root@pam``, which
199 corresponds to the Unix superuser. You can use the
200 ``proxmox-backup-manager`` command line tool to list or manipulate
203 .. code-block:: console
205 # proxmox-backup-manager user list
206 ┌─────────────┬────────┬────────┬───────────┬──────────┬─────────────────┬────────────────────┐
207 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
208 ╞═════════════╪════════╪════════╪═══════════╪══════════╪═════════════════╪════════════════════╡
209 │ root@pam │ 1 │ │ │ │ │ Superuser │
210 └─────────────┴────────┴────────┴───────────┴──────────┴─────────────────┴────────────────────┘
212 The superuser has full administration rights on everything, so you
213 normally want to add other users with less privileges:
215 .. code-block:: console
217 # proxmox-backup-manager user create john@pbs --email john@example.com
219 The create command lets you specify many option like ``--email`` or
220 ``--password``, but you can update or change any of them using the
221 update command later:
223 .. code-block:: console
225 # proxmox-backup-manager user update john@pbs --firstname John --lastname Smith
226 # proxmox-backup-manager user update john@pbs --comment "An example user."
229 .. todo:: Mention how to set password without passing plaintext password as cli argument.
232 The resulting use list looks like this:
234 .. code-block:: console
236 # proxmox-backup-manager user list
237 ┌──────────┬────────┬────────┬───────────┬──────────┬──────────────────┬──────────────────┐
238 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
239 ╞══════════╪════════╪════════╪═══════════╪══════════╪══════════════════╪══════════════════╡
240 │ john@pbs │ 1 │ │ John │ Smith │ john@example.com │ An example user. │
241 ├──────────┼────────┼────────┼───────────┼──────────┼──────────────────┼──────────────────┤
242 │ root@pam │ 1 │ │ │ │ │ Superuser │
243 └──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
245 Newly created users do not have an permissions. Please read the next
246 section to learn how to set access permissions.
248 If you want to disable an user account, you can do that by setting ``--enable`` to ``0``
250 .. code-block:: console
252 # proxmox-backup-manager user update john@pbs --enable 0
254 Or completely remove the users with:
256 .. code-block:: console
258 # proxmox-backup-manager user remove john@pbs
270 The command line client is called :command:`proxmox-backup-client`.
273 Respository Locations
274 ~~~~~~~~~~~~~~~~~~~~~
276 The client uses the following notation to specify a datastore repository
277 on the backup server.
279 [[username@]server:]datastore
281 The default value for ``username`` ist ``root``. If no server is specified, the
282 default is the local host (``localhost``).
284 You can pass the repository with the ``--repository`` command
285 line option, or by setting the ``PBS_REPOSITORY`` environment
289 Environment Variables
290 ~~~~~~~~~~~~~~~~~~~~~
293 The default backup repository.
296 When set, this value is used for the password required for the
299 ``PBS_ENCRYPTION_PASSWORD``
300 When set, this value is used to access the secret encryption key (if
301 protected by password).
303 ``PBS_FINGERPRINT`` When set, this value is used to verify the server
304 certificate (only used if the system CA certificates cannot
305 validate the certificate).
311 Most commands support the ``--output-format`` parameter. It accepts
312 the following values:
314 :``text``: Text format (default). Structured data is rendered as a table.
316 :``json``: JSON (single line).
318 :``json-pretty``: JSON (multiple lines, nicely formatted).
321 Please use the following environment variables to modify output behavior:
323 ``PROXMOX_OUTPUT_FORMAT``
324 Defines the default output format.
326 ``PROXMOX_OUTPUT_NO_BORDER``
327 If set (to any value), do not render table borders.
329 ``PROXMOX_OUTPUT_NO_HEADER``
330 If set (to any value), do not render table headers.
332 .. note:: The ``text`` format is designed to be human readable, and
333 not meant to be parsed by automation tools. Please use the ``json``
334 format if you need to process the output.
337 .. _creating-backups:
342 This section explains how to create a backup from within the machine. This can
343 be a physical host, a virtual machine, or a container. Such backups may contain file
344 and image archives. There are no restrictions in this case.
346 .. note:: If you want to backup virtual machines or containers on Proxmov VE, see :ref:`pve-integration`.
348 For the following example you need to have a backup server set up, working
349 credentials and need to know the repository name.
350 In the following examples we use ``backup-server:store1``.
352 .. code-block:: console
354 # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
355 Starting backup: host/elsa/2019-12-03T09:35:01Z
357 skip mount point: "/boot/efi"
358 skip mount point: "/dev"
359 skip mount point: "/run"
360 skip mount point: "/sys"
361 Uploaded 12129 chunks in 87 seconds (564 MB/s).
362 End Time: 2019-12-03T10:36:29+01:00
364 This will prompt you for a password and then uploads a file archive named
365 ``root.pxar`` containing all the files in the ``/`` directory.
367 .. Caution:: Please note that the proxmox-backup-client does not
368 automatically include mount points. Instead, you will see a short
369 ``skip mount point`` notice for each of them. The idea is to
370 create a separate file archive for each mounted disk. You can
371 explicitly include them using the ``--include-dev`` option
372 (i.e. ``--include-dev /boot/efi``). You can use this option
373 multiple times for each mount point that should be included.
375 The ``--repository`` option can get quite long and is used by all
376 commands. You can avoid having to enter this value by setting the
377 environment variable ``PBS_REPOSITORY``.
379 .. code-block:: console
381 # export PBS_REPOSTORY=backup-server:store1
383 After this you can execute all commands without specifying the ``--repository``
386 One single backup is allowed to contain more than one archive. For example, if
387 you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
389 .. code-block:: console
391 # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
393 This creates a backup of both disks.
395 The backup command takes a list of backup specifications, which
396 include the archive name on the server, the type of the archive, and the
397 archive source at the client. The format is:
399 <archive-name>.<type>:<source-path>
401 Common types are ``.pxar`` for file archives, and ``.img`` for block
402 device images. To create a backup of a block device run the following command:
404 .. code-block:: console
406 # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
408 Excluding files/folders from a backup
409 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
411 Sometimes it is desired to exclude certain files or folders from a backup archive.
412 To tell the Proxmox backup client when and how to ignore files and directories,
413 place a text file called ``.pxarexclude`` in the filesystem hierarchy.
414 Whenever the backup client encounters such a file in a directory, it interprets
415 each line as glob match patterns for files and directories that are to be excluded
418 The file must contain a single glob pattern per line. Empty lines are ignored.
419 The same is true for lines starting with ``#``, which indicates a comment.
420 A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
421 to an explicit inclusion. This makes it possible to exclude all entries in a
422 directory except for a few single files/subdirectories.
423 Lines ending in ``/`` match only on directories.
424 The directory containing the ``.pxarexclude`` file is considered to be the root of
425 the given patterns. It is only possible to match files in this directory and its subdirectories.
427 ``\`` is used to escape special glob characters.
428 ``?`` matches any single character.
429 ``*`` matches any character, including an empty string.
430 ``**`` is used to match subdirectories. It can be used to, for example, exclude
431 all files ending in ``.tmp`` within the directory or subdirectories with the
432 following pattern ``**/*.tmp``.
433 ``[...]`` matches a single character from any of the provided characters within
434 the brackets. ``[!...]`` does the complementary and matches any singe character
435 not contained within the brackets. It is also possible to specify ranges with two
436 characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
437 alphabetic character and ``[0-9]`` matches any one single digit.
439 The order of the glob match patterns defines if a file is included or
440 excluded, later entries win over previous ones.
441 This is also true for match patterns encountered deeper down the directory tree,
442 which can override a previous exclusion.
443 Be aware that excluded directories will **not** be read by the backup client.
444 A ``.pxarexclude`` file in a subdirectory will have no effect.
445 ``.pxarexclude`` files are treated as regular files and will be included in the
448 For example, consider the following directory structure:
450 .. code-block:: console
454 . .. .pxarexclude subfolder0 subfolder1
457 . .. file0 file1 file2 file3 .pxarexclude
460 . .. file0 file1 file2 file3
462 The different ``.pxarexclude`` files contain the following:
464 .. code-block:: console
466 # cat folder/.pxarexclude
471 .. code-block:: console
473 # cat folder/subfolder0/.pxarexclude
476 This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of
477 ``subfolder1`` except ``file2``.
479 Restoring this backup will result in:
481 .. code-block:: console
485 . .. .pxarexclude subfolder0 subfolder1
488 . .. file0 file2 .pxarexclude
496 Proxmox backup supports client side encryption with AES-256 in GCM_
497 mode. First you need to create an encryption key:
499 .. code-block:: console
501 # proxmox-backup-client key create my-backup.key
502 Encryption Key Password: **************
504 The key is password protected by default. If you do not need this
505 extra protection, you can also create it without a password:
507 .. code-block:: console
509 # proxmox-backup-client key create /path/to/my-backup.key --kdf none
512 .. code-block:: console
514 # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
516 Encryption Key Password: **************
520 You can avoid entering the passwords by setting the environment
521 variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
523 .. todo:: Explain master-key
529 The regular creation of backups is a necessary step to avoid data
530 loss. More important, however, is the restoration. It is good practice to perform
531 periodic recovery tests to ensure that you can access the data in
534 First, you need to find the snapshot which you want to restore. The snapshot
535 command gives a list of all snapshots on the server:
537 .. code-block:: console
539 # proxmox-backup-client snapshots
541 host/elsa/2019-12-03T09:30:15Z | 51788646825 | root.pxar catalog.pcat1 index.json
542 host/elsa/2019-12-03T09:35:01Z | 51790622048 | root.pxar catalog.pcat1 index.json
545 You can inspect the catalog to find specific files.
547 .. code-block:: console
549 # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z
551 d "./root.pxar.didx/etc/cifs-utils"
552 l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
553 d "./root.pxar.didx/etc/console-setup"
556 The restore command lets you restore a single archive from the
559 .. code-block:: console
561 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
563 To get the contents of any archive you can restore the ``ìndex.json`` file in the
564 repository and restore it to '-'. This will dump the content to the standard output.
566 .. code-block:: console
568 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
574 If you only want to restore a few individual files, it is often easier
575 to use the interactive recovery shell.
577 .. code-block:: console
579 # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar
580 Starting interactive shell
582 bin boot dev etc home lib lib32
585 The interactive recovery shell is a minimalistic command line interface that
586 utilizes the metadata stored in the catalog to quickly list, navigate and
587 search files in a file archive.
588 To restore files, you can select them individually or match them with a glob
591 Using the catalog for navigation reduces the overhead considerably because only
592 the catalog needs to be downloaded and, optionally, decrypted.
593 The actual chunks are only accessed if the metadata in the catalog is not enough
594 or for the actual restore.
596 Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
597 working directory and list directory contents in the archive.
598 ``pwd`` shows the full path of the current working directory with respect to the
601 Being able to quickly search the contents of the archive is a often needed feature.
602 That's where the catalog is most valuable.
605 .. code-block:: console
607 pxar:/ > find etc/ **/*.txt --select
609 pxar:/ > list-selected
611 pxar:/ > restore-selected /target/path
614 This will find and print all files ending in ``.txt`` located in ``etc/`` or a
615 subdirectory and add the corresponding pattern to the list for subsequent restores.
616 ``list-selected`` shows these patterns and ``restore-selected`` finally restores
617 all files in the archive matching the patterns to ``/target/path`` on the local
618 host. This will scan the whole archive.
620 With ``restore /target/path`` you can restore the sub-archive given by the current
621 working directory to the local target path ``/target/path`` on your host.
622 By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
623 further limited to files matching the pattern.
626 .. code-block:: console
629 pxar:/etc/ > restore /target/ --pattern **/*.conf
632 The above will scan trough all the directories below ``/etc`` and restore all
633 files ending in ``.conf``.
635 .. todo:: Explain interactive restore in more detail
637 Mounting of Archives via FUSE
638 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
640 The :term:`FUSE` implementation for the pxar archive allows you to mount a
641 file archive as a read-only filesystem to a mountpoint on your host.
643 .. code-block:: console
645 # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt
647 bin dev home lib32 libx32 media opt root sbin sys usr
648 boot etc lib lib64 lost+found mnt proc run srv tmp var
650 This allows you to access the full content of the archive in a seamless manner.
652 .. note:: As the FUSE connection needs to fetch and decrypt chunks from the
653 backup servers datastore, this can cause some additional network and CPU
654 load on your host, depending on the operations you perform on the mounted
657 To unmount the filesystem use the ``umount`` command on the mountpoint:
659 .. code-block:: console
666 The client tool prompts you to enter the logon password as soon as you
667 want to access the backup server. The server checks your credentials
668 and responds with a ticket that is valid for two hours. The client
669 tool automatically stores that ticket and uses it for further requests
672 You can also manually trigger this login/logout using the login and
675 .. code-block:: console
677 # proxmox-backup-client login
680 To remove the ticket, issue a logout:
682 .. code-block:: console
684 # proxmox-backup-client logout
687 Pruning and Removing Backups
688 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
690 You can manually delete a backup snapshot using the ``forget``
693 .. code-block:: console
695 # proxmox-backup-client forget <snapshot>
698 .. caution:: This command removes all archives in this backup
699 snapshot. They will be inaccessible and unrecoverable.
702 The manual removal is sometimes required, but normally the prune
703 command is used to systematically delete older backups. Prune lets
704 you specify which backup snapshots you want to keep. The
705 following retention options are available:
708 Keep the last ``<N>`` backup snapshots.
710 ``--keep-hourly <N>``
711 Keep backups for the last ``<N>`` hours. If there is more than one
712 backup for a single hour, only the latest is kept.
715 Keep backups for the last ``<N>`` days. If there is more than one
716 backup for a single day, only the latest is kept.
718 ``--keep-weekly <N>``
719 Keep backups for the last ``<N>`` weeks. If there is more than one
720 backup for a single week, only the latest is kept.
722 .. note:: Weeks start on Monday and end on Sunday. The software
723 uses the `ISO week date`_ system and handles weeks at
724 the end of the year correctly.
726 ``--keep-monthly <N>``
727 Keep backups for the last ``<N>`` months. If there is more than one
728 backup for a single month, only the latest is kept.
730 ``--keep-yearly <N>``
731 Keep backups for the last ``<N>`` years. If there is more than one
732 backup for a single year, only the latest is kept.
734 The retention options are processed in the order given above. Each option
735 only covers backups within its time period. The next option does not take care
736 of already covered backups. It will only consider older backups.
738 For example, the ``--keep-monthly`` option does not consider any backup that is
739 younger than one month.
741 .. todo:: check if the previous statement is correct
743 Unfinished and incomplete backups will be removed by the prune command unless
744 they are newer than the last successful backup. In this case, the last failed
747 .. code-block:: console
749 # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3
752 You can use the ``--dry-run`` option to test your settings. This only
753 shows the list of existing snapshots and which action prune would take.
755 .. code-block:: console
757 # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
758 retention options: --keep-daily 1 --keep-weekly 3
759 Testing prune on store "store2" group "host/elsa"
760 host/elsa/2019-12-04T13:20:37Z keep
761 host/elsa/2019-12-03T09:35:01Z remove
762 host/elsa/2019-11-22T11:54:47Z keep
763 host/elsa/2019-11-21T12:36:25Z remove
764 host/elsa/2019-11-10T10:42:20Z keep
767 .. note:: Neither the ``prune`` command nor the ``forget`` command free space
768 in the chunk-store. The chunk-store still contains the data blocks. To free
769 space you need to perform :ref:`garbage-collection`.
772 .. _garbage-collection:
777 The ``prune`` command removes only the backup index files, not the data
778 from the data store. This task is left to the garbage collection
779 command. It is recommended to carry out garbage collection on a regular basis.
781 The garbage collection works in two phases. In the first phase, all
782 data blocks that are still in use are marked. In the second phase,
783 unused data blocks are removed.
785 .. note:: This command needs to read all existing backup index files
786 and touches the complete chunk-store. This can take a long time
787 depending on the number of chunks and the speed of the underlying
791 .. code-block:: console
793 # proxmox-backup-client garbage-collect
794 starting garbage collection on store store2
795 Start GC phase1 (mark used chunks)
796 Start GC phase2 (sweep unused chunks)
797 percentage done: 1, chunk count: 219
798 percentage done: 2, chunk count: 453
800 percentage done: 99, chunk count: 21188
801 Removed bytes: 411368505
803 Original data bytes: 327160886391
804 Disk bytes: 52767414743 (16 %)
806 Average chunk size: 2486565
810 .. todo:: howto run garbage-collection at regular intervalls (cron)
815 `Proxmox VE`_ integration
816 -------------------------
819 .. include:: command-line-tools.rst
821 .. include:: services.rst
823 .. include host system admin at the end
825 .. include:: sysadmin.rst