4 The administration guide.
13 When doing deduplication, there are different strategies to get
14 optimal results in terms of performance and/or deduplication rates.
15 Depending on the type of data, one can split data into fixed or variable
18 Fixed sized chunking needs almost no CPU performance, and is used to
19 backup virtual machine images.
21 Variable sized chunking needs more CPU power, but is essential to get
22 good deduplication rates for file archives.
24 Therefore, the backup server supports both strategies.
27 File Archives: ``<name>.pxar``
28 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
30 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
32 A file archive stores a whole directory tree. Content is stored using
33 the :ref:`pxar-format`, split into variable sized chunks. The format
34 is specially optimized to achieve good deduplication rates.
37 Image Archives: ``<name>.img``
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
40 This is used for virtual machine images and other large binary
41 data. Content is split into fixed sized chunks.
47 This type is used to store smaller (< 16MB) binaries like
48 configuration data. Larger files should be stored as image archive.
50 .. caution:: Please do not store all files as BLOBs. Instead, use the
51 file archive to store whole directory trees.
54 Catalog File: ``catalog.pcat1``
55 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
57 The catalog file is basically an index for file archive. It contains
58 the list of files, and is used to speedup search operations.
61 The Manifest: ``index.json``
62 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64 The manifest contains the list of all backup files, including
65 file sizes and checksums. It is used to verify the consistency of a
72 The backup server groups backups by *type*, where *type* is one of:
75 This type is used for :term:`virtual machine`\ s. Typically
76 contains the virtual machine configuration and an image archive
80 This type is used for :term:`container`\ s. Contains the container
81 configuration and a single file archive for the container content.
84 This type is used for physical host, or if you want to run backups
85 manually from inside virtual machines or containers. Such backups
86 may contain file and image archives (no restrictions here).
92 An unique ID. Usually the virtual machine or container ID. ``host``
93 type backups normally use the hostname.
99 The time when the backup was made.
105 We call the tuple ``<type>/<ID>`` a backup group. Such group
106 may contains one or more backup snapshots.
112 We call the triplet ``<type>/<ID>/<time>`` a backup snapshot. It
113 uniquely identifies a specific backup within a datastore.
115 .. code-block:: console
116 :caption: Backup Snapshot Examples
118 vm/104/2019-10-09T08:01:06Z
119 host/elsa/2019-11-08T09:48:14Z
121 As you can see, the time is formatted as RFC3399_ using Coordinated
122 Universal Time (UTC_, identified by the trailing *Z*).
128 A datastore is a place to store backups. The current implementation
129 uses a directory inside a standard unix file system (``ext4``, ``xfs``
130 or ``zfs``) to store backup data.
132 Datastores are identified by a simple *ID*. You can configure that
133 when setting up the backup server.
136 Backup Server Management
137 ------------------------
139 The command line tool to configure and manage the server is called
140 :command:`proxmox-backup-manager`.
143 Datastore Configuration
144 ~~~~~~~~~~~~~~~~~~~~~~~
146 A :term:`datastore` is a place to store backups. You can configure
147 several datastores, but you need at least one of them. The datastore is identified by a simple `name` and point to a directory.
149 The following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
151 .. code-block:: console
153 # proxmox-backup-manager datastore create store1 /backup/disk1/store1
155 To list existing datastores use:
157 .. code-block:: console
159 # proxmox-backup-manager datastore list
160 store1 /backup/disk1/store1
162 Finally, it is also possible to remove the datastore configuration:
164 .. code-block:: console
166 # proxmox-backup-manager datastore remove store1
168 .. note:: Above command removes the datastore configuration. It does
169 not delete any data from the underlying directory.
175 .. todo:: Add datastore file layout example
181 The command line client is called :command:`proxmox-backup-client`.
184 Respository Locations
185 ~~~~~~~~~~~~~~~~~~~~~
187 The client uses a special repository notation to specify a datastore
188 on the backup server.
190 [[username@]server:]datastore
192 If you do not specify a ``username`` the default is ``root@pam``. The
193 default for server is to use the local host (``localhost``).
195 You can pass the repository by setting the ``--repository`` command
196 line options, or by setting the ``PBS_REPOSITORY`` environment
200 Environment Variables
201 ~~~~~~~~~~~~~~~~~~~~~~
204 The default backup repository.
207 When set, this value is used for the password required for the
210 ``PBS_ENCRYPTION_PASSWORD``
211 When set, this value is used to access the secret encryption key (if
212 protected by password).
218 This section explains how to create backup on physical host, or from
219 inside virtual machines or containers. Such backups may contain file
220 and image archives (no restrictions here).
222 .. note:: If you want to backup virtual machines or containers see :ref:`pve-integration`.
224 The prerequisite is that you have already set up (or can access) a
225 backup server. It is assumed that you know the repository name and
226 credentials. In the following examples we simply use ``backup-server:store1``.
228 .. code-block:: console
230 # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
231 Starting backup: host/elsa/2019-12-03T09:35:01Z
233 skip mount point: "/boot/efi"
234 skip mount point: "/dev"
235 skip mount point: "/run"
236 skip mount point: "/sys"
237 Uploaded 12129 chunks in 87 seconds (564 MB/s).
238 End Time: 2019-12-03T10:36:29+01:00
240 This will prompt you for a password and then uploads a file archive named
241 ``root.pxar`` containing all the files in the ``/`` directory.
243 .. Caution:: Please note that proxmox-backup-client does not
244 automatically include mount points. Instead, you will see a short
245 ``skip mount point`` notice for each of them. The idea is that you
246 create a separate file archive for each mounted disk. You can also
247 explicitly include them using the ``--include-dev`` option
248 (i.e. ``--include-dev /boot/efi``). You can use this option
249 multiple times, once for each mount point you want to include.
251 The ``--repository`` option is sometimes quite long and is used by all
252 commands. You can avoid having to enter this value by setting the
253 environment variable ``PBS_REPOSITORY``.
255 .. code-block:: console
257 # export PBS_REPOSTORY=backup-server:store1
259 You can then execute all commands without specifying the ``--repository``
262 One signle backup is allowed to contain more than one archive. For example, assume you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
264 .. code-block:: console
266 # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
268 This create a backup of both disks.
270 The backup command takes a list of backup specifications, which
271 include archive name on the server, the type of the archive, and the
272 archive source at the client. The format is quite simple to understand:
274 <archive-name>.<type>:<source-path>
276 Common types are ``.pxar`` for file archives, and ``.img`` for block
277 device images. Thus it is quite easy to create a backup for a block
280 .. code-block:: console
282 # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
288 Proxmox backup support client side encryption using AES-256 in GCM_
289 mode. You first need to create an encryption key in order to use that:
291 .. code-block:: console
293 # proxmox-backup-client key create my-backup.key
294 Encryption Key Password: **************
296 The key is password protected by default. If you do not need this
297 extra protection, you can also create it without a password:
299 .. code-block:: console
301 # proxmox-backup-client key create /path/to/my-backup.key --kdf none
304 .. code-block:: console
306 # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
308 Encryption Key Password: **************
312 You can avoid having to enter the passwords by setting the environment
313 variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
315 .. todo:: Explain master-key
321 The regular creation of backups is a necessary step to avoid data
322 loss. More important, however, is the restoration. Be sure to perform
323 periodic recovery tests to ensure that you can access your data in
326 First, you need to find the snapshot you want to restore. The snapshot
327 command gives you a list of all snapshots on the server:
329 .. code-block:: console
331 # proxmox-backup-client snapshots
333 host/elsa/2019-12-03T09:30:15Z | 51788646825 | root.pxar catalog.pcat1 index.json
334 host/elsa/2019-12-03T09:35:01Z | 51790622048 | root.pxar catalog.pcat1 index.json
337 You can also inspect the catalog to find specific files.
339 .. code-block:: console
341 # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z
343 d "./root.pxar.didx/etc/cifs-utils"
344 l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
345 d "./root.pxar.didx/etc/console-setup"
348 The restore command lets you restore a single archive from the
351 .. code-block:: console
353 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
355 You can instead simply download the contents of any archive using '-'
356 instead of ``/target/path``. This dumps the content to standard
359 .. code-block:: console
361 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
367 If you only want to restore a few individual files, it is often easier
368 to use the interactive recovery shell.
370 .. code-block:: console
372 # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar
373 Starting interactive shell
375 bin boot dev etc home lib lib32
378 The interactive recovery shell is a minimalistic command line interface that
379 utilizes the metadata stored in the catalog for you to quickly list, navigate and
380 search files contained within a file archive.
381 You can select individual files as well as select files matched by a glob pattern
384 The use of the catalog for navigation reduces the overhead otherwise caused by
385 network traffic and decryption, as instead of downloading and decrypting
386 individual encrypted chunks from the chunk store to access the metadata, we only
387 need to download and decrypt the catalog.
388 The actual chunks are only accessed if the metadata in the catalog is not enough
389 or for the actual restore.
391 Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
392 working directory and list directory contents of the archive.
393 ``pwd`` shows the full path of the current working directory with respect to the
396 Being able to quickly search the contents of the archive is a often needed feature.
397 That's where the catalog is most valuable.
400 .. code-block:: console
402 pxar:/ > find etc/ **/*.txt --select
404 pxar:/ > list-selected
406 pxar:/ > restore-selected /target/path
409 This will find and print all files ending in ``.txt`` located in ``etc/`` or a
410 subdirectory and add the corresponding pattern to the list for subsequent restores.
411 ``list-selected`` shows these patterns and ``restore-selected`` finally restores
412 all files in the archive matching the patterns to ``/target/path`` on the local
413 host. This will scan the whole archive.
415 With ``restore /target/path`` you can restore the sub-archive given by the current
416 working directory to the local target path ``/target/path`` on your host.
417 By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
418 further limited to files matching the pattern.
421 .. code-block:: console
424 pxar:/etc/ > restore /target/ --pattern **/*.conf
427 The above will scan trough all the directories below ``/etc`` and restore all
428 files ending in ``.conf``.
430 .. todo:: Explain interactive restore in more detail
432 Mounting of Archives via FUSE
433 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
435 The :term:`FUSE` implementation for the pxar archive allows you to mount a
436 file archive as a read-only filesystem to a mountpoint on your host.
438 .. code-block:: console
440 # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt
442 bin dev home lib32 libx32 media opt root sbin sys usr
443 boot etc lib lib64 lost+found mnt proc run srv tmp var
445 This allows you to access the full content of the archive in a seamless manner.
447 .. note:: As the FUSE connection needs to fetch and decrypt chunks from the
448 backup servers datastore, this can cause some additional network and CPU
449 load on your host, depending on the operations you perform on the mounted
452 To unmount the filesystem simply use the ``umount`` command on the mountpoint:
454 .. code-block:: console
461 The client tool prompts you to enter the logon password as soon as you
462 want to access the backup server. The server checks your credentials
463 and responds with a ticket that is valid for two hours. The client
464 tool automatically stores that ticket and use it for further requests
467 You can also manually trigger this login/logout using the login and
470 .. code-block:: console
472 # proxmox-backup-client login
475 To remove the ticket, simply issue a logout:
477 .. code-block:: console
479 # proxmox-backup-client logout
482 Pruning and Removing Backups
483 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
485 You can manually delete a backup snapshot using the ``forget``
488 .. code-block:: console
490 # proxmox-backup-client forget <snapshot>
493 .. caution:: This command removes all the archives in this backup
494 snapshot so that they are inaccessible and unrecoverable.
497 Such manual removal is sometimes required, but normally the prune
498 command is used to systematically delete older backups. Prune lets
499 you specify which backup snapshots you want to keep. There are the
500 following retention options:
503 Keep the last ``<N>`` backup snapshots.
505 ``--keep-hourly <N>``
506 Keep backups for the last ``<N>`` different hours. If there is more than one
507 backup for a single hour, only the latest one is kept.
510 Keep backups for the last ``<N>`` different days. If there is more than one
511 backup for a single day, only the latest one is kept.
513 ``--keep-weekly <N>``
514 Keep backups for the last ``<N>`` different weeks. If there is more than one
515 backup for a single week, only the latest one is kept.
517 .. note:: The weeks start on Monday and end on Sunday. The software
518 uses the `ISO week date`_ system and correctly handles weeks at
521 ``--keep-monthly <N>``
522 Keep backups for the last ``<N>`` different months. If there is more than one
523 backup for a single month, only the latest one is kept.
525 ``--keep-yearly <N>``
526 Keep backups for the last ``<N>`` different years. If there is more than one
527 backup for a single year, only the latest one is kept.
530 Those retention options are processed in the order given above. Each
531 option covers a specific period of time. We say that backups within
532 this period are covered by this option. The next option does not take
533 care of already covered backups and only considers older backups.
535 The prune command also looks for unfinished and incomplete backups and
536 removes them unless they are newer than the last successful backup. In
537 this case, the last failed backup is retained.
539 .. code-block:: console
541 # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3
544 You can use the ``--dry-run`` option to test your settings. This just
545 shows the list of existing snapshots and what action prune would take
548 .. code-block:: console
550 # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
551 retention options: --keep-daily 1 --keep-weekly 3
552 Testing prune on store "store2" group "host/elsa"
553 host/elsa/2019-12-04T13:20:37Z keep
554 host/elsa/2019-12-03T09:35:01Z remove
555 host/elsa/2019-11-22T11:54:47Z keep
556 host/elsa/2019-11-21T12:36:25Z remove
557 host/elsa/2019-11-10T10:42:20Z keep
560 .. note:: Neither the ``prune`` command nor the ``forget`` command free space
561 in the chunk-store. The chunk-store still contains the data blocks
562 unless you are performing :ref:`garbage-collection`.
565 .. _garbage-collection:
570 The ``prune`` command removes only the backup index files, not the data
571 from the data store. This task is left to the garbage collection
572 command. It is therefore recommended to carry out garbage collection
575 The garbage collection works in two phases. In the first phase, all
576 data blocks that are still in use are marked. In the second phase,
577 unused data blocks are removed.
579 .. note:: This command needs to read all existing backup index files
580 and touches the complete chunk store. This can take a long time
581 depending on the number of chunk and the speed of the underlying
585 .. code-block:: console
587 # proxmox-backup-client garbage-collect
588 starting garbage collection on store store2
589 Start GC phase1 (mark used chunks)
590 Start GC phase2 (sweep unused chunks)
591 percentage done: 1, chunk count: 219
592 percentage done: 2, chunk count: 453
594 percentage done: 99, chunk count: 21188
595 Removed bytes: 411368505
597 Original data bytes: 327160886391
598 Disk bytes: 52767414743 (16 %)
600 Average chunk size: 2486565
604 .. todo:: howto run garbage-collection at regular intervalls (cron)
609 `Proxmox VE`_ integration
610 -------------------------
613 .. include:: command-line-tools.rst
615 .. include:: services.rst
617 .. include host system admin at the end
619 .. include:: sysadmin.rst