7 When doing deduplication, there are different strategies to get
8 optimal results in terms of performance and/or deduplication rates.
9 Depending on the type of data, it can be split into *fixed* or *variable*
12 Fixed sized chunking requires minimal CPU power, and is used to
13 backup virtual machine images.
15 Variable sized chunking needs more CPU power, but is essential to get
16 good deduplication rates for file archives.
18 The Proxmox Backup Server supports both strategies.
21 Image Archives: ``<name>.img``
22 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
24 This is used for virtual machine images and other large binary
25 data. Content is split into fixed-sized chunks.
28 File Archives: ``<name>.pxar``
29 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
31 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
33 A file archive stores a full directory tree. Content is stored using
34 the :ref:`pxar-format`, split into variable-sized chunks. The format
35 is optimized to achieve good deduplication rates.
41 This type is used to store smaller (< 16MB) binary data such as
42 configuration files. Larger files should be stored as image archive.
44 .. caution:: Please do not store all files as BLOBs. Instead, use the
45 file archive to store whole directory trees.
48 Catalog File: ``catalog.pcat1``
49 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
51 The catalog file is an index for file archives. It contains
52 the list of files and is used to speed up search operations.
55 The Manifest: ``index.json``
56 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
58 The manifest contains the list of all backup files, their
59 sizes and checksums. It is used to verify the consistency of a
66 The backup server groups backups by *type*, where *type* is one of:
69 This type is used for :term:`virtual machine`\ s. Typically
70 consists of the virtual machine's configuration file and an image archive
74 This type is used for :term:`container`\ s. Consists of the container's
75 configuration and a single file archive for the filesystem content.
78 This type is used for backups created from within the backed up machine.
79 Typically this would be a physical host but could also be a virtual machine
80 or container. Such backups may contain file and image archives, there are no restrictions in this regard.
86 A unique ID. Usually the virtual machine or container ID. ``host``
87 type backups normally use the hostname.
93 The time when the backup was made.
99 The tuple ``<type>/<ID>`` is called a backup group. Such a group
100 may contain one or more backup snapshots.
106 The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It
107 uniquely identifies a specific backup within a datastore.
109 .. code-block:: console
110 :caption: Backup Snapshot Examples
112 vm/104/2019-10-09T08:01:06Z
113 host/elsa/2019-11-08T09:48:14Z
115 As you can see, the time format is RFC3399_ with Coordinated
116 Universal Time (UTC_, identified by the trailing *Z*).