4 The administration guide.
13 When doing deduplication, there are different strategies to get
14 optimal results in terms of performance and/or deduplication rates.
15 Depending on the type of data, one can split data into fixed or variable
18 Fixed sized chunking needs almost no CPU performance, and is used to
19 backup virtual machine images.
21 Variable sized chunking needs more CPU power, but is essential to get
22 good deduplication rates for file archives.
24 Therefore, the backup server supports both strategies.
27 File Archives: ``<name>.pxar``
28 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
30 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
32 A file archive stores a whole directory tree. Content is stored using
33 the :ref:`pxar-format`, split into variable sized chunks. The format
34 is specially optimized to achieve good deduplication rates.
37 Image Archives: ``<name>.img``
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
40 This is used for virtual machine images and other large binary
41 data. Content is split into fixed sized chunks.
47 This type is used to store smaller (< 16MB) binaries like
48 configuration data. Larger files should be stored as image archive.
50 .. caution:: Please do not store all files as BLOBs. Instead, use the
51 file archive to store whole directory trees.
54 Catalog File: ``catalog.pcat1``
55 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
57 The catalog file is basically an index for file archive. It contains
58 the list of files, and is used to speedup search operations.
61 The Manifest: ``index.json``
62 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64 The manifest contains the list of all backup files, including
65 file sizes and checksums. It is used to verify the consistency of a
72 The backup server groups backups by *type*, where *type* is one of:
75 This type is used for :term:`virtual machine`\ s. Typically
76 contains the virtual machine configuration and an image archive
80 This type is used for :term:`container`\ s. Contains the container
81 configuration and a single file archive for the container content.
84 This type is used for physical host, or if you want to run backups
85 manually from inside virtual machines or containers. Such backups
86 may contain file and image archives (no restrictions here).
92 An unique ID. Usually the virtual machine or container ID. ``host``
93 type backups normally use the hostname.
99 The time when the backup was made.
105 We call the triplet ``<type>/<ID>/<time>`` a backup snapshot. It
106 uniquely identifies a specific backup within a datastore.
108 .. code-block:: console
109 :caption: Backup Snapshot Examples
111 vm/104/2019-10-09T08:01:06Z
112 host/elsa/2019-11-08T09:48:14Z
114 As you can see, the time is formatted as RFC3399_ using Coordinated
115 Universal Time (UTC_, identified by the trailing *Z*).
121 A datastore is a place to store backups. The current implementation
122 uses a directory inside a standard unix file system (``ext4``, ``xfs``
123 or ``zfs``) to store backup data.
125 Datastores are identified by a simple *ID*. You can configure that
126 when setting up the backup server.
129 Backup Server Management
130 ------------------------
132 The command line tool to configure and manage the server is called
133 :command:`proxmox-backup-manager`.
136 Datastore Configuration
137 ~~~~~~~~~~~~~~~~~~~~~~~
139 A :term:`datastore` is a place to store backups. You can configure
140 several datastores, but you need at least one of them. The datastore is identified by a simple `name` and point to a directory.
142 The following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
144 .. code-block:: console
146 # proxmox-backup-manager datastore create store1 /backup/disk1/store1
148 To list existing datastores use:
150 .. code-block:: console
152 # proxmox-backup-manager datastore list
153 store1 /backup/disk1/store1
155 Finally, it is also possible to remove the datastore configuration:
157 .. code-block:: console
159 # proxmox-backup-manager datastore remove store1
161 .. note:: Above command removes the datastore configuration. It does
162 not delete any data from the underlying directory.
168 .. todo:: Add datastore file layout example
174 The command line client is called :command:`proxmox-backup-client`.
177 Respository Locations
178 ~~~~~~~~~~~~~~~~~~~~~
180 The client uses a special repository notation to specify a datastore
181 on the backup server.
183 [[username@]server:]datastore
185 If you do not specify a ``username`` the default is ``root@pam``. The
186 default for server is to use the local host (``localhost``).
188 You can pass the repository by setting the ``--repository`` command
189 line options, or by setting the ``PBS_REPOSITORY`` environment
193 Environment Variables
194 ~~~~~~~~~~~~~~~~~~~~~~
197 The default backup repository.
200 When set, this value is used for the password required for the
203 ``PBS_ENCRYPTION_PASSWORD``
204 When set, this value is used to access the secret encryption key (if
205 protected by password).
211 This section explains how to create backup on physical host, or from
212 inside virtual machines or containers. Such backups may contain file
213 and image archives (no restrictions here).
215 .. note:: If you want to backup virtual machines or containers see :ref:`pve-integration`.
217 The prerequisite is that you have already set up (or can access) a
218 backup server. It is assumed that you know the repository name and
219 credentials. In the following examples we simply use ``backup-server:store1``.
221 .. code-block:: console
223 # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
224 Starting backup: host/elsa/2019-12-03T09:35:01Z
226 skip mount point: "/boot/efi"
227 skip mount point: "/dev"
228 skip mount point: "/run"
229 skip mount point: "/sys"
230 Uploaded 12129 chunks in 87 seconds (564 MB/s).
231 End Time: 2019-12-03T10:36:29+01:00
233 This will prompt you for a password and then uploads a file archive named
234 ``root.pxar`` containing all the files in the ``/`` directory.
236 .. Caution:: Please note that proxmox-backup-client does not
237 automatically include mount points. Instead, you will see a short
238 ``skip mount point`` notice for each of them. The idea is that you
239 create a separate file archive for each mounted disk. You can also
240 explicitly include them using the ``--include-dev`` option
241 (i.e. ``--include-dev /boot/efi``). You can use this option
242 multiple times, once for each mount point you want to include.
244 The ``--repository`` option is sometimes quite long and is used by all
245 commands. You can avoid having to enter this value by setting the
246 environment variable ``PBS_REPOSITORY``.
248 .. code-block:: console
250 # export PBS_REPOSTORY=backup-server:store1
252 You can then execute all commands without specifying the ``--repository``
255 One signle backup is allowed to contain more than one archive. For example, assume you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
257 .. code-block:: console
259 # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
261 This create a backup of both disks.
263 The backup command takes a list of backup specifications, which
264 include archive name on the server, the type of the archive, and the
265 archive source at the client. The format is quite simple to understand:
267 <archive-name>.<type>:<source-path>
269 Common types are ``.pxar`` for file archives, and ``.img`` for block
270 device images. Thus it is quite easy to create a backup for a block
273 .. code-block:: console
275 # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
281 Proxmox backup support client side encryption using AES-256 in GCM_
282 mode. You first need to create an encryption key in order to use that:
284 .. code-block:: console
286 # proxmox-backup-client key create my-backup.key
287 Encryption Key Password: **************
289 The key is password protected by default. If you do not need this
290 extra protection, you can also create it without a password:
292 .. code-block:: console
294 # proxmox-backup-client key create /path/to/my-backup.key --kdf none
297 .. code-block:: console
299 # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
301 Encryption Key Password: **************
305 You can avoid having to enter the passwords by setting the environment
306 variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
308 .. todo:: Explain master-key
314 The regular creation of backups is a necessary step to avoid data
315 loss. More important, however, is the restoration. Be sure to perform
316 periodic recovery tests to ensure that you can access your data in
319 First, you need to find the snapshot you want to restore. The snapshot
320 command gives you a list of all snapshots on the server:
322 .. code-block:: console
324 # proxmox-backup-client snapshots
326 host/elsa/2019-12-03T09:30:15Z | 51788646825 | root.pxar catalog.pcat1 index.json
327 host/elsa/2019-12-03T09:35:01Z | 51790622048 | root.pxar catalog.pcat1 index.json
330 You can also inspect the catalog to find specific files.
332 .. code-block:: console
334 # proxmox-backup-client catalog host/elsa/2019-12-03T09:35:01Z
336 d "./root.pxar.didx/etc/cifs-utils"
337 l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
338 d "./root.pxar.didx/etc/console-setup"
341 The restore command lets you restore a single archive from the
344 .. code-block:: console
346 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
348 You can instead simply download the contents of any archive using '-'
349 instead of ``/target/path``. This dumps the content to standard
352 .. code-block:: console
354 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
360 If you only want to restore a few individual files, it is often easier
361 to use the interactive recovery shell.
363 .. code-block:: console
365 # proxmox-backup-client shell host/elsa/2019-12-03T09:35:01Z root.pxar
366 Starting interactive shell
368 bin boot dev etc home lib lib32
371 .. todo:: Explain interactive restore
377 The client tool prompts you to enter the logon password as soon as you
378 want to access the backup server. The server checks your credentials
379 and responds with a ticket that is valid for two hours. The client
380 tool automatically stores that ticket and use it for further requests
383 You can also manually trigger this login/logout using the login and
386 .. code-block:: console
388 # proxmox-backup-client login
391 To remove the ticket, simply issue a logout:
393 .. code-block:: console
395 # proxmox-backup-client logout
400 `Proxmox VE`_ integration
401 -------------------------
404 .. include:: command-line-tools.rst
406 .. include:: services.rst
408 .. include host system admin at the end
410 .. include:: sysadmin.rst