4 .. The administration guide.
5 .. todo:: either add a bit more explanation or remove the previous sentence
13 When doing deduplication, there are different strategies to get
14 optimal results in terms of performance and/or deduplication rates.
15 Depending on the type of data, it can be split into *fixed* or *variable*
18 Fixed sized chunking requires minimal CPU power, and is used to
19 backup virtual machine images.
21 Variable sized chunking needs more CPU power, but is essential to get
22 good deduplication rates for file archives.
24 The Proxmox Backup Server supports both strategies.
27 File Archives: ``<name>.pxar``
28 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
30 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
32 A file archive stores a full directory tree. Content is stored using
33 the :ref:`pxar-format`, split into variable-sized chunks. The format
34 is optimized to achieve good deduplication rates.
37 Image Archives: ``<name>.img``
38 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
40 This is used for virtual machine images and other large binary
41 data. Content is split into fixed-sized chunks.
47 This type is used to store smaller (< 16MB) binary data such as
48 configuration files. Larger files should be stored as image archive.
50 .. caution:: Please do not store all files as BLOBs. Instead, use the
51 file archive to store whole directory trees.
54 Catalog File: ``catalog.pcat1``
55 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
57 The catalog file is an index for file archives. It contains
58 the list of files and is used to speed up search operations.
61 The Manifest: ``index.json``
62 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64 The manifest contains the list of all backup files, their
65 sizes and checksums. It is used to verify the consistency of a
72 The backup server groups backups by *type*, where *type* is one of:
75 This type is used for :term:`virtual machine`\ s. Typically
76 consists of the virtual machine's configuration file and an image archive
80 This type is used for :term:`container`\ s. Consists of the container's
81 configuration and a single file archive for the filesystem content.
84 This type is used for backups created from within the backed up machine.
85 Typically this would be a physical host but could also be a virtual machine
86 or container. Such backups may contain file and image archives, there are no restrictions in this regard.
92 A unique ID. Usually the virtual machine or container ID. ``host``
93 type backups normally use the hostname.
99 The time when the backup was made.
105 The tuple ``<type>/<ID>`` is called a backup group. Such a group
106 may contain one or more backup snapshots.
112 The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It
113 uniquely identifies a specific backup within a datastore.
115 .. code-block:: console
116 :caption: Backup Snapshot Examples
118 vm/104/2019-10-09T08:01:06Z
119 host/elsa/2019-11-08T09:48:14Z
121 As you can see, the time format is RFC3399_ with Coordinated
122 Universal Time (UTC_, identified by the trailing *Z*).
124 Backup Server Management
125 ------------------------
127 The command line tool to configure and manage the backup server is called
128 :command:`proxmox-backup-manager`.
135 A datastore is a place where backups are stored. The current implementation
136 uses a directory inside a standard unix file system (``ext4``, ``xfs``
137 or ``zfs``) to store the backup data.
139 Datastores are identified by a simple *ID*. You can configure it
140 when setting up the backup server.
142 .. note:: The `File Layout`_ requires the file system to support at least *65538*
143 subdirectories per directory. That number comes from the 2\ :sup:`16`
144 pre-created chunk namespace directories, and the ``.`` and ``..`` default
145 directory entries. This requirement excludes certain filesystems and
146 filesystem configuration from being supported for a datastore. For example,
147 ``ext3`` as a whole or ``ext4`` with the ``dir_nlink`` feature manually disabled.
152 .. image:: images/screenshots/pbs-gui-disks.png
157 Proxmox Backup Server comes with a set of disk utilities, which are
158 accessed using the ``disk`` subcommand. This subcommand allows you to initialize
159 disks, create various filesystems, and get information about the disks.
161 To view the disks connected to the system, navigate to **Administration ->
162 Disks** in the web interface or use the ``list`` subcommand of
165 .. code-block:: console
167 # proxmox-backup-manager disk list
168 ┌──────┬────────┬─────┬───────────┬─────────────┬───────────────┬─────────┬────────┐
169 │ name │ used │ gpt │ disk-type │ size │ model │ wearout │ status │
170 ╞══════╪════════╪═════╪═══════════╪═════════════╪═══════════════╪═════════╪════════╡
171 │ sda │ lvm │ 1 │ hdd │ 34359738368 │ QEMU_HARDDISK │ - │ passed │
172 ├──────┼────────┼─────┼───────────┼─────────────┼───────────────┼─────────┼────────┤
173 │ sdb │ unused │ 1 │ hdd │ 68719476736 │ QEMU_HARDDISK │ - │ passed │
174 ├──────┼────────┼─────┼───────────┼─────────────┼───────────────┼─────────┼────────┤
175 │ sdc │ unused │ 1 │ hdd │ 68719476736 │ QEMU_HARDDISK │ - │ passed │
176 └──────┴────────┴─────┴───────────┴─────────────┴───────────────┴─────────┴────────┘
178 To initialize a disk with a new GPT, use the ``initialize`` subcommand:
180 .. code-block:: console
182 # proxmox-backup-manager disk initialize sdX
184 .. image:: images/screenshots/pbs-gui-disks-dir-create.png
187 :alt: Create a directory
189 You can create an ``ext4`` or ``xfs`` filesystem on a disk using ``fs
190 create``, or by navigating to **Administration -> Disks -> Directory** in the
191 web interface and creating one from there. The following command creates an
192 ``ext4`` filesystem and passes the ``--add-datastore`` parameter, in order to
193 automatically create a datastore on the disk (in this case ``sdd``). This will
194 create a datastore at the location ``/mnt/datastore/store1``:
198 .. code-block:: console
200 # proxmox-backup-manager disk fs create store1 --disk sdd --filesystem ext4 --add-datastore true
201 create datastore 'store1' on disk sdd
207 .. image:: images/screenshots/pbs-gui-disks-zfs-create.png
210 :alt: Create a directory
212 You can also create a ``zpool`` with various raid levels from **Administration
213 -> Disks -> Zpool** in the web interface, or by using ``zpool create``. The command
214 below creates a mirrored ``zpool`` using two disks (``sdb`` & ``sdc``) and
215 mounts it on the root directory (default):
219 .. code-block:: console
221 # proxmox-backup-manager disk zpool create zpool1 --devices sdb,sdc --raidlevel mirror
222 create Mirror zpool 'zpool1' on devices 'sdb,sdc'
223 # "zpool" "create" "-o" "ashift=12" "zpool1" "mirror" "sdb" "sdc"
228 You can also pass the ``--add-datastore`` parameter here, to automatically
229 create a datastore from the disk.
231 You can use ``disk fs list`` and ``disk zpool list`` to keep track of your
232 filesystems and zpools respectively.
234 If a disk supports S.M.A.R.T. capability, and you have this enabled, you can
235 display S.M.A.R.T. attributes from the web interface or by using the command:
237 .. code-block:: console
239 # proxmox-backup-manager disk smart-attributes sdX
242 Datastore Configuration
243 ~~~~~~~~~~~~~~~~~~~~~~~
245 .. image:: images/screenshots/pbs-gui-datastore.png
248 :alt: Datastore Overview
250 You can configure multiple datastores. Minimum one datastore needs to be
251 configured. The datastore is identified by a simple *name* and points to a
252 directory on the filesystem. Each datastore also has associated retention
253 settings of how many backup snapshots for each interval of ``hourly``,
254 ``daily``, ``weekly``, ``monthly``, ``yearly`` as well as a time-independent
255 number of backups to keep in that store. :ref:`Pruning <pruning>` and
256 :ref:`garbage collection <garbage-collection>` can also be configured to run
257 periodically based on a configured :term:`schedule` per datastore.
261 .. image:: images/screenshots/pbs-gui-datastore-create-general.png
264 :alt: Create a data store
266 You can create a new datastore from the web GUI, by navigating to **Datastore** in
267 the menu tree and clicking **Create**. Here:
269 * *Name* refers to the name of the datastore
270 * *Backing Path* is the path to the directory upon which you want to create the
272 * *GC Schedule* refers to the time and intervals at which garbage collection
274 * *Prune Schedule* refers to the frequency at which pruning takes place
275 * *Prune Options* set the amount of backups which you would like to keep (see :ref:`Pruning <pruning>`).
277 Alternatively you can create a new datastore from the command line. The
278 following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
280 .. code-block:: console
282 # proxmox-backup-manager datastore create store1 /backup/disk1/store1
287 To list existing datastores from the command line run:
289 .. code-block:: console
291 # proxmox-backup-manager datastore list
292 ┌────────┬──────────────────────┬─────────────────────────────┐
293 │ name │ path │ comment │
294 ╞════════╪══════════════════════╪═════════════════════════════╡
295 │ store1 │ /backup/disk1/store1 │ This is my default storage. │
296 └────────┴──────────────────────┴─────────────────────────────┘
298 You can change the garbage collection and prune settings of a datastore, by
299 editing the datastore from the GUI or by using the ``update`` subcommand. For
300 example, the below command changes the garbage collection schedule using the
301 ``update`` subcommand and prints the properties of the datastore with the
304 .. code-block:: console
306 # proxmox-backup-manager datastore update store1 --gc-schedule 'Tue 04:27'
307 # proxmox-backup-manager datastore show store1
308 ┌────────────────┬─────────────────────────────┐
310 ╞════════════════╪═════════════════════════════╡
312 ├────────────────┼─────────────────────────────┤
313 │ path │ /backup/disk1/store1 │
314 ├────────────────┼─────────────────────────────┤
315 │ comment │ This is my default storage. │
316 ├────────────────┼─────────────────────────────┤
317 │ gc-schedule │ Tue 04:27 │
318 ├────────────────┼─────────────────────────────┤
320 ├────────────────┼─────────────────────────────┤
321 │ prune-schedule │ daily │
322 └────────────────┴─────────────────────────────┘
324 Finally, it is possible to remove the datastore configuration:
326 .. code-block:: console
328 # proxmox-backup-manager datastore remove store1
330 .. note:: The above command removes only the datastore configuration. It does
331 not delete any data from the underlying directory.
337 After creating a datastore, the following default layout will appear:
339 .. code-block:: console
341 # ls -arilh /backup/disk1/store1
342 276493 -rw-r--r-- 1 backup backup 0 Jul 8 12:35 .lock
343 276490 drwxr-x--- 1 backup backup 1064960 Jul 8 12:35 .chunks
345 `.lock` is an empty file used for process locking.
347 The `.chunks` directory contains folders, starting from `0000` and taking hexadecimal values until `ffff`. These
348 directories will store the chunked data after a backup operation has been executed.
350 .. code-block:: console
352 # ls -arilh /backup/disk1/store1/.chunks
353 545824 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 ffff
354 545823 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffe
355 415621 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffd
356 415620 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffc
357 353187 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffb
358 344995 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffa
359 144079 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff9
360 144078 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff8
361 144077 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff7
363 403180 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000c
364 403179 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000b
365 403177 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000a
366 402530 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0009
367 402513 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0008
368 402509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0007
369 276509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0006
370 276508 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0005
371 276507 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0004
372 276501 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0003
373 276499 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0002
374 276498 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0001
375 276494 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0000
376 276489 drwxr-xr-x 3 backup backup 4.0K Jul 8 12:35 ..
377 276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 .
384 .. image:: images/screenshots/pbs-gui-user-management.png
387 :alt: User management
389 Proxmox Backup Server supports several authentication realms, and you need to
390 choose the realm when you add a new user. Possible realms are:
392 :pam: Linux PAM standard authentication. Use this if you want to
393 authenticate as Linux system user (Users need to exist on the
396 :pbs: Proxmox Backup Server realm. This type stores hashed passwords in
397 ``/etc/proxmox-backup/shadow.json``.
399 After installation, there is a single user ``root@pam``, which
400 corresponds to the Unix superuser. You can use the
401 ``proxmox-backup-manager`` command line tool to list or manipulate
404 .. code-block:: console
406 # proxmox-backup-manager user list
407 ┌─────────────┬────────┬────────┬───────────┬──────────┬────────────────┬────────────────────┐
408 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
409 ╞═════════════╪════════╪════════╪═══════════╪══════════╪════════════════╪════════════════════╡
410 │ root@pam │ 1 │ │ │ │ │ Superuser │
411 └─────────────┴────────┴────────┴───────────┴──────────┴────────────────┴────────────────────┘
413 .. image:: images/screenshots/pbs-gui-user-management-add-user.png
418 The superuser has full administration rights on everything, so you
419 normally want to add other users with less privileges. You can create a new
420 user with the ``user create`` subcommand or through the web interface, under
421 **Configuration -> User Management**. The ``create`` subcommand lets you specify
422 many options like ``--email`` or ``--password``. You can update or change any
423 user properties using the ``update`` subcommand later (**Edit** in the GUI):
426 .. code-block:: console
428 # proxmox-backup-manager user create john@pbs --email john@example.com
429 # proxmox-backup-manager user update john@pbs --firstname John --lastname Smith
430 # proxmox-backup-manager user update john@pbs --comment "An example user."
432 .. todo:: Mention how to set password without passing plaintext password as cli argument.
435 The resulting user list looks like this:
437 .. code-block:: console
439 # proxmox-backup-manager user list
440 ┌──────────┬────────┬────────┬───────────┬──────────┬──────────────────┬──────────────────┐
441 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
442 ╞══════════╪════════╪════════╪═══════════╪══════════╪══════════════════╪══════════════════╡
443 │ john@pbs │ 1 │ │ John │ Smith │ john@example.com │ An example user. │
444 ├──────────┼────────┼────────┼───────────┼──────────┼──────────────────┼──────────────────┤
445 │ root@pam │ 1 │ │ │ │ │ Superuser │
446 └──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
448 Newly created users do not have any permissions. Please read the next
449 section to learn how to set access permissions.
451 If you want to disable a user account, you can do that by setting ``--enable`` to ``0``
453 .. code-block:: console
455 # proxmox-backup-manager user update john@pbs --enable 0
457 Or completely remove the user with:
459 .. code-block:: console
461 # proxmox-backup-manager user remove john@pbs
467 By default new users do not have any permission. Instead you need to
468 specify what is allowed and what is not. You can do this by assigning
469 roles to users on specific objects like datastores or remotes. The
470 following roles exist:
473 Disable Access - nothing is allowed.
479 Can view things, but is not allowed to change settings.
482 Can do anything on datastores.
485 Can view datastore settings and list content. But
486 is not allowed to read the actual data.
489 Can Inspect datastore content and can do restores.
492 Can backup and restore owned backups.
494 **DatastorePowerUser**
495 Can backup, restore, and prune owned backups.
498 Can do anything on remotes.
501 Can view remote settings.
503 **RemoteSyncOperator**
504 Is allowed to read data from a remote.
508 :alt: Add permissions for user
510 You can manage datastore permissions from **Configuration -> Permissions** in
511 the web interface. Likewise, you can use the ``acl`` subcommand to manage and
512 monitor user permissions from the command line. For example, the command below
513 will add the user ``john@pbs`` as a **DatastoreAdmin** for the datastore
514 ``store1``, located at ``/backup/disk1/store1``:
516 .. code-block:: console
518 # proxmox-backup-manager acl update /datastore/store1 DatastoreAdmin --userid john@pbs
520 You can monitor the roles of each user using the following command:
522 .. code-block:: console
524 # proxmox-backup-manager acl list
525 ┌──────────┬──────────────────┬───────────┬────────────────┐
526 │ ugid │ path │ propagate │ roleid │
527 ╞══════════╪══════════════════╪═══════════╪════════════════╡
528 │ john@pbs │ /datastore/disk1 │ 1 │ DatastoreAdmin │
529 └──────────┴──────────────────┴───────────┴────────────────┘
531 A single user can be assigned multiple permission sets for different data stores.
534 Naming convention is important here. For data stores on the host,
535 you must use the convention ``/datastore/{storename}``. For example, to set
536 permissions for a data store mounted at ``/mnt/backup/disk4/store2``, you would use
537 ``/datastore/store2`` for the path. For remote stores, use the convention
538 ``/remote/{remote}/{storename}``, where ``{remote}`` signifies the name of the
539 remote (see `Remote` below) and ``{storename}`` is the name of the data store on
545 Proxmox Backup Server provides both a web interface and a command line tool for
546 network configuration. You can find the configuration options in the web
547 interface under the **Network Interfaces** section of the **Configuration** menu
548 tree item. The command line tool is accessed via the ``network`` subcommand.
549 These interfaces allow you to carry out some basic network management tasks,
550 such as adding, configuring, and removing network interfaces.
552 .. note:: Any changes made to the network configuration are not
553 applied, until you click on **Apply Configuration** or enter the ``network
554 reload`` command. This allows you to make many changes at once. It also allows
555 you to ensure that your changes are correct before applying them, as making a
556 mistake here can render the server inaccessible over the network.
558 To get a list of available interfaces, use the following command:
560 .. code-block:: console
562 # proxmox-backup-manager network list
563 ┌───────┬────────┬───────────┬────────┬─────────┬───────────────────┬──────────────┬──────────────┐
564 │ name │ type │ autostart │ method │ method6 │ address │ gateway │ ports/slaves │
565 ╞═══════╪════════╪═══════════╪════════╪═════════╪═══════════════════╪══════════════╪══════════════╡
566 │ bond0 │ bond │ 1 │ manual │ │ │ │ ens18 ens19 │
567 ├───────┼────────┼───────────┼────────┼─────────┼───────────────────┼──────────────┼──────────────┤
568 │ ens18 │ eth │ 1 │ manual │ │ │ │ │
569 ├───────┼────────┼───────────┼────────┼─────────┼───────────────────┼──────────────┼──────────────┤
570 │ ens19 │ eth │ 1 │ manual │ │ │ │ │
571 ├───────┼────────┼───────────┼────────┼─────────┼───────────────────┼──────────────┼──────────────┤
572 │ vmbr0 │ bridge │ 1 │ static │ │ x.x.x.x/x │ x.x.x.x │ bond0 │
573 └───────┴────────┴───────────┴────────┴─────────┴───────────────────┴──────────────┴──────────────┘
575 .. image:: images/screenshots/pbs-gui-network-create-bond.png
578 :alt: Add a network interface
580 To add a new network interface, select an interface type from the **Create** menu
581 in the web interface, or use the ``create`` subcommand with the relevant
582 parameters. The following command shows a template for creating a new bridge:
586 .. code-block:: console
588 # proxmox-backup-manager network create vmbr1 --autostart true --cidr x.x.x.x/x --gateway x.x.x.x --bridge_ports iface_name --type bridge
590 You can make changes to the configuration of a network interface with the
591 ``update`` subcommand:
593 .. code-block:: console
595 # proxmox-backup-manager network update vmbr1 --cidr y.y.y.y/y
597 You can also remove a network interface:
599 .. code-block:: console
601 # proxmox-backup-manager network remove vmbr1
603 The pending changes for the network configuration file will appear at the bottom of the
604 web interface. You can also view these changes, by using the command:
606 .. code-block:: console
608 # proxmox-backup-manager network changes
610 If you would like to cancel all changes at this point, you can either click on
611 the **Revert** button or use the following command:
613 .. code-block:: console
615 # proxmox-backup-manager network revert
617 If you are happy with the changes and would like to write them into the
618 configuration file, select **Apply Configuration**. The corresponding command
621 .. code-block:: console
623 # proxmox-backup-manager network reload
625 You can also configure DNS settings, from the **DNS** section
626 of **Configuration** or by using the ``dns`` subcommand of
627 ``proxmox-backup-manager``.
632 A remote refers to a separate Proxmox Backup Server installation and a user on that
633 installation, from which you can `sync` datastores to a local datastore with a
636 To add a remote, you need its hostname or ip, a userid and password on the
637 remote, and its certificate fingerprint. To get the fingerprint, use the
638 ``proxmox-backup-manager cert info`` command on the remote.
640 .. code-block:: console
642 # proxmox-backup-manager cert info |grep Fingerprint
643 Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
645 Using the information specified above, add the remote with:
647 .. code-block:: console
649 # proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
651 Use the ``list``, ``show``, ``update``, ``remove`` subcommands of
652 ``proxmox-backup-manager remote`` to manage your remotes:
654 .. code-block:: console
656 # proxmox-backup-manager remote update pbs2 --host pbs2.example
657 # proxmox-backup-manager remote list
658 ┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐
659 │ name │ host │ userid │ fingerprint │ comment │
660 ╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡
661 │ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │ │
662 └──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘
663 # proxmox-backup-manager remote remove pbs2
669 Sync jobs are configured to pull the contents of a datastore on a `Remote` to a
670 local datastore. You can either start the sync job manually on the GUI or
671 provide it with a :term:`schedule` to run regularly. The
672 ``proxmox-backup-manager sync-job`` command is used to manage sync jobs:
674 .. code-block:: console
676 # proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30'
677 # proxmox-backup-manager sync-job update pbs2-local --comment 'offsite'
678 # proxmox-backup-manager sync-job list
679 ┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐
680 │ id │ store │ remote │ remote-store │ schedule │ comment │
681 ╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡
682 │ pbs2-local │ local │ pbs2 │ local │ Wed 02:30 │ offsite │
683 └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
684 # proxmox-backup-manager sync-job remove pbs2-local
688 You can monitor and run :ref:`garbage collection <garbage-collection>` on the
689 Proxmox Backup Server using the ``garbage-collection`` subcommand of
690 ``proxmox-backup-manager``. You can use the ``start`` subcommand to manually start garbage
691 collection on an entire data store and the ``status`` subcommand to see
692 attributes relating to the :ref:`garbage collection <garbage-collection>`.
698 The command line client is called :command:`proxmox-backup-client`.
704 The client uses the following notation to specify a datastore repository
705 on the backup server.
707 [[username@]server:]datastore
709 The default value for ``username`` ist ``root``. If no server is specified,
710 the default is the local host (``localhost``).
712 You can pass the repository with the ``--repository`` command
713 line option, or by setting the ``PBS_REPOSITORY`` environment
717 Environment Variables
718 ~~~~~~~~~~~~~~~~~~~~~
721 The default backup repository.
724 When set, this value is used for the password required for the
727 ``PBS_ENCRYPTION_PASSWORD``
728 When set, this value is used to access the secret encryption key (if
729 protected by password).
731 ``PBS_FINGERPRINT`` When set, this value is used to verify the server
732 certificate (only used if the system CA certificates cannot
733 validate the certificate).
739 Most commands support the ``--output-format`` parameter. It accepts
740 the following values:
742 :``text``: Text format (default). Structured data is rendered as a table.
744 :``json``: JSON (single line).
746 :``json-pretty``: JSON (multiple lines, nicely formatted).
749 Please use the following environment variables to modify output behavior:
751 ``PROXMOX_OUTPUT_FORMAT``
752 Defines the default output format.
754 ``PROXMOX_OUTPUT_NO_BORDER``
755 If set (to any value), do not render table borders.
757 ``PROXMOX_OUTPUT_NO_HEADER``
758 If set (to any value), do not render table headers.
760 .. note:: The ``text`` format is designed to be human readable, and
761 not meant to be parsed by automation tools. Please use the ``json``
762 format if you need to process the output.
765 .. _creating-backups:
770 This section explains how to create a backup from within the machine. This can
771 be a physical host, a virtual machine, or a container. Such backups may contain file
772 and image archives. There are no restrictions in this case.
774 .. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`.
776 For the following example you need to have a backup server set up, working
777 credentials and need to know the repository name.
778 In the following examples we use ``backup-server:store1``.
780 .. code-block:: console
782 # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
783 Starting backup: host/elsa/2019-12-03T09:35:01Z
785 skip mount point: "/boot/efi"
786 skip mount point: "/dev"
787 skip mount point: "/run"
788 skip mount point: "/sys"
789 Uploaded 12129 chunks in 87 seconds (564 MB/s).
790 End Time: 2019-12-03T10:36:29+01:00
792 This will prompt you for a password and then uploads a file archive named
793 ``root.pxar`` containing all the files in the ``/`` directory.
795 .. Caution:: Please note that the proxmox-backup-client does not
796 automatically include mount points. Instead, you will see a short
797 ``skip mount point`` notice for each of them. The idea is to
798 create a separate file archive for each mounted disk. You can
799 explicitly include them using the ``--include-dev`` option
800 (i.e. ``--include-dev /boot/efi``). You can use this option
801 multiple times for each mount point that should be included.
803 The ``--repository`` option can get quite long and is used by all
804 commands. You can avoid having to enter this value by setting the
805 environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set
806 over multiple sessions, you should instead add the below line to your
809 .. code-block:: console
811 # export PBS_REPOSITORY=backup-server:store1
813 After this you can execute all commands without specifying the ``--repository``
816 One single backup is allowed to contain more than one archive. For example, if
817 you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
819 .. code-block:: console
821 # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
823 This creates a backup of both disks.
825 The backup command takes a list of backup specifications, which
826 include the archive name on the server, the type of the archive, and the
827 archive source at the client. The format is:
829 <archive-name>.<type>:<source-path>
831 Common types are ``.pxar`` for file archives, and ``.img`` for block
832 device images. To create a backup of a block device run the following command:
834 .. code-block:: console
836 # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
838 Excluding files/folders from a backup
839 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
841 Sometimes it is desired to exclude certain files or folders from a backup archive.
842 To tell the Proxmox Backup client when and how to ignore files and directories,
843 place a text file called ``.pxarexclude`` in the filesystem hierarchy.
844 Whenever the backup client encounters such a file in a directory, it interprets
845 each line as glob match patterns for files and directories that are to be excluded
848 The file must contain a single glob pattern per line. Empty lines are ignored.
849 The same is true for lines starting with ``#``, which indicates a comment.
850 A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
851 to an explicit inclusion. This makes it possible to exclude all entries in a
852 directory except for a few single files/subdirectories.
853 Lines ending in ``/`` match only on directories.
854 The directory containing the ``.pxarexclude`` file is considered to be the root of
855 the given patterns. It is only possible to match files in this directory and its subdirectories.
857 ``\`` is used to escape special glob characters.
858 ``?`` matches any single character.
859 ``*`` matches any character, including an empty string.
860 ``**`` is used to match subdirectories. It can be used to, for example, exclude
861 all files ending in ``.tmp`` within the directory or subdirectories with the
862 following pattern ``**/*.tmp``.
863 ``[...]`` matches a single character from any of the provided characters within
864 the brackets. ``[!...]`` does the complementary and matches any single character
865 not contained within the brackets. It is also possible to specify ranges with two
866 characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
867 alphabetic character and ``[0-9]`` matches any one single digit.
869 The order of the glob match patterns defines whether a file is included or
870 excluded, that is to say later entries override previous ones.
871 This is also true for match patterns encountered deeper down the directory tree,
872 which can override a previous exclusion.
873 Be aware that excluded directories will **not** be read by the backup client.
874 Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect.
875 ``.pxarexclude`` files are treated as regular files and will be included in the
878 For example, consider the following directory structure:
880 .. code-block:: console
884 . .. .pxarexclude subfolder0 subfolder1
887 . .. file0 file1 file2 file3 .pxarexclude
890 . .. file0 file1 file2 file3
892 The different ``.pxarexclude`` files contain the following:
894 .. code-block:: console
896 # cat folder/.pxarexclude
901 .. code-block:: console
903 # cat folder/subfolder0/.pxarexclude
906 This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of
907 ``subfolder1`` except ``file2``.
909 Restoring this backup will result in:
911 .. code-block:: console
915 . .. .pxarexclude subfolder0 subfolder1
918 . .. file0 file2 .pxarexclude
926 Proxmox Backup supports client-side encryption with AES-256 in GCM_
927 mode. To set this up, you first need to create an encryption key:
929 .. code-block:: console
931 # proxmox-backup-client key create my-backup.key
932 Encryption Key Password: **************
934 The key is password protected by default. If you do not need this
935 extra protection, you can also create it without a password:
937 .. code-block:: console
939 # proxmox-backup-client key create /path/to/my-backup.key --kdf none
941 Having created this key, it is now possible to create an encrypted backup, by
942 passing the ``--keyfile`` parameter, with the path to the key file.
944 .. code-block:: console
946 # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
948 Encryption Key Password: **************
951 .. Note:: If you do not specify the name of the backup key, the key will be
952 created in the default location
953 ``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client``
954 will also search this location by default, in case the ``--keyfile``
955 parameter is not specified.
957 You can avoid entering the passwords by setting the environment
958 variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
960 Using a master key to store and recover encryption keys
961 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
963 You can also use ``proxmox-backup-client key`` to create an RSA public/private
964 key pair, which can be used to store an encrypted version of the symmetric
965 backup encryption key alongside each backup and recover it later.
967 To set up a master key:
969 1. Create an encryption key for the backup:
971 .. code-block:: console
973 # proxmox-backup-client key create
974 creating default key at: "~/.config/proxmox-backup/encryption-key.json"
975 Encryption Key Password: **********
978 The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``.
980 2. Create an RSA public/private key pair:
982 .. code-block:: console
984 # proxmox-backup-client key create-master-key
985 Master Key Password: *********
988 This will create two files in your current directory, ``master-public.pem``
989 and ``master-private.pem``.
991 3. Import the newly created ``master-public.pem`` public certificate, so that
992 ``proxmox-backup-client`` can find and use it upon backup.
994 .. code-block:: console
996 # proxmox-backup-client key import-master-pubkey /path/to/master-public.pem
997 Imported public master key to "~/.config/proxmox-backup/master-public.pem"
999 4. With all these files in place, run a backup job:
1001 .. code-block:: console
1003 # proxmox-backup-client backup etc.pxar:/etc
1005 The key will be stored in your backup, under the name ``rsa-encrypted.key``.
1007 .. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key
1008 is in the default path. If you specified another path upon creation, you
1009 must pass the ``--keyfile`` parameter.
1011 5. To test that everything worked, you can restore the key from the backup:
1013 .. code-block:: console
1015 # proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target
1017 .. Note:: You should not need an encryption key to extract this file. However, if
1018 a key exists at the default location
1019 (``~/.config/proxmox-backup/encryption-key.json``) the program will prompt
1020 you for an encryption key password. Simply moving ``encryption-key.json``
1021 out of this directory will fix this issue.
1023 6. Then, use the previously generated master key to decrypt the file:
1025 .. code-block:: console
1027 # openssl rsautl -decrypt -inkey master-private.pem -in rsa-encrypted.key -out /path/to/target
1028 Enter pass phrase for ./master-private.pem: *********
1030 7. The target file will now contain the encryption key information in plain
1031 text. The success of this can be confirmed by passing the resulting ``json``
1032 file, with the ``--keyfile`` parameter, when decrypting files from the backup.
1034 .. warning:: Without their key, backed up files will be inaccessible. Thus, you should
1035 keep keys ordered and in a place that is separate from the contents being
1036 backed up. It can happen, for example, that you back up an entire system, using
1037 a key on that system. If the system then becomes inaccessable for any reason
1038 and needs to be restored, this will not be possible as the encryption key will be
1039 lost along with the broken system. In preparation for the worst case scenario,
1040 you should consider keeping a paper copy of this key locked away in
1046 The regular creation of backups is a necessary step to avoiding data
1047 loss. More importantly, however, is the restoration. It is good practice to perform
1048 periodic recovery tests to ensure that you can access the data in
1051 First, you need to find the snapshot which you want to restore. The snapshot
1052 command provides a list of all the snapshots on the server:
1054 .. code-block:: console
1056 # proxmox-backup-client snapshots
1057 ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐
1058 │ snapshot │ size │ files │
1059 ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡
1060 │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │
1061 ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
1062 │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │
1063 ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
1066 You can inspect the catalog to find specific files.
1068 .. code-block:: console
1070 # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z
1072 d "./root.pxar.didx/etc/cifs-utils"
1073 l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
1074 d "./root.pxar.didx/etc/console-setup"
1077 The restore command lets you restore a single archive from the
1080 .. code-block:: console
1082 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
1084 To get the contents of any archive, you can restore the ``index.json`` file in the
1085 repository to the target path '-'. This will dump the contents to the standard output.
1087 .. code-block:: console
1089 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
1092 Interactive Restores
1093 ^^^^^^^^^^^^^^^^^^^^
1095 If you only want to restore a few individual files, it is often easier
1096 to use the interactive recovery shell.
1098 .. code-block:: console
1100 # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar
1101 Starting interactive shell
1103 bin boot dev etc home lib lib32
1106 The interactive recovery shell is a minimalistic command line interface that
1107 utilizes the metadata stored in the catalog to quickly list, navigate and
1108 search files in a file archive.
1109 To restore files, you can select them individually or match them with a glob
1112 Using the catalog for navigation reduces the overhead considerably because only
1113 the catalog needs to be downloaded and, optionally, decrypted.
1114 The actual chunks are only accessed if the metadata in the catalog is not enough
1115 or for the actual restore.
1117 Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
1118 working directory and list directory contents in the archive.
1119 ``pwd`` shows the full path of the current working directory with respect to the
1122 Being able to quickly search the contents of the archive is a commmonly needed feature.
1123 That's where the catalog is most valuable.
1126 .. code-block:: console
1128 pxar:/ > find etc/**/*.txt --select
1130 pxar:/ > list-selected
1132 pxar:/ > restore-selected /target/path
1135 This will find and print all files ending in ``.txt`` located in ``etc/`` or a
1136 subdirectory and add the corresponding pattern to the list for subsequent restores.
1137 ``list-selected`` shows these patterns and ``restore-selected`` finally restores
1138 all files in the archive matching the patterns to ``/target/path`` on the local
1139 host. This will scan the whole archive.
1141 With ``restore /target/path`` you can restore the sub-archive given by the current
1142 working directory to the local target path ``/target/path`` on your host.
1143 By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
1144 further limited to files matching the pattern.
1147 .. code-block:: console
1150 pxar:/etc/ > restore /target/ --pattern **/*.conf
1153 The above will scan trough all the directories below ``/etc`` and restore all
1154 files ending in ``.conf``.
1156 .. todo:: Explain interactive restore in more detail
1158 Mounting of Archives via FUSE
1159 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1161 The :term:`FUSE` implementation for the pxar archive allows you to mount a
1162 file archive as a read-only filesystem to a mountpoint on your host.
1164 .. code-block:: console
1166 # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint
1167 # ls /mnt/mountpoint
1168 bin dev home lib32 libx32 media opt root sbin sys usr
1169 boot etc lib lib64 lost+found mnt proc run srv tmp var
1171 This allows you to access the full contents of the archive in a seamless manner.
1173 .. note:: As the FUSE connection needs to fetch and decrypt chunks from the
1174 backup server's datastore, this can cause some additional network and CPU
1175 load on your host, depending on the operations you perform on the mounted
1178 To unmount the filesystem use the ``umount`` command on the mountpoint:
1180 .. code-block:: console
1182 # umount /mnt/mountpoint
1187 The client tool prompts you to enter the logon password as soon as you
1188 want to access the backup server. The server checks your credentials
1189 and responds with a ticket that is valid for two hours. The client
1190 tool automatically stores that ticket and uses it for further requests
1193 You can also manually trigger this login/logout using the login and
1196 .. code-block:: console
1198 # proxmox-backup-client login
1199 Password: **********
1201 To remove the ticket, issue a logout:
1203 .. code-block:: console
1205 # proxmox-backup-client logout
1210 Pruning and Removing Backups
1211 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1213 You can manually delete a backup snapshot using the ``forget``
1216 .. code-block:: console
1218 # proxmox-backup-client forget <snapshot>
1221 .. caution:: This command removes all archives in this backup
1222 snapshot. They will be inaccessible and unrecoverable.
1225 Although manual removal is sometimes required, the ``prune``
1226 command is normally used to systematically delete older backups. Prune lets
1227 you specify which backup snapshots you want to keep. The
1228 following retention options are available:
1231 Keep the last ``<N>`` backup snapshots.
1233 ``--keep-hourly <N>``
1234 Keep backups for the last ``<N>`` hours. If there is more than one
1235 backup for a single hour, only the latest is kept.
1237 ``--keep-daily <N>``
1238 Keep backups for the last ``<N>`` days. If there is more than one
1239 backup for a single day, only the latest is kept.
1241 ``--keep-weekly <N>``
1242 Keep backups for the last ``<N>`` weeks. If there is more than one
1243 backup for a single week, only the latest is kept.
1245 .. note:: Weeks start on Monday and end on Sunday. The software
1246 uses the `ISO week date`_ system and handles weeks at
1247 the end of the year correctly.
1249 ``--keep-monthly <N>``
1250 Keep backups for the last ``<N>`` months. If there is more than one
1251 backup for a single month, only the latest is kept.
1253 ``--keep-yearly <N>``
1254 Keep backups for the last ``<N>`` years. If there is more than one
1255 backup for a single year, only the latest is kept.
1257 The retention options are processed in the order given above. Each option
1258 only covers backups within its time period. The next option does not take care
1259 of already covered backups. It will only consider older backups.
1261 Unfinished and incomplete backups will be removed by the prune command unless
1262 they are newer than the last successful backup. In this case, the last failed
1265 .. code-block:: console
1267 # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3
1270 You can use the ``--dry-run`` option to test your settings. This only
1271 shows the list of existing snapshots and what actions prune would take.
1273 .. code-block:: console
1275 # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
1276 ┌────────────────────────────────┬──────┐
1278 ╞════════════════════════════════╪══════╡
1279 │ host/elsa/2019-12-04T13:20:37Z │ 1 │
1280 ├────────────────────────────────┼──────┤
1281 │ host/elsa/2019-12-03T09:35:01Z │ 0 │
1282 ├────────────────────────────────┼──────┤
1283 │ host/elsa/2019-11-22T11:54:47Z │ 1 │
1284 ├────────────────────────────────┼──────┤
1285 │ host/elsa/2019-11-21T12:36:25Z │ 0 │
1286 ├────────────────────────────────┼──────┤
1287 │ host/elsa/2019-11-10T10:42:20Z │ 1 │
1288 └────────────────────────────────┴──────┘
1290 .. note:: Neither the ``prune`` command nor the ``forget`` command free space
1291 in the chunk-store. The chunk-store still contains the data blocks. To free
1292 space you need to perform :ref:`garbage-collection`.
1295 .. _garbage-collection:
1300 The ``prune`` command removes only the backup index files, not the data
1301 from the data store. This task is left to the garbage collection
1302 command. It is recommended to carry out garbage collection on a regular basis.
1304 The garbage collection works in two phases. In the first phase, all
1305 data blocks that are still in use are marked. In the second phase,
1306 unused data blocks are removed.
1308 .. note:: This command needs to read all existing backup index files
1309 and touches the complete chunk-store. This can take a long time
1310 depending on the number of chunks and the speed of the underlying
1313 .. note:: The garbage collection will only remove chunks that haven't been used
1314 for at least one day (exactly 24h 5m). This grace period is necessary because
1315 chunks in use are marked by touching the chunk which updates the ``atime``
1316 (access time) property. Filesystems are mounted with the ``relatime`` option
1317 by default. This results in a better performance by only updating the
1318 ``atime`` property if the last access has been at least 24 hours ago. The
1319 downside is, that touching a chunk within these 24 hours will not always
1320 update its ``atime`` property.
1322 Chunks in the grace period will be logged at the end of the garbage
1323 collection task as *Pending removals*.
1325 .. code-block:: console
1327 # proxmox-backup-client garbage-collect
1328 starting garbage collection on store store2
1329 Start GC phase1 (mark used chunks)
1330 Start GC phase2 (sweep unused chunks)
1331 percentage done: 1, chunk count: 219
1332 percentage done: 2, chunk count: 453
1334 percentage done: 99, chunk count: 21188
1335 Removed bytes: 411368505
1337 Original data bytes: 327160886391
1338 Disk bytes: 52767414743 (16 %)
1340 Average chunk size: 2486565
1344 .. todo:: howto run garbage-collection at regular intervalls (cron)
1348 The backup client also comes with a benchmarking tool. This tool measures
1349 various metrics relating to compression and encryption speeds. You can run a
1350 benchmark using the ``benchmark`` subcommand of ``proxmox-backup-client``:
1352 .. code-block:: console
1354 # proxmox-backup-client benchmark
1355 Uploaded 656 chunks in 5 seconds.
1356 Time per request: 7659 microseconds.
1357 TLS speed: 547.60 MB/s
1358 SHA256 speed: 585.76 MB/s
1359 Compression speed: 1923.96 MB/s
1360 Decompress speed: 7885.24 MB/s
1361 AES256/GCM speed: 3974.03 MB/s
1362 ┌───────────────────────────────────┬─────────────────────┐
1364 ╞═══════════════════════════════════╪═════════════════════╡
1365 │ TLS (maximal backup upload speed) │ 547.60 MB/s (93%) │
1366 ├───────────────────────────────────┼─────────────────────┤
1367 │ SHA256 checksum computation speed │ 585.76 MB/s (28%) │
1368 ├───────────────────────────────────┼─────────────────────┤
1369 │ ZStd level 1 compression speed │ 1923.96 MB/s (89%) │
1370 ├───────────────────────────────────┼─────────────────────┤
1371 │ ZStd level 1 decompression speed │ 7885.24 MB/s (98%) │
1372 ├───────────────────────────────────┼─────────────────────┤
1373 │ AES256 GCM encryption speed │ 3974.03 MB/s (104%) │
1374 └───────────────────────────────────┴─────────────────────┘
1376 .. note:: The percentages given in the output table correspond to a
1377 comparison against a Ryzen 7 2700X. The TLS test connects to the
1378 local host, so there is no network involved.
1380 You can also pass the ``--output-format`` parameter to output stats in ``json``,
1381 rather than the default table format.
1383 .. _pve-integration:
1385 `Proxmox VE`_ integration
1386 -------------------------
1388 You need to define a new storage with type 'pbs' on your `Proxmox VE`_
1389 node. The following example uses ``store2`` as storage name, and
1390 assumes the server address is ``localhost``, and you want to connect
1393 .. code-block:: console
1395 # pvesm add pbs store2 --server localhost --datastore store2
1396 # pvesm set store2 --username user1@pbs --password <secret>
1398 If your backup server uses a self signed certificate, you need to add
1399 the certificate fingerprint to the configuration. You can get the
1400 fingerprint by running the following command on the backup server:
1402 .. code-block:: console
1404 # proxmox-backup-manager cert info |grep Fingerprint
1405 Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
1407 Please add that fingerprint to your configuration to establish a trust
1410 .. code-block:: console
1412 # pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
1414 After that you should be able to see storage status with:
1416 .. code-block:: console
1418 # pvesm status --storage store2
1419 Name Type Status Total Used Available %
1420 store2 pbs active 3905109820 1336687816 2568422004 34.23%
1424 .. include:: command-line-tools.rst
1426 .. include:: services.rst