-Administration Guide
-====================
+Backup Management
+=================
-The administration guide.
-
-.. todo:: either add a bit more explanation or remove the previous sentence
+.. The administration guide.
+ .. todo:: either add a bit more explanation or remove the previous sentence
Terminology
-----------
When doing deduplication, there are different strategies to get
optimal results in terms of performance and/or deduplication rates.
-Depending on the type of data, one can split data into *fixed* or *variable*
+Depending on the type of data, it can be split into *fixed* or *variable*
sized chunks.
-Fixed sized chunking needs almost no CPU performance, and is used to
+Fixed sized chunking requires minimal CPU power, and is used to
backup virtual machine images.
Variable sized chunking needs more CPU power, but is essential to get
good deduplication rates for file archives.
-The backup server supports both strategies.
+The Proxmox Backup Server supports both strategies.
File Archives: ``<name>.pxar``
.. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
A file archive stores a full directory tree. Content is stored using
-the :ref:`pxar-format`, split into variable sized chunks. The format
+the :ref:`pxar-format`, split into variable-sized chunks. The format
is optimized to achieve good deduplication rates.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is used for virtual machine images and other large binary
-data. Content is split into fixed sized chunks.
+data. Content is split into fixed-sized chunks.
Binary Data (BLOBs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The catalog file is an index for file archives. It contains
-the list of files and is used to speed-up search operations.
+the list of files and is used to speed up search operations.
The Manifest: ``index.json``
``vm``
This type is used for :term:`virtual machine`\ s. Typically
- contains the virtual machine's configuration and an image archive
+ consists of the virtual machine's configuration file and an image archive
for each disk.
``ct``
- This type is used for :term:`container`\ s. Contains the container's
- configuration and a single file archive for the container content.
+ This type is used for :term:`container`\ s. Consists of the container's
+ configuration and a single file archive for the filesystem content.
``host``
This type is used for backups created from within the backed up machine.
Backup ID
~~~~~~~~~
-An unique ID. Usually the virtual machine or container ID. ``host``
+A unique ID. Usually the virtual machine or container ID. ``host``
type backups normally use the hostname.
As you can see, the time format is RFC3399_ with Coordinated
Universal Time (UTC_, identified by the trailing *Z*).
+Backup Server Management
+------------------------
+
+The command line tool to configure and manage the backup server is called
+:command:`proxmox-backup-manager`.
+
+
:term:`DataStore`
~~~~~~~~~~~~~~~~~
when setting up the backup server.
-Backup Server Management
-------------------------
-
-The command line tool to configure and manage the backup server is called
-:command:`proxmox-backup-manager`.
-
Datastore Configuration
~~~~~~~~~~~~~~~~~~~~~~~
-A :term:`datastore` is a place to store backups. You can configure
-multiple datastores. At least one datastore needs to be
-configured. The datastore is identified by a simple `name` and points
-to a directory.
+You can configure multiple datastores. Minimum one datastore needs to be
+configured. The datastore is identified by a simple `name` and points to a
+directory on the filesystem. Each datastore also has associated retention
+settings of how many backup snapshots for each interval of ``hourly``,
+``daily``, ``weekly``, ``monthly``, ``yearly`` as well as an time independent
+number of backups to keep in that store. :ref:`Pruning <pruning>` and
+:ref:`garbage collection <garbage-collection>` can also be configured to run
+periodically based on a configured :term:`schedule` per datastore.
The following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
│ store1 │ /backup/disk1/store1 │ This is my default storage. │
└────────┴──────────────────────┴─────────────────────────────┘
+You can change settings of a datastore, for example to set a prune and garbage
+collection schedule or retention settings using ``update`` subcommand and view
+a datastore with the ``show`` subcommand:
+
+.. code-block:: console
+
+ # proxmox-backup-manager datastore update store1 --keep-last 7 --prune-schedule daily --gc-schedule 'Tue 04:27'
+ # proxmox-backup-manager datastore show store1
+ ┌────────────────┬─────────────────────────────┐
+ │ Name │ Value │
+ ╞════════════════╪═════════════════════════════╡
+ │ name │ store1 │
+ ├────────────────┼─────────────────────────────┤
+ │ path │ /backup/disk1/store1 │
+ ├────────────────┼─────────────────────────────┤
+ │ comment │ This is my default storage. │
+ ├────────────────┼─────────────────────────────┤
+ │ gc-schedule │ Tue 04:27 │
+ ├────────────────┼─────────────────────────────┤
+ │ keep-last │ 7 │
+ ├────────────────┼─────────────────────────────┤
+ │ prune-schedule │ daily │
+ └────────────────┴─────────────────────────────┘
+
Finally, it is possible to remove the datastore configuration:
.. code-block:: console
File Layout
^^^^^^^^^^^
-.. todo:: Add datastore file layout example
+After creating a datastore, the following default layout will appear:
+
+.. code-block:: console
+
+ # ls -arilh /backup/disk1/store1
+ 276493 -rw-r--r-- 1 backup backup 0 Jul 8 12:35 .lock
+ 276490 drwxr-x--- 1 backup backup 1064960 Jul 8 12:35 .chunks
+
+`.lock` is an empty file used for process locking.
+
+The `.chunks` directory contains folders, starting from `0000` and taking hexadecimal values until `ffff`. These
+directories will store the chunked data after a backup operation has been executed.
+
+.. code-block:: console
+
+ # ls -arilh /backup/disk1/store1/.chunks
+ 545824 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 ffff
+ 545823 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffe
+ 415621 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffd
+ 415620 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffc
+ 353187 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffb
+ 344995 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffa
+ 144079 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff9
+ 144078 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff8
+ 144077 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff7
+ ...
+ 403180 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000c
+ 403179 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000b
+ 403177 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000a
+ 402530 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0009
+ 402513 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0008
+ 402509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0007
+ 276509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0006
+ 276508 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0005
+ 276507 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0004
+ 276501 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0003
+ 276499 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0002
+ 276498 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0001
+ 276494 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0000
+ 276489 drwxr-xr-x 3 backup backup 4.0K Jul 8 12:35 ..
+ 276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 .
+
User Management
~~~~~~~~~~~~~~~
-Proxmox Backup support several authentication realms, and you need to
+Proxmox Backup Server supports several authentication realms, and you need to
choose the realm when you add a new user. Possible realms are:
:pam: Linux PAM standard authentication. Use this if you want to
- authenticate as Linux system user (Users needs to exist on the
+ authenticate as Linux system user (Users need to exist on the
system).
:pbs: Proxmox Backup Server realm. This type stores hashed passwords in
.. code-block:: console
# proxmox-backup-manager user list
- â\94\8câ\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94¬â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94¬â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94¬â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94¬â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94¬â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94¬â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\90
- │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
- â\95\9eâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95ªâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95ªâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95ªâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95ªâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95ªâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95ªâ\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95\90â\95¡
- │ root@pam │ 1 │ │ │ │ │ Superuser │
- â\94\94â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94´â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94´â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94´â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94´â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94´â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94´â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\80â\94\98
+ ┌─────────────┬────────┬────────┬───────────┬──────────┬────────────────┬────────────────────┐
+ │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
+ ╞═════════════╪════════╪════════╪═══════════╪══════════╪════════════════╪════════════════════╡
+ │ root@pam │ 1 │ │ │ │ │ Superuser │
+ └─────────────┴────────┴────────┴───────────┴──────────┴────────────────┴────────────────────┘
The superuser has full administration rights on everything, so you
normally want to add other users with less privileges:
# proxmox-backup-manager user create john@pbs --email john@example.com
-The create command lets you specify many option like ``--email`` or
-``--password``, but you can update or change any of them using the
+The create command lets you specify many options like ``--email`` or
+``--password``. You can update or change any of them using the
update command later:
.. code-block:: console
# proxmox-backup-manager user update john@pbs --firstname John --lastname Smith
# proxmox-backup-manager user update john@pbs --comment "An example user."
-
.. todo:: Mention how to set password without passing plaintext password as cli argument.
-The resulting use list looks like this:
+The resulting user list looks like this:
.. code-block:: console
│ root@pam │ 1 │ │ │ │ │ Superuser │
└──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
-Newly created users do not have an permissions. Please read the next
+Newly created users do not have any permissions. Please read the next
section to learn how to set access permissions.
+If you want to disable a user account, you can do that by setting ``--enable`` to ``0``
+
+.. code-block:: console
+
+ # proxmox-backup-manager user update john@pbs --enable 0
+
+Or completely remove the user with:
+
+.. code-block:: console
+
+ # proxmox-backup-manager user remove john@pbs
+
Access Control
~~~~~~~~~~~~~~
+By default new users do not have any permission. Instead you need to
+specify what is allowed and what is not. You can do this by assigning
+roles to users on specific objects like datastores or remotes. The
+following roles exist:
+
+**NoAccess**
+ Disable Access - nothing is allowed.
+
+**Admin**
+ The Administrator can do anything.
+
+**Audit**
+ An Auditor can view things, but is not allowed to change settings.
+
+**DatastoreAdmin**
+ Can do anything on datastores.
+
+**DatastoreAudit**
+ Can view datastore settings and list content. But
+ is not allowed to read the actual data.
+
+**DataStoreReader**
+ Can Inspect datastore content and can do restores.
+
+**DataStoreBackup**
+ Can backup and restore owned backups.
+
+**DatastorePowerUser**
+ Can backup, restore, and prune owned backups.
+
+**RemoteAdmin**
+ Can do anything on remotes.
+
+**RemoteAudit**
+ Can view remote settings.
+
+**RemoteSyncOperator**
+ Is allowed to read data from a remote.
+
+
+:term:`Remote`
+~~~~~~~~~~~~~~
+
+A remote is a different Proxmox Backup Server installation and a user on that
+installation, from which you can `sync` datastores to a local datastore with a
+`Sync Job`.
+
+For adding a remote you need its hostname or ip, a userid and password on the
+remote and its certificate fingerprint to add it. To get the fingerprint use
+the ``proxmox-backup-manager cert info`` command on the remote.
+
+.. code-block:: console
+
+ # proxmox-backup-manager cert info |grep Fingerprint
+ Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
+
+With the needed information add the remote with:
+
+.. code-block:: console
+
+ # proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
+
+Use the ``list``, ``show``, ``update``, ``remove`` subcommands of
+``proxmox-backup-manager remote`` to manage your remotes:
+
+.. code-block:: console
+
+ # proxmox-backup-manager remote update pbs2 --host pbs2.example
+ # proxmox-backup-manager remote list
+ ┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐
+ │ name │ host │ userid │ fingerprint │ comment │
+ ╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡
+ │ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │ │
+ └──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘
+ # proxmox-backup-manager remote remove pbs2
+
+
+Sync Jobs
+~~~~~~~~~
+
+Sync jobs are configured to pull the contents of a datastore on a `Remote` to a
+local datastore. You can either start the sync job manually on the GUI or
+provide it with a :term:`schedule` to run regularly. The
+``proxmox-backup-manager sync-job`` command is used to manage sync jobs:
+
+.. code-block:: console
+
+ # proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30'
+ # proxmox-backup-manager sync-job update pbs2-local --comment 'offsite'
+ # proxmox-backup-manager sync-job list
+ ┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐
+ │ id │ store │ remote │ remote-store │ schedule │ comment │
+ ╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡
+ │ pbs2-local │ local │ pbs2 │ local │ Wed 02:30 │ offsite │
+ └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
+ # proxmox-backup-manager sync-job remove pbs2-local
Backup Client usage
[[username@]server:]datastore
-The default value for ``username`` ist ``root``. If no server is specified, the
-default is the local host (``localhost``).
+The default value for ``username`` ist ``root``. If no server is specified,
+the default is the local host (``localhost``).
You can pass the repository with the ``--repository`` command
line option, or by setting the ``PBS_REPOSITORY`` environment
be a physical host, a virtual machine, or a container. Such backups may contain file
and image archives. There are no restrictions in this case.
-.. note:: If you want to backup virtual machines or containers on Proxmov VE, see :ref:`pve-integration`.
+.. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`.
For the following example you need to have a backup server set up, working
credentials and need to know the repository name.
.. code-block:: console
- # export PBS_REPOSTORY=backup-server:store1
+ # export PBS_REPOSITORY=backup-server:store1
After this you can execute all commands without specifying the ``--repository``
option.
.. code-block:: console
# proxmox-backup-client snapshots
- ...
- host/elsa/2019-12-03T09:30:15Z | 51788646825 | root.pxar catalog.pcat1 index.json
- host/elsa/2019-12-03T09:35:01Z | 51790622048 | root.pxar catalog.pcat1 index.json
+ ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐
+ │ snapshot │ size │ files │
+ ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡
+ │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │
+ ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
+ │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │
+ ├────────────────────────────────┼─────────────┼────────────────────────────────────┤
...
You can inspect the catalog to find specific files.
# proxmox-backup-client logout
+.. _pruning:
+
Pruning and Removing Backups
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
only covers backups within its time period. The next option does not take care
of already covered backups. It will only consider older backups.
-For example, the ``--keep-monthly`` option does not consider any backup that is
-younger than one month.
-
-.. todo:: check if the previous statement is correct
-
Unfinished and incomplete backups will be removed by the prune command unless
they are newer than the last successful backup. In this case, the last failed
backup is retained.
.. code-block:: console
# proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
- retention options: --keep-daily 1 --keep-weekly 3
- Testing prune on store "store2" group "host/elsa"
- host/elsa/2019-12-04T13:20:37Z keep
- host/elsa/2019-12-03T09:35:01Z remove
- host/elsa/2019-11-22T11:54:47Z keep
- host/elsa/2019-11-21T12:36:25Z remove
- host/elsa/2019-11-10T10:42:20Z keep
-
+ ┌────────────────────────────────┬──────┐
+ │ snapshot │ keep │
+ ╞════════════════════════════════╪══════╡
+ │ host/elsa/2019-12-04T13:20:37Z │ 1 │
+ ├────────────────────────────────┼──────┤
+ │ host/elsa/2019-12-03T09:35:01Z │ 0 │
+ ├────────────────────────────────┼──────┤
+ │ host/elsa/2019-11-22T11:54:47Z │ 1 │
+ ├────────────────────────────────┼──────┤
+ │ host/elsa/2019-11-21T12:36:25Z │ 0 │
+ ├────────────────────────────────┼──────┤
+ │ host/elsa/2019-11-10T10:42:20Z │ 1 │
+ └────────────────────────────────┴──────┘
.. note:: Neither the ``prune`` command nor the ``forget`` command free space
in the chunk-store. The chunk-store still contains the data blocks. To free
`Proxmox VE`_ integration
-------------------------
+You need to define a new storage with type 'pbs' on your `Proxmox VE`_
+node. The following example uses ``store2`` as storage name, and
+assumes the server address is ``localhost``, and you want to connect
+as ``user1@pbs``.
-.. include:: command-line-tools.rst
+.. code-block:: console
-.. include:: services.rst
+ # pvesm add pbs store2 --server localhost --datastore store2
+ # pvesm set store2 --username user1@pbs --password <secret>
+
+If your backup server uses a self signed certificate, you need to add
+the certificate fingerprint to the configuration. You can get the
+fingerprint by running the following command on the backup server:
+
+.. code-block:: console
+
+ # proxmox-backup-manager cert info |grep Fingerprint
+ Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
-.. include host system admin at the end
+Please add that fingerprint to your configuration to establish a trust
+relationship:
-.. include:: sysadmin.rst
+.. code-block:: console
+
+ # pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
+
+After that you should be able to see storage status with:
+
+.. code-block:: console
+
+ # pvesm status --storage store2
+ Name Type Status Total Used Available %
+ store2 pbs active 3905109820 1336687816 2568422004 34.23%
+
+
+
+.. include:: command-line-tools.rst
+
+.. include:: services.rst
projects / proxmox-backup.git / blobdiff