]>
Commit | Line | Data |
---|---|---|
24406ebc TL |
1 | Backup Management |
2 | ================= | |
7e688b71 | 3 | |
24406ebc TL |
4 | .. The administration guide. |
5 | .. todo:: either add a bit more explanation or remove the previous sentence | |
c4f1b69f | 6 | |
fea8789c DM |
7 | Terminology |
8 | ----------- | |
9 | ||
85e139b7 DM |
10 | Backup Content |
11 | ~~~~~~~~~~~~~~ | |
12 | ||
13 | When doing deduplication, there are different strategies to get | |
14 | optimal results in terms of performance and/or deduplication rates. | |
8c6e5ce2 | 15 | Depending on the type of data, it can be split into *fixed* or *variable* |
85e139b7 DM |
16 | sized chunks. |
17 | ||
8c6e5ce2 | 18 | Fixed sized chunking requires minimal CPU power, and is used to |
85e139b7 DM |
19 | backup virtual machine images. |
20 | ||
21 | Variable sized chunking needs more CPU power, but is essential to get | |
22 | good deduplication rates for file archives. | |
23 | ||
8c6e5ce2 | 24 | The Proxmox Backup Server supports both strategies. |
85e139b7 DM |
25 | |
26 | ||
e11cfb93 DW |
27 | Image Archives: ``<name>.img`` |
28 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
29 | ||
30 | This is used for virtual machine images and other large binary | |
31 | data. Content is split into fixed-sized chunks. | |
32 | ||
33 | ||
57905a61 DM |
34 | File Archives: ``<name>.pxar`` |
35 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
85e139b7 DM |
36 | |
37 | .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/ | |
38 | ||
4f3db187 | 39 | A file archive stores a full directory tree. Content is stored using |
8c6e5ce2 | 40 | the :ref:`pxar-format`, split into variable-sized chunks. The format |
4f3db187 | 41 | is optimized to achieve good deduplication rates. |
85e139b7 DM |
42 | |
43 | ||
85e139b7 DM |
44 | Binary Data (BLOBs) |
45 | ^^^^^^^^^^^^^^^^^^^ | |
46 | ||
4f3db187 AL |
47 | This type is used to store smaller (< 16MB) binary data such as |
48 | configuration files. Larger files should be stored as image archive. | |
85e139b7 DM |
49 | |
50 | .. caution:: Please do not store all files as BLOBs. Instead, use the | |
51 | file archive to store whole directory trees. | |
52 | ||
53 | ||
57905a61 DM |
54 | Catalog File: ``catalog.pcat1`` |
55 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
56 | ||
4f3db187 | 57 | The catalog file is an index for file archives. It contains |
8c6e5ce2 | 58 | the list of files and is used to speed up search operations. |
57905a61 DM |
59 | |
60 | ||
61 | The Manifest: ``index.json`` | |
62 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
63 | ||
4f3db187 AL |
64 | The manifest contains the list of all backup files, their |
65 | sizes and checksums. It is used to verify the consistency of a | |
57905a61 DM |
66 | backup. |
67 | ||
68 | ||
fea8789c DM |
69 | Backup Type |
70 | ~~~~~~~~~~~ | |
71 | ||
72 | The backup server groups backups by *type*, where *type* is one of: | |
73 | ||
74 | ``vm`` | |
a129fdd9 | 75 | This type is used for :term:`virtual machine`\ s. Typically |
8c6e5ce2 | 76 | consists of the virtual machine's configuration file and an image archive |
fea8789c DM |
77 | for each disk. |
78 | ||
79 | ``ct`` | |
8c6e5ce2 OB |
80 | This type is used for :term:`container`\ s. Consists of the container's |
81 | configuration and a single file archive for the filesystem content. | |
fea8789c DM |
82 | |
83 | ``host`` | |
4f3db187 AL |
84 | This type is used for backups created from within the backed up machine. |
85 | Typically this would be a physical host but could also be a virtual machine | |
86 | or container. Such backups may contain file and image archives, there are no restrictions in this regard. | |
fea8789c DM |
87 | |
88 | ||
89 | Backup ID | |
90 | ~~~~~~~~~ | |
91 | ||
8c6e5ce2 | 92 | A unique ID. Usually the virtual machine or container ID. ``host`` |
fea8789c DM |
93 | type backups normally use the hostname. |
94 | ||
95 | ||
96 | Backup Time | |
97 | ~~~~~~~~~~~ | |
98 | ||
99 | The time when the backup was made. | |
100 | ||
101 | ||
6e5a0c03 DM |
102 | Backup Group |
103 | ~~~~~~~~~~~~ | |
104 | ||
4f3db187 AL |
105 | The tuple ``<type>/<ID>`` is called a backup group. Such a group |
106 | may contain one or more backup snapshots. | |
6e5a0c03 DM |
107 | |
108 | ||
fea8789c DM |
109 | Backup Snapshot |
110 | ~~~~~~~~~~~~~~~ | |
111 | ||
4f3db187 | 112 | The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It |
fea8789c DM |
113 | uniquely identifies a specific backup within a datastore. |
114 | ||
115 | .. code-block:: console | |
116 | :caption: Backup Snapshot Examples | |
117 | ||
118 | vm/104/2019-10-09T08:01:06Z | |
119 | host/elsa/2019-11-08T09:48:14Z | |
120 | ||
4f3db187 | 121 | As you can see, the time format is RFC3399_ with Coordinated |
fea8789c DM |
122 | Universal Time (UTC_, identified by the trailing *Z*). |
123 | ||
8c6e5ce2 OB |
124 | Backup Server Management |
125 | ------------------------ | |
126 | ||
127 | The command line tool to configure and manage the backup server is called | |
128 | :command:`proxmox-backup-manager`. | |
129 | ||
298c6aae | 130 | .. _datastore_intro: |
fea8789c DM |
131 | |
132 | :term:`DataStore` | |
133 | ~~~~~~~~~~~~~~~~~ | |
134 | ||
9d3f183b DW |
135 | A datastore refers to a location at which backups are stored. The current |
136 | implementation uses a directory inside a standard unix file system (``ext4``, | |
137 | ``xfs`` or ``zfs``) to store the backup data. | |
fea8789c | 138 | |
9d3f183b DW |
139 | Datastores are identified by a simple *ID*. You can configure this |
140 | when setting up the datastore. The configuration information for datastores | |
141 | is stored in the file ``/etc/proxmox-backup/datastore.cfg``. | |
fea8789c | 142 | |
538c2b6d TL |
143 | .. note:: The `File Layout`_ requires the file system to support at least *65538* |
144 | subdirectories per directory. That number comes from the 2\ :sup:`16` | |
145 | pre-created chunk namespace directories, and the ``.`` and ``..`` default | |
146 | directory entries. This requirement excludes certain filesystems and | |
147 | filesystem configuration from being supported for a datastore. For example, | |
148 | ``ext3`` as a whole or ``ext4`` with the ``dir_nlink`` feature manually disabled. | |
fea8789c | 149 | |
dce9dd6f DW |
150 | Disk Management |
151 | ~~~~~~~~~~~~~~~ | |
b0c10a88 DW |
152 | |
153 | .. image:: images/screenshots/pbs-gui-disks.png | |
b0c10a88 DW |
154 | :align: right |
155 | :alt: List of disks | |
156 | ||
dce9dd6f DW |
157 | Proxmox Backup Server comes with a set of disk utilities, which are |
158 | accessed using the ``disk`` subcommand. This subcommand allows you to initialize | |
159 | disks, create various filesystems, and get information about the disks. | |
160 | ||
b0c10a88 DW |
161 | To view the disks connected to the system, navigate to **Administration -> |
162 | Disks** in the web interface or use the ``list`` subcommand of | |
dce9dd6f DW |
163 | ``disk``: |
164 | ||
165 | .. code-block:: console | |
166 | ||
167 | # proxmox-backup-manager disk list | |
168 | ┌──────┬────────┬─────┬───────────┬─────────────┬───────────────┬─────────┬────────┐ | |
169 | │ name │ used │ gpt │ disk-type │ size │ model │ wearout │ status │ | |
170 | ╞══════╪════════╪═════╪═══════════╪═════════════╪═══════════════╪═════════╪════════╡ | |
171 | │ sda │ lvm │ 1 │ hdd │ 34359738368 │ QEMU_HARDDISK │ - │ passed │ | |
172 | ├──────┼────────┼─────┼───────────┼─────────────┼───────────────┼─────────┼────────┤ | |
173 | │ sdb │ unused │ 1 │ hdd │ 68719476736 │ QEMU_HARDDISK │ - │ passed │ | |
174 | ├──────┼────────┼─────┼───────────┼─────────────┼───────────────┼─────────┼────────┤ | |
175 | │ sdc │ unused │ 1 │ hdd │ 68719476736 │ QEMU_HARDDISK │ - │ passed │ | |
176 | └──────┴────────┴─────┴───────────┴─────────────┴───────────────┴─────────┴────────┘ | |
177 | ||
178 | To initialize a disk with a new GPT, use the ``initialize`` subcommand: | |
179 | ||
180 | .. code-block:: console | |
181 | ||
182 | # proxmox-backup-manager disk initialize sdX | |
183 | ||
b0c10a88 | 184 | .. image:: images/screenshots/pbs-gui-disks-dir-create.png |
b0c10a88 DW |
185 | :align: right |
186 | :alt: Create a directory | |
187 | ||
188 | You can create an ``ext4`` or ``xfs`` filesystem on a disk using ``fs | |
189 | create``, or by navigating to **Administration -> Disks -> Directory** in the | |
190 | web interface and creating one from there. The following command creates an | |
191 | ``ext4`` filesystem and passes the ``--add-datastore`` parameter, in order to | |
192 | automatically create a datastore on the disk (in this case ``sdd``). This will | |
193 | create a datastore at the location ``/mnt/datastore/store1``: | |
194 | ||
dce9dd6f DW |
195 | .. code-block:: console |
196 | ||
197 | # proxmox-backup-manager disk fs create store1 --disk sdd --filesystem ext4 --add-datastore true | |
dce9dd6f | 198 | |
b0c10a88 | 199 | .. image:: images/screenshots/pbs-gui-disks-zfs-create.png |
b0c10a88 | 200 | :align: right |
fe233f3b | 201 | :alt: Create ZFS |
b0c10a88 DW |
202 | |
203 | You can also create a ``zpool`` with various raid levels from **Administration | |
204 | -> Disks -> Zpool** in the web interface, or by using ``zpool create``. The command | |
205 | below creates a mirrored ``zpool`` using two disks (``sdb`` & ``sdc``) and | |
206 | mounts it on the root directory (default): | |
207 | ||
dce9dd6f DW |
208 | .. code-block:: console |
209 | ||
210 | # proxmox-backup-manager disk zpool create zpool1 --devices sdb,sdc --raidlevel mirror | |
dce9dd6f | 211 | |
fe233f3b | 212 | .. note:: You can also pass the ``--add-datastore`` parameter here, to automatically |
dce9dd6f DW |
213 | create a datastore from the disk. |
214 | ||
215 | You can use ``disk fs list`` and ``disk zpool list`` to keep track of your | |
216 | filesystems and zpools respectively. | |
217 | ||
9d3f183b DW |
218 | Proxmox Backup Server uses the package smartmontools. This is a set of tools |
219 | used to monitor and control the S.M.A.R.T. system for local hard disks. If a | |
220 | disk supports S.M.A.R.T. capability, and you have this enabled, you can | |
ff30b912 | 221 | display S.M.A.R.T. attributes from the web interface or by using the command: |
dce9dd6f DW |
222 | |
223 | .. code-block:: console | |
224 | ||
225 | # proxmox-backup-manager disk smart-attributes sdX | |
58ea88c8 | 226 | |
9d3f183b DW |
227 | .. note:: This functionality may also be accessed directly through the use of |
228 | the ``smartctl`` command, which comes as part of the smartmontools package | |
229 | (see ``man smartctl`` for more details). | |
230 | ||
ff30b912 | 231 | |
58ea88c8 DM |
232 | Datastore Configuration |
233 | ~~~~~~~~~~~~~~~~~~~~~~~ | |
234 | ||
ff30b912 | 235 | .. image:: images/screenshots/pbs-gui-datastore.png |
ff30b912 DW |
236 | :align: right |
237 | :alt: Datastore Overview | |
238 | ||
8c6e5ce2 | 239 | You can configure multiple datastores. Minimum one datastore needs to be |
ff30b912 | 240 | configured. The datastore is identified by a simple *name* and points to a |
22231524 SI |
241 | directory on the filesystem. Each datastore also has associated retention |
242 | settings of how many backup snapshots for each interval of ``hourly``, | |
aef49768 | 243 | ``daily``, ``weekly``, ``monthly``, ``yearly`` as well as a time-independent |
22231524 SI |
244 | number of backups to keep in that store. :ref:`Pruning <pruning>` and |
245 | :ref:`garbage collection <garbage-collection>` can also be configured to run | |
1eff9a1e | 246 | periodically based on a configured schedule (see :ref:`calendar-events`) per datastore. |
58ea88c8 | 247 | |
ff30b912 DW |
248 | Creating a Datastore |
249 | ^^^^^^^^^^^^^^^^^^^^ | |
250 | .. image:: images/screenshots/pbs-gui-datastore-create-general.png | |
ff30b912 | 251 | :align: right |
092378ba | 252 | :alt: Create a datastore |
ff30b912 DW |
253 | |
254 | You can create a new datastore from the web GUI, by navigating to **Datastore** in | |
255 | the menu tree and clicking **Create**. Here: | |
256 | ||
257 | * *Name* refers to the name of the datastore | |
258 | * *Backing Path* is the path to the directory upon which you want to create the | |
259 | datastore | |
260 | * *GC Schedule* refers to the time and intervals at which garbage collection | |
261 | runs | |
262 | * *Prune Schedule* refers to the frequency at which pruning takes place | |
263 | * *Prune Options* set the amount of backups which you would like to keep (see :ref:`Pruning <pruning>`). | |
264 | ||
265 | Alternatively you can create a new datastore from the command line. The | |
266 | following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1` | |
58ea88c8 DM |
267 | |
268 | .. code-block:: console | |
269 | ||
270 | # proxmox-backup-manager datastore create store1 /backup/disk1/store1 | |
271 | ||
ff30b912 DW |
272 | Managing Datastores |
273 | ^^^^^^^^^^^^^^^^^^^ | |
274 | ||
275 | To list existing datastores from the command line run: | |
58ea88c8 DM |
276 | |
277 | .. code-block:: console | |
278 | ||
279 | # proxmox-backup-manager datastore list | |
17ec699d DM |
280 | ┌────────┬──────────────────────┬─────────────────────────────┐ |
281 | │ name │ path │ comment │ | |
282 | ╞════════╪══════════════════════╪═════════════════════════════╡ | |
283 | │ store1 │ /backup/disk1/store1 │ This is my default storage. │ | |
284 | └────────┴──────────────────────┴─────────────────────────────┘ | |
58ea88c8 | 285 | |
ff30b912 DW |
286 | You can change the garbage collection and prune settings of a datastore, by |
287 | editing the datastore from the GUI or by using the ``update`` subcommand. For | |
288 | example, the below command changes the garbage collection schedule using the | |
289 | ``update`` subcommand and prints the properties of the datastore with the | |
290 | ``show`` subcommand: | |
22231524 SI |
291 | |
292 | .. code-block:: console | |
293 | ||
ff30b912 | 294 | # proxmox-backup-manager datastore update store1 --gc-schedule 'Tue 04:27' |
22231524 SI |
295 | # proxmox-backup-manager datastore show store1 |
296 | ┌────────────────┬─────────────────────────────┐ | |
297 | │ Name │ Value │ | |
298 | ╞════════════════╪═════════════════════════════╡ | |
299 | │ name │ store1 │ | |
300 | ├────────────────┼─────────────────────────────┤ | |
301 | │ path │ /backup/disk1/store1 │ | |
302 | ├────────────────┼─────────────────────────────┤ | |
303 | │ comment │ This is my default storage. │ | |
304 | ├────────────────┼─────────────────────────────┤ | |
305 | │ gc-schedule │ Tue 04:27 │ | |
306 | ├────────────────┼─────────────────────────────┤ | |
307 | │ keep-last │ 7 │ | |
308 | ├────────────────┼─────────────────────────────┤ | |
309 | │ prune-schedule │ daily │ | |
310 | └────────────────┴─────────────────────────────┘ | |
311 | ||
4f3db187 | 312 | Finally, it is possible to remove the datastore configuration: |
58ea88c8 DM |
313 | |
314 | .. code-block:: console | |
315 | ||
316 | # proxmox-backup-manager datastore remove store1 | |
317 | ||
4f3db187 | 318 | .. note:: The above command removes only the datastore configuration. It does |
58ea88c8 DM |
319 | not delete any data from the underlying directory. |
320 | ||
321 | ||
fea8789c DM |
322 | File Layout |
323 | ^^^^^^^^^^^ | |
324 | ||
8c6e5ce2 OB |
325 | After creating a datastore, the following default layout will appear: |
326 | ||
327 | .. code-block:: console | |
24406ebc | 328 | |
8c6e5ce2 OB |
329 | # ls -arilh /backup/disk1/store1 |
330 | 276493 -rw-r--r-- 1 backup backup 0 Jul 8 12:35 .lock | |
331 | 276490 drwxr-x--- 1 backup backup 1064960 Jul 8 12:35 .chunks | |
332 | ||
333 | `.lock` is an empty file used for process locking. | |
334 | ||
335 | The `.chunks` directory contains folders, starting from `0000` and taking hexadecimal values until `ffff`. These | |
336 | directories will store the chunked data after a backup operation has been executed. | |
337 | ||
338 | .. code-block:: console | |
24406ebc | 339 | |
8c6e5ce2 OB |
340 | # ls -arilh /backup/disk1/store1/.chunks |
341 | 545824 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 ffff | |
342 | 545823 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffe | |
343 | 415621 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffd | |
344 | 415620 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffc | |
345 | 353187 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffb | |
346 | 344995 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fffa | |
347 | 144079 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff9 | |
348 | 144078 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff8 | |
349 | 144077 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 fff7 | |
350 | ... | |
351 | 403180 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000c | |
352 | 403179 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000b | |
353 | 403177 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 000a | |
354 | 402530 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0009 | |
355 | 402513 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0008 | |
356 | 402509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0007 | |
357 | 276509 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0006 | |
358 | 276508 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0005 | |
359 | 276507 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0004 | |
360 | 276501 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0003 | |
361 | 276499 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0002 | |
362 | 276498 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0001 | |
363 | 276494 drwxr-x--- 2 backup backup 4.0K Jul 8 12:35 0000 | |
364 | 276489 drwxr-xr-x 3 backup backup 4.0K Jul 8 12:35 .. | |
365 | 276490 drwxr-x--- 1 backup backup 1.1M Jul 8 12:35 . | |
366 | ||
298c6aae | 367 | .. _user_mgmt: |
fea8789c | 368 | |
17ec699d DM |
369 | User Management |
370 | ~~~~~~~~~~~~~~~ | |
371 | ||
9fa39a46 | 372 | .. image:: images/screenshots/pbs-gui-user-management.png |
9fa39a46 DW |
373 | :align: right |
374 | :alt: User management | |
375 | ||
8c6e5ce2 | 376 | Proxmox Backup Server supports several authentication realms, and you need to |
17ec699d DM |
377 | choose the realm when you add a new user. Possible realms are: |
378 | ||
379 | :pam: Linux PAM standard authentication. Use this if you want to | |
8c6e5ce2 | 380 | authenticate as Linux system user (Users need to exist on the |
17ec699d DM |
381 | system). |
382 | ||
383 | :pbs: Proxmox Backup Server realm. This type stores hashed passwords in | |
384 | ``/etc/proxmox-backup/shadow.json``. | |
385 | ||
386 | After installation, there is a single user ``root@pam``, which | |
9d3f183b DW |
387 | corresponds to the Unix superuser. User configuration information is stored in the file |
388 | ``/etc/proxmox-backup/user.cfg``. You can use the | |
17ec699d DM |
389 | ``proxmox-backup-manager`` command line tool to list or manipulate |
390 | users: | |
391 | ||
392 | .. code-block:: console | |
393 | ||
394 | # proxmox-backup-manager user list | |
26d29e0e DM |
395 | ┌─────────────┬────────┬────────┬───────────┬──────────┬────────────────┬────────────────────┐ |
396 | │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │ | |
397 | ╞═════════════╪════════╪════════╪═══════════╪══════════╪════════════════╪════════════════════╡ | |
398 | │ root@pam │ 1 │ │ │ │ │ Superuser │ | |
399 | └─────────────┴────────┴────────┴───────────┴──────────┴────────────────┴────────────────────┘ | |
17ec699d | 400 | |
9fa39a46 | 401 | .. image:: images/screenshots/pbs-gui-user-management-add-user.png |
9fa39a46 DW |
402 | :align: right |
403 | :alt: Add a new user | |
17ec699d | 404 | |
9fa39a46 DW |
405 | The superuser has full administration rights on everything, so you |
406 | normally want to add other users with less privileges. You can create a new | |
407 | user with the ``user create`` subcommand or through the web interface, under | |
408 | **Configuration -> User Management**. The ``create`` subcommand lets you specify | |
409 | many options like ``--email`` or ``--password``. You can update or change any | |
410 | user properties using the ``update`` subcommand later (**Edit** in the GUI): | |
17ec699d | 411 | |
17ec699d DM |
412 | |
413 | .. code-block:: console | |
414 | ||
9fa39a46 | 415 | # proxmox-backup-manager user create john@pbs --email john@example.com |
17ec699d DM |
416 | # proxmox-backup-manager user update john@pbs --firstname John --lastname Smith |
417 | # proxmox-backup-manager user update john@pbs --comment "An example user." | |
418 | ||
17ec699d DM |
419 | .. todo:: Mention how to set password without passing plaintext password as cli argument. |
420 | ||
421 | ||
8c6e5ce2 | 422 | The resulting user list looks like this: |
17ec699d DM |
423 | |
424 | .. code-block:: console | |
425 | ||
426 | # proxmox-backup-manager user list | |
427 | ┌──────────┬────────┬────────┬───────────┬──────────┬──────────────────┬──────────────────┐ | |
428 | │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │ | |
429 | ╞══════════╪════════╪════════╪═══════════╪══════════╪══════════════════╪══════════════════╡ | |
430 | │ john@pbs │ 1 │ │ John │ Smith │ john@example.com │ An example user. │ | |
431 | ├──────────┼────────┼────────┼───────────┼──────────┼──────────────────┼──────────────────┤ | |
432 | │ root@pam │ 1 │ │ │ │ │ Superuser │ | |
433 | └──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘ | |
434 | ||
8c6e5ce2 | 435 | Newly created users do not have any permissions. Please read the next |
17ec699d DM |
436 | section to learn how to set access permissions. |
437 | ||
8c6e5ce2 | 438 | If you want to disable a user account, you can do that by setting ``--enable`` to ``0`` |
8f3b3cc1 DM |
439 | |
440 | .. code-block:: console | |
441 | ||
442 | # proxmox-backup-manager user update john@pbs --enable 0 | |
443 | ||
8c6e5ce2 | 444 | Or completely remove the user with: |
8f3b3cc1 DM |
445 | |
446 | .. code-block:: console | |
447 | ||
448 | # proxmox-backup-manager user remove john@pbs | |
449 | ||
450 | ||
298c6aae OB |
451 | .. _user_acl: |
452 | ||
17ec699d DM |
453 | Access Control |
454 | ~~~~~~~~~~~~~~ | |
455 | ||
8c6e5ce2 OB |
456 | By default new users do not have any permission. Instead you need to |
457 | specify what is allowed and what is not. You can do this by assigning | |
8df51d48 DM |
458 | roles to users on specific objects like datastores or remotes. The |
459 | following roles exist: | |
460 | ||
8c6e5ce2 OB |
461 | **NoAccess** |
462 | Disable Access - nothing is allowed. | |
463 | ||
8df51d48 | 464 | **Admin** |
4cda7603 | 465 | Can do anything. |
8df51d48 DM |
466 | |
467 | **Audit** | |
4cda7603 | 468 | Can view things, but is not allowed to change settings. |
8df51d48 | 469 | |
8df51d48 DM |
470 | **DatastoreAdmin** |
471 | Can do anything on datastores. | |
472 | ||
473 | **DatastoreAudit** | |
474 | Can view datastore settings and list content. But | |
475 | is not allowed to read the actual data. | |
476 | ||
74fc8447 | 477 | **DatastoreReader** |
8df51d48 DM |
478 | Can Inspect datastore content and can do restores. |
479 | ||
74fc8447 | 480 | **DatastoreBackup** |
8df51d48 DM |
481 | Can backup and restore owned backups. |
482 | ||
483 | **DatastorePowerUser** | |
484 | Can backup, restore, and prune owned backups. | |
485 | ||
486 | **RemoteAdmin** | |
487 | Can do anything on remotes. | |
488 | ||
489 | **RemoteAudit** | |
490 | Can view remote settings. | |
491 | ||
492 | **RemoteSyncOperator** | |
493 | Is allowed to read data from a remote. | |
494 | ||
fe233f3b | 495 | .. image:: images/screenshots/pbs-gui-permissions-add.png |
9fa39a46 DW |
496 | :align: right |
497 | :alt: Add permissions for user | |
498 | ||
9d3f183b DW |
499 | Access permission information is stored in ``/etc/proxmox-backup/acl.cfg``. The |
500 | file contains 5 fields, separated using a colon (':') as a delimiter. A typical | |
501 | entry takes the form: | |
502 | ||
503 | ``acl:1:/datastore:john@pbs:DatastoreBackup`` | |
504 | ||
505 | The data represented in each field is as follows: | |
506 | ||
507 | #. ``acl`` identifier | |
508 | #. A ``1`` or ``0``, representing whether propagation is enabled or disabled, | |
509 | respectively | |
510 | #. The object on which the permission is set. This can be a specific object | |
511 | (single datastore, remote, etc.) or a top level object, which with | |
512 | propagation enabled, represents all children of the object also. | |
513 | #. The user for which the permission is set | |
514 | #. The role being set | |
515 | ||
516 | You can manage datastore permissions from **Configuration -> Permissions** in the | |
517 | web interface. Likewise, you can use the ``acl`` subcommand to manage and | |
9fa39a46 DW |
518 | monitor user permissions from the command line. For example, the command below |
519 | will add the user ``john@pbs`` as a **DatastoreAdmin** for the datastore | |
520 | ``store1``, located at ``/backup/disk1/store1``: | |
1e68497c DW |
521 | |
522 | .. code-block:: console | |
523 | ||
524 | # proxmox-backup-manager acl update /datastore/store1 DatastoreAdmin --userid john@pbs | |
525 | ||
526 | You can monitor the roles of each user using the following command: | |
527 | ||
528 | .. code-block:: console | |
529 | ||
530 | # proxmox-backup-manager acl list | |
531 | ┌──────────┬──────────────────┬───────────┬────────────────┐ | |
532 | │ ugid │ path │ propagate │ roleid │ | |
533 | ╞══════════╪══════════════════╪═══════════╪════════════════╡ | |
534 | │ john@pbs │ /datastore/disk1 │ 1 │ DatastoreAdmin │ | |
535 | └──────────┴──────────────────┴───────────┴────────────────┘ | |
536 | ||
092378ba | 537 | A single user can be assigned multiple permission sets for different datastores. |
1e68497c DW |
538 | |
539 | .. Note:: | |
092378ba | 540 | Naming convention is important here. For datastores on the host, |
1e68497c | 541 | you must use the convention ``/datastore/{storename}``. For example, to set |
092378ba | 542 | permissions for a datastore mounted at ``/mnt/backup/disk4/store2``, you would use |
1e68497c DW |
543 | ``/datastore/store2`` for the path. For remote stores, use the convention |
544 | ``/remote/{remote}/{storename}``, where ``{remote}`` signifies the name of the | |
092378ba | 545 | remote (see `Remote` below) and ``{storename}`` is the name of the datastore on |
1e68497c | 546 | the remote. |
17ec699d | 547 | |
24b638bd DW |
548 | Network Management |
549 | ~~~~~~~~~~~~~~~~~~ | |
36be1921 DW |
550 | |
551 | Proxmox Backup Server provides both a web interface and a command line tool for | |
552 | network configuration. You can find the configuration options in the web | |
553 | interface under the **Network Interfaces** section of the **Configuration** menu | |
554 | tree item. The command line tool is accessed via the ``network`` subcommand. | |
555 | These interfaces allow you to carry out some basic network management tasks, | |
556 | such as adding, configuring, and removing network interfaces. | |
557 | ||
558 | .. note:: Any changes made to the network configuration are not | |
559 | applied, until you click on **Apply Configuration** or enter the ``network | |
560 | reload`` command. This allows you to make many changes at once. It also allows | |
561 | you to ensure that your changes are correct before applying them, as making a | |
562 | mistake here can render the server inaccessible over the network. | |
24b638bd DW |
563 | |
564 | To get a list of available interfaces, use the following command: | |
565 | ||
566 | .. code-block:: console | |
567 | ||
568 | # proxmox-backup-manager network list | |
0623674f DW |
569 | ┌───────┬────────┬───────────┬────────┬─────────────┬──────────────┬──────────────┐ |
570 | │ name │ type │ autostart │ method │ address │ gateway │ ports/slaves │ | |
571 | ╞═══════╪════════╪═══════════╪════════╪═════════════╪══════════════╪══════════════╡ | |
572 | │ bond0 │ bond │ 1 │ static │ x.x.x.x/x │ x.x.x.x │ ens18 ens19 │ | |
573 | ├───────┼────────┼───────────┼────────┼─────────────┼──────────────┼──────────────┤ | |
574 | │ ens18 │ eth │ 1 │ manual │ │ │ │ | |
575 | ├───────┼────────┼───────────┼────────┼─────────────┼──────────────┼──────────────┤ | |
576 | │ ens19 │ eth │ 1 │ manual │ │ │ │ | |
577 | └───────┴────────┴───────────┴────────┴─────────────┴──────────────┴──────────────┘ | |
24b638bd | 578 | |
36be1921 | 579 | .. image:: images/screenshots/pbs-gui-network-create-bond.png |
36be1921 DW |
580 | :align: right |
581 | :alt: Add a network interface | |
582 | ||
0623674f | 583 | To add a new network interface, use the ``create`` subcommand with the relevant |
9d3f183b DW |
584 | parameters. For example, you may want to set up a bond, for the purpose of |
585 | network redundancy. The following command shows a template for creating the bond shown | |
0623674f | 586 | in the list above: |
36be1921 | 587 | |
24b638bd DW |
588 | .. code-block:: console |
589 | ||
0623674f | 590 | # proxmox-backup-manager network create bond0 --type bond --bond_mode active-backup --slaves ens18,ens19 --autostart true --cidr x.x.x.x/x --gateway x.x.x.x |
24b638bd DW |
591 | |
592 | You can make changes to the configuration of a network interface with the | |
593 | ``update`` subcommand: | |
594 | ||
595 | .. code-block:: console | |
596 | ||
0623674f | 597 | # proxmox-backup-manager network update bond0 --cidr y.y.y.y/y |
24b638bd DW |
598 | |
599 | You can also remove a network interface: | |
600 | ||
601 | .. code-block:: console | |
602 | ||
0623674f | 603 | # proxmox-backup-manager network remove bond0 |
24b638bd | 604 | |
36be1921 DW |
605 | The pending changes for the network configuration file will appear at the bottom of the |
606 | web interface. You can also view these changes, by using the command: | |
24b638bd DW |
607 | |
608 | .. code-block:: console | |
609 | ||
610 | # proxmox-backup-manager network changes | |
611 | ||
36be1921 DW |
612 | If you would like to cancel all changes at this point, you can either click on |
613 | the **Revert** button or use the following command: | |
24b638bd DW |
614 | |
615 | .. code-block:: console | |
616 | ||
617 | # proxmox-backup-manager network revert | |
618 | ||
619 | If you are happy with the changes and would like to write them into the | |
36be1921 DW |
620 | configuration file, select **Apply Configuration**. The corresponding command |
621 | is: | |
24b638bd DW |
622 | |
623 | .. code-block:: console | |
624 | ||
625 | # proxmox-backup-manager network reload | |
626 | ||
9d3f183b DW |
627 | .. note:: This command and corresponding GUI button rely on the ``ifreload`` |
628 | command, from the package ``ifupdown2``. This package is included within the | |
629 | Proxmox Backup Server installation, however, you may have to install it yourself, | |
630 | if you have installed Proxmox Backup Server on top of Debian or Proxmox VE. | |
631 | ||
36be1921 DW |
632 | You can also configure DNS settings, from the **DNS** section |
633 | of **Configuration** or by using the ``dns`` subcommand of | |
24b638bd DW |
634 | ``proxmox-backup-manager``. |
635 | ||
298c6aae OB |
636 | .. _backup_remote: |
637 | ||
9634ca07 SI |
638 | :term:`Remote` |
639 | ~~~~~~~~~~~~~~ | |
640 | ||
aef49768 | 641 | A remote refers to a separate Proxmox Backup Server installation and a user on that |
9634ca07 | 642 | installation, from which you can `sync` datastores to a local datastore with a |
bc060895 | 643 | `Sync Job`. You can configure remotes in the web interface, under **Configuration |
9d3f183b DW |
644 | -> Remotes**. Alternatively, you can use the ``remote`` subcommand. The |
645 | configuration information for remotes is stored in the file | |
646 | ``/etc/proxmox-backup/remote.cfg``. | |
bc060895 DW |
647 | |
648 | .. image:: images/screenshots/pbs-gui-remote-add.png | |
bc060895 DW |
649 | :align: right |
650 | :alt: Add a remote | |
9634ca07 | 651 | |
aef49768 DW |
652 | To add a remote, you need its hostname or ip, a userid and password on the |
653 | remote, and its certificate fingerprint. To get the fingerprint, use the | |
bc060895 DW |
654 | ``proxmox-backup-manager cert info`` command on the remote, or navigate to |
655 | **Dashboard** in the remote's web interface and select **Show Fingerprint**. | |
9634ca07 SI |
656 | |
657 | .. code-block:: console | |
658 | ||
659 | # proxmox-backup-manager cert info |grep Fingerprint | |
660 | Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe | |
661 | ||
bc060895 DW |
662 | Using the information specified above, you can add a remote from the **Remotes** |
663 | configuration panel, or by using the command: | |
9634ca07 SI |
664 | |
665 | .. code-block:: console | |
666 | ||
667 | # proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe | |
668 | ||
669 | Use the ``list``, ``show``, ``update``, ``remove`` subcommands of | |
670 | ``proxmox-backup-manager remote`` to manage your remotes: | |
671 | ||
672 | .. code-block:: console | |
673 | ||
674 | # proxmox-backup-manager remote update pbs2 --host pbs2.example | |
675 | # proxmox-backup-manager remote list | |
676 | ┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐ | |
677 | │ name │ host │ userid │ fingerprint │ comment │ | |
678 | ╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡ | |
679 | │ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │ │ | |
680 | └──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘ | |
681 | # proxmox-backup-manager remote remove pbs2 | |
682 | ||
229adeb7 | 683 | .. _syncjobs: |
9634ca07 SI |
684 | |
685 | Sync Jobs | |
686 | ~~~~~~~~~ | |
687 | ||
bc060895 | 688 | .. image:: images/screenshots/pbs-gui-syncjob-add.png |
bc060895 | 689 | :align: right |
9d3f183b | 690 | :alt: Add a Sync Job |
bc060895 | 691 | |
9d3f183b DW |
692 | Sync jobs are configured to pull the contents of a datastore on a **Remote** to |
693 | a local datastore. You can manage sync jobs under **Configuration -> Sync Jobs** | |
694 | in the web interface, or using the ``proxmox-backup-manager sync-job`` command. | |
695 | The configuration information for sync jobs is stored at | |
696 | ``/etc/proxmox-backup/sync.cfg``. To create a new sync job, click the add button | |
697 | in the GUI, or use the ``create`` subcommand. After creating a sync job, you can | |
698 | either start it manually on the GUI or provide it with a schedule (see | |
699 | :ref:`calendar-events`) to run regularly. | |
9634ca07 SI |
700 | |
701 | .. code-block:: console | |
702 | ||
703 | # proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30' | |
704 | # proxmox-backup-manager sync-job update pbs2-local --comment 'offsite' | |
705 | # proxmox-backup-manager sync-job list | |
706 | ┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐ | |
707 | │ id │ store │ remote │ remote-store │ schedule │ comment │ | |
708 | ╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡ | |
709 | │ pbs2-local │ local │ pbs2 │ local │ Wed 02:30 │ offsite │ | |
710 | └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘ | |
711 | # proxmox-backup-manager sync-job remove pbs2-local | |
712 | ||
bc060895 | 713 | |
fd3f6901 DW |
714 | Garbage Collection |
715 | ~~~~~~~~~~~~~~~~~~ | |
716 | You can monitor and run :ref:`garbage collection <garbage-collection>` on the | |
717 | Proxmox Backup Server using the ``garbage-collection`` subcommand of | |
718 | ``proxmox-backup-manager``. You can use the ``start`` subcommand to manually start garbage | |
092378ba | 719 | collection on an entire datastore and the ``status`` subcommand to see |
fd3f6901 DW |
720 | attributes relating to the :ref:`garbage collection <garbage-collection>`. |
721 | ||
9634ca07 | 722 | |
cb01363c DM |
723 | Backup Client usage |
724 | ------------------- | |
58ea88c8 DM |
725 | |
726 | The command line client is called :command:`proxmox-backup-client`. | |
727 | ||
a129fdd9 | 728 | |
0c1c492d TL |
729 | Repository Locations |
730 | ~~~~~~~~~~~~~~~~~~~~ | |
58ea88c8 | 731 | |
4f3db187 | 732 | The client uses the following notation to specify a datastore repository |
58ea88c8 DM |
733 | on the backup server. |
734 | ||
ba20987a | 735 | [[username@]server[:port]:]datastore |
58ea88c8 | 736 | |
abf8b5d4 | 737 | The default value for ``username`` ist ``root@pam``. If no server is specified, |
8c6e5ce2 | 738 | the default is the local host (``localhost``). |
58ea88c8 | 739 | |
ba20987a DC |
740 | You can specify a port if your backup server is only reachable on a different |
741 | port (e.g. with NAT and port forwarding). | |
742 | ||
60b9b48e DC |
743 | Note that if the server is an IPv6 address, you have to write it with |
744 | square brackets (e.g. [fe80::01]). | |
745 | ||
4f3db187 AL |
746 | You can pass the repository with the ``--repository`` command |
747 | line option, or by setting the ``PBS_REPOSITORY`` environment | |
58ea88c8 DM |
748 | variable. |
749 | ||
ba20987a DC |
750 | Here some examples of valid repositories and the real values |
751 | ||
752 | ================================ ============ ================== =========== | |
753 | Example User Host:Port Datastore | |
754 | ================================ ============ ================== =========== | |
755 | mydatastore ``root@pam`` localhost:8007 mydatastore | |
756 | myhostname:mydatastore ``root@pam`` myhostname:8007 mydatastore | |
757 | user@pbs@myhostname:mydatastore ``user@pbs`` myhostname:8007 mydatastore | |
758 | 192.168.55.55:1234:mydatastore ``root@pam`` 192.168.55.55:1234 mydatastore | |
759 | [ff80::51]:mydatastore ``root@pam`` [ff80::51]:8007 mydatastore | |
760 | [ff80::51]:1234:mydatastore ``root@pam`` [ff80::51]:1234 mydatastore | |
761 | ================================ ============ ================== =========== | |
58ea88c8 DM |
762 | |
763 | Environment Variables | |
53ea6556 | 764 | ~~~~~~~~~~~~~~~~~~~~~ |
58ea88c8 DM |
765 | |
766 | ``PBS_REPOSITORY`` | |
767 | The default backup repository. | |
768 | ||
769 | ``PBS_PASSWORD`` | |
770 | When set, this value is used for the password required for the | |
771 | backup server. | |
772 | ||
773 | ``PBS_ENCRYPTION_PASSWORD`` | |
58ea88c8 DM |
774 | When set, this value is used to access the secret encryption key (if |
775 | protected by password). | |
776 | ||
3243f93c DM |
777 | ``PBS_FINGERPRINT`` When set, this value is used to verify the server |
778 | certificate (only used if the system CA certificates cannot | |
779 | validate the certificate). | |
780 | ||
53ea6556 DM |
781 | |
782 | Output Format | |
783 | ~~~~~~~~~~~~~ | |
784 | ||
4f3db187 AL |
785 | Most commands support the ``--output-format`` parameter. It accepts |
786 | the following values: | |
53ea6556 DM |
787 | |
788 | :``text``: Text format (default). Structured data is rendered as a table. | |
789 | ||
790 | :``json``: JSON (single line). | |
791 | ||
792 | :``json-pretty``: JSON (multiple lines, nicely formatted). | |
793 | ||
794 | ||
795 | Please use the following environment variables to modify output behavior: | |
796 | ||
797 | ``PROXMOX_OUTPUT_FORMAT`` | |
798 | Defines the default output format. | |
799 | ||
800 | ``PROXMOX_OUTPUT_NO_BORDER`` | |
801 | If set (to any value), do not render table borders. | |
802 | ||
803 | ``PROXMOX_OUTPUT_NO_HEADER`` | |
804 | If set (to any value), do not render table headers. | |
805 | ||
4f3db187 | 806 | .. note:: The ``text`` format is designed to be human readable, and |
53ea6556 | 807 | not meant to be parsed by automation tools. Please use the ``json`` |
4f3db187 | 808 | format if you need to process the output. |
53ea6556 DM |
809 | |
810 | ||
cee53b34 | 811 | .. _creating-backups: |
58ea88c8 DM |
812 | |
813 | Creating Backups | |
814 | ~~~~~~~~~~~~~~~~ | |
815 | ||
4f3db187 AL |
816 | This section explains how to create a backup from within the machine. This can |
817 | be a physical host, a virtual machine, or a container. Such backups may contain file | |
818 | and image archives. There are no restrictions in this case. | |
a129fdd9 | 819 | |
8c6e5ce2 | 820 | .. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`. |
a129fdd9 | 821 | |
4f3db187 AL |
822 | For the following example you need to have a backup server set up, working |
823 | credentials and need to know the repository name. | |
824 | In the following examples we use ``backup-server:store1``. | |
a129fdd9 DM |
825 | |
826 | .. code-block:: console | |
827 | ||
828 | # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1 | |
829 | Starting backup: host/elsa/2019-12-03T09:35:01Z | |
830 | Client name: elsa | |
831 | skip mount point: "/boot/efi" | |
832 | skip mount point: "/dev" | |
833 | skip mount point: "/run" | |
834 | skip mount point: "/sys" | |
835 | Uploaded 12129 chunks in 87 seconds (564 MB/s). | |
836 | End Time: 2019-12-03T10:36:29+01:00 | |
837 | ||
838 | This will prompt you for a password and then uploads a file archive named | |
839 | ``root.pxar`` containing all the files in the ``/`` directory. | |
840 | ||
4f3db187 | 841 | .. Caution:: Please note that the proxmox-backup-client does not |
ed858b0a | 842 | automatically include mount points. Instead, you will see a short |
4f3db187 AL |
843 | ``skip mount point`` notice for each of them. The idea is to |
844 | create a separate file archive for each mounted disk. You can | |
a129fdd9 DM |
845 | explicitly include them using the ``--include-dev`` option |
846 | (i.e. ``--include-dev /boot/efi``). You can use this option | |
4f3db187 | 847 | multiple times for each mount point that should be included. |
a129fdd9 | 848 | |
4f3db187 | 849 | The ``--repository`` option can get quite long and is used by all |
a129fdd9 | 850 | commands. You can avoid having to enter this value by setting the |
74fc8447 DW |
851 | environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set |
852 | over multiple sessions, you should instead add the below line to your | |
853 | ``.bashrc`` file. | |
a129fdd9 DM |
854 | |
855 | .. code-block:: console | |
856 | ||
78ee20d7 | 857 | # export PBS_REPOSITORY=backup-server:store1 |
a129fdd9 | 858 | |
4f3db187 | 859 | After this you can execute all commands without specifying the ``--repository`` |
a129fdd9 DM |
860 | option. |
861 | ||
4f3db187 AL |
862 | One single backup is allowed to contain more than one archive. For example, if |
863 | you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``: | |
a129fdd9 DM |
864 | |
865 | .. code-block:: console | |
866 | ||
867 | # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 | |
868 | ||
4f3db187 | 869 | This creates a backup of both disks. |
a129fdd9 DM |
870 | |
871 | The backup command takes a list of backup specifications, which | |
4f3db187 AL |
872 | include the archive name on the server, the type of the archive, and the |
873 | archive source at the client. The format is: | |
a129fdd9 DM |
874 | |
875 | <archive-name>.<type>:<source-path> | |
876 | ||
877 | Common types are ``.pxar`` for file archives, and ``.img`` for block | |
4f3db187 | 878 | device images. To create a backup of a block device run the following command: |
a129fdd9 DM |
879 | |
880 | .. code-block:: console | |
881 | ||
882 | # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata | |
883 | ||
50b8f9dd CE |
884 | Excluding files/folders from a backup |
885 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
886 | ||
887 | Sometimes it is desired to exclude certain files or folders from a backup archive. | |
4cda7603 | 888 | To tell the Proxmox Backup client when and how to ignore files and directories, |
4f3db187 AL |
889 | place a text file called ``.pxarexclude`` in the filesystem hierarchy. |
890 | Whenever the backup client encounters such a file in a directory, it interprets | |
891 | each line as glob match patterns for files and directories that are to be excluded | |
892 | from the backup. | |
893 | ||
894 | The file must contain a single glob pattern per line. Empty lines are ignored. | |
895 | The same is true for lines starting with ``#``, which indicates a comment. | |
896 | A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion | |
897 | to an explicit inclusion. This makes it possible to exclude all entries in a | |
898 | directory except for a few single files/subdirectories. | |
899 | Lines ending in ``/`` match only on directories. | |
900 | The directory containing the ``.pxarexclude`` file is considered to be the root of | |
901 | the given patterns. It is only possible to match files in this directory and its subdirectories. | |
902 | ||
903 | ``\`` is used to escape special glob characters. | |
904 | ``?`` matches any single character. | |
905 | ``*`` matches any character, including an empty string. | |
906 | ``**`` is used to match subdirectories. It can be used to, for example, exclude | |
907 | all files ending in ``.tmp`` within the directory or subdirectories with the | |
50b8f9dd CE |
908 | following pattern ``**/*.tmp``. |
909 | ``[...]`` matches a single character from any of the provided characters within | |
0c1c492d | 910 | the brackets. ``[!...]`` does the complementary and matches any single character |
4f3db187 AL |
911 | not contained within the brackets. It is also possible to specify ranges with two |
912 | characters separated by ``-``. For example, ``[a-z]`` matches any lowercase | |
913 | alphabetic character and ``[0-9]`` matches any one single digit. | |
50b8f9dd | 914 | |
aef49768 DW |
915 | The order of the glob match patterns defines whether a file is included or |
916 | excluded, that is to say later entries override previous ones. | |
50b8f9dd | 917 | This is also true for match patterns encountered deeper down the directory tree, |
4f3db187 AL |
918 | which can override a previous exclusion. |
919 | Be aware that excluded directories will **not** be read by the backup client. | |
aef49768 | 920 | Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect. |
4f3db187 | 921 | ``.pxarexclude`` files are treated as regular files and will be included in the |
50b8f9dd CE |
922 | backup archive. |
923 | ||
4f3db187 | 924 | For example, consider the following directory structure: |
50b8f9dd CE |
925 | |
926 | .. code-block:: console | |
927 | ||
928 | # ls -aR folder | |
929 | folder/: | |
930 | . .. .pxarexclude subfolder0 subfolder1 | |
931 | ||
932 | folder/subfolder0: | |
933 | . .. file0 file1 file2 file3 .pxarexclude | |
934 | ||
935 | folder/subfolder1: | |
936 | . .. file0 file1 file2 file3 | |
937 | ||
4f3db187 | 938 | The different ``.pxarexclude`` files contain the following: |
50b8f9dd CE |
939 | |
940 | .. code-block:: console | |
941 | ||
942 | # cat folder/.pxarexclude | |
943 | /subfolder0/file1 | |
944 | /subfolder1/* | |
945 | !/subfolder1/file2 | |
946 | ||
947 | .. code-block:: console | |
948 | ||
949 | # cat folder/subfolder0/.pxarexclude | |
950 | file3 | |
951 | ||
952 | This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of | |
953 | ``subfolder1`` except ``file2``. | |
954 | ||
4f3db187 | 955 | Restoring this backup will result in: |
50b8f9dd CE |
956 | |
957 | .. code-block:: console | |
958 | ||
959 | ls -aR restored | |
960 | restored/: | |
961 | . .. .pxarexclude subfolder0 subfolder1 | |
962 | ||
963 | restored/subfolder0: | |
964 | . .. file0 file2 .pxarexclude | |
965 | ||
966 | restored/subfolder1: | |
967 | . .. file2 | |
a129fdd9 | 968 | |
58ea88c8 | 969 | Encryption |
747c3bc0 | 970 | ~~~~~~~~~~ |
58ea88c8 | 971 | |
aef49768 DW |
972 | Proxmox Backup supports client-side encryption with AES-256 in GCM_ |
973 | mode. To set this up, you first need to create an encryption key: | |
5a499f32 DM |
974 | |
975 | .. code-block:: console | |
976 | ||
977 | # proxmox-backup-client key create my-backup.key | |
978 | Encryption Key Password: ************** | |
979 | ||
980 | The key is password protected by default. If you do not need this | |
981 | extra protection, you can also create it without a password: | |
982 | ||
983 | .. code-block:: console | |
984 | ||
4f3db187 | 985 | # proxmox-backup-client key create /path/to/my-backup.key --kdf none |
5a499f32 | 986 | |
16a18dad DW |
987 | Having created this key, it is now possible to create an encrypted backup, by |
988 | passing the ``--keyfile`` parameter, with the path to the key file. | |
5a499f32 DM |
989 | |
990 | .. code-block:: console | |
991 | ||
992 | # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key | |
993 | Password: ********* | |
994 | Encryption Key Password: ************** | |
995 | ... | |
996 | ||
16a18dad DW |
997 | .. Note:: If you do not specify the name of the backup key, the key will be |
998 | created in the default location | |
999 | ``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client`` | |
1000 | will also search this location by default, in case the ``--keyfile`` | |
1001 | parameter is not specified. | |
5a499f32 | 1002 | |
4f3db187 | 1003 | You can avoid entering the passwords by setting the environment |
5a499f32 DM |
1004 | variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``. |
1005 | ||
16a18dad | 1006 | Using a master key to store and recover encryption keys |
c23e257c | 1007 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
16a18dad DW |
1008 | |
1009 | You can also use ``proxmox-backup-client key`` to create an RSA public/private | |
1010 | key pair, which can be used to store an encrypted version of the symmetric | |
1011 | backup encryption key alongside each backup and recover it later. | |
1012 | ||
1013 | To set up a master key: | |
1014 | ||
1015 | 1. Create an encryption key for the backup: | |
1016 | ||
1017 | .. code-block:: console | |
1018 | ||
1019 | # proxmox-backup-client key create | |
1020 | creating default key at: "~/.config/proxmox-backup/encryption-key.json" | |
1021 | Encryption Key Password: ********** | |
1022 | ... | |
1023 | ||
1024 | The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``. | |
1025 | ||
1026 | 2. Create an RSA public/private key pair: | |
1027 | ||
1028 | .. code-block:: console | |
1029 | ||
1030 | # proxmox-backup-client key create-master-key | |
1031 | Master Key Password: ********* | |
1032 | ... | |
1033 | ||
1034 | This will create two files in your current directory, ``master-public.pem`` | |
1035 | and ``master-private.pem``. | |
1036 | ||
1037 | 3. Import the newly created ``master-public.pem`` public certificate, so that | |
1038 | ``proxmox-backup-client`` can find and use it upon backup. | |
1039 | ||
1040 | .. code-block:: console | |
1041 | ||
1042 | # proxmox-backup-client key import-master-pubkey /path/to/master-public.pem | |
1043 | Imported public master key to "~/.config/proxmox-backup/master-public.pem" | |
1044 | ||
1045 | 4. With all these files in place, run a backup job: | |
1046 | ||
1047 | .. code-block:: console | |
1048 | ||
1049 | # proxmox-backup-client backup etc.pxar:/etc | |
1050 | ||
1051 | The key will be stored in your backup, under the name ``rsa-encrypted.key``. | |
1052 | ||
1053 | .. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key | |
1054 | is in the default path. If you specified another path upon creation, you | |
1055 | must pass the ``--keyfile`` parameter. | |
1056 | ||
1057 | 5. To test that everything worked, you can restore the key from the backup: | |
1058 | ||
1059 | .. code-block:: console | |
1060 | ||
1061 | # proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target | |
1062 | ||
1063 | .. Note:: You should not need an encryption key to extract this file. However, if | |
1064 | a key exists at the default location | |
1065 | (``~/.config/proxmox-backup/encryption-key.json``) the program will prompt | |
1066 | you for an encryption key password. Simply moving ``encryption-key.json`` | |
1067 | out of this directory will fix this issue. | |
1068 | ||
1069 | 6. Then, use the previously generated master key to decrypt the file: | |
1070 | ||
1071 | .. code-block:: console | |
1072 | ||
1073 | # openssl rsautl -decrypt -inkey master-private.pem -in rsa-encrypted.key -out /path/to/target | |
1074 | Enter pass phrase for ./master-private.pem: ********* | |
1075 | ||
1076 | 7. The target file will now contain the encryption key information in plain | |
1077 | text. The success of this can be confirmed by passing the resulting ``json`` | |
1078 | file, with the ``--keyfile`` parameter, when decrypting files from the backup. | |
5a499f32 | 1079 | |
16a18dad DW |
1080 | .. warning:: Without their key, backed up files will be inaccessible. Thus, you should |
1081 | keep keys ordered and in a place that is separate from the contents being | |
1082 | backed up. It can happen, for example, that you back up an entire system, using | |
1083 | a key on that system. If the system then becomes inaccessable for any reason | |
1084 | and needs to be restored, this will not be possible as the encryption key will be | |
74fc8447 DW |
1085 | lost along with the broken system. In preparation for the worst case scenario, |
1086 | you should consider keeping a paper copy of this key locked away in | |
1087 | a safe place. | |
58ea88c8 DM |
1088 | |
1089 | Restoring Data | |
1090 | ~~~~~~~~~~~~~~ | |
1091 | ||
aef49768 DW |
1092 | The regular creation of backups is a necessary step to avoiding data |
1093 | loss. More importantly, however, is the restoration. It is good practice to perform | |
4f3db187 | 1094 | periodic recovery tests to ensure that you can access the data in |
64b85116 DM |
1095 | case of problems. |
1096 | ||
4f3db187 | 1097 | First, you need to find the snapshot which you want to restore. The snapshot |
aef49768 | 1098 | command provides a list of all the snapshots on the server: |
64b85116 DM |
1099 | |
1100 | .. code-block:: console | |
1101 | ||
1102 | # proxmox-backup-client snapshots | |
96feecd6 DM |
1103 | ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐ |
1104 | │ snapshot │ size │ files │ | |
1105 | ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡ | |
1106 | │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │ | |
1107 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
1108 | │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │ | |
1109 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
64b85116 DM |
1110 | ... |
1111 | ||
4f3db187 | 1112 | You can inspect the catalog to find specific files. |
64b85116 DM |
1113 | |
1114 | .. code-block:: console | |
1115 | ||
3c50a9d8 | 1116 | # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z |
64b85116 DM |
1117 | ... |
1118 | d "./root.pxar.didx/etc/cifs-utils" | |
1119 | l "./root.pxar.didx/etc/cifs-utils/idmap-plugin" | |
1120 | d "./root.pxar.didx/etc/console-setup" | |
1121 | ... | |
1122 | ||
1123 | The restore command lets you restore a single archive from the | |
1124 | backup. | |
1125 | ||
1126 | .. code-block:: console | |
1127 | ||
1128 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/ | |
1129 | ||
4cda7603 | 1130 | To get the contents of any archive, you can restore the ``index.json`` file in the |
aef49768 | 1131 | repository to the target path '-'. This will dump the contents to the standard output. |
64b85116 DM |
1132 | |
1133 | .. code-block:: console | |
1134 | ||
1135 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json - | |
1136 | ||
1137 | ||
1138 | Interactive Restores | |
1139 | ^^^^^^^^^^^^^^^^^^^^ | |
1140 | ||
1141 | If you only want to restore a few individual files, it is often easier | |
1142 | to use the interactive recovery shell. | |
1143 | ||
1144 | .. code-block:: console | |
1145 | ||
3c50a9d8 | 1146 | # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar |
64b85116 DM |
1147 | Starting interactive shell |
1148 | pxar:/ > ls | |
1149 | bin boot dev etc home lib lib32 | |
1150 | ... | |
1151 | ||
3f0983b7 | 1152 | The interactive recovery shell is a minimalistic command line interface that |
4f3db187 AL |
1153 | utilizes the metadata stored in the catalog to quickly list, navigate and |
1154 | search files in a file archive. | |
1155 | To restore files, you can select them individually or match them with a glob | |
1156 | pattern. | |
1157 | ||
1158 | Using the catalog for navigation reduces the overhead considerably because only | |
1159 | the catalog needs to be downloaded and, optionally, decrypted. | |
3f0983b7 CE |
1160 | The actual chunks are only accessed if the metadata in the catalog is not enough |
1161 | or for the actual restore. | |
1162 | ||
1163 | Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change | |
4f3db187 | 1164 | working directory and list directory contents in the archive. |
3f0983b7 CE |
1165 | ``pwd`` shows the full path of the current working directory with respect to the |
1166 | archive root. | |
1167 | ||
aef49768 | 1168 | Being able to quickly search the contents of the archive is a commmonly needed feature. |
3f0983b7 CE |
1169 | That's where the catalog is most valuable. |
1170 | For example: | |
1171 | ||
1172 | .. code-block:: console | |
1173 | ||
a83674ad | 1174 | pxar:/ > find etc/**/*.txt --select |
3f0983b7 CE |
1175 | "/etc/X11/rgb.txt" |
1176 | pxar:/ > list-selected | |
1177 | etc/**/*.txt | |
1178 | pxar:/ > restore-selected /target/path | |
1179 | ... | |
1180 | ||
1181 | This will find and print all files ending in ``.txt`` located in ``etc/`` or a | |
1182 | subdirectory and add the corresponding pattern to the list for subsequent restores. | |
1183 | ``list-selected`` shows these patterns and ``restore-selected`` finally restores | |
1184 | all files in the archive matching the patterns to ``/target/path`` on the local | |
1185 | host. This will scan the whole archive. | |
1186 | ||
1187 | With ``restore /target/path`` you can restore the sub-archive given by the current | |
1188 | working directory to the local target path ``/target/path`` on your host. | |
1189 | By additionally passing a glob pattern with ``--pattern <glob>``, the restore is | |
1190 | further limited to files matching the pattern. | |
1191 | For example: | |
1192 | ||
1193 | .. code-block:: console | |
1194 | ||
1195 | pxar:/ > cd /etc/ | |
1196 | pxar:/etc/ > restore /target/ --pattern **/*.conf | |
1197 | ... | |
1198 | ||
1199 | The above will scan trough all the directories below ``/etc`` and restore all | |
1200 | files ending in ``.conf``. | |
1201 | ||
1202 | .. todo:: Explain interactive restore in more detail | |
64b85116 | 1203 | |
c7971d7f CE |
1204 | Mounting of Archives via FUSE |
1205 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1206 | ||
1207 | The :term:`FUSE` implementation for the pxar archive allows you to mount a | |
1208 | file archive as a read-only filesystem to a mountpoint on your host. | |
1209 | ||
1210 | .. code-block:: console | |
1211 | ||
74fc8447 DW |
1212 | # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint |
1213 | # ls /mnt/mountpoint | |
c7971d7f CE |
1214 | bin dev home lib32 libx32 media opt root sbin sys usr |
1215 | boot etc lib lib64 lost+found mnt proc run srv tmp var | |
1216 | ||
aef49768 | 1217 | This allows you to access the full contents of the archive in a seamless manner. |
c7971d7f CE |
1218 | |
1219 | .. note:: As the FUSE connection needs to fetch and decrypt chunks from the | |
aef49768 | 1220 | backup server's datastore, this can cause some additional network and CPU |
c7971d7f CE |
1221 | load on your host, depending on the operations you perform on the mounted |
1222 | filesystem. | |
1223 | ||
4f3db187 | 1224 | To unmount the filesystem use the ``umount`` command on the mountpoint: |
c7971d7f CE |
1225 | |
1226 | .. code-block:: console | |
1227 | ||
74fc8447 | 1228 | # umount /mnt/mountpoint |
58ea88c8 | 1229 | |
ac456d85 DM |
1230 | Login and Logout |
1231 | ~~~~~~~~~~~~~~~~ | |
1232 | ||
1233 | The client tool prompts you to enter the logon password as soon as you | |
1234 | want to access the backup server. The server checks your credentials | |
1235 | and responds with a ticket that is valid for two hours. The client | |
4f3db187 | 1236 | tool automatically stores that ticket and uses it for further requests |
ac456d85 DM |
1237 | to this server. |
1238 | ||
1239 | You can also manually trigger this login/logout using the login and | |
1240 | logout commands: | |
1241 | ||
1242 | .. code-block:: console | |
1243 | ||
1244 | # proxmox-backup-client login | |
1245 | Password: ********** | |
1246 | ||
4f3db187 | 1247 | To remove the ticket, issue a logout: |
ac456d85 DM |
1248 | |
1249 | .. code-block:: console | |
1250 | ||
1251 | # proxmox-backup-client logout | |
1252 | ||
1253 | ||
22231524 SI |
1254 | .. _pruning: |
1255 | ||
6e5a0c03 DM |
1256 | Pruning and Removing Backups |
1257 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
1258 | ||
1259 | You can manually delete a backup snapshot using the ``forget`` | |
1260 | command: | |
1261 | ||
1262 | .. code-block:: console | |
1263 | ||
1264 | # proxmox-backup-client forget <snapshot> | |
1265 | ||
1266 | ||
4f3db187 AL |
1267 | .. caution:: This command removes all archives in this backup |
1268 | snapshot. They will be inaccessible and unrecoverable. | |
6e5a0c03 DM |
1269 | |
1270 | ||
4cda7603 DW |
1271 | Although manual removal is sometimes required, the ``prune`` |
1272 | command is normally used to systematically delete older backups. Prune lets | |
4f3db187 AL |
1273 | you specify which backup snapshots you want to keep. The |
1274 | following retention options are available: | |
52b2be97 DM |
1275 | |
1276 | ``--keep-last <N>`` | |
1277 | Keep the last ``<N>`` backup snapshots. | |
1278 | ||
102d8d41 | 1279 | ``--keep-hourly <N>`` |
4f3db187 AL |
1280 | Keep backups for the last ``<N>`` hours. If there is more than one |
1281 | backup for a single hour, only the latest is kept. | |
102d8d41 | 1282 | |
52b2be97 | 1283 | ``--keep-daily <N>`` |
4f3db187 AL |
1284 | Keep backups for the last ``<N>`` days. If there is more than one |
1285 | backup for a single day, only the latest is kept. | |
52b2be97 DM |
1286 | |
1287 | ``--keep-weekly <N>`` | |
4f3db187 AL |
1288 | Keep backups for the last ``<N>`` weeks. If there is more than one |
1289 | backup for a single week, only the latest is kept. | |
52b2be97 | 1290 | |
4f3db187 AL |
1291 | .. note:: Weeks start on Monday and end on Sunday. The software |
1292 | uses the `ISO week date`_ system and handles weeks at | |
1293 | the end of the year correctly. | |
1af66370 | 1294 | |
52b2be97 | 1295 | ``--keep-monthly <N>`` |
4f3db187 AL |
1296 | Keep backups for the last ``<N>`` months. If there is more than one |
1297 | backup for a single month, only the latest is kept. | |
52b2be97 DM |
1298 | |
1299 | ``--keep-yearly <N>`` | |
4f3db187 AL |
1300 | Keep backups for the last ``<N>`` years. If there is more than one |
1301 | backup for a single year, only the latest is kept. | |
1302 | ||
1303 | The retention options are processed in the order given above. Each option | |
1304 | only covers backups within its time period. The next option does not take care | |
1305 | of already covered backups. It will only consider older backups. | |
52b2be97 | 1306 | |
4f3db187 AL |
1307 | Unfinished and incomplete backups will be removed by the prune command unless |
1308 | they are newer than the last successful backup. In this case, the last failed | |
1309 | backup is retained. | |
02d22dec | 1310 | |
6e5a0c03 DM |
1311 | .. code-block:: console |
1312 | ||
1313 | # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3 | |
1314 | ||
1315 | ||
4f3db187 | 1316 | You can use the ``--dry-run`` option to test your settings. This only |
aef49768 | 1317 | shows the list of existing snapshots and what actions prune would take. |
84322d8c DM |
1318 | |
1319 | .. code-block:: console | |
1320 | ||
1321 | # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3 | |
a66d5898 DM |
1322 | ┌────────────────────────────────┬──────┐ |
1323 | │ snapshot │ keep │ | |
1324 | ╞════════════════════════════════╪══════╡ | |
1325 | │ host/elsa/2019-12-04T13:20:37Z │ 1 │ | |
1326 | ├────────────────────────────────┼──────┤ | |
1327 | │ host/elsa/2019-12-03T09:35:01Z │ 0 │ | |
1328 | ├────────────────────────────────┼──────┤ | |
1329 | │ host/elsa/2019-11-22T11:54:47Z │ 1 │ | |
1330 | ├────────────────────────────────┼──────┤ | |
1331 | │ host/elsa/2019-11-21T12:36:25Z │ 0 │ | |
1332 | ├────────────────────────────────┼──────┤ | |
1333 | │ host/elsa/2019-11-10T10:42:20Z │ 1 │ | |
1334 | └────────────────────────────────┴──────┘ | |
84322d8c | 1335 | |
52b2be97 | 1336 | .. note:: Neither the ``prune`` command nor the ``forget`` command free space |
4f3db187 AL |
1337 | in the chunk-store. The chunk-store still contains the data blocks. To free |
1338 | space you need to perform :ref:`garbage-collection`. | |
6e5a0c03 DM |
1339 | |
1340 | ||
1341 | .. _garbage-collection: | |
1342 | ||
1343 | Garbage Collection | |
1344 | ~~~~~~~~~~~~~~~~~~ | |
1345 | ||
e1c356ec | 1346 | The ``prune`` command removes only the backup index files, not the data |
092378ba | 1347 | from the datastore. This task is left to the garbage collection |
4f3db187 | 1348 | command. It is recommended to carry out garbage collection on a regular basis. |
e1c356ec DM |
1349 | |
1350 | The garbage collection works in two phases. In the first phase, all | |
1351 | data blocks that are still in use are marked. In the second phase, | |
1352 | unused data blocks are removed. | |
1353 | ||
1354 | .. note:: This command needs to read all existing backup index files | |
f0188322 CE |
1355 | and touches the complete chunk-store. This can take a long time |
1356 | depending on the number of chunks and the speed of the underlying | |
e1c356ec DM |
1357 | disks. |
1358 | ||
8314ca9c AL |
1359 | .. note:: The garbage collection will only remove chunks that haven't been used |
1360 | for at least one day (exactly 24h 5m). This grace period is necessary because | |
1361 | chunks in use are marked by touching the chunk which updates the ``atime`` | |
1362 | (access time) property. Filesystems are mounted with the ``relatime`` option | |
1363 | by default. This results in a better performance by only updating the | |
1364 | ``atime`` property if the last access has been at least 24 hours ago. The | |
255ed621 TL |
1365 | downside is, that touching a chunk within these 24 hours will not always |
1366 | update its ``atime`` property. | |
8314ca9c | 1367 | |
255ed621 TL |
1368 | Chunks in the grace period will be logged at the end of the garbage |
1369 | collection task as *Pending removals*. | |
e1c356ec DM |
1370 | |
1371 | .. code-block:: console | |
1372 | ||
1373 | # proxmox-backup-client garbage-collect | |
1374 | starting garbage collection on store store2 | |
1375 | Start GC phase1 (mark used chunks) | |
1376 | Start GC phase2 (sweep unused chunks) | |
1377 | percentage done: 1, chunk count: 219 | |
1378 | percentage done: 2, chunk count: 453 | |
1379 | ... | |
1380 | percentage done: 99, chunk count: 21188 | |
1381 | Removed bytes: 411368505 | |
1382 | Removed chunks: 203 | |
1383 | Original data bytes: 327160886391 | |
1384 | Disk bytes: 52767414743 (16 %) | |
1385 | Disk chunks: 21221 | |
1386 | Average chunk size: 2486565 | |
1387 | TASK OK | |
1388 | ||
1389 | ||
1390 | .. todo:: howto run garbage-collection at regular intervalls (cron) | |
6e5a0c03 | 1391 | |
a8d69fcf DW |
1392 | Benchmarking |
1393 | ~~~~~~~~~~~~ | |
1394 | The backup client also comes with a benchmarking tool. This tool measures | |
1395 | various metrics relating to compression and encryption speeds. You can run a | |
1396 | benchmark using the ``benchmark`` subcommand of ``proxmox-backup-client``: | |
1397 | ||
1398 | .. code-block:: console | |
1399 | ||
1400 | # proxmox-backup-client benchmark | |
1401 | Uploaded 656 chunks in 5 seconds. | |
1402 | Time per request: 7659 microseconds. | |
1403 | TLS speed: 547.60 MB/s | |
1404 | SHA256 speed: 585.76 MB/s | |
1405 | Compression speed: 1923.96 MB/s | |
1406 | Decompress speed: 7885.24 MB/s | |
1407 | AES256/GCM speed: 3974.03 MB/s | |
1408 | ┌───────────────────────────────────┬─────────────────────┐ | |
1409 | │ Name │ Value │ | |
1410 | ╞═══════════════════════════════════╪═════════════════════╡ | |
1411 | │ TLS (maximal backup upload speed) │ 547.60 MB/s (93%) │ | |
1412 | ├───────────────────────────────────┼─────────────────────┤ | |
1413 | │ SHA256 checksum computation speed │ 585.76 MB/s (28%) │ | |
1414 | ├───────────────────────────────────┼─────────────────────┤ | |
1415 | │ ZStd level 1 compression speed │ 1923.96 MB/s (89%) │ | |
1416 | ├───────────────────────────────────┼─────────────────────┤ | |
1417 | │ ZStd level 1 decompression speed │ 7885.24 MB/s (98%) │ | |
1418 | ├───────────────────────────────────┼─────────────────────┤ | |
1419 | │ AES256 GCM encryption speed │ 3974.03 MB/s (104%) │ | |
1420 | └───────────────────────────────────┴─────────────────────┘ | |
1421 | ||
9624c5ee DM |
1422 | .. note:: The percentages given in the output table correspond to a |
1423 | comparison against a Ryzen 7 2700X. The TLS test connects to the | |
1424 | local host, so there is no network involved. | |
503dd339 | 1425 | |
a8d69fcf DW |
1426 | You can also pass the ``--output-format`` parameter to output stats in ``json``, |
1427 | rather than the default table format. | |
6e5a0c03 | 1428 | |
a129fdd9 DM |
1429 | .. _pve-integration: |
1430 | ||
58ea88c8 DM |
1431 | `Proxmox VE`_ integration |
1432 | ------------------------- | |
cb01363c | 1433 | |
f9dcfa41 DM |
1434 | You need to define a new storage with type 'pbs' on your `Proxmox VE`_ |
1435 | node. The following example uses ``store2`` as storage name, and | |
1436 | assumes the server address is ``localhost``, and you want to connect | |
1437 | as ``user1@pbs``. | |
1438 | ||
1439 | .. code-block:: console | |
1440 | ||
1441 | # pvesm add pbs store2 --server localhost --datastore store2 | |
1442 | # pvesm set store2 --username user1@pbs --password <secret> | |
1443 | ||
2dd58db7 DW |
1444 | .. note:: If you would rather not pass your password as plain text, you can pass |
1445 | the ``--password`` parameter, without any arguments. This will cause the | |
1446 | program to prompt you for a password upon entering the command. | |
1447 | ||
f9dcfa41 DM |
1448 | If your backup server uses a self signed certificate, you need to add |
1449 | the certificate fingerprint to the configuration. You can get the | |
1450 | fingerprint by running the following command on the backup server: | |
1451 | ||
1452 | .. code-block:: console | |
1453 | ||
fe233f3b | 1454 | # proxmox-backup-manager cert info | grep Fingerprint |
f9dcfa41 DM |
1455 | Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe |
1456 | ||
1457 | Please add that fingerprint to your configuration to establish a trust | |
1458 | relationship: | |
1459 | ||
1460 | .. code-block:: console | |
1461 | ||
1462 | # pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe | |
1463 | ||
1464 | After that you should be able to see storage status with: | |
1465 | ||
1466 | .. code-block:: console | |
1467 | ||
1468 | # pvesm status --storage store2 | |
1469 | Name Type Status Total Used Available % | |
1470 | store2 pbs active 3905109820 1336687816 2568422004 34.23% | |
1471 | ||
9d3f183b DW |
1472 | Having added the PBS datastore to `Proxmox VE`_, you can backup VMs and |
1473 | containers in the same way you would for any other storage device within the | |
1474 | environment (see `PVE Admin Guide: Backup and Restore | |
1475 | <https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_vzdump>`_. | |
f9dcfa41 | 1476 | |
cb01363c DM |
1477 | |
1478 | .. include:: command-line-tools.rst | |
1479 | ||
1480 | .. include:: services.rst |