]>
Commit | Line | Data |
---|---|---|
04e24b14 DW |
1 | Backup Client Usage |
2 | =================== | |
3 | ||
4 | The command line client is called :command:`proxmox-backup-client`. | |
5 | ||
6 | ||
7 | Repository Locations | |
8 | -------------------- | |
9 | ||
10 | The client uses the following notation to specify a datastore repository | |
11 | on the backup server. | |
12 | ||
13 | [[username@]server[:port]:]datastore | |
14 | ||
4aef06f1 | 15 | The default value for ``username`` is ``root@pam``. If no server is specified, |
04e24b14 DW |
16 | the default is the local host (``localhost``). |
17 | ||
18 | You can specify a port if your backup server is only reachable on a different | |
19 | port (e.g. with NAT and port forwarding). | |
20 | ||
4aef06f1 TL |
21 | Note that if the server is an IPv6 address, you have to write it with square |
22 | brackets (for example, `[fe80::01]`). | |
04e24b14 | 23 | |
4aef06f1 TL |
24 | You can pass the repository with the ``--repository`` command line option, or |
25 | by setting the ``PBS_REPOSITORY`` environment variable. | |
04e24b14 DW |
26 | |
27 | Here some examples of valid repositories and the real values | |
28 | ||
c0af05e1 | 29 | ================================ ================== ================== =========== |
4aef06f1 | 30 | Example User Host:Port Datastore |
c0af05e1 | 31 | ================================ ================== ================== =========== |
4aef06f1 TL |
32 | mydatastore ``root@pam`` localhost:8007 mydatastore |
33 | myhostname:mydatastore ``root@pam`` myhostname:8007 mydatastore | |
34 | user@pbs@myhostname:mydatastore ``user@pbs`` myhostname:8007 mydatastore | |
90e94aa2 | 35 | user\@pbs!token@host:store ``user@pbs!token`` myhostname:8007 mydatastore |
4aef06f1 TL |
36 | 192.168.55.55:1234:mydatastore ``root@pam`` 192.168.55.55:1234 mydatastore |
37 | [ff80::51]:mydatastore ``root@pam`` [ff80::51]:8007 mydatastore | |
38 | [ff80::51]:1234:mydatastore ``root@pam`` [ff80::51]:1234 mydatastore | |
c0af05e1 | 39 | ================================ ================== ================== =========== |
04e24b14 DW |
40 | |
41 | Environment Variables | |
42 | --------------------- | |
43 | ||
44 | ``PBS_REPOSITORY`` | |
45 | The default backup repository. | |
46 | ||
47 | ``PBS_PASSWORD`` | |
4aef06f1 TL |
48 | When set, this value is used for the password required for the backup server. |
49 | You can also set this to a API token secret. | |
04e24b14 DW |
50 | |
51 | ``PBS_ENCRYPTION_PASSWORD`` | |
52 | When set, this value is used to access the secret encryption key (if | |
53 | protected by password). | |
54 | ||
55 | ``PBS_FINGERPRINT`` When set, this value is used to verify the server | |
4aef06f1 TL |
56 | certificate (only used if the system CA certificates cannot validate the |
57 | certificate). | |
04e24b14 DW |
58 | |
59 | ||
60 | Output Format | |
61 | ------------- | |
62 | ||
f458e97f | 63 | .. include:: output-format.rst |
04e24b14 DW |
64 | |
65 | ||
ec1ae7e6 | 66 | .. _client_creating_backups: |
04e24b14 DW |
67 | |
68 | Creating Backups | |
69 | ---------------- | |
70 | ||
71 | This section explains how to create a backup from within the machine. This can | |
72 | be a physical host, a virtual machine, or a container. Such backups may contain file | |
73 | and image archives. There are no restrictions in this case. | |
74 | ||
75 | .. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`. | |
76 | ||
77 | For the following example you need to have a backup server set up, working | |
78 | credentials and need to know the repository name. | |
79 | In the following examples we use ``backup-server:store1``. | |
80 | ||
81 | .. code-block:: console | |
82 | ||
83 | # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1 | |
84 | Starting backup: host/elsa/2019-12-03T09:35:01Z | |
85 | Client name: elsa | |
86 | skip mount point: "/boot/efi" | |
87 | skip mount point: "/dev" | |
88 | skip mount point: "/run" | |
89 | skip mount point: "/sys" | |
90 | Uploaded 12129 chunks in 87 seconds (564 MB/s). | |
91 | End Time: 2019-12-03T10:36:29+01:00 | |
92 | ||
93 | This will prompt you for a password and then uploads a file archive named | |
94 | ``root.pxar`` containing all the files in the ``/`` directory. | |
95 | ||
96 | .. Caution:: Please note that the proxmox-backup-client does not | |
97 | automatically include mount points. Instead, you will see a short | |
98 | ``skip mount point`` notice for each of them. The idea is to | |
99 | create a separate file archive for each mounted disk. You can | |
100 | explicitly include them using the ``--include-dev`` option | |
101 | (i.e. ``--include-dev /boot/efi``). You can use this option | |
102 | multiple times for each mount point that should be included. | |
103 | ||
104 | The ``--repository`` option can get quite long and is used by all | |
105 | commands. You can avoid having to enter this value by setting the | |
106 | environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set | |
107 | over multiple sessions, you should instead add the below line to your | |
108 | ``.bashrc`` file. | |
109 | ||
110 | .. code-block:: console | |
111 | ||
112 | # export PBS_REPOSITORY=backup-server:store1 | |
113 | ||
114 | After this you can execute all commands without specifying the ``--repository`` | |
115 | option. | |
116 | ||
117 | One single backup is allowed to contain more than one archive. For example, if | |
3bbb70b3 | 118 | you want to backup two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``: |
04e24b14 DW |
119 | |
120 | .. code-block:: console | |
121 | ||
122 | # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 | |
123 | ||
124 | This creates a backup of both disks. | |
125 | ||
126 | The backup command takes a list of backup specifications, which | |
127 | include the archive name on the server, the type of the archive, and the | |
128 | archive source at the client. The format is: | |
129 | ||
130 | <archive-name>.<type>:<source-path> | |
131 | ||
132 | Common types are ``.pxar`` for file archives, and ``.img`` for block | |
133 | device images. To create a backup of a block device run the following command: | |
134 | ||
135 | .. code-block:: console | |
136 | ||
137 | # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata | |
138 | ||
139 | ||
140 | Excluding files/folders from a backup | |
141 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
142 | ||
143 | Sometimes it is desired to exclude certain files or folders from a backup archive. | |
144 | To tell the Proxmox Backup client when and how to ignore files and directories, | |
145 | place a text file called ``.pxarexclude`` in the filesystem hierarchy. | |
146 | Whenever the backup client encounters such a file in a directory, it interprets | |
147 | each line as glob match patterns for files and directories that are to be excluded | |
148 | from the backup. | |
149 | ||
150 | The file must contain a single glob pattern per line. Empty lines are ignored. | |
151 | The same is true for lines starting with ``#``, which indicates a comment. | |
152 | A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion | |
153 | to an explicit inclusion. This makes it possible to exclude all entries in a | |
154 | directory except for a few single files/subdirectories. | |
155 | Lines ending in ``/`` match only on directories. | |
156 | The directory containing the ``.pxarexclude`` file is considered to be the root of | |
157 | the given patterns. It is only possible to match files in this directory and its subdirectories. | |
158 | ||
159 | ``\`` is used to escape special glob characters. | |
160 | ``?`` matches any single character. | |
161 | ``*`` matches any character, including an empty string. | |
162 | ``**`` is used to match subdirectories. It can be used to, for example, exclude | |
163 | all files ending in ``.tmp`` within the directory or subdirectories with the | |
164 | following pattern ``**/*.tmp``. | |
165 | ``[...]`` matches a single character from any of the provided characters within | |
166 | the brackets. ``[!...]`` does the complementary and matches any single character | |
167 | not contained within the brackets. It is also possible to specify ranges with two | |
168 | characters separated by ``-``. For example, ``[a-z]`` matches any lowercase | |
169 | alphabetic character and ``[0-9]`` matches any one single digit. | |
170 | ||
171 | The order of the glob match patterns defines whether a file is included or | |
172 | excluded, that is to say later entries override previous ones. | |
173 | This is also true for match patterns encountered deeper down the directory tree, | |
174 | which can override a previous exclusion. | |
175 | Be aware that excluded directories will **not** be read by the backup client. | |
176 | Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect. | |
177 | ``.pxarexclude`` files are treated as regular files and will be included in the | |
178 | backup archive. | |
179 | ||
180 | For example, consider the following directory structure: | |
181 | ||
182 | .. code-block:: console | |
183 | ||
184 | # ls -aR folder | |
185 | folder/: | |
186 | . .. .pxarexclude subfolder0 subfolder1 | |
187 | ||
188 | folder/subfolder0: | |
189 | . .. file0 file1 file2 file3 .pxarexclude | |
190 | ||
191 | folder/subfolder1: | |
192 | . .. file0 file1 file2 file3 | |
193 | ||
194 | The different ``.pxarexclude`` files contain the following: | |
195 | ||
196 | .. code-block:: console | |
197 | ||
198 | # cat folder/.pxarexclude | |
199 | /subfolder0/file1 | |
200 | /subfolder1/* | |
201 | !/subfolder1/file2 | |
202 | ||
203 | .. code-block:: console | |
204 | ||
205 | # cat folder/subfolder0/.pxarexclude | |
206 | file3 | |
207 | ||
208 | This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of | |
209 | ``subfolder1`` except ``file2``. | |
210 | ||
211 | Restoring this backup will result in: | |
212 | ||
213 | .. code-block:: console | |
214 | ||
215 | ls -aR restored | |
216 | restored/: | |
217 | . .. .pxarexclude subfolder0 subfolder1 | |
218 | ||
219 | restored/subfolder0: | |
220 | . .. file0 file2 .pxarexclude | |
221 | ||
222 | restored/subfolder1: | |
223 | . .. file2 | |
224 | ||
225 | ||
ec1ae7e6 | 226 | .. _client_encryption: |
d9f365d7 | 227 | |
04e24b14 DW |
228 | Encryption |
229 | ---------- | |
230 | ||
231 | Proxmox Backup supports client-side encryption with AES-256 in GCM_ | |
232 | mode. To set this up, you first need to create an encryption key: | |
233 | ||
234 | .. code-block:: console | |
235 | ||
236 | # proxmox-backup-client key create my-backup.key | |
237 | Encryption Key Password: ************** | |
238 | ||
239 | The key is password protected by default. If you do not need this | |
240 | extra protection, you can also create it without a password: | |
241 | ||
242 | .. code-block:: console | |
243 | ||
244 | # proxmox-backup-client key create /path/to/my-backup.key --kdf none | |
245 | ||
246 | Having created this key, it is now possible to create an encrypted backup, by | |
247 | passing the ``--keyfile`` parameter, with the path to the key file. | |
248 | ||
249 | .. code-block:: console | |
250 | ||
251 | # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key | |
252 | Password: ********* | |
253 | Encryption Key Password: ************** | |
254 | ... | |
255 | ||
256 | .. Note:: If you do not specify the name of the backup key, the key will be | |
257 | created in the default location | |
258 | ``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client`` | |
259 | will also search this location by default, in case the ``--keyfile`` | |
260 | parameter is not specified. | |
261 | ||
262 | You can avoid entering the passwords by setting the environment | |
263 | variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``. | |
264 | ||
265 | ||
266 | Using a master key to store and recover encryption keys | |
267 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
268 | ||
269 | You can also use ``proxmox-backup-client key`` to create an RSA public/private | |
270 | key pair, which can be used to store an encrypted version of the symmetric | |
271 | backup encryption key alongside each backup and recover it later. | |
272 | ||
273 | To set up a master key: | |
274 | ||
275 | 1. Create an encryption key for the backup: | |
276 | ||
277 | .. code-block:: console | |
278 | ||
279 | # proxmox-backup-client key create | |
280 | creating default key at: "~/.config/proxmox-backup/encryption-key.json" | |
281 | Encryption Key Password: ********** | |
282 | ... | |
283 | ||
284 | The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``. | |
285 | ||
286 | 2. Create an RSA public/private key pair: | |
287 | ||
288 | .. code-block:: console | |
289 | ||
290 | # proxmox-backup-client key create-master-key | |
291 | Master Key Password: ********* | |
292 | ... | |
293 | ||
294 | This will create two files in your current directory, ``master-public.pem`` | |
295 | and ``master-private.pem``. | |
296 | ||
297 | 3. Import the newly created ``master-public.pem`` public certificate, so that | |
298 | ``proxmox-backup-client`` can find and use it upon backup. | |
299 | ||
300 | .. code-block:: console | |
301 | ||
302 | # proxmox-backup-client key import-master-pubkey /path/to/master-public.pem | |
303 | Imported public master key to "~/.config/proxmox-backup/master-public.pem" | |
304 | ||
305 | 4. With all these files in place, run a backup job: | |
306 | ||
307 | .. code-block:: console | |
308 | ||
309 | # proxmox-backup-client backup etc.pxar:/etc | |
310 | ||
311 | The key will be stored in your backup, under the name ``rsa-encrypted.key``. | |
312 | ||
313 | .. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key | |
314 | is in the default path. If you specified another path upon creation, you | |
315 | must pass the ``--keyfile`` parameter. | |
316 | ||
317 | 5. To test that everything worked, you can restore the key from the backup: | |
318 | ||
319 | .. code-block:: console | |
320 | ||
321 | # proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target | |
322 | ||
323 | .. Note:: You should not need an encryption key to extract this file. However, if | |
324 | a key exists at the default location | |
325 | (``~/.config/proxmox-backup/encryption-key.json``) the program will prompt | |
326 | you for an encryption key password. Simply moving ``encryption-key.json`` | |
327 | out of this directory will fix this issue. | |
328 | ||
329 | 6. Then, use the previously generated master key to decrypt the file: | |
330 | ||
331 | .. code-block:: console | |
332 | ||
2ea5abcd FG |
333 | # proxmox-backup-client key import-with-master-key /path/to/target --master-keyfile /path/to/master-private.pem --encrypted-keyfile /path/to/rsa-encrypted.key |
334 | Master Key Password: ****** | |
335 | New Password: ****** | |
336 | Verify Password: ****** | |
04e24b14 DW |
337 | |
338 | 7. The target file will now contain the encryption key information in plain | |
339 | text. The success of this can be confirmed by passing the resulting ``json`` | |
340 | file, with the ``--keyfile`` parameter, when decrypting files from the backup. | |
341 | ||
342 | .. warning:: Without their key, backed up files will be inaccessible. Thus, you should | |
343 | keep keys ordered and in a place that is separate from the contents being | |
344 | backed up. It can happen, for example, that you back up an entire system, using | |
cdbc18fc | 345 | a key on that system. If the system then becomes inaccessible for any reason |
04e24b14 | 346 | and needs to be restored, this will not be possible as the encryption key will be |
4accbc58 DW |
347 | lost along with the broken system. |
348 | ||
c5608cf8 DW |
349 | It is recommended that you keep your master key safe, but easily accessible, in |
350 | order for quick disaster recovery. For this reason, the best place to store it | |
351 | is in your password manager, where it is immediately recoverable. As a backup to | |
352 | this, you should also save the key to a USB drive and store that in a secure | |
353 | place. This way, it is detached from any system, but is still easy to recover | |
354 | from, in case of emergency. Finally, in preparation for the worst case scenario, | |
355 | you should also consider keeping a paper copy of your master key locked away in | |
356 | a safe place. The ``paperkey`` subcommand can be used to create a QR encoded | |
357 | version of your master key. The following command sends the output of the | |
358 | ``paperkey`` command to a text file, for easy printing. | |
4accbc58 DW |
359 | |
360 | .. code-block:: console | |
361 | ||
362 | proxmox-backup-client key paperkey --output-format text > qrkey.txt | |
04e24b14 DW |
363 | |
364 | ||
365 | Restoring Data | |
366 | -------------- | |
367 | ||
368 | The regular creation of backups is a necessary step to avoiding data | |
369 | loss. More importantly, however, is the restoration. It is good practice to perform | |
370 | periodic recovery tests to ensure that you can access the data in | |
371 | case of problems. | |
372 | ||
373 | First, you need to find the snapshot which you want to restore. The snapshot | |
abf9b6da | 374 | list command provides a list of all the snapshots on the server: |
04e24b14 DW |
375 | |
376 | .. code-block:: console | |
377 | ||
abf9b6da | 378 | # proxmox-backup-client snapshot list |
04e24b14 DW |
379 | ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐ |
380 | │ snapshot │ size │ files │ | |
381 | ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡ | |
382 | │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │ | |
383 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
384 | │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │ | |
385 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
386 | ... | |
387 | ||
388 | You can inspect the catalog to find specific files. | |
389 | ||
390 | .. code-block:: console | |
391 | ||
392 | # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z | |
393 | ... | |
394 | d "./root.pxar.didx/etc/cifs-utils" | |
395 | l "./root.pxar.didx/etc/cifs-utils/idmap-plugin" | |
396 | d "./root.pxar.didx/etc/console-setup" | |
397 | ... | |
398 | ||
399 | The restore command lets you restore a single archive from the | |
400 | backup. | |
401 | ||
402 | .. code-block:: console | |
403 | ||
404 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/ | |
405 | ||
406 | To get the contents of any archive, you can restore the ``index.json`` file in the | |
407 | repository to the target path '-'. This will dump the contents to the standard output. | |
408 | ||
409 | .. code-block:: console | |
410 | ||
411 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json - | |
412 | ||
413 | ||
414 | Interactive Restores | |
415 | ~~~~~~~~~~~~~~~~~~~~ | |
416 | ||
417 | If you only want to restore a few individual files, it is often easier | |
418 | to use the interactive recovery shell. | |
419 | ||
420 | .. code-block:: console | |
421 | ||
422 | # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar | |
423 | Starting interactive shell | |
424 | pxar:/ > ls | |
425 | bin boot dev etc home lib lib32 | |
426 | ... | |
427 | ||
3bbb70b3 | 428 | The interactive recovery shell is a minimal command line interface that |
04e24b14 DW |
429 | utilizes the metadata stored in the catalog to quickly list, navigate and |
430 | search files in a file archive. | |
431 | To restore files, you can select them individually or match them with a glob | |
432 | pattern. | |
433 | ||
434 | Using the catalog for navigation reduces the overhead considerably because only | |
435 | the catalog needs to be downloaded and, optionally, decrypted. | |
436 | The actual chunks are only accessed if the metadata in the catalog is not enough | |
437 | or for the actual restore. | |
438 | ||
439 | Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change | |
440 | working directory and list directory contents in the archive. | |
441 | ``pwd`` shows the full path of the current working directory with respect to the | |
442 | archive root. | |
443 | ||
3bbb70b3 | 444 | Being able to quickly search the contents of the archive is a commonly needed feature. |
04e24b14 DW |
445 | That's where the catalog is most valuable. |
446 | For example: | |
447 | ||
448 | .. code-block:: console | |
449 | ||
450 | pxar:/ > find etc/**/*.txt --select | |
451 | "/etc/X11/rgb.txt" | |
452 | pxar:/ > list-selected | |
453 | etc/**/*.txt | |
454 | pxar:/ > restore-selected /target/path | |
455 | ... | |
456 | ||
457 | This will find and print all files ending in ``.txt`` located in ``etc/`` or a | |
458 | subdirectory and add the corresponding pattern to the list for subsequent restores. | |
459 | ``list-selected`` shows these patterns and ``restore-selected`` finally restores | |
460 | all files in the archive matching the patterns to ``/target/path`` on the local | |
461 | host. This will scan the whole archive. | |
462 | ||
463 | With ``restore /target/path`` you can restore the sub-archive given by the current | |
464 | working directory to the local target path ``/target/path`` on your host. | |
465 | By additionally passing a glob pattern with ``--pattern <glob>``, the restore is | |
466 | further limited to files matching the pattern. | |
467 | For example: | |
468 | ||
469 | .. code-block:: console | |
470 | ||
471 | pxar:/ > cd /etc/ | |
472 | pxar:/etc/ > restore /target/ --pattern **/*.conf | |
473 | ... | |
474 | ||
475 | The above will scan trough all the directories below ``/etc`` and restore all | |
476 | files ending in ``.conf``. | |
477 | ||
478 | .. todo:: Explain interactive restore in more detail | |
479 | ||
480 | Mounting of Archives via FUSE | |
481 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
482 | ||
483 | The :term:`FUSE` implementation for the pxar archive allows you to mount a | |
484 | file archive as a read-only filesystem to a mountpoint on your host. | |
485 | ||
486 | .. code-block:: console | |
487 | ||
488 | # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint | |
489 | # ls /mnt/mountpoint | |
490 | bin dev home lib32 libx32 media opt root sbin sys usr | |
491 | boot etc lib lib64 lost+found mnt proc run srv tmp var | |
492 | ||
493 | This allows you to access the full contents of the archive in a seamless manner. | |
494 | ||
495 | .. note:: As the FUSE connection needs to fetch and decrypt chunks from the | |
496 | backup server's datastore, this can cause some additional network and CPU | |
497 | load on your host, depending on the operations you perform on the mounted | |
498 | filesystem. | |
499 | ||
500 | To unmount the filesystem use the ``umount`` command on the mountpoint: | |
501 | ||
502 | .. code-block:: console | |
503 | ||
504 | # umount /mnt/mountpoint | |
505 | ||
506 | Login and Logout | |
507 | ---------------- | |
508 | ||
509 | The client tool prompts you to enter the logon password as soon as you | |
510 | want to access the backup server. The server checks your credentials | |
511 | and responds with a ticket that is valid for two hours. The client | |
512 | tool automatically stores that ticket and uses it for further requests | |
513 | to this server. | |
514 | ||
515 | You can also manually trigger this login/logout using the login and | |
516 | logout commands: | |
517 | ||
518 | .. code-block:: console | |
519 | ||
520 | # proxmox-backup-client login | |
521 | Password: ********** | |
522 | ||
523 | To remove the ticket, issue a logout: | |
524 | ||
525 | .. code-block:: console | |
526 | ||
527 | # proxmox-backup-client logout | |
528 | ||
529 | ||
08379a21 DW |
530 | .. _changing-backup-owner: |
531 | ||
532 | Changing the Owner of a Backup Group | |
533 | ------------------------------------ | |
534 | ||
535 | By default, the owner of a backup group is the user which was used to originally | |
536 | create that backup group (or in the case of sync jobs, ``root@pam``). This | |
537 | means that if a user ``mike@pbs`` created a backup, another user ``john@pbs`` | |
538 | can not be used to create backups in that same backup group. In case you want | |
539 | to change the owner of a backup, you can do so with the below command, using a | |
540 | user that has ``Datastore.Modify`` privileges on the datastore. | |
541 | ||
542 | .. code-block:: console | |
543 | ||
544 | # proxmox-backup-client change-owner vm/103 john@pbs | |
545 | ||
546 | This can also be done from within the web interface, by navigating to the | |
547 | `Content` section of the datastore that contains the backup group and | |
548 | selecting the user icon under the `Actions` column. Common cases for this could | |
549 | be to change the owner of a sync job from ``root@pam``, or to repurpose a | |
550 | backup group. | |
551 | ||
552 | ||
4c3efb53 | 553 | .. _backup-pruning: |
04e24b14 DW |
554 | |
555 | Pruning and Removing Backups | |
556 | ---------------------------- | |
557 | ||
558 | You can manually delete a backup snapshot using the ``forget`` | |
559 | command: | |
560 | ||
561 | .. code-block:: console | |
562 | ||
abf9b6da | 563 | # proxmox-backup-client snapshot forget <snapshot> |
04e24b14 DW |
564 | |
565 | ||
566 | .. caution:: This command removes all archives in this backup | |
567 | snapshot. They will be inaccessible and unrecoverable. | |
568 | ||
569 | ||
570 | Although manual removal is sometimes required, the ``prune`` | |
571 | command is normally used to systematically delete older backups. Prune lets | |
572 | you specify which backup snapshots you want to keep. The | |
573 | following retention options are available: | |
574 | ||
575 | ``--keep-last <N>`` | |
576 | Keep the last ``<N>`` backup snapshots. | |
577 | ||
578 | ``--keep-hourly <N>`` | |
579 | Keep backups for the last ``<N>`` hours. If there is more than one | |
580 | backup for a single hour, only the latest is kept. | |
581 | ||
582 | ``--keep-daily <N>`` | |
583 | Keep backups for the last ``<N>`` days. If there is more than one | |
584 | backup for a single day, only the latest is kept. | |
585 | ||
586 | ``--keep-weekly <N>`` | |
587 | Keep backups for the last ``<N>`` weeks. If there is more than one | |
588 | backup for a single week, only the latest is kept. | |
589 | ||
590 | .. note:: Weeks start on Monday and end on Sunday. The software | |
591 | uses the `ISO week date`_ system and handles weeks at | |
592 | the end of the year correctly. | |
593 | ||
594 | ``--keep-monthly <N>`` | |
595 | Keep backups for the last ``<N>`` months. If there is more than one | |
596 | backup for a single month, only the latest is kept. | |
597 | ||
598 | ``--keep-yearly <N>`` | |
599 | Keep backups for the last ``<N>`` years. If there is more than one | |
600 | backup for a single year, only the latest is kept. | |
601 | ||
602 | The retention options are processed in the order given above. Each option | |
603 | only covers backups within its time period. The next option does not take care | |
604 | of already covered backups. It will only consider older backups. | |
605 | ||
606 | Unfinished and incomplete backups will be removed by the prune command unless | |
607 | they are newer than the last successful backup. In this case, the last failed | |
608 | backup is retained. | |
609 | ||
610 | .. code-block:: console | |
611 | ||
612 | # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3 | |
613 | ||
614 | ||
615 | You can use the ``--dry-run`` option to test your settings. This only | |
616 | shows the list of existing snapshots and what actions prune would take. | |
617 | ||
618 | .. code-block:: console | |
619 | ||
620 | # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3 | |
621 | ┌────────────────────────────────┬──────┐ | |
622 | │ snapshot │ keep │ | |
623 | ╞════════════════════════════════╪══════╡ | |
624 | │ host/elsa/2019-12-04T13:20:37Z │ 1 │ | |
625 | ├────────────────────────────────┼──────┤ | |
626 | │ host/elsa/2019-12-03T09:35:01Z │ 0 │ | |
627 | ├────────────────────────────────┼──────┤ | |
628 | │ host/elsa/2019-11-22T11:54:47Z │ 1 │ | |
629 | ├────────────────────────────────┼──────┤ | |
630 | │ host/elsa/2019-11-21T12:36:25Z │ 0 │ | |
631 | ├────────────────────────────────┼──────┤ | |
632 | │ host/elsa/2019-11-10T10:42:20Z │ 1 │ | |
633 | └────────────────────────────────┴──────┘ | |
634 | ||
635 | .. note:: Neither the ``prune`` command nor the ``forget`` command free space | |
636 | in the chunk-store. The chunk-store still contains the data blocks. To free | |
a98e2287 | 637 | space you need to perform :ref:`client_garbage-collection`. |
04e24b14 DW |
638 | |
639 | ||
ec1ae7e6 | 640 | .. _client_garbage-collection: |
04e24b14 DW |
641 | |
642 | Garbage Collection | |
643 | ------------------ | |
644 | ||
645 | The ``prune`` command removes only the backup index files, not the data | |
646 | from the datastore. This task is left to the garbage collection | |
647 | command. It is recommended to carry out garbage collection on a regular basis. | |
648 | ||
649 | The garbage collection works in two phases. In the first phase, all | |
650 | data blocks that are still in use are marked. In the second phase, | |
651 | unused data blocks are removed. | |
652 | ||
653 | .. note:: This command needs to read all existing backup index files | |
654 | and touches the complete chunk-store. This can take a long time | |
655 | depending on the number of chunks and the speed of the underlying | |
656 | disks. | |
657 | ||
658 | .. note:: The garbage collection will only remove chunks that haven't been used | |
659 | for at least one day (exactly 24h 5m). This grace period is necessary because | |
660 | chunks in use are marked by touching the chunk which updates the ``atime`` | |
661 | (access time) property. Filesystems are mounted with the ``relatime`` option | |
662 | by default. This results in a better performance by only updating the | |
663 | ``atime`` property if the last access has been at least 24 hours ago. The | |
664 | downside is, that touching a chunk within these 24 hours will not always | |
665 | update its ``atime`` property. | |
666 | ||
667 | Chunks in the grace period will be logged at the end of the garbage | |
668 | collection task as *Pending removals*. | |
669 | ||
670 | .. code-block:: console | |
671 | ||
672 | # proxmox-backup-client garbage-collect | |
673 | starting garbage collection on store store2 | |
674 | Start GC phase1 (mark used chunks) | |
675 | Start GC phase2 (sweep unused chunks) | |
676 | percentage done: 1, chunk count: 219 | |
677 | percentage done: 2, chunk count: 453 | |
678 | ... | |
679 | percentage done: 99, chunk count: 21188 | |
680 | Removed bytes: 411368505 | |
681 | Removed chunks: 203 | |
682 | Original data bytes: 327160886391 | |
683 | Disk bytes: 52767414743 (16 %) | |
684 | Disk chunks: 21221 | |
685 | Average chunk size: 2486565 | |
686 | TASK OK | |
687 | ||
688 | ||
3bbb70b3 | 689 | .. todo:: howto run garbage-collection at regular intervals (cron) |
04e24b14 DW |
690 | |
691 | Benchmarking | |
692 | ------------ | |
693 | ||
694 | The backup client also comes with a benchmarking tool. This tool measures | |
695 | various metrics relating to compression and encryption speeds. You can run a | |
696 | benchmark using the ``benchmark`` subcommand of ``proxmox-backup-client``: | |
697 | ||
698 | .. code-block:: console | |
699 | ||
700 | # proxmox-backup-client benchmark | |
701 | Uploaded 656 chunks in 5 seconds. | |
702 | Time per request: 7659 microseconds. | |
703 | TLS speed: 547.60 MB/s | |
704 | SHA256 speed: 585.76 MB/s | |
705 | Compression speed: 1923.96 MB/s | |
706 | Decompress speed: 7885.24 MB/s | |
707 | AES256/GCM speed: 3974.03 MB/s | |
708 | ┌───────────────────────────────────┬─────────────────────┐ | |
709 | │ Name │ Value │ | |
710 | ╞═══════════════════════════════════╪═════════════════════╡ | |
711 | │ TLS (maximal backup upload speed) │ 547.60 MB/s (93%) │ | |
712 | ├───────────────────────────────────┼─────────────────────┤ | |
713 | │ SHA256 checksum computation speed │ 585.76 MB/s (28%) │ | |
714 | ├───────────────────────────────────┼─────────────────────┤ | |
715 | │ ZStd level 1 compression speed │ 1923.96 MB/s (89%) │ | |
716 | ├───────────────────────────────────┼─────────────────────┤ | |
717 | │ ZStd level 1 decompression speed │ 7885.24 MB/s (98%) │ | |
718 | ├───────────────────────────────────┼─────────────────────┤ | |
719 | │ AES256 GCM encryption speed │ 3974.03 MB/s (104%) │ | |
720 | └───────────────────────────────────┴─────────────────────┘ | |
721 | ||
722 | .. note:: The percentages given in the output table correspond to a | |
723 | comparison against a Ryzen 7 2700X. The TLS test connects to the | |
724 | local host, so there is no network involved. | |
725 | ||
726 | You can also pass the ``--output-format`` parameter to output stats in ``json``, | |
727 | rather than the default table format. |