]>
Commit | Line | Data |
---|---|---|
04e24b14 DW |
1 | Backup Client Usage |
2 | =================== | |
3 | ||
4 | The command line client is called :command:`proxmox-backup-client`. | |
5 | ||
654c56e0 | 6 | .. _client_repository: |
04e24b14 | 7 | |
fd8b00ae TL |
8 | Backup Repository Locations |
9 | --------------------------- | |
04e24b14 DW |
10 | |
11 | The client uses the following notation to specify a datastore repository | |
12 | on the backup server. | |
13 | ||
14 | [[username@]server[:port]:]datastore | |
15 | ||
4aef06f1 | 16 | The default value for ``username`` is ``root@pam``. If no server is specified, |
04e24b14 DW |
17 | the default is the local host (``localhost``). |
18 | ||
19 | You can specify a port if your backup server is only reachable on a different | |
20 | port (e.g. with NAT and port forwarding). | |
21 | ||
4aef06f1 TL |
22 | Note that if the server is an IPv6 address, you have to write it with square |
23 | brackets (for example, `[fe80::01]`). | |
04e24b14 | 24 | |
4aef06f1 TL |
25 | You can pass the repository with the ``--repository`` command line option, or |
26 | by setting the ``PBS_REPOSITORY`` environment variable. | |
04e24b14 DW |
27 | |
28 | Here some examples of valid repositories and the real values | |
29 | ||
c0af05e1 | 30 | ================================ ================== ================== =========== |
4aef06f1 | 31 | Example User Host:Port Datastore |
c0af05e1 | 32 | ================================ ================== ================== =========== |
4aef06f1 TL |
33 | mydatastore ``root@pam`` localhost:8007 mydatastore |
34 | myhostname:mydatastore ``root@pam`` myhostname:8007 mydatastore | |
35 | user@pbs@myhostname:mydatastore ``user@pbs`` myhostname:8007 mydatastore | |
90e94aa2 | 36 | user\@pbs!token@host:store ``user@pbs!token`` myhostname:8007 mydatastore |
4aef06f1 TL |
37 | 192.168.55.55:1234:mydatastore ``root@pam`` 192.168.55.55:1234 mydatastore |
38 | [ff80::51]:mydatastore ``root@pam`` [ff80::51]:8007 mydatastore | |
39 | [ff80::51]:1234:mydatastore ``root@pam`` [ff80::51]:1234 mydatastore | |
c0af05e1 | 40 | ================================ ================== ================== =========== |
04e24b14 DW |
41 | |
42 | Environment Variables | |
43 | --------------------- | |
44 | ||
45 | ``PBS_REPOSITORY`` | |
46 | The default backup repository. | |
47 | ||
48 | ``PBS_PASSWORD`` | |
4aef06f1 TL |
49 | When set, this value is used for the password required for the backup server. |
50 | You can also set this to a API token secret. | |
04e24b14 | 51 | |
65ab4ca9 TL |
52 | ``PBS_PASSWORD_FD``, ``PBS_PASSWORD_FILE``, ``PBS_PASSWORD_CMD`` |
53 | Like ``PBS_PASSWORD``, but read data from an open file descriptor, a file | |
54 | name or from the `stdout` of a command, respectively. The first defined | |
55 | environment variable from the order above is preferred. | |
99a73fad | 56 | |
04e24b14 DW |
57 | ``PBS_ENCRYPTION_PASSWORD`` |
58 | When set, this value is used to access the secret encryption key (if | |
59 | protected by password). | |
60 | ||
65ab4ca9 TL |
61 | ``PBS_ENCRYPTION_PASSWORD_FD``, ``PBS_ENCRYPTION_PASSWORD_FILE``, ``PBS_ENCRYPTION_PASSWORD_CMD`` |
62 | Like ``PBS_ENCRYPTION_PASSWORD``, but read data from an open file descriptor, | |
63 | a file name or from the `stdout` of a command, respectively. The first | |
64 | defined environment variable from the order above is preferred. | |
99a73fad | 65 | |
04e24b14 | 66 | ``PBS_FINGERPRINT`` When set, this value is used to verify the server |
4aef06f1 TL |
67 | certificate (only used if the system CA certificates cannot validate the |
68 | certificate). | |
04e24b14 DW |
69 | |
70 | ||
99a73fad DM |
71 | .. Note:: Passwords must be valid UTF8 an may not contain |
72 | newlines. For your convienience, we just use the first line as | |
73 | password, so you can add arbitrary comments after the | |
74 | first newline. | |
75 | ||
76 | ||
04e24b14 DW |
77 | Output Format |
78 | ------------- | |
79 | ||
f458e97f | 80 | .. include:: output-format.rst |
04e24b14 DW |
81 | |
82 | ||
ec1ae7e6 | 83 | .. _client_creating_backups: |
04e24b14 DW |
84 | |
85 | Creating Backups | |
86 | ---------------- | |
87 | ||
88 | This section explains how to create a backup from within the machine. This can | |
89 | be a physical host, a virtual machine, or a container. Such backups may contain file | |
90 | and image archives. There are no restrictions in this case. | |
91 | ||
92 | .. note:: If you want to backup virtual machines or containers on Proxmox VE, see :ref:`pve-integration`. | |
93 | ||
94 | For the following example you need to have a backup server set up, working | |
95 | credentials and need to know the repository name. | |
96 | In the following examples we use ``backup-server:store1``. | |
97 | ||
98 | .. code-block:: console | |
99 | ||
100 | # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1 | |
101 | Starting backup: host/elsa/2019-12-03T09:35:01Z | |
102 | Client name: elsa | |
103 | skip mount point: "/boot/efi" | |
104 | skip mount point: "/dev" | |
105 | skip mount point: "/run" | |
106 | skip mount point: "/sys" | |
107 | Uploaded 12129 chunks in 87 seconds (564 MB/s). | |
108 | End Time: 2019-12-03T10:36:29+01:00 | |
109 | ||
110 | This will prompt you for a password and then uploads a file archive named | |
111 | ``root.pxar`` containing all the files in the ``/`` directory. | |
112 | ||
113 | .. Caution:: Please note that the proxmox-backup-client does not | |
114 | automatically include mount points. Instead, you will see a short | |
115 | ``skip mount point`` notice for each of them. The idea is to | |
116 | create a separate file archive for each mounted disk. You can | |
117 | explicitly include them using the ``--include-dev`` option | |
118 | (i.e. ``--include-dev /boot/efi``). You can use this option | |
119 | multiple times for each mount point that should be included. | |
120 | ||
121 | The ``--repository`` option can get quite long and is used by all | |
122 | commands. You can avoid having to enter this value by setting the | |
123 | environment variable ``PBS_REPOSITORY``. Note that if you would like this to remain set | |
124 | over multiple sessions, you should instead add the below line to your | |
125 | ``.bashrc`` file. | |
126 | ||
127 | .. code-block:: console | |
128 | ||
129 | # export PBS_REPOSITORY=backup-server:store1 | |
130 | ||
131 | After this you can execute all commands without specifying the ``--repository`` | |
132 | option. | |
133 | ||
134 | One single backup is allowed to contain more than one archive. For example, if | |
3bbb70b3 | 135 | you want to backup two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``: |
04e24b14 DW |
136 | |
137 | .. code-block:: console | |
138 | ||
139 | # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 | |
140 | ||
141 | This creates a backup of both disks. | |
142 | ||
143 | The backup command takes a list of backup specifications, which | |
144 | include the archive name on the server, the type of the archive, and the | |
145 | archive source at the client. The format is: | |
146 | ||
147 | <archive-name>.<type>:<source-path> | |
148 | ||
149 | Common types are ``.pxar`` for file archives, and ``.img`` for block | |
150 | device images. To create a backup of a block device run the following command: | |
151 | ||
152 | .. code-block:: console | |
153 | ||
154 | # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata | |
155 | ||
156 | ||
157 | Excluding files/folders from a backup | |
158 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
159 | ||
160 | Sometimes it is desired to exclude certain files or folders from a backup archive. | |
161 | To tell the Proxmox Backup client when and how to ignore files and directories, | |
162 | place a text file called ``.pxarexclude`` in the filesystem hierarchy. | |
163 | Whenever the backup client encounters such a file in a directory, it interprets | |
164 | each line as glob match patterns for files and directories that are to be excluded | |
165 | from the backup. | |
166 | ||
167 | The file must contain a single glob pattern per line. Empty lines are ignored. | |
168 | The same is true for lines starting with ``#``, which indicates a comment. | |
169 | A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion | |
170 | to an explicit inclusion. This makes it possible to exclude all entries in a | |
171 | directory except for a few single files/subdirectories. | |
172 | Lines ending in ``/`` match only on directories. | |
173 | The directory containing the ``.pxarexclude`` file is considered to be the root of | |
174 | the given patterns. It is only possible to match files in this directory and its subdirectories. | |
175 | ||
176 | ``\`` is used to escape special glob characters. | |
177 | ``?`` matches any single character. | |
178 | ``*`` matches any character, including an empty string. | |
179 | ``**`` is used to match subdirectories. It can be used to, for example, exclude | |
180 | all files ending in ``.tmp`` within the directory or subdirectories with the | |
181 | following pattern ``**/*.tmp``. | |
182 | ``[...]`` matches a single character from any of the provided characters within | |
183 | the brackets. ``[!...]`` does the complementary and matches any single character | |
184 | not contained within the brackets. It is also possible to specify ranges with two | |
185 | characters separated by ``-``. For example, ``[a-z]`` matches any lowercase | |
186 | alphabetic character and ``[0-9]`` matches any one single digit. | |
187 | ||
188 | The order of the glob match patterns defines whether a file is included or | |
189 | excluded, that is to say later entries override previous ones. | |
190 | This is also true for match patterns encountered deeper down the directory tree, | |
191 | which can override a previous exclusion. | |
192 | Be aware that excluded directories will **not** be read by the backup client. | |
193 | Thus, a ``.pxarexclude`` file in an excluded subdirectory will have no effect. | |
194 | ``.pxarexclude`` files are treated as regular files and will be included in the | |
195 | backup archive. | |
196 | ||
197 | For example, consider the following directory structure: | |
198 | ||
199 | .. code-block:: console | |
200 | ||
201 | # ls -aR folder | |
202 | folder/: | |
203 | . .. .pxarexclude subfolder0 subfolder1 | |
204 | ||
205 | folder/subfolder0: | |
206 | . .. file0 file1 file2 file3 .pxarexclude | |
207 | ||
208 | folder/subfolder1: | |
209 | . .. file0 file1 file2 file3 | |
210 | ||
211 | The different ``.pxarexclude`` files contain the following: | |
212 | ||
213 | .. code-block:: console | |
214 | ||
215 | # cat folder/.pxarexclude | |
216 | /subfolder0/file1 | |
217 | /subfolder1/* | |
218 | !/subfolder1/file2 | |
219 | ||
220 | .. code-block:: console | |
221 | ||
222 | # cat folder/subfolder0/.pxarexclude | |
223 | file3 | |
224 | ||
225 | This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of | |
226 | ``subfolder1`` except ``file2``. | |
227 | ||
228 | Restoring this backup will result in: | |
229 | ||
230 | .. code-block:: console | |
231 | ||
232 | ls -aR restored | |
233 | restored/: | |
234 | . .. .pxarexclude subfolder0 subfolder1 | |
235 | ||
236 | restored/subfolder0: | |
237 | . .. file0 file2 .pxarexclude | |
238 | ||
239 | restored/subfolder1: | |
240 | . .. file2 | |
241 | ||
242 | ||
ec1ae7e6 | 243 | .. _client_encryption: |
d9f365d7 | 244 | |
04e24b14 DW |
245 | Encryption |
246 | ---------- | |
247 | ||
248 | Proxmox Backup supports client-side encryption with AES-256 in GCM_ | |
249 | mode. To set this up, you first need to create an encryption key: | |
250 | ||
251 | .. code-block:: console | |
252 | ||
253 | # proxmox-backup-client key create my-backup.key | |
254 | Encryption Key Password: ************** | |
255 | ||
256 | The key is password protected by default. If you do not need this | |
257 | extra protection, you can also create it without a password: | |
258 | ||
259 | .. code-block:: console | |
260 | ||
261 | # proxmox-backup-client key create /path/to/my-backup.key --kdf none | |
262 | ||
263 | Having created this key, it is now possible to create an encrypted backup, by | |
264 | passing the ``--keyfile`` parameter, with the path to the key file. | |
265 | ||
266 | .. code-block:: console | |
267 | ||
268 | # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key | |
269 | Password: ********* | |
270 | Encryption Key Password: ************** | |
271 | ... | |
272 | ||
273 | .. Note:: If you do not specify the name of the backup key, the key will be | |
274 | created in the default location | |
275 | ``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client`` | |
276 | will also search this location by default, in case the ``--keyfile`` | |
277 | parameter is not specified. | |
278 | ||
279 | You can avoid entering the passwords by setting the environment | |
280 | variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``. | |
281 | ||
282 | ||
283 | Using a master key to store and recover encryption keys | |
284 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
285 | ||
286 | You can also use ``proxmox-backup-client key`` to create an RSA public/private | |
287 | key pair, which can be used to store an encrypted version of the symmetric | |
288 | backup encryption key alongside each backup and recover it later. | |
289 | ||
290 | To set up a master key: | |
291 | ||
292 | 1. Create an encryption key for the backup: | |
293 | ||
294 | .. code-block:: console | |
295 | ||
296 | # proxmox-backup-client key create | |
297 | creating default key at: "~/.config/proxmox-backup/encryption-key.json" | |
298 | Encryption Key Password: ********** | |
299 | ... | |
300 | ||
301 | The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``. | |
302 | ||
303 | 2. Create an RSA public/private key pair: | |
304 | ||
305 | .. code-block:: console | |
306 | ||
307 | # proxmox-backup-client key create-master-key | |
308 | Master Key Password: ********* | |
309 | ... | |
310 | ||
311 | This will create two files in your current directory, ``master-public.pem`` | |
312 | and ``master-private.pem``. | |
313 | ||
314 | 3. Import the newly created ``master-public.pem`` public certificate, so that | |
315 | ``proxmox-backup-client`` can find and use it upon backup. | |
316 | ||
317 | .. code-block:: console | |
318 | ||
319 | # proxmox-backup-client key import-master-pubkey /path/to/master-public.pem | |
320 | Imported public master key to "~/.config/proxmox-backup/master-public.pem" | |
321 | ||
322 | 4. With all these files in place, run a backup job: | |
323 | ||
324 | .. code-block:: console | |
325 | ||
326 | # proxmox-backup-client backup etc.pxar:/etc | |
327 | ||
328 | The key will be stored in your backup, under the name ``rsa-encrypted.key``. | |
329 | ||
330 | .. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key | |
331 | is in the default path. If you specified another path upon creation, you | |
332 | must pass the ``--keyfile`` parameter. | |
333 | ||
334 | 5. To test that everything worked, you can restore the key from the backup: | |
335 | ||
336 | .. code-block:: console | |
337 | ||
338 | # proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target | |
339 | ||
340 | .. Note:: You should not need an encryption key to extract this file. However, if | |
341 | a key exists at the default location | |
342 | (``~/.config/proxmox-backup/encryption-key.json``) the program will prompt | |
343 | you for an encryption key password. Simply moving ``encryption-key.json`` | |
344 | out of this directory will fix this issue. | |
345 | ||
346 | 6. Then, use the previously generated master key to decrypt the file: | |
347 | ||
348 | .. code-block:: console | |
349 | ||
2ea5abcd FG |
350 | # proxmox-backup-client key import-with-master-key /path/to/target --master-keyfile /path/to/master-private.pem --encrypted-keyfile /path/to/rsa-encrypted.key |
351 | Master Key Password: ****** | |
352 | New Password: ****** | |
353 | Verify Password: ****** | |
04e24b14 DW |
354 | |
355 | 7. The target file will now contain the encryption key information in plain | |
356 | text. The success of this can be confirmed by passing the resulting ``json`` | |
357 | file, with the ``--keyfile`` parameter, when decrypting files from the backup. | |
358 | ||
359 | .. warning:: Without their key, backed up files will be inaccessible. Thus, you should | |
360 | keep keys ordered and in a place that is separate from the contents being | |
361 | backed up. It can happen, for example, that you back up an entire system, using | |
cdbc18fc | 362 | a key on that system. If the system then becomes inaccessible for any reason |
04e24b14 | 363 | and needs to be restored, this will not be possible as the encryption key will be |
4accbc58 DW |
364 | lost along with the broken system. |
365 | ||
c5608cf8 DW |
366 | It is recommended that you keep your master key safe, but easily accessible, in |
367 | order for quick disaster recovery. For this reason, the best place to store it | |
368 | is in your password manager, where it is immediately recoverable. As a backup to | |
369 | this, you should also save the key to a USB drive and store that in a secure | |
370 | place. This way, it is detached from any system, but is still easy to recover | |
371 | from, in case of emergency. Finally, in preparation for the worst case scenario, | |
372 | you should also consider keeping a paper copy of your master key locked away in | |
373 | a safe place. The ``paperkey`` subcommand can be used to create a QR encoded | |
374 | version of your master key. The following command sends the output of the | |
375 | ``paperkey`` command to a text file, for easy printing. | |
4accbc58 DW |
376 | |
377 | .. code-block:: console | |
378 | ||
379 | proxmox-backup-client key paperkey --output-format text > qrkey.txt | |
04e24b14 DW |
380 | |
381 | ||
382 | Restoring Data | |
383 | -------------- | |
384 | ||
385 | The regular creation of backups is a necessary step to avoiding data | |
386 | loss. More importantly, however, is the restoration. It is good practice to perform | |
387 | periodic recovery tests to ensure that you can access the data in | |
388 | case of problems. | |
389 | ||
390 | First, you need to find the snapshot which you want to restore. The snapshot | |
abf9b6da | 391 | list command provides a list of all the snapshots on the server: |
04e24b14 DW |
392 | |
393 | .. code-block:: console | |
394 | ||
abf9b6da | 395 | # proxmox-backup-client snapshot list |
04e24b14 DW |
396 | ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐ |
397 | │ snapshot │ size │ files │ | |
398 | ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡ | |
399 | │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │ | |
400 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
401 | │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │ | |
402 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
403 | ... | |
404 | ||
405 | You can inspect the catalog to find specific files. | |
406 | ||
407 | .. code-block:: console | |
408 | ||
409 | # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z | |
410 | ... | |
411 | d "./root.pxar.didx/etc/cifs-utils" | |
412 | l "./root.pxar.didx/etc/cifs-utils/idmap-plugin" | |
413 | d "./root.pxar.didx/etc/console-setup" | |
414 | ... | |
415 | ||
416 | The restore command lets you restore a single archive from the | |
417 | backup. | |
418 | ||
419 | .. code-block:: console | |
420 | ||
421 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/ | |
422 | ||
423 | To get the contents of any archive, you can restore the ``index.json`` file in the | |
424 | repository to the target path '-'. This will dump the contents to the standard output. | |
425 | ||
426 | .. code-block:: console | |
427 | ||
428 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json - | |
429 | ||
430 | ||
431 | Interactive Restores | |
432 | ~~~~~~~~~~~~~~~~~~~~ | |
433 | ||
434 | If you only want to restore a few individual files, it is often easier | |
435 | to use the interactive recovery shell. | |
436 | ||
437 | .. code-block:: console | |
438 | ||
439 | # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar | |
440 | Starting interactive shell | |
441 | pxar:/ > ls | |
442 | bin boot dev etc home lib lib32 | |
443 | ... | |
444 | ||
3bbb70b3 | 445 | The interactive recovery shell is a minimal command line interface that |
04e24b14 DW |
446 | utilizes the metadata stored in the catalog to quickly list, navigate and |
447 | search files in a file archive. | |
448 | To restore files, you can select them individually or match them with a glob | |
449 | pattern. | |
450 | ||
451 | Using the catalog for navigation reduces the overhead considerably because only | |
452 | the catalog needs to be downloaded and, optionally, decrypted. | |
453 | The actual chunks are only accessed if the metadata in the catalog is not enough | |
454 | or for the actual restore. | |
455 | ||
456 | Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change | |
457 | working directory and list directory contents in the archive. | |
458 | ``pwd`` shows the full path of the current working directory with respect to the | |
459 | archive root. | |
460 | ||
3bbb70b3 | 461 | Being able to quickly search the contents of the archive is a commonly needed feature. |
04e24b14 DW |
462 | That's where the catalog is most valuable. |
463 | For example: | |
464 | ||
465 | .. code-block:: console | |
466 | ||
467 | pxar:/ > find etc/**/*.txt --select | |
468 | "/etc/X11/rgb.txt" | |
469 | pxar:/ > list-selected | |
470 | etc/**/*.txt | |
471 | pxar:/ > restore-selected /target/path | |
472 | ... | |
473 | ||
474 | This will find and print all files ending in ``.txt`` located in ``etc/`` or a | |
475 | subdirectory and add the corresponding pattern to the list for subsequent restores. | |
476 | ``list-selected`` shows these patterns and ``restore-selected`` finally restores | |
477 | all files in the archive matching the patterns to ``/target/path`` on the local | |
478 | host. This will scan the whole archive. | |
479 | ||
985e84e3 DW |
480 | The ``restore`` command can be used to restore all the files contained within |
481 | the backup archive. This is most helpful when paired with the ``--pattern | |
482 | <glob>`` option, as it allows you to restore all files matching a specific | |
483 | pattern. For example, if you wanted to restore configuration files | |
484 | located in ``/etc``, you could do the following: | |
04e24b14 DW |
485 | |
486 | .. code-block:: console | |
487 | ||
985e84e3 | 488 | pxar:/ > restore target/ --pattern etc/**/*.conf |
04e24b14 DW |
489 | ... |
490 | ||
5f3b2330 | 491 | The above will scan through all the directories below ``/etc`` and restore all |
04e24b14 DW |
492 | files ending in ``.conf``. |
493 | ||
494 | .. todo:: Explain interactive restore in more detail | |
495 | ||
496 | Mounting of Archives via FUSE | |
497 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
498 | ||
499 | The :term:`FUSE` implementation for the pxar archive allows you to mount a | |
500 | file archive as a read-only filesystem to a mountpoint on your host. | |
501 | ||
502 | .. code-block:: console | |
503 | ||
504 | # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint | |
505 | # ls /mnt/mountpoint | |
506 | bin dev home lib32 libx32 media opt root sbin sys usr | |
507 | boot etc lib lib64 lost+found mnt proc run srv tmp var | |
508 | ||
509 | This allows you to access the full contents of the archive in a seamless manner. | |
510 | ||
511 | .. note:: As the FUSE connection needs to fetch and decrypt chunks from the | |
512 | backup server's datastore, this can cause some additional network and CPU | |
513 | load on your host, depending on the operations you perform on the mounted | |
514 | filesystem. | |
515 | ||
516 | To unmount the filesystem use the ``umount`` command on the mountpoint: | |
517 | ||
518 | .. code-block:: console | |
519 | ||
520 | # umount /mnt/mountpoint | |
521 | ||
522 | Login and Logout | |
523 | ---------------- | |
524 | ||
525 | The client tool prompts you to enter the logon password as soon as you | |
526 | want to access the backup server. The server checks your credentials | |
527 | and responds with a ticket that is valid for two hours. The client | |
528 | tool automatically stores that ticket and uses it for further requests | |
529 | to this server. | |
530 | ||
531 | You can also manually trigger this login/logout using the login and | |
532 | logout commands: | |
533 | ||
534 | .. code-block:: console | |
535 | ||
536 | # proxmox-backup-client login | |
537 | Password: ********** | |
538 | ||
539 | To remove the ticket, issue a logout: | |
540 | ||
541 | .. code-block:: console | |
542 | ||
543 | # proxmox-backup-client logout | |
544 | ||
545 | ||
08379a21 DW |
546 | .. _changing-backup-owner: |
547 | ||
548 | Changing the Owner of a Backup Group | |
549 | ------------------------------------ | |
550 | ||
551 | By default, the owner of a backup group is the user which was used to originally | |
552 | create that backup group (or in the case of sync jobs, ``root@pam``). This | |
553 | means that if a user ``mike@pbs`` created a backup, another user ``john@pbs`` | |
554 | can not be used to create backups in that same backup group. In case you want | |
555 | to change the owner of a backup, you can do so with the below command, using a | |
556 | user that has ``Datastore.Modify`` privileges on the datastore. | |
557 | ||
558 | .. code-block:: console | |
559 | ||
560 | # proxmox-backup-client change-owner vm/103 john@pbs | |
561 | ||
562 | This can also be done from within the web interface, by navigating to the | |
563 | `Content` section of the datastore that contains the backup group and | |
564 | selecting the user icon under the `Actions` column. Common cases for this could | |
565 | be to change the owner of a sync job from ``root@pam``, or to repurpose a | |
566 | backup group. | |
567 | ||
568 | ||
4c3efb53 | 569 | .. _backup-pruning: |
04e24b14 DW |
570 | |
571 | Pruning and Removing Backups | |
572 | ---------------------------- | |
573 | ||
574 | You can manually delete a backup snapshot using the ``forget`` | |
575 | command: | |
576 | ||
577 | .. code-block:: console | |
578 | ||
abf9b6da | 579 | # proxmox-backup-client snapshot forget <snapshot> |
04e24b14 DW |
580 | |
581 | ||
582 | .. caution:: This command removes all archives in this backup | |
583 | snapshot. They will be inaccessible and unrecoverable. | |
584 | ||
585 | ||
586 | Although manual removal is sometimes required, the ``prune`` | |
587 | command is normally used to systematically delete older backups. Prune lets | |
588 | you specify which backup snapshots you want to keep. The | |
589 | following retention options are available: | |
590 | ||
591 | ``--keep-last <N>`` | |
592 | Keep the last ``<N>`` backup snapshots. | |
593 | ||
594 | ``--keep-hourly <N>`` | |
595 | Keep backups for the last ``<N>`` hours. If there is more than one | |
596 | backup for a single hour, only the latest is kept. | |
597 | ||
598 | ``--keep-daily <N>`` | |
599 | Keep backups for the last ``<N>`` days. If there is more than one | |
600 | backup for a single day, only the latest is kept. | |
601 | ||
602 | ``--keep-weekly <N>`` | |
603 | Keep backups for the last ``<N>`` weeks. If there is more than one | |
604 | backup for a single week, only the latest is kept. | |
605 | ||
606 | .. note:: Weeks start on Monday and end on Sunday. The software | |
607 | uses the `ISO week date`_ system and handles weeks at | |
608 | the end of the year correctly. | |
609 | ||
610 | ``--keep-monthly <N>`` | |
611 | Keep backups for the last ``<N>`` months. If there is more than one | |
612 | backup for a single month, only the latest is kept. | |
613 | ||
614 | ``--keep-yearly <N>`` | |
615 | Keep backups for the last ``<N>`` years. If there is more than one | |
616 | backup for a single year, only the latest is kept. | |
617 | ||
618 | The retention options are processed in the order given above. Each option | |
619 | only covers backups within its time period. The next option does not take care | |
620 | of already covered backups. It will only consider older backups. | |
621 | ||
622 | Unfinished and incomplete backups will be removed by the prune command unless | |
623 | they are newer than the last successful backup. In this case, the last failed | |
624 | backup is retained. | |
625 | ||
626 | .. code-block:: console | |
627 | ||
628 | # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3 | |
629 | ||
630 | ||
631 | You can use the ``--dry-run`` option to test your settings. This only | |
632 | shows the list of existing snapshots and what actions prune would take. | |
633 | ||
634 | .. code-block:: console | |
635 | ||
636 | # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3 | |
637 | ┌────────────────────────────────┬──────┐ | |
638 | │ snapshot │ keep │ | |
639 | ╞════════════════════════════════╪══════╡ | |
640 | │ host/elsa/2019-12-04T13:20:37Z │ 1 │ | |
641 | ├────────────────────────────────┼──────┤ | |
642 | │ host/elsa/2019-12-03T09:35:01Z │ 0 │ | |
643 | ├────────────────────────────────┼──────┤ | |
644 | │ host/elsa/2019-11-22T11:54:47Z │ 1 │ | |
645 | ├────────────────────────────────┼──────┤ | |
646 | │ host/elsa/2019-11-21T12:36:25Z │ 0 │ | |
647 | ├────────────────────────────────┼──────┤ | |
648 | │ host/elsa/2019-11-10T10:42:20Z │ 1 │ | |
649 | └────────────────────────────────┴──────┘ | |
650 | ||
651 | .. note:: Neither the ``prune`` command nor the ``forget`` command free space | |
652 | in the chunk-store. The chunk-store still contains the data blocks. To free | |
a98e2287 | 653 | space you need to perform :ref:`client_garbage-collection`. |
04e24b14 DW |
654 | |
655 | ||
ec1ae7e6 | 656 | .. _client_garbage-collection: |
04e24b14 DW |
657 | |
658 | Garbage Collection | |
659 | ------------------ | |
660 | ||
661 | The ``prune`` command removes only the backup index files, not the data | |
662 | from the datastore. This task is left to the garbage collection | |
663 | command. It is recommended to carry out garbage collection on a regular basis. | |
664 | ||
665 | The garbage collection works in two phases. In the first phase, all | |
666 | data blocks that are still in use are marked. In the second phase, | |
667 | unused data blocks are removed. | |
668 | ||
669 | .. note:: This command needs to read all existing backup index files | |
670 | and touches the complete chunk-store. This can take a long time | |
671 | depending on the number of chunks and the speed of the underlying | |
672 | disks. | |
673 | ||
674 | .. note:: The garbage collection will only remove chunks that haven't been used | |
675 | for at least one day (exactly 24h 5m). This grace period is necessary because | |
676 | chunks in use are marked by touching the chunk which updates the ``atime`` | |
677 | (access time) property. Filesystems are mounted with the ``relatime`` option | |
678 | by default. This results in a better performance by only updating the | |
679 | ``atime`` property if the last access has been at least 24 hours ago. The | |
680 | downside is, that touching a chunk within these 24 hours will not always | |
681 | update its ``atime`` property. | |
682 | ||
683 | Chunks in the grace period will be logged at the end of the garbage | |
684 | collection task as *Pending removals*. | |
685 | ||
686 | .. code-block:: console | |
687 | ||
688 | # proxmox-backup-client garbage-collect | |
689 | starting garbage collection on store store2 | |
690 | Start GC phase1 (mark used chunks) | |
691 | Start GC phase2 (sweep unused chunks) | |
692 | percentage done: 1, chunk count: 219 | |
693 | percentage done: 2, chunk count: 453 | |
694 | ... | |
695 | percentage done: 99, chunk count: 21188 | |
696 | Removed bytes: 411368505 | |
697 | Removed chunks: 203 | |
698 | Original data bytes: 327160886391 | |
699 | Disk bytes: 52767414743 (16 %) | |
700 | Disk chunks: 21221 | |
701 | Average chunk size: 2486565 | |
702 | TASK OK | |
703 | ||
704 | ||
3bbb70b3 | 705 | .. todo:: howto run garbage-collection at regular intervals (cron) |
04e24b14 DW |
706 | |
707 | Benchmarking | |
708 | ------------ | |
709 | ||
710 | The backup client also comes with a benchmarking tool. This tool measures | |
654c56e0 TL |
711 | various metrics relating to compression and encryption speeds. If a Proxmox |
712 | Backup repository (remote or local) is specified, the TLS upload speed will get | |
713 | measured too. | |
714 | ||
715 | You can run a benchmark using the ``benchmark`` subcommand of | |
716 | ``proxmox-backup-client``: | |
717 | ||
718 | .. note:: The TLS speed test is only included if a :ref:`backup server | |
719 | repository is specified <client_repository>`. | |
04e24b14 DW |
720 | |
721 | .. code-block:: console | |
722 | ||
723 | # proxmox-backup-client benchmark | |
81f5d03e DM |
724 | Uploaded 1517 chunks in 5 seconds. |
725 | Time per request: 3309 microseconds. | |
726 | TLS speed: 1267.41 MB/s | |
727 | SHA256 speed: 2066.73 MB/s | |
728 | Compression speed: 775.11 MB/s | |
729 | Decompress speed: 1233.35 MB/s | |
730 | AES256/GCM speed: 3688.27 MB/s | |
731 | Verify speed: 783.43 MB/s | |
04e24b14 DW |
732 | ┌───────────────────────────────────┬─────────────────────┐ |
733 | │ Name │ Value │ | |
734 | ╞═══════════════════════════════════╪═════════════════════╡ | |
81f5d03e | 735 | │ TLS (maximal backup upload speed) │ 1267.41 MB/s (103%) │ |
04e24b14 | 736 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 737 | │ SHA256 checksum computation speed │ 2066.73 MB/s (102%) │ |
04e24b14 | 738 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 739 | │ ZStd level 1 compression speed │ 775.11 MB/s (103%) │ |
04e24b14 | 740 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 741 | │ ZStd level 1 decompression speed │ 1233.35 MB/s (103%) │ |
04e24b14 | 742 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e DM |
743 | │ Chunk verification speed │ 783.43 MB/s (103%) │ |
744 | ├───────────────────────────────────┼─────────────────────┤ | |
745 | │ AES256 GCM encryption speed │ 3688.27 MB/s (101%) │ | |
04e24b14 DW |
746 | └───────────────────────────────────┴─────────────────────┘ |
747 | ||
81f5d03e | 748 | |
04e24b14 | 749 | .. note:: The percentages given in the output table correspond to a |
654c56e0 | 750 | comparison against a Ryzen 7 2700X. |
04e24b14 DW |
751 | |
752 | You can also pass the ``--output-format`` parameter to output stats in ``json``, | |
753 | rather than the default table format. |