]>
Commit | Line | Data |
---|---|---|
04e24b14 DW |
1 | Backup Client Usage |
2 | =================== | |
3 | ||
5fb852af DW |
4 | The command line client for Proxmox Backup Server is called |
5 | :command:`proxmox-backup-client`. | |
04e24b14 | 6 | |
654c56e0 | 7 | .. _client_repository: |
04e24b14 | 8 | |
fd8b00ae TL |
9 | Backup Repository Locations |
10 | --------------------------- | |
04e24b14 | 11 | |
5fb852af DW |
12 | The client uses the following format to specify a datastore repository |
13 | on the backup server (where username is specified in the form of user@realm): | |
04e24b14 DW |
14 | |
15 | [[username@]server[:port]:]datastore | |
16 | ||
4aef06f1 | 17 | The default value for ``username`` is ``root@pam``. If no server is specified, |
04e24b14 DW |
18 | the default is the local host (``localhost``). |
19 | ||
5fb852af DW |
20 | You can specify a port if your backup server is only reachable on a non-default |
21 | port (for example, with NAT and port forwarding configurations). | |
04e24b14 | 22 | |
5fb852af | 23 | Note that if the server uses an IPv6 address, you have to write it with square |
4aef06f1 | 24 | brackets (for example, `[fe80::01]`). |
04e24b14 | 25 | |
4aef06f1 TL |
26 | You can pass the repository with the ``--repository`` command line option, or |
27 | by setting the ``PBS_REPOSITORY`` environment variable. | |
04e24b14 | 28 | |
5fb852af DW |
29 | Below are some examples of valid repositories and their corresponding real |
30 | values: | |
04e24b14 | 31 | |
c0af05e1 | 32 | ================================ ================== ================== =========== |
4aef06f1 | 33 | Example User Host:Port Datastore |
c0af05e1 | 34 | ================================ ================== ================== =========== |
4aef06f1 TL |
35 | mydatastore ``root@pam`` localhost:8007 mydatastore |
36 | myhostname:mydatastore ``root@pam`` myhostname:8007 mydatastore | |
37 | user@pbs@myhostname:mydatastore ``user@pbs`` myhostname:8007 mydatastore | |
90e94aa2 | 38 | user\@pbs!token@host:store ``user@pbs!token`` myhostname:8007 mydatastore |
4aef06f1 TL |
39 | 192.168.55.55:1234:mydatastore ``root@pam`` 192.168.55.55:1234 mydatastore |
40 | [ff80::51]:mydatastore ``root@pam`` [ff80::51]:8007 mydatastore | |
41 | [ff80::51]:1234:mydatastore ``root@pam`` [ff80::51]:1234 mydatastore | |
c0af05e1 | 42 | ================================ ================== ================== =========== |
04e24b14 DW |
43 | |
44 | Environment Variables | |
45 | --------------------- | |
46 | ||
47 | ``PBS_REPOSITORY`` | |
48 | The default backup repository. | |
49 | ||
50 | ``PBS_PASSWORD`` | |
5fb852af DW |
51 | When set, this value is used as the password for the backup server. |
52 | You can also set this to an API token secret. | |
04e24b14 | 53 | |
65ab4ca9 TL |
54 | ``PBS_PASSWORD_FD``, ``PBS_PASSWORD_FILE``, ``PBS_PASSWORD_CMD`` |
55 | Like ``PBS_PASSWORD``, but read data from an open file descriptor, a file | |
56 | name or from the `stdout` of a command, respectively. The first defined | |
57 | environment variable from the order above is preferred. | |
99a73fad | 58 | |
04e24b14 DW |
59 | ``PBS_ENCRYPTION_PASSWORD`` |
60 | When set, this value is used to access the secret encryption key (if | |
61 | protected by password). | |
62 | ||
65ab4ca9 TL |
63 | ``PBS_ENCRYPTION_PASSWORD_FD``, ``PBS_ENCRYPTION_PASSWORD_FILE``, ``PBS_ENCRYPTION_PASSWORD_CMD`` |
64 | Like ``PBS_ENCRYPTION_PASSWORD``, but read data from an open file descriptor, | |
65 | a file name or from the `stdout` of a command, respectively. The first | |
66 | defined environment variable from the order above is preferred. | |
99a73fad | 67 | |
5fb852af DW |
68 | ``PBS_FINGERPRINT`` |
69 | When set, this value is used to verify the server certificate (only used if | |
70 | the system CA certificates cannot validate the certificate). | |
04e24b14 DW |
71 | |
72 | ||
5fb852af | 73 | .. Note:: Passwords must be valid UTF-8 and may not contain newlines. For your |
c10a6755 | 74 | convenience, Proxmox Backup Server only uses the first line as password, so |
5fb852af | 75 | you can add arbitrary comments after the first newline. |
99a73fad DM |
76 | |
77 | ||
04e24b14 DW |
78 | Output Format |
79 | ------------- | |
80 | ||
f458e97f | 81 | .. include:: output-format.rst |
04e24b14 DW |
82 | |
83 | ||
ec1ae7e6 | 84 | .. _client_creating_backups: |
04e24b14 DW |
85 | |
86 | Creating Backups | |
87 | ---------------- | |
88 | ||
89 | This section explains how to create a backup from within the machine. This can | |
5fb852af DW |
90 | be a physical host, a virtual machine, or a container. Such backups may contain |
91 | file and image archives. There are no restrictions in this case. | |
04e24b14 | 92 | |
5fb852af DW |
93 | .. Note:: If you want to backup virtual machines or containers on Proxmox VE, |
94 | see :ref:`pve-integration`. | |
04e24b14 | 95 | |
5fb852af DW |
96 | For the following example, you need to have a backup server set up, have working |
97 | credentials, and know the repository name. | |
98 | In the following examples, we use ``backup-server:store1``. | |
04e24b14 DW |
99 | |
100 | .. code-block:: console | |
101 | ||
102 | # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1 | |
103 | Starting backup: host/elsa/2019-12-03T09:35:01Z | |
104 | Client name: elsa | |
105 | skip mount point: "/boot/efi" | |
106 | skip mount point: "/dev" | |
107 | skip mount point: "/run" | |
108 | skip mount point: "/sys" | |
109 | Uploaded 12129 chunks in 87 seconds (564 MB/s). | |
110 | End Time: 2019-12-03T10:36:29+01:00 | |
111 | ||
5fb852af | 112 | This will prompt you for a password, then upload a file archive named |
04e24b14 DW |
113 | ``root.pxar`` containing all the files in the ``/`` directory. |
114 | ||
5fb852af | 115 | .. Caution:: Please note that proxmox-backup-client does not |
04e24b14 | 116 | automatically include mount points. Instead, you will see a short |
5fb852af | 117 | ``skip mount point`` message for each of them. The idea is to |
04e24b14 DW |
118 | create a separate file archive for each mounted disk. You can |
119 | explicitly include them using the ``--include-dev`` option | |
120 | (i.e. ``--include-dev /boot/efi``). You can use this option | |
121 | multiple times for each mount point that should be included. | |
122 | ||
123 | The ``--repository`` option can get quite long and is used by all | |
124 | commands. You can avoid having to enter this value by setting the | |
5fb852af DW |
125 | environment variable ``PBS_REPOSITORY``. Note that if you would like this to |
126 | remain set over multiple sessions, you should instead add the below line to your | |
04e24b14 DW |
127 | ``.bashrc`` file. |
128 | ||
129 | .. code-block:: console | |
130 | ||
131 | # export PBS_REPOSITORY=backup-server:store1 | |
132 | ||
5fb852af DW |
133 | After this, you can execute all commands without having to specify the |
134 | ``--repository`` option. | |
04e24b14 | 135 | |
5fb852af DW |
136 | A single backup is allowed to contain more than one archive. For example, if |
137 | you want to back up two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``: | |
04e24b14 DW |
138 | |
139 | .. code-block:: console | |
140 | ||
141 | # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 | |
142 | ||
143 | This creates a backup of both disks. | |
144 | ||
145 | The backup command takes a list of backup specifications, which | |
146 | include the archive name on the server, the type of the archive, and the | |
147 | archive source at the client. The format is: | |
148 | ||
149 | <archive-name>.<type>:<source-path> | |
150 | ||
5fb852af DW |
151 | Common types are ``.pxar`` for file archives and ``.img`` for block |
152 | device images. To create a backup of a block device, run the following command: | |
04e24b14 DW |
153 | |
154 | .. code-block:: console | |
155 | ||
156 | # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata | |
157 | ||
158 | ||
5fb852af DW |
159 | Excluding Files/Directories from a Backup |
160 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
04e24b14 | 161 | |
5fb852af | 162 | Sometimes it is desired to exclude certain files or directories from a backup archive. |
04e24b14 | 163 | To tell the Proxmox Backup client when and how to ignore files and directories, |
5fb852af | 164 | place a text file named ``.pxarexclude`` in the filesystem hierarchy. |
04e24b14 | 165 | Whenever the backup client encounters such a file in a directory, it interprets |
5fb852af | 166 | each line as a glob match pattern for files and directories that are to be excluded |
04e24b14 DW |
167 | from the backup. |
168 | ||
5fb852af DW |
169 | The file must contain a single glob pattern per line. Empty lines and lines |
170 | starting with ``#`` (indicating a comment) are ignored. | |
04e24b14 DW |
171 | A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion |
172 | to an explicit inclusion. This makes it possible to exclude all entries in a | |
173 | directory except for a few single files/subdirectories. | |
174 | Lines ending in ``/`` match only on directories. | |
175 | The directory containing the ``.pxarexclude`` file is considered to be the root of | |
176 | the given patterns. It is only possible to match files in this directory and its subdirectories. | |
177 | ||
178 | ``\`` is used to escape special glob characters. | |
179 | ``?`` matches any single character. | |
180 | ``*`` matches any character, including an empty string. | |
5fb852af DW |
181 | ``**`` is used to match current directory and subdirectories. For example, with |
182 | the pattern ``**/*.tmp``, it would exclude all files ending in ``.tmp`` within | |
183 | a directory and its subdirectories. | |
04e24b14 DW |
184 | ``[...]`` matches a single character from any of the provided characters within |
185 | the brackets. ``[!...]`` does the complementary and matches any single character | |
186 | not contained within the brackets. It is also possible to specify ranges with two | |
187 | characters separated by ``-``. For example, ``[a-z]`` matches any lowercase | |
5fb852af | 188 | alphabetic character, and ``[0-9]`` matches any single digit. |
04e24b14 DW |
189 | |
190 | The order of the glob match patterns defines whether a file is included or | |
5fb852af | 191 | excluded, that is to say, later entries override earlier ones. |
04e24b14 DW |
192 | This is also true for match patterns encountered deeper down the directory tree, |
193 | which can override a previous exclusion. | |
5fb852af DW |
194 | |
195 | .. Note:: Excluded directories will **not** be read by the backup client. Thus, | |
196 | a ``.pxarexclude`` file in an excluded subdirectory will have no effect. | |
197 | ``.pxarexclude`` files are treated as regular files and will be included in | |
198 | the backup archive. | |
04e24b14 DW |
199 | |
200 | For example, consider the following directory structure: | |
201 | ||
202 | .. code-block:: console | |
203 | ||
204 | # ls -aR folder | |
205 | folder/: | |
206 | . .. .pxarexclude subfolder0 subfolder1 | |
207 | ||
208 | folder/subfolder0: | |
209 | . .. file0 file1 file2 file3 .pxarexclude | |
210 | ||
211 | folder/subfolder1: | |
212 | . .. file0 file1 file2 file3 | |
213 | ||
214 | The different ``.pxarexclude`` files contain the following: | |
215 | ||
216 | .. code-block:: console | |
217 | ||
218 | # cat folder/.pxarexclude | |
219 | /subfolder0/file1 | |
220 | /subfolder1/* | |
221 | !/subfolder1/file2 | |
222 | ||
223 | .. code-block:: console | |
224 | ||
225 | # cat folder/subfolder0/.pxarexclude | |
226 | file3 | |
227 | ||
228 | This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of | |
229 | ``subfolder1`` except ``file2``. | |
230 | ||
231 | Restoring this backup will result in: | |
232 | ||
233 | .. code-block:: console | |
234 | ||
235 | ls -aR restored | |
236 | restored/: | |
237 | . .. .pxarexclude subfolder0 subfolder1 | |
238 | ||
239 | restored/subfolder0: | |
240 | . .. file0 file2 .pxarexclude | |
241 | ||
242 | restored/subfolder1: | |
243 | . .. file2 | |
244 | ||
245 | ||
ec1ae7e6 | 246 | .. _client_encryption: |
d9f365d7 | 247 | |
04e24b14 DW |
248 | Encryption |
249 | ---------- | |
250 | ||
251 | Proxmox Backup supports client-side encryption with AES-256 in GCM_ | |
252 | mode. To set this up, you first need to create an encryption key: | |
253 | ||
254 | .. code-block:: console | |
255 | ||
256 | # proxmox-backup-client key create my-backup.key | |
257 | Encryption Key Password: ************** | |
258 | ||
259 | The key is password protected by default. If you do not need this | |
260 | extra protection, you can also create it without a password: | |
261 | ||
262 | .. code-block:: console | |
263 | ||
264 | # proxmox-backup-client key create /path/to/my-backup.key --kdf none | |
265 | ||
266 | Having created this key, it is now possible to create an encrypted backup, by | |
267 | passing the ``--keyfile`` parameter, with the path to the key file. | |
268 | ||
269 | .. code-block:: console | |
270 | ||
271 | # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key | |
272 | Password: ********* | |
273 | Encryption Key Password: ************** | |
274 | ... | |
275 | ||
276 | .. Note:: If you do not specify the name of the backup key, the key will be | |
277 | created in the default location | |
278 | ``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client`` | |
279 | will also search this location by default, in case the ``--keyfile`` | |
280 | parameter is not specified. | |
281 | ||
282 | You can avoid entering the passwords by setting the environment | |
283 | variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``. | |
284 | ||
285 | ||
5fb852af | 286 | Using a Master Key to Store and Recover Encryption Keys |
04e24b14 DW |
287 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
288 | ||
289 | You can also use ``proxmox-backup-client key`` to create an RSA public/private | |
290 | key pair, which can be used to store an encrypted version of the symmetric | |
291 | backup encryption key alongside each backup and recover it later. | |
292 | ||
293 | To set up a master key: | |
294 | ||
295 | 1. Create an encryption key for the backup: | |
296 | ||
297 | .. code-block:: console | |
298 | ||
299 | # proxmox-backup-client key create | |
300 | creating default key at: "~/.config/proxmox-backup/encryption-key.json" | |
301 | Encryption Key Password: ********** | |
302 | ... | |
303 | ||
304 | The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``. | |
305 | ||
306 | 2. Create an RSA public/private key pair: | |
307 | ||
308 | .. code-block:: console | |
309 | ||
310 | # proxmox-backup-client key create-master-key | |
311 | Master Key Password: ********* | |
312 | ... | |
313 | ||
314 | This will create two files in your current directory, ``master-public.pem`` | |
315 | and ``master-private.pem``. | |
316 | ||
317 | 3. Import the newly created ``master-public.pem`` public certificate, so that | |
318 | ``proxmox-backup-client`` can find and use it upon backup. | |
319 | ||
320 | .. code-block:: console | |
321 | ||
322 | # proxmox-backup-client key import-master-pubkey /path/to/master-public.pem | |
323 | Imported public master key to "~/.config/proxmox-backup/master-public.pem" | |
324 | ||
325 | 4. With all these files in place, run a backup job: | |
326 | ||
327 | .. code-block:: console | |
328 | ||
329 | # proxmox-backup-client backup etc.pxar:/etc | |
330 | ||
331 | The key will be stored in your backup, under the name ``rsa-encrypted.key``. | |
332 | ||
333 | .. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key | |
334 | is in the default path. If you specified another path upon creation, you | |
335 | must pass the ``--keyfile`` parameter. | |
336 | ||
337 | 5. To test that everything worked, you can restore the key from the backup: | |
338 | ||
339 | .. code-block:: console | |
340 | ||
341 | # proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target | |
342 | ||
343 | .. Note:: You should not need an encryption key to extract this file. However, if | |
344 | a key exists at the default location | |
345 | (``~/.config/proxmox-backup/encryption-key.json``) the program will prompt | |
346 | you for an encryption key password. Simply moving ``encryption-key.json`` | |
347 | out of this directory will fix this issue. | |
348 | ||
349 | 6. Then, use the previously generated master key to decrypt the file: | |
350 | ||
351 | .. code-block:: console | |
352 | ||
2ea5abcd FG |
353 | # proxmox-backup-client key import-with-master-key /path/to/target --master-keyfile /path/to/master-private.pem --encrypted-keyfile /path/to/rsa-encrypted.key |
354 | Master Key Password: ****** | |
355 | New Password: ****** | |
356 | Verify Password: ****** | |
04e24b14 DW |
357 | |
358 | 7. The target file will now contain the encryption key information in plain | |
359 | text. The success of this can be confirmed by passing the resulting ``json`` | |
360 | file, with the ``--keyfile`` parameter, when decrypting files from the backup. | |
361 | ||
362 | .. warning:: Without their key, backed up files will be inaccessible. Thus, you should | |
363 | keep keys ordered and in a place that is separate from the contents being | |
364 | backed up. It can happen, for example, that you back up an entire system, using | |
cdbc18fc | 365 | a key on that system. If the system then becomes inaccessible for any reason |
5fb852af | 366 | and needs to be restored, this will not be possible, as the encryption key will be |
4accbc58 DW |
367 | lost along with the broken system. |
368 | ||
c5608cf8 DW |
369 | It is recommended that you keep your master key safe, but easily accessible, in |
370 | order for quick disaster recovery. For this reason, the best place to store it | |
371 | is in your password manager, where it is immediately recoverable. As a backup to | |
372 | this, you should also save the key to a USB drive and store that in a secure | |
373 | place. This way, it is detached from any system, but is still easy to recover | |
374 | from, in case of emergency. Finally, in preparation for the worst case scenario, | |
375 | you should also consider keeping a paper copy of your master key locked away in | |
376 | a safe place. The ``paperkey`` subcommand can be used to create a QR encoded | |
377 | version of your master key. The following command sends the output of the | |
378 | ``paperkey`` command to a text file, for easy printing. | |
4accbc58 DW |
379 | |
380 | .. code-block:: console | |
381 | ||
382 | proxmox-backup-client key paperkey --output-format text > qrkey.txt | |
04e24b14 DW |
383 | |
384 | ||
385 | Restoring Data | |
386 | -------------- | |
387 | ||
5fb852af DW |
388 | The regular creation of backups is a necessary step in avoiding data loss. More |
389 | importantly, however, is the restoration. It is good practice to perform | |
390 | periodic recovery tests to ensure that you can access the data in case of | |
391 | disaster. | |
04e24b14 DW |
392 | |
393 | First, you need to find the snapshot which you want to restore. The snapshot | |
abf9b6da | 394 | list command provides a list of all the snapshots on the server: |
04e24b14 DW |
395 | |
396 | .. code-block:: console | |
397 | ||
abf9b6da | 398 | # proxmox-backup-client snapshot list |
04e24b14 DW |
399 | ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐ |
400 | │ snapshot │ size │ files │ | |
401 | ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡ | |
402 | │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │ | |
403 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
404 | │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │ | |
405 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
406 | ... | |
407 | ||
408 | You can inspect the catalog to find specific files. | |
409 | ||
410 | .. code-block:: console | |
411 | ||
412 | # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z | |
413 | ... | |
414 | d "./root.pxar.didx/etc/cifs-utils" | |
415 | l "./root.pxar.didx/etc/cifs-utils/idmap-plugin" | |
416 | d "./root.pxar.didx/etc/console-setup" | |
417 | ... | |
418 | ||
419 | The restore command lets you restore a single archive from the | |
420 | backup. | |
421 | ||
422 | .. code-block:: console | |
423 | ||
424 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/ | |
425 | ||
426 | To get the contents of any archive, you can restore the ``index.json`` file in the | |
427 | repository to the target path '-'. This will dump the contents to the standard output. | |
428 | ||
429 | .. code-block:: console | |
430 | ||
431 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json - | |
432 | ||
433 | ||
434 | Interactive Restores | |
435 | ~~~~~~~~~~~~~~~~~~~~ | |
436 | ||
437 | If you only want to restore a few individual files, it is often easier | |
438 | to use the interactive recovery shell. | |
439 | ||
440 | .. code-block:: console | |
441 | ||
442 | # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar | |
443 | Starting interactive shell | |
444 | pxar:/ > ls | |
445 | bin boot dev etc home lib lib32 | |
446 | ... | |
447 | ||
3bbb70b3 | 448 | The interactive recovery shell is a minimal command line interface that |
04e24b14 | 449 | utilizes the metadata stored in the catalog to quickly list, navigate and |
5fb852af | 450 | search for files in a file archive. |
04e24b14 DW |
451 | To restore files, you can select them individually or match them with a glob |
452 | pattern. | |
453 | ||
454 | Using the catalog for navigation reduces the overhead considerably because only | |
455 | the catalog needs to be downloaded and, optionally, decrypted. | |
5fb852af DW |
456 | The actual chunks are only accessed if the metadata in the catalog is |
457 | insufficient or for the actual restore. | |
04e24b14 | 458 | |
5fb852af | 459 | Similar to common UNIX shells, ``cd`` and ``ls`` are the commands used to change |
04e24b14 DW |
460 | working directory and list directory contents in the archive. |
461 | ``pwd`` shows the full path of the current working directory with respect to the | |
462 | archive root. | |
463 | ||
5fb852af DW |
464 | The ability to quickly search the contents of the archive is a commonly required |
465 | feature. That's where the catalog is most valuable. For example: | |
04e24b14 DW |
466 | |
467 | .. code-block:: console | |
468 | ||
469 | pxar:/ > find etc/**/*.txt --select | |
470 | "/etc/X11/rgb.txt" | |
471 | pxar:/ > list-selected | |
472 | etc/**/*.txt | |
473 | pxar:/ > restore-selected /target/path | |
474 | ... | |
475 | ||
5fb852af DW |
476 | This will find and print all files ending in ``.txt`` located in ``etc/`` or its |
477 | subdirectories, and add the corresponding pattern to the list for subsequent restores. | |
04e24b14 DW |
478 | ``list-selected`` shows these patterns and ``restore-selected`` finally restores |
479 | all files in the archive matching the patterns to ``/target/path`` on the local | |
480 | host. This will scan the whole archive. | |
481 | ||
985e84e3 DW |
482 | The ``restore`` command can be used to restore all the files contained within |
483 | the backup archive. This is most helpful when paired with the ``--pattern | |
484 | <glob>`` option, as it allows you to restore all files matching a specific | |
485 | pattern. For example, if you wanted to restore configuration files | |
486 | located in ``/etc``, you could do the following: | |
04e24b14 DW |
487 | |
488 | .. code-block:: console | |
489 | ||
985e84e3 | 490 | pxar:/ > restore target/ --pattern etc/**/*.conf |
04e24b14 DW |
491 | ... |
492 | ||
5f3b2330 | 493 | The above will scan through all the directories below ``/etc`` and restore all |
04e24b14 DW |
494 | files ending in ``.conf``. |
495 | ||
496 | .. todo:: Explain interactive restore in more detail | |
497 | ||
498 | Mounting of Archives via FUSE | |
499 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
500 | ||
501 | The :term:`FUSE` implementation for the pxar archive allows you to mount a | |
5fb852af | 502 | file archive as a read-only filesystem to a mount point on your host. |
04e24b14 DW |
503 | |
504 | .. code-block:: console | |
505 | ||
506 | # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint | |
507 | # ls /mnt/mountpoint | |
508 | bin dev home lib32 libx32 media opt root sbin sys usr | |
509 | boot etc lib lib64 lost+found mnt proc run srv tmp var | |
510 | ||
511 | This allows you to access the full contents of the archive in a seamless manner. | |
512 | ||
513 | .. note:: As the FUSE connection needs to fetch and decrypt chunks from the | |
514 | backup server's datastore, this can cause some additional network and CPU | |
515 | load on your host, depending on the operations you perform on the mounted | |
516 | filesystem. | |
517 | ||
5fb852af | 518 | To unmount the filesystem, use the ``umount`` command on the mount point: |
04e24b14 DW |
519 | |
520 | .. code-block:: console | |
521 | ||
522 | # umount /mnt/mountpoint | |
523 | ||
524 | Login and Logout | |
525 | ---------------- | |
526 | ||
5fb852af | 527 | The client tool prompts you to enter the login password as soon as you |
04e24b14 DW |
528 | want to access the backup server. The server checks your credentials |
529 | and responds with a ticket that is valid for two hours. The client | |
530 | tool automatically stores that ticket and uses it for further requests | |
531 | to this server. | |
532 | ||
533 | You can also manually trigger this login/logout using the login and | |
534 | logout commands: | |
535 | ||
536 | .. code-block:: console | |
537 | ||
538 | # proxmox-backup-client login | |
539 | Password: ********** | |
540 | ||
541 | To remove the ticket, issue a logout: | |
542 | ||
543 | .. code-block:: console | |
544 | ||
545 | # proxmox-backup-client logout | |
546 | ||
547 | ||
08379a21 DW |
548 | .. _changing-backup-owner: |
549 | ||
550 | Changing the Owner of a Backup Group | |
551 | ------------------------------------ | |
552 | ||
553 | By default, the owner of a backup group is the user which was used to originally | |
554 | create that backup group (or in the case of sync jobs, ``root@pam``). This | |
555 | means that if a user ``mike@pbs`` created a backup, another user ``john@pbs`` | |
5fb852af | 556 | can not be used to create backups in that same backup group. In case you want |
08379a21 DW |
557 | to change the owner of a backup, you can do so with the below command, using a |
558 | user that has ``Datastore.Modify`` privileges on the datastore. | |
559 | ||
560 | .. code-block:: console | |
561 | ||
562 | # proxmox-backup-client change-owner vm/103 john@pbs | |
563 | ||
564 | This can also be done from within the web interface, by navigating to the | |
565 | `Content` section of the datastore that contains the backup group and | |
566 | selecting the user icon under the `Actions` column. Common cases for this could | |
567 | be to change the owner of a sync job from ``root@pam``, or to repurpose a | |
568 | backup group. | |
569 | ||
570 | ||
4c3efb53 | 571 | .. _backup-pruning: |
04e24b14 DW |
572 | |
573 | Pruning and Removing Backups | |
574 | ---------------------------- | |
575 | ||
576 | You can manually delete a backup snapshot using the ``forget`` | |
577 | command: | |
578 | ||
579 | .. code-block:: console | |
580 | ||
abf9b6da | 581 | # proxmox-backup-client snapshot forget <snapshot> |
04e24b14 DW |
582 | |
583 | ||
584 | .. caution:: This command removes all archives in this backup | |
585 | snapshot. They will be inaccessible and unrecoverable. | |
586 | ||
587 | ||
588 | Although manual removal is sometimes required, the ``prune`` | |
589 | command is normally used to systematically delete older backups. Prune lets | |
590 | you specify which backup snapshots you want to keep. The | |
591 | following retention options are available: | |
592 | ||
593 | ``--keep-last <N>`` | |
594 | Keep the last ``<N>`` backup snapshots. | |
595 | ||
596 | ``--keep-hourly <N>`` | |
597 | Keep backups for the last ``<N>`` hours. If there is more than one | |
598 | backup for a single hour, only the latest is kept. | |
599 | ||
600 | ``--keep-daily <N>`` | |
601 | Keep backups for the last ``<N>`` days. If there is more than one | |
602 | backup for a single day, only the latest is kept. | |
603 | ||
604 | ``--keep-weekly <N>`` | |
605 | Keep backups for the last ``<N>`` weeks. If there is more than one | |
606 | backup for a single week, only the latest is kept. | |
607 | ||
608 | .. note:: Weeks start on Monday and end on Sunday. The software | |
609 | uses the `ISO week date`_ system and handles weeks at | |
610 | the end of the year correctly. | |
611 | ||
612 | ``--keep-monthly <N>`` | |
613 | Keep backups for the last ``<N>`` months. If there is more than one | |
614 | backup for a single month, only the latest is kept. | |
615 | ||
616 | ``--keep-yearly <N>`` | |
617 | Keep backups for the last ``<N>`` years. If there is more than one | |
618 | backup for a single year, only the latest is kept. | |
619 | ||
620 | The retention options are processed in the order given above. Each option | |
621 | only covers backups within its time period. The next option does not take care | |
622 | of already covered backups. It will only consider older backups. | |
623 | ||
624 | Unfinished and incomplete backups will be removed by the prune command unless | |
625 | they are newer than the last successful backup. In this case, the last failed | |
626 | backup is retained. | |
627 | ||
628 | .. code-block:: console | |
629 | ||
630 | # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3 | |
631 | ||
632 | ||
633 | You can use the ``--dry-run`` option to test your settings. This only | |
634 | shows the list of existing snapshots and what actions prune would take. | |
635 | ||
636 | .. code-block:: console | |
637 | ||
638 | # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3 | |
639 | ┌────────────────────────────────┬──────┐ | |
640 | │ snapshot │ keep │ | |
641 | ╞════════════════════════════════╪══════╡ | |
642 | │ host/elsa/2019-12-04T13:20:37Z │ 1 │ | |
643 | ├────────────────────────────────┼──────┤ | |
644 | │ host/elsa/2019-12-03T09:35:01Z │ 0 │ | |
645 | ├────────────────────────────────┼──────┤ | |
646 | │ host/elsa/2019-11-22T11:54:47Z │ 1 │ | |
647 | ├────────────────────────────────┼──────┤ | |
648 | │ host/elsa/2019-11-21T12:36:25Z │ 0 │ | |
649 | ├────────────────────────────────┼──────┤ | |
650 | │ host/elsa/2019-11-10T10:42:20Z │ 1 │ | |
651 | └────────────────────────────────┴──────┘ | |
652 | ||
653 | .. note:: Neither the ``prune`` command nor the ``forget`` command free space | |
654 | in the chunk-store. The chunk-store still contains the data blocks. To free | |
a98e2287 | 655 | space you need to perform :ref:`client_garbage-collection`. |
04e24b14 | 656 | |
38517ca0 DC |
657 | It is also possible to protect single snapshots from being pruned or deleted: |
658 | ||
659 | .. code-block:: console | |
660 | ||
661 | # proxmox-backup-client snapshot protected update <snapshot> true | |
662 | ||
663 | This will set the protected flag on the snapshot and prevent pruning or manual | |
664 | deletion of this snapshot untilt he flag is removed again with: | |
665 | ||
666 | .. code-block:: console | |
667 | ||
668 | # proxmox-backup-client snapshot protected update <snapshot> false | |
669 | ||
670 | When a group is with a protected snapshot is deleted, only the non-protected | |
671 | ones are removed and the group will remain. | |
672 | ||
673 | .. note:: This flag will not be synced when using pull or sync jobs. If you | |
674 | want to protect a synced snapshot, you have to manually to this again on | |
56d3b59c | 675 | the target backup server. |
04e24b14 | 676 | |
ec1ae7e6 | 677 | .. _client_garbage-collection: |
04e24b14 DW |
678 | |
679 | Garbage Collection | |
680 | ------------------ | |
681 | ||
682 | The ``prune`` command removes only the backup index files, not the data | |
683 | from the datastore. This task is left to the garbage collection | |
684 | command. It is recommended to carry out garbage collection on a regular basis. | |
685 | ||
686 | The garbage collection works in two phases. In the first phase, all | |
687 | data blocks that are still in use are marked. In the second phase, | |
688 | unused data blocks are removed. | |
689 | ||
690 | .. note:: This command needs to read all existing backup index files | |
691 | and touches the complete chunk-store. This can take a long time | |
692 | depending on the number of chunks and the speed of the underlying | |
693 | disks. | |
694 | ||
695 | .. note:: The garbage collection will only remove chunks that haven't been used | |
696 | for at least one day (exactly 24h 5m). This grace period is necessary because | |
697 | chunks in use are marked by touching the chunk which updates the ``atime`` | |
698 | (access time) property. Filesystems are mounted with the ``relatime`` option | |
699 | by default. This results in a better performance by only updating the | |
700 | ``atime`` property if the last access has been at least 24 hours ago. The | |
5fb852af | 701 | downside is that touching a chunk within these 24 hours will not always |
04e24b14 DW |
702 | update its ``atime`` property. |
703 | ||
704 | Chunks in the grace period will be logged at the end of the garbage | |
705 | collection task as *Pending removals*. | |
706 | ||
707 | .. code-block:: console | |
708 | ||
709 | # proxmox-backup-client garbage-collect | |
710 | starting garbage collection on store store2 | |
711 | Start GC phase1 (mark used chunks) | |
712 | Start GC phase2 (sweep unused chunks) | |
713 | percentage done: 1, chunk count: 219 | |
714 | percentage done: 2, chunk count: 453 | |
715 | ... | |
716 | percentage done: 99, chunk count: 21188 | |
717 | Removed bytes: 411368505 | |
718 | Removed chunks: 203 | |
719 | Original data bytes: 327160886391 | |
720 | Disk bytes: 52767414743 (16 %) | |
721 | Disk chunks: 21221 | |
722 | Average chunk size: 2486565 | |
723 | TASK OK | |
724 | ||
5fb852af DW |
725 | Garbage collection can also be scheduled using ``promxox-backup-manager`` or |
726 | from the Proxmox Backup Server's web interface. | |
04e24b14 DW |
727 | |
728 | Benchmarking | |
729 | ------------ | |
730 | ||
731 | The backup client also comes with a benchmarking tool. This tool measures | |
654c56e0 TL |
732 | various metrics relating to compression and encryption speeds. If a Proxmox |
733 | Backup repository (remote or local) is specified, the TLS upload speed will get | |
734 | measured too. | |
735 | ||
736 | You can run a benchmark using the ``benchmark`` subcommand of | |
737 | ``proxmox-backup-client``: | |
738 | ||
739 | .. note:: The TLS speed test is only included if a :ref:`backup server | |
740 | repository is specified <client_repository>`. | |
04e24b14 DW |
741 | |
742 | .. code-block:: console | |
743 | ||
744 | # proxmox-backup-client benchmark | |
81f5d03e DM |
745 | Uploaded 1517 chunks in 5 seconds. |
746 | Time per request: 3309 microseconds. | |
747 | TLS speed: 1267.41 MB/s | |
748 | SHA256 speed: 2066.73 MB/s | |
749 | Compression speed: 775.11 MB/s | |
750 | Decompress speed: 1233.35 MB/s | |
751 | AES256/GCM speed: 3688.27 MB/s | |
752 | Verify speed: 783.43 MB/s | |
04e24b14 DW |
753 | ┌───────────────────────────────────┬─────────────────────┐ |
754 | │ Name │ Value │ | |
755 | ╞═══════════════════════════════════╪═════════════════════╡ | |
81f5d03e | 756 | │ TLS (maximal backup upload speed) │ 1267.41 MB/s (103%) │ |
04e24b14 | 757 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 758 | │ SHA256 checksum computation speed │ 2066.73 MB/s (102%) │ |
04e24b14 | 759 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 760 | │ ZStd level 1 compression speed │ 775.11 MB/s (103%) │ |
04e24b14 | 761 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 762 | │ ZStd level 1 decompression speed │ 1233.35 MB/s (103%) │ |
04e24b14 | 763 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e DM |
764 | │ Chunk verification speed │ 783.43 MB/s (103%) │ |
765 | ├───────────────────────────────────┼─────────────────────┤ | |
766 | │ AES256 GCM encryption speed │ 3688.27 MB/s (101%) │ | |
04e24b14 DW |
767 | └───────────────────────────────────┴─────────────────────┘ |
768 | ||
81f5d03e | 769 | |
04e24b14 | 770 | .. note:: The percentages given in the output table correspond to a |
654c56e0 | 771 | comparison against a Ryzen 7 2700X. |
04e24b14 DW |
772 | |
773 | You can also pass the ``--output-format`` parameter to output stats in ``json``, | |
774 | rather than the default table format. |