]>
Commit | Line | Data |
---|---|---|
04e24b14 DW |
1 | Backup Client Usage |
2 | =================== | |
3 | ||
34407477 | 4 | The command-line client for `Proxmox Backup`_ Server is called |
5fb852af | 5 | :command:`proxmox-backup-client`. |
04e24b14 | 6 | |
654c56e0 | 7 | .. _client_repository: |
04e24b14 | 8 | |
fd8b00ae TL |
9 | Backup Repository Locations |
10 | --------------------------- | |
04e24b14 | 11 | |
5fb852af DW |
12 | The client uses the following format to specify a datastore repository |
13 | on the backup server (where username is specified in the form of user@realm): | |
04e24b14 DW |
14 | |
15 | [[username@]server[:port]:]datastore | |
16 | ||
4aef06f1 | 17 | The default value for ``username`` is ``root@pam``. If no server is specified, |
04e24b14 DW |
18 | the default is the local host (``localhost``). |
19 | ||
5fb852af DW |
20 | You can specify a port if your backup server is only reachable on a non-default |
21 | port (for example, with NAT and port forwarding configurations). | |
04e24b14 | 22 | |
5fb852af | 23 | Note that if the server uses an IPv6 address, you have to write it with square |
4aef06f1 | 24 | brackets (for example, `[fe80::01]`). |
04e24b14 | 25 | |
8c83b75a | 26 | You can pass the repository with the ``--repository`` command-line option, or |
4aef06f1 | 27 | by setting the ``PBS_REPOSITORY`` environment variable. |
04e24b14 | 28 | |
ca8cf671 DC |
29 | The web interface provides copyable repository text in the datastore summary |
30 | with the `Show Connection Information` button. | |
31 | ||
5fb852af DW |
32 | Below are some examples of valid repositories and their corresponding real |
33 | values: | |
04e24b14 | 34 | |
c0af05e1 | 35 | ================================ ================== ================== =========== |
4aef06f1 | 36 | Example User Host:Port Datastore |
c0af05e1 | 37 | ================================ ================== ================== =========== |
4aef06f1 TL |
38 | mydatastore ``root@pam`` localhost:8007 mydatastore |
39 | myhostname:mydatastore ``root@pam`` myhostname:8007 mydatastore | |
40 | user@pbs@myhostname:mydatastore ``user@pbs`` myhostname:8007 mydatastore | |
4f04ecb2 | 41 | user\@pbs!token@host:store ``user@pbs!token`` host:8007 store |
4aef06f1 TL |
42 | 192.168.55.55:1234:mydatastore ``root@pam`` 192.168.55.55:1234 mydatastore |
43 | [ff80::51]:mydatastore ``root@pam`` [ff80::51]:8007 mydatastore | |
44 | [ff80::51]:1234:mydatastore ``root@pam`` [ff80::51]:1234 mydatastore | |
c0af05e1 | 45 | ================================ ================== ================== =========== |
04e24b14 DW |
46 | |
47 | Environment Variables | |
48 | --------------------- | |
49 | ||
50 | ``PBS_REPOSITORY`` | |
51 | The default backup repository. | |
52 | ||
53 | ``PBS_PASSWORD`` | |
5fb852af DW |
54 | When set, this value is used as the password for the backup server. |
55 | You can also set this to an API token secret. | |
04e24b14 | 56 | |
65ab4ca9 TL |
57 | ``PBS_PASSWORD_FD``, ``PBS_PASSWORD_FILE``, ``PBS_PASSWORD_CMD`` |
58 | Like ``PBS_PASSWORD``, but read data from an open file descriptor, a file | |
59 | name or from the `stdout` of a command, respectively. The first defined | |
60 | environment variable from the order above is preferred. | |
99a73fad | 61 | |
04e24b14 DW |
62 | ``PBS_ENCRYPTION_PASSWORD`` |
63 | When set, this value is used to access the secret encryption key (if | |
64 | protected by password). | |
65 | ||
65ab4ca9 TL |
66 | ``PBS_ENCRYPTION_PASSWORD_FD``, ``PBS_ENCRYPTION_PASSWORD_FILE``, ``PBS_ENCRYPTION_PASSWORD_CMD`` |
67 | Like ``PBS_ENCRYPTION_PASSWORD``, but read data from an open file descriptor, | |
68 | a file name or from the `stdout` of a command, respectively. The first | |
69 | defined environment variable from the order above is preferred. | |
99a73fad | 70 | |
5fb852af DW |
71 | ``PBS_FINGERPRINT`` |
72 | When set, this value is used to verify the server certificate (only used if | |
73 | the system CA certificates cannot validate the certificate). | |
04e24b14 | 74 | |
fc65ec43 SH |
75 | ``ALL_PROXY`` |
76 | When set, the client uses the specified HTTP proxy for all connections to the | |
77 | backup server. Currently only HTTP proxies are supported. Valid proxy | |
78 | configurations have the following format: | |
79 | `[http://][user:password@]<host>[:port]`. Default `port` is 1080, if not | |
80 | otherwise specified. | |
81 | ||
82 | ||
83 | .. Note:: The recommended solution for shielding hosts is using tunnels such as | |
84 | wireguard, instead of using an HTTP proxy. | |
85 | ||
04e24b14 | 86 | |
5fb852af | 87 | .. Note:: Passwords must be valid UTF-8 and may not contain newlines. For your |
c10a6755 | 88 | convenience, Proxmox Backup Server only uses the first line as password, so |
5fb852af | 89 | you can add arbitrary comments after the first newline. |
99a73fad DM |
90 | |
91 | ||
04e24b14 DW |
92 | Output Format |
93 | ------------- | |
94 | ||
f458e97f | 95 | .. include:: output-format.rst |
04e24b14 DW |
96 | |
97 | ||
ec1ae7e6 | 98 | .. _client_creating_backups: |
04e24b14 DW |
99 | |
100 | Creating Backups | |
101 | ---------------- | |
102 | ||
103 | This section explains how to create a backup from within the machine. This can | |
5fb852af DW |
104 | be a physical host, a virtual machine, or a container. Such backups may contain |
105 | file and image archives. There are no restrictions in this case. | |
04e24b14 | 106 | |
5fb852af DW |
107 | .. Note:: If you want to backup virtual machines or containers on Proxmox VE, |
108 | see :ref:`pve-integration`. | |
04e24b14 | 109 | |
5fb852af DW |
110 | For the following example, you need to have a backup server set up, have working |
111 | credentials, and know the repository name. | |
112 | In the following examples, we use ``backup-server:store1``. | |
04e24b14 DW |
113 | |
114 | .. code-block:: console | |
115 | ||
116 | # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1 | |
117 | Starting backup: host/elsa/2019-12-03T09:35:01Z | |
118 | Client name: elsa | |
119 | skip mount point: "/boot/efi" | |
120 | skip mount point: "/dev" | |
121 | skip mount point: "/run" | |
122 | skip mount point: "/sys" | |
123 | Uploaded 12129 chunks in 87 seconds (564 MB/s). | |
124 | End Time: 2019-12-03T10:36:29+01:00 | |
125 | ||
5fb852af | 126 | This will prompt you for a password, then upload a file archive named |
04e24b14 DW |
127 | ``root.pxar`` containing all the files in the ``/`` directory. |
128 | ||
5fb852af | 129 | .. Caution:: Please note that proxmox-backup-client does not |
04e24b14 | 130 | automatically include mount points. Instead, you will see a short |
5fb852af | 131 | ``skip mount point`` message for each of them. The idea is to |
04e24b14 DW |
132 | create a separate file archive for each mounted disk. You can |
133 | explicitly include them using the ``--include-dev`` option | |
134 | (i.e. ``--include-dev /boot/efi``). You can use this option | |
135 | multiple times for each mount point that should be included. | |
136 | ||
a3b10267 TL |
137 | The ``--repository`` option can get quite long and is used by all commands. You |
138 | can avoid having to enter this value by setting the environment variable | |
139 | ``PBS_REPOSITORY``. Note that if you would like this to remain set over | |
140 | multiple sessions, you should instead add the below line to your ``.bashrc`` | |
141 | file. | |
04e24b14 DW |
142 | |
143 | .. code-block:: console | |
144 | ||
145 | # export PBS_REPOSITORY=backup-server:store1 | |
146 | ||
5fb852af DW |
147 | After this, you can execute all commands without having to specify the |
148 | ``--repository`` option. | |
04e24b14 | 149 | |
5fb852af DW |
150 | A single backup is allowed to contain more than one archive. For example, if |
151 | you want to back up two disks mounted at ``/mnt/disk1`` and ``/mnt/disk2``: | |
04e24b14 DW |
152 | |
153 | .. code-block:: console | |
154 | ||
155 | # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 | |
156 | ||
157 | This creates a backup of both disks. | |
158 | ||
6481fd24 | 159 | If you want to use a namespace for the backup target, you can add the `--ns` |
f71a4ce6 TL |
160 | parameter: |
161 | ||
162 | .. code-block:: console | |
163 | ||
164 | # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2 --ns a/b/c | |
165 | ||
166 | The backup command takes a list of backup specifications, which include the | |
167 | archive name on the server, the type of the archive, and the archive source at | |
168 | the client. The format is: | |
04e24b14 DW |
169 | |
170 | <archive-name>.<type>:<source-path> | |
171 | ||
5fb852af DW |
172 | Common types are ``.pxar`` for file archives and ``.img`` for block |
173 | device images. To create a backup of a block device, run the following command: | |
04e24b14 DW |
174 | |
175 | .. code-block:: console | |
176 | ||
177 | # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata | |
178 | ||
179 | ||
5fb852af DW |
180 | Excluding Files/Directories from a Backup |
181 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
04e24b14 | 182 | |
a3b10267 TL |
183 | Sometimes it is desired to exclude certain files or directories from a backup |
184 | archive. To tell the Proxmox Backup client when and how to ignore files and | |
185 | directories, place a text file named ``.pxarexclude`` in the filesystem | |
186 | hierarchy. Whenever the backup client encounters such a file in a directory, | |
187 | it interprets each line as a glob match pattern for files and directories that | |
188 | are to be excluded from the backup. | |
04e24b14 | 189 | |
5fb852af DW |
190 | The file must contain a single glob pattern per line. Empty lines and lines |
191 | starting with ``#`` (indicating a comment) are ignored. | |
a3b10267 TL |
192 | A ``!`` at the beginning of a line reverses the glob match pattern from an |
193 | exclusion to an explicit inclusion. This makes it possible to exclude all | |
194 | entries in a directory except for a few single files/subdirectories. | |
04e24b14 | 195 | Lines ending in ``/`` match only on directories. |
a3b10267 TL |
196 | The directory containing the ``.pxarexclude`` file is considered to be the root |
197 | of the given patterns. It is only possible to match files in this directory and | |
198 | its subdirectories. | |
04e24b14 | 199 | |
b2df21bb FE |
200 | .. Note:: Patterns without a leading ``/`` will also match in subdirectories, |
201 | while patterns with a leading ``/`` will only match in the current directory. | |
202 | ||
04e24b14 DW |
203 | ``\`` is used to escape special glob characters. |
204 | ``?`` matches any single character. | |
205 | ``*`` matches any character, including an empty string. | |
5fb852af DW |
206 | ``**`` is used to match current directory and subdirectories. For example, with |
207 | the pattern ``**/*.tmp``, it would exclude all files ending in ``.tmp`` within | |
208 | a directory and its subdirectories. | |
04e24b14 | 209 | ``[...]`` matches a single character from any of the provided characters within |
a3b10267 TL |
210 | the brackets. ``[!...]`` does the complementary and matches any single |
211 | character not contained within the brackets. It is also possible to specify | |
212 | ranges with two characters separated by ``-``. For example, ``[a-z]`` matches | |
213 | any lowercase alphabetic character, and ``[0-9]`` matches any single digit. | |
04e24b14 DW |
214 | |
215 | The order of the glob match patterns defines whether a file is included or | |
5fb852af | 216 | excluded, that is to say, later entries override earlier ones. |
a3b10267 TL |
217 | This is also true for match patterns encountered deeper down the directory |
218 | tree, which can override a previous exclusion. | |
5fb852af DW |
219 | |
220 | .. Note:: Excluded directories will **not** be read by the backup client. Thus, | |
221 | a ``.pxarexclude`` file in an excluded subdirectory will have no effect. | |
222 | ``.pxarexclude`` files are treated as regular files and will be included in | |
223 | the backup archive. | |
04e24b14 DW |
224 | |
225 | For example, consider the following directory structure: | |
226 | ||
227 | .. code-block:: console | |
228 | ||
229 | # ls -aR folder | |
230 | folder/: | |
231 | . .. .pxarexclude subfolder0 subfolder1 | |
232 | ||
233 | folder/subfolder0: | |
234 | . .. file0 file1 file2 file3 .pxarexclude | |
235 | ||
236 | folder/subfolder1: | |
237 | . .. file0 file1 file2 file3 | |
238 | ||
239 | The different ``.pxarexclude`` files contain the following: | |
240 | ||
241 | .. code-block:: console | |
242 | ||
243 | # cat folder/.pxarexclude | |
244 | /subfolder0/file1 | |
245 | /subfolder1/* | |
246 | !/subfolder1/file2 | |
247 | ||
248 | .. code-block:: console | |
249 | ||
250 | # cat folder/subfolder0/.pxarexclude | |
251 | file3 | |
252 | ||
253 | This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of | |
254 | ``subfolder1`` except ``file2``. | |
255 | ||
256 | Restoring this backup will result in: | |
257 | ||
258 | .. code-block:: console | |
259 | ||
bc9b8990 | 260 | # ls -aR restored |
04e24b14 DW |
261 | restored/: |
262 | . .. .pxarexclude subfolder0 subfolder1 | |
263 | ||
264 | restored/subfolder0: | |
265 | . .. file0 file2 .pxarexclude | |
266 | ||
267 | restored/subfolder1: | |
268 | . .. file2 | |
269 | ||
93f2568f GG |
270 | The same syntax can also be used directly in the cli with the ``--exclude`` |
271 | parameter. For example: | |
272 | ||
273 | .. code-block:: console | |
274 | ||
275 | # proxmox-backup-client backup.pxar:./linux --exclude /usr | |
276 | ||
277 | Multiple paths can be excluded like this: | |
278 | ||
279 | .. code-block:: console | |
280 | ||
281 | # proxmox-backup-client backup.pxar:./linux --exclude=/usr --exclude=/rust | |
04e24b14 | 282 | |
ec1ae7e6 | 283 | .. _client_encryption: |
d9f365d7 | 284 | |
04e24b14 DW |
285 | Encryption |
286 | ---------- | |
287 | ||
288 | Proxmox Backup supports client-side encryption with AES-256 in GCM_ | |
289 | mode. To set this up, you first need to create an encryption key: | |
290 | ||
291 | .. code-block:: console | |
292 | ||
293 | # proxmox-backup-client key create my-backup.key | |
294 | Encryption Key Password: ************** | |
295 | ||
296 | The key is password protected by default. If you do not need this | |
297 | extra protection, you can also create it without a password: | |
298 | ||
299 | .. code-block:: console | |
300 | ||
301 | # proxmox-backup-client key create /path/to/my-backup.key --kdf none | |
302 | ||
303 | Having created this key, it is now possible to create an encrypted backup, by | |
304 | passing the ``--keyfile`` parameter, with the path to the key file. | |
305 | ||
306 | .. code-block:: console | |
307 | ||
308 | # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key | |
309 | Password: ********* | |
310 | Encryption Key Password: ************** | |
311 | ... | |
312 | ||
313 | .. Note:: If you do not specify the name of the backup key, the key will be | |
314 | created in the default location | |
315 | ``~/.config/proxmox-backup/encryption-key.json``. ``proxmox-backup-client`` | |
316 | will also search this location by default, in case the ``--keyfile`` | |
317 | parameter is not specified. | |
318 | ||
319 | You can avoid entering the passwords by setting the environment | |
320 | variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``. | |
321 | ||
322 | ||
5fb852af | 323 | Using a Master Key to Store and Recover Encryption Keys |
04e24b14 DW |
324 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
325 | ||
326 | You can also use ``proxmox-backup-client key`` to create an RSA public/private | |
327 | key pair, which can be used to store an encrypted version of the symmetric | |
328 | backup encryption key alongside each backup and recover it later. | |
329 | ||
330 | To set up a master key: | |
331 | ||
332 | 1. Create an encryption key for the backup: | |
333 | ||
334 | .. code-block:: console | |
335 | ||
336 | # proxmox-backup-client key create | |
337 | creating default key at: "~/.config/proxmox-backup/encryption-key.json" | |
338 | Encryption Key Password: ********** | |
339 | ... | |
340 | ||
341 | The resulting file will be saved to ``~/.config/proxmox-backup/encryption-key.json``. | |
342 | ||
343 | 2. Create an RSA public/private key pair: | |
344 | ||
345 | .. code-block:: console | |
346 | ||
347 | # proxmox-backup-client key create-master-key | |
348 | Master Key Password: ********* | |
349 | ... | |
350 | ||
351 | This will create two files in your current directory, ``master-public.pem`` | |
352 | and ``master-private.pem``. | |
353 | ||
354 | 3. Import the newly created ``master-public.pem`` public certificate, so that | |
355 | ``proxmox-backup-client`` can find and use it upon backup. | |
356 | ||
357 | .. code-block:: console | |
358 | ||
359 | # proxmox-backup-client key import-master-pubkey /path/to/master-public.pem | |
360 | Imported public master key to "~/.config/proxmox-backup/master-public.pem" | |
361 | ||
362 | 4. With all these files in place, run a backup job: | |
363 | ||
364 | .. code-block:: console | |
365 | ||
366 | # proxmox-backup-client backup etc.pxar:/etc | |
367 | ||
368 | The key will be stored in your backup, under the name ``rsa-encrypted.key``. | |
369 | ||
370 | .. Note:: The ``--keyfile`` parameter can be excluded, if the encryption key | |
371 | is in the default path. If you specified another path upon creation, you | |
372 | must pass the ``--keyfile`` parameter. | |
373 | ||
374 | 5. To test that everything worked, you can restore the key from the backup: | |
375 | ||
376 | .. code-block:: console | |
377 | ||
378 | # proxmox-backup-client restore /path/to/backup/ rsa-encrypted.key /path/to/target | |
379 | ||
380 | .. Note:: You should not need an encryption key to extract this file. However, if | |
381 | a key exists at the default location | |
382 | (``~/.config/proxmox-backup/encryption-key.json``) the program will prompt | |
383 | you for an encryption key password. Simply moving ``encryption-key.json`` | |
384 | out of this directory will fix this issue. | |
385 | ||
386 | 6. Then, use the previously generated master key to decrypt the file: | |
387 | ||
388 | .. code-block:: console | |
389 | ||
2ea5abcd FG |
390 | # proxmox-backup-client key import-with-master-key /path/to/target --master-keyfile /path/to/master-private.pem --encrypted-keyfile /path/to/rsa-encrypted.key |
391 | Master Key Password: ****** | |
392 | New Password: ****** | |
393 | Verify Password: ****** | |
04e24b14 DW |
394 | |
395 | 7. The target file will now contain the encryption key information in plain | |
396 | text. The success of this can be confirmed by passing the resulting ``json`` | |
397 | file, with the ``--keyfile`` parameter, when decrypting files from the backup. | |
398 | ||
399 | .. warning:: Without their key, backed up files will be inaccessible. Thus, you should | |
400 | keep keys ordered and in a place that is separate from the contents being | |
401 | backed up. It can happen, for example, that you back up an entire system, using | |
cdbc18fc | 402 | a key on that system. If the system then becomes inaccessible for any reason |
5fb852af | 403 | and needs to be restored, this will not be possible, as the encryption key will be |
4accbc58 DW |
404 | lost along with the broken system. |
405 | ||
c5608cf8 DW |
406 | It is recommended that you keep your master key safe, but easily accessible, in |
407 | order for quick disaster recovery. For this reason, the best place to store it | |
408 | is in your password manager, where it is immediately recoverable. As a backup to | |
6aa3eeb5 | 409 | this, you should also save the key to a USB flash drive and store that in a secure |
c5608cf8 DW |
410 | place. This way, it is detached from any system, but is still easy to recover |
411 | from, in case of emergency. Finally, in preparation for the worst case scenario, | |
412 | you should also consider keeping a paper copy of your master key locked away in | |
413 | a safe place. The ``paperkey`` subcommand can be used to create a QR encoded | |
414 | version of your master key. The following command sends the output of the | |
415 | ``paperkey`` command to a text file, for easy printing. | |
4accbc58 DW |
416 | |
417 | .. code-block:: console | |
418 | ||
419 | proxmox-backup-client key paperkey --output-format text > qrkey.txt | |
04e24b14 DW |
420 | |
421 | ||
422 | Restoring Data | |
423 | -------------- | |
424 | ||
5fb852af DW |
425 | The regular creation of backups is a necessary step in avoiding data loss. More |
426 | importantly, however, is the restoration. It is good practice to perform | |
427 | periodic recovery tests to ensure that you can access the data in case of | |
428 | disaster. | |
04e24b14 DW |
429 | |
430 | First, you need to find the snapshot which you want to restore. The snapshot | |
abf9b6da | 431 | list command provides a list of all the snapshots on the server: |
04e24b14 DW |
432 | |
433 | .. code-block:: console | |
434 | ||
abf9b6da | 435 | # proxmox-backup-client snapshot list |
04e24b14 DW |
436 | ┌────────────────────────────────┬─────────────┬────────────────────────────────────┐ |
437 | │ snapshot │ size │ files │ | |
438 | ╞════════════════════════════════╪═════════════╪════════════════════════════════════╡ | |
439 | │ host/elsa/2019-12-03T09:30:15Z │ 51788646825 │ root.pxar catalog.pcat1 index.json │ | |
440 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
441 | │ host/elsa/2019-12-03T09:35:01Z │ 51790622048 │ root.pxar catalog.pcat1 index.json │ | |
442 | ├────────────────────────────────┼─────────────┼────────────────────────────────────┤ | |
443 | ... | |
444 | ||
f71a4ce6 TL |
445 | |
446 | .. tip:: List will by default only output the backup snapshots of the root | |
447 | namespace itself. To list backups from another namespace use the ``--ns | |
448 | <ns>`` option | |
449 | ||
04e24b14 DW |
450 | You can inspect the catalog to find specific files. |
451 | ||
452 | .. code-block:: console | |
453 | ||
454 | # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z | |
455 | ... | |
456 | d "./root.pxar.didx/etc/cifs-utils" | |
457 | l "./root.pxar.didx/etc/cifs-utils/idmap-plugin" | |
458 | d "./root.pxar.didx/etc/console-setup" | |
459 | ... | |
460 | ||
461 | The restore command lets you restore a single archive from the | |
462 | backup. | |
463 | ||
464 | .. code-block:: console | |
465 | ||
466 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/ | |
467 | ||
468 | To get the contents of any archive, you can restore the ``index.json`` file in the | |
469 | repository to the target path '-'. This will dump the contents to the standard output. | |
470 | ||
471 | .. code-block:: console | |
472 | ||
473 | # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json - | |
474 | ||
475 | ||
476 | Interactive Restores | |
477 | ~~~~~~~~~~~~~~~~~~~~ | |
478 | ||
479 | If you only want to restore a few individual files, it is often easier | |
480 | to use the interactive recovery shell. | |
481 | ||
482 | .. code-block:: console | |
483 | ||
484 | # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar | |
485 | Starting interactive shell | |
486 | pxar:/ > ls | |
487 | bin boot dev etc home lib lib32 | |
488 | ... | |
489 | ||
8c83b75a | 490 | The interactive recovery shell is a minimal command-line interface that |
04e24b14 | 491 | utilizes the metadata stored in the catalog to quickly list, navigate and |
5fb852af | 492 | search for files in a file archive. |
04e24b14 DW |
493 | To restore files, you can select them individually or match them with a glob |
494 | pattern. | |
495 | ||
496 | Using the catalog for navigation reduces the overhead considerably because only | |
497 | the catalog needs to be downloaded and, optionally, decrypted. | |
5fb852af DW |
498 | The actual chunks are only accessed if the metadata in the catalog is |
499 | insufficient or for the actual restore. | |
04e24b14 | 500 | |
5fb852af | 501 | Similar to common UNIX shells, ``cd`` and ``ls`` are the commands used to change |
04e24b14 DW |
502 | working directory and list directory contents in the archive. |
503 | ``pwd`` shows the full path of the current working directory with respect to the | |
504 | archive root. | |
505 | ||
5fb852af DW |
506 | The ability to quickly search the contents of the archive is a commonly required |
507 | feature. That's where the catalog is most valuable. For example: | |
04e24b14 DW |
508 | |
509 | .. code-block:: console | |
510 | ||
511 | pxar:/ > find etc/**/*.txt --select | |
512 | "/etc/X11/rgb.txt" | |
513 | pxar:/ > list-selected | |
514 | etc/**/*.txt | |
515 | pxar:/ > restore-selected /target/path | |
516 | ... | |
517 | ||
5fb852af DW |
518 | This will find and print all files ending in ``.txt`` located in ``etc/`` or its |
519 | subdirectories, and add the corresponding pattern to the list for subsequent restores. | |
04e24b14 DW |
520 | ``list-selected`` shows these patterns and ``restore-selected`` finally restores |
521 | all files in the archive matching the patterns to ``/target/path`` on the local | |
522 | host. This will scan the whole archive. | |
523 | ||
985e84e3 DW |
524 | The ``restore`` command can be used to restore all the files contained within |
525 | the backup archive. This is most helpful when paired with the ``--pattern | |
526 | <glob>`` option, as it allows you to restore all files matching a specific | |
527 | pattern. For example, if you wanted to restore configuration files | |
528 | located in ``/etc``, you could do the following: | |
04e24b14 DW |
529 | |
530 | .. code-block:: console | |
531 | ||
985e84e3 | 532 | pxar:/ > restore target/ --pattern etc/**/*.conf |
04e24b14 DW |
533 | ... |
534 | ||
5f3b2330 | 535 | The above will scan through all the directories below ``/etc`` and restore all |
04e24b14 DW |
536 | files ending in ``.conf``. |
537 | ||
538 | .. todo:: Explain interactive restore in more detail | |
539 | ||
540 | Mounting of Archives via FUSE | |
541 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
542 | ||
543 | The :term:`FUSE` implementation for the pxar archive allows you to mount a | |
5fb852af | 544 | file archive as a read-only filesystem to a mount point on your host. |
04e24b14 DW |
545 | |
546 | .. code-block:: console | |
547 | ||
548 | # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt/mountpoint | |
549 | # ls /mnt/mountpoint | |
550 | bin dev home lib32 libx32 media opt root sbin sys usr | |
551 | boot etc lib lib64 lost+found mnt proc run srv tmp var | |
552 | ||
553 | This allows you to access the full contents of the archive in a seamless manner. | |
554 | ||
555 | .. note:: As the FUSE connection needs to fetch and decrypt chunks from the | |
556 | backup server's datastore, this can cause some additional network and CPU | |
557 | load on your host, depending on the operations you perform on the mounted | |
558 | filesystem. | |
559 | ||
5fb852af | 560 | To unmount the filesystem, use the ``umount`` command on the mount point: |
04e24b14 DW |
561 | |
562 | .. code-block:: console | |
563 | ||
564 | # umount /mnt/mountpoint | |
565 | ||
566 | Login and Logout | |
567 | ---------------- | |
568 | ||
5fb852af | 569 | The client tool prompts you to enter the login password as soon as you |
04e24b14 DW |
570 | want to access the backup server. The server checks your credentials |
571 | and responds with a ticket that is valid for two hours. The client | |
572 | tool automatically stores that ticket and uses it for further requests | |
573 | to this server. | |
574 | ||
575 | You can also manually trigger this login/logout using the login and | |
576 | logout commands: | |
577 | ||
578 | .. code-block:: console | |
579 | ||
580 | # proxmox-backup-client login | |
581 | Password: ********** | |
582 | ||
583 | To remove the ticket, issue a logout: | |
584 | ||
585 | .. code-block:: console | |
586 | ||
587 | # proxmox-backup-client logout | |
588 | ||
589 | ||
08379a21 DW |
590 | .. _changing-backup-owner: |
591 | ||
592 | Changing the Owner of a Backup Group | |
593 | ------------------------------------ | |
594 | ||
595 | By default, the owner of a backup group is the user which was used to originally | |
596 | create that backup group (or in the case of sync jobs, ``root@pam``). This | |
597 | means that if a user ``mike@pbs`` created a backup, another user ``john@pbs`` | |
5fb852af | 598 | can not be used to create backups in that same backup group. In case you want |
08379a21 DW |
599 | to change the owner of a backup, you can do so with the below command, using a |
600 | user that has ``Datastore.Modify`` privileges on the datastore. | |
601 | ||
602 | .. code-block:: console | |
603 | ||
604 | # proxmox-backup-client change-owner vm/103 john@pbs | |
605 | ||
606 | This can also be done from within the web interface, by navigating to the | |
a3b10267 TL |
607 | `Content` section of the datastore that contains the backup group and selecting |
608 | the user icon under the `Actions` column. Common cases for this could be to | |
609 | change the owner of a sync job from ``root@pam``, or to repurpose a backup | |
610 | group. | |
08379a21 DW |
611 | |
612 | ||
4c3efb53 | 613 | .. _backup-pruning: |
04e24b14 DW |
614 | |
615 | Pruning and Removing Backups | |
616 | ---------------------------- | |
617 | ||
f71a4ce6 | 618 | You can manually delete a backup snapshot using the ``forget`` command: |
04e24b14 DW |
619 | |
620 | .. code-block:: console | |
621 | ||
abf9b6da | 622 | # proxmox-backup-client snapshot forget <snapshot> |
04e24b14 DW |
623 | |
624 | ||
f71a4ce6 TL |
625 | .. caution:: This command removes all archives in this backup snapshot. They |
626 | will be inaccessible and *unrecoverable*. | |
627 | ||
628 | Don't forget to add the namespace ``--ns`` parameter if you want to forget a | |
629 | snapshot that is contained in the root namespace: | |
630 | ||
631 | .. code-block:: console | |
632 | ||
633 | # proxmox-backup-client snapshot forget <snapshot> --ns <ns> | |
634 | ||
635 | ||
04e24b14 DW |
636 | |
637 | ||
638 | Although manual removal is sometimes required, the ``prune`` | |
639 | command is normally used to systematically delete older backups. Prune lets | |
640 | you specify which backup snapshots you want to keep. The | |
641 | following retention options are available: | |
642 | ||
643 | ``--keep-last <N>`` | |
644 | Keep the last ``<N>`` backup snapshots. | |
645 | ||
646 | ``--keep-hourly <N>`` | |
8df3a17c FE |
647 | Keep backups for the last ``<N>`` hours. If there is more than one backup for |
648 | a single hour, only the latest is kept. Hours without backups do not count. | |
04e24b14 DW |
649 | |
650 | ``--keep-daily <N>`` | |
8df3a17c FE |
651 | Keep backups for the last ``<N>`` days. If there is more than one backup for a |
652 | single day, only the latest is kept. Days without backups do not count. | |
04e24b14 DW |
653 | |
654 | ``--keep-weekly <N>`` | |
8df3a17c FE |
655 | Keep backups for the last ``<N>`` weeks. If there is more than one backup for |
656 | a single week, only the latest is kept. Weeks without backup do not count. | |
04e24b14 DW |
657 | |
658 | .. note:: Weeks start on Monday and end on Sunday. The software | |
659 | uses the `ISO week date`_ system and handles weeks at | |
660 | the end of the year correctly. | |
661 | ||
662 | ``--keep-monthly <N>`` | |
8df3a17c FE |
663 | Keep backups for the last ``<N>`` months. If there is more than one backup for |
664 | a single month, only the latest is kept. Months without backups do not count. | |
04e24b14 DW |
665 | |
666 | ``--keep-yearly <N>`` | |
8df3a17c FE |
667 | Keep backups for the last ``<N>`` years. If there is more than one backup for |
668 | a single year, only the latest is kept. Years without backups do not count. | |
04e24b14 DW |
669 | |
670 | The retention options are processed in the order given above. Each option | |
671 | only covers backups within its time period. The next option does not take care | |
672 | of already covered backups. It will only consider older backups. | |
673 | ||
674 | Unfinished and incomplete backups will be removed by the prune command unless | |
675 | they are newer than the last successful backup. In this case, the last failed | |
676 | backup is retained. | |
677 | ||
678 | .. code-block:: console | |
679 | ||
680 | # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3 | |
681 | ||
682 | ||
683 | You can use the ``--dry-run`` option to test your settings. This only | |
684 | shows the list of existing snapshots and what actions prune would take. | |
685 | ||
686 | .. code-block:: console | |
687 | ||
688 | # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3 | |
689 | ┌────────────────────────────────┬──────┐ | |
690 | │ snapshot │ keep │ | |
691 | ╞════════════════════════════════╪══════╡ | |
692 | │ host/elsa/2019-12-04T13:20:37Z │ 1 │ | |
693 | ├────────────────────────────────┼──────┤ | |
694 | │ host/elsa/2019-12-03T09:35:01Z │ 0 │ | |
695 | ├────────────────────────────────┼──────┤ | |
696 | │ host/elsa/2019-11-22T11:54:47Z │ 1 │ | |
697 | ├────────────────────────────────┼──────┤ | |
698 | │ host/elsa/2019-11-21T12:36:25Z │ 0 │ | |
699 | ├────────────────────────────────┼──────┤ | |
700 | │ host/elsa/2019-11-10T10:42:20Z │ 1 │ | |
701 | └────────────────────────────────┴──────┘ | |
702 | ||
703 | .. note:: Neither the ``prune`` command nor the ``forget`` command free space | |
704 | in the chunk-store. The chunk-store still contains the data blocks. To free | |
a98e2287 | 705 | space you need to perform :ref:`client_garbage-collection`. |
04e24b14 | 706 | |
38517ca0 DC |
707 | It is also possible to protect single snapshots from being pruned or deleted: |
708 | ||
709 | .. code-block:: console | |
710 | ||
711 | # proxmox-backup-client snapshot protected update <snapshot> true | |
712 | ||
713 | This will set the protected flag on the snapshot and prevent pruning or manual | |
6481fd24 | 714 | deletion of this snapshot until the flag is removed again with: |
38517ca0 DC |
715 | |
716 | .. code-block:: console | |
717 | ||
718 | # proxmox-backup-client snapshot protected update <snapshot> false | |
719 | ||
6481fd24 DW |
720 | When a group with a protected snapshot is deleted, only the non-protected |
721 | ones are removed, and the rest will remain. | |
38517ca0 DC |
722 | |
723 | .. note:: This flag will not be synced when using pull or sync jobs. If you | |
6481fd24 | 724 | want to protect a synced snapshot, you have to do this again manually on |
56d3b59c | 725 | the target backup server. |
04e24b14 | 726 | |
ec1ae7e6 | 727 | .. _client_garbage-collection: |
04e24b14 DW |
728 | |
729 | Garbage Collection | |
730 | ------------------ | |
731 | ||
732 | The ``prune`` command removes only the backup index files, not the data | |
733 | from the datastore. This task is left to the garbage collection | |
734 | command. It is recommended to carry out garbage collection on a regular basis. | |
735 | ||
736 | The garbage collection works in two phases. In the first phase, all | |
737 | data blocks that are still in use are marked. In the second phase, | |
738 | unused data blocks are removed. | |
739 | ||
740 | .. note:: This command needs to read all existing backup index files | |
741 | and touches the complete chunk-store. This can take a long time | |
742 | depending on the number of chunks and the speed of the underlying | |
743 | disks. | |
744 | ||
745 | .. note:: The garbage collection will only remove chunks that haven't been used | |
746 | for at least one day (exactly 24h 5m). This grace period is necessary because | |
747 | chunks in use are marked by touching the chunk which updates the ``atime`` | |
748 | (access time) property. Filesystems are mounted with the ``relatime`` option | |
749 | by default. This results in a better performance by only updating the | |
750 | ``atime`` property if the last access has been at least 24 hours ago. The | |
5fb852af | 751 | downside is that touching a chunk within these 24 hours will not always |
04e24b14 DW |
752 | update its ``atime`` property. |
753 | ||
754 | Chunks in the grace period will be logged at the end of the garbage | |
755 | collection task as *Pending removals*. | |
756 | ||
757 | .. code-block:: console | |
758 | ||
759 | # proxmox-backup-client garbage-collect | |
760 | starting garbage collection on store store2 | |
761 | Start GC phase1 (mark used chunks) | |
762 | Start GC phase2 (sweep unused chunks) | |
763 | percentage done: 1, chunk count: 219 | |
764 | percentage done: 2, chunk count: 453 | |
765 | ... | |
766 | percentage done: 99, chunk count: 21188 | |
767 | Removed bytes: 411368505 | |
768 | Removed chunks: 203 | |
769 | Original data bytes: 327160886391 | |
770 | Disk bytes: 52767414743 (16 %) | |
771 | Disk chunks: 21221 | |
772 | Average chunk size: 2486565 | |
773 | TASK OK | |
774 | ||
35e80f3e | 775 | Garbage collection can also be scheduled using ``proxmox-backup-manager`` or |
5fb852af | 776 | from the Proxmox Backup Server's web interface. |
04e24b14 DW |
777 | |
778 | Benchmarking | |
779 | ------------ | |
780 | ||
781 | The backup client also comes with a benchmarking tool. This tool measures | |
654c56e0 TL |
782 | various metrics relating to compression and encryption speeds. If a Proxmox |
783 | Backup repository (remote or local) is specified, the TLS upload speed will get | |
784 | measured too. | |
785 | ||
786 | You can run a benchmark using the ``benchmark`` subcommand of | |
787 | ``proxmox-backup-client``: | |
788 | ||
789 | .. note:: The TLS speed test is only included if a :ref:`backup server | |
790 | repository is specified <client_repository>`. | |
04e24b14 DW |
791 | |
792 | .. code-block:: console | |
793 | ||
794 | # proxmox-backup-client benchmark | |
81f5d03e DM |
795 | Uploaded 1517 chunks in 5 seconds. |
796 | Time per request: 3309 microseconds. | |
797 | TLS speed: 1267.41 MB/s | |
798 | SHA256 speed: 2066.73 MB/s | |
799 | Compression speed: 775.11 MB/s | |
800 | Decompress speed: 1233.35 MB/s | |
801 | AES256/GCM speed: 3688.27 MB/s | |
802 | Verify speed: 783.43 MB/s | |
04e24b14 DW |
803 | ┌───────────────────────────────────┬─────────────────────┐ |
804 | │ Name │ Value │ | |
805 | ╞═══════════════════════════════════╪═════════════════════╡ | |
81f5d03e | 806 | │ TLS (maximal backup upload speed) │ 1267.41 MB/s (103%) │ |
04e24b14 | 807 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 808 | │ SHA256 checksum computation speed │ 2066.73 MB/s (102%) │ |
04e24b14 | 809 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 810 | │ ZStd level 1 compression speed │ 775.11 MB/s (103%) │ |
04e24b14 | 811 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e | 812 | │ ZStd level 1 decompression speed │ 1233.35 MB/s (103%) │ |
04e24b14 | 813 | ├───────────────────────────────────┼─────────────────────┤ |
81f5d03e DM |
814 | │ Chunk verification speed │ 783.43 MB/s (103%) │ |
815 | ├───────────────────────────────────┼─────────────────────┤ | |
816 | │ AES256 GCM encryption speed │ 3688.27 MB/s (101%) │ | |
04e24b14 DW |
817 | └───────────────────────────────────┴─────────────────────┘ |
818 | ||
81f5d03e | 819 | |
04e24b14 | 820 | .. note:: The percentages given in the output table correspond to a |
654c56e0 | 821 | comparison against a Ryzen 7 2700X. |
04e24b14 DW |
822 | |
823 | You can also pass the ``--output-format`` parameter to output stats in ``json``, | |
824 | rather than the default table format. |