]> git.proxmox.com Git - proxmox-backup.git/blob - docs/administration-guide.rst
administration-guide.rst: add section "Proxmox VE integration"
[proxmox-backup.git] / docs / administration-guide.rst
1 Administration Guide
2 ====================
3
4 The administration guide.
5
6 .. todo:: either add a bit more explanation or remove the previous sentence
7
8 Terminology
9 -----------
10
11 Backup Content
12 ~~~~~~~~~~~~~~
13
14 When doing deduplication, there are different strategies to get
15 optimal results in terms of performance and/or deduplication rates.
16 Depending on the type of data, one can split data into *fixed* or *variable*
17 sized chunks.
18
19 Fixed sized chunking needs almost no CPU performance, and is used to
20 backup virtual machine images.
21
22 Variable sized chunking needs more CPU power, but is essential to get
23 good deduplication rates for file archives.
24
25 The backup server supports both strategies.
26
27
28 File Archives: ``<name>.pxar``
29 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
30
31 .. see https://moinakg.wordpress.com/2013/06/22/high-performance-content-defined-chunking/
32
33 A file archive stores a full directory tree. Content is stored using
34 the :ref:`pxar-format`, split into variable sized chunks. The format
35 is optimized to achieve good deduplication rates.
36
37
38 Image Archives: ``<name>.img``
39 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
40
41 This is used for virtual machine images and other large binary
42 data. Content is split into fixed sized chunks.
43
44
45 Binary Data (BLOBs)
46 ^^^^^^^^^^^^^^^^^^^
47
48 This type is used to store smaller (< 16MB) binary data such as
49 configuration files. Larger files should be stored as image archive.
50
51 .. caution:: Please do not store all files as BLOBs. Instead, use the
52 file archive to store whole directory trees.
53
54
55 Catalog File: ``catalog.pcat1``
56 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
57
58 The catalog file is an index for file archives. It contains
59 the list of files and is used to speed-up search operations.
60
61
62 The Manifest: ``index.json``
63 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64
65 The manifest contains the list of all backup files, their
66 sizes and checksums. It is used to verify the consistency of a
67 backup.
68
69
70 Backup Type
71 ~~~~~~~~~~~
72
73 The backup server groups backups by *type*, where *type* is one of:
74
75 ``vm``
76 This type is used for :term:`virtual machine`\ s. Typically
77 contains the virtual machine's configuration and an image archive
78 for each disk.
79
80 ``ct``
81 This type is used for :term:`container`\ s. Contains the container's
82 configuration and a single file archive for the container content.
83
84 ``host``
85 This type is used for backups created from within the backed up machine.
86 Typically this would be a physical host but could also be a virtual machine
87 or container. Such backups may contain file and image archives, there are no restrictions in this regard.
88
89
90 Backup ID
91 ~~~~~~~~~
92
93 An unique ID. Usually the virtual machine or container ID. ``host``
94 type backups normally use the hostname.
95
96
97 Backup Time
98 ~~~~~~~~~~~
99
100 The time when the backup was made.
101
102
103 Backup Group
104 ~~~~~~~~~~~~
105
106 The tuple ``<type>/<ID>`` is called a backup group. Such a group
107 may contain one or more backup snapshots.
108
109
110 Backup Snapshot
111 ~~~~~~~~~~~~~~~
112
113 The triplet ``<type>/<ID>/<time>`` is called a backup snapshot. It
114 uniquely identifies a specific backup within a datastore.
115
116 .. code-block:: console
117 :caption: Backup Snapshot Examples
118
119 vm/104/2019-10-09T08:01:06Z
120 host/elsa/2019-11-08T09:48:14Z
121
122 As you can see, the time format is RFC3399_ with Coordinated
123 Universal Time (UTC_, identified by the trailing *Z*).
124
125
126 :term:`DataStore`
127 ~~~~~~~~~~~~~~~~~
128
129 A datastore is a place where backups are stored. The current implementation
130 uses a directory inside a standard unix file system (``ext4``, ``xfs``
131 or ``zfs``) to store the backup data.
132
133 Datastores are identified by a simple *ID*. You can configure it
134 when setting up the backup server.
135
136
137 Backup Server Management
138 ------------------------
139
140 The command line tool to configure and manage the backup server is called
141 :command:`proxmox-backup-manager`.
142
143
144 Datastore Configuration
145 ~~~~~~~~~~~~~~~~~~~~~~~
146
147 A :term:`datastore` is a place to store backups. You can configure
148 multiple datastores. At least one datastore needs to be
149 configured. The datastore is identified by a simple `name` and points
150 to a directory.
151
152 The following command creates a new datastore called ``store1`` on :file:`/backup/disk1/store1`
153
154 .. code-block:: console
155
156 # proxmox-backup-manager datastore create store1 /backup/disk1/store1
157
158 To list existing datastores run:
159
160 .. code-block:: console
161
162 # proxmox-backup-manager datastore list
163 ┌────────┬──────────────────────┬─────────────────────────────┐
164 │ name │ path │ comment │
165 ╞════════╪══════════════════════╪═════════════════════════════╡
166 │ store1 │ /backup/disk1/store1 │ This is my default storage. │
167 └────────┴──────────────────────┴─────────────────────────────┘
168
169 Finally, it is possible to remove the datastore configuration:
170
171 .. code-block:: console
172
173 # proxmox-backup-manager datastore remove store1
174
175 .. note:: The above command removes only the datastore configuration. It does
176 not delete any data from the underlying directory.
177
178
179 File Layout
180 ^^^^^^^^^^^
181
182 .. todo:: Add datastore file layout example
183
184
185 User Management
186 ~~~~~~~~~~~~~~~
187
188 Proxmox Backup support several authentication realms, and you need to
189 choose the realm when you add a new user. Possible realms are:
190
191 :pam: Linux PAM standard authentication. Use this if you want to
192 authenticate as Linux system user (Users needs to exist on the
193 system).
194
195 :pbs: Proxmox Backup Server realm. This type stores hashed passwords in
196 ``/etc/proxmox-backup/shadow.json``.
197
198 After installation, there is a single user ``root@pam``, which
199 corresponds to the Unix superuser. You can use the
200 ``proxmox-backup-manager`` command line tool to list or manipulate
201 users:
202
203 .. code-block:: console
204
205 # proxmox-backup-manager user list
206 ┌─────────────┬────────┬────────┬───────────┬──────────┬────────────────┬────────────────────┐
207 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
208 ╞═════════════╪════════╪════════╪═══════════╪══════════╪════════════════╪════════════════════╡
209 │ root@pam │ 1 │ │ │ │ │ Superuser │
210 └─────────────┴────────┴────────┴───────────┴──────────┴────────────────┴────────────────────┘
211
212 The superuser has full administration rights on everything, so you
213 normally want to add other users with less privileges:
214
215 .. code-block:: console
216
217 # proxmox-backup-manager user create john@pbs --email john@example.com
218
219 The create command lets you specify many option like ``--email`` or
220 ``--password``, but you can update or change any of them using the
221 update command later:
222
223 .. code-block:: console
224
225 # proxmox-backup-manager user update john@pbs --firstname John --lastname Smith
226 # proxmox-backup-manager user update john@pbs --comment "An example user."
227
228
229 .. todo:: Mention how to set password without passing plaintext password as cli argument.
230
231
232 The resulting use list looks like this:
233
234 .. code-block:: console
235
236 # proxmox-backup-manager user list
237 ┌──────────┬────────┬────────┬───────────┬──────────┬──────────────────┬──────────────────┐
238 │ userid │ enable │ expire │ firstname │ lastname │ email │ comment │
239 ╞══════════╪════════╪════════╪═══════════╪══════════╪══════════════════╪══════════════════╡
240 │ john@pbs │ 1 │ │ John │ Smith │ john@example.com │ An example user. │
241 ├──────────┼────────┼────────┼───────────┼──────────┼──────────────────┼──────────────────┤
242 │ root@pam │ 1 │ │ │ │ │ Superuser │
243 └──────────┴────────┴────────┴───────────┴──────────┴──────────────────┴──────────────────┘
244
245 Newly created users do not have an permissions. Please read the next
246 section to learn how to set access permissions.
247
248 If you want to disable an user account, you can do that by setting ``--enable`` to ``0``
249
250 .. code-block:: console
251
252 # proxmox-backup-manager user update john@pbs --enable 0
253
254 Or completely remove the users with:
255
256 .. code-block:: console
257
258 # proxmox-backup-manager user remove john@pbs
259
260
261 Access Control
262 ~~~~~~~~~~~~~~
263
264 Users do not have any permission by default. Instead you need to
265 specify what is allowed and what not. You can do this by assigning
266 roles to users on specific objects like datastores or remotes. The
267 following roles exist:
268
269 **Admin**
270 The Administrator can do anything.
271
272 **Audit**
273 An Auditor can view things, but is not allowed to change settings.
274
275 **NoAccess**
276 Disable Access - nothing is allowed.
277
278 **DatastoreAdmin**
279 Can do anything on datastores.
280
281 **DatastoreAudit**
282 Can view datastore settings and list content. But
283 is not allowed to read the actual data.
284
285 **DataStoreReader**
286 Can Inspect datastore content and can do restores.
287
288 **DataStoreBackup**
289 Can backup and restore owned backups.
290
291 **DatastorePowerUser**
292 Can backup, restore, and prune owned backups.
293
294 **RemoteAdmin**
295 Can do anything on remotes.
296
297 **RemoteAudit**
298 Can view remote settings.
299
300 **RemoteSyncOperator**
301 Is allowed to read data from a remote.
302
303
304
305 Backup Client usage
306 -------------------
307
308 The command line client is called :command:`proxmox-backup-client`.
309
310
311 Respository Locations
312 ~~~~~~~~~~~~~~~~~~~~~
313
314 The client uses the following notation to specify a datastore repository
315 on the backup server.
316
317 [[username@]server:]datastore
318
319 The default value for ``username`` ist ``root``. If no server is specified, the
320 default is the local host (``localhost``).
321
322 You can pass the repository with the ``--repository`` command
323 line option, or by setting the ``PBS_REPOSITORY`` environment
324 variable.
325
326
327 Environment Variables
328 ~~~~~~~~~~~~~~~~~~~~~
329
330 ``PBS_REPOSITORY``
331 The default backup repository.
332
333 ``PBS_PASSWORD``
334 When set, this value is used for the password required for the
335 backup server.
336
337 ``PBS_ENCRYPTION_PASSWORD``
338 When set, this value is used to access the secret encryption key (if
339 protected by password).
340
341 ``PBS_FINGERPRINT`` When set, this value is used to verify the server
342 certificate (only used if the system CA certificates cannot
343 validate the certificate).
344
345
346 Output Format
347 ~~~~~~~~~~~~~
348
349 Most commands support the ``--output-format`` parameter. It accepts
350 the following values:
351
352 :``text``: Text format (default). Structured data is rendered as a table.
353
354 :``json``: JSON (single line).
355
356 :``json-pretty``: JSON (multiple lines, nicely formatted).
357
358
359 Please use the following environment variables to modify output behavior:
360
361 ``PROXMOX_OUTPUT_FORMAT``
362 Defines the default output format.
363
364 ``PROXMOX_OUTPUT_NO_BORDER``
365 If set (to any value), do not render table borders.
366
367 ``PROXMOX_OUTPUT_NO_HEADER``
368 If set (to any value), do not render table headers.
369
370 .. note:: The ``text`` format is designed to be human readable, and
371 not meant to be parsed by automation tools. Please use the ``json``
372 format if you need to process the output.
373
374
375 .. _creating-backups:
376
377 Creating Backups
378 ~~~~~~~~~~~~~~~~
379
380 This section explains how to create a backup from within the machine. This can
381 be a physical host, a virtual machine, or a container. Such backups may contain file
382 and image archives. There are no restrictions in this case.
383
384 .. note:: If you want to backup virtual machines or containers on Proxmov VE, see :ref:`pve-integration`.
385
386 For the following example you need to have a backup server set up, working
387 credentials and need to know the repository name.
388 In the following examples we use ``backup-server:store1``.
389
390 .. code-block:: console
391
392 # proxmox-backup-client backup root.pxar:/ --repository backup-server:store1
393 Starting backup: host/elsa/2019-12-03T09:35:01Z
394 Client name: elsa
395 skip mount point: "/boot/efi"
396 skip mount point: "/dev"
397 skip mount point: "/run"
398 skip mount point: "/sys"
399 Uploaded 12129 chunks in 87 seconds (564 MB/s).
400 End Time: 2019-12-03T10:36:29+01:00
401
402 This will prompt you for a password and then uploads a file archive named
403 ``root.pxar`` containing all the files in the ``/`` directory.
404
405 .. Caution:: Please note that the proxmox-backup-client does not
406 automatically include mount points. Instead, you will see a short
407 ``skip mount point`` notice for each of them. The idea is to
408 create a separate file archive for each mounted disk. You can
409 explicitly include them using the ``--include-dev`` option
410 (i.e. ``--include-dev /boot/efi``). You can use this option
411 multiple times for each mount point that should be included.
412
413 The ``--repository`` option can get quite long and is used by all
414 commands. You can avoid having to enter this value by setting the
415 environment variable ``PBS_REPOSITORY``.
416
417 .. code-block:: console
418
419 # export PBS_REPOSTORY=backup-server:store1
420
421 After this you can execute all commands without specifying the ``--repository``
422 option.
423
424 One single backup is allowed to contain more than one archive. For example, if
425 you want to backup two disks mounted at ``/mmt/disk1`` and ``/mnt/disk2``:
426
427 .. code-block:: console
428
429 # proxmox-backup-client backup disk1.pxar:/mnt/disk1 disk2.pxar:/mnt/disk2
430
431 This creates a backup of both disks.
432
433 The backup command takes a list of backup specifications, which
434 include the archive name on the server, the type of the archive, and the
435 archive source at the client. The format is:
436
437 <archive-name>.<type>:<source-path>
438
439 Common types are ``.pxar`` for file archives, and ``.img`` for block
440 device images. To create a backup of a block device run the following command:
441
442 .. code-block:: console
443
444 # proxmox-backup-client backup mydata.img:/dev/mylvm/mydata
445
446 Excluding files/folders from a backup
447 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
448
449 Sometimes it is desired to exclude certain files or folders from a backup archive.
450 To tell the Proxmox backup client when and how to ignore files and directories,
451 place a text file called ``.pxarexclude`` in the filesystem hierarchy.
452 Whenever the backup client encounters such a file in a directory, it interprets
453 each line as glob match patterns for files and directories that are to be excluded
454 from the backup.
455
456 The file must contain a single glob pattern per line. Empty lines are ignored.
457 The same is true for lines starting with ``#``, which indicates a comment.
458 A ``!`` at the beginning of a line reverses the glob match pattern from an exclusion
459 to an explicit inclusion. This makes it possible to exclude all entries in a
460 directory except for a few single files/subdirectories.
461 Lines ending in ``/`` match only on directories.
462 The directory containing the ``.pxarexclude`` file is considered to be the root of
463 the given patterns. It is only possible to match files in this directory and its subdirectories.
464
465 ``\`` is used to escape special glob characters.
466 ``?`` matches any single character.
467 ``*`` matches any character, including an empty string.
468 ``**`` is used to match subdirectories. It can be used to, for example, exclude
469 all files ending in ``.tmp`` within the directory or subdirectories with the
470 following pattern ``**/*.tmp``.
471 ``[...]`` matches a single character from any of the provided characters within
472 the brackets. ``[!...]`` does the complementary and matches any singe character
473 not contained within the brackets. It is also possible to specify ranges with two
474 characters separated by ``-``. For example, ``[a-z]`` matches any lowercase
475 alphabetic character and ``[0-9]`` matches any one single digit.
476
477 The order of the glob match patterns defines if a file is included or
478 excluded, later entries win over previous ones.
479 This is also true for match patterns encountered deeper down the directory tree,
480 which can override a previous exclusion.
481 Be aware that excluded directories will **not** be read by the backup client.
482 A ``.pxarexclude`` file in a subdirectory will have no effect.
483 ``.pxarexclude`` files are treated as regular files and will be included in the
484 backup archive.
485
486 For example, consider the following directory structure:
487
488 .. code-block:: console
489
490 # ls -aR folder
491 folder/:
492 . .. .pxarexclude subfolder0 subfolder1
493
494 folder/subfolder0:
495 . .. file0 file1 file2 file3 .pxarexclude
496
497 folder/subfolder1:
498 . .. file0 file1 file2 file3
499
500 The different ``.pxarexclude`` files contain the following:
501
502 .. code-block:: console
503
504 # cat folder/.pxarexclude
505 /subfolder0/file1
506 /subfolder1/*
507 !/subfolder1/file2
508
509 .. code-block:: console
510
511 # cat folder/subfolder0/.pxarexclude
512 file3
513
514 This would exclude ``file1`` and ``file3`` in ``subfolder0`` and all of
515 ``subfolder1`` except ``file2``.
516
517 Restoring this backup will result in:
518
519 .. code-block:: console
520
521 ls -aR restored
522 restored/:
523 . .. .pxarexclude subfolder0 subfolder1
524
525 restored/subfolder0:
526 . .. file0 file2 .pxarexclude
527
528 restored/subfolder1:
529 . .. file2
530
531 Encryption
532 ^^^^^^^^^^
533
534 Proxmox backup supports client side encryption with AES-256 in GCM_
535 mode. First you need to create an encryption key:
536
537 .. code-block:: console
538
539 # proxmox-backup-client key create my-backup.key
540 Encryption Key Password: **************
541
542 The key is password protected by default. If you do not need this
543 extra protection, you can also create it without a password:
544
545 .. code-block:: console
546
547 # proxmox-backup-client key create /path/to/my-backup.key --kdf none
548
549
550 .. code-block:: console
551
552 # proxmox-backup-client backup etc.pxar:/etc --keyfile /path/to/my-backup.key
553 Password: *********
554 Encryption Key Password: **************
555 ...
556
557
558 You can avoid entering the passwords by setting the environment
559 variables ``PBS_PASSWORD`` and ``PBS_ENCRYPTION_PASSWORD``.
560
561 .. todo:: Explain master-key
562
563
564 Restoring Data
565 ~~~~~~~~~~~~~~
566
567 The regular creation of backups is a necessary step to avoid data
568 loss. More important, however, is the restoration. It is good practice to perform
569 periodic recovery tests to ensure that you can access the data in
570 case of problems.
571
572 First, you need to find the snapshot which you want to restore. The snapshot
573 command gives a list of all snapshots on the server:
574
575 .. code-block:: console
576
577 # proxmox-backup-client snapshots
578 ...
579 host/elsa/2019-12-03T09:30:15Z | 51788646825 | root.pxar catalog.pcat1 index.json
580 host/elsa/2019-12-03T09:35:01Z | 51790622048 | root.pxar catalog.pcat1 index.json
581 ...
582
583 You can inspect the catalog to find specific files.
584
585 .. code-block:: console
586
587 # proxmox-backup-client catalog dump host/elsa/2019-12-03T09:35:01Z
588 ...
589 d "./root.pxar.didx/etc/cifs-utils"
590 l "./root.pxar.didx/etc/cifs-utils/idmap-plugin"
591 d "./root.pxar.didx/etc/console-setup"
592 ...
593
594 The restore command lets you restore a single archive from the
595 backup.
596
597 .. code-block:: console
598
599 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z root.pxar /target/path/
600
601 To get the contents of any archive you can restore the ``ìndex.json`` file in the
602 repository and restore it to '-'. This will dump the content to the standard output.
603
604 .. code-block:: console
605
606 # proxmox-backup-client restore host/elsa/2019-12-03T09:35:01Z index.json -
607
608
609 Interactive Restores
610 ^^^^^^^^^^^^^^^^^^^^
611
612 If you only want to restore a few individual files, it is often easier
613 to use the interactive recovery shell.
614
615 .. code-block:: console
616
617 # proxmox-backup-client catalog shell host/elsa/2019-12-03T09:35:01Z root.pxar
618 Starting interactive shell
619 pxar:/ > ls
620 bin boot dev etc home lib lib32
621 ...
622
623 The interactive recovery shell is a minimalistic command line interface that
624 utilizes the metadata stored in the catalog to quickly list, navigate and
625 search files in a file archive.
626 To restore files, you can select them individually or match them with a glob
627 pattern.
628
629 Using the catalog for navigation reduces the overhead considerably because only
630 the catalog needs to be downloaded and, optionally, decrypted.
631 The actual chunks are only accessed if the metadata in the catalog is not enough
632 or for the actual restore.
633
634 Similar to common UNIX shells ``cd`` and ``ls`` are the commands used to change
635 working directory and list directory contents in the archive.
636 ``pwd`` shows the full path of the current working directory with respect to the
637 archive root.
638
639 Being able to quickly search the contents of the archive is a often needed feature.
640 That's where the catalog is most valuable.
641 For example:
642
643 .. code-block:: console
644
645 pxar:/ > find etc/ **/*.txt --select
646 "/etc/X11/rgb.txt"
647 pxar:/ > list-selected
648 etc/**/*.txt
649 pxar:/ > restore-selected /target/path
650 ...
651
652 This will find and print all files ending in ``.txt`` located in ``etc/`` or a
653 subdirectory and add the corresponding pattern to the list for subsequent restores.
654 ``list-selected`` shows these patterns and ``restore-selected`` finally restores
655 all files in the archive matching the patterns to ``/target/path`` on the local
656 host. This will scan the whole archive.
657
658 With ``restore /target/path`` you can restore the sub-archive given by the current
659 working directory to the local target path ``/target/path`` on your host.
660 By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
661 further limited to files matching the pattern.
662 For example:
663
664 .. code-block:: console
665
666 pxar:/ > cd /etc/
667 pxar:/etc/ > restore /target/ --pattern **/*.conf
668 ...
669
670 The above will scan trough all the directories below ``/etc`` and restore all
671 files ending in ``.conf``.
672
673 .. todo:: Explain interactive restore in more detail
674
675 Mounting of Archives via FUSE
676 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
677
678 The :term:`FUSE` implementation for the pxar archive allows you to mount a
679 file archive as a read-only filesystem to a mountpoint on your host.
680
681 .. code-block:: console
682
683 # proxmox-backup-client mount host/backup-client/2020-01-29T11:29:22Z root.pxar /mnt
684 # ls /mnt
685 bin dev home lib32 libx32 media opt root sbin sys usr
686 boot etc lib lib64 lost+found mnt proc run srv tmp var
687
688 This allows you to access the full content of the archive in a seamless manner.
689
690 .. note:: As the FUSE connection needs to fetch and decrypt chunks from the
691 backup servers datastore, this can cause some additional network and CPU
692 load on your host, depending on the operations you perform on the mounted
693 filesystem.
694
695 To unmount the filesystem use the ``umount`` command on the mountpoint:
696
697 .. code-block:: console
698
699 # umount /mnt
700
701 Login and Logout
702 ~~~~~~~~~~~~~~~~
703
704 The client tool prompts you to enter the logon password as soon as you
705 want to access the backup server. The server checks your credentials
706 and responds with a ticket that is valid for two hours. The client
707 tool automatically stores that ticket and uses it for further requests
708 to this server.
709
710 You can also manually trigger this login/logout using the login and
711 logout commands:
712
713 .. code-block:: console
714
715 # proxmox-backup-client login
716 Password: **********
717
718 To remove the ticket, issue a logout:
719
720 .. code-block:: console
721
722 # proxmox-backup-client logout
723
724
725 Pruning and Removing Backups
726 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
727
728 You can manually delete a backup snapshot using the ``forget``
729 command:
730
731 .. code-block:: console
732
733 # proxmox-backup-client forget <snapshot>
734
735
736 .. caution:: This command removes all archives in this backup
737 snapshot. They will be inaccessible and unrecoverable.
738
739
740 The manual removal is sometimes required, but normally the prune
741 command is used to systematically delete older backups. Prune lets
742 you specify which backup snapshots you want to keep. The
743 following retention options are available:
744
745 ``--keep-last <N>``
746 Keep the last ``<N>`` backup snapshots.
747
748 ``--keep-hourly <N>``
749 Keep backups for the last ``<N>`` hours. If there is more than one
750 backup for a single hour, only the latest is kept.
751
752 ``--keep-daily <N>``
753 Keep backups for the last ``<N>`` days. If there is more than one
754 backup for a single day, only the latest is kept.
755
756 ``--keep-weekly <N>``
757 Keep backups for the last ``<N>`` weeks. If there is more than one
758 backup for a single week, only the latest is kept.
759
760 .. note:: Weeks start on Monday and end on Sunday. The software
761 uses the `ISO week date`_ system and handles weeks at
762 the end of the year correctly.
763
764 ``--keep-monthly <N>``
765 Keep backups for the last ``<N>`` months. If there is more than one
766 backup for a single month, only the latest is kept.
767
768 ``--keep-yearly <N>``
769 Keep backups for the last ``<N>`` years. If there is more than one
770 backup for a single year, only the latest is kept.
771
772 The retention options are processed in the order given above. Each option
773 only covers backups within its time period. The next option does not take care
774 of already covered backups. It will only consider older backups.
775
776 For example, the ``--keep-monthly`` option does not consider any backup that is
777 younger than one month.
778
779 .. todo:: check if the previous statement is correct
780
781 Unfinished and incomplete backups will be removed by the prune command unless
782 they are newer than the last successful backup. In this case, the last failed
783 backup is retained.
784
785 .. code-block:: console
786
787 # proxmox-backup-client prune <group> --keep-daily 7 --keep-weekly 4 --keep-monthly 3
788
789
790 You can use the ``--dry-run`` option to test your settings. This only
791 shows the list of existing snapshots and which action prune would take.
792
793 .. code-block:: console
794
795 # proxmox-backup-client prune host/elsa --dry-run --keep-daily 1 --keep-weekly 3
796 retention options: --keep-daily 1 --keep-weekly 3
797 Testing prune on store "store2" group "host/elsa"
798 host/elsa/2019-12-04T13:20:37Z keep
799 host/elsa/2019-12-03T09:35:01Z remove
800 host/elsa/2019-11-22T11:54:47Z keep
801 host/elsa/2019-11-21T12:36:25Z remove
802 host/elsa/2019-11-10T10:42:20Z keep
803
804
805 .. note:: Neither the ``prune`` command nor the ``forget`` command free space
806 in the chunk-store. The chunk-store still contains the data blocks. To free
807 space you need to perform :ref:`garbage-collection`.
808
809
810 .. _garbage-collection:
811
812 Garbage Collection
813 ~~~~~~~~~~~~~~~~~~
814
815 The ``prune`` command removes only the backup index files, not the data
816 from the data store. This task is left to the garbage collection
817 command. It is recommended to carry out garbage collection on a regular basis.
818
819 The garbage collection works in two phases. In the first phase, all
820 data blocks that are still in use are marked. In the second phase,
821 unused data blocks are removed.
822
823 .. note:: This command needs to read all existing backup index files
824 and touches the complete chunk-store. This can take a long time
825 depending on the number of chunks and the speed of the underlying
826 disks.
827
828
829 .. code-block:: console
830
831 # proxmox-backup-client garbage-collect
832 starting garbage collection on store store2
833 Start GC phase1 (mark used chunks)
834 Start GC phase2 (sweep unused chunks)
835 percentage done: 1, chunk count: 219
836 percentage done: 2, chunk count: 453
837 ...
838 percentage done: 99, chunk count: 21188
839 Removed bytes: 411368505
840 Removed chunks: 203
841 Original data bytes: 327160886391
842 Disk bytes: 52767414743 (16 %)
843 Disk chunks: 21221
844 Average chunk size: 2486565
845 TASK OK
846
847
848 .. todo:: howto run garbage-collection at regular intervalls (cron)
849
850
851 .. _pve-integration:
852
853 `Proxmox VE`_ integration
854 -------------------------
855
856 You need to define a new storage with type 'pbs' on your `Proxmox VE`_
857 node. The following example uses ``store2`` as storage name, and
858 assumes the server address is ``localhost``, and you want to connect
859 as ``user1@pbs``.
860
861 .. code-block:: console
862
863 # pvesm add pbs store2 --server localhost --datastore store2
864 # pvesm set store2 --username user1@pbs --password <secret>
865
866 If your backup server uses a self signed certificate, you need to add
867 the certificate fingerprint to the configuration. You can get the
868 fingerprint by running the following command on the backup server:
869
870 .. code-block:: console
871
872 # proxmox-backup-manager cert info |grep Fingerprint
873 Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
874
875 Please add that fingerprint to your configuration to establish a trust
876 relationship:
877
878 .. code-block:: console
879
880 # pvesm set store2 --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
881
882 After that you should be able to see storage status with:
883
884 .. code-block:: console
885
886 # pvesm status --storage store2
887 Name Type Status Total Used Available %
888 store2 pbs active 3905109820 1336687816 2568422004 34.23%
889
890
891
892 .. include:: command-line-tools.rst
893
894 .. include:: services.rst
895
896 .. include host system admin at the end
897
898 .. include:: sysadmin.rst