X-Git-Url: https://git.proxmox.com/?a=blobdiff_plain;f=vzdump.adoc;h=a2b908dd18d6dc9cb4ec8c841db7cf932cc134ac;hb=d2a0a9cc74f5de7fc8099bd3bbb131647c5c6071;hp=0461140de480509a551eb9c220d9be7cba2adc35;hpb=3802f512b9733615ce60d31588fb2d3e13441a0c;p=pve-docs.git diff --git a/vzdump.adoc b/vzdump.adoc index 0461140..a2b908d 100644 --- a/vzdump.adoc +++ b/vzdump.adoc @@ -25,7 +25,7 @@ Backup and Restore :pve-toplevel: endif::manvolnum[] -Backups are a requirements for any sensible IT deployment, and {pve} +Backups are a requirement for any sensible IT deployment, and {pve} provides a fully integrated solution, using the capabilities of each storage and each guest system type. This allows the system administrator to fine tune via the `mode` option between consistency @@ -78,17 +78,17 @@ consistency, the use of the `snapshot` mode is recommended instead. `snapshot` mode:: This mode provides the lowest operation downtime, at the cost of a -small inconstancy risk. It works by performing a Proxmox VE live +small inconsistency risk. It works by performing a {pve} live backup, in which data blocks are copied while the VM is running. If the guest agent is enabled (`agent: 1`) and running, it calls `guest-fsfreeze-freeze` and `guest-fsfreeze-thaw` to improve consistency. -A technical overview of the Proxmox VE live backup for QemuServer can +A technical overview of the {pve} live backup for QemuServer can be found online https://git.proxmox.com/?p=pve-qemu.git;a=blob_plain;f=backup.txt[here]. -NOTE: Proxmox VE live backup provides snapshot-like semantics on any +NOTE: {pve} live backup provides snapshot-like semantics on any storage type. It does not require that the underlying storage supports snapshots. Also please note that since the backups are done via a background Qemu process, a stopped VM will appear as running for a @@ -111,7 +111,7 @@ started (resumed) again. This results in minimal downtime, but needs additional space to hold the container copy. + When the container is on a local file system and the target storage of -the backup is an NFS server, you should set `--tmpdir` to reside on a +the backup is an NFS/CIFS server, you should set `--tmpdir` to reside on a local file system too, as this will result in a many fold performance improvement. Use of a local `tmpdir` is also required if you want to backup a local container using ACLs in suspend mode if the backup @@ -143,9 +143,143 @@ backup time into the filename, for example vzdump-lxc-105-2009_10_09-11_04_43.tar -That way it is possible to store several backup in the same -directory. The parameter `maxfiles` can be used to specify the -maximum number of backups to keep. +That way it is possible to store several backup in the same directory. You can +limit the number of backups that are kept with various retention options, see +the xref:vzdump_retention[Backup Retention] section below. + +Backup File Compression +----------------------- + +The backup file can be compressed with one of the following algorithms: `lzo` +footnote:[Lempel–Ziv–Oberhumer a lossless data compression algorithm +https://en.wikipedia.org/wiki/Lempel-Ziv-Oberhumer], `gzip` footnote:[gzip - +based on the DEFLATE algorithm https://en.wikipedia.org/wiki/Gzip] or `zstd` +footnote:[Zstandard a lossless data compression algorithm +https://en.wikipedia.org/wiki/Zstandard]. + +Currently, Zstandard (zstd) is the fastest of these three algorithms. +Multi-threading is another advantage of zstd over lzo and gzip. Lzo and gzip +are more widely used and often installed by default. + +You can install pigz footnote:[pigz - parallel implementation of gzip +https://zlib.net/pigz/] as a drop-in replacement for gzip to provide better +performance due to multi-threading. For pigz & zstd, the amount of +threads/cores can be adjusted. See the +xref:vzdump_configuration[configuration options] below. + +The extension of the backup file name can usually be used to determine which +compression algorithm has been used to create the backup. + +|=== +|.zst | Zstandard (zstd) compression +|.gz or .tgz | gzip compression +|.lzo | lzo compression +|=== + +If the backup file name doesn't end with one of the above file extensions, then +it was not compressed by vzdump. + +Backup Encryption +----------------- + +For Proxmox Backup Server storages, you can optionally set up client-side +encryption of backups, see xref:storage_pbs_encryption[the corresponding section.] + +[[vzdump_retention]] +Backup Retention +---------------- + +With the `prune-backups` option you can specify which backups you want to keep +in a flexible manner. The following retention options are available: + +`keep-all ` :: +Keep all backups. If this is `true`, no other options can be set. + +`keep-last ` :: +Keep the last `` backups. + +`keep-hourly ` :: +Keep backups for the last `` hours. If there is more than one +backup for a single hour, only the latest is kept. + +`keep-daily ` :: +Keep backups for the last `` days. If there is more than one +backup for a single day, only the latest is kept. + +`keep-weekly ` :: +Keep backups for the last `` weeks. If there is more than one +backup for a single week, only the latest is kept. + +NOTE: Weeks start on Monday and end on Sunday. The software uses the +`ISO week date`-system and handles weeks at the end of the year correctly. + +`keep-monthly ` :: +Keep backups for the last `` months. If there is more than one +backup for a single month, only the latest is kept. + +`keep-yearly ` :: +Keep backups for the last `` years. If there is more than one +backup for a single year, only the latest is kept. + +The retention options are processed in the order given above. Each option +only covers backups within its time period. The next option does not take care +of already covered backups. It will only consider older backups. + +Specify the retention options you want to use as a +comma-separated list, for example: + + # vzdump 777 --prune-backups keep-last=3,keep-daily=13,keep-yearly=9 + +While you can pass `prune-backups` directly to `vzdump`, it is often more +sensible to configure the setting on the storage level, which can be done via +the web interface. + +NOTE: The old `maxfiles` option is deprecated and should be replaced either by +`keep-last` or, in case `maxfiles` was `0` for unlimited retention, by +`keep-all`. + + +Prune Simulator +~~~~~~~~~~~~~~~ + +You can use the https://pbs.proxmox.com/docs/prune-simulator[prune simulator +of the Proxmox Backup Server documentation] to explore the effect of different +retention options with various backup schedules. + +Retention Settings Example +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The backup frequency and retention of old backups may depend on how often data +changes, and how important an older state may be, in a specific work load. +When backups act as a company's document archive, there may also be legal +requirements for how long backups must be kept. + +For this example, we assume that you are doing daily backups, have a retention +period of 10 years, and the period between backups stored gradually grows. + +`keep-last=3` - even if only daily backups are taken, an admin may want to + create an extra one just before or after a big upgrade. Setting keep-last + ensures this. + +`keep-hourly` is not set - for daily backups this is not relevant. You cover + extra manual backups already, with keep-last. + +`keep-daily=13` - together with keep-last, which covers at least one + day, this ensures that you have at least two weeks of backups. + +`keep-weekly=8` - ensures that you have at least two full months of + weekly backups. + +`keep-monthly=11` - together with the previous keep settings, this + ensures that you have at least a year of monthly backups. + +`keep-yearly=9` - this is for the long term archive. As you covered the + current year with the previous options, you would set this to nine for the + remaining ones, giving you a total of at least 10 years of coverage. + +We recommend that you use a higher retention period than is minimally required +by your environment; you can always reduce it if you find it is unnecessarily +high, but you cannot recreate backups once they have been removed. [[vzdump_restore]] Restore @@ -166,7 +300,7 @@ Bandwidth Limit Restoring one or more big backups may need a lot of resources, especially storage bandwidth for both reading from the backup storage and writing to -the target storage. This can negatively effect other virtual guest as access +the target storage. This can negatively affect other virtual guests as access to storage can get congested. To avoid this you can set bandwidth limits for a backup job. {pve} @@ -200,10 +334,62 @@ time, thus we implemented the possibility to set a default bandwidth limit per configured storage, this can be done with: ---- -# pvesm set STORAGEID --bwlimit KIBs +# pvesm set STORAGEID --bwlimit restore=KIBs ---- - +Live-Restore +~~~~~~~~~~~~ + +Restoring a large backup can take a long time, in which a guest is still +unavailable. For VM backups stored on a Proxmox Backup Server, this wait +time can be mitigated using the live-restore option. + +Enabling live-restore via either the checkbox in the GUI or the `--live-restore` +argument of `qmrestore` causes the VM to start as soon as the restore +begins. Data is copied in the background, prioritizing chunks that the VM is +actively accessing. + +Note that this comes with two caveats: + +* During live-restore, the VM will operate with limited disk read speeds, as + data has to be loaded from the backup server (once loaded, it is immediately + available on the destination storage however, so accessing data twice only + incurs the penalty the first time). Write speeds are largely unaffected. +* If the live-restore fails for any reason, the VM will be left in an + undefined state - that is, not all data might have been copied from the + backup, and it is _most likely_ not possible to keep any data that was written + during the failed restore operation. + +This mode of operation is especially useful for large VMs, where only a small +amount of data is required for initial operation, e.g. web servers - once the OS +and necessary services have been started, the VM is operational, while the +background task continues copying seldom used data. + +Single File Restore +~~~~~~~~~~~~~~~~~~~ + +The 'File Restore' button in the 'Backups' tab of the storage GUI can be used to +open a file browser directly on the data contained in a backup. This feature +is only available for backups on a Proxmox Backup Server. + +For containers, the first layer of the file tree shows all included 'pxar' +archives, which can be opened and browsed freely. For VMs, the first layer shows +contained drive images, which can be opened to reveal a list of supported +storage technologies found on the drive. In the most basic case, this will be an +entry called 'part', representing a partition table, which contains entries for +each partition found on the drive. Note that for VMs, not all data might be +accessible (unsupported guest file systems, storage technologies, etc...). + +Files and directories can be downloaded using the 'Download' button, the latter +being compressed into a zip archive on the fly. + +To enable secure access to VM images, which might contain untrusted data, a +temporary VM (not visible as a guest) is started. This does not mean that data +downloaded from such an archive is inherently safe, but it avoids exposing the +hypervisor system to danger. The VM will stop itself after a timeout. This +entire process happens transparently from a user's point of view. + +[[vzdump_configuration]] Configuration ------------- @@ -255,7 +441,16 @@ You can also manually specify (additional) exclude paths, for example: # vzdump 777 --exclude-path /tmp/ --exclude-path '/var/foo*' -(only excludes tmp directories) +excludes the directory `/tmp/` and any file or directory named `/var/foo`, +`/var/foobar`, and so on. + +Paths that do not start with a `/` are not anchored to the container's root, +but will match relative to any subdirectory. For example: + + # vzdump 777 --exclude-path bar + +excludes any file or directory named `/bar`, `/var/bar`, `/var/foo/bar`, and +so on, but not `/bar2`. Configuration files are also stored inside the backup archive (in `./etc/vzdump/`) and will be correctly restored.