ifdef::manvolnum[]
vzdump(1)
=========
-include::attributes.txt[]
:pve-toplevel:
NAME
ifndef::manvolnum[]
Backup and Restore
==================
-include::attributes.txt[]
:pve-toplevel:
endif::manvolnum[]
-Backups are a requirements for any sensible IT deployment, and {pve}
+Backups are a requirement for any sensible IT deployment, and {pve}
provides a fully integrated solution, using the capabilities of each
storage and each guest system type. This allows the system
administrator to fine tune via the `mode` option between consistency
Backup jobs can be scheduled so that they are executed automatically
on specific days and times, for selectable nodes and guest systems.
Configuration of scheduled backups is done at the Datacenter level in
-the GUI, which will generate a cron entry in /etc/cron.d/vzdump.
+the GUI, which will generate a job entry in /etc/pve/jobs.cfg, which
+will in turn be parsed and executed by the `pvescheduler` daemon.
+These jobs use the xref:chapter_calendar_events[calendar events] for
+defining the schedule.
Backup modes
------------
`stop` mode::
This mode provides the highest consistency of the backup, at the cost
-of a downtime in the VM operation. It works by executing an orderly
-shutdown of the VM, and then runs a background Qemu process to backup
-the VM data. After the backup is complete, the Qemu process resumes
-the VM to full operation mode if it was previously running.
+of a short downtime in the VM operation. It works by executing an
+orderly shutdown of the VM, and then runs a background Qemu process to
+backup the VM data. After the backup is started, the VM goes to full
+operation mode if it was previously running. Consistency is guaranteed
+by using the live backup feature.
`suspend` mode::
`snapshot` mode::
This mode provides the lowest operation downtime, at the cost of a
-small inconstancy risk. It works by performing a Proxmox VE live
+small inconsistency risk. It works by performing a {pve} live
backup, in which data blocks are copied while the VM is running. If the
guest agent is enabled (`agent: 1`) and running, it calls
`guest-fsfreeze-freeze` and `guest-fsfreeze-thaw` to improve
consistency.
-A technical overview of the Proxmox VE live backup for QemuServer can
+A technical overview of the {pve} live backup for QemuServer can
be found online
-https://git.proxmox.com/?p=pve-qemu-kvm.git;a=blob;f=backup.txt[here].
+https://git.proxmox.com/?p=pve-qemu.git;a=blob_plain;f=backup.txt[here].
-NOTE: Proxmox VE live backup provides snapshot-like semantics on any
+NOTE: {pve} live backup provides snapshot-like semantics on any
storage type. It does not require that the underlying storage supports
-snapshots.
+snapshots. Also please note that since the backups are done via
+a background Qemu process, a stopped VM will appear as running for a
+short amount of time while the VM disks are being read by Qemu.
+However the VM itself is not booted, only its disk(s) are read.
.Backup modes for Containers:
additional space to hold the container copy.
+
When the container is on a local file system and the target storage of
-the backup is an NFS server, you should set `--tmpdir` to reside on a
+the backup is an NFS/CIFS server, you should set `--tmpdir` to reside on a
local file system too, as this will result in a many fold performance
improvement. Use of a local `tmpdir` is also required if you want to
backup a local container using ACLs in suspend mode if the backup
supports snapshots. Using the `backup=no` mount point option individual volumes
can be excluded from the backup (and thus this requirement).
-NOTE: bind and device mount points are skipped during backup operations, like
-volume mount points with the backup option disabled.
-
+// see PVE::VZDump::LXC::prepare()
+NOTE: By default additional mount points besides the Root Disk mount point are
+not included in backups. For volume mount points you can set the *Backup* option
+to include the mount point in the backup. Device and bind mounts are never
+backed up as their content is managed outside the {pve} storage library.
Backup File Names
-----------------
vzdump-lxc-105-2009_10_09-11_04_43.tar
-That way it is possible to store several backup in the same
-directory. The parameter `maxfiles` can be used to specify the
-maximum number of backups to keep.
+That way it is possible to store several backup in the same directory. You can
+limit the number of backups that are kept with various retention options, see
+the xref:vzdump_retention[Backup Retention] section below.
+
+Backup File Compression
+-----------------------
+
+The backup file can be compressed with one of the following algorithms: `lzo`
+footnote:[Lempel–Ziv–Oberhumer a lossless data compression algorithm
+https://en.wikipedia.org/wiki/Lempel-Ziv-Oberhumer], `gzip` footnote:[gzip -
+based on the DEFLATE algorithm https://en.wikipedia.org/wiki/Gzip] or `zstd`
+footnote:[Zstandard a lossless data compression algorithm
+https://en.wikipedia.org/wiki/Zstandard].
+
+Currently, Zstandard (zstd) is the fastest of these three algorithms.
+Multi-threading is another advantage of zstd over lzo and gzip. Lzo and gzip
+are more widely used and often installed by default.
+
+You can install pigz footnote:[pigz - parallel implementation of gzip
+https://zlib.net/pigz/] as a drop-in replacement for gzip to provide better
+performance due to multi-threading. For pigz & zstd, the amount of
+threads/cores can be adjusted. See the
+xref:vzdump_configuration[configuration options] below.
+
+The extension of the backup file name can usually be used to determine which
+compression algorithm has been used to create the backup.
+
+|===
+|.zst | Zstandard (zstd) compression
+|.gz or .tgz | gzip compression
+|.lzo | lzo compression
+|===
+
+If the backup file name doesn't end with one of the above file extensions, then
+it was not compressed by vzdump.
+
+Backup Encryption
+-----------------
+
+For Proxmox Backup Server storages, you can optionally set up client-side
+encryption of backups, see xref:storage_pbs_encryption[the corresponding section.]
+
+[[vzdump_retention]]
+Backup Retention
+----------------
+
+With the `prune-backups` option you can specify which backups you want to keep
+in a flexible manner. The following retention options are available:
+
+`keep-all <boolean>` ::
+Keep all backups. If this is `true`, no other options can be set.
+
+`keep-last <N>` ::
+Keep the last `<N>` backups.
+
+`keep-hourly <N>` ::
+Keep backups for the last `<N>` hours. If there is more than one
+backup for a single hour, only the latest is kept.
+
+`keep-daily <N>` ::
+Keep backups for the last `<N>` days. If there is more than one
+backup for a single day, only the latest is kept.
+
+`keep-weekly <N>` ::
+Keep backups for the last `<N>` weeks. If there is more than one
+backup for a single week, only the latest is kept.
+
+NOTE: Weeks start on Monday and end on Sunday. The software uses the
+`ISO week date`-system and handles weeks at the end of the year correctly.
+
+`keep-monthly <N>` ::
+Keep backups for the last `<N>` months. If there is more than one
+backup for a single month, only the latest is kept.
+
+`keep-yearly <N>` ::
+Keep backups for the last `<N>` years. If there is more than one
+backup for a single year, only the latest is kept.
+
+The retention options are processed in the order given above. Each option
+only covers backups within its time period. The next option does not take care
+of already covered backups. It will only consider older backups.
+
+Specify the retention options you want to use as a
+comma-separated list, for example:
+
+ # vzdump 777 --prune-backups keep-last=3,keep-daily=13,keep-yearly=9
+
+While you can pass `prune-backups` directly to `vzdump`, it is often more
+sensible to configure the setting on the storage level, which can be done via
+the web interface.
+
+NOTE: The old `maxfiles` option is deprecated and should be replaced either by
+`keep-last` or, in case `maxfiles` was `0` for unlimited retention, by
+`keep-all`.
+
+
+Prune Simulator
+~~~~~~~~~~~~~~~
+
+You can use the https://pbs.proxmox.com/docs/prune-simulator[prune simulator
+of the Proxmox Backup Server documentation] to explore the effect of different
+retention options with various backup schedules.
+
+Retention Settings Example
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+The backup frequency and retention of old backups may depend on how often data
+changes, and how important an older state may be, in a specific work load.
+When backups act as a company's document archive, there may also be legal
+requirements for how long backups must be kept.
+
+For this example, we assume that you are doing daily backups, have a retention
+period of 10 years, and the period between backups stored gradually grows.
+
+`keep-last=3` - even if only daily backups are taken, an admin may want to
+ create an extra one just before or after a big upgrade. Setting keep-last
+ ensures this.
+
+`keep-hourly` is not set - for daily backups this is not relevant. You cover
+ extra manual backups already, with keep-last.
+
+`keep-daily=13` - together with keep-last, which covers at least one
+ day, this ensures that you have at least two weeks of backups.
+
+`keep-weekly=8` - ensures that you have at least two full months of
+ weekly backups.
+
+`keep-monthly=11` - together with the previous keep settings, this
+ ensures that you have at least a year of monthly backups.
+
+`keep-yearly=9` - this is for the long term archive. As you covered the
+ current year with the previous options, you would set this to nine for the
+ remaining ones, giving you a total of at least 10 years of coverage.
+
+We recommend that you use a higher retention period than is minimally required
+by your environment; you can always reduce it if you find it is unnecessarily
+high, but you cannot recreate backups once they have been removed.
+
+[[vzdump_protection]]
+Backup Protection
+-----------------
+
+You can mark a backup as `protected` to prevent its removal. Attempting to
+remove a protected backup via {pve}'s UI, CLI or API will fail. However, this
+is enforced by {pve} and not the file-system, that means that a manual removal
+of a backup file itself is still possible for anyone with write access to the
+underlying backup storage.
+
+NOTE: Protected backups are ignored by pruning and do not count towards the
+retention settings.
+
+For filesystem-based storages, the protection is implemented via a sentinel file
+`<backup-name>.protected`. For Proxmox Backup Server, it is handled on the
+server side (available since Proxmox Backup Server version 2.1).
+
+[[vzdump_restore]]
Restore
-------
-The resulting archive files can be restored with the following programs.
+A backup archive can be restored through the {pve} web GUI or through the
+following CLI tools:
`pct restore`:: Container restore utility
-`qmrestore`:: QemuServer restore utility
+`qmrestore`:: Virtual Machine restore utility
For details see the corresponding manual pages.
+Bandwidth Limit
+~~~~~~~~~~~~~~~
+
+Restoring one or more big backups may need a lot of resources, especially
+storage bandwidth for both reading from the backup storage and writing to
+the target storage. This can negatively affect other virtual guests as access
+to storage can get congested.
+
+To avoid this you can set bandwidth limits for a backup job. {pve}
+implements two kinds of limits for restoring and archive:
+
+* per-restore limit: denotes the maximal amount of bandwidth for
+ reading from a backup archive
+
+* per-storage write limit: denotes the maximal amount of bandwidth used for
+ writing to a specific storage
+
+The read limit indirectly affects the write limit, as we cannot write more
+than we read. A smaller per-job limit will overwrite a bigger per-storage
+limit. A bigger per-job limit will only overwrite the per-storage limit if
+you have `Data.Allocate' permissions on the affected storage.
+
+You can use the `--bwlimit <integer>` option from the restore CLI commands
+to set up a restore job specific bandwidth limit. Kibit/s is used as unit
+for the limit, this means passing `10240' will limit the read speed of the
+backup to 10 MiB/s, ensuring that the rest of the possible storage bandwidth
+is available for the already running virtual guests, and thus the backup
+does not impact their operations.
+
+NOTE: You can use `0` for the `bwlimit` parameter to disable all limits for
+a specific restore job. This can be helpful if you need to restore a very
+important virtual guest as fast as possible. (Needs `Data.Allocate'
+permissions on storage)
+
+Most times your storage's generally available bandwidth stays the same over
+time, thus we implemented the possibility to set a default bandwidth limit
+per configured storage, this can be done with:
+
+----
+# pvesm set STORAGEID --bwlimit restore=KIBs
+----
+
+Live-Restore
+~~~~~~~~~~~~
+
+Restoring a large backup can take a long time, in which a guest is still
+unavailable. For VM backups stored on a Proxmox Backup Server, this wait
+time can be mitigated using the live-restore option.
+
+Enabling live-restore via either the checkbox in the GUI or the `--live-restore`
+argument of `qmrestore` causes the VM to start as soon as the restore
+begins. Data is copied in the background, prioritizing chunks that the VM is
+actively accessing.
+
+Note that this comes with two caveats:
+
+* During live-restore, the VM will operate with limited disk read speeds, as
+ data has to be loaded from the backup server (once loaded, it is immediately
+ available on the destination storage however, so accessing data twice only
+ incurs the penalty the first time). Write speeds are largely unaffected.
+* If the live-restore fails for any reason, the VM will be left in an
+ undefined state - that is, not all data might have been copied from the
+ backup, and it is _most likely_ not possible to keep any data that was written
+ during the failed restore operation.
+
+This mode of operation is especially useful for large VMs, where only a small
+amount of data is required for initial operation, e.g. web servers - once the OS
+and necessary services have been started, the VM is operational, while the
+background task continues copying seldom used data.
+
+Single File Restore
+~~~~~~~~~~~~~~~~~~~
+
+The 'File Restore' button in the 'Backups' tab of the storage GUI can be used to
+open a file browser directly on the data contained in a backup. This feature
+is only available for backups on a Proxmox Backup Server.
+
+For containers, the first layer of the file tree shows all included 'pxar'
+archives, which can be opened and browsed freely. For VMs, the first layer shows
+contained drive images, which can be opened to reveal a list of supported
+storage technologies found on the drive. In the most basic case, this will be an
+entry called 'part', representing a partition table, which contains entries for
+each partition found on the drive. Note that for VMs, not all data might be
+accessible (unsupported guest file systems, storage technologies, etc...).
+
+Files and directories can be downloaded using the 'Download' button, the latter
+being compressed into a zip archive on the fly.
+
+To enable secure access to VM images, which might contain untrusted data, a
+temporary VM (not visible as a guest) is started. This does not mean that data
+downloaded from such an archive is inherently safe, but it avoids exposing the
+hypervisor system to danger. The VM will stop itself after a timeout. This
+entire process happens transparently from a user's point of view.
+
+[[vzdump_configuration]]
Configuration
-------------
# vzdump 777 --exclude-path /tmp/ --exclude-path '/var/foo*'
-(only excludes tmp directories)
+excludes the directory `/tmp/` and any file or directory named `/var/foo`,
+`/var/foobar`, and so on.
+
+Paths that do not start with a `/` are not anchored to the container's root,
+but will match relative to any subdirectory. For example:
+
+ # vzdump 777 --exclude-path bar
+
+excludes any file or directory named `/bar`, `/var/bar`, `/var/foo/bar`, and
+so on, but not `/bar2`.
Configuration files are also stored inside the backup archive
(in `./etc/vzdump/`) and will be correctly restored.
projects / pve-docs.git / blobdiff