10 vzdump - Backup Utility for VMs and Containers
16 include::vzdump.1-synopsis.adoc[]
28 Backups are a requirement for any sensible IT deployment, and {pve}
29 provides a fully integrated solution, using the capabilities of each
30 storage and each guest system type. This allows the system
31 administrator to fine tune via the `mode` option between consistency
32 of the backups and downtime of the guest system.
34 {pve} backups are always full backups - containing the VM/CT
35 configuration and all data. Backups can be started via the GUI or via
36 the `vzdump` command-line tool.
40 Before a backup can run, a backup storage must be defined. Refer to the
41 xref:chapter_storage[storage documentation] on how to add a storage. It can
42 either be a Proxmox Backup Server storage, where backups are stored as
43 de-duplicated chunks and metadata, or a file-level storage, where backups are
44 stored as regular files. Using Proxmox Backup Server on a dedicated host is
45 recommended, because of its advanced features. Using an NFS server is a good
46 alternative. In both cases, you might want to save those backups later to a tape
47 drive, for off-site archiving.
51 Backup jobs can be scheduled so that they are executed automatically on specific
52 days and times, for selectable nodes and guest systems. See the
53 xref:vzdump_jobs[Backup Jobs] section for more.
58 There are several ways to provide consistency (option `mode`),
59 depending on the guest type.
61 .Backup modes for VMs:
65 This mode provides the highest consistency of the backup, at the cost
66 of a short downtime in the VM operation. It works by executing an
67 orderly shutdown of the VM, and then runs a background QEMU process to
68 backup the VM data. After the backup is started, the VM goes to full
69 operation mode if it was previously running. Consistency is guaranteed
70 by using the live backup feature.
74 This mode is provided for compatibility reason, and suspends the VM
75 before calling the `snapshot` mode. Since suspending the VM results in
76 a longer downtime and does not necessarily improve the data
77 consistency, the use of the `snapshot` mode is recommended instead.
81 This mode provides the lowest operation downtime, at the cost of a
82 small inconsistency risk. It works by performing a {pve} live
83 backup, in which data blocks are copied while the VM is running. If the
84 guest agent is enabled (`agent: 1`) and running, it calls
85 `guest-fsfreeze-freeze` and `guest-fsfreeze-thaw` to improve
88 A technical overview of the {pve} live backup for QemuServer can
90 https://git.proxmox.com/?p=pve-qemu.git;a=blob_plain;f=backup.txt[here].
92 NOTE: {pve} live backup provides snapshot-like semantics on any
93 storage type. It does not require that the underlying storage supports
94 snapshots. Also please note that since the backups are done via
95 a background QEMU process, a stopped VM will appear as running for a
96 short amount of time while the VM disks are being read by QEMU.
97 However the VM itself is not booted, only its disk(s) are read.
99 .Backup modes for Containers:
103 Stop the container for the duration of the backup. This potentially
104 results in a very long downtime.
108 This mode uses rsync to copy the container data to a temporary
109 location (see option `--tmpdir`). Then the container is suspended and
110 a second rsync copies changed files. After that, the container is
111 started (resumed) again. This results in minimal downtime, but needs
112 additional space to hold the container copy.
114 When the container is on a local file system and the target storage of
115 the backup is an NFS/CIFS server, you should set `--tmpdir` to reside on a
116 local file system too, as this will result in a many fold performance
117 improvement. Use of a local `tmpdir` is also required if you want to
118 backup a local container using ACLs in suspend mode if the backup
119 storage is an NFS server.
123 This mode uses the snapshotting facilities of the underlying
124 storage. First, the container will be suspended to ensure data consistency.
125 A temporary snapshot of the container's volumes will be made and the
126 snapshot content will be archived in a tar file. Finally, the temporary
127 snapshot is deleted again.
129 NOTE: `snapshot` mode requires that all backed up volumes are on a storage that
130 supports snapshots. Using the `backup=no` mount point option individual volumes
131 can be excluded from the backup (and thus this requirement).
133 // see PVE::VZDump::LXC::prepare()
134 NOTE: By default additional mount points besides the Root Disk mount point are
135 not included in backups. For volume mount points you can set the *Backup* option
136 to include the mount point in the backup. Device and bind mounts are never
137 backed up as their content is managed outside the {pve} storage library.
142 When a backup for a VM is started, QEMU will install a "copy-before-write"
143 filter in its block layer. This filter ensures that upon new guest writes, old
144 data still needed for the backup is sent to the backup target first. The guest
145 write blocks until this operation is finished so guest IO to not-yet-backed-up
146 sectors will be limited by the speed of the backup target.
148 With backup fleecing, such old data is cached in a fleecing image rather than
149 sent directly to the backup target. This can help guest IO performance and even
150 prevent hangs in certain scenarios, at the cost of requiring more storage space.
151 Use e.g. `vzdump 123 --fleecing enabled=1,storage=local-lvm` to enable backup
152 fleecing, with fleecing images created on the storage `local-lvm`.
154 The fleecing storage should be a fast local storage, with thin provisioning and
155 discard support. Examples are LVM-thin, RBD, ZFS with `sparse 1` in the storage
156 configuration, many file-based storages. Ideally, the fleecing storage is a
157 dedicated storage, so it running full will not affect other guests and just fail
158 the backup. Parts of the fleecing image that have been backed up will be
159 discarded to try and keep the space usage low.
161 For file-based storages that do not support discard (e.g. NFS before version
162 4.2), you should set `preallocation off` in the storage configuration. In
163 combination with `qcow2` (used automatically as the format for the fleecing
164 image when the storage supports it), this has the advantage that already
165 allocated parts of the image can be re-used later, which can still help save
166 quite a bit of space.
168 WARNING: On a storage that's not thinly provisioned, e.g. LVM or ZFS without the
169 `sparse` option, the full size of the original disk needs to be reserved for the
170 fleecing image up-front. On a thinly provisioned storage, the fleecing image can
171 grow to the same size as the original image only if the guest re-writes a whole
172 disk while the backup is busy with another disk.
177 Newer versions of vzdump encode the guest type and the
178 backup time into the filename, for example
180 vzdump-lxc-105-2009_10_09-11_04_43.tar
182 That way it is possible to store several backup in the same directory. You can
183 limit the number of backups that are kept with various retention options, see
184 the xref:vzdump_retention[Backup Retention] section below.
186 Backup File Compression
187 -----------------------
189 The backup file can be compressed with one of the following algorithms: `lzo`
190 footnote:[Lempel–Ziv–Oberhumer a lossless data compression algorithm
191 https://en.wikipedia.org/wiki/Lempel-Ziv-Oberhumer], `gzip` footnote:[gzip -
192 based on the DEFLATE algorithm https://en.wikipedia.org/wiki/Gzip] or `zstd`
193 footnote:[Zstandard a lossless data compression algorithm
194 https://en.wikipedia.org/wiki/Zstandard].
196 Currently, Zstandard (zstd) is the fastest of these three algorithms.
197 Multi-threading is another advantage of zstd over lzo and gzip. Lzo and gzip
198 are more widely used and often installed by default.
200 You can install pigz footnote:[pigz - parallel implementation of gzip
201 https://zlib.net/pigz/] as a drop-in replacement for gzip to provide better
202 performance due to multi-threading. For pigz & zstd, the amount of
203 threads/cores can be adjusted. See the
204 xref:vzdump_configuration[configuration options] below.
206 The extension of the backup file name can usually be used to determine which
207 compression algorithm has been used to create the backup.
210 |.zst | Zstandard (zstd) compression
211 |.gz or .tgz | gzip compression
212 |.lzo | lzo compression
215 If the backup file name doesn't end with one of the above file extensions, then
216 it was not compressed by vzdump.
221 For Proxmox Backup Server storages, you can optionally set up client-side
222 encryption of backups, see xref:storage_pbs_encryption[the corresponding section.]
228 [thumbnail="screenshot/gui-cluster-backup-overview.png"]
230 Besides triggering a backup manually, you can also setup periodic jobs that
231 backup all, or a selection of virtual guest to a storage. You can manage the
232 jobs in the UI under 'Datacenter' -> 'Backup' or via the `/cluster/backup` API
233 endpoint. Both will generate job entries in `/etc/pve/jobs.cfg`, which are
234 parsed and executed by the `pvescheduler` daemon.
236 [thumbnail="screenshot/gui-cluster-backup-edit-01-general.png", float="left"]
238 A job is either configured for all cluster nodes or a specific node, and is
239 executed according to a given schedule. The format for the schedule is very
240 similar to `systemd` calendar events, see the
241 xref:chapter_calendar_events[calendar events] section for details. The
242 'Schedule' field in the UI can be freely edited, and it contains several
243 examples that can be used as a starting point in its drop-down list.
245 You can configure job-specific xref:vzdump_retention[retention options]
246 overriding those from the storage or node configuration, as well as a
247 xref:vzdump_notes[template for notes] for additional information to be saved
248 together with the backup.
250 Since scheduled backups miss their execution when the host was offline or the
251 pvescheduler was disabled during the scheduled time, it is possible to configure
252 the behaviour for catching up. By enabling the `Repeat missed` option (in the
253 'Advanced' tab in the UI, `repeat-missed` in the config), you can tell the
254 scheduler that it should run missed jobs as soon as possible.
256 [thumbnail="screenshot/gui-cluster-backup-edit-04-advanced.png"]
258 There are a few settings for tuning backup performance (some of which are
259 exposed in the 'Advanced' tab in the UI). The most notable is `bwlimit` for
260 limiting IO bandwidth. The amount of threads used for the compressor can be
261 controlled with the `pigz` (replacing `gzip`), respectively, `zstd` setting.
262 Furthermore, there are `ionice` (when the BFQ scheduler is used) and, as part of
263 the `performance` setting, `max-workers` (affects VM backups only) and
264 `pbs-entries-max` (affects container backups only). See the
265 xref:vzdump_configuration[configuration options] for details.
271 With the `prune-backups` option you can specify which backups you want to keep
272 in a flexible manner.
274 [thumbnail="screenshot/gui-cluster-backup-edit-02-retention.png"]
276 The following retention options are available:
278 `keep-all <boolean>` ::
279 Keep all backups. If this is `true`, no other options can be set.
282 Keep the last `<N>` backups.
285 Keep backups for the last `<N>` hours. If there is more than one
286 backup for a single hour, only the latest is kept.
289 Keep backups for the last `<N>` days. If there is more than one
290 backup for a single day, only the latest is kept.
293 Keep backups for the last `<N>` weeks. If there is more than one
294 backup for a single week, only the latest is kept.
296 NOTE: Weeks start on Monday and end on Sunday. The software uses the
297 `ISO week date`-system and handles weeks at the end of the year correctly.
299 `keep-monthly <N>` ::
300 Keep backups for the last `<N>` months. If there is more than one
301 backup for a single month, only the latest is kept.
304 Keep backups for the last `<N>` years. If there is more than one
305 backup for a single year, only the latest is kept.
307 The retention options are processed in the order given above. Each option
308 only covers backups within its time period. The next option does not take care
309 of already covered backups. It will only consider older backups.
311 Specify the retention options you want to use as a
312 comma-separated list, for example:
314 # vzdump 777 --prune-backups keep-last=3,keep-daily=13,keep-yearly=9
316 While you can pass `prune-backups` directly to `vzdump`, it is often more
317 sensible to configure the setting on the storage level, which can be done via
320 NOTE: The old `maxfiles` option is deprecated and should be replaced either by
321 `keep-last` or, in case `maxfiles` was `0` for unlimited retention, by
328 You can use the https://pbs.proxmox.com/docs/prune-simulator[prune simulator
329 of the Proxmox Backup Server documentation] to explore the effect of different
330 retention options with various backup schedules.
332 Retention Settings Example
333 ~~~~~~~~~~~~~~~~~~~~~~~~~~
335 The backup frequency and retention of old backups may depend on how often data
336 changes, and how important an older state may be, in a specific work load.
337 When backups act as a company's document archive, there may also be legal
338 requirements for how long backups must be kept.
340 For this example, we assume that you are doing daily backups, have a retention
341 period of 10 years, and the period between backups stored gradually grows.
343 `keep-last=3` - even if only daily backups are taken, an admin may want to
344 create an extra one just before or after a big upgrade. Setting keep-last
347 `keep-hourly` is not set - for daily backups this is not relevant. You cover
348 extra manual backups already, with keep-last.
350 `keep-daily=13` - together with keep-last, which covers at least one
351 day, this ensures that you have at least two weeks of backups.
353 `keep-weekly=8` - ensures that you have at least two full months of
356 `keep-monthly=11` - together with the previous keep settings, this
357 ensures that you have at least a year of monthly backups.
359 `keep-yearly=9` - this is for the long term archive. As you covered the
360 current year with the previous options, you would set this to nine for the
361 remaining ones, giving you a total of at least 10 years of coverage.
363 We recommend that you use a higher retention period than is minimally required
364 by your environment; you can always reduce it if you find it is unnecessarily
365 high, but you cannot recreate backups once they have been removed.
367 [[vzdump_protection]]
371 You can mark a backup as `protected` to prevent its removal. Attempting to
372 remove a protected backup via {pve}'s UI, CLI or API will fail. However, this
373 is enforced by {pve} and not the file-system, that means that a manual removal
374 of a backup file itself is still possible for anyone with write access to the
375 underlying backup storage.
377 NOTE: Protected backups are ignored by pruning and do not count towards the
380 For filesystem-based storages, the protection is implemented via a sentinel file
381 `<backup-name>.protected`. For Proxmox Backup Server, it is handled on the
382 server side (available since Proxmox Backup Server version 2.1).
384 Use the storage option `max-protected-backups` to control how many protected
385 backups per guest are allowed on the storage. Use `-1` for unlimited. The
386 default is unlimited for users with `Datastore.Allocate` privilege and `5` for
393 You can add notes to backups using the 'Edit Notes' button in the UI or via the
396 [thumbnail="screenshot/gui-cluster-backup-edit-03-template.png"]
398 It is also possible to specify a template for generating notes dynamically for
399 a backup job and for manual backup. The template string can contain variables,
400 surrounded by two curly braces, which will be replaced by the corresponding
401 value when the backup is executed.
403 Currently supported are:
405 * `{{cluster}}` the cluster name, if any
406 * `{{guestname}}` the virtual guest's assigned name
407 * `{{node}}` the host name of the node the backup is being created
408 * `{{vmid}}` the numerical VMID of the guest
410 When specified via API or CLI, it needs to be a single line, where newline and
411 backslash need to be escaped as literal `\n` and `\\` respectively.
417 A backup archive can be restored through the {pve} web GUI or through the
421 `pct restore`:: Container restore utility
423 `qmrestore`:: Virtual Machine restore utility
425 For details see the corresponding manual pages.
430 Restoring one or more big backups may need a lot of resources, especially
431 storage bandwidth for both reading from the backup storage and writing to
432 the target storage. This can negatively affect other virtual guests as access
433 to storage can get congested.
435 To avoid this you can set bandwidth limits for a backup job. {pve}
436 implements two kinds of limits for restoring and archive:
438 * per-restore limit: denotes the maximal amount of bandwidth for
439 reading from a backup archive
441 * per-storage write limit: denotes the maximal amount of bandwidth used for
442 writing to a specific storage
444 The read limit indirectly affects the write limit, as we cannot write more
445 than we read. A smaller per-job limit will overwrite a bigger per-storage
446 limit. A bigger per-job limit will only overwrite the per-storage limit if
447 you have `Data.Allocate' permissions on the affected storage.
449 You can use the `--bwlimit <integer>` option from the restore CLI commands
450 to set up a restore job specific bandwidth limit. KiB/s is used as unit
451 for the limit, this means passing `10240' will limit the read speed of the
452 backup to 10 MiB/s, ensuring that the rest of the possible storage bandwidth
453 is available for the already running virtual guests, and thus the backup
454 does not impact their operations.
456 NOTE: You can use `0` for the `bwlimit` parameter to disable all limits for
457 a specific restore job. This can be helpful if you need to restore a very
458 important virtual guest as fast as possible. (Needs `Data.Allocate'
459 permissions on storage)
461 Most times your storage's generally available bandwidth stays the same over
462 time, thus we implemented the possibility to set a default bandwidth limit
463 per configured storage, this can be done with:
466 # pvesm set STORAGEID --bwlimit restore=KIBs
472 Restoring a large backup can take a long time, in which a guest is still
473 unavailable. For VM backups stored on a Proxmox Backup Server, this wait
474 time can be mitigated using the live-restore option.
476 Enabling live-restore via either the checkbox in the GUI or the `--live-restore`
477 argument of `qmrestore` causes the VM to start as soon as the restore
478 begins. Data is copied in the background, prioritizing chunks that the VM is
481 Note that this comes with two caveats:
483 * During live-restore, the VM will operate with limited disk read speeds, as
484 data has to be loaded from the backup server (once loaded, it is immediately
485 available on the destination storage however, so accessing data twice only
486 incurs the penalty the first time). Write speeds are largely unaffected.
487 * If the live-restore fails for any reason, the VM will be left in an
488 undefined state - that is, not all data might have been copied from the
489 backup, and it is _most likely_ not possible to keep any data that was written
490 during the failed restore operation.
492 This mode of operation is especially useful for large VMs, where only a small
493 amount of data is required for initial operation, e.g. web servers - once the OS
494 and necessary services have been started, the VM is operational, while the
495 background task continues copying seldom used data.
500 The 'File Restore' button in the 'Backups' tab of the storage GUI can be used to
501 open a file browser directly on the data contained in a backup. This feature
502 is only available for backups on a Proxmox Backup Server.
504 For containers, the first layer of the file tree shows all included 'pxar'
505 archives, which can be opened and browsed freely. For VMs, the first layer shows
506 contained drive images, which can be opened to reveal a list of supported
507 storage technologies found on the drive. In the most basic case, this will be an
508 entry called 'part', representing a partition table, which contains entries for
509 each partition found on the drive. Note that for VMs, not all data might be
510 accessible (unsupported guest file systems, storage technologies, etc...).
512 Files and directories can be downloaded using the 'Download' button, the latter
513 being compressed into a zip archive on the fly.
515 To enable secure access to VM images, which might contain untrusted data, a
516 temporary VM (not visible as a guest) is started. This does not mean that data
517 downloaded from such an archive is inherently safe, but it avoids exposing the
518 hypervisor system to danger. The VM will stop itself after a timeout. This
519 entire process happens transparently from a user's point of view.
521 NOTE: For troubleshooting purposes, each temporary VM instance generates a log
522 file in `/var/log/proxmox-backup/file-restore/`. The log file might contain
523 additional information in case an attempt to restore individual files or
524 accessing file systems contained in a backup archive fails.
526 [[vzdump_configuration]]
530 Global configuration is stored in `/etc/vzdump.conf`. The file uses a
531 simple colon separated key/value format. Each line has the following
536 Blank lines in the file are ignored, and lines starting with a `#`
537 character are treated as comments and are also ignored. Values from
538 this file are used as default, and can be overwritten on the command
541 We currently support the following options:
543 include::vzdump.conf.5-opts.adoc[]
546 .Example `vzdump.conf` Configuration
548 tmpdir: /mnt/fast_local_disk
549 storage: my_backup_storage
557 You can specify a hook script with option `--script`. This script is
558 called at various phases of the backup process, with parameters
559 accordingly set. You can find an example in the documentation
560 directory (`vzdump-hook-script.pl`).
565 NOTE: this option is only available for container backups.
567 `vzdump` skips the following files by default (disable with the option
574 You can also manually specify (additional) exclude paths, for example:
576 # vzdump 777 --exclude-path /tmp/ --exclude-path '/var/foo*'
578 excludes the directory `/tmp/` and any file or directory named `/var/foo`,
579 `/var/foobar`, and so on.
581 Paths that do not start with a `/` are not anchored to the container's root,
582 but will match relative to any subdirectory. For example:
584 # vzdump 777 --exclude-path bar
586 excludes any file or directory named `/bar`, `/var/bar`, `/var/foo/bar`, and
587 so on, but not `/bar2`.
589 Configuration files are also stored inside the backup archive
590 (in `./etc/vzdump/`) and will be correctly restored.
595 Simply dump guest 777 - no snapshot, just archive the guest private area and
596 configuration files to the default dump directory (usually
597 `/var/lib/vz/dump/`).
601 Use rsync and suspend/resume to create a snapshot (minimal downtime).
603 # vzdump 777 --mode suspend
605 Backup all guest systems and send notification mails to root and admin.
606 Due to `mailto` being set and `notification-mode` being set to `auto` by
607 default, the notification mails are sent via the system's `sendmail`
608 command instead of the notification system.
610 # vzdump --all --mode suspend --mailto root --mailto admin
612 Use snapshot mode (no downtime) and non-default dump directory.
614 # vzdump 777 --dumpdir /mnt/backup --mode snapshot
616 Backup more than one guest (selectively)
618 # vzdump 101 102 103 --mailto root
620 Backup all guests excluding 101 and 102
622 # vzdump --mode suspend --exclude 101,102
624 Restore a container to a new CT 600
626 # pct restore 600 /mnt/backup/vzdump-lxc-777.tar
628 Restore a QemuServer VM to VM 601
630 # qmrestore /mnt/backup/vzdump-qemu-888.vma 601
632 Clone an existing container 101 to a new container 300 with a 4GB root
633 file system, using pipes
635 # vzdump 101 --stdout | pct restore --rootfs 4 300 -
639 include::pve-copyright.adoc[]