debian/patches/0001-RFC-Efficient-VM-backup-for-qemu.patch

   1 From 793525f2a3b92fdc0ce27e48c3421171b87c367c Mon Sep 17 00:00:00 2001
   2 From: Dietmar Maurer <dietmar@proxmox.com>
   3 Date: Tue, 13 Nov 2012 09:24:50 +0100
   4 Subject: [PATCH v3 1/6] RFC: Efficient VM backup for qemu
   5
   6 This series provides a way to efficiently backup VMs.
   7
   8 * Backup to a single archive file
   9 * Backup contain all data to restore VM (full backup)
  10 * Do not depend on storage type or image format
  11 * Avoid use of temporary storage
  12 * store sparse images efficiently
  13
  14 The file docs/backup-rfc.txt contains more details.
  15
  16 Changes since v1:
  17
  18 * fix spelling errors
  19 * move BackupInfo from BDS to BackupBlockJob
  20 * introduce BackupDriver to allow more than one backup format
  21 * vma: add suport to store vmstate (size is not known in advance)
  22 * add ability to store VM state
  23
  24 Changes since v2:
  25
  26 * BackupDriver: remove cancel_cb
  27 * use enum for BackupFormat
  28 * vma: use bdrv_open instead of bdrv_file_open
  29
  30 Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
  31 ---
  32  docs/backup-rfc.txt |  119 +++++++++++++++++++++++++++++++++++++++++++++++++++
  33  1 files changed, 119 insertions(+), 0 deletions(-)
  34  create mode 100644 docs/backup-rfc.txt
  35
  36 diff --git a/docs/backup-rfc.txt b/docs/backup-rfc.txt
  37 new file mode 100644
  38 index 0000000..5b4b3df
  39 --- /dev/null
  40 +++ b/docs/backup-rfc.txt
  41 @@ -0,0 +1,119 @@
  42 +RFC: Efficient VM backup for qemu
  43 +
  44 +=Requirements=
  45 +
  46 +* Backup to a single archive file
  47 +* Backup needs to contain all data to restore VM (full backup)
  48 +* Do not depend on storage type or image format
  49 +* Avoid use of temporary storage
  50 +* store sparse images efficiently
  51 +
  52 +=Introduction=
  53 +
  54 +Most VM backup solutions use some kind of snapshot to get a consistent
  55 +VM view at a specific point in time. For example, we previously used
  56 +LVM to create a snapshot of all used VM images, which are then copied
  57 +into a tar file.
  58 +
  59 +That basically means that any data written during backup involve
  60 +considerable overhead. For LVM we get the following steps:
  61 +
  62 +1.) read original data (VM write)
  63 +2.) write original data into snapshot (VM write)
  64 +3.) write new data (VM write)
  65 +4.) read data from snapshot (backup)
  66 +5.) write data from snapshot into tar file (backup)
  67 +
  68 +Another approach to backup VM images is to create a new qcow2 image
  69 +which use the old image as base. During backup, writes are redirected
  70 +to the new image, so the old image represents a 'snapshot'. After
  71 +backup, data need to be copied back from new image into the old
  72 +one (commit). So a simple write during backup triggers the following
  73 +steps:
  74 +
  75 +1.) write new data to new image (VM write)
  76 +2.) read data from old image (backup)
  77 +3.) write data from old image into tar file (backup)
  78 +
  79 +4.) read data from new image (commit)
  80 +5.) write data to old image (commit)
  81 +
  82 +This is in fact the same overhead as before. Other tools like qemu
  83 +livebackup produces similar overhead (2 reads, 3 writes).
  84 +
  85 +Some storage types/formats supports internal snapshots using some kind
  86 +of reference counting (rados, sheepdog, dm-thin, qcow2). It would be possible
  87 +to use that for backups, but for now we want to be storage-independent.
  88 +
  89 +Note: It turned out that taking a qcow2 snapshot can take a very long
  90 +time on larger files.
  91 +
  92 +=Make it more efficient=
  93 +
  94 +The be more efficient, we simply need to avoid unnecessary steps. The
  95 +following steps are always required:
  96 +
  97 +1.) read old data before it gets overwritten
  98 +2.) write that data into the backup archive
  99 +3.) write new data (VM write)
 100 +
 101 +As you can see, this involves only one read, an two writes.
 102 +
 103 +To make that work, our backup archive need to be able to store image
 104 +data 'out of order'. It is important to notice that this will not work
 105 +with traditional archive formats like tar.
 106 +
 107 +During backup we simply intercept writes, then read existing data and
 108 +store that directly into the archive. After that we can continue the
 109 +write.
 110 +
 111 +==Advantages==
 112 +
 113 +* very good performance (1 read, 2 writes)
 114 +* works on any storage type and image format.
 115 +* avoid usage of temporary storage
 116 +* we can define a new and simple archive format, which is able to
 117 +  store sparse files efficiently.
 118 +
 119 +Note: Storing sparse files is a mess with existing archive
 120 +formats. For example, tar requires information about holes at the
 121 +beginning of the archive.
 122 +
 123 +==Disadvantages==
 124 +
 125 +* we need to define a new archive format
 126 +
 127 +Note: Most existing archive formats are optimized to store small files
 128 +including file attributes. We simply do not need that for VM archives.
 129 +
 130 +* archive contains data 'out of order'
 131 +
 132 +If you want to access image data in sequential order, you need to
 133 +re-order archive data. It would be possible to to that on the fly,
 134 +using temporary files.
 135 +
 136 +Fortunately, a normal restore/extract works perfectly with 'out of
 137 +order' data, because the target files are seekable.
 138 +
 139 +* slow backup storage can slow down VM during backup
 140 +
 141 +It is important to note that we only do sequential writes to the
 142 +backup storage. Furthermore one can compress the backup stream. IMHO,
 143 +it is better to slow down the VM a bit. All other solutions creates
 144 +large amounts of temporary data during backup.
 145 +
 146 +=Archive format requirements=
 147 +
 148 +The basic requirement for such new format is that we can store image
 149 +date 'out of order'. It is also very likely that we have less than 256
 150 +drives/images per VM, and we want to be able to store VM configuration
 151 +files.
 152 +
 153 +We have defined a very simply format with those properties, see:
 154 +
 155 +docs/specs/vma_spec.txt
 156 +
 157 +Please let us know if you know an existing format which provides the
 158 +same functionality.
 159 +
 160 +
 161 --
 162 1.7.2.5
 163