]>
Commit | Line | Data |
---|---|---|
d69aed69 DM |
1 | Efficient VM backup for qemu |
2 | ||
3 | =Requirements= | |
4 | ||
5 | * Backup to a single archive file | |
6 | * Backup needs to contain all data to restore VM (full backup) | |
7 | * Do not depend on storage type or image format | |
8 | * Avoid use of temporary storage | |
9 | * store sparse images efficiently | |
10 | ||
11 | =Introduction= | |
12 | ||
13 | Most VM backup solutions use some kind of snapshot to get a consistent | |
14 | VM view at a specific point in time. For example, we previously used | |
15 | LVM to create a snapshot of all used VM images, which are then copied | |
16 | into a tar file. | |
17 | ||
18 | That basically means that any data written during backup involve | |
19 | considerable overhead. For LVM we get the following steps: | |
20 | ||
21 | 1.) read original data (VM write) | |
22 | 2.) write original data into snapshot (VM write) | |
23 | 3.) write new data (VM write) | |
24 | 4.) read data from snapshot (backup) | |
25 | 5.) write data from snapshot into tar file (backup) | |
26 | ||
27 | Another approach to backup VM images is to create a new qcow2 image | |
28 | which use the old image as base. During backup, writes are redirected | |
29 | to the new image, so the old image represents a 'snapshot'. After | |
30 | backup, data need to be copied back from new image into the old | |
31 | one (commit). So a simple write during backup triggers the following | |
32 | steps: | |
33 | ||
34 | 1.) write new data to new image (VM write) | |
35 | 2.) read data from old image (backup) | |
36 | 3.) write data from old image into tar file (backup) | |
37 | ||
38 | 4.) read data from new image (commit) | |
39 | 5.) write data to old image (commit) | |
40 | ||
41 | This is in fact the same overhead as before. Other tools like qemu | |
42 | livebackup produces similar overhead (2 reads, 3 writes). | |
43 | ||
44 | Some storage types/formats supports internal snapshots using some kind | |
45 | of reference counting (rados, sheepdog, dm-thin, qcow2). It would be possible | |
46 | to use that for backups, but for now we want to be storage-independent. | |
47 | ||
48 | =Make it more efficient= | |
49 | ||
50 | The be more efficient, we simply need to avoid unnecessary steps. The | |
51 | following steps are always required: | |
52 | ||
53 | 1.) read old data before it gets overwritten | |
54 | 2.) write that data into the backup archive | |
55 | 3.) write new data (VM write) | |
56 | ||
57 | As you can see, this involves only one read, and two writes. | |
58 | ||
59 | To make that work, our backup archive need to be able to store image | |
60 | data 'out of order'. It is important to notice that this will not work | |
61 | with traditional archive formats like tar. | |
62 | ||
63 | During backup we simply intercept writes, then read existing data and | |
64 | store that directly into the archive. After that we can continue the | |
65 | write. | |
66 | ||
67 | ==Advantages== | |
68 | ||
69 | * very good performance (1 read, 2 writes) | |
70 | * works on any storage type and image format. | |
71 | * avoid usage of temporary storage | |
72 | * we can define a new and simple archive format, which is able to | |
73 | store sparse files efficiently. | |
74 | ||
75 | Note: Storing sparse files is a mess with existing archive | |
76 | formats. For example, tar requires information about holes at the | |
77 | beginning of the archive. | |
78 | ||
79 | ==Disadvantages== | |
80 | ||
81 | * we need to define a new archive format | |
82 | ||
83 | Note: Most existing archive formats are optimized to store small files | |
84 | including file attributes. We simply do not need that for VM archives. | |
85 | ||
86 | * archive contains data 'out of order' | |
87 | ||
88 | If you want to access image data in sequential order, you need to | |
89 | re-order archive data. It would be possible to to that on the fly, | |
90 | using temporary files. | |
91 | ||
92 | Fortunately, a normal restore/extract works perfectly with 'out of | |
93 | order' data, because the target files are seekable. | |
94 | ||
95 | * slow backup storage can slow down VM during backup | |
96 | ||
97 | It is important to note that we only do sequential writes to the | |
98 | backup storage. Furthermore one can compress the backup stream. IMHO, | |
99 | it is better to slow down the VM a bit. All other solutions creates | |
100 | large amounts of temporary data during backup. | |
101 | ||
102 | =Archive format requirements= | |
103 | ||
104 | The basic requirement for such new format is that we can store image | |
105 | date 'out of order'. It is also very likely that we have less than 256 | |
106 | drives/images per VM, and we want to be able to store VM configuration | |
107 | files. | |
108 | ||
109 | We have defined a very simply format with those properties, see: | |
110 | ||
e0c71673 | 111 | https://git.proxmox.com/?p=pve-qemu-kvm.git;a=blob;f=vma_spec.txt; |
d69aed69 DM |
112 | |
113 | Please let us know if you know an existing format which provides the | |
114 | same functionality. | |
115 | ||
116 |