]>
Commit | Line | Data |
---|---|---|
c024f553 DM |
1 | ifdef::manvolnum[] |
2 | pvesr(1) | |
3 | ======== | |
4 | :pve-toplevel: | |
5 | ||
6 | NAME | |
7 | ---- | |
8 | ||
236bec37 | 9 | pvesr - Proxmox VE Storage Replication |
c024f553 DM |
10 | |
11 | SYNOPSIS | |
12 | -------- | |
13 | ||
14 | include::pvesr.1-synopsis.adoc[] | |
15 | ||
16 | DESCRIPTION | |
17 | ----------- | |
18 | endif::manvolnum[] | |
19 | ||
20 | ifndef::manvolnum[] | |
21 | Storage Replication | |
22 | =================== | |
23 | :pve-toplevel: | |
24 | endif::manvolnum[] | |
25 | ||
45c218cf DM |
26 | The `pvesr` command line tool manages the {PVE} storage replication |
27 | framework. Storage replication brings redundancy for guests using | |
28 | local storage's and reduces migration time when you migrate a guest. | |
29 | ||
30 | It replicates virtual disks to another node so that all data is | |
31 | available without using shared storage. Replication uses storage | |
32 | snapshots to minimize traffic sent over the network. New data is sent | |
33 | incrementally after the initial sync. In the case of a node failure, | |
34 | your guest data is still available on the replicated node. | |
236bec37 WL |
35 | |
36 | The minimal replication interval are 1 minute and the maximal interval is once a week. | |
37 | Interval schedule format is a subset of `systemd` calendar events. | |
38 | Every interval time your guest vdisk data will be synchronized, | |
39 | but only the new data will replicated. This reduce the amount of data to a minimum. | |
40 | New data are data what are written to the vdisk after the last replication. | |
41 | ||
42 | Every guest can replicate to many target nodes, but only one replication job per target node is allowed. | |
43 | ||
44 | The migration of guests, where storage replication is activated, is currently only offline possible. | |
45 | When the guest will migrate to the target of the replication, only the delta of the data must migrated and | |
46 | the replication direction will switched automatically in the opposite direction. | |
47 | If you migrate to a node where you do not replicate, it will send the whole vdisk data to the new node and after the migration it continuous the replication job as usually. | |
48 | ||
49 | WARNING: High-Availability is possible with Storage Replication but this can lead to lose data. So be aware of this problem before you use this combination. | |
50 | ||
51 | Supported Storage Types | |
52 | ----------------------- | |
53 | ||
54 | .Storage Types | |
55 | [width="100%",options="header"] | |
56 | |============================================ | |
57 | |Description |PVE type |Snapshots|Stable | |
58 | |ZFS (local) |zfspool |yes |yes | |
59 | |============================================ | |
60 | ||
61 | Schedule | |
62 | -------- | |
63 | ||
64 | Proxmox VE has a very flexible replication scheduler with will explained in detail here. | |
65 | ||
66 | A schedule string has following format. | |
67 | ||
68 | [day of the week] <time> | |
69 | ||
70 | Day of the week are taken of sun, mon, tue, wed, thu, fri and sat. | |
71 | It is allowed to make a list of this token. | |
72 | Or alternative you can use a span. | |
73 | ||
74 | The second part are the time, here the string contains two sub-string separate by `:`. | |
75 | First comes the hours, than the minutes. | |
76 | 0..23 are the allowed values for hours and 0..59 for minutes. | |
77 | If hours is missing it will substitute with `*`, this also counts for the day of the week. | |
78 | The `*` will be interpreted as wildcard. In case of the "all day of the week" it would be sun..sat, for the hours 0..23 or for the minutes 0..59. | |
79 | ||
80 | Examples: | |
81 | ~~~~~~~~~ | |
82 | ||
83 | .Schedule Examples | |
84 | [width="100%",options="header"] | |
85 | |============================================ | |
86 | |String |Meaning | |
87 | |"mon,tue,wed,thu,fri" => "mon..fir" |All working days at 0:00. | |
88 | |"sat,sun" => "sat..sun" |Only on weekend at 0:00. | |
89 | |"mon,wed,fri" |Only on Monday, Wednesday and Friday at 0:00. | |
90 | |"12:05" => "sun..sat 12:05" |All weekdays at 12:05 PM. | |
91 | |"*/5" |All five minutes on all weekdays. | |
92 | |"mon..wed 30/10" |Monday, Tuesday, Wednesday at all hours and 30, 40, 50 minutes. | |
93 | |"fri 12..13:5/20" |Friday at 12:05, 12:25, 12:45, 13:05, 13:25 and 13:45. | |
94 | |"12/2:05" |All weekdays every 2 hours start at 12:05 until 22:05 | |
95 | |============================================ | |
96 | The simplest case is the `*` which means every min what is the minimum interval. | |
97 | ||
98 | ||
99 | Error State | |
100 | ----------- | |
101 | ||
102 | It can happen that a job will have a error. | |
103 | Common issues for example: | |
104 | ||
105 | Network is not working. | |
106 | ||
107 | No space on the disk. | |
108 | ... | |
109 | ||
110 | In this case storage replication will retry every 30 miuntes. | |
111 | ||
112 | ||
113 | Migrating a guest in case of Error | |
114 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
115 | ||
116 | Assumed that you have two guests (KVM 100| LXC 200) running on node A (pve1) and replicate to node B (pve2). | |
117 | Node A is fails and can not get back online. Now you have to migrate the guest to Node B manually. | |
118 | ||
119 | Connect to node B with ssh. | |
120 | ||
121 | - check if that the cluster has quorum. | |
122 | ||
123 | pvecm status | |
124 | ||
125 | Is quorum is ok continuing with the skip[ the next step. | |
126 | ||
127 | - If you have no quorum, it is important to make your pmxcfs writable. | |
128 | ||
129 | WARNING: If expected is set, please avoid large changes on the nodes before the cluster is quorate online because there is a little change to a split brain. | |
130 | ||
131 | pvecm expected 1 | |
132 | ||
133 | - move the guest config form the origin node A to node B. | |
134 | cd /etc/pve/node && mv pve1/qemu-server/100.conf pve2/qemu-server/100.conf | |
135 | cd /etc/pve/node && mv pve1/lxc/200.conf pve2/lxc/100.conf | |
136 | ||
137 | - Now you can start the guests. | |
138 | ||
139 | qm start 100 | |
140 | pct start 100 | |
141 | ||
142 | ||
143 | Create Replication | |
144 | ------------------ | |
145 | ||
146 | [thumbnail="gui-create-replica-on-node.png"] | |
147 | ||
148 | You can easy create a replication job on the GUI or as well on the command line. | |
149 | There are only a few setting to configure. | |
150 | ||
151 | The only required parameter on the GUI is the target node. | |
152 | If you create a job on the command line also a unique id is needed. | |
153 | This id will automatically generated in case of GUI created jobs. | |
154 | The id contains the VMID and a consecutive number separated by a minus. | |
155 | All other parameters are optional. | |
156 | ||
157 | Update Replication Job | |
158 | ---------------------- | |
159 | ||
160 | For now is possible to update rate limit and the scheduler. | |
161 | It can be done on the GUI or on the cli. | |
162 | ||
163 | ||
164 | Using the Command Line Interface | |
165 | -------------------------------- | |
166 | ||
167 | Examples | |
168 | ~~~~~~~~ | |
169 | ||
170 | Create a replication job for VM 100 first job, which will run all 5 min with max bandwidth of 10MByte/s. | |
171 | ||
172 | pvesr create-local-job 100-0 pve1 --schedule */5 --rate 10 | |
173 | ||
174 | Disable a running job for ID 100-0 | |
175 | ||
176 | pvesr disable 100-0 | |
177 | ||
178 | Enable a running job for ID 100-0 | |
179 | ||
180 | pvesr disable 100-0 | |
181 | ||
182 | Change the schedule interval to once a hour | |
183 | ||
184 | pvesr update 100-0 --schedule '*/00' | |
c024f553 DM |
185 | |
186 | ifdef::manvolnum[] | |
187 | include::pve-copyright.adoc[] | |
188 | endif::manvolnum[] |