pvesr.adoc

   1 ifdef::manvolnum[]
   2 pvesr(1)
   3 ========
   4 :pve-toplevel:
   5
   6 NAME
   7 ----
   8
   9 pvesr - Proxmox VE Storage Replication
  10
  11 SYNOPSIS
  12 --------
  13
  14 include::pvesr.1-synopsis.adoc[]
  15
  16 DESCRIPTION
  17 -----------
  18 endif::manvolnum[]
  19
  20 ifndef::manvolnum[]
  21 Storage Replication
  22 ===================
  23 :pve-toplevel:
  24 endif::manvolnum[]
  25
  26 The `pvesr` command line tool manages the {PVE} storage replication
  27 framework. Storage replication brings redundancy for guests using
  28 local storage's and reduces migration time when you migrate a guest.
  29
  30 It replicates virtual disks to another node so that all data is
  31 available without using shared storage. Replication uses storage
  32 snapshots to minimize traffic sent over the network. New data is sent
  33 incrementally after the initial sync. In the case of a node failure,
  34 your guest data is still available on the replicated node.
  35
  36 The minimal replication interval are 1 minute and the maximal interval is once a week.
  37 Interval schedule format is a subset of `systemd` calendar events.
  38 Every interval time your guest vdisk data will be synchronized,
  39 but only the new data will replicated. This reduce the amount of data to a minimum.
  40 New data are data what are written to the vdisk after the last replication.
  41
  42 Every guest can replicate to many target nodes, but only one replication job per target node is allowed.
  43
  44 The migration of guests, where storage replication is activated, is currently only offline possible.
  45 When the guest will migrate to the target of the replication, only the delta of the data must migrated and
  46 the replication direction will switched automatically in the opposite direction.
  47 If you migrate to a node where you do not replicate, it will send the whole vdisk data to the new node and after the migration it continuous the replication job as usually.
  48
  49 WARNING: High-Availability is possible with Storage Replication but this can lead to lose data. So be aware of this problem before you use this combination.
  50
  51 Supported Storage Types
  52 -----------------------
  53
  54 .Storage Types
  55 [width="100%",options="header"]
  56 |============================================
  57 |Description    |PVE type    |Snapshots|Stable
  58 |ZFS (local)    |zfspool     |yes      |yes
  59 |============================================
  60
  61 Schedule
  62 --------
  63
  64 Proxmox VE has a very flexible replication scheduler with will explained in detail here.
  65
  66 A schedule string has following format.
  67
  68 [day of the week] <time>
  69
  70 Day of the week are taken of sun, mon, tue, wed, thu, fri and sat.
  71 It is allowed to make a list of this token.
  72 Or alternative you can use a span.
  73
  74 The second part are the time, here the string contains two sub-string separate by `:`.
  75 First comes the hours, than the minutes.
  76 0..23 are the allowed values for hours and 0..59 for minutes.
  77 If hours is missing it will substitute with `*`, this also counts for the day of the week.
  78 The `*` will be interpreted as wildcard. In case of the "all day of the week" it would be sun..sat, for the hours 0..23 or for the minutes 0..59.
  79
  80 Examples:
  81 ~~~~~~~~~
  82
  83 .Schedule Examples
  84 [width="100%",options="header"]
  85 |============================================
  86 |String                                 |Meaning
  87 |"mon,tue,wed,thu,fri" => "mon..fir"    |All working days at 0:00.
  88 |"sat,sun" =>  "sat..sun"               |Only on weekend at 0:00.
  89 |"mon,wed,fri"                          |Only on Monday, Wednesday and Friday at 0:00.
  90 |"12:05" => "sun..sat 12:05"            |All weekdays at 12:05 PM.
  91 |"*/5"                                  |All five minutes on all weekdays.
  92 |"mon..wed 30/10"                       |Monday, Tuesday, Wednesday at all hours and 30, 40, 50 minutes.
  93 |"fri 12..13:5/20"                      |Friday at 12:05, 12:25, 12:45, 13:05, 13:25 and 13:45.
  94 |"12/2:05"                              |All weekdays every 2 hours start at 12:05 until 22:05
  95 |============================================
  96 The simplest case is the `*` which means every min what is the minimum interval.
  97
  98
  99 Error State
 100 -----------
 101
 102 It can happen that a job will have a error.
 103 Common issues for example:
 104
 105 Network is not working.
 106
 107 No space on the disk.
 108 ...
 109
 110 In this case storage replication will retry every 30 miuntes.
 111
 112
 113 Migrating a guest in case of Error
 114 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 115
 116 Assumed that you have two guests (KVM 100| LXC 200) running on node A (pve1) and replicate to node B (pve2).
 117 Node A is fails and can not get back online. Now you have to migrate the guest to Node B manually.
 118
 119 Connect to node B with ssh.
 120
 121 - check if that the cluster has quorum.
 122
 123  pvecm status
 124
 125 Is quorum is ok continuing with the skip[ the next step.
 126
 127 - If you have no quorum, it is important to make your pmxcfs writable.
 128
 129 WARNING: If expected is set, please avoid large changes on the nodes before the cluster is quorate online because there is a little change to a split brain.
 130
 131  pvecm expected 1
 132
 133 - move the guest config form the origin node A to node B.
 134  cd /etc/pve/node && mv pve1/qemu-server/100.conf pve2/qemu-server/100.conf
 135  cd /etc/pve/node && mv pve1/lxc/200.conf pve2/lxc/100.conf
 136
 137 - Now you can start the guests.
 138
 139  qm start 100
 140  pct start 100
 141
 142
 143 Create Replication
 144 ------------------
 145
 146 [thumbnail="gui-create-replica-on-node.png"]
 147
 148 You can easy create a replication job on the GUI or as well on the command line.
 149 There are only a few setting to configure.
 150
 151 The only required parameter on the GUI is the target node.
 152 If you create a job on the command line also a unique id is needed.
 153 This id will automatically generated in case of GUI created jobs.
 154 The id contains the VMID and a consecutive number separated by a minus.
 155 All other parameters are optional.
 156
 157 Update Replication Job
 158 ----------------------
 159
 160 For now is possible to update rate limit and the scheduler.
 161 It can be done on the GUI or on the cli.
 162
 163
 164 Using the Command Line Interface
 165 --------------------------------
 166
 167 Examples
 168 ~~~~~~~~
 169
 170 Create a replication job for VM 100 first job, which will run all 5 min with max bandwidth of 10MByte/s.
 171
 172  pvesr create-local-job 100-0 pve1 --schedule */5 --rate 10
 173
 174 Disable a running job for ID 100-0
 175
 176  pvesr disable 100-0
 177
 178 Enable a running job for ID 100-0
 179
 180  pvesr disable 100-0
 181
 182 Change the schedule interval to once a hour
 183
 184  pvesr update 100-0 --schedule '*/00'
 185
 186 ifdef::manvolnum[]
 187 include::pve-copyright.adoc[]
 188 endif::manvolnum[]