+[[chapter_pvesr]]
ifdef::manvolnum[]
pvesr(1)
========
NAME
----
-pvesr - Proxmox VE Storage Replication
+pvesr - Proxmox VE Storage Replication
SYNOPSIS
--------
:pve-toplevel:
endif::manvolnum[]
-The {PVE} storage replication tool (`pvesr`) ...
+The `pvesr` command-line tool manages the {PVE} storage replication
+framework. Storage replication brings redundancy for guests using
+local storage and reduces migration time.
+
+It replicates guest volumes to another node so that all data is available
+without using shared storage. Replication uses snapshots to minimize traffic
+sent over the network. Therefore, new data is sent only incrementally after
+the initial full sync. In the case of a node failure, your guest data is
+still available on the replicated node.
+
+The replication is done automatically in configurable intervals.
+The minimum replication interval is one minute, and the maximal interval
+once a week. The format used to specify those intervals is a subset of
+`systemd` calendar events, see
+xref:pvesr_schedule_time_format[Schedule Format] section:
+
+It is possible to replicate a guest to multiple target nodes,
+but not twice to the same target node.
+
+Each replications bandwidth can be limited, to avoid overloading a storage
+or server.
+
+Only changes since the last replication (so-called `deltas`) need to be
+transferred if the guest is migrated to a node to which it already is
+replicated. This reduces the time needed significantly. The replication
+direction automatically switches if you migrate a guest to the replication
+target node.
+
+For example: VM100 is currently on `nodeA` and gets replicated to `nodeB`.
+You migrate it to `nodeB`, so now it gets automatically replicated back from
+`nodeB` to `nodeA`.
+
+If you migrate to a node where the guest is not replicated, the whole disk
+data must send over. After the migration, the replication job continues to
+replicate this guest to the configured nodes.
+
+[IMPORTANT]
+====
+High-Availability is allowed in combination with storage replication, but there
+may be some data loss between the last synced time and the time a node failed.
+====
+
+Supported Storage Types
+-----------------------
+
+.Storage Types
+[width="100%",options="header"]
+|=============================================
+|Description |Plugin type |Snapshots|Stable
+|ZFS (local) |zfspool |yes |yes
+|=============================================
+
+[[pvesr_schedule_time_format]]
+Schedule Format
+---------------
+Replication uses xref:chapter_calendar_events[calendar events] for
+configuring the schedule.
+
+Error Handling
+--------------
+
+If a replication job encounters problems, it is placed in an error state.
+In this state, the configured replication intervals get suspended
+temporarily. The failed replication is repeatedly tried again in a
+30 minute interval.
+Once this succeeds, the original schedule gets activated again.
+
+Possible issues
+~~~~~~~~~~~~~~~
+
+Some of the most common issues are in the following list. Depending on your
+setup there may be another cause.
+
+* Network is not working.
+
+* No free space left on the replication target storage.
+
+* Storage with same storage ID available on the target node
+
+NOTE: You can always use the replication log to find out what is causing the problem.
+
+Migrating a guest in case of Error
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+// FIXME: move this to better fitting chapter (sysadmin ?) and only link to
+// it here
+
+In the case of a grave error, a virtual guest may get stuck on a failed
+node. You then need to move it manually to a working node again.
+
+Example
+~~~~~~~
+
+Let's assume that you have two guests (VM 100 and CT 200) running on node A
+and replicate to node B.
+Node A failed and can not get back online. Now you have to migrate the guest
+to Node B manually.
+
+- connect to node B over ssh or open its shell via the web UI
+
+- check if that the cluster is quorate
++
+----
+# pvecm status
+----
+
+- If you have no quorum, we strongly advise to fix this first and make the
+ node operable again. Only if this is not possible at the moment, you may
+ use the following command to enforce quorum on the current node:
++
+----
+# pvecm expected 1
+----
+
+WARNING: Avoid changes which affect the cluster if `expected votes` are set
+(for example adding/removing nodes, storages, virtual guests) at all costs.
+Only use it to get vital guests up and running again or to resolve the quorum
+issue itself.
+
+- move both guest configuration files form the origin node A to node B:
++
+----
+# mv /etc/pve/nodes/A/qemu-server/100.conf /etc/pve/nodes/B/qemu-server/100.conf
+# mv /etc/pve/nodes/A/lxc/200.conf /etc/pve/nodes/B/lxc/200.conf
+----
+
+- Now you can start the guests again:
++
+----
+# qm start 100
+# pct start 200
+----
+
+Remember to replace the VMIDs and node names with your respective values.
+
+Managing Jobs
+-------------
+
+[thumbnail="screenshot/gui-qemu-add-replication-job.png"]
+
+You can use the web GUI to create, modify, and remove replication jobs
+easily. Additionally, the command-line interface (CLI) tool `pvesr` can be
+used to do this.
+
+You can find the replication panel on all levels (datacenter, node, virtual
+guest) in the web GUI. They differ in which jobs get shown:
+all, node- or guest-specific jobs.
+
+When adding a new job, you need to specify the guest if not already selected
+as well as the target node. The replication
+xref:pvesr_schedule_time_format[schedule] can be set if the default of `all
+15 minutes` is not desired. You may impose a rate-limit on a replication
+job. The rate limit can help to keep the load on the storage acceptable.
+
+A replication job is identified by a cluster-wide unique ID. This ID is
+composed of the VMID in addition to a job number.
+This ID must only be specified manually if the CLI tool is used.
+
+
+Command-line Interface Examples
+-------------------------------
+
+Create a replication job which runs every 5 minutes with a limited bandwidth
+of 10 Mbps (megabytes per second) for the guest with ID 100.
+
+----
+# pvesr create-local-job 100-0 pve1 --schedule "*/5" --rate 10
+----
+
+Disable an active job with ID `100-0`.
+
+----
+# pvesr disable 100-0
+----
+
+Enable a deactivated job with ID `100-0`.
+
+----
+# pvesr enable 100-0
+----
+
+Change the schedule interval of the job with ID `100-0` to once per hour.
+
+----
+# pvesr update 100-0 --schedule '*/00'
+----
ifdef::manvolnum[]
include::pve-copyright.adoc[]
projects / pve-docs.git / blobdiff