]> git.proxmox.com Git - proxmox-backup.git/blob - docs/managing-remotes.rst
docs: document new include/exclude paramenter
[proxmox-backup.git] / docs / managing-remotes.rst
1 Managing Remotes & Sync
2 =======================
3
4 .. _backup_remote:
5
6 :term:`Remote`
7 --------------
8
9 A remote refers to a separate `Proxmox Backup`_ Server installation and a user
10 on that installation, from which you can `sync` datastores to a local datastore
11 with a `Sync Job`. You can configure remotes in the web interface, under
12 **Configuration -> Remotes**. Alternatively, you can use the ``remote``
13 subcommand. The configuration information for remotes is stored in the file
14 ``/etc/proxmox-backup/remote.cfg``.
15
16 .. image:: images/screenshots/pbs-gui-remote-add.png
17 :target: _images/pbs-gui-remote-add.png
18 :align: right
19 :alt: Add a remote
20
21 To add a remote, you need its hostname or IP address, a userid and password on
22 the remote, and its certificate fingerprint. To get the fingerprint, use the
23 ``proxmox-backup-manager cert info`` command on the remote, or navigate to
24 **Dashboard** in the remote's web interface and select **Show Fingerprint**.
25
26 .. code-block:: console
27
28 # proxmox-backup-manager cert info |grep Fingerprint
29 Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
30
31 Using the information specified above, you can add a remote from the **Remotes**
32 configuration panel, or by using the command:
33
34 .. code-block:: console
35
36 # proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
37
38 Use the ``list``, ``show``, ``update``, ``remove`` subcommands of
39 ``proxmox-backup-manager remote`` to manage your remotes:
40
41 .. code-block:: console
42
43 # proxmox-backup-manager remote update pbs2 --host pbs2.example
44 # proxmox-backup-manager remote list
45 ┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐
46 │ name │ host │ userid │ fingerprint │ comment │
47 ╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡
48 │ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │ │
49 └──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘
50 # proxmox-backup-manager remote remove pbs2
51
52
53 .. _syncjobs:
54
55 Sync Jobs
56 ---------
57
58 .. image:: images/screenshots/pbs-gui-syncjob-add.png
59 :target: _images/pbs-gui-syncjob-add.png
60 :align: right
61 :alt: Add a Sync Job
62
63 Sync jobs are configured to pull the contents of a datastore on a **Remote** to
64 a local datastore. You can manage sync jobs in the web interface, from the
65 **Sync Jobs** tab of the **Datastore** panel or from that of the Datastore
66 itself. Alternatively, you can manage them with the ``proxmox-backup-manager
67 sync-job`` command. The configuration information for sync jobs is stored at
68 ``/etc/proxmox-backup/sync.cfg``. To create a new sync job, click the add button
69 in the GUI, or use the ``create`` subcommand. After creating a sync job, you can
70 either start it manually from the GUI or provide it with a schedule (see
71 :ref:`calendar-event-scheduling`) to run regularly.
72
73 .. code-block:: console
74
75 # proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30'
76 # proxmox-backup-manager sync-job update pbs2-local --comment 'offsite'
77 # proxmox-backup-manager sync-job list
78 ┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐
79 │ id │ store │ remote │ remote-store │ schedule │ comment │
80 ╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡
81 │ pbs2-local │ local │ pbs2 │ local │ Wed 02:30 │ offsite │
82 └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
83 # proxmox-backup-manager sync-job remove pbs2-local
84
85 To set up sync jobs, the configuring user needs the following permissions:
86
87 #. ``Remote.Read`` on the ``/remote/{remote}/{remote-store}`` path
88 #. At least ``Datastore.Backup`` on the local target datastore (``/datastore/{store}``)
89
90 .. note:: A sync job can only sync backup groups that the configured remote's
91 user/API token can read. If a remote is configured with a user/API token that
92 only has ``Datastore.Backup`` privileges, only the limited set of accessible
93 snapshots owned by that user/API token can be synced.
94
95 If the ``remove-vanished`` option is set, ``Datastore.Prune`` is required on
96 the local datastore as well. If the ``owner`` option is not set (defaulting to
97 ``root@pam``) or is set to something other than the configuring user,
98 ``Datastore.Modify`` is required as well.
99
100 If the ``group-filter`` option is set, only backup groups matching at least one
101 of the specified criteria are synced. The available criteria are:
102
103 * Backup type, for example, to only sync groups of the `ct` (Container) type:
104 .. code-block:: console
105
106 # proxmox-backup-manager sync-job update ID --group-filter type:ct
107 * Full group identifier, to sync a specific backup group:
108 .. code-block:: console
109
110 # proxmox-backup-manager sync-job update ID --group-filter group:vm/100
111 * Regular expression, matched against the full group identifier
112 .. code-block:: console
113
114 # proxmox-backup-manager sync-job update ID --group-filter regex:'^vm/1\d{2,3}$'
115
116 The same filter is applied to local groups, for handling of the
117 ``remove-vanished`` option.
118
119 A ``group-filter`` can be inverted by adding ``exclude:`` to its beginning.
120
121 * Regular expression example, excluding the match:
122 .. code-block:: console
123
124 # proxmox-backup-manager sync-job update ID --group-filter exclude:regex:'^vm/1\d{2,3}$'
125
126 For mixing include and exclude filter, following rules apply:
127
128 - no filters: all backup groups
129 - include: only those matching the include filters
130 - exclude: all but those matching the exclude filters
131 - both: those matching the include filters, but without those matching the exclude filters
132 .. note:: The ``protected`` flag of remote backup snapshots will not be synced.
133
134 Namespace Support
135 ^^^^^^^^^^^^^^^^^
136
137 Sync jobs can be configured to not only sync datastores, but also subsets of
138 datastores in the form of namespaces or namespace sub-trees. The following
139 parameters influence how namespaces are treated as part of a sync job's
140 execution:
141
142 - ``remote-ns``: the remote namespace anchor (default: the root namespace)
143
144 - ``ns``: the local namespace anchor (default: the root namespace)
145
146 - ``max-depth``: whether to recursively iterate over sub-namespaces of the remote
147 namespace anchor (default: `None`)
148
149 If ``max-depth`` is set to `0`, groups are synced from ``remote-ns`` into
150 ``ns``, without any recursion. If it is set to `None` (left empty), recursion
151 depth will depend on the value of ``remote-ns`` and the remote side's
152 availability of namespace support:
153
154 - ``remote-ns`` set to something other than the root namespace: remote *must*
155 support namespaces, full recursion starting at ``remote-ns``.
156
157 - ``remote-ns`` set to root namespace and remote *supports* namespaces: full
158 recursion starting at root namespace.
159
160 - ``remote-ns`` set to root namespace and remote *does not support* namespaces:
161 backwards-compat mode, only root namespace will be synced into ``ns``, no
162 recursion.
163
164 Any other value of ``max-depth`` will limit recursion to at most ``max-depth``
165 levels, for example: ``remote-ns`` set to `location_a/department_b` and
166 ``max-depth`` set to `1` will result in `location_a/department_b` and at most
167 one more level of sub-namespaces being synced.
168
169 The namespace tree starting at ``remote-ns`` will be mapped into ``ns`` up to a
170 depth of ``max-depth``.
171
172 For example, with the following namespaces at the remote side:
173
174 - `location_a`
175
176 - `location_a/department_x`
177
178 - `location_a/department_x/team_one`
179
180 - `location_a/department_x/team_two`
181
182 - `location_a/department_y`
183
184 - `location_a/department_y/team_one`
185
186 - `location_a/department_y/team_two`
187
188 - `location_b`
189
190 and ``remote-ns`` being set to `location_a/department_x` and ``ns`` set to
191 `location_a_dep_x` resulting in the following namespace tree on the sync
192 target:
193
194 - `location_a_dep_x` (containing the remote's `location_a/department_x`)
195
196 - `location_a_dep_x/team_one` (containing the remote's `location_a/department_x/team_one`)
197
198 - `location_a_dep_x/team_two` (containing the remote's `location_a/department_x/team_two`)
199
200 with the rest of the remote namespaces and groups not being synced (by this
201 sync job).
202
203 If a remote namespace is included in the sync job scope, but does not exist
204 locally, it will be created (provided the sync job owner has sufficient
205 privileges).
206
207 If the ``remove-vanished`` option is set, namespaces that are included in the
208 sync job scope but only exist locally are treated as vanished and removed
209 (provided the sync job owner has sufficient privileges).
210
211 .. note:: All other limitations on sync scope (such as remote user/API token
212 privileges, group filters) also apply for sync jobs involving one or
213 multiple namespaces.
214
215 Bandwidth Limit
216 ^^^^^^^^^^^^^^^
217
218 Syncing a datastore to an archive can produce a lot of traffic and impact other
219 users of the network. In order to avoid network or storage congestion, you can
220 limit the bandwidth of the sync job by setting the ``rate-in`` option either in
221 the web interface or using the ``proxmox-backup-manager`` command-line tool:
222
223 .. code-block:: console
224
225 # proxmox-backup-manager sync-job update ID --rate-in 20MiB