docs/managing-remotes.rst

   1 Managing Remotes & Sync
   2 =======================
   3
   4 .. _backup_remote:
   5
   6 :term:`Remote`
   7 --------------
   8
   9 A remote refers to a separate `Proxmox Backup`_ Server installation and a user
  10 on that installation, from which you can `sync` datastores to a local datastore
  11 with a `Sync Job`. You can configure remotes in the web interface, under
  12 **Configuration -> Remotes**. Alternatively, you can use the ``remote``
  13 subcommand. The configuration information for remotes is stored in the file
  14 ``/etc/proxmox-backup/remote.cfg``.
  15
  16 .. image:: images/screenshots/pbs-gui-remote-add.png
  17   :target: _images/pbs-gui-remote-add.png
  18   :align: right
  19   :alt: Add a remote
  20
  21 To add a remote, you need its hostname or IP address, a userid and password on
  22 the remote, and its certificate fingerprint. To get the fingerprint, use the
  23 ``proxmox-backup-manager cert info`` command on the remote, or navigate to
  24 **Dashboard** in the remote's web interface and select **Show Fingerprint**.
  25
  26 .. code-block:: console
  27
  28   # proxmox-backup-manager cert info |grep Fingerprint
  29   Fingerprint (sha256): 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
  30
  31 Using the information specified above, you can add a remote from the **Remotes**
  32 configuration panel, or by using the command:
  33
  34 .. code-block:: console
  35
  36   # proxmox-backup-manager remote create pbs2 --host pbs2.mydomain.example --userid sync@pam --password 'SECRET' --fingerprint 64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe
  37
  38 Use the ``list``, ``show``, ``update``, ``remove`` subcommands of
  39 ``proxmox-backup-manager remote`` to manage your remotes:
  40
  41 .. code-block:: console
  42
  43   # proxmox-backup-manager remote update pbs2 --host pbs2.example
  44   # proxmox-backup-manager remote list
  45   ┌──────┬──────────────┬──────────┬───────────────────────────────────────────┬─────────┐
  46   │ name │ host         │ userid   │ fingerprint                               │ comment │
  47   ╞══════╪══════════════╪══════════╪═══════════════════════════════════════════╪═════════╡
  48   │ pbs2 │ pbs2.example │ sync@pam │64:d3:ff:3a:50:38:53:5a:9b:f7:50:...:ab:fe │         │
  49   └──────┴──────────────┴──────────┴───────────────────────────────────────────┴─────────┘
  50   # proxmox-backup-manager remote remove pbs2
  51
  52
  53 .. _syncjobs:
  54
  55 Sync Jobs
  56 ---------
  57
  58 .. image:: images/screenshots/pbs-gui-syncjob-add.png
  59   :target: _images/pbs-gui-syncjob-add.png
  60   :align: right
  61   :alt: Add a Sync Job
  62
  63 Sync jobs are configured to pull the contents of a datastore on a **Remote** to
  64 a local datastore. You can manage sync jobs in the web interface, from the
  65 **Sync Jobs** tab of the **Datastore** panel or from that of the Datastore
  66 itself. Alternatively, you can manage them with the ``proxmox-backup-manager
  67 sync-job`` command. The configuration information for sync jobs is stored at
  68 ``/etc/proxmox-backup/sync.cfg``. To create a new sync job, click the add button
  69 in the GUI, or use the ``create`` subcommand. After creating a sync job, you can
  70 either start it manually from the GUI or provide it with a schedule (see
  71 :ref:`calendar-event-scheduling`) to run regularly.
  72
  73 .. code-block:: console
  74
  75   # proxmox-backup-manager sync-job create pbs2-local --remote pbs2 --remote-store local --store local --schedule 'Wed 02:30'
  76   # proxmox-backup-manager sync-job update pbs2-local --comment 'offsite'
  77   # proxmox-backup-manager sync-job list
  78   ┌────────────┬───────┬────────┬──────────────┬───────────┬─────────┐
  79   │ id         │ store │ remote │ remote-store │ schedule  │ comment │
  80   ╞════════════╪═══════╪════════╪══════════════╪═══════════╪═════════╡
  81   │ pbs2-local │ local │ pbs2   │ local        │ Wed 02:30 │ offsite │
  82   └────────────┴───────┴────────┴──────────────┴───────────┴─────────┘
  83   # proxmox-backup-manager sync-job remove pbs2-local
  84
  85 To set up sync jobs, the configuring user needs the following permissions:
  86
  87 #. ``Remote.Read`` on the ``/remote/{remote}/{remote-store}`` path
  88 #. At least ``Datastore.Backup`` on the local target datastore (``/datastore/{store}``)
  89
  90 .. note:: A sync job can only sync backup groups that the configured remote's
  91   user/API token can read. If a remote is configured with a user/API token that
  92   only has ``Datastore.Backup`` privileges, only the limited set of accessible
  93   snapshots owned by that user/API token can be synced.
  94
  95 If the ``remove-vanished`` option is set, ``Datastore.Prune`` is required on
  96 the local datastore as well. If the ``owner`` option is not set (defaulting to
  97 ``root@pam``) or is set to something other than the configuring user,
  98 ``Datastore.Modify`` is required as well.
  99
 100 If the ``group-filter`` option is set, only backup groups matching at least one
 101 of the specified criteria are synced. The available criteria are:
 102
 103 * Backup type, for example, to only sync groups of the `ct` (Container) type:
 104     .. code-block:: console
 105
 106      # proxmox-backup-manager sync-job update ID --group-filter type:ct
 107 * Full group identifier, to sync a specific backup group:
 108     .. code-block:: console
 109
 110      # proxmox-backup-manager sync-job update ID --group-filter group:vm/100
 111 * Regular expression, matched against the full group identifier
 112     .. code-block:: console
 113
 114      # proxmox-backup-manager sync-job update ID --group-filter regex:'^vm/1\d{2,3}$'
 115
 116 The same filter is applied to local groups, for handling of the
 117 ``remove-vanished`` option.
 118
 119 .. note:: The ``protected`` flag of remote backup snapshots will not be synced.
 120
 121 Namespace Support
 122 ^^^^^^^^^^^^^^^^^
 123
 124 Sync jobs can be configured to not only sync datastores, but also subsets of
 125 datastores in the form of namespaces or namespace sub-trees. The following
 126 parameters influence how namespaces are treated as part of a sync job's
 127 execution:
 128
 129 - ``remote-ns``: the remote namespace anchor (default: the root namespace)
 130
 131 - ``ns``: the local namespace anchor (default: the root namespace)
 132
 133 - ``max-depth``: whether to recursively iterate over sub-namespaces of the remote
 134   namespace anchor (default: `None`)
 135
 136 If ``max-depth`` is set to `0`, groups are synced from ``remote-ns`` into
 137 ``ns``, without any recursion. If it is set to `None` (left empty), recursion
 138 depth will depend on the value of ``remote-ns`` and the remote side's
 139 availability of namespace support:
 140
 141 - ``remote-ns`` set to something other than the root namespace: remote *must*
 142   support namespaces, full recursion starting at ``remote-ns``.
 143
 144 - ``remote-ns`` set to root namespace and remote *supports* namespaces: full
 145   recursion starting at root namespace.
 146
 147 - ``remote-ns`` set to root namespace and remote *does not support* namespaces:
 148   backwards-compat mode, only root namespace will be synced into ``ns``, no
 149   recursion.
 150
 151 Any other value of ``max-depth`` will limit recursion to at most ``max-depth``
 152 levels, for example: ``remote-ns`` set to `location_a/department_b` and
 153 ``max-depth`` set to `1` will result in `location_a/department_b` and at most
 154 one more level of sub-namespaces being synced.
 155
 156 The namespace tree starting at ``remote-ns`` will be mapped into ``ns`` up to a
 157 depth of ``max-depth``.
 158
 159 For example, with the following namespaces at the remote side:
 160
 161 - `location_a`
 162
 163   - `location_a/department_x`
 164
 165     - `location_a/department_x/team_one`
 166
 167     - `location_a/department_x/team_two`
 168
 169   - `location_a/department_y`
 170
 171     - `location_a/department_y/team_one`
 172
 173     - `location_a/department_y/team_two`
 174
 175 - `location_b`
 176
 177 and ``remote-ns`` being set to `location_a/department_x` and ``ns`` set to
 178 `location_a_dep_x` resulting in the following namespace tree on the sync
 179 target:
 180
 181 - `location_a_dep_x` (containing the remote's `location_a/department_x`)
 182
 183   - `location_a_dep_x/team_one` (containing the remote's `location_a/department_x/team_one`)
 184
 185   - `location_a_dep_x/team_two` (containing the remote's `location_a/department_x/team_two`)
 186
 187 with the rest of the remote namespaces and groups not being synced (by this
 188 sync job).
 189
 190 If a remote namespace is included in the sync job scope, but does not exist
 191 locally, it will be created (provided the sync job owner has sufficient
 192 privileges).
 193
 194 If the ``remove-vanished`` option is set, namespaces that are included in the
 195 sync job scope but only exist locally are treated as vanished and removed
 196 (provided the sync job owner has sufficient privileges).
 197
 198 .. note:: All other limitations on sync scope (such as remote user/API token
 199    privileges, group filters) also apply for sync jobs involving one or
 200    multiple namespaces.
 201
 202 Bandwidth Limit
 203 ^^^^^^^^^^^^^^^
 204
 205 Syncing a datastore to an archive can produce a lot of traffic and impact other
 206 users of the network. In order to avoid network or storage congestion, you can
 207 limit the bandwidth of the sync job by setting the ``rate-in`` option either in
 208 the web interface or using the ``proxmox-backup-manager`` command-line tool:
 209
 210 .. code-block:: console
 211
 212     # proxmox-backup-manager sync-job update ID --rate-in 20MiB