ceph/doc/cephfs/snap-schedule.rst

   1 .. _snap-schedule:
   2
   3 ==========================
   4 Snapshot Scheduling Module
   5 ==========================
   6 This module implements scheduled snapshots for CephFS.
   7 It provides a user interface to add, query and remove snapshots schedules and
   8 retention policies, as well as a scheduler that takes the snapshots and prunes
   9 existing snapshots accordingly.
  10
  11
  12 How to enable
  13 =============
  14
  15 The *snap_schedule* module is enabled with::
  16
  17   ceph mgr module enable snap_schedule
  18
  19 Usage
  20 =====
  21
  22 This module uses :doc:`/dev/cephfs-snapshots`, please consider this documentation
  23 as well.
  24
  25 This module's subcommands live under the `ceph fs snap-schedule` namespace.
  26 Arguments can either be supplied as positional arguments or as keyword
  27 arguments. Once a keyword argument was encountered, all following arguments are
  28 assumed to be keyword arguments too.
  29
  30 Snapshot schedules are identified by path, their repeat interval and their start
  31 time. The
  32 repeat interval defines the time between two subsequent snapshots. It is
  33 specified by a number and a period multiplier, one of `h(our)`, `d(ay)` and
  34 `w(eek)`. E.g. a repeat interval of `12h` specifies one snapshot every 12
  35 hours.
  36 The start time is specified as a time string (more details about passing times
  37 below). By default
  38 the start time is last midnight. So when a snapshot schedule with repeat
  39 interval `1h` is added at 13:50
  40 with the default start time, the first snapshot will be taken at 14:00.
  41
  42 Retention specifications are identified by path and the retention spec itself. A
  43 retention spec consists of either a number and a time period separated by a
  44 space or concatenated pairs of `<number><time period>`.
  45 The semantics are that a spec will ensure `<number>` snapshots are kept that are
  46 at least `<time period>` apart. For Example `7d` means the user wants to keep 7
  47 snapshots that are at least one day (but potentially longer) apart from each other.
  48 The following time periods are recognized: `h(our), d(ay), w(eek), m(onth),
  49 y(ear)` and `n`. The latter is a special modifier where e.g. `10n` means keep
  50 the last 10 snapshots regardless of timing,
  51
  52 All subcommands take optional `fs` and `subvol` arguments to specify paths in
  53 multi-fs setups and :doc:`/cephfs/fs-volumes` managed setups. If not
  54 passed `fs` defaults to the first file system listed in the fs_map, `subvolume`
  55 defaults to nothing.
  56 When using :doc:`/cephfs/fs-volumes` the argument `fs` is equivalent to a
  57 `volume`.
  58
  59 When a timestamp is passed (the `start` argument in the `add`, `remove`,
  60 `activate` and `deactivate` subcommands) the ISO format `%Y-%m-%dT%H:%M:%S` will
  61 always be accepted. When either python3.7 or newer is used or
  62 https://github.com/movermeyer/backports.datetime_fromisoformat is installed, any
  63 valid ISO timestamp that is parsed by python's `datetime.fromisoformat` is valid.
  64
  65 When no subcommand is supplied a synopsis is printed::
  66
  67   #> ceph fs snap-schedule
  68   no valid command found; 8 closest matches:
  69   fs snap-schedule status [<path>] [<subvol>] [<fs>] [<format>]
  70   fs snap-schedule list <path> [<subvol>] [--recursive] [<fs>] [<format>]
  71   fs snap-schedule add <path> <snap_schedule> [<start>] [<fs>] [<subvol>]
  72   fs snap-schedule remove <path> [<repeat>] [<start>] [<subvol>] [<fs>]
  73   fs snap-schedule retention add <path> <retention_spec_or_period> [<retention_count>] [<fs>] [<subvol>]
  74   fs snap-schedule retention remove <path> <retention_spec_or_period> [<retention_count>] [<fs>] [<subvol>]
  75   fs snap-schedule activate <path> [<repeat>] [<start>] [<subvol>] [<fs>]
  76   fs snap-schedule deactivate <path> [<repeat>] [<start>] [<subvol>] [<fs>]
  77   Error EINVAL: invalid command
  78
  79 Inspect snapshot schedules
  80 --------------------------
  81
  82 The module offers two subcommands to inspect existing schedules: `list` and
  83 `status`. Bother offer plain and json output via the optional `format` argument.
  84 The default is plain.
  85 The `list` sub-command will list all schedules on a path in a short single line
  86 format. It offers a `recursive` argument to list all schedules in the specified
  87 directory and all contained directories.
  88 The `status` subcommand prints all available schedules and retention specs for a
  89 path.
  90
  91 Examples::
  92
  93   ceph fs snap-schedule status /
  94   ceph fs snap-schedule status /foo/bar --format=json
  95   ceph fs snap-schedule list /
  96   ceph fs snap-schedule list / --recursive=true # list all schedules in the tree
  97
  98
  99 Add and remove schedules
 100 ------------------------
 101 The `add` and `remove` subcommands add and remove snapshots schedules
 102 respectively. Both require at least a `path` argument, `add` additionally
 103 requires a `schedule` argument as described in the USAGE section.
 104
 105 Multiple different schedules can be added to a path. Two schedules are considered
 106 different from each other if they differ in their repeat interval and their
 107 start time.
 108
 109 If multiple schedules have been set on a path, `remove` can remove individual
 110 schedules on a path by specifying the exact repeat interval and start time, or
 111 the subcommand can remove all schedules on a path when just a `path` is
 112 specified.
 113
 114 Examples::
 115
 116   ceph fs snap-schedule add / 1h
 117   ceph fs snap-schedule add / 1h 11:55
 118   ceph fs snap-schedule add / 2h 11:55
 119   ceph fs snap-schedule remove / 1h 11:55 # removes one single schedule
 120   ceph fs snap-schedule remove / 1h # removes all schedules with --repeat=1h
 121   ceph fs snap-schedule remove / # removes all schedules on path /
 122
 123 Add and remove retention policies
 124 ---------------------------------
 125 The `retention add` and `retention remove` subcommands allow to manage
 126 retention policies. One path has exactly one retention policy. A policy can
 127 however contain multiple count-time period pairs in order to specify complex
 128 retention policies.
 129 Retention policies can be added and removed individually or in bulk via the
 130 forms `ceph fs snap-schedule retention add <path> <time period> <count>` and
 131 `ceph fs snap-schedule retention add <path> <countTime period>[countTime period]`
 132
 133 Examples::
 134
 135   ceph fs snap-schedule retention add / h 24 # keep 24 snapshots at least an hour apart
 136   ceph fs snap-schedule retention add / d 7 # and 7 snapshots at least a day apart
 137   ceph fs snap-schedule retention remove / h 24 # remove retention for 24 hourlies
 138   ceph fs snap-schedule retention add / 24h4w # add 24 hourly and 4 weekly to retention
 139   ceph fs snap-schedule retention remove / 7d4w # remove 7 daily and 4 weekly, leaves 24 hourly
 140
 141 Active and inactive schedules
 142 -----------------------------
 143 Snapshot schedules can be added for a path that doesn't exist yet in the
 144 directory tree. Similarly a path can be removed without affecting any snapshot
 145 schedules on that path.
 146 If a directory is not present when a snapshot is scheduled to be taken, the
 147 schedule will be set to inactive and will be excluded from scheduling until
 148 it is activated again.
 149 A schedule can manually be set to inactive to pause the creating of scheduled
 150 snapshots.
 151 The module provides the `activate` and `deactivate` subcommands for this
 152 purpose.
 153
 154 Examples::
 155
 156   ceph fs snap-schedule activate / # activate all schedules on the root directory
 157   ceph fs snap-schedule deactivate / 1d # deactivates daily snapshots on the root directory
 158
 159 Limitations
 160 -----------
 161 Snapshots are scheduled using python Timers. Under normal circumstances
 162 specifying 1h as the schedule will result in snapshots 1 hour apart fairly
 163 precisely. If the mgr daemon is under heavy load however, the Timer threads
 164 might not get scheduled right away, resulting in a slightly delayed snapshot. If
 165 this happens, the next snapshot will be schedule as if the previous one was not
 166 delayed, i.e. one or more delayed snapshots will not cause drift in the overall
 167 schedule.
 168
 169 In order to somewhat limit the overall number of snapshots in a file system, the
 170 module will only keep a maximum of 50 snapshots per directory. If the retention
 171 policy results in more then 50 retained snapshots, the retention list will be
 172 shortened to the newest 50 snapshots.
 173
 174 Data storage
 175 ------------
 176 The snapshot schedule data is stored in a rados object in the cephfs metadata
 177 pool. At runtime all data lives in a sqlite database that is serialized and
 178 stored as a rados object.