->= 12.0.0
-------
-* The "journaler allow split entries" config setting has been removed.
-* The 'apply' mode of cephfs-journal-tool has been removed
-* Added new configuration "public bind addr" to support dynamic environments
- like Kubernetes. When set the Ceph MON daemon could bind locally to an IP
- address and advertise a different IP address "public addr" on the network.
-
-12.0.0
-------
-
- * When assigning a network to the public network and not to
- the cluster network the network specification of the public
- network will be used for the cluster network as well.
- In older versions this would lead to cluster services
- being bound to 0.0.0.0:<port>, thus making the
- cluster service even more publicly available than the
- public services. When only specifying a cluster network it
- will still result in the public services binding to 0.0.0.0.
-
-* Some variants of the omap_get_keys and omap_get_vals librados
- functions have been deprecated in favor of omap_get_vals2 and
- omap_get_keys2. The new methods include an output argument
- indicating whether there are additional keys left to fetch.
- Previously this had to be inferred from the requested key count vs
- the number of keys returned, but this breaks with new OSD-side
- limits on the number of keys or bytes that can be returned by a
- single omap request. These limits were introduced by kraken but
- are effectively disabled by default (by setting a very large limit
- of 1 GB) because users of the newly deprecated interface cannot
- tell whether they should fetch more keys or not. In the case of
- the standalone calls in the C++ interface
- (IoCtx::get_omap_{keys,vals}), librados has been updated to loop on
- the client side to provide a correct result via multiple calls to
- the OSD. In the case of the methods used for building
- multi-operation transactions, however, client-side looping is not
- practical, and the methods have been deprecated. Note that use of
- either the IoCtx methods on older librados versions or the
- deprecated methods on any version of librados will lead to
- incomplete results if/when the new OSD limits are enabled.
-
-* In previous versions, if a client sent an op to the wrong OSD, the OSD
- would reply with ENXIO. The rationale here is that the client or OSD is
- clearly buggy and we want to surface the error as clearly as possible.
- We now only send the ENXIO reply if the osd_enxio_on_misdirected_op option
- is enabled (it's off by default). This means that a VM using librbd that
- previously would have gotten an EIO and gone read-only will now see a
- blocked/hung IO instead.
-
-* When configuring ceph-fuse mounts in /etc/fstab, a new syntax is
- available that uses "ceph.<arg>=<val>" in the options column, instead
- of putting configuration in the device column. The old style syntax
- still works. See the documentation page "Mount CephFS in your
- file systems table" for details.
-
-12.0.1
-------
-
-* The original librados rados_objects_list_open (C) and objects_begin
- (C++) object listing API, deprecated in Hammer, has finally been
- removed. Users of this interface must update their software to use
- either the rados_nobjects_list_open (C) and nobjects_begin (C++) API or
- the new rados_object_list_begin (C) and object_list_begin (C++) API
- before updating the client-side librados library to Luminous.
-
- Object enumeration (via any API) with the latest librados version
- and pre-Hammer OSDs is no longer supported. Note that no in-tree
- Ceph services rely on object enumeration via the deprecated APIs, so
- only external librados users might be affected.
-
- The newest (and recommended) rados_object_list_begin (C) and
- object_list_begin (C++) API is only usable on clusters with the
- SORTBITWISE flag enabled (Jewel and later). (Note that this flag is
- required to be set before upgrading beyond Jewel.)
-
-* The rados copy-get-classic operation has been removed since it has not been
- used by the OSD since before hammer. It is unlikely any librados user is
- using this operation explicitly since there is also the more modern copy-get.
-
-* The RGW api for getting object torrent has changed its params from 'get_torrent'
- to 'torrent' so that it can be compatible with Amazon S3. Now the request for
- object torrent is like 'GET /ObjectName?torrent'.
-
-* The configuration option "osd pool erasure code stripe width" has
- been replaced by "osd pool erasure code stripe unit", and given the
- ability to be overridden by the erasure code profile setting
- "stripe_unit". For more details see "Erasure Code Profiles" in the
- documentation.
-
-* rbd and cephfs can use erasure coding with bluestore. This may be
- enabled by setting 'allow_ec_overwrites' to 'true' for a pool. Since
- this relies on bluestore's checksumming to do deep scrubbing,
- enabling this on a pool stored on filestore is not allowed.
-
-* The 'rados df' JSON output now prints numeric values as numbers instead of
- strings.
-
-* There was a bug introduced in Jewel (#19119) that broke the mapping behavior
- when an "out" OSD that still existed in the CRUSH map was removed with 'osd rm'.
- This could result in 'misdirected op' and other errors. The bug is now fixed,
- but the fix itself introduces the same risk because the behavior may vary between
- clients and OSDs. To avoid problems, please ensure that all OSDs are removed
- from the CRUSH map before deleting them. That is, be sure to do::
-
- ceph osd crush rm osd.123
-
- before::
-
- ceph osd rm osd.123
-
-12.0.2
-------
-
-* The original librados rados_objects_list_open (C) and objects_begin
- (C++) object listing API, deprecated in Hammer, has finally been
- removed. Users of this interface must update their software to use
- either the rados_nobjects_list_open (C) and nobjects_begin (C++) API or
- the new rados_object_list_begin (C) and object_list_begin (C++) API
- before updating the client-side librados library to Luminous.
-
- Object enumeration (via any API) with the latest librados version
- and pre-Hammer OSDs is no longer supported. Note that no in-tree
- Ceph services rely on object enumeration via the deprecated APIs, so
- only external librados users might be affected.
-
- The newest (and recommended) rados_object_list_begin (C) and
- object_list_begin (C++) API is only usable on clusters with the
- SORTBITWISE flag enabled (Jewel and later). (Note that this flag is
- required to be set before upgrading beyond Jewel.)
-* CephFS clients without the 'p' flag in their authentication capability
- string will no longer be able to set quotas or any layout fields. This
- flag previously only restricted modification of the pool and namespace
- fields in layouts.
-* CephFS directory fragmentation (large directory support) is enabled
- by default on new filesystems. To enable it on existing filesystems
- use "ceph fs set <fs_name> allow_dirfrags".
-* CephFS will generate a health warning if you have fewer standby daemons
- than it thinks you wanted. By default this will be 1 if you ever had
- a standby, and 0 if you did not. You can customize this using
- ``ceph fs set <fs> standby_count_wanted <number>``. Setting it
- to zero will effectively disable the health check.
-* The "ceph mds tell ..." command has been removed. It is superceded
- by "ceph tell mds.<id> ..."
-
-12.1.0
-------
-
-* The ``mon_osd_max_op_age`` option has been renamed to
- ``mon_osd_warn_op_age`` (default: 32 seconds), to indicate we
- generate a warning at this age. There is also a new
- ``mon_osd_err_op_age_ratio`` that is a expressed as a multitple of
- ``mon_osd_warn_op_age`` (default: 128, for roughly 60 minutes) to
- control when an error is generated.
-
-* The default maximum size for a single RADOS object has been reduced from
- 100GB to 128MB. The 100GB limit was completely impractical in practice
- while the 128MB limit is a bit high but not unreasonable. If you have an
- application written directly to librados that is using objects larger than
- 128MB you may need to adjust ``osd_max_object_size``.
-
-* The semantics of the 'rados ls' and librados object listing
- operations have always been a bit confusing in that "whiteout"
- objects (which logically don't exist and will return ENOENT if you
- try to access them) are included in the results. Previously
- whiteouts only occurred in cache tier pools. In luminous, logically
- deleted but snapshotted objects now result in a whiteout object, and
- as a result they will appear in 'rados ls' results, even though
- trying to read such an object will result in ENOENT. The 'rados
- listsnaps' operation can be used in such a case to enumerate which
- snapshots are present.
-
- This may seem a bit strange, but is less strange than having a
- deleted-but-snapshotted object not appear at all and be completely
- hidden from librados's ability to enumerate objects. Future
- versions of Ceph will likely include an alternative object
- enumeration interface that makes it more natural and efficient to
- enumerate all objects along with their snapshot and clone metadata.
-
-* The deprecated 'crush_ruleset' property has finally been removed; please use
- 'crush_rule' instead for the 'osd pool get ...' and 'osd pool set ..' commands.
-
-* The 'osd pool default crush replicated ruleset' option has been
- removed and replaced by the 'osd pool default crush rule' option.
- By default it is -1, which means the mon will pick the first type
- replicated rule in the CRUSH map for replicated pools. Erasure
- coded pools have rules that are automatically created for them if they are
- not specified at pool creation time.
-
-* The `status` ceph-mgr module is enabled by default, and initially provides two
- commands: `ceph tell mgr osd status` and `ceph tell mgr fs status`. These
- are high level colorized views to complement the existing CLI.
-
-12.1.1
-------
-
-* choose_args encoding has been changed to make it architecture-independent.
- If you deployed Luminous dev releases or 12.1.0 rc release and made use of
- the CRUSH choose_args feature, you need to remove all choose_args mappings
- from your CRUSH map before starting the upgrade.
-
-* The 'ceph health' structured output (JSON or XML) no longer contains
- a 'timechecks' section describing the time sync status. This
- information is now available via the 'ceph time-sync-status'
- command.
-
-* Certain extra fields in the 'ceph health' structured output that
- used to appear if the mons were low on disk space (which duplicated
- the information in the normal health warning messages) are now gone.
-
-* The "ceph -w" output no longer contains audit log entries by default.
- Add a "--watch-channel=audit" or "--watch-channel=*" to see them.
+>= 12.1.2
+---------
+* When running 'df' on a CephFS filesystem comprising exactly one data pool,
+ the result now reflects the file storage space used and available in that
+ data pool (fuse client only).
+* Added new commands "pg force-recovery" and "pg-force-backfill". Use them
+ to boost recovery or backfill priority of specified pgs, so they're
+ recovered/backfilled before any other. Note that these commands don't
+ interrupt ongoing recovery/backfill, but merely queue specified pgs
+ before others so they're recovered/backfilled as soon as possible.
+ New commands "pg cancel-force-recovery" and "pg cancel-force-backfill"
+ restore default recovery/backfill priority of previously forced pgs.