>= 12.1.2 --------- * When running 'df' on a CephFS filesystem comprising exactly one data pool, the result now reflects the file storage space used and available in that data pool (fuse client only). * Added new commands "pg force-recovery" and "pg-force-backfill". Use them to boost recovery or backfill priority of specified pgs, so they're recovered/backfilled before any other. Note that these commands don't interrupt ongoing recovery/backfill, but merely queue specified pgs before others so they're recovered/backfilled as soon as possible. New commands "pg cancel-force-recovery" and "pg cancel-force-backfill" restore default recovery/backfill priority of previously forced pgs. 12.2.1 ------ * Clusters will need to upgrade to 12.2.1 before upgrading to any Mimic 13.y.z version (either a development release or an eventual stable Mimic release). - *CephFS*: * Limiting MDS cache via a memory limit is now supported using the new mds_cache_memory_limit config option (1GB by default). A cache reservation can also be specified using mds_cache_reservation as a percentage of the limit (5% by default). Limits by inode count are still supported using mds_cache_size. Setting mds_cache_size to 0 (the default) disables the inode limit. * The maximum number of PGs per OSD before the monitor issues a warning has been reduced from 300 to 200 PGs. 200 is still twice the generally recommended target of 100 PGs per OSD. This limit can be adjusted via the ``mon_max_pg_per_osd`` option on the monitors. The older ``mon_pg_warn_max_per_osd`` option has been removed. * Creating pools or adjusting pg_num will now fail if the change would make the number of PGs per OSD exceed the configured ``mon_max_pg_per_osd`` limit. The option can be adjusted if it is really necessary to create a pool with more PGs. 12.2.3 ------ - *RBD*: * The RBD C API's rbd_discard method now enforces a maximum length of 2GB to match the C++ API's Image::discard method. This restriction prevents overflow of the result code. - *CephFS*: * The CephFS client now catches failures to clear dentries during startup and refuses to start as consistency and untrimmable cache issues may develop. The new option client_die_on_failed_dentry_invalidate (default: true) may be turned off to allow the client to proceed (dangerous!). 12.2.5 ------ - *CephFS*: * Upgrading an MDS cluster to 12.2.3+ will result in all active MDS exiting due to feature incompatibilities once an upgraded MDS comes online (even as standby). Operators may ignore the error messages and continue upgrading/restarting or follow this upgrade sequence: Reduce the number of ranks to 1 (`ceph fs set max_mds 1`), deactivate all other ranks (`ceph mds deactivate :`), shutdown standbys leaving the one active MDS, upgrade the single active MDS, then upgrade/start standbys. Finally, restore the previous max_mds. See also: https://tracker.ceph.com/issues/23172 * *rados list-inconsistent-obj format changes:* * Various error strings have been improved. For example, the "oi" or "oi_attr" in errors which stands for object info is now "info" (e.g. oi_attr_missing is now info_missing). * The object's "selected_object_info" is now in json format instead of string. * The attribute errors (attr_value_mismatch, attr_name_mismatch) only apply to user attributes. Only user attributes are output and have the internal leading underscore stripped. * If there are hash information errors (hinfo_missing, hinfo_corrupted, hinfo_inconsistency) then "hashinfo" is added with the json format of the information. If the information is corrupt then "hashinfo" is a string containing the value. * If there are snapset errors (snapset_missing, snapset_corrupted, snapset_inconsistency) then "snapset" is added with the json format of the information. If the information is corrupt then "snapset" is a string containing the value. * If there are object information errors (info_missing, info_corrupted, obj_size_info_mismatch, object_info_inconsistency) then "object_info" is added with the json format of the information instead of a string. If the information is corrupt then "object_info" is a string containing the value. * *rados list-inconsistent-snapset format changes:* * Various error strings have been improved. For example, the "ss_attr" in errors which stands for snapset info is now "snapset" (e.g. ss_attr_missing is now snapset_missing). The error snapset_mismatch has been renamed to snapset_error to better reflect what it means. * The head snapset information is output in json format as "snapset." This means that even when there are no head errors, the head object will be output when any shard has an error. This head object is there to show the snapset that was used in determining errors. 12.2.9 ------ * 12.2.9 contains the pg hard hard limit patches(https://tracker.ceph.com/issues/23979). A partial upgrade during recovery/backfill, can cause the osds on the previous version, to fail with assert(trim_to <= info.last_complete). The workaround for users is to upgrade and restart all OSDs to a version with the pg hard limit, or only upgrade when all PGs are active+clean. This patch will be reverted in 12.2.10, until a clean upgrade path is added to the pg log hard limit patches. See also: http://tracker.ceph.com/issues/36686