[ceph.git] / ceph / doc / cephfs / upgrading.rst

Upgrading the MDS Cluster
=========================

Currently the MDS cluster does not have built-in versioning or file system
flags to support seamless upgrades of the MDSs without potentially causing
assertions or other faults due to incompatible messages or other functional
differences. For this reason, it's necessary during any cluster upgrade to
reduce the number of active MDS for a file system to one first so that two
active MDS do not communicate with different versions.  Further, it's also
necessary to take standbys offline as any new CompatSet flags will propagate
via the MDSMap to all MDS and cause older MDS to suicide.

The proper sequence for upgrading the MDS cluster is:

1. Reduce the number of ranks to 1:

::

    ceph fs set <fs_name> max_mds 1

2. Wait for cluster to stop non-zero ranks where only rank 0 is active and the rest are standbys.

::

    ceph status # wait for MDS to finish stopping

3. Take all standbys offline, e.g. using systemctl:

::

    systemctl stop ceph-mds.target

4. Confirm only one MDS is online and is rank 0 for your FS:

::

    ceph status

5. Upgrade the single active MDS, e.g. using systemctl:

::

    # use package manager to update cluster
    systemctl restart ceph-mds.target

6. Upgrade/start the standby daemons.

::

    # use package manager to update cluster
    systemctl restart ceph-mds.target

7. Restore the previous max_mds for your cluster:

::

    ceph fs set <fs_name> max_mds <old_max_mds>


Upgrading pre-Firefly filesystems past Jewel
============================================

.. tip::

    This advice only applies to users with filesystems
    created using versions of Ceph older than *Firefly* (0.80).
    Users creating new filesystems may disregard this advice.

Pre-firefly versions of Ceph used a now-deprecated format
for storing CephFS directory objects, called TMAPs.  Support
for reading these in RADOS will be removed after the Jewel
release of Ceph, so for upgrading CephFS users it is important
to ensure that any old directory objects have been converted.

After installing Jewel on all your MDS and OSD servers, and restarting
the services, run the following command:

::
    
    cephfs-data-scan tmap_upgrade <metadata pool name>

This only needs to be run once, and it is not necessary to
stop any other services while it runs.  The command may take some
time to execute, as it iterates overall objects in your metadata
pool.  It is safe to continue using your filesystem as normal while
it executes.  If the command aborts for any reason, it is safe
to simply run it again.

If you are upgrading a pre-Firefly CephFS filesystem to a newer Ceph version
than Jewel, you must first upgrade to Jewel and run the ``tmap_upgrade``
command before completing your upgrade to the latest version.
Commit	Line	Data
94b18763 FG	1	Upgrading the MDS Cluster
	2	=========================
	3
	4	Currently the MDS cluster does not have built-in versioning or file system
	5	flags to support seamless upgrades of the MDSs without potentially causing
	6	assertions or other faults due to incompatible messages or other functional
	7	differences. For this reason, it's necessary during any cluster upgrade to
	8	reduce the number of active MDS for a file system to one first so that two
	9	active MDS do not communicate with different versions. Further, it's also
11fdf7f2	10	necessary to take standbys offline as any new CompatSet flags will propagate
94b18763 FG	11	via the MDSMap to all MDS and cause older MDS to suicide.
	12
	13	The proper sequence for upgrading the MDS cluster is:
	14
	15	1. Reduce the number of ranks to 1:
	16
	17	::
	18
	19	ceph fs set <fs_name> max_mds 1
	20
11fdf7f2	21	2. Wait for cluster to stop non-zero ranks where only rank 0 is active and the rest are standbys.
94b18763 FG	22
	23	::
	24
94b18763 FG	25	ceph status # wait for MDS to finish stopping
	26
	27	3. Take all standbys offline, e.g. using systemctl:
	28
	29	::
	30
	31	systemctl stop ceph-mds.target
94b18763	32
11fdf7f2	33	4. Confirm only one MDS is online and is rank 0 for your FS:
94b18763 FG	34
	35	::
	36
11fdf7f2 TL	37	ceph status
	38
	39	5. Upgrade the single active MDS, e.g. using systemctl:
	40
	41	::
	42
	43	# use package manager to update cluster
94b18763 FG	44	systemctl restart ceph-mds.target
94b18763 FG	45
11fdf7f2 TL	46	6. Upgrade/start the standby daemons.
	47
	48	::
	49
	50	# use package manager to update cluster
	51	systemctl restart ceph-mds.target
94b18763	52
11fdf7f2	53	7. Restore the previous max_mds for your cluster:
94b18763 FG	54
	55	::
	56
	57	ceph fs set <fs_name> max_mds <old_max_mds>
	58
7c673cae FG	59
	60	Upgrading pre-Firefly filesystems past Jewel
	61	============================================
	62
	63	.. tip::
	64
	65	This advice only applies to users with filesystems
	66	created using versions of Ceph older than Firefly (0.80).
	67	Users creating new filesystems may disregard this advice.
	68
	69	Pre-firefly versions of Ceph used a now-deprecated format
	70	for storing CephFS directory objects, called TMAPs. Support
	71	for reading these in RADOS will be removed after the Jewel
	72	release of Ceph, so for upgrading CephFS users it is important
	73	to ensure that any old directory objects have been converted.
	74
	75	After installing Jewel on all your MDS and OSD servers, and restarting
	76	the services, run the following command:
	77
	78	::
	79
	80	cephfs-data-scan tmap_upgrade <metadata pool name>
	81
	82	This only needs to be run once, and it is not necessary to
	83	stop any other services while it runs. The command may take some
	84	time to execute, as it iterates overall objects in your metadata
	85	pool. It is safe to continue using your filesystem as normal while
	86	it executes. If the command aborts for any reason, it is safe
	87	to simply run it again.
	88
	89	If you are upgrading a pre-Firefly CephFS filesystem to a newer Ceph version
	90	than Jewel, you must first upgrade to Jewel and run the ``tmap_upgrade``
	91	command before completing your upgrade to the latest version.
	92