[ceph.git] / ceph / doc / man / 8 / ceph-volume.rst

:orphan:

=======================================================
 ceph-volume -- Ceph OSD deployment and inspection tool
=======================================================

.. program:: ceph-volume

Synopsis
========

| **ceph-volume** [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
|                 [--log-path LOG_PATH]

| **ceph-volume** **inventory**

| **ceph-volume** **lvm** [ *trigger* | *create* | *activate* | *prepare*
| *zap* | *list* | *batch*]

| **ceph-volume** **simple** [ *trigger* | *scan* | *activate* ]


Description
===========

:program:`ceph-volume` is a single purpose command line tool to deploy logical
volumes as OSDs, trying to maintain a similar API to ``ceph-disk`` when
preparing, activating, and creating OSDs.

It deviates from ``ceph-disk`` by not interacting or relying on the udev rules
that come installed for Ceph. These rules allow automatic detection of
previously setup devices that are in turn fed into ``ceph-disk`` to activate
them.


Commands
========

inventory
---------

This subcommand provides information about a host's physical disc inventory and
reports metadata about these discs. Among this metadata one can find disc
specific data items (like model, size, rotational or solid state) as well as
data items specific to ceph using a device, such as if it is available for
use with ceph or if logical volumes are present.

Examples::

    ceph-volume inventory
    ceph-volume inventory /dev/sda
    ceph-volume inventory --format json-pretty

Optional arguments:

* [-h, --help]          show the help message and exit
* [--format]            report format, valid values are ``plain`` (default),
                        ``json`` and ``json-pretty``

lvm
---

By making use of LVM tags, the ``lvm`` sub-command is able to store and later
re-discover and query devices associated with OSDs so that they can later
activated.

Subcommands:

**batch**
Creates OSDs from a list of devices using a ``filestore``
or ``bluestore`` (default) setup. It will create all necessary volume groups
and logical volumes required to have a working OSD.

Example usage with three devices::

    ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc

Optional arguments:

* [-h, --help]          show the help message and exit
* [--bluestore]         Use the bluestore objectstore (default)
* [--filestore]         Use the filestore objectstore
* [--yes]               Skip the report and prompt to continue provisioning
* [--prepare]           Only prepare OSDs, do not activate
* [--dmcrypt]           Enable encryption for the underlying OSD devices
* [--crush-device-class] Define a CRUSH device class to assign the OSD to
* [--no-systemd]         Do not enable or create any systemd units
* [--report]         Report what the potential outcome would be for the
                     current input (requires devices to be passed in)
* [--format]         Output format when reporting (used along with
                     --report), can be one of 'pretty' (default) or 'json'
* [--block-db-size]     Set (or override) the "bluestore_block_db_size" value,
                        in bytes
* [--journal-size]      Override the "osd_journal_size" value, in megabytes

Required positional arguments:

* <DEVICE>    Full path to a raw device, like ``/dev/sda``. Multiple
              ``<DEVICE>`` paths can be passed in.


**activate**
Enables a systemd unit that persists the OSD ID and its UUID (also called
``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
enabled and needs to be mounted.

Usage::

    ceph-volume lvm activate --bluestore <osd id> <osd fsid>

Optional Arguments:

* [-h, --help]  show the help message and exit
* [--auto-detect-objectstore] Automatically detect the objecstore by inspecting
  the OSD
* [--bluestore] bluestore objectstore (default)
* [--filestore] filestore objectstore
* [--all] Activate all OSDs found in the system
* [--no-systemd] Skip creating and enabling systemd units and starting of OSD
  services

Multiple OSDs can be activated at once by using the (idempotent) ``--all`` flag::

    ceph-volume lvm activate --all


**prepare**
Prepares a logical volume to be used as an OSD and journal using a ``filestore``
or ``bluestore`` (default) setup. It will not create or modify the logical volumes
except for adding extra metadata.

Usage::

    ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device>

Optional arguments:

* [-h, --help]          show the help message and exit
* [--journal JOURNAL]   A logical group name, path to a logical volume, or path to a device
* [--bluestore]         Use the bluestore objectstore (default)
* [--block.wal]         Path to a bluestore block.wal logical volume or partition
* [--block.db]          Path to a bluestore block.db logical volume or partition
* [--filestore]         Use the filestore objectstore
* [--dmcrypt]           Enable encryption for the underlying OSD devices
* [--osd-id OSD_ID]     Reuse an existing OSD id
* [--osd-fsid OSD_FSID] Reuse an existing OSD fsid
* [--crush-device-class] Define a CRUSH device class to assign the OSD to

Required arguments:

* --data                A logical group name or a path to a logical volume

For encrypting an OSD, the ``--dmcrypt`` flag must be added when preparing
(also supported in the ``create`` sub-command).


**create**
Wraps the two-step process to provision a new osd (calling ``prepare`` first
and then ``activate``) into a single one. The reason to prefer ``prepare`` and
then ``activate`` is to gradually introduce new OSDs into a cluster, and
avoiding large amounts of data being rebalanced.

The single-call process unifies exactly what ``prepare`` and ``activate`` do,
with the convenience of doing it all at once. Flags and general usage are
equivalent to those of the ``prepare`` and ``activate`` subcommand.

**trigger**
This subcommand is not meant to be used directly, and it is used by systemd so
that it proxies input to ``ceph-volume lvm activate`` by parsing the
input from systemd, detecting the UUID and ID associated with an OSD.

Usage::

    ceph-volume lvm trigger <SYSTEMD-DATA>

The systemd "data" is expected to be in the format of::

    <OSD ID>-<OSD UUID>

The lvs associated with the OSD need to have been prepared previously,
so that all needed tags and metadata exist.

Positional arguments:

* <SYSTEMD_DATA>  Data from a systemd unit containing ID and UUID of the OSD.

**list**
List devices or logical volumes associated with Ceph. An association is
determined if a device has information relating to an OSD. This is
verified by querying LVM's metadata and correlating it with devices.

The lvs associated with the OSD need to have been prepared previously by
ceph-volume so that all needed tags and metadata exist.

Usage::

    ceph-volume lvm list

List a particular device, reporting all metadata about it::

    ceph-volume lvm list /dev/sda1

List a logical volume, along with all its metadata (vg is a volume
group, and lv the logical volume name)::

    ceph-volume lvm list {vg/lv}

Positional arguments:

* <DEVICE>  Either in the form of ``vg/lv`` for logical volumes,
  ``/path/to/sda1`` or ``/path/to/sda`` for regular devices.


**zap**
Zaps the given logical volume or partition. If given a path to a logical
volume it must be in the format of vg/lv. Any filesystems present
on the given lv or partition will be removed and all data will be purged.

However, the lv or partition will be kept intact.

Usage, for logical volumes::

      ceph-volume lvm zap {vg/lv}

Usage, for logical partitions::

      ceph-volume lvm zap /dev/sdc1

For full removal of the device use the ``--destroy`` flag (allowed for all
device types)::

      ceph-volume lvm zap --destroy /dev/sdc1

Multiple devices can be removed by specifying the OSD ID and/or the OSD FSID::

      ceph-volume lvm zap --destroy --osd-id 1
      ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59


Positional arguments:

* <DEVICE>  Either in the form of ``vg/lv`` for logical volumes,
  ``/path/to/sda1`` or ``/path/to/sda`` for regular devices.


simple
------

Scan legacy OSD directories or data devices that may have been created by
ceph-disk, or manually.

Subcommands:

**activate**
Enables a systemd unit that persists the OSD ID and its UUID (also called
``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
enabled and needs to be mounted, while reading information that was previously
created and persisted at ``/etc/ceph/osd/`` in JSON format.

Usage::

    ceph-volume simple activate --bluestore <osd id> <osd fsid>

Optional Arguments:

* [-h, --help]  show the help message and exit
* [--bluestore] bluestore objectstore (default)
* [--filestore] filestore objectstore

Note: It requires a matching JSON file with the following format::

    /etc/ceph/osd/<osd id>-<osd fsid>.json


**scan**
Scan a running OSD or data device for an OSD for metadata that can later be
used to activate and manage the OSD with ceph-volume. The scan method will
create a JSON file with the required information plus anything found in the OSD
directory as well.

Optionally, the JSON blob can be sent to stdout for further inspection.

Usage on data devices::

    ceph-volume simple scan <data device>

Running OSD directories::

    ceph-volume simple scan <path to osd dir>


Optional arguments:

* [-h, --help]          show the help message and exit
* [--stdout]            Send the JSON blob to stdout
* [--force]             If the JSON file exists at destination, overwrite it

Required Positional arguments:

* <DATA DEVICE or OSD DIR>  Actual data partition or a path to the running OSD

**trigger**
This subcommand is not meant to be used directly, and it is used by systemd so
that it proxies input to ``ceph-volume simple activate`` by parsing the
input from systemd, detecting the UUID and ID associated with an OSD.

Usage::

    ceph-volume simple trigger <SYSTEMD-DATA>

The systemd "data" is expected to be in the format of::

    <OSD ID>-<OSD UUID>

The JSON file associated with the OSD need to have been persisted previously by
a scan (or manually), so that all needed metadata can be used.

Positional arguments:

* <SYSTEMD_DATA>  Data from a systemd unit containing ID and UUID of the OSD.


Availability
============

:program:`ceph-volume` is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to
the documentation at http://docs.ceph.com/ for more information.


See also
========

:doc:`ceph-osd <ceph-osd>`\(8),
:doc:`ceph-disk <ceph-disk>`\(8),
Commit	Line	Data
b5b8bbf5 FG	1	:orphan:
b5b8bbf5 FG	2
91327a77 AA	3	=======================================================
	4	ceph-volume -- Ceph OSD deployment and inspection tool
	5	=======================================================
b5b8bbf5 FG	6
	7	.. program:: ceph-volume
	8
	9	Synopsis
	10	========
	11
	12	\| ceph-volume [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
	13	\| [--log-path LOG_PATH]
	14
91327a77 AA	15	\| ceph-volume inventory
91327a77 AA	16
b32b8144	17	\| ceph-volume lvm [ trigger \| create \| activate \| prepare
1adf2230	18	\| zap \| list \| batch]
b32b8144 FG	19
	20	\| ceph-volume simple [ trigger \| scan \| activate ]
	21
b5b8bbf5 FG	22
	23	Description
	24	===========
	25
	26	:program:`ceph-volume` is a single purpose command line tool to deploy logical
	27	volumes as OSDs, trying to maintain a similar API to ``ceph-disk`` when
	28	preparing, activating, and creating OSDs.
	29
	30	It deviates from ``ceph-disk`` by not interacting or relying on the udev rules
	31	that come installed for Ceph. These rules allow automatic detection of
	32	previously setup devices that are in turn fed into ``ceph-disk`` to activate
	33	them.
	34
	35
	36	Commands
	37	========
	38
91327a77 AA	39	inventory
	40	---------
	41
	42	This subcommand provides information about a host's physical disc inventory and
	43	reports metadata about these discs. Among this metadata one can find disc
	44	specific data items (like model, size, rotational or solid state) as well as
	45	data items specific to ceph using a device, such as if it is available for
	46	use with ceph or if logical volumes are present.
	47
	48	Examples::
	49
	50	ceph-volume inventory
	51	ceph-volume inventory /dev/sda
	52	ceph-volume inventory --format json-pretty
	53
	54	Optional arguments:
	55
	56	* [-h, --help] show the help message and exit
	57	* [--format] report format, valid values are ``plain`` (default),
	58	``json`` and ``json-pretty``
	59
b5b8bbf5 FG	60	lvm
	61	---
	62
	63	By making use of LVM tags, the ``lvm`` sub-command is able to store and later
	64	re-discover and query devices associated with OSDs so that they can later
	65	activated.
	66
	67	Subcommands:
	68
1adf2230 AA	69	batch
	70	Creates OSDs from a list of devices using a ``filestore``
	71	or ``bluestore`` (default) setup. It will create all necessary volume groups
	72	and logical volumes required to have a working OSD.
	73
	74	Example usage with three devices::
	75
	76	ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
	77
	78	Optional arguments:
	79
	80	* [-h, --help] show the help message and exit
	81	* [--bluestore] Use the bluestore objectstore (default)
	82	* [--filestore] Use the filestore objectstore
	83	* [--yes] Skip the report and prompt to continue provisioning
91327a77	84	* [--prepare] Only prepare OSDs, do not activate
1adf2230 AA	85	* [--dmcrypt] Enable encryption for the underlying OSD devices
	86	* [--crush-device-class] Define a CRUSH device class to assign the OSD to
	87	* [--no-systemd] Do not enable or create any systemd units
	88	* [--report] Report what the potential outcome would be for the
	89	current input (requires devices to be passed in)
	90	* [--format] Output format when reporting (used along with
	91	--report), can be one of 'pretty' (default) or 'json'
91327a77 AA	92	* [--block-db-size] Set (or override) the "bluestore_block_db_size" value,
	93	in bytes
	94	* [--journal-size] Override the "osd_journal_size" value, in megabytes
1adf2230 AA	95
	96	Required positional arguments:
	97
	98	* <DEVICE> Full path to a raw device, like ``/dev/sda``. Multiple
	99	``<DEVICE>`` paths can be passed in.
	100
	101
b5b8bbf5 FG	102	activate
	103	Enables a systemd unit that persists the OSD ID and its UUID (also called
	104	``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
	105	enabled and needs to be mounted.
	106
	107	Usage::
	108
94b18763	109	ceph-volume lvm activate --bluestore <osd id> <osd fsid>
b5b8bbf5 FG	110
	111	Optional Arguments:
	112
	113	* [-h, --help] show the help message and exit
94b18763 FG	114	* [--auto-detect-objectstore] Automatically detect the objecstore by inspecting
	115	the OSD
	116	* [--bluestore] bluestore objectstore (default)
	117	* [--filestore] filestore objectstore
	118	* [--all] Activate all OSDs found in the system
	119	* [--no-systemd] Skip creating and enabling systemd units and starting of OSD
	120	services
b5b8bbf5	121
28e407b8 AA	122	Multiple OSDs can be activated at once by using the (idempotent) ``--all`` flag::
	123
	124	ceph-volume lvm activate --all
	125
b5b8bbf5 FG	126
b5b8bbf5 FG	127	prepare
94b18763 FG	128	Prepares a logical volume to be used as an OSD and journal using a ``filestore``
94b18763 FG	129	or ``bluestore`` (default) setup. It will not create or modify the logical volumes
b5b8bbf5 FG	130	except for adding extra metadata.
	131
	132	Usage::
	133
	134	ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device>
	135
	136	Optional arguments:
	137
	138	* [-h, --help] show the help message and exit
	139	* [--journal JOURNAL] A logical group name, path to a logical volume, or path to a device
b32b8144	140	* [--bluestore] Use the bluestore objectstore (default)
94b18763 FG	141	* [--block.wal] Path to a bluestore block.wal logical volume or partition
94b18763 FG	142	* [--block.db] Path to a bluestore block.db logical volume or partition
b32b8144 FG	143	* [--filestore] Use the filestore objectstore
b32b8144 FG	144	* [--dmcrypt] Enable encryption for the underlying OSD devices
b5b8bbf5 FG	145	* [--osd-id OSD_ID] Reuse an existing OSD id
b5b8bbf5 FG	146	* [--osd-fsid OSD_FSID] Reuse an existing OSD fsid
94b18763	147	* [--crush-device-class] Define a CRUSH device class to assign the OSD to
b5b8bbf5 FG	148
	149	Required arguments:
	150
	151	* --data A logical group name or a path to a logical volume
	152
28e407b8 AA	153	For encrypting an OSD, the ``--dmcrypt`` flag must be added when preparing
	154	(also supported in the ``create`` sub-command).
	155
	156
b5b8bbf5 FG	157	create
	158	Wraps the two-step process to provision a new osd (calling ``prepare`` first
	159	and then ``activate``) into a single one. The reason to prefer ``prepare`` and
	160	then ``activate`` is to gradually introduce new OSDs into a cluster, and
	161	avoiding large amounts of data being rebalanced.
	162
	163	The single-call process unifies exactly what ``prepare`` and ``activate`` do,
	164	with the convenience of doing it all at once. Flags and general usage are
94b18763	165	equivalent to those of the ``prepare`` and ``activate`` subcommand.
b5b8bbf5 FG	166
	167	trigger
	168	This subcommand is not meant to be used directly, and it is used by systemd so
	169	that it proxies input to ``ceph-volume lvm activate`` by parsing the
	170	input from systemd, detecting the UUID and ID associated with an OSD.
	171
	172	Usage::
	173
	174	ceph-volume lvm trigger <SYSTEMD-DATA>
	175
	176	The systemd "data" is expected to be in the format of::
	177
	178	<OSD ID>-<OSD UUID>
	179
	180	The lvs associated with the OSD need to have been prepared previously,
	181	so that all needed tags and metadata exist.
	182
	183	Positional arguments:
	184
	185	* <SYSTEMD_DATA> Data from a systemd unit containing ID and UUID of the OSD.
	186
b32b8144 FG	187	list
	188	List devices or logical volumes associated with Ceph. An association is
	189	determined if a device has information relating to an OSD. This is
	190	verified by querying LVM's metadata and correlating it with devices.
	191
	192	The lvs associated with the OSD need to have been prepared previously by
	193	ceph-volume so that all needed tags and metadata exist.
	194
	195	Usage::
	196
	197	ceph-volume lvm list
	198
	199	List a particular device, reporting all metadata about it::
	200
	201	ceph-volume lvm list /dev/sda1
	202
	203	List a logical volume, along with all its metadata (vg is a volume
	204	group, and lv the logical volume name)::
	205
	206	ceph-volume lvm list {vg/lv}
	207
	208	Positional arguments:
	209
94b18763 FG	210	* <DEVICE> Either in the form of ``vg/lv`` for logical volumes,
94b18763 FG	211	``/path/to/sda1`` or ``/path/to/sda`` for regular devices.
b32b8144 FG	212
	213
	214	zap
	215	Zaps the given logical volume or partition. If given a path to a logical
	216	volume it must be in the format of vg/lv. Any filesystems present
	217	on the given lv or partition will be removed and all data will be purged.
	218
	219	However, the lv or partition will be kept intact.
	220
	221	Usage, for logical volumes::
	222
	223	ceph-volume lvm zap {vg/lv}
	224
	225	Usage, for logical partitions::
	226
	227	ceph-volume lvm zap /dev/sdc1
	228
f64942e4 AA	229	For full removal of the device use the ``--destroy`` flag (allowed for all
	230	device types)::
	231
	232	ceph-volume lvm zap --destroy /dev/sdc1
	233
	234	Multiple devices can be removed by specifying the OSD ID and/or the OSD FSID::
	235
	236	ceph-volume lvm zap --destroy --osd-id 1
	237	ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
	238
	239
b32b8144 FG	240	Positional arguments:
b32b8144 FG	241
94b18763 FG	242	* <DEVICE> Either in the form of ``vg/lv`` for logical volumes,
94b18763 FG	243	``/path/to/sda1`` or ``/path/to/sda`` for regular devices.
b32b8144 FG	244
	245
	246	simple
	247	------
	248
	249	Scan legacy OSD directories or data devices that may have been created by
	250	ceph-disk, or manually.
	251
	252	Subcommands:
	253
	254	activate
	255	Enables a systemd unit that persists the OSD ID and its UUID (also called
	256	``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
	257	enabled and needs to be mounted, while reading information that was previously
	258	created and persisted at ``/etc/ceph/osd/`` in JSON format.
	259
	260	Usage::
	261
	262	ceph-volume simple activate --bluestore <osd id> <osd fsid>
	263
	264	Optional Arguments:
	265
	266	* [-h, --help] show the help message and exit
	267	* [--bluestore] bluestore objectstore (default)
	268	* [--filestore] filestore objectstore
	269
	270	Note: It requires a matching JSON file with the following format::
	271
	272	/etc/ceph/osd/<osd id>-<osd fsid>.json
	273
	274
	275	scan
	276	Scan a running OSD or data device for an OSD for metadata that can later be
	277	used to activate and manage the OSD with ceph-volume. The scan method will
	278	create a JSON file with the required information plus anything found in the OSD
	279	directory as well.
	280
	281	Optionally, the JSON blob can be sent to stdout for further inspection.
	282
	283	Usage on data devices::
	284
	285	ceph-volume simple scan <data device>
	286
	287	Running OSD directories::
	288
	289	ceph-volume simple scan <path to osd dir>
	290
	291
	292	Optional arguments:
	293
	294	* [-h, --help] show the help message and exit
	295	* [--stdout] Send the JSON blob to stdout
	296	* [--force] If the JSON file exists at destination, overwrite it
	297
	298	Required Positional arguments:
	299
	300	* <DATA DEVICE or OSD DIR> Actual data partition or a path to the running OSD
	301
	302	trigger
	303	This subcommand is not meant to be used directly, and it is used by systemd so
	304	that it proxies input to ``ceph-volume simple activate`` by parsing the
	305	input from systemd, detecting the UUID and ID associated with an OSD.
	306
	307	Usage::
308
309	ceph-volume simple trigger <SYSTEMD-DATA>
310
311	The systemd "data" is expected to be in the format of::
312
313	<OSD ID>-<OSD UUID>
314
315	The JSON file associated with the OSD need to have been persisted previously by
316	a scan (or manually), so that all needed metadata can be used.
317
318	Positional arguments:
319
320	* <SYSTEMD_DATA> Data from a systemd unit containing ID and UUID of the OSD.
321
322
b5b8bbf5 FG	323	Availability
	324	============
	325
	326	:program:`ceph-volume` is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to
	327	the documentation at http://docs.ceph.com/ for more information.
	328
	329
	330	See also
	331	========
	332
	333	:doc:`ceph-osd <ceph-osd>`\(8),
	334	:doc:`ceph-disk <ceph-disk>`\(8),