]>
Commit | Line | Data |
---|---|---|
b5b8bbf5 FG |
1 | :orphan: |
2 | ||
91327a77 AA |
3 | ======================================================= |
4 | ceph-volume -- Ceph OSD deployment and inspection tool | |
5 | ======================================================= | |
b5b8bbf5 FG |
6 | |
7 | .. program:: ceph-volume | |
8 | ||
9 | Synopsis | |
10 | ======== | |
11 | ||
12 | | **ceph-volume** [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL] | |
13 | | [--log-path LOG_PATH] | |
14 | ||
91327a77 AA |
15 | | **ceph-volume** **inventory** |
16 | ||
b32b8144 | 17 | | **ceph-volume** **lvm** [ *trigger* | *create* | *activate* | *prepare* |
ec96510d | 18 | | *zap* | *list* | *batch* | *new-wal* | *new-db* | *migrate* ] |
b32b8144 FG |
19 | |
20 | | **ceph-volume** **simple** [ *trigger* | *scan* | *activate* ] | |
21 | ||
b5b8bbf5 FG |
22 | |
23 | Description | |
24 | =========== | |
25 | ||
26 | :program:`ceph-volume` is a single purpose command line tool to deploy logical | |
27 | volumes as OSDs, trying to maintain a similar API to ``ceph-disk`` when | |
28 | preparing, activating, and creating OSDs. | |
29 | ||
30 | It deviates from ``ceph-disk`` by not interacting or relying on the udev rules | |
31 | that come installed for Ceph. These rules allow automatic detection of | |
32 | previously setup devices that are in turn fed into ``ceph-disk`` to activate | |
33 | them. | |
34 | ||
35 | ||
36 | Commands | |
37 | ======== | |
38 | ||
91327a77 AA |
39 | inventory |
40 | --------- | |
41 | ||
42 | This subcommand provides information about a host's physical disc inventory and | |
43 | reports metadata about these discs. Among this metadata one can find disc | |
44 | specific data items (like model, size, rotational or solid state) as well as | |
45 | data items specific to ceph using a device, such as if it is available for | |
46 | use with ceph or if logical volumes are present. | |
47 | ||
48 | Examples:: | |
49 | ||
50 | ceph-volume inventory | |
51 | ceph-volume inventory /dev/sda | |
52 | ceph-volume inventory --format json-pretty | |
53 | ||
54 | Optional arguments: | |
55 | ||
56 | * [-h, --help] show the help message and exit | |
57 | * [--format] report format, valid values are ``plain`` (default), | |
58 | ``json`` and ``json-pretty`` | |
59 | ||
b5b8bbf5 FG |
60 | lvm |
61 | --- | |
62 | ||
63 | By making use of LVM tags, the ``lvm`` sub-command is able to store and later | |
64 | re-discover and query devices associated with OSDs so that they can later | |
65 | activated. | |
66 | ||
67 | Subcommands: | |
68 | ||
1adf2230 AA |
69 | **batch** |
70 | Creates OSDs from a list of devices using a ``filestore`` | |
71 | or ``bluestore`` (default) setup. It will create all necessary volume groups | |
72 | and logical volumes required to have a working OSD. | |
73 | ||
74 | Example usage with three devices:: | |
75 | ||
76 | ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc | |
77 | ||
78 | Optional arguments: | |
79 | ||
80 | * [-h, --help] show the help message and exit | |
81 | * [--bluestore] Use the bluestore objectstore (default) | |
82 | * [--filestore] Use the filestore objectstore | |
83 | * [--yes] Skip the report and prompt to continue provisioning | |
91327a77 | 84 | * [--prepare] Only prepare OSDs, do not activate |
1adf2230 AA |
85 | * [--dmcrypt] Enable encryption for the underlying OSD devices |
86 | * [--crush-device-class] Define a CRUSH device class to assign the OSD to | |
87 | * [--no-systemd] Do not enable or create any systemd units | |
9f95a23c TL |
88 | * [--osds-per-device] Provision more than 1 (the default) OSD per device |
89 | * [--report] Report what the potential outcome would be for the current input (requires devices to be passed in) | |
90 | * [--format] Output format when reporting (used along with --report), can be one of 'pretty' (default) or 'json' | |
91 | * [--block-db-size] Set (or override) the "bluestore_block_db_size" value, in bytes | |
91327a77 | 92 | * [--journal-size] Override the "osd_journal_size" value, in megabytes |
1adf2230 AA |
93 | |
94 | Required positional arguments: | |
95 | ||
96 | * <DEVICE> Full path to a raw device, like ``/dev/sda``. Multiple | |
97 | ``<DEVICE>`` paths can be passed in. | |
98 | ||
99 | ||
b5b8bbf5 FG |
100 | **activate** |
101 | Enables a systemd unit that persists the OSD ID and its UUID (also called | |
102 | ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is | |
103 | enabled and needs to be mounted. | |
104 | ||
105 | Usage:: | |
106 | ||
94b18763 | 107 | ceph-volume lvm activate --bluestore <osd id> <osd fsid> |
b5b8bbf5 FG |
108 | |
109 | Optional Arguments: | |
110 | ||
111 | * [-h, --help] show the help message and exit | |
11fdf7f2 | 112 | * [--auto-detect-objectstore] Automatically detect the objectstore by inspecting |
94b18763 FG |
113 | the OSD |
114 | * [--bluestore] bluestore objectstore (default) | |
115 | * [--filestore] filestore objectstore | |
116 | * [--all] Activate all OSDs found in the system | |
117 | * [--no-systemd] Skip creating and enabling systemd units and starting of OSD | |
118 | services | |
b5b8bbf5 | 119 | |
28e407b8 AA |
120 | Multiple OSDs can be activated at once by using the (idempotent) ``--all`` flag:: |
121 | ||
122 | ceph-volume lvm activate --all | |
123 | ||
b5b8bbf5 FG |
124 | |
125 | **prepare** | |
94b18763 FG |
126 | Prepares a logical volume to be used as an OSD and journal using a ``filestore`` |
127 | or ``bluestore`` (default) setup. It will not create or modify the logical volumes | |
b5b8bbf5 FG |
128 | except for adding extra metadata. |
129 | ||
130 | Usage:: | |
131 | ||
132 | ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device> | |
133 | ||
134 | Optional arguments: | |
135 | ||
136 | * [-h, --help] show the help message and exit | |
137 | * [--journal JOURNAL] A logical group name, path to a logical volume, or path to a device | |
b32b8144 | 138 | * [--bluestore] Use the bluestore objectstore (default) |
94b18763 FG |
139 | * [--block.wal] Path to a bluestore block.wal logical volume or partition |
140 | * [--block.db] Path to a bluestore block.db logical volume or partition | |
b32b8144 FG |
141 | * [--filestore] Use the filestore objectstore |
142 | * [--dmcrypt] Enable encryption for the underlying OSD devices | |
b5b8bbf5 FG |
143 | * [--osd-id OSD_ID] Reuse an existing OSD id |
144 | * [--osd-fsid OSD_FSID] Reuse an existing OSD fsid | |
94b18763 | 145 | * [--crush-device-class] Define a CRUSH device class to assign the OSD to |
b5b8bbf5 FG |
146 | |
147 | Required arguments: | |
148 | ||
149 | * --data A logical group name or a path to a logical volume | |
150 | ||
28e407b8 AA |
151 | For encrypting an OSD, the ``--dmcrypt`` flag must be added when preparing |
152 | (also supported in the ``create`` sub-command). | |
153 | ||
154 | ||
b5b8bbf5 FG |
155 | **create** |
156 | Wraps the two-step process to provision a new osd (calling ``prepare`` first | |
157 | and then ``activate``) into a single one. The reason to prefer ``prepare`` and | |
158 | then ``activate`` is to gradually introduce new OSDs into a cluster, and | |
159 | avoiding large amounts of data being rebalanced. | |
160 | ||
161 | The single-call process unifies exactly what ``prepare`` and ``activate`` do, | |
162 | with the convenience of doing it all at once. Flags and general usage are | |
94b18763 | 163 | equivalent to those of the ``prepare`` and ``activate`` subcommand. |
b5b8bbf5 FG |
164 | |
165 | **trigger** | |
166 | This subcommand is not meant to be used directly, and it is used by systemd so | |
167 | that it proxies input to ``ceph-volume lvm activate`` by parsing the | |
168 | input from systemd, detecting the UUID and ID associated with an OSD. | |
169 | ||
170 | Usage:: | |
171 | ||
172 | ceph-volume lvm trigger <SYSTEMD-DATA> | |
173 | ||
174 | The systemd "data" is expected to be in the format of:: | |
175 | ||
176 | <OSD ID>-<OSD UUID> | |
177 | ||
178 | The lvs associated with the OSD need to have been prepared previously, | |
179 | so that all needed tags and metadata exist. | |
180 | ||
181 | Positional arguments: | |
182 | ||
183 | * <SYSTEMD_DATA> Data from a systemd unit containing ID and UUID of the OSD. | |
184 | ||
b32b8144 FG |
185 | **list** |
186 | List devices or logical volumes associated with Ceph. An association is | |
187 | determined if a device has information relating to an OSD. This is | |
188 | verified by querying LVM's metadata and correlating it with devices. | |
189 | ||
190 | The lvs associated with the OSD need to have been prepared previously by | |
191 | ceph-volume so that all needed tags and metadata exist. | |
192 | ||
193 | Usage:: | |
194 | ||
195 | ceph-volume lvm list | |
196 | ||
197 | List a particular device, reporting all metadata about it:: | |
198 | ||
199 | ceph-volume lvm list /dev/sda1 | |
200 | ||
201 | List a logical volume, along with all its metadata (vg is a volume | |
202 | group, and lv the logical volume name):: | |
203 | ||
204 | ceph-volume lvm list {vg/lv} | |
205 | ||
206 | Positional arguments: | |
207 | ||
94b18763 FG |
208 | * <DEVICE> Either in the form of ``vg/lv`` for logical volumes, |
209 | ``/path/to/sda1`` or ``/path/to/sda`` for regular devices. | |
b32b8144 FG |
210 | |
211 | ||
212 | **zap** | |
213 | Zaps the given logical volume or partition. If given a path to a logical | |
9f95a23c | 214 | volume it must be in the format of vg/lv. Any file systems present |
b32b8144 FG |
215 | on the given lv or partition will be removed and all data will be purged. |
216 | ||
217 | However, the lv or partition will be kept intact. | |
218 | ||
219 | Usage, for logical volumes:: | |
220 | ||
221 | ceph-volume lvm zap {vg/lv} | |
222 | ||
223 | Usage, for logical partitions:: | |
224 | ||
225 | ceph-volume lvm zap /dev/sdc1 | |
226 | ||
f64942e4 AA |
227 | For full removal of the device use the ``--destroy`` flag (allowed for all |
228 | device types):: | |
229 | ||
230 | ceph-volume lvm zap --destroy /dev/sdc1 | |
231 | ||
232 | Multiple devices can be removed by specifying the OSD ID and/or the OSD FSID:: | |
233 | ||
234 | ceph-volume lvm zap --destroy --osd-id 1 | |
235 | ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59 | |
236 | ||
237 | ||
b32b8144 FG |
238 | Positional arguments: |
239 | ||
94b18763 FG |
240 | * <DEVICE> Either in the form of ``vg/lv`` for logical volumes, |
241 | ``/path/to/sda1`` or ``/path/to/sda`` for regular devices. | |
b32b8144 FG |
242 | |
243 | ||
ec96510d FG |
244 | **new-wal** |
245 | Attaches the given logical volume to OSD as a WAL. Logical volume | |
246 | name format is vg/lv. Fails if OSD has already got attached WAL. | |
247 | ||
248 | Usage:: | |
249 | ||
250 | ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target TARGET_LV | |
251 | ||
252 | Optional arguments: | |
253 | ||
254 | * [-h, --help] show the help message and exit | |
255 | ||
256 | Required arguments: | |
257 | ||
258 | * --osd-id OSD_ID OSD id to attach new WAL to | |
259 | * --osd-fsid OSD_FSID OSD fsid to attach new WAL to | |
260 | * --target TARGET_LV logical volume name to attach as WAL | |
261 | ||
262 | ||
263 | **new-db** | |
264 | Attaches the given logical volume to OSD as a DB. Logical volume | |
265 | name format is vg/lv. Fails if OSD has already got attached DB. | |
266 | ||
267 | Usage:: | |
268 | ||
269 | ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> | |
270 | ||
271 | Optional arguments: | |
272 | ||
273 | * [-h, --help] show the help message and exit | |
274 | ||
275 | Required arguments: | |
276 | ||
277 | * --osd-id OSD_ID OSD id to attach new DB to | |
278 | * --osd-fsid OSD_FSID OSD fsid to attach new DB to | |
279 | * --target TARGET_LV logical volume name to attach as DB | |
280 | ||
281 | **migrate** | |
282 | ||
283 | Moves BlueFS data from source volume(s) to the target one, source volumes | |
284 | (except the main, i.e. data or block one) are removed on success. LVM volumes | |
285 | are permitted for Target only, both already attached or new one. In the latter | |
286 | case it is attached to the OSD replacing one of the source devices. Following | |
287 | replacement rules apply (in the order of precedence, stop on the first match): | |
288 | ||
289 | - if source list has DB volume - target device replaces it. | |
290 | - if source list has WAL volume - target device replace it. | |
291 | - if source list has slow volume only - operation is not permitted, | |
292 | requires explicit allocation via new-db/new-wal command. | |
293 | ||
294 | Usage:: | |
295 | ||
296 | ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target TARGET_LV --from {data|db|wal} [{data|db|wal} ...] | |
297 | ||
298 | Optional arguments: | |
299 | ||
300 | * [-h, --help] show the help message and exit | |
301 | ||
302 | Required arguments: | |
303 | ||
304 | * --osd-id OSD_ID OSD id to perform migration at | |
305 | * --osd-fsid OSD_FSID OSD fsid to perform migration at | |
306 | * --target TARGET_LV logical volume to move data to | |
307 | * --from TYPE_LIST list of source device type names, e.g. --from db wal | |
308 | ||
b32b8144 FG |
309 | simple |
310 | ------ | |
311 | ||
312 | Scan legacy OSD directories or data devices that may have been created by | |
313 | ceph-disk, or manually. | |
314 | ||
315 | Subcommands: | |
316 | ||
317 | **activate** | |
318 | Enables a systemd unit that persists the OSD ID and its UUID (also called | |
319 | ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is | |
320 | enabled and needs to be mounted, while reading information that was previously | |
321 | created and persisted at ``/etc/ceph/osd/`` in JSON format. | |
322 | ||
323 | Usage:: | |
324 | ||
325 | ceph-volume simple activate --bluestore <osd id> <osd fsid> | |
326 | ||
327 | Optional Arguments: | |
328 | ||
329 | * [-h, --help] show the help message and exit | |
330 | * [--bluestore] bluestore objectstore (default) | |
331 | * [--filestore] filestore objectstore | |
332 | ||
333 | Note: It requires a matching JSON file with the following format:: | |
334 | ||
335 | /etc/ceph/osd/<osd id>-<osd fsid>.json | |
336 | ||
337 | ||
338 | **scan** | |
339 | Scan a running OSD or data device for an OSD for metadata that can later be | |
340 | used to activate and manage the OSD with ceph-volume. The scan method will | |
341 | create a JSON file with the required information plus anything found in the OSD | |
342 | directory as well. | |
343 | ||
344 | Optionally, the JSON blob can be sent to stdout for further inspection. | |
345 | ||
a8e16298 TL |
346 | Usage on all running OSDs:: |
347 | ||
9f95a23c | 348 | ceph-volume simple scan |
a8e16298 | 349 | |
b32b8144 FG |
350 | Usage on data devices:: |
351 | ||
352 | ceph-volume simple scan <data device> | |
353 | ||
354 | Running OSD directories:: | |
355 | ||
356 | ceph-volume simple scan <path to osd dir> | |
357 | ||
358 | ||
359 | Optional arguments: | |
360 | ||
361 | * [-h, --help] show the help message and exit | |
362 | * [--stdout] Send the JSON blob to stdout | |
363 | * [--force] If the JSON file exists at destination, overwrite it | |
364 | ||
a8e16298 | 365 | Optional Positional arguments: |
b32b8144 FG |
366 | |
367 | * <DATA DEVICE or OSD DIR> Actual data partition or a path to the running OSD | |
368 | ||
369 | **trigger** | |
370 | This subcommand is not meant to be used directly, and it is used by systemd so | |
371 | that it proxies input to ``ceph-volume simple activate`` by parsing the | |
372 | input from systemd, detecting the UUID and ID associated with an OSD. | |
373 | ||
374 | Usage:: | |
375 | ||
376 | ceph-volume simple trigger <SYSTEMD-DATA> | |
377 | ||
378 | The systemd "data" is expected to be in the format of:: | |
379 | ||
380 | <OSD ID>-<OSD UUID> | |
381 | ||
382 | The JSON file associated with the OSD need to have been persisted previously by | |
383 | a scan (or manually), so that all needed metadata can be used. | |
384 | ||
385 | Positional arguments: | |
386 | ||
387 | * <SYSTEMD_DATA> Data from a systemd unit containing ID and UUID of the OSD. | |
388 | ||
389 | ||
b5b8bbf5 FG |
390 | Availability |
391 | ============ | |
392 | ||
393 | :program:`ceph-volume` is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to | |
394 | the documentation at http://docs.ceph.com/ for more information. | |
395 | ||
396 | ||
397 | See also | |
398 | ======== | |
399 | ||
400 | :doc:`ceph-osd <ceph-osd>`\(8), |