]>
Commit | Line | Data |
---|---|---|
b5b8bbf5 FG |
1 | :orphan: |
2 | ||
91327a77 AA |
3 | ======================================================= |
4 | ceph-volume -- Ceph OSD deployment and inspection tool | |
5 | ======================================================= | |
b5b8bbf5 FG |
6 | |
7 | .. program:: ceph-volume | |
8 | ||
9 | Synopsis | |
10 | ======== | |
11 | ||
20effc67 TL |
12 | **ceph-volume** [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL] |
13 | [--log-path LOG_PATH] | |
b5b8bbf5 | 14 | |
20effc67 | 15 | **ceph-volume** **inventory** |
91327a77 | 16 | |
20effc67 | 17 | **ceph-volume** **lvm** [ *trigger* | *create* | *activate* | *prepare* |
522d829b | 18 | | *zap* | *list* | *batch* | *new-wal* | *new-db* | *migrate* ] |
b32b8144 | 19 | |
20effc67 | 20 | **ceph-volume** **simple** [ *trigger* | *scan* | *activate* ] |
b32b8144 | 21 | |
b5b8bbf5 FG |
22 | |
23 | Description | |
24 | =========== | |
25 | ||
26 | :program:`ceph-volume` is a single purpose command line tool to deploy logical | |
27 | volumes as OSDs, trying to maintain a similar API to ``ceph-disk`` when | |
28 | preparing, activating, and creating OSDs. | |
29 | ||
30 | It deviates from ``ceph-disk`` by not interacting or relying on the udev rules | |
31 | that come installed for Ceph. These rules allow automatic detection of | |
32 | previously setup devices that are in turn fed into ``ceph-disk`` to activate | |
33 | them. | |
34 | ||
35 | ||
36 | Commands | |
37 | ======== | |
38 | ||
91327a77 AA |
39 | inventory |
40 | --------- | |
41 | ||
20effc67 TL |
42 | .. program:: ceph-volume inventory |
43 | ||
91327a77 AA |
44 | This subcommand provides information about a host's physical disc inventory and |
45 | reports metadata about these discs. Among this metadata one can find disc | |
46 | specific data items (like model, size, rotational or solid state) as well as | |
47 | data items specific to ceph using a device, such as if it is available for | |
48 | use with ceph or if logical volumes are present. | |
49 | ||
50 | Examples:: | |
51 | ||
52 | ceph-volume inventory | |
53 | ceph-volume inventory /dev/sda | |
54 | ceph-volume inventory --format json-pretty | |
55 | ||
56 | Optional arguments: | |
57 | ||
20effc67 TL |
58 | .. option:: -h, --help |
59 | ||
60 | show the help message and exit | |
61 | ||
62 | .. option:: --format | |
63 | ||
64 | report format, valid values are ``plain`` (default), | |
65 | ``json`` and ``json-pretty`` | |
91327a77 | 66 | |
b5b8bbf5 FG |
67 | lvm |
68 | --- | |
69 | ||
20effc67 TL |
70 | .. program:: ceph-volume lvm |
71 | ||
b5b8bbf5 FG |
72 | By making use of LVM tags, the ``lvm`` sub-command is able to store and later |
73 | re-discover and query devices associated with OSDs so that they can later | |
74 | activated. | |
75 | ||
76 | Subcommands: | |
77 | ||
20effc67 TL |
78 | batch |
79 | ^^^^^ | |
80 | ||
81 | .. program:: ceph-volume lvm batch | |
82 | ||
1adf2230 AA |
83 | Creates OSDs from a list of devices using a ``filestore`` |
84 | or ``bluestore`` (default) setup. It will create all necessary volume groups | |
85 | and logical volumes required to have a working OSD. | |
86 | ||
87 | Example usage with three devices:: | |
88 | ||
89 | ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc | |
90 | ||
91 | Optional arguments: | |
92 | ||
20effc67 TL |
93 | .. option:: -h, --help |
94 | ||
95 | show the help message and exit | |
96 | ||
97 | .. option:: --bluestore | |
98 | ||
99 | Use the bluestore objectstore (default) | |
100 | ||
101 | .. option:: --filestore | |
102 | ||
103 | Use the filestore objectstore | |
104 | ||
105 | .. option:: --yes | |
106 | ||
107 | Skip the report and prompt to continue provisioning | |
108 | ||
109 | .. option:: --prepare | |
110 | ||
111 | Only prepare OSDs, do not activate | |
112 | ||
113 | .. option:: --dmcrypt | |
114 | ||
115 | Enable encryption for the underlying OSD devices | |
116 | ||
117 | .. option:: --crush-device-class | |
118 | ||
119 | Define a CRUSH device class to assign the OSD to | |
120 | ||
121 | .. option:: --no-systemd | |
122 | ||
123 | Do not enable or create any systemd units | |
124 | ||
125 | .. option:: --osds-per-device | |
126 | ||
127 | Provision more than 1 (the default) OSD per device | |
128 | ||
129 | .. option:: --report | |
130 | ||
131 | Report what the potential outcome would be for the current input (requires devices | |
132 | to be passed in) | |
133 | ||
134 | .. option:: --format | |
135 | ||
136 | Output format when reporting (used along with --report), can be one of 'pretty' | |
137 | (default) or 'json' | |
138 | ||
139 | .. option:: --block-db-size | |
140 | ||
141 | Set (or override) the "bluestore_block_db_size" value, in bytes | |
142 | ||
143 | .. option:: --journal-size | |
144 | ||
145 | Override the "osd_journal_size" value, in megabytes | |
1adf2230 AA |
146 | |
147 | Required positional arguments: | |
148 | ||
20effc67 TL |
149 | .. describe:: <DEVICE> |
150 | ||
151 | Full path to a raw device, like ``/dev/sda``. Multiple | |
152 | ``<DEVICE>`` paths can be passed in. | |
1adf2230 | 153 | |
20effc67 TL |
154 | activate |
155 | ^^^^^^^^ | |
156 | ||
157 | .. program:: ceph-volume lvm activate | |
1adf2230 | 158 | |
b5b8bbf5 FG |
159 | Enables a systemd unit that persists the OSD ID and its UUID (also called |
160 | ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is | |
161 | enabled and needs to be mounted. | |
162 | ||
163 | Usage:: | |
164 | ||
94b18763 | 165 | ceph-volume lvm activate --bluestore <osd id> <osd fsid> |
b5b8bbf5 FG |
166 | |
167 | Optional Arguments: | |
168 | ||
20effc67 TL |
169 | .. option:: -h, --help |
170 | ||
171 | show the help message and exit | |
172 | ||
173 | .. option:: --auto-detect-objectstore | |
174 | ||
175 | Automatically detect the objectstore by inspecting | |
176 | the OSD | |
177 | ||
178 | .. option:: --bluestore | |
179 | ||
180 | bluestore objectstore (default) | |
181 | ||
182 | .. option:: --filestore | |
183 | ||
184 | filestore objectstore | |
185 | ||
186 | .. option:: --all | |
187 | ||
188 | Activate all OSDs found in the system | |
189 | ||
190 | .. option:: --no-systemd | |
191 | ||
192 | Skip creating and enabling systemd units and starting of OSD | |
193 | services | |
b5b8bbf5 | 194 | |
28e407b8 AA |
195 | Multiple OSDs can be activated at once by using the (idempotent) ``--all`` flag:: |
196 | ||
197 | ceph-volume lvm activate --all | |
198 | ||
b5b8bbf5 | 199 | |
20effc67 TL |
200 | prepare |
201 | ^^^^^^^ | |
202 | ||
203 | .. program:: ceph-volume lvm prepare | |
204 | ||
94b18763 FG |
205 | Prepares a logical volume to be used as an OSD and journal using a ``filestore`` |
206 | or ``bluestore`` (default) setup. It will not create or modify the logical volumes | |
b5b8bbf5 FG |
207 | except for adding extra metadata. |
208 | ||
209 | Usage:: | |
210 | ||
211 | ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device> | |
212 | ||
213 | Optional arguments: | |
214 | ||
20effc67 TL |
215 | .. option:: -h, --help |
216 | ||
217 | show the help message and exit | |
218 | ||
219 | .. option:: --journal JOURNAL | |
220 | ||
221 | logical group name, path to a logical volume, or path to a device | |
222 | ||
223 | .. option:: --bluestore | |
224 | ||
225 | Use the bluestore objectstore (default) | |
226 | ||
227 | .. option:: --block.wal | |
228 | ||
229 | Path to a bluestore block.wal logical volume or partition | |
230 | ||
231 | .. option:: --block.db | |
232 | ||
233 | Path to a bluestore block.db logical volume or partition | |
234 | ||
235 | .. option:: --filestore | |
236 | ||
237 | Use the filestore objectstore | |
238 | ||
239 | .. option:: --dmcrypt | |
240 | ||
241 | Enable encryption for the underlying OSD devices | |
242 | ||
243 | .. option:: --osd-id OSD_ID | |
244 | ||
245 | Reuse an existing OSD id | |
246 | ||
247 | .. option:: --osd-fsid OSD_FSID | |
248 | ||
249 | Reuse an existing OSD fsid | |
250 | ||
251 | .. option:: --crush-device-class | |
252 | ||
253 | Define a CRUSH device class to assign the OSD to | |
b5b8bbf5 FG |
254 | |
255 | Required arguments: | |
256 | ||
20effc67 TL |
257 | .. option:: --data |
258 | ||
259 | A logical group name or a path to a logical volume | |
b5b8bbf5 | 260 | |
28e407b8 AA |
261 | For encrypting an OSD, the ``--dmcrypt`` flag must be added when preparing |
262 | (also supported in the ``create`` sub-command). | |
263 | ||
264 | ||
20effc67 TL |
265 | create |
266 | ^^^^^^ | |
267 | ||
b5b8bbf5 FG |
268 | Wraps the two-step process to provision a new osd (calling ``prepare`` first |
269 | and then ``activate``) into a single one. The reason to prefer ``prepare`` and | |
270 | then ``activate`` is to gradually introduce new OSDs into a cluster, and | |
271 | avoiding large amounts of data being rebalanced. | |
272 | ||
273 | The single-call process unifies exactly what ``prepare`` and ``activate`` do, | |
274 | with the convenience of doing it all at once. Flags and general usage are | |
94b18763 | 275 | equivalent to those of the ``prepare`` and ``activate`` subcommand. |
b5b8bbf5 | 276 | |
20effc67 TL |
277 | trigger |
278 | ^^^^^^^ | |
279 | ||
b5b8bbf5 FG |
280 | This subcommand is not meant to be used directly, and it is used by systemd so |
281 | that it proxies input to ``ceph-volume lvm activate`` by parsing the | |
282 | input from systemd, detecting the UUID and ID associated with an OSD. | |
283 | ||
284 | Usage:: | |
285 | ||
286 | ceph-volume lvm trigger <SYSTEMD-DATA> | |
287 | ||
288 | The systemd "data" is expected to be in the format of:: | |
289 | ||
290 | <OSD ID>-<OSD UUID> | |
291 | ||
292 | The lvs associated with the OSD need to have been prepared previously, | |
293 | so that all needed tags and metadata exist. | |
294 | ||
295 | Positional arguments: | |
296 | ||
20effc67 TL |
297 | .. describe:: <SYSTEMD_DATA> |
298 | ||
299 | Data from a systemd unit containing ID and UUID of the OSD. | |
300 | ||
301 | list | |
302 | ^^^^ | |
b5b8bbf5 | 303 | |
b32b8144 FG |
304 | List devices or logical volumes associated with Ceph. An association is |
305 | determined if a device has information relating to an OSD. This is | |
306 | verified by querying LVM's metadata and correlating it with devices. | |
307 | ||
308 | The lvs associated with the OSD need to have been prepared previously by | |
309 | ceph-volume so that all needed tags and metadata exist. | |
310 | ||
311 | Usage:: | |
312 | ||
313 | ceph-volume lvm list | |
314 | ||
315 | List a particular device, reporting all metadata about it:: | |
316 | ||
317 | ceph-volume lvm list /dev/sda1 | |
318 | ||
319 | List a logical volume, along with all its metadata (vg is a volume | |
320 | group, and lv the logical volume name):: | |
321 | ||
322 | ceph-volume lvm list {vg/lv} | |
323 | ||
324 | Positional arguments: | |
325 | ||
20effc67 TL |
326 | .. describe:: <DEVICE> |
327 | ||
328 | Either in the form of ``vg/lv`` for logical volumes, | |
329 | ``/path/to/sda1`` or ``/path/to/sda`` for regular devices. | |
b32b8144 FG |
330 | |
331 | ||
20effc67 TL |
332 | zap |
333 | ^^^ | |
334 | ||
b32b8144 | 335 | Zaps the given logical volume or partition. If given a path to a logical |
9f95a23c | 336 | volume it must be in the format of vg/lv. Any file systems present |
b32b8144 FG |
337 | on the given lv or partition will be removed and all data will be purged. |
338 | ||
339 | However, the lv or partition will be kept intact. | |
340 | ||
341 | Usage, for logical volumes:: | |
342 | ||
343 | ceph-volume lvm zap {vg/lv} | |
344 | ||
345 | Usage, for logical partitions:: | |
346 | ||
347 | ceph-volume lvm zap /dev/sdc1 | |
348 | ||
f64942e4 AA |
349 | For full removal of the device use the ``--destroy`` flag (allowed for all |
350 | device types):: | |
351 | ||
352 | ceph-volume lvm zap --destroy /dev/sdc1 | |
353 | ||
354 | Multiple devices can be removed by specifying the OSD ID and/or the OSD FSID:: | |
355 | ||
356 | ceph-volume lvm zap --destroy --osd-id 1 | |
357 | ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59 | |
358 | ||
359 | ||
b32b8144 FG |
360 | Positional arguments: |
361 | ||
20effc67 TL |
362 | .. describe:: <DEVICE> |
363 | ||
364 | Either in the form of ``vg/lv`` for logical volumes, | |
365 | ``/path/to/sda1`` or ``/path/to/sda`` for regular devices. | |
b32b8144 FG |
366 | |
367 | ||
522d829b TL |
368 | new-wal |
369 | ^^^^^^^ | |
370 | ||
20effc67 TL |
371 | .. program:: ceph-volume lvm new-wal |
372 | ||
522d829b TL |
373 | Attaches the given logical volume to OSD as a WAL. Logical volume |
374 | name format is vg/lv. Fails if OSD has already got attached WAL. | |
375 | ||
376 | Usage:: | |
377 | ||
378 | ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> | |
379 | ||
380 | Optional arguments: | |
381 | ||
382 | .. option:: -h, --help | |
383 | ||
384 | show the help message and exit | |
385 | ||
386 | .. option:: --no-systemd | |
387 | ||
388 | Skip checking OSD systemd unit | |
389 | ||
390 | Required arguments: | |
391 | ||
392 | .. option:: --target | |
393 | ||
394 | logical volume name to attach as WAL | |
395 | ||
396 | new-db | |
397 | ^^^^^^ | |
398 | ||
20effc67 TL |
399 | .. program:: ceph-volume lvm new-db |
400 | ||
522d829b TL |
401 | Attaches the given logical volume to OSD as a DB. Logical volume |
402 | name format is vg/lv. Fails if OSD has already got attached DB. | |
403 | ||
404 | Usage:: | |
405 | ||
406 | ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> | |
407 | ||
408 | Optional arguments: | |
409 | ||
410 | .. option:: -h, --help | |
411 | ||
412 | show the help message and exit | |
413 | ||
414 | .. option:: --no-systemd | |
415 | ||
416 | Skip checking OSD systemd unit | |
417 | ||
418 | Required arguments: | |
419 | ||
420 | .. option:: --target | |
421 | ||
422 | logical volume name to attach as DB | |
423 | ||
424 | migrate | |
425 | ^^^^^^^ | |
426 | ||
20effc67 TL |
427 | .. program:: ceph-volume lvm migrate |
428 | ||
522d829b TL |
429 | Moves BlueFS data from source volume(s) to the target one, source volumes |
430 | (except the main, i.e. data or block one) are removed on success. LVM volumes | |
431 | are permitted for Target only, both already attached or new one. In the latter | |
432 | case it is attached to the OSD replacing one of the source devices. Following | |
433 | replacement rules apply (in the order of precedence, stop on the first match): | |
434 | ||
435 | - if source list has DB volume - target device replaces it. | |
436 | - if source list has WAL volume - target device replace it. | |
437 | - if source list has slow volume only - operation is not permitted, | |
438 | requires explicit allocation via new-db/new-wal command. | |
439 | ||
440 | Usage:: | |
441 | ||
442 | ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...] | |
443 | ||
444 | Optional arguments: | |
445 | ||
446 | .. option:: -h, --help | |
447 | ||
448 | show the help message and exit | |
449 | ||
450 | .. option:: --no-systemd | |
451 | ||
452 | Skip checking OSD systemd unit | |
453 | ||
454 | Required arguments: | |
455 | ||
456 | .. option:: --from | |
457 | ||
458 | list of source device type names | |
459 | ||
460 | .. option:: --target | |
461 | ||
462 | logical volume to move data to | |
463 | ||
b32b8144 FG |
464 | simple |
465 | ------ | |
466 | ||
467 | Scan legacy OSD directories or data devices that may have been created by | |
468 | ceph-disk, or manually. | |
469 | ||
470 | Subcommands: | |
471 | ||
20effc67 TL |
472 | activate |
473 | ^^^^^^^^ | |
474 | ||
475 | .. program:: ceph-volume simple activate | |
476 | ||
b32b8144 FG |
477 | Enables a systemd unit that persists the OSD ID and its UUID (also called |
478 | ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is | |
479 | enabled and needs to be mounted, while reading information that was previously | |
480 | created and persisted at ``/etc/ceph/osd/`` in JSON format. | |
481 | ||
482 | Usage:: | |
483 | ||
484 | ceph-volume simple activate --bluestore <osd id> <osd fsid> | |
485 | ||
486 | Optional Arguments: | |
487 | ||
20effc67 TL |
488 | .. option:: -h, --help |
489 | ||
490 | show the help message and exit | |
491 | ||
492 | .. option:: --bluestore | |
493 | ||
494 | bluestore objectstore (default) | |
495 | ||
496 | .. option:: --filestore | |
497 | ||
498 | filestore objectstore | |
b32b8144 | 499 | |
20effc67 TL |
500 | .. note:: |
501 | ||
502 | It requires a matching JSON file with the following format:: | |
b32b8144 FG |
503 | |
504 | /etc/ceph/osd/<osd id>-<osd fsid>.json | |
505 | ||
506 | ||
20effc67 TL |
507 | scan |
508 | ^^^^ | |
509 | ||
510 | .. program:: ceph-volume simple scan | |
511 | ||
b32b8144 FG |
512 | Scan a running OSD or data device for an OSD for metadata that can later be |
513 | used to activate and manage the OSD with ceph-volume. The scan method will | |
514 | create a JSON file with the required information plus anything found in the OSD | |
515 | directory as well. | |
516 | ||
517 | Optionally, the JSON blob can be sent to stdout for further inspection. | |
518 | ||
a8e16298 TL |
519 | Usage on all running OSDs:: |
520 | ||
9f95a23c | 521 | ceph-volume simple scan |
a8e16298 | 522 | |
b32b8144 FG |
523 | Usage on data devices:: |
524 | ||
525 | ceph-volume simple scan <data device> | |
526 | ||
527 | Running OSD directories:: | |
528 | ||
529 | ceph-volume simple scan <path to osd dir> | |
530 | ||
531 | ||
532 | Optional arguments: | |
533 | ||
20effc67 TL |
534 | .. option:: -h, --help |
535 | ||
536 | show the help message and exit | |
537 | ||
538 | .. option:: --stdout | |
539 | ||
540 | Send the JSON blob to stdout | |
541 | ||
542 | .. option:: --force | |
543 | ||
544 | If the JSON file exists at destination, overwrite it | |
b32b8144 | 545 | |
a8e16298 | 546 | Optional Positional arguments: |
b32b8144 | 547 | |
20effc67 TL |
548 | .. describe:: <DATA DEVICE or OSD DIR> |
549 | ||
550 | Actual data partition or a path to the running OSD | |
551 | ||
552 | trigger | |
553 | ^^^^^^^ | |
b32b8144 | 554 | |
b32b8144 FG |
555 | This subcommand is not meant to be used directly, and it is used by systemd so |
556 | that it proxies input to ``ceph-volume simple activate`` by parsing the | |
557 | input from systemd, detecting the UUID and ID associated with an OSD. | |
558 | ||
559 | Usage:: | |
560 | ||
561 | ceph-volume simple trigger <SYSTEMD-DATA> | |
562 | ||
563 | The systemd "data" is expected to be in the format of:: | |
564 | ||
565 | <OSD ID>-<OSD UUID> | |
566 | ||
567 | The JSON file associated with the OSD need to have been persisted previously by | |
568 | a scan (or manually), so that all needed metadata can be used. | |
569 | ||
570 | Positional arguments: | |
571 | ||
20effc67 TL |
572 | .. describe:: <SYSTEMD_DATA> |
573 | ||
574 | Data from a systemd unit containing ID and UUID of the OSD. | |
b32b8144 FG |
575 | |
576 | ||
b5b8bbf5 FG |
577 | Availability |
578 | ============ | |
579 | ||
580 | :program:`ceph-volume` is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to | |
581 | the documentation at http://docs.ceph.com/ for more information. | |
582 | ||
583 | ||
584 | See also | |
585 | ======== | |
586 | ||
587 | :doc:`ceph-osd <ceph-osd>`\(8), |