ceph/doc/man/8/ceph-volume.rst

   1 :orphan:
   2
   3 =======================================================
   4  ceph-volume -- Ceph OSD deployment and inspection tool
   5 =======================================================
   6
   7 .. program:: ceph-volume
   8
   9 Synopsis
  10 ========
  11
  12 **ceph-volume** [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
  13 [--log-path LOG_PATH]
  14
  15 **ceph-volume** **inventory**
  16
  17 **ceph-volume** **lvm** [ *trigger* | *create* | *activate* | *prepare*
  18 | *zap* | *list* | *batch* | *new-wal* | *new-db* | *migrate* ]
  19
  20 **ceph-volume** **simple** [ *trigger* | *scan* | *activate* ]
  21
  22
  23 Description
  24 ===========
  25
  26 :program:`ceph-volume` is a single purpose command line tool to deploy logical
  27 volumes as OSDs, trying to maintain a similar API to ``ceph-disk`` when
  28 preparing, activating, and creating OSDs.
  29
  30 It deviates from ``ceph-disk`` by not interacting or relying on the udev rules
  31 that come installed for Ceph. These rules allow automatic detection of
  32 previously setup devices that are in turn fed into ``ceph-disk`` to activate
  33 them.
  34
  35
  36 Commands
  37 ========
  38
  39 inventory
  40 ---------
  41
  42 .. program:: ceph-volume inventory
  43
  44 This subcommand provides information about a host's physical disc inventory and
  45 reports metadata about these discs. Among this metadata one can find disc
  46 specific data items (like model, size, rotational or solid state) as well as
  47 data items specific to ceph using a device, such as if it is available for
  48 use with ceph or if logical volumes are present.
  49
  50 Examples::
  51
  52     ceph-volume inventory
  53     ceph-volume inventory /dev/sda
  54     ceph-volume inventory --format json-pretty
  55
  56 Optional arguments:
  57
  58 .. option:: -h, --help
  59
  60    show the help message and exit
  61
  62 .. option:: --format
  63
  64    report format, valid values are ``plain`` (default),
  65    ``json`` and ``json-pretty``
  66
  67 lvm
  68 ---
  69
  70 .. program:: ceph-volume lvm
  71
  72 By making use of LVM tags, the ``lvm`` sub-command is able to store and later
  73 re-discover and query devices associated with OSDs so that they can later
  74 activated.
  75
  76 Subcommands:
  77
  78 batch
  79 ^^^^^
  80
  81 .. program:: ceph-volume lvm batch
  82
  83 Creates OSDs from a list of devices using a ``bluestore`` (default) setup.
  84 It will create all necessary volume groups and logical volumes required to have a working OSD.
  85
  86 Example usage with three devices::
  87
  88     ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
  89
  90 Optional arguments:
  91
  92 .. option:: -h, --help
  93
  94    show the help message and exit
  95
  96 .. option:: --bluestore
  97
  98    Use the bluestore objectstore (default)
  99
 100 .. option:: --yes
 101
 102    Skip the report and prompt to continue provisioning
 103
 104 .. option:: --prepare
 105
 106    Only prepare OSDs, do not activate
 107
 108 .. option:: --dmcrypt
 109
 110    Enable encryption for the underlying OSD devices
 111
 112 .. option:: --crush-device-class
 113
 114    Define a CRUSH device class to assign the OSD to
 115
 116 .. option:: --no-systemd
 117
 118    Do not enable or create any systemd units
 119
 120 .. option:: --osds-per-device
 121
 122    Provision more than 1 (the default) OSD per device
 123
 124 .. option:: --report
 125
 126    Report what the potential outcome would be for the current input (requires devices
 127    to be passed in)
 128
 129 .. option:: --format
 130
 131    Output format when reporting (used along with --report), can be one of 'pretty'
 132    (default) or 'json'
 133
 134 .. option:: --block-db-size
 135
 136    Set (or override) the "bluestore_block_db_size" value, in bytes
 137
 138 .. option:: --journal-size
 139
 140    Override the "osd_journal_size" value, in megabytes
 141
 142 Required positional arguments:
 143
 144 .. describe:: <DEVICE>
 145
 146    Full path to a raw device, like ``/dev/sda``. Multiple
 147    ``<DEVICE>`` paths can be passed in.
 148
 149 activate
 150 ^^^^^^^^
 151
 152 .. program:: ceph-volume lvm activate
 153
 154 Enables a systemd unit that persists the OSD ID and its UUID (also called
 155 ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
 156 enabled and needs to be mounted.
 157
 158 Usage::
 159
 160     ceph-volume lvm activate --bluestore <osd id> <osd fsid>
 161
 162 Optional Arguments:
 163
 164 .. option:: -h, --help
 165
 166    show the help message and exit
 167
 168 .. option:: --auto-detect-objectstore
 169
 170    Automatically detect the objectstore by inspecting
 171    the OSD
 172
 173 .. option:: --bluestore
 174
 175    bluestore objectstore (default)
 176
 177 .. option:: --all
 178
 179    Activate all OSDs found in the system
 180
 181 .. option:: --no-systemd
 182
 183    Skip creating and enabling systemd units and starting of OSD
 184    services
 185
 186 Multiple OSDs can be activated at once by using the (idempotent) ``--all`` flag::
 187
 188     ceph-volume lvm activate --all
 189
 190
 191 prepare
 192 ^^^^^^^
 193
 194 .. program:: ceph-volume lvm prepare
 195
 196 Prepares a logical volume to be used as an OSD and journal using a ``bluestore`` (default) setup.
 197 It will not create or modify the logical volumes except for adding extra metadata.
 198
 199 Usage::
 200
 201     ceph-volume lvm prepare --bluestore --data <data lv> --journal <journal device>
 202
 203 Optional arguments:
 204
 205 .. option:: -h, --help
 206
 207    show the help message and exit
 208
 209 .. option:: --journal JOURNAL
 210
 211    logical group name, path to a logical volume, or path to a device
 212
 213 .. option:: --bluestore
 214
 215    Use the bluestore objectstore (default)
 216
 217 .. option:: --block.wal
 218
 219    Path to a bluestore block.wal logical volume or partition
 220
 221 .. option:: --block.db
 222
 223    Path to a bluestore block.db logical volume or partition
 224
 225 .. option:: --dmcrypt
 226
 227    Enable encryption for the underlying OSD devices
 228
 229 .. option:: --osd-id OSD_ID
 230
 231    Reuse an existing OSD id
 232
 233 .. option:: --osd-fsid OSD_FSID
 234
 235    Reuse an existing OSD fsid
 236
 237 .. option:: --crush-device-class
 238
 239    Define a CRUSH device class to assign the OSD to
 240
 241 Required arguments:
 242
 243 .. option:: --data
 244
 245    A logical group name or a path to a logical volume
 246
 247 For encrypting an OSD, the ``--dmcrypt`` flag must be added when preparing
 248 (also supported in the ``create`` sub-command).
 249
 250
 251 create
 252 ^^^^^^
 253
 254 Wraps the two-step process to provision a new osd (calling ``prepare`` first
 255 and then ``activate``) into a single one. The reason to prefer ``prepare`` and
 256 then ``activate`` is to gradually introduce new OSDs into a cluster, and
 257 avoiding large amounts of data being rebalanced.
 258
 259 The single-call process unifies exactly what ``prepare`` and ``activate`` do,
 260 with the convenience of doing it all at once. Flags and general usage are
 261 equivalent to those of the ``prepare`` and ``activate`` subcommand.
 262
 263 trigger
 264 ^^^^^^^
 265
 266 This subcommand is not meant to be used directly, and it is used by systemd so
 267 that it proxies input to ``ceph-volume lvm activate`` by parsing the
 268 input from systemd, detecting the UUID and ID associated with an OSD.
 269
 270 Usage::
 271
 272     ceph-volume lvm trigger <SYSTEMD-DATA>
 273
 274 The systemd "data" is expected to be in the format of::
 275
 276     <OSD ID>-<OSD UUID>
 277
 278 The lvs associated with the OSD need to have been prepared previously,
 279 so that all needed tags and metadata exist.
 280
 281 Positional arguments:
 282
 283 .. describe:: <SYSTEMD_DATA>
 284
 285    Data from a systemd unit containing ID and UUID of the OSD.
 286
 287 list
 288 ^^^^
 289
 290 List devices or logical volumes associated with Ceph. An association is
 291 determined if a device has information relating to an OSD. This is
 292 verified by querying LVM's metadata and correlating it with devices.
 293
 294 The lvs associated with the OSD need to have been prepared previously by
 295 ceph-volume so that all needed tags and metadata exist.
 296
 297 Usage::
 298
 299     ceph-volume lvm list
 300
 301 List a particular device, reporting all metadata about it::
 302
 303     ceph-volume lvm list /dev/sda1
 304
 305 List a logical volume, along with all its metadata (vg is a volume
 306 group, and lv the logical volume name)::
 307
 308     ceph-volume lvm list {vg/lv}
 309
 310 Positional arguments:
 311
 312 .. describe:: <DEVICE>
 313
 314    Either in the form of ``vg/lv`` for logical volumes,
 315    ``/path/to/sda1`` or ``/path/to/sda`` for regular devices.
 316
 317
 318 zap
 319 ^^^
 320
 321 Zaps the given logical volume or partition. If given a path to a logical
 322 volume it must be in the format of vg/lv. Any file systems present
 323 on the given lv or partition will be removed and all data will be purged.
 324
 325 However, the lv or partition will be kept intact.
 326
 327 Usage, for logical volumes::
 328
 329       ceph-volume lvm zap {vg/lv}
 330
 331 Usage, for logical partitions::
 332
 333       ceph-volume lvm zap /dev/sdc1
 334
 335 For full removal of the device use the ``--destroy`` flag (allowed for all
 336 device types)::
 337
 338       ceph-volume lvm zap --destroy /dev/sdc1
 339
 340 Multiple devices can be removed by specifying the OSD ID and/or the OSD FSID::
 341
 342       ceph-volume lvm zap --destroy --osd-id 1
 343       ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
 344
 345
 346 Positional arguments:
 347
 348 .. describe:: <DEVICE>
 349
 350    Either in the form of ``vg/lv`` for logical volumes,
 351    ``/path/to/sda1`` or ``/path/to/sda`` for regular devices.
 352
 353
 354 new-wal
 355 ^^^^^^^
 356
 357 .. program:: ceph-volume lvm new-wal
 358
 359 Attaches the given logical volume to OSD as a WAL. Logical volume
 360 name format is vg/lv. Fails if OSD has already got attached WAL.
 361
 362 Usage::
 363
 364     ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
 365
 366 Optional arguments:
 367
 368 .. option:: -h, --help
 369
 370    show the help message and exit
 371
 372 .. option:: --no-systemd
 373
 374    Skip checking OSD systemd unit
 375
 376 Required arguments:
 377
 378 .. option:: --target
 379
 380    logical volume name to attach as WAL
 381
 382 new-db
 383 ^^^^^^
 384
 385 .. program:: ceph-volume lvm new-db
 386
 387 Attaches the given logical volume to OSD as a DB. Logical volume
 388 name format is vg/lv. Fails if OSD has already got attached DB.
 389
 390 Usage::
 391
 392     ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
 393
 394 Optional arguments:
 395
 396 .. option:: -h, --help
 397
 398    show the help message and exit
 399
 400 .. option:: --no-systemd
 401
 402    Skip checking OSD systemd unit
 403
 404 Required arguments:
 405
 406 .. option:: --target
 407
 408    logical volume name to attach as DB
 409
 410 migrate
 411 ^^^^^^^
 412
 413 .. program:: ceph-volume lvm migrate
 414
 415 Moves BlueFS data from source volume(s) to the target one, source volumes
 416 (except the main, i.e. data or block one) are removed on success. LVM volumes
 417 are permitted for Target only, both already attached or new one. In the latter
 418 case it is attached to the OSD replacing one of the source devices. Following
 419 replacement rules apply (in the order of precedence, stop on the first match):
 420
 421     - if source list has DB volume - target device replaces it.
 422     - if source list has WAL volume - target device replace it.
 423     - if source list has slow volume only - operation is not permitted,
 424       requires explicit allocation via new-db/new-wal command.
 425
 426 Usage::
 427
 428     ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...]
 429
 430 Optional arguments:
 431
 432 .. option:: -h, --help
 433
 434    show the help message and exit
 435
 436 .. option:: --no-systemd
 437
 438    Skip checking OSD systemd unit
 439
 440 Required arguments:
 441
 442 .. option:: --from
 443
 444    list of source device type names
 445
 446 .. option:: --target
 447
 448    logical volume to move data to
 449
 450 simple
 451 ------
 452
 453 Scan legacy OSD directories or data devices that may have been created by
 454 ceph-disk, or manually.
 455
 456 Subcommands:
 457
 458 activate
 459 ^^^^^^^^
 460
 461 .. program:: ceph-volume simple activate
 462
 463 Enables a systemd unit that persists the OSD ID and its UUID (also called
 464 ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
 465 enabled and needs to be mounted, while reading information that was previously
 466 created and persisted at ``/etc/ceph/osd/`` in JSON format.
 467
 468 Usage::
 469
 470     ceph-volume simple activate --bluestore <osd id> <osd fsid>
 471
 472 Optional Arguments:
 473
 474 .. option:: -h, --help
 475
 476    show the help message and exit
 477
 478 .. option:: --bluestore
 479
 480    bluestore objectstore (default)
 481
 482 .. note::
 483
 484    It requires a matching JSON file with the following format::
 485
 486     /etc/ceph/osd/<osd id>-<osd fsid>.json
 487
 488
 489 scan
 490 ^^^^
 491
 492 .. program:: ceph-volume simple scan
 493
 494 Scan a running OSD or data device for an OSD for metadata that can later be
 495 used to activate and manage the OSD with ceph-volume. The scan method will
 496 create a JSON file with the required information plus anything found in the OSD
 497 directory as well.
 498
 499 Optionally, the JSON blob can be sent to stdout for further inspection.
 500
 501 Usage on all running OSDs::
 502
 503     ceph-volume simple scan
 504
 505 Usage on data devices::
 506
 507     ceph-volume simple scan <data device>
 508
 509 Running OSD directories::
 510
 511     ceph-volume simple scan <path to osd dir>
 512
 513
 514 Optional arguments:
 515
 516 .. option:: -h, --help
 517
 518    show the help message and exit
 519
 520 .. option:: --stdout
 521
 522    Send the JSON blob to stdout
 523
 524 .. option:: --force
 525
 526    If the JSON file exists at destination, overwrite it
 527
 528 Optional Positional arguments:
 529
 530 .. describe:: <DATA DEVICE or OSD DIR>
 531
 532    Actual data partition or a path to the running OSD
 533
 534 trigger
 535 ^^^^^^^
 536
 537 This subcommand is not meant to be used directly, and it is used by systemd so
 538 that it proxies input to ``ceph-volume simple activate`` by parsing the
 539 input from systemd, detecting the UUID and ID associated with an OSD.
 540
 541 Usage::
 542
 543     ceph-volume simple trigger <SYSTEMD-DATA>
 544
 545 The systemd "data" is expected to be in the format of::
 546
 547     <OSD ID>-<OSD UUID>
 548
 549 The JSON file associated with the OSD need to have been persisted previously by
 550 a scan (or manually), so that all needed metadata can be used.
 551
 552 Positional arguments:
 553
 554 .. describe:: <SYSTEMD_DATA>
 555
 556    Data from a systemd unit containing ID and UUID of the OSD.
 557
 558
 559 Availability
 560 ============
 561
 562 :program:`ceph-volume` is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to
 563 the documentation at http://docs.ceph.com/ for more information.
 564
 565
 566 See also
 567 ========
 568
 569 :doc:`ceph-osd <ceph-osd>`\(8),