ceph/doc/man/8/ceph-volume.rst

   1 :orphan:
   2
   3 =======================================================
   4  ceph-volume -- Ceph OSD deployment and inspection tool
   5 =======================================================
   6
   7 .. program:: ceph-volume
   8
   9 Synopsis
  10 ========
  11
  12 **ceph-volume** [-h] [--cluster CLUSTER] [--log-level LOG_LEVEL]
  13 [--log-path LOG_PATH]
  14
  15 **ceph-volume** **inventory**
  16
  17 **ceph-volume** **lvm** [ *trigger* | *create* | *activate* | *prepare*
  18 | *zap* | *list* | *batch* | *new-wal* | *new-db* | *migrate* ]
  19
  20 **ceph-volume** **simple** [ *trigger* | *scan* | *activate* ]
  21
  22
  23 Description
  24 ===========
  25
  26 :program:`ceph-volume` is a single purpose command line tool to deploy logical
  27 volumes as OSDs, trying to maintain a similar API to ``ceph-disk`` when
  28 preparing, activating, and creating OSDs.
  29
  30 It deviates from ``ceph-disk`` by not interacting or relying on the udev rules
  31 that come installed for Ceph. These rules allow automatic detection of
  32 previously setup devices that are in turn fed into ``ceph-disk`` to activate
  33 them.
  34
  35
  36 Commands
  37 ========
  38
  39 inventory
  40 ---------
  41
  42 .. program:: ceph-volume inventory
  43
  44 This subcommand provides information about a host's physical disc inventory and
  45 reports metadata about these discs. Among this metadata one can find disc
  46 specific data items (like model, size, rotational or solid state) as well as
  47 data items specific to ceph using a device, such as if it is available for
  48 use with ceph or if logical volumes are present.
  49
  50 Examples::
  51
  52     ceph-volume inventory
  53     ceph-volume inventory /dev/sda
  54     ceph-volume inventory --format json-pretty
  55
  56 Optional arguments:
  57
  58 .. option:: -h, --help
  59
  60    show the help message and exit
  61
  62 .. option:: --format
  63
  64    report format, valid values are ``plain`` (default),
  65    ``json`` and ``json-pretty``
  66
  67 lvm
  68 ---
  69
  70 .. program:: ceph-volume lvm
  71
  72 By making use of LVM tags, the ``lvm`` sub-command is able to store and later
  73 re-discover and query devices associated with OSDs so that they can later
  74 activated.
  75
  76 Subcommands:
  77
  78 batch
  79 ^^^^^
  80
  81 .. program:: ceph-volume lvm batch
  82
  83 Creates OSDs from a list of devices using a ``filestore``
  84 or ``bluestore`` (default) setup. It will create all necessary volume groups
  85 and logical volumes required to have a working OSD.
  86
  87 Example usage with three devices::
  88
  89     ceph-volume lvm batch --bluestore /dev/sda /dev/sdb /dev/sdc
  90
  91 Optional arguments:
  92
  93 .. option:: -h, --help
  94
  95    show the help message and exit
  96
  97 .. option:: --bluestore
  98
  99    Use the bluestore objectstore (default)
 100
 101 .. option:: --filestore
 102
 103    Use the filestore objectstore
 104
 105 .. option:: --yes
 106
 107    Skip the report and prompt to continue provisioning
 108
 109 .. option:: --prepare
 110
 111    Only prepare OSDs, do not activate
 112
 113 .. option:: --dmcrypt
 114
 115    Enable encryption for the underlying OSD devices
 116
 117 .. option:: --crush-device-class
 118
 119    Define a CRUSH device class to assign the OSD to
 120
 121 .. option:: --no-systemd
 122
 123    Do not enable or create any systemd units
 124
 125 .. option:: --osds-per-device
 126
 127    Provision more than 1 (the default) OSD per device
 128
 129 .. option:: --report
 130
 131    Report what the potential outcome would be for the current input (requires devices
 132    to be passed in)
 133
 134 .. option:: --format
 135
 136    Output format when reporting (used along with --report), can be one of 'pretty'
 137    (default) or 'json'
 138
 139 .. option:: --block-db-size
 140
 141    Set (or override) the "bluestore_block_db_size" value, in bytes
 142
 143 .. option:: --journal-size
 144
 145    Override the "osd_journal_size" value, in megabytes
 146
 147 Required positional arguments:
 148
 149 .. describe:: <DEVICE>
 150
 151    Full path to a raw device, like ``/dev/sda``. Multiple
 152    ``<DEVICE>`` paths can be passed in.
 153
 154 activate
 155 ^^^^^^^^
 156
 157 .. program:: ceph-volume lvm activate
 158
 159 Enables a systemd unit that persists the OSD ID and its UUID (also called
 160 ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
 161 enabled and needs to be mounted.
 162
 163 Usage::
 164
 165     ceph-volume lvm activate --bluestore <osd id> <osd fsid>
 166
 167 Optional Arguments:
 168
 169 .. option:: -h, --help
 170
 171    show the help message and exit
 172
 173 .. option:: --auto-detect-objectstore
 174
 175    Automatically detect the objectstore by inspecting
 176    the OSD
 177
 178 .. option:: --bluestore
 179
 180    bluestore objectstore (default)
 181
 182 .. option:: --filestore
 183
 184    filestore objectstore
 185
 186 .. option:: --all
 187
 188    Activate all OSDs found in the system
 189
 190 .. option:: --no-systemd
 191
 192    Skip creating and enabling systemd units and starting of OSD
 193    services
 194
 195 Multiple OSDs can be activated at once by using the (idempotent) ``--all`` flag::
 196
 197     ceph-volume lvm activate --all
 198
 199
 200 prepare
 201 ^^^^^^^
 202
 203 .. program:: ceph-volume lvm prepare
 204
 205 Prepares a logical volume to be used as an OSD and journal using a ``filestore``
 206 or ``bluestore`` (default) setup. It will not create or modify the logical volumes
 207 except for adding extra metadata.
 208
 209 Usage::
 210
 211     ceph-volume lvm prepare --filestore --data <data lv> --journal <journal device>
 212
 213 Optional arguments:
 214
 215 .. option:: -h, --help
 216
 217    show the help message and exit
 218
 219 .. option:: --journal JOURNAL
 220
 221    logical group name, path to a logical volume, or path to a device
 222
 223 .. option:: --bluestore
 224
 225    Use the bluestore objectstore (default)
 226
 227 .. option:: --block.wal
 228
 229    Path to a bluestore block.wal logical volume or partition
 230
 231 .. option:: --block.db
 232
 233    Path to a bluestore block.db logical volume or partition
 234
 235 .. option:: --filestore
 236
 237    Use the filestore objectstore
 238
 239 .. option:: --dmcrypt
 240
 241    Enable encryption for the underlying OSD devices
 242
 243 .. option:: --osd-id OSD_ID
 244
 245    Reuse an existing OSD id
 246
 247 .. option:: --osd-fsid OSD_FSID
 248
 249    Reuse an existing OSD fsid
 250
 251 .. option:: --crush-device-class
 252
 253    Define a CRUSH device class to assign the OSD to
 254
 255 Required arguments:
 256
 257 .. option:: --data
 258
 259    A logical group name or a path to a logical volume
 260
 261 For encrypting an OSD, the ``--dmcrypt`` flag must be added when preparing
 262 (also supported in the ``create`` sub-command).
 263
 264
 265 create
 266 ^^^^^^
 267
 268 Wraps the two-step process to provision a new osd (calling ``prepare`` first
 269 and then ``activate``) into a single one. The reason to prefer ``prepare`` and
 270 then ``activate`` is to gradually introduce new OSDs into a cluster, and
 271 avoiding large amounts of data being rebalanced.
 272
 273 The single-call process unifies exactly what ``prepare`` and ``activate`` do,
 274 with the convenience of doing it all at once. Flags and general usage are
 275 equivalent to those of the ``prepare`` and ``activate`` subcommand.
 276
 277 trigger
 278 ^^^^^^^
 279
 280 This subcommand is not meant to be used directly, and it is used by systemd so
 281 that it proxies input to ``ceph-volume lvm activate`` by parsing the
 282 input from systemd, detecting the UUID and ID associated with an OSD.
 283
 284 Usage::
 285
 286     ceph-volume lvm trigger <SYSTEMD-DATA>
 287
 288 The systemd "data" is expected to be in the format of::
 289
 290     <OSD ID>-<OSD UUID>
 291
 292 The lvs associated with the OSD need to have been prepared previously,
 293 so that all needed tags and metadata exist.
 294
 295 Positional arguments:
 296
 297 .. describe:: <SYSTEMD_DATA>
 298
 299    Data from a systemd unit containing ID and UUID of the OSD.
 300
 301 list
 302 ^^^^
 303
 304 List devices or logical volumes associated with Ceph. An association is
 305 determined if a device has information relating to an OSD. This is
 306 verified by querying LVM's metadata and correlating it with devices.
 307
 308 The lvs associated with the OSD need to have been prepared previously by
 309 ceph-volume so that all needed tags and metadata exist.
 310
 311 Usage::
 312
 313     ceph-volume lvm list
 314
 315 List a particular device, reporting all metadata about it::
 316
 317     ceph-volume lvm list /dev/sda1
 318
 319 List a logical volume, along with all its metadata (vg is a volume
 320 group, and lv the logical volume name)::
 321
 322     ceph-volume lvm list {vg/lv}
 323
 324 Positional arguments:
 325
 326 .. describe:: <DEVICE>
 327
 328    Either in the form of ``vg/lv`` for logical volumes,
 329    ``/path/to/sda1`` or ``/path/to/sda`` for regular devices.
 330
 331
 332 zap
 333 ^^^
 334
 335 Zaps the given logical volume or partition. If given a path to a logical
 336 volume it must be in the format of vg/lv. Any file systems present
 337 on the given lv or partition will be removed and all data will be purged.
 338
 339 However, the lv or partition will be kept intact.
 340
 341 Usage, for logical volumes::
 342
 343       ceph-volume lvm zap {vg/lv}
 344
 345 Usage, for logical partitions::
 346
 347       ceph-volume lvm zap /dev/sdc1
 348
 349 For full removal of the device use the ``--destroy`` flag (allowed for all
 350 device types)::
 351
 352       ceph-volume lvm zap --destroy /dev/sdc1
 353
 354 Multiple devices can be removed by specifying the OSD ID and/or the OSD FSID::
 355
 356       ceph-volume lvm zap --destroy --osd-id 1
 357       ceph-volume lvm zap --destroy --osd-id 1 --osd-fsid C9605912-8395-4D76-AFC0-7DFDAC315D59
 358
 359
 360 Positional arguments:
 361
 362 .. describe:: <DEVICE>
 363
 364    Either in the form of ``vg/lv`` for logical volumes,
 365    ``/path/to/sda1`` or ``/path/to/sda`` for regular devices.
 366
 367
 368 new-wal
 369 ^^^^^^^
 370
 371 .. program:: ceph-volume lvm new-wal
 372
 373 Attaches the given logical volume to OSD as a WAL. Logical volume
 374 name format is vg/lv. Fails if OSD has already got attached WAL.
 375
 376 Usage::
 377
 378     ceph-volume lvm new-wal --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
 379
 380 Optional arguments:
 381
 382 .. option:: -h, --help
 383
 384    show the help message and exit
 385
 386 .. option:: --no-systemd
 387
 388    Skip checking OSD systemd unit
 389
 390 Required arguments:
 391
 392 .. option:: --target
 393
 394    logical volume name to attach as WAL
 395
 396 new-db
 397 ^^^^^^
 398
 399 .. program:: ceph-volume lvm new-db
 400
 401 Attaches the given logical volume to OSD as a DB. Logical volume
 402 name format is vg/lv. Fails if OSD has already got attached DB.
 403
 404 Usage::
 405
 406     ceph-volume lvm new-db --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv>
 407
 408 Optional arguments:
 409
 410 .. option:: -h, --help
 411
 412    show the help message and exit
 413
 414 .. option:: --no-systemd
 415
 416    Skip checking OSD systemd unit
 417
 418 Required arguments:
 419
 420 .. option:: --target
 421
 422    logical volume name to attach as DB
 423
 424 migrate
 425 ^^^^^^^
 426
 427 .. program:: ceph-volume lvm migrate
 428
 429 Moves BlueFS data from source volume(s) to the target one, source volumes
 430 (except the main, i.e. data or block one) are removed on success. LVM volumes
 431 are permitted for Target only, both already attached or new one. In the latter
 432 case it is attached to the OSD replacing one of the source devices. Following
 433 replacement rules apply (in the order of precedence, stop on the first match):
 434
 435     - if source list has DB volume - target device replaces it.
 436     - if source list has WAL volume - target device replace it.
 437     - if source list has slow volume only - operation is not permitted,
 438       requires explicit allocation via new-db/new-wal command.
 439
 440 Usage::
 441
 442     ceph-volume lvm migrate --osd-id OSD_ID --osd-fsid OSD_FSID --target <target lv> --from {data|db|wal} [{data|db|wal} ...]
 443
 444 Optional arguments:
 445
 446 .. option:: -h, --help
 447
 448    show the help message and exit
 449
 450 .. option:: --no-systemd
 451
 452    Skip checking OSD systemd unit
 453
 454 Required arguments:
 455
 456 .. option:: --from
 457
 458    list of source device type names
 459
 460 .. option:: --target
 461
 462    logical volume to move data to
 463
 464 simple
 465 ------
 466
 467 Scan legacy OSD directories or data devices that may have been created by
 468 ceph-disk, or manually.
 469
 470 Subcommands:
 471
 472 activate
 473 ^^^^^^^^
 474
 475 .. program:: ceph-volume simple activate
 476
 477 Enables a systemd unit that persists the OSD ID and its UUID (also called
 478 ``fsid`` in Ceph CLI tools), so that at boot time it can understand what OSD is
 479 enabled and needs to be mounted, while reading information that was previously
 480 created and persisted at ``/etc/ceph/osd/`` in JSON format.
 481
 482 Usage::
 483
 484     ceph-volume simple activate --bluestore <osd id> <osd fsid>
 485
 486 Optional Arguments:
 487
 488 .. option:: -h, --help
 489
 490    show the help message and exit
 491
 492 .. option:: --bluestore
 493
 494    bluestore objectstore (default)
 495
 496 .. option:: --filestore
 497
 498    filestore objectstore
 499
 500 .. note::
 501
 502    It requires a matching JSON file with the following format::
 503
 504     /etc/ceph/osd/<osd id>-<osd fsid>.json
 505
 506
 507 scan
 508 ^^^^
 509
 510 .. program:: ceph-volume simple scan
 511
 512 Scan a running OSD or data device for an OSD for metadata that can later be
 513 used to activate and manage the OSD with ceph-volume. The scan method will
 514 create a JSON file with the required information plus anything found in the OSD
 515 directory as well.
 516
 517 Optionally, the JSON blob can be sent to stdout for further inspection.
 518
 519 Usage on all running OSDs::
 520
 521     ceph-volume simple scan
 522
 523 Usage on data devices::
 524
 525     ceph-volume simple scan <data device>
 526
 527 Running OSD directories::
 528
 529     ceph-volume simple scan <path to osd dir>
 530
 531
 532 Optional arguments:
 533
 534 .. option:: -h, --help
 535
 536    show the help message and exit
 537
 538 .. option:: --stdout
 539
 540    Send the JSON blob to stdout
 541
 542 .. option:: --force
 543
 544    If the JSON file exists at destination, overwrite it
 545
 546 Optional Positional arguments:
 547
 548 .. describe:: <DATA DEVICE or OSD DIR>
 549
 550    Actual data partition or a path to the running OSD
 551
 552 trigger
 553 ^^^^^^^
 554
 555 This subcommand is not meant to be used directly, and it is used by systemd so
 556 that it proxies input to ``ceph-volume simple activate`` by parsing the
 557 input from systemd, detecting the UUID and ID associated with an OSD.
 558
 559 Usage::
 560
 561     ceph-volume simple trigger <SYSTEMD-DATA>
 562
 563 The systemd "data" is expected to be in the format of::
 564
 565     <OSD ID>-<OSD UUID>
 566
 567 The JSON file associated with the OSD need to have been persisted previously by
 568 a scan (or manually), so that all needed metadata can be used.
 569
 570 Positional arguments:
 571
 572 .. describe:: <SYSTEMD_DATA>
 573
 574    Data from a systemd unit containing ID and UUID of the OSD.
 575
 576
 577 Availability
 578 ============
 579
 580 :program:`ceph-volume` is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to
 581 the documentation at http://docs.ceph.com/ for more information.
 582
 583
 584 See also
 585 ========
 586
 587 :doc:`ceph-osd <ceph-osd>`\(8),