]>
Commit | Line | Data |
---|---|---|
81eedcae TL |
1 | .. _configuring-ceph: |
2 | ||
7c673cae FG |
3 | ================== |
4 | Configuring Ceph | |
5 | ================== | |
6 | ||
aee94f69 | 7 | When Ceph services start, the initialization process activates a set of |
05a536ef TL |
8 | daemons that run in the background. A :term:`Ceph Storage Cluster` runs at |
9 | least three types of daemons: | |
7c673cae FG |
10 | |
11 | - :term:`Ceph Monitor` (``ceph-mon``) | |
11fdf7f2 | 12 | - :term:`Ceph Manager` (``ceph-mgr``) |
7c673cae FG |
13 | - :term:`Ceph OSD Daemon` (``ceph-osd``) |
14 | ||
aee94f69 TL |
15 | Any Ceph Storage Cluster that supports the :term:`Ceph File System` also runs |
16 | at least one :term:`Ceph Metadata Server` (``ceph-mds``). Any Cluster that | |
17 | supports :term:`Ceph Object Storage` runs Ceph RADOS Gateway daemons | |
18 | (``radosgw``). | |
7c673cae | 19 | |
aee94f69 TL |
20 | Each daemon has a number of configuration options, and each of those options |
21 | has a default value. Adjust the behavior of the system by changing these | |
22 | configuration options. Make sure to understand the consequences before | |
23 | overriding the default values, as it is possible to significantly degrade the | |
24 | performance and stability of your cluster. Remember that default values | |
05a536ef TL |
25 | sometimes change between releases. For this reason, it is best to review the |
26 | version of this documentation that applies to your Ceph release. | |
7c673cae | 27 | |
11fdf7f2 TL |
28 | Option names |
29 | ============ | |
7c673cae | 30 | |
05a536ef TL |
31 | Each of the Ceph configuration options has a unique name that consists of words |
32 | formed with lowercase characters and connected with underscore characters | |
33 | (``_``). | |
7c673cae | 34 | |
05a536ef TL |
35 | When option names are specified on the command line, underscore (``_``) and |
36 | dash (``-``) characters can be used interchangeably (for example, | |
11fdf7f2 | 37 | ``--mon-host`` is equivalent to ``--mon_host``). |
7c673cae | 38 | |
05a536ef TL |
39 | When option names appear in configuration files, spaces can also be used in |
40 | place of underscores or dashes. However, for the sake of clarity and | |
41 | convenience, we suggest that you consistently use underscores, as we do | |
f67539c2 | 42 | throughout this documentation. |
7c673cae | 43 | |
11fdf7f2 TL |
44 | Config sources |
45 | ============== | |
7c673cae | 46 | |
05a536ef TL |
47 | Each Ceph daemon, process, and library pulls its configuration from one or more |
48 | of the several sources listed below. Sources that occur later in the list | |
49 | override those that occur earlier in the list (when both are present). | |
11fdf7f2 TL |
50 | |
51 | - the compiled-in default value | |
52 | - the monitor cluster's centralized configuration database | |
53 | - a configuration file stored on the local host | |
54 | - environment variables | |
05a536ef TL |
55 | - command-line arguments |
56 | - runtime overrides that are set by an administrator | |
11fdf7f2 TL |
57 | |
58 | One of the first things a Ceph process does on startup is parse the | |
05a536ef TL |
59 | configuration options provided via the command line, via the environment, and |
60 | via the local configuration file. Next, the process contacts the monitor | |
61 | cluster to retrieve centrally-stored configuration for the entire cluster. | |
62 | After a complete view of the configuration is available, the startup of the | |
63 | daemon or process will commence. | |
11fdf7f2 | 64 | |
9f95a23c TL |
65 | .. _bootstrap-options: |
66 | ||
11fdf7f2 TL |
67 | Bootstrap options |
68 | ----------------- | |
69 | ||
05a536ef TL |
70 | Bootstrap options are configuration options that affect the process's ability |
71 | to contact the monitors, to authenticate, and to retrieve the cluster-stored | |
72 | configuration. For this reason, these options might need to be stored locally | |
73 | on the node, and set by means of a local configuration file. These options | |
74 | include the following: | |
20effc67 TL |
75 | |
76 | .. confval:: mon_host | |
77 | .. confval:: mon_host_override | |
78 | ||
79 | - :confval:`mon_dns_srv_name` | |
05a536ef TL |
80 | - :confval:`mon_data`, :confval:`osd_data`, :confval:`mds_data`, |
81 | :confval:`mgr_data`, and similar options that define which local directory | |
82 | the daemon stores its data in. | |
83 | - :confval:`keyring`, :confval:`keyfile`, and/or :confval:`key`, which can be | |
84 | used to specify the authentication credential to use to authenticate with the | |
85 | monitor. Note that in most cases the default keyring location is in the data | |
86 | directory specified above. | |
87 | ||
88 | In most cases, there is no reason to modify the default values of these | |
89 | options. However, there is one exception to this: the :confval:`mon_host` | |
90 | option that identifies the addresses of the cluster's monitors. But when | |
91 | :ref:`DNS is used to identify monitors<mon-dns-lookup>`, a local Ceph | |
20effc67 | 92 | configuration file can be avoided entirely. |
11fdf7f2 | 93 | |
05a536ef | 94 | |
11fdf7f2 TL |
95 | Skipping monitor config |
96 | ----------------------- | |
97 | ||
05a536ef TL |
98 | The option ``--no-mon-config`` can be passed in any command in order to skip |
99 | the step that retrieves configuration information from the cluster's monitors. | |
100 | Skipping this retrieval step can be useful in cases where configuration is | |
101 | managed entirely via configuration files, or when maintenance activity needs to | |
102 | be done but the monitor cluster is down. | |
7c673cae | 103 | |
11fdf7f2 | 104 | .. _ceph-conf-file: |
7c673cae | 105 | |
11fdf7f2 TL |
106 | Configuration sections |
107 | ====================== | |
7c673cae | 108 | |
05a536ef TL |
109 | Each of the configuration options associated with a single process or daemon |
110 | has a single value. However, the values for a configuration option can vary | |
111 | across daemon types, and can vary even across different daemons of the same | |
112 | type. Ceph options that are stored in the monitor configuration database or in | |
113 | local configuration files are grouped into sections |---| so-called "configuration | |
114 | sections" |---| to indicate which daemons or clients they apply to. | |
115 | ||
7c673cae | 116 | |
05a536ef | 117 | These sections include the following: |
7c673cae | 118 | |
20effc67 | 119 | .. confsec:: global |
7c673cae | 120 | |
20effc67 TL |
121 | Settings under ``global`` affect all daemons and clients |
122 | in a Ceph Storage Cluster. | |
7c673cae | 123 | |
20effc67 | 124 | :example: ``log_file = /var/log/ceph/$cluster-$type.$id.log`` |
7c673cae | 125 | |
20effc67 | 126 | .. confsec:: mon |
7c673cae | 127 | |
20effc67 TL |
128 | Settings under ``mon`` affect all ``ceph-mon`` daemons in |
129 | the Ceph Storage Cluster, and override the same setting in | |
130 | ``global``. | |
7c673cae | 131 | |
20effc67 | 132 | :example: ``mon_cluster_log_to_syslog = true`` |
7c673cae | 133 | |
20effc67 | 134 | .. confsec:: mgr |
7c673cae | 135 | |
20effc67 TL |
136 | Settings in the ``mgr`` section affect all ``ceph-mgr`` daemons in |
137 | the Ceph Storage Cluster, and override the same setting in | |
138 | ``global``. | |
7c673cae | 139 | |
20effc67 | 140 | :example: ``mgr_stats_period = 10`` |
7c673cae | 141 | |
20effc67 | 142 | .. confsec:: osd |
7c673cae | 143 | |
20effc67 TL |
144 | Settings under ``osd`` affect all ``ceph-osd`` daemons in |
145 | the Ceph Storage Cluster, and override the same setting in | |
146 | ``global``. | |
7c673cae | 147 | |
20effc67 | 148 | :example: ``osd_op_queue = wpq`` |
7c673cae | 149 | |
20effc67 | 150 | .. confsec:: mds |
7c673cae | 151 | |
20effc67 TL |
152 | Settings in the ``mds`` section affect all ``ceph-mds`` daemons in |
153 | the Ceph Storage Cluster, and override the same setting in | |
154 | ``global``. | |
7c673cae | 155 | |
20effc67 | 156 | :example: ``mds_cache_memory_limit = 10G`` |
7c673cae | 157 | |
20effc67 | 158 | .. confsec:: client |
7c673cae | 159 | |
05a536ef TL |
160 | Settings under ``client`` affect all Ceph clients |
161 | (for example, mounted Ceph File Systems, mounted Ceph Block Devices) | |
162 | as well as RADOS Gateway (RGW) daemons. | |
7c673cae | 163 | |
20effc67 | 164 | :example: ``objecter_inflight_ops = 512`` |
7c673cae | 165 | |
7c673cae | 166 | |
05a536ef | 167 | Configuration sections can also specify an individual daemon or client name. For example, |
11fdf7f2 | 168 | ``mon.foo``, ``osd.123``, and ``client.smith`` are all valid section names. |
7c673cae | 169 | |
7c673cae | 170 | |
05a536ef TL |
171 | Any given daemon will draw its settings from the global section, the daemon- or |
172 | client-type section, and the section sharing its name. Settings in the | |
173 | most-specific section take precedence so precedence: for example, if the same | |
174 | option is specified in both :confsec:`global`, :confsec:`mon`, and ``mon.foo`` | |
175 | on the same source (i.e. that is, in the same configuration file), the | |
176 | ``mon.foo`` setting will be used. | |
7c673cae | 177 | |
9f95a23c | 178 | If multiple values of the same configuration option are specified in the same |
05a536ef | 179 | section, the last value specified takes precedence. |
7c673cae | 180 | |
05a536ef TL |
181 | Note that values from the local configuration file always take precedence over |
182 | values from the monitor configuration database, regardless of the section in | |
183 | which they appear. | |
7c673cae FG |
184 | |
185 | .. _ceph-metavariables: | |
186 | ||
187 | Metavariables | |
188 | ============= | |
189 | ||
05a536ef TL |
190 | Metavariables dramatically simplify Ceph storage cluster configuration. When a |
191 | metavariable is set in a configuration value, Ceph expands the metavariable at | |
192 | the time the configuration value is used. In this way, Ceph metavariables | |
193 | behave similarly to the way that variable expansion works in the Bash shell. | |
7c673cae | 194 | |
05a536ef | 195 | Ceph supports the following metavariables: |
7c673cae | 196 | |
20effc67 | 197 | .. describe:: $cluster |
7c673cae | 198 | |
20effc67 TL |
199 | Expands to the Ceph Storage Cluster name. Useful when running |
200 | multiple Ceph Storage Clusters on the same hardware. | |
7c673cae | 201 | |
20effc67 TL |
202 | :example: ``/etc/ceph/$cluster.keyring`` |
203 | :default: ``ceph`` | |
7c673cae | 204 | |
20effc67 | 205 | .. describe:: $type |
7c673cae | 206 | |
05a536ef | 207 | Expands to a daemon or process type (for example, ``mds``, ``osd``, or ``mon``) |
7c673cae | 208 | |
20effc67 | 209 | :example: ``/var/lib/ceph/$type`` |
7c673cae | 210 | |
20effc67 | 211 | .. describe:: $id |
7c673cae | 212 | |
20effc67 TL |
213 | Expands to the daemon or client identifier. For |
214 | ``osd.0``, this would be ``0``; for ``mds.a``, it would | |
215 | be ``a``. | |
7c673cae | 216 | |
20effc67 | 217 | :example: ``/var/lib/ceph/$type/$cluster-$id`` |
7c673cae | 218 | |
20effc67 | 219 | .. describe:: $host |
7c673cae | 220 | |
20effc67 | 221 | Expands to the host name where the process is running. |
7c673cae | 222 | |
20effc67 | 223 | .. describe:: $name |
7c673cae | 224 | |
20effc67 | 225 | Expands to ``$type.$id``. |
7c673cae | 226 | |
20effc67 | 227 | :example: ``/var/run/ceph/$cluster-$name.asok`` |
7c673cae | 228 | |
20effc67 | 229 | .. describe:: $pid |
7c673cae | 230 | |
20effc67 | 231 | Expands to daemon pid. |
7c673cae | 232 | |
20effc67 | 233 | :example: ``/var/run/ceph/$cluster-$name-$pid.asok`` |
7c673cae FG |
234 | |
235 | ||
05a536ef TL |
236 | Ceph configuration file |
237 | ======================= | |
7c673cae | 238 | |
11fdf7f2 TL |
239 | On startup, Ceph processes search for a configuration file in the |
240 | following locations: | |
7c673cae | 241 | |
05a536ef | 242 | #. ``$CEPH_CONF`` (that is, the path following the ``$CEPH_CONF`` |
11fdf7f2 | 243 | environment variable) |
05a536ef | 244 | #. ``-c path/path`` (that is, the ``-c`` command line argument) |
11fdf7f2 TL |
245 | #. ``/etc/ceph/$cluster.conf`` |
246 | #. ``~/.ceph/$cluster.conf`` | |
05a536ef | 247 | #. ``./$cluster.conf`` (that is, in the current working directory) |
11fdf7f2 | 248 | #. On FreeBSD systems only, ``/usr/local/etc/ceph/$cluster.conf`` |
7c673cae | 249 | |
05a536ef | 250 | Here ``$cluster`` is the cluster's name (default: ``ceph``). |
7c673cae | 251 | |
05a536ef TL |
252 | The Ceph configuration file uses an ``ini`` style syntax. You can add "comment |
253 | text" after a pound sign (#) or a semi-colon semicolon (;). For example: | |
7c673cae | 254 | |
11fdf7f2 | 255 | .. code-block:: ini |
7c673cae | 256 | |
05a536ef TL |
257 | # <--A number (#) sign number sign (#) precedes a comment. |
258 | ; A comment may be anything. | |
259 | # Comments always follow a semi-colon semicolon (;) or a pound sign (#) on each line. | |
260 | # The end of the line terminates a comment. | |
261 | # We recommend that you provide comments in your configuration file(s). | |
7c673cae | 262 | |
7c673cae | 263 | |
11fdf7f2 | 264 | .. _ceph-conf-settings: |
7c673cae | 265 | |
11fdf7f2 TL |
266 | Config file section names |
267 | ------------------------- | |
7c673cae | 268 | |
11fdf7f2 | 269 | The configuration file is divided into sections. Each section must begin with a |
05a536ef TL |
270 | valid configuration section name (see `Configuration sections`_, above) that is |
271 | surrounded by square brackets. For example: | |
7c673cae | 272 | |
11fdf7f2 | 273 | .. code-block:: ini |
7c673cae | 274 | |
05a536ef TL |
275 | [global] |
276 | debug_ms = 0 | |
7c673cae | 277 | |
05a536ef TL |
278 | [osd] |
279 | debug_ms = 1 | |
7c673cae | 280 | |
05a536ef TL |
281 | [osd.1] |
282 | debug_ms = 10 | |
7c673cae | 283 | |
05a536ef TL |
284 | [osd.2] |
285 | debug_ms = 10 | |
7c673cae | 286 | |
9f95a23c TL |
287 | Config file option values |
288 | ------------------------- | |
289 | ||
05a536ef TL |
290 | The value of a configuration option is a string. If the string is too long to |
291 | fit on a single line, you can put a backslash (``\``) at the end of the line | |
292 | and the backslash will act as a line continuation marker. In such a case, the | |
293 | value of the option will be the string after ``=`` in the current line, | |
294 | combined with the string in the next line. Here is an example:: | |
9f95a23c TL |
295 | |
296 | [global] | |
297 | foo = long long ago\ | |
298 | long ago | |
299 | ||
05a536ef TL |
300 | In this example, the value of the "``foo``" option is "``long long ago long |
301 | ago``". | |
9f95a23c | 302 | |
05a536ef TL |
303 | An option value typically ends with either a newline or a comment. For |
304 | example: | |
9f95a23c TL |
305 | |
306 | .. code-block:: ini | |
307 | ||
308 | [global] | |
f67539c2 TL |
309 | obscure_one = difficult to explain # I will try harder in next release |
310 | simpler_one = nothing to explain | |
9f95a23c | 311 | |
05a536ef TL |
312 | In this example, the value of the "``obscure one``" option is "``difficult to |
313 | explain``" and the value of the "``simpler one`` options is "``nothing to | |
314 | explain``". | |
9f95a23c | 315 | |
05a536ef TL |
316 | When an option value contains spaces, it can be enclosed within single quotes |
317 | or double quotes in order to make its scope clear and in order to make sure | |
318 | that the first space in the value is not interpreted as the end of the value. | |
319 | For example: | |
9f95a23c TL |
320 | |
321 | .. code-block:: ini | |
322 | ||
323 | [global] | |
324 | line = "to be, or not to be" | |
325 | ||
05a536ef TL |
326 | In option values, there are four characters that are treated as escape |
327 | characters: ``=``, ``#``, ``;`` and ``[``. They are permitted to occur in an | |
328 | option value only if they are immediately preceded by the backslash character | |
329 | (``\``). For example: | |
9f95a23c TL |
330 | |
331 | .. code-block:: ini | |
332 | ||
333 | [global] | |
334 | secret = "i love \# and \[" | |
335 | ||
05a536ef | 336 | Each configuration option falls under one of the following types: |
f67539c2 | 337 | |
20effc67 TL |
338 | .. describe:: int |
339 | ||
05a536ef TL |
340 | 64-bit signed integer. Some SI suffixes are supported, such as "K", "M", |
341 | "G", "T", "P", and "E" (meaning, respectively, 10\ :sup:`3`, 10\ :sup:`6`, | |
342 | 10\ :sup:`9`, etc.). "B" is the only supported unit string. Thus "1K", "1M", | |
343 | "128B" and "-1" are all valid option values. When a negative value is | |
344 | assigned to a threshold option, this can indicate that the option is | |
345 | "unlimited" -- that is, that there is no threshold or limit in effect. | |
20effc67 TL |
346 | |
347 | :example: ``42``, ``-1`` | |
348 | ||
349 | .. describe:: uint | |
f67539c2 | 350 | |
05a536ef TL |
351 | This differs from ``integer`` only in that negative values are not |
352 | permitted. | |
f67539c2 | 353 | |
20effc67 | 354 | :example: ``256``, ``0`` |
f67539c2 | 355 | |
20effc67 | 356 | .. describe:: str |
f67539c2 | 357 | |
05a536ef TL |
358 | A string encoded in UTF-8. Certain characters are not permitted. Reference |
359 | the above notes for the details. | |
f67539c2 | 360 | |
20effc67 | 361 | :example: ``"hello world"``, ``"i love \#"``, ``yet-another-name`` |
f67539c2 | 362 | |
20effc67 | 363 | .. describe:: boolean |
f67539c2 | 364 | |
05a536ef TL |
365 | Typically either of the two values ``true`` or ``false``. However, any |
366 | integer is permitted: "0" implies ``false``, and any non-zero value implies | |
367 | ``true``. | |
f67539c2 | 368 | |
20effc67 | 369 | :example: ``true``, ``false``, ``1``, ``0`` |
f67539c2 | 370 | |
20effc67 | 371 | .. describe:: addr |
f67539c2 | 372 | |
05a536ef TL |
373 | A single address, optionally prefixed with ``v1``, ``v2`` or ``any`` for the |
374 | messenger protocol. If no prefix is specified, the ``v2`` protocol is used. | |
375 | For more details, see :ref:`address_formats`. | |
f67539c2 | 376 | |
20effc67 | 377 | :example: ``v1:1.2.3.4:567``, ``v2:1.2.3.4:567``, ``1.2.3.4:567``, ``2409:8a1e:8fb6:aa20:1260:4bff:fe92:18f5::567``, ``[::1]:6789`` |
f67539c2 | 378 | |
20effc67 | 379 | .. describe:: addrvec |
f67539c2 | 380 | |
05a536ef TL |
381 | A set of addresses separated by ",". The addresses can be optionally quoted |
382 | with ``[`` and ``]``. | |
f67539c2 | 383 | |
20effc67 | 384 | :example: ``[v1:1.2.3.4:567,v2:1.2.3.4:568]``, ``v1:1.2.3.4:567,v1:1.2.3.14:567`` ``[2409:8a1e:8fb6:aa20:1260:4bff:fe92:18f5::567], [2409:8a1e:8fb6:aa20:1260:4bff:fe92:18f5::568]`` |
f67539c2 | 385 | |
20effc67 | 386 | .. describe:: uuid |
f67539c2 | 387 | |
05a536ef TL |
388 | The string format of a uuid defined by `RFC4122 |
389 | <https://www.ietf.org/rfc/rfc4122.txt>`_. Certain variants are also | |
390 | supported: for more details, see `Boost document | |
391 | <https://www.boost.org/doc/libs/1_74_0/libs/uuid/doc/uuid.html#String%20Generator>`_. | |
f67539c2 | 392 | |
20effc67 TL |
393 | :example: ``f81d4fae-7dec-11d0-a765-00a0c91e6bf6`` |
394 | ||
395 | .. describe:: size | |
396 | ||
05a536ef TL |
397 | 64-bit unsigned integer. Both SI prefixes and IEC prefixes are supported. |
398 | "B" is the only supported unit string. Negative values are not permitted. | |
20effc67 TL |
399 | |
400 | :example: ``1Ki``, ``1K``, ``1KiB`` and ``1B``. | |
401 | ||
402 | .. describe:: secs | |
403 | ||
05a536ef TL |
404 | Denotes a duration of time. The default unit of time is the second. |
405 | The following units of time are supported: | |
f67539c2 | 406 | |
05a536ef TL |
407 | * second: ``s``, ``sec``, ``second``, ``seconds`` |
408 | * minute: ``m``, ``min``, ``minute``, ``minutes`` | |
409 | * hour: ``hs``, ``hr``, ``hour``, ``hours`` | |
410 | * day: ``d``, ``day``, ``days`` | |
411 | * week: ``w``, ``wk``, ``week``, ``weeks`` | |
412 | * month: ``mo``, ``month``, ``months`` | |
413 | * year: ``y``, ``yr``, ``year``, ``years`` | |
20effc67 TL |
414 | |
415 | :example: ``1 m``, ``1m`` and ``1 week`` | |
7c673cae | 416 | |
f91f0fd5 TL |
417 | .. _ceph-conf-database: |
418 | ||
11fdf7f2 TL |
419 | Monitor configuration database |
420 | ============================== | |
7c673cae | 421 | |
05a536ef TL |
422 | The monitor cluster manages a database of configuration options that can be |
423 | consumed by the entire cluster. This allows for streamlined central | |
424 | configuration management of the entire system. For ease of administration and | |
425 | transparency, the vast majority of configuration options can and should be | |
426 | stored in this database. | |
7c673cae | 427 | |
05a536ef TL |
428 | Some settings might need to be stored in local configuration files because they |
429 | affect the ability of the process to connect to the monitors, to authenticate, | |
430 | and to fetch configuration information. In most cases this applies only to the | |
431 | ``mon_host`` option. This issue can be avoided by using :ref:`DNS SRV | |
432 | records<mon-dns-lookup>`. | |
7c673cae | 433 | |
11fdf7f2 TL |
434 | Sections and masks |
435 | ------------------ | |
7c673cae | 436 | |
05a536ef TL |
437 | Configuration options stored by the monitor can be stored in a global section, |
438 | in a daemon-type section, or in a specific daemon section. In this, they are | |
439 | no different from the options in a configuration file. | |
7c673cae | 440 | |
05a536ef TL |
441 | In addition, options may have a *mask* associated with them to further restrict |
442 | which daemons or clients the option applies to. Masks take two forms: | |
7c673cae | 443 | |
05a536ef TL |
444 | #. ``type:location`` where ``type`` is a CRUSH property like ``rack`` or |
445 | ``host``, and ``location`` is a value for that property. For example, | |
11fdf7f2 TL |
446 | ``host:foo`` would limit the option only to daemons or clients |
447 | running on a particular host. | |
05a536ef TL |
448 | #. ``class:device-class`` where ``device-class`` is the name of a CRUSH |
449 | device class (for example, ``hdd`` or ``ssd``). For example, | |
11fdf7f2 | 450 | ``class:ssd`` would limit the option only to OSDs backed by SSDs. |
05a536ef | 451 | (This mask has no effect on non-OSD daemons or clients.) |
7c673cae | 452 | |
05a536ef TL |
453 | In commands that specify a configuration option, the argument of the option (in |
454 | the following examples, this is the "who" string) may be a section name, a | |
455 | mask, or a combination of both separated by a slash character (``/``). For | |
456 | example, ``osd/rack:foo`` would refer to all OSD daemons in the ``foo`` rack. | |
7c673cae | 457 | |
05a536ef TL |
458 | When configuration options are shown, the section name and mask are presented |
459 | in separate fields or columns to make them more readable. | |
7c673cae | 460 | |
11fdf7f2 TL |
461 | Commands |
462 | -------- | |
7c673cae | 463 | |
11fdf7f2 | 464 | The following CLI commands are used to configure the cluster: |
7c673cae | 465 | |
05a536ef | 466 | * ``ceph config dump`` dumps the entire monitor configuration |
1e59de90 | 467 | database for the cluster. |
7c673cae | 468 | |
05a536ef TL |
469 | * ``ceph config get <who>`` dumps the configuration options stored in |
470 | the monitor configuration database for a specific daemon or client | |
471 | (for example, ``mds.a``). | |
1e59de90 | 472 | |
05a536ef TL |
473 | * ``ceph config get <who> <option>`` shows either a configuration value |
474 | stored in the monitor configuration database for a specific daemon or client | |
475 | (for example, ``mds.a``), or, if that value is not present in the monitor | |
1e59de90 | 476 | configuration database, the compiled-in default value. |
7c673cae | 477 | |
05a536ef TL |
478 | * ``ceph config set <who> <option> <value>`` specifies a configuration |
479 | option in the monitor configuration database. | |
7c673cae | 480 | |
05a536ef TL |
481 | * ``ceph config show <who>`` shows the configuration for a running daemon. |
482 | These settings might differ from those stored by the monitors if there are | |
483 | also local configuration files in use or if options have been overridden on | |
484 | the command line or at run time. The source of the values of the options is | |
485 | displayed in the output. | |
7c673cae | 486 | |
05a536ef TL |
487 | * ``ceph config assimilate-conf -i <input file> -o <output file>`` ingests a |
488 | configuration file from *input file* and moves any valid options into the | |
489 | monitor configuration database. Any settings that are unrecognized, are | |
490 | invalid, or cannot be controlled by the monitor will be returned in an | |
491 | abbreviated configuration file stored in *output file*. This command is | |
492 | useful for transitioning from legacy configuration files to centralized | |
493 | monitor-based configuration. | |
7c673cae | 494 | |
1e59de90 | 495 | Note that ``ceph config set <who> <option> <value>`` and ``ceph config get |
05a536ef TL |
496 | <who> <option>`` will not necessarily return the same values. The latter |
497 | command will show compiled-in default values. In order to determine whether a | |
498 | configuration option is present in the monitor configuration database, run | |
499 | ``ceph config dump``. | |
7c673cae | 500 | |
11fdf7f2 TL |
501 | Help |
502 | ==== | |
7c673cae | 503 | |
05a536ef | 504 | To get help for a particular option, run the following command: |
11fdf7f2 | 505 | |
39ae355f | 506 | .. prompt:: bash $ |
11fdf7f2 | 507 | |
39ae355f | 508 | ceph config help <option> |
11fdf7f2 | 509 | |
39ae355f TL |
510 | For example: |
511 | ||
512 | .. prompt:: bash $ | |
513 | ||
514 | ceph config help log_file | |
515 | ||
05a536ef | 516 | :: |
11fdf7f2 | 517 | |
05a536ef | 518 | log_file - path to log file |
11fdf7f2 TL |
519 | (std::string, basic) |
520 | Default (non-daemon): | |
521 | Default (daemon): /var/log/ceph/$cluster-$name.log | |
522 | Can update at runtime: false | |
523 | See also: [log_to_stderr,err_to_stderr,log_to_syslog,err_to_syslog] | |
524 | ||
39ae355f TL |
525 | or: |
526 | ||
527 | .. prompt:: bash $ | |
528 | ||
529 | ceph config help log_file -f json-pretty | |
530 | ||
531 | :: | |
11fdf7f2 | 532 | |
11fdf7f2 TL |
533 | { |
534 | "name": "log_file", | |
535 | "type": "std::string", | |
536 | "level": "basic", | |
537 | "desc": "path to log file", | |
538 | "long_desc": "", | |
539 | "default": "", | |
540 | "daemon_default": "/var/log/ceph/$cluster-$name.log", | |
541 | "tags": [], | |
542 | "services": [], | |
543 | "see_also": [ | |
544 | "log_to_stderr", | |
545 | "err_to_stderr", | |
546 | "log_to_syslog", | |
547 | "err_to_syslog" | |
548 | ], | |
549 | "enum_values": [], | |
550 | "min": "", | |
551 | "max": "", | |
552 | "can_update_at_runtime": false | |
553 | } | |
554 | ||
05a536ef TL |
555 | The ``level`` property can be ``basic``, ``advanced``, or ``dev``. The `dev` |
556 | options are intended for use by developers, generally for testing purposes, and | |
557 | are not recommended for use by operators. | |
558 | ||
559 | .. note:: This command uses the configuration schema that is compiled into the | |
560 | running monitors. If you have a mixed-version cluster (as might exist, for | |
561 | example, during an upgrade), you might want to query the option schema from | |
562 | a specific running daemon by running a command of the following form: | |
563 | ||
564 | .. prompt:: bash $ | |
7c673cae | 565 | |
05a536ef | 566 | ceph daemon <name> config help [option] |
7c673cae FG |
567 | |
568 | Runtime Changes | |
569 | =============== | |
570 | ||
39ae355f | 571 | In most cases, Ceph permits changes to the configuration of a daemon at |
05a536ef | 572 | run time. This can be used for increasing or decreasing the amount of logging |
39ae355f | 573 | output, for enabling or disabling debug settings, and for runtime optimization. |
c07f9fc5 | 574 | |
05a536ef TL |
575 | Use the ``ceph config set`` command to update configuration options. For |
576 | example, to enable the most verbose debug log level on a specific OSD, run a | |
577 | command of the following form: | |
c07f9fc5 | 578 | |
39ae355f | 579 | .. prompt:: bash $ |
c07f9fc5 | 580 | |
39ae355f TL |
581 | ceph config set osd.123 debug_ms 20 |
582 | ||
583 | .. note:: If an option has been customized in a local configuration file, the | |
584 | `central config | |
585 | <https://ceph.io/en/news/blog/2018/new-mimic-centralized-configuration-management/>`_ | |
05a536ef TL |
586 | setting will be ignored because it has a lower priority than the local |
587 | configuration file. | |
588 | ||
589 | .. note:: Log levels range from 0 to 20. | |
c07f9fc5 | 590 | |
11fdf7f2 TL |
591 | Override values |
592 | --------------- | |
c07f9fc5 | 593 | |
05a536ef TL |
594 | Options can be set temporarily by using the Ceph CLI ``tell`` or ``daemon`` |
595 | interfaces on the Ceph CLI. These *override* values are ephemeral, which means | |
596 | that they affect only the current instance of the daemon and revert to | |
597 | persistently configured values when the daemon restarts. | |
c07f9fc5 | 598 | |
11fdf7f2 | 599 | Override values can be set in two ways: |
c07f9fc5 | 600 | |
39ae355f TL |
601 | #. From any host, send a message to a daemon with a command of the following |
602 | form: | |
05a536ef | 603 | |
39ae355f TL |
604 | .. prompt:: bash $ |
605 | ||
606 | ceph tell <name> config set <option> <value> | |
c07f9fc5 | 607 | |
39ae355f | 608 | For example: |
05a536ef | 609 | |
39ae355f | 610 | .. prompt:: bash $ |
c07f9fc5 | 611 | |
39ae355f | 612 | ceph tell osd.123 config set debug_osd 20 |
c07f9fc5 | 613 | |
39ae355f TL |
614 | The ``tell`` command can also accept a wildcard as the daemon identifier. |
615 | For example, to adjust the debug level on all OSD daemons, run a command of | |
05a536ef TL |
616 | the following form: |
617 | ||
39ae355f | 618 | .. prompt:: bash $ |
c07f9fc5 | 619 | |
39ae355f | 620 | ceph tell osd.* config set debug_osd 20 |
c07f9fc5 | 621 | |
39ae355f | 622 | #. On the host where the daemon is running, connect to the daemon via a socket |
05a536ef | 623 | in ``/var/run/ceph`` by running a command of the following form: |
c07f9fc5 | 624 | |
39ae355f | 625 | .. prompt:: bash $ |
c07f9fc5 | 626 | |
39ae355f | 627 | ceph daemon <name> config set <option> <value> |
c07f9fc5 | 628 | |
39ae355f | 629 | For example: |
05a536ef | 630 | |
39ae355f | 631 | .. prompt:: bash $ |
c07f9fc5 | 632 | |
39ae355f | 633 | ceph daemon osd.4 config set debug_osd 20 |
c07f9fc5 | 634 | |
39ae355f | 635 | .. note:: In the output of the ``ceph config show`` command, these temporary |
05a536ef | 636 | values are shown to have a source of ``override``. |
c07f9fc5 | 637 | |
c07f9fc5 | 638 | |
11fdf7f2 TL |
639 | Viewing runtime settings |
640 | ======================== | |
c07f9fc5 | 641 | |
05a536ef TL |
642 | You can see the current settings specified for a running daemon with the ``ceph |
643 | config show`` command. For example, to see the (non-default) settings for the | |
644 | daemon ``osd.0``, run the following command: | |
39ae355f TL |
645 | |
646 | .. prompt:: bash $ | |
647 | ||
648 | ceph config show osd.0 | |
649 | ||
05a536ef | 650 | To see a specific setting, run the following command: |
39ae355f TL |
651 | |
652 | .. prompt:: bash $ | |
653 | ||
654 | ceph config show osd.0 debug_osd | |
655 | ||
05a536ef TL |
656 | To see all settings (including those with default values), run the following |
657 | command: | |
c07f9fc5 | 658 | |
39ae355f | 659 | .. prompt:: bash $ |
c07f9fc5 | 660 | |
39ae355f | 661 | ceph config show-with-defaults osd.0 |
c07f9fc5 | 662 | |
05a536ef TL |
663 | You can see all settings for a daemon that is currently running by connecting |
664 | to it on the local host via the admin socket. For example, to dump all | |
665 | current settings, run the following command: | |
c07f9fc5 | 666 | |
39ae355f | 667 | .. prompt:: bash $ |
c07f9fc5 | 668 | |
39ae355f | 669 | ceph daemon osd.0 config show |
c07f9fc5 | 670 | |
05a536ef TL |
671 | To see non-default settings and to see where each value came from (for example, |
672 | a config file, the monitor, or an override), run the following command: | |
c07f9fc5 | 673 | |
39ae355f | 674 | .. prompt:: bash $ |
c07f9fc5 | 675 | |
39ae355f | 676 | ceph daemon osd.0 config diff |
c07f9fc5 | 677 | |
05a536ef | 678 | To see the value of a single setting, run the following command: |
c07f9fc5 | 679 | |
39ae355f | 680 | .. prompt:: bash $ |
c07f9fc5 | 681 | |
39ae355f | 682 | ceph daemon osd.0 config get debug_osd |
7c673cae | 683 | |
7c673cae | 684 | |
05a536ef TL |
685 | Changes introduced in Octopus |
686 | ============================= | |
9f95a23c | 687 | |
1e59de90 | 688 | The Octopus release changed the way the configuration file is parsed. |
f67539c2 | 689 | These changes are as follows: |
9f95a23c | 690 | |
05a536ef TL |
691 | - Repeated configuration options are allowed, and no warnings will be |
692 | displayed. This means that the setting that comes last in the file is the one | |
693 | that takes effect. Prior to this change, Ceph displayed warning messages when | |
694 | lines containing duplicate options were encountered, such as:: | |
9f95a23c TL |
695 | |
696 | warning line 42: 'foo' in section 'bar' redefined | |
05a536ef TL |
697 | - Prior to Octopus, options containing invalid UTF-8 characters were ignored |
698 | with warning messages. But in Octopus, they are treated as fatal errors. | |
699 | - The backslash character ``\`` is used as the line-continuation marker that | |
700 | combines the next line with the current one. Prior to Octopus, there was a | |
701 | requirement that any end-of-line backslash be followed by a non-empty line. | |
702 | But in Octopus, an empty line following a backslash is allowed. | |
9f95a23c | 703 | - In the configuration file, each line specifies an individual configuration |
f67539c2 | 704 | option. The option's name and its value are separated with ``=``, and the |
05a536ef | 705 | value may be enclosed within single or double quotes. If an invalid |
9f95a23c | 706 | configuration is specified, we will treat it as an invalid configuration |
05a536ef | 707 | file:: |
9f95a23c TL |
708 | |
709 | bad option ==== bad value | |
05a536ef TL |
710 | - Prior to Octopus, if no section name was specified in the configuration file, |
711 | all options would be set as though they were within the :confsec:`global` | |
712 | section. This approach is discouraged. Since Octopus, any configuration | |
713 | file that has no section name must contain only a single option. | |
f67539c2 | 714 | |
05a536ef | 715 | .. |---| unicode:: U+2014 .. EM DASH :trim: |