]>
Commit | Line | Data |
---|---|---|
81eedcae TL |
1 | .. _configuring-ceph: |
2 | ||
7c673cae FG |
3 | ================== |
4 | Configuring Ceph | |
5 | ================== | |
6 | ||
f67539c2 | 7 | When Ceph services start, the initialization process activates a series |
7c673cae | 8 | of daemons that run in the background. A :term:`Ceph Storage Cluster` runs |
f67539c2 | 9 | at a minimum three types of daemons: |
7c673cae FG |
10 | |
11 | - :term:`Ceph Monitor` (``ceph-mon``) | |
11fdf7f2 | 12 | - :term:`Ceph Manager` (``ceph-mgr``) |
7c673cae FG |
13 | - :term:`Ceph OSD Daemon` (``ceph-osd``) |
14 | ||
f67539c2 | 15 | Ceph Storage Clusters that support the :term:`Ceph File System` also run at |
11fdf7f2 | 16 | least one :term:`Ceph Metadata Server` (``ceph-mds``). Clusters that |
f67539c2 TL |
17 | support :term:`Ceph Object Storage` run Ceph RADOS Gateway daemons |
18 | (``radosgw``) as well. | |
7c673cae | 19 | |
f67539c2 TL |
20 | Each daemon has a number of configuration options, each of which has a |
21 | default value. You may adjust the behavior of the system by changing these | |
22 | configuration options. Be careful to understand the consequences before | |
23 | overriding default values, as it is possible to significantly degrade the | |
24 | performance and stability of your cluster. Also note that default values | |
25 | sometimes change between releases, so it is best to review the version of | |
26 | this documentation that aligns with your Ceph release. | |
7c673cae | 27 | |
11fdf7f2 TL |
28 | Option names |
29 | ============ | |
7c673cae | 30 | |
11fdf7f2 TL |
31 | All Ceph configuration options have a unique name consisting of words |
32 | formed with lower-case characters and connected with underscore | |
33 | (``_``) characters. | |
7c673cae | 34 | |
11fdf7f2 TL |
35 | When option names are specified on the command line, either underscore |
36 | (``_``) or dash (``-``) characters can be used interchangeable (e.g., | |
37 | ``--mon-host`` is equivalent to ``--mon_host``). | |
7c673cae | 38 | |
11fdf7f2 | 39 | When option names appear in configuration files, spaces can also be |
f67539c2 TL |
40 | used in place of underscore or dash. We suggest, though, that for |
41 | clarity and convenience you consistently use underscores, as we do | |
42 | throughout this documentation. | |
7c673cae | 43 | |
11fdf7f2 TL |
44 | Config sources |
45 | ============== | |
7c673cae | 46 | |
11fdf7f2 TL |
47 | Each Ceph daemon, process, and library will pull its configuration |
48 | from several sources, listed below. Sources later in the list will | |
49 | override those earlier in the list when both are present. | |
50 | ||
51 | - the compiled-in default value | |
52 | - the monitor cluster's centralized configuration database | |
53 | - a configuration file stored on the local host | |
54 | - environment variables | |
55 | - command line arguments | |
56 | - runtime overrides set by an administrator | |
57 | ||
58 | One of the first things a Ceph process does on startup is parse the | |
59 | configuration options provided via the command line, environment, and | |
60 | local configuration file. The process will then contact the monitor | |
61 | cluster to retrieve configuration stored centrally for the entire | |
62 | cluster. Once a complete view of the configuration is available, the | |
63 | daemon or process startup will proceed. | |
64 | ||
9f95a23c TL |
65 | .. _bootstrap-options: |
66 | ||
11fdf7f2 TL |
67 | Bootstrap options |
68 | ----------------- | |
69 | ||
20effc67 TL |
70 | Some configuration options affect the process's ability to contact the |
71 | monitors, to authenticate, and to retrieve the cluster-stored configuration. | |
72 | For this reason, these options might need to be stored locally on the node, and | |
73 | set by means of a local configuration file. These options include the | |
74 | following: | |
75 | ||
76 | .. confval:: mon_host | |
77 | .. confval:: mon_host_override | |
78 | ||
79 | - :confval:`mon_dns_srv_name` | |
80 | - ``mon_data``, ``osd_data``, ``mds_data``, ``mgr_data``, and | |
81 | similar options that define which local directory the daemon | |
82 | stores its data in. | |
83 | - :confval:`keyring`, :confval:`keyfile`, and/or :confval:`key`, which can be used to | |
84 | specify the authentication credential to use to authenticate with | |
85 | the monitor. Note that in most cases the default keyring location | |
86 | is in the data directory specified above. | |
87 | ||
88 | In most cases, the default values of these options are suitable. There is one | |
89 | exception to this: the :confval:`mon_host` option that identifies the addresses | |
90 | of the cluster's monitors. When DNS is used to identify monitors, a local Ceph | |
91 | configuration file can be avoided entirely. | |
11fdf7f2 TL |
92 | |
93 | Skipping monitor config | |
94 | ----------------------- | |
95 | ||
20effc67 TL |
96 | Pass the option ``--no-mon-config`` to any process to skip the step that |
97 | retrieves configuration information from the cluster monitors. This is useful | |
98 | in cases where configuration is managed entirely via configuration files, or | |
99 | when the monitor cluster is down and some maintenance activity needs to be | |
100 | done. | |
7c673cae FG |
101 | |
102 | ||
11fdf7f2 | 103 | .. _ceph-conf-file: |
7c673cae | 104 | |
7c673cae | 105 | |
11fdf7f2 TL |
106 | Configuration sections |
107 | ====================== | |
7c673cae | 108 | |
11fdf7f2 TL |
109 | Any given process or daemon has a single value for each configuration |
110 | option. However, values for an option may vary across different | |
111 | daemon types even daemons of the same type. Ceph options that are | |
112 | stored in the monitor configuration database or in local configuration | |
113 | files are grouped into sections to indicate which daemons or clients | |
114 | they apply to. | |
7c673cae | 115 | |
11fdf7f2 | 116 | These sections include: |
7c673cae | 117 | |
20effc67 | 118 | .. confsec:: global |
7c673cae | 119 | |
20effc67 TL |
120 | Settings under ``global`` affect all daemons and clients |
121 | in a Ceph Storage Cluster. | |
7c673cae | 122 | |
20effc67 | 123 | :example: ``log_file = /var/log/ceph/$cluster-$type.$id.log`` |
7c673cae | 124 | |
20effc67 | 125 | .. confsec:: mon |
7c673cae | 126 | |
20effc67 TL |
127 | Settings under ``mon`` affect all ``ceph-mon`` daemons in |
128 | the Ceph Storage Cluster, and override the same setting in | |
129 | ``global``. | |
7c673cae | 130 | |
20effc67 | 131 | :example: ``mon_cluster_log_to_syslog = true`` |
7c673cae | 132 | |
20effc67 | 133 | .. confsec:: mgr |
7c673cae | 134 | |
20effc67 TL |
135 | Settings in the ``mgr`` section affect all ``ceph-mgr`` daemons in |
136 | the Ceph Storage Cluster, and override the same setting in | |
137 | ``global``. | |
7c673cae | 138 | |
20effc67 | 139 | :example: ``mgr_stats_period = 10`` |
7c673cae | 140 | |
20effc67 | 141 | .. confsec:: osd |
7c673cae | 142 | |
20effc67 TL |
143 | Settings under ``osd`` affect all ``ceph-osd`` daemons in |
144 | the Ceph Storage Cluster, and override the same setting in | |
145 | ``global``. | |
7c673cae | 146 | |
20effc67 | 147 | :example: ``osd_op_queue = wpq`` |
7c673cae | 148 | |
20effc67 | 149 | .. confsec:: mds |
7c673cae | 150 | |
20effc67 TL |
151 | Settings in the ``mds`` section affect all ``ceph-mds`` daemons in |
152 | the Ceph Storage Cluster, and override the same setting in | |
153 | ``global``. | |
7c673cae | 154 | |
20effc67 | 155 | :example: ``mds_cache_memory_limit = 10G`` |
7c673cae | 156 | |
20effc67 | 157 | .. confsec:: client |
7c673cae | 158 | |
20effc67 TL |
159 | Settings under ``client`` affect all Ceph Clients |
160 | (e.g., mounted Ceph File Systems, mounted Ceph Block Devices, | |
161 | etc.) as well as Rados Gateway (RGW) daemons. | |
7c673cae | 162 | |
20effc67 | 163 | :example: ``objecter_inflight_ops = 512`` |
7c673cae | 164 | |
7c673cae | 165 | |
11fdf7f2 TL |
166 | Sections may also specify an individual daemon or client name. For example, |
167 | ``mon.foo``, ``osd.123``, and ``client.smith`` are all valid section names. | |
7c673cae | 168 | |
7c673cae | 169 | |
11fdf7f2 TL |
170 | Any given daemon will draw its settings from the global section, the |
171 | daemon or client type section, and the section sharing its name. | |
172 | Settings in the most-specific section take precedence, so for example | |
20effc67 | 173 | if the same option is specified in both :confsec:`global`, :confsec:`mon`, and |
11fdf7f2 TL |
174 | ``mon.foo`` on the same source (i.e., in the same configurationfile), |
175 | the ``mon.foo`` value will be used. | |
7c673cae | 176 | |
9f95a23c TL |
177 | If multiple values of the same configuration option are specified in the same |
178 | section, the last value wins. | |
179 | ||
11fdf7f2 TL |
180 | Note that values from the local configuration file always take |
181 | precedence over values from the monitor configuration database, | |
182 | regardless of which section they appear in. | |
7c673cae FG |
183 | |
184 | ||
185 | .. _ceph-metavariables: | |
186 | ||
187 | Metavariables | |
188 | ============= | |
189 | ||
11fdf7f2 TL |
190 | Metavariables simplify Ceph Storage Cluster configuration |
191 | dramatically. When a metavariable is set in a configuration value, | |
192 | Ceph expands the metavariable into a concrete value at the time the | |
193 | configuration value is used. Ceph metavariables are similar to variable expansion in the Bash shell. | |
7c673cae FG |
194 | |
195 | Ceph supports the following metavariables: | |
196 | ||
20effc67 | 197 | .. describe:: $cluster |
7c673cae | 198 | |
20effc67 TL |
199 | Expands to the Ceph Storage Cluster name. Useful when running |
200 | multiple Ceph Storage Clusters on the same hardware. | |
7c673cae | 201 | |
20effc67 TL |
202 | :example: ``/etc/ceph/$cluster.keyring`` |
203 | :default: ``ceph`` | |
7c673cae | 204 | |
20effc67 | 205 | .. describe:: $type |
7c673cae | 206 | |
20effc67 | 207 | Expands to a daemon or process type (e.g., ``mds``, ``osd``, or ``mon``) |
7c673cae | 208 | |
20effc67 | 209 | :example: ``/var/lib/ceph/$type`` |
7c673cae | 210 | |
20effc67 | 211 | .. describe:: $id |
7c673cae | 212 | |
20effc67 TL |
213 | Expands to the daemon or client identifier. For |
214 | ``osd.0``, this would be ``0``; for ``mds.a``, it would | |
215 | be ``a``. | |
7c673cae | 216 | |
20effc67 | 217 | :example: ``/var/lib/ceph/$type/$cluster-$id`` |
7c673cae | 218 | |
20effc67 | 219 | .. describe:: $host |
7c673cae | 220 | |
20effc67 | 221 | Expands to the host name where the process is running. |
7c673cae | 222 | |
20effc67 | 223 | .. describe:: $name |
7c673cae | 224 | |
20effc67 | 225 | Expands to ``$type.$id``. |
7c673cae | 226 | |
20effc67 | 227 | :example: ``/var/run/ceph/$cluster-$name.asok`` |
7c673cae | 228 | |
20effc67 | 229 | .. describe:: $pid |
7c673cae | 230 | |
20effc67 | 231 | Expands to daemon pid. |
7c673cae | 232 | |
20effc67 | 233 | :example: ``/var/run/ceph/$cluster-$name-$pid.asok`` |
7c673cae FG |
234 | |
235 | ||
7c673cae | 236 | |
11fdf7f2 TL |
237 | The Configuration File |
238 | ====================== | |
7c673cae | 239 | |
11fdf7f2 TL |
240 | On startup, Ceph processes search for a configuration file in the |
241 | following locations: | |
7c673cae | 242 | |
11fdf7f2 TL |
243 | #. ``$CEPH_CONF`` (*i.e.,* the path following the ``$CEPH_CONF`` |
244 | environment variable) | |
245 | #. ``-c path/path`` (*i.e.,* the ``-c`` command line argument) | |
246 | #. ``/etc/ceph/$cluster.conf`` | |
247 | #. ``~/.ceph/$cluster.conf`` | |
248 | #. ``./$cluster.conf`` (*i.e.,* in the current working directory) | |
249 | #. On FreeBSD systems only, ``/usr/local/etc/ceph/$cluster.conf`` | |
7c673cae | 250 | |
11fdf7f2 | 251 | where ``$cluster`` is the cluster's name (default ``ceph``). |
7c673cae | 252 | |
f67539c2 TL |
253 | The Ceph configuration file uses an *ini* style syntax. You can add comment |
254 | text after a pound sign (#) or a semi-colon (;). For example: | |
7c673cae | 255 | |
11fdf7f2 | 256 | .. code-block:: ini |
7c673cae | 257 | |
11fdf7f2 TL |
258 | # <--A number (#) sign precedes a comment. |
259 | ; A comment may be anything. | |
260 | # Comments always follow a semi-colon (;) or a pound (#) on each line. | |
261 | # The end of the line terminates a comment. | |
262 | # We recommend that you provide comments in your configuration file(s). | |
7c673cae | 263 | |
7c673cae | 264 | |
11fdf7f2 | 265 | .. _ceph-conf-settings: |
7c673cae | 266 | |
11fdf7f2 TL |
267 | Config file section names |
268 | ------------------------- | |
7c673cae | 269 | |
11fdf7f2 TL |
270 | The configuration file is divided into sections. Each section must begin with a |
271 | valid configuration section name (see `Configuration sections`_, above) | |
272 | surrounded by square brackets. For example, | |
7c673cae | 273 | |
11fdf7f2 | 274 | .. code-block:: ini |
7c673cae | 275 | |
11fdf7f2 | 276 | [global] |
f67539c2 | 277 | debug_ms = 0 |
7c673cae | 278 | |
11fdf7f2 | 279 | [osd] |
f67539c2 | 280 | debug_ms = 1 |
7c673cae | 281 | |
11fdf7f2 | 282 | [osd.1] |
f67539c2 | 283 | debug_ms = 10 |
7c673cae | 284 | |
11fdf7f2 | 285 | [osd.2] |
f67539c2 | 286 | debug_ms = 10 |
7c673cae | 287 | |
7c673cae | 288 | |
9f95a23c TL |
289 | Config file option values |
290 | ------------------------- | |
291 | ||
292 | The value of a configuration option is a string. If it is too long to | |
293 | fit in a single line, you can put a backslash (``\``) at the end of line | |
294 | as the line continuation marker, so the value of the option will be | |
295 | the string after ``=`` in current line combined with the string in the next | |
296 | line:: | |
297 | ||
298 | [global] | |
299 | foo = long long ago\ | |
300 | long ago | |
301 | ||
302 | In the example above, the value of "``foo``" would be "``long long ago long ago``". | |
303 | ||
304 | Normally, the option value ends with a new line, or a comment, like | |
305 | ||
306 | .. code-block:: ini | |
307 | ||
308 | [global] | |
f67539c2 TL |
309 | obscure_one = difficult to explain # I will try harder in next release |
310 | simpler_one = nothing to explain | |
9f95a23c TL |
311 | |
312 | In the example above, the value of "``obscure one``" would be "``difficult to explain``"; | |
313 | and the value of "``simpler one`` would be "``nothing to explain``". | |
314 | ||
315 | If an option value contains spaces, and we want to make it explicit, we | |
316 | could quote the value using single or double quotes, like | |
317 | ||
318 | .. code-block:: ini | |
319 | ||
320 | [global] | |
321 | line = "to be, or not to be" | |
322 | ||
323 | Certain characters are not allowed to be present in the option values directly. | |
324 | They are ``=``, ``#``, ``;`` and ``[``. If we have to, we need to escape them, | |
325 | like | |
326 | ||
327 | .. code-block:: ini | |
328 | ||
329 | [global] | |
330 | secret = "i love \# and \[" | |
331 | ||
f67539c2 TL |
332 | Every configuration option is typed with one of the types below: |
333 | ||
20effc67 TL |
334 | .. describe:: int |
335 | ||
336 | 64-bit signed integer, Some SI prefixes are supported, like "K", "M", "G", | |
337 | "T", "P", "E", meaning, respectively, 10\ :sup:`3`, 10\ :sup:`6`, | |
338 | 10\ :sup:`9`, etc. And "B" is the only supported unit. So, "1K", "1M", "128B" and "-1" are all valid | |
339 | option values. Some times, a negative value implies "unlimited" when it comes to | |
340 | an option for threshold or limit. | |
341 | ||
342 | :example: ``42``, ``-1`` | |
343 | ||
344 | .. describe:: uint | |
f67539c2 | 345 | |
20effc67 | 346 | It is almost identical to ``integer``. But a negative value will be rejected. |
f67539c2 | 347 | |
20effc67 | 348 | :example: ``256``, ``0`` |
f67539c2 | 349 | |
20effc67 | 350 | .. describe:: str |
f67539c2 | 351 | |
20effc67 TL |
352 | Free style strings encoded in UTF-8, but some characters are not allowed. Please |
353 | reference the above notes for the details. | |
f67539c2 | 354 | |
20effc67 | 355 | :example: ``"hello world"``, ``"i love \#"``, ``yet-another-name`` |
f67539c2 | 356 | |
20effc67 | 357 | .. describe:: boolean |
f67539c2 | 358 | |
20effc67 TL |
359 | one of the two values ``true`` or ``false``. But an integer is also accepted, |
360 | where "0" implies ``false``, and any non-zero values imply ``true``. | |
f67539c2 | 361 | |
20effc67 | 362 | :example: ``true``, ``false``, ``1``, ``0`` |
f67539c2 | 363 | |
20effc67 | 364 | .. describe:: addr |
f67539c2 | 365 | |
20effc67 TL |
366 | a single address optionally prefixed with ``v1``, ``v2`` or ``any`` for the messenger |
367 | protocol. If the prefix is not specified, ``v2`` protocol is used. Please see | |
368 | :ref:`address_formats` for more details. | |
f67539c2 | 369 | |
20effc67 | 370 | :example: ``v1:1.2.3.4:567``, ``v2:1.2.3.4:567``, ``1.2.3.4:567``, ``2409:8a1e:8fb6:aa20:1260:4bff:fe92:18f5::567``, ``[::1]:6789`` |
f67539c2 | 371 | |
20effc67 | 372 | .. describe:: addrvec |
f67539c2 | 373 | |
20effc67 | 374 | a set of addresses separated by ",". The addresses can be optionally quoted with ``[`` and ``]``. |
f67539c2 | 375 | |
20effc67 | 376 | :example: ``[v1:1.2.3.4:567,v2:1.2.3.4:568]``, ``v1:1.2.3.4:567,v1:1.2.3.14:567`` ``[2409:8a1e:8fb6:aa20:1260:4bff:fe92:18f5::567], [2409:8a1e:8fb6:aa20:1260:4bff:fe92:18f5::568]`` |
f67539c2 | 377 | |
20effc67 | 378 | .. describe:: uuid |
f67539c2 | 379 | |
20effc67 TL |
380 | the string format of a uuid defined by `RFC4122 <https://www.ietf.org/rfc/rfc4122.txt>`_. |
381 | And some variants are also supported, for more details, see | |
382 | `Boost document <https://www.boost.org/doc/libs/1_74_0/libs/uuid/doc/uuid.html#String%20Generator>`_. | |
f67539c2 | 383 | |
20effc67 TL |
384 | :example: ``f81d4fae-7dec-11d0-a765-00a0c91e6bf6`` |
385 | ||
386 | .. describe:: size | |
387 | ||
388 | denotes a 64-bit unsigned integer. Both SI prefixes and IEC prefixes are | |
389 | supported. And "B" is the only supported unit. A negative value will be | |
390 | rejected. | |
391 | ||
392 | :example: ``1Ki``, ``1K``, ``1KiB`` and ``1B``. | |
393 | ||
394 | .. describe:: secs | |
395 | ||
396 | denotes a duration of time. By default the unit is second if not specified. | |
397 | Following units of time are supported: | |
f67539c2 TL |
398 | |
399 | * second: "s", "sec", "second", "seconds" | |
400 | * minute: "m", "min", "minute", "minutes" | |
401 | * hour: "hs", "hr", "hour", "hours" | |
402 | * day: "d", "day", "days" | |
403 | * week: "w", "wk", "week", "weeks" | |
404 | * month: "mo", "month", "months" | |
405 | * year: "y", "yr", "year", "years" | |
20effc67 TL |
406 | |
407 | :example: ``1 m``, ``1m`` and ``1 week`` | |
7c673cae | 408 | |
f91f0fd5 TL |
409 | .. _ceph-conf-database: |
410 | ||
11fdf7f2 TL |
411 | Monitor configuration database |
412 | ============================== | |
7c673cae | 413 | |
11fdf7f2 TL |
414 | The monitor cluster manages a database of configuration options that |
415 | can be consumed by the entire cluster, enabling streamlined central | |
416 | configuration management for the entire system. The vast majority of | |
417 | configuration options can and should be stored here for ease of | |
418 | administration and transparency. | |
7c673cae | 419 | |
11fdf7f2 TL |
420 | A handful of settings may still need to be stored in local |
421 | configuration files because they affect the ability to connect to the | |
422 | monitors, authenticate, and fetch configuration information. In most | |
423 | cases this is limited to the ``mon_host`` option, although this can | |
424 | also be avoided through the use of DNS SRV records. | |
7c673cae | 425 | |
11fdf7f2 TL |
426 | Sections and masks |
427 | ------------------ | |
7c673cae | 428 | |
11fdf7f2 TL |
429 | Configuration options stored by the monitor can live in a global |
430 | section, daemon type section, or specific daemon section, just like | |
431 | options in a configuration file can. | |
7c673cae | 432 | |
11fdf7f2 TL |
433 | In addition, options may also have a *mask* associated with them to |
434 | further restrict which daemons or clients the option applies to. | |
435 | Masks take two forms: | |
7c673cae | 436 | |
11fdf7f2 TL |
437 | #. ``type:location`` where *type* is a CRUSH property like `rack` or |
438 | `host`, and *location* is a value for that property. For example, | |
439 | ``host:foo`` would limit the option only to daemons or clients | |
440 | running on a particular host. | |
441 | #. ``class:device-class`` where *device-class* is the name of a CRUSH | |
442 | device class (e.g., ``hdd`` or ``ssd``). For example, | |
443 | ``class:ssd`` would limit the option only to OSDs backed by SSDs. | |
444 | (This mask has no effect for non-OSD daemons or clients.) | |
7c673cae | 445 | |
11fdf7f2 TL |
446 | When setting a configuration option, the `who` may be a section name, |
447 | a mask, or a combination of both separated by a slash (``/``) | |
448 | character. For example, ``osd/rack:foo`` would mean all OSD daemons | |
449 | in the ``foo`` rack. | |
7c673cae | 450 | |
11fdf7f2 TL |
451 | When viewing configuration options, the section name and mask are |
452 | generally separated out into separate fields or columns to ease readability. | |
7c673cae FG |
453 | |
454 | ||
11fdf7f2 TL |
455 | Commands |
456 | -------- | |
7c673cae | 457 | |
11fdf7f2 | 458 | The following CLI commands are used to configure the cluster: |
7c673cae | 459 | |
11fdf7f2 TL |
460 | * ``ceph config dump`` will dump the entire configuration database for |
461 | the cluster. | |
7c673cae | 462 | |
11fdf7f2 TL |
463 | * ``ceph config get <who>`` will dump the configuration for a specific |
464 | daemon or client (e.g., ``mds.a``), as stored in the monitors' | |
465 | configuration database. | |
7c673cae | 466 | |
11fdf7f2 TL |
467 | * ``ceph config set <who> <option> <value>`` will set a configuration |
468 | option in the monitors' configuration database. | |
7c673cae | 469 | |
11fdf7f2 TL |
470 | * ``ceph config show <who>`` will show the reported running |
471 | configuration for a running daemon. These settings may differ from | |
472 | those stored by the monitors if there are also local configuration | |
473 | files in use or options have been overridden on the command line or | |
474 | at run time. The source of the option values is reported as part | |
475 | of the output. | |
7c673cae | 476 | |
11fdf7f2 TL |
477 | * ``ceph config assimilate-conf -i <input file> -o <output file>`` |
478 | will ingest a configuration file from *input file* and move any | |
479 | valid options into the monitors' configuration database. Any | |
480 | settings that are unrecognized, invalid, or cannot be controlled by | |
481 | the monitor will be returned in an abbreviated config file stored in | |
482 | *output file*. This command is useful for transitioning from legacy | |
483 | configuration files to centralized monitor-based configuration. | |
7c673cae | 484 | |
7c673cae | 485 | |
11fdf7f2 TL |
486 | Help |
487 | ==== | |
7c673cae | 488 | |
39ae355f | 489 | You can get help for a particular option with: |
11fdf7f2 | 490 | |
39ae355f | 491 | .. prompt:: bash $ |
11fdf7f2 | 492 | |
39ae355f | 493 | ceph config help <option> |
11fdf7f2 | 494 | |
39ae355f | 495 | Note that this will use the configuration schema that is compiled into the running monitors. If you have a mixed-version cluster (e.g., during an upgrade), you might also want to query the option schema from a specific running daemon: |
11fdf7f2 | 496 | |
39ae355f TL |
497 | .. prompt:: bash $ |
498 | ||
499 | ceph daemon <name> config help [option] | |
500 | ||
501 | For example: | |
502 | ||
503 | .. prompt:: bash $ | |
504 | ||
505 | ceph config help log_file | |
506 | ||
507 | :: | |
11fdf7f2 | 508 | |
11fdf7f2 TL |
509 | log_file - path to log file |
510 | (std::string, basic) | |
511 | Default (non-daemon): | |
512 | Default (daemon): /var/log/ceph/$cluster-$name.log | |
513 | Can update at runtime: false | |
514 | See also: [log_to_stderr,err_to_stderr,log_to_syslog,err_to_syslog] | |
515 | ||
39ae355f TL |
516 | or: |
517 | ||
518 | .. prompt:: bash $ | |
519 | ||
520 | ceph config help log_file -f json-pretty | |
521 | ||
522 | :: | |
11fdf7f2 | 523 | |
11fdf7f2 TL |
524 | { |
525 | "name": "log_file", | |
526 | "type": "std::string", | |
527 | "level": "basic", | |
528 | "desc": "path to log file", | |
529 | "long_desc": "", | |
530 | "default": "", | |
531 | "daemon_default": "/var/log/ceph/$cluster-$name.log", | |
532 | "tags": [], | |
533 | "services": [], | |
534 | "see_also": [ | |
535 | "log_to_stderr", | |
536 | "err_to_stderr", | |
537 | "log_to_syslog", | |
538 | "err_to_syslog" | |
539 | ], | |
540 | "enum_values": [], | |
541 | "min": "", | |
542 | "max": "", | |
543 | "can_update_at_runtime": false | |
544 | } | |
545 | ||
546 | The ``level`` property can be any of `basic`, `advanced`, or `dev`. | |
547 | The `dev` options are intended for use by developers, generally for | |
548 | testing purposes, and are not recommended for use by operators. | |
7c673cae | 549 | |
7c673cae FG |
550 | |
551 | Runtime Changes | |
552 | =============== | |
553 | ||
39ae355f TL |
554 | In most cases, Ceph permits changes to the configuration of a daemon at |
555 | runtime. This can be used for increasing or decreasing the amount of logging | |
556 | output, for enabling or disabling debug settings, and for runtime optimization. | |
c07f9fc5 | 557 | |
39ae355f TL |
558 | Configuration options can be updated via the ``ceph config set`` command. For |
559 | example, to enable the debug log level on a specific OSD, run a command of this form: | |
c07f9fc5 | 560 | |
39ae355f | 561 | .. prompt:: bash $ |
c07f9fc5 | 562 | |
39ae355f TL |
563 | ceph config set osd.123 debug_ms 20 |
564 | ||
565 | .. note:: If an option has been customized in a local configuration file, the | |
566 | `central config | |
567 | <https://ceph.io/en/news/blog/2018/new-mimic-centralized-configuration-management/>`_ | |
568 | setting will be ignored (it has a lower priority than the local | |
569 | configuration file). | |
c07f9fc5 | 570 | |
11fdf7f2 TL |
571 | Override values |
572 | --------------- | |
c07f9fc5 | 573 | |
39ae355f TL |
574 | Options can be set temporarily by using the `tell` or `daemon` interfaces on |
575 | the Ceph CLI. These *override* values are ephemeral, which means that they | |
576 | affect only the current instance of the daemon and revert to persistently | |
577 | configured values when the daemon restarts. | |
c07f9fc5 | 578 | |
11fdf7f2 | 579 | Override values can be set in two ways: |
c07f9fc5 | 580 | |
39ae355f TL |
581 | #. From any host, send a message to a daemon with a command of the following |
582 | form: | |
583 | ||
584 | .. prompt:: bash $ | |
585 | ||
586 | ceph tell <name> config set <option> <value> | |
c07f9fc5 | 587 | |
39ae355f TL |
588 | For example: |
589 | ||
590 | .. prompt:: bash $ | |
c07f9fc5 | 591 | |
39ae355f | 592 | ceph tell osd.123 config set debug_osd 20 |
c07f9fc5 | 593 | |
39ae355f TL |
594 | The ``tell`` command can also accept a wildcard as the daemon identifier. |
595 | For example, to adjust the debug level on all OSD daemons, run a command of | |
596 | this form: | |
597 | ||
598 | .. prompt:: bash $ | |
c07f9fc5 | 599 | |
39ae355f | 600 | ceph tell osd.* config set debug_osd 20 |
c07f9fc5 | 601 | |
39ae355f TL |
602 | #. On the host where the daemon is running, connect to the daemon via a socket |
603 | in ``/var/run/ceph`` by running a command of this form: | |
c07f9fc5 | 604 | |
39ae355f | 605 | .. prompt:: bash $ |
c07f9fc5 | 606 | |
39ae355f | 607 | ceph daemon <name> config set <option> <value> |
c07f9fc5 | 608 | |
39ae355f TL |
609 | For example: |
610 | ||
611 | .. prompt:: bash $ | |
c07f9fc5 | 612 | |
39ae355f | 613 | ceph daemon osd.4 config set debug_osd 20 |
c07f9fc5 | 614 | |
39ae355f TL |
615 | .. note:: In the output of the ``ceph config show`` command, these temporary |
616 | values are shown with a source of ``override``. | |
c07f9fc5 | 617 | |
c07f9fc5 | 618 | |
11fdf7f2 TL |
619 | Viewing runtime settings |
620 | ======================== | |
c07f9fc5 | 621 | |
39ae355f TL |
622 | You can see the current options set for a running daemon with the ``ceph config show`` command. For example: |
623 | ||
624 | .. prompt:: bash $ | |
625 | ||
626 | ceph config show osd.0 | |
627 | ||
628 | will show you the (non-default) options for that daemon. You can also look at a specific option with: | |
629 | ||
630 | .. prompt:: bash $ | |
631 | ||
632 | ceph config show osd.0 debug_osd | |
633 | ||
634 | or view all options (even those with default values) with: | |
c07f9fc5 | 635 | |
39ae355f | 636 | .. prompt:: bash $ |
c07f9fc5 | 637 | |
39ae355f | 638 | ceph config show-with-defaults osd.0 |
c07f9fc5 | 639 | |
39ae355f | 640 | You can also observe settings for a running daemon by connecting to it from the local host via the admin socket. For example: |
c07f9fc5 | 641 | |
39ae355f | 642 | .. prompt:: bash $ |
c07f9fc5 | 643 | |
39ae355f | 644 | ceph daemon osd.0 config show |
c07f9fc5 | 645 | |
39ae355f | 646 | will dump all current settings: |
c07f9fc5 | 647 | |
39ae355f | 648 | .. prompt:: bash $ |
c07f9fc5 | 649 | |
39ae355f | 650 | ceph daemon osd.0 config diff |
c07f9fc5 | 651 | |
39ae355f | 652 | will show only non-default settings (as well as where the value came from: a config file, the monitor, an override, etc.), and: |
c07f9fc5 | 653 | |
39ae355f | 654 | .. prompt:: bash $ |
c07f9fc5 | 655 | |
39ae355f | 656 | ceph daemon osd.0 config get debug_osd |
7c673cae | 657 | |
11fdf7f2 | 658 | will report the value of a single option. |
7c673cae | 659 | |
7c673cae FG |
660 | |
661 | ||
f67539c2 | 662 | Changes since Nautilus |
9f95a23c TL |
663 | ====================== |
664 | ||
f67539c2 TL |
665 | With the Octopus release We changed the way the configuration file is parsed. |
666 | These changes are as follows: | |
9f95a23c | 667 | |
f67539c2 TL |
668 | - Repeated configuration options are allowed, and no warnings will be printed. |
669 | The value of the last one is used, which means that the setting last in the file | |
670 | is the one that takes effect. Before this change, we would print warning messages | |
671 | when lines with duplicated options were encountered, like:: | |
9f95a23c TL |
672 | |
673 | warning line 42: 'foo' in section 'bar' redefined | |
f67539c2 TL |
674 | |
675 | - Invalid UTF-8 options were ignored with warning messages. But since Octopus, | |
9f95a23c | 676 | they are treated as fatal errors. |
f67539c2 TL |
677 | |
678 | - Backslash ``\`` is used as the line continuation marker to combine the next | |
679 | line with current one. Before Octopus, it was required to follow a backslash with | |
680 | a non-empty line. But in Octopus, an empty line following a backslash is now allowed. | |
681 | ||
9f95a23c | 682 | - In the configuration file, each line specifies an individual configuration |
f67539c2 TL |
683 | option. The option's name and its value are separated with ``=``, and the |
684 | value may be quoted using single or double quotes. If an invalid | |
9f95a23c TL |
685 | configuration is specified, we will treat it as an invalid configuration |
686 | file :: | |
687 | ||
688 | bad option ==== bad value | |
f67539c2 | 689 | |
9f95a23c | 690 | - Before Octopus, if no section name was specified in the configuration file, |
20effc67 | 691 | all options would be set as though they were within the :confsec:`global` section. This is |
f67539c2 | 692 | now discouraged. Since Octopus, only a single option is allowed for |
9f95a23c | 693 | configuration files without a section name. |