]>
Commit | Line | Data |
---|---|---|
7c673cae FG |
1 | ================================= |
2 | Configuration Management System | |
3 | ================================= | |
4 | ||
5 | The configuration management system exists to provide every daemon with the | |
6 | proper configuration information. The configuration can be viewed as a set of | |
7 | key-value pairs. | |
8 | ||
9 | How can the configuration be set? Well, there are several sources: | |
20effc67 | 10 | |
7c673cae FG |
11 | - the ceph configuration file, usually named ceph.conf |
12 | - command line arguments:: | |
20effc67 | 13 | |
7c673cae | 14 | --debug-ms=1 |
20effc67 TL |
15 | --debug-monc=10 |
16 | ||
17 | etc. | |
18 | - arguments injected at runtime using ``injectargs`` or ``config set`` | |
7c673cae FG |
19 | |
20 | ||
21 | The Configuration File | |
22 | ====================== | |
23 | ||
24 | Most configuration settings originate in the Ceph configuration file. | |
25 | ||
26 | How do we find the configuration file? Well, in order, we check: | |
20effc67 | 27 | |
7c673cae | 28 | - the default locations |
20effc67 TL |
29 | - the environment variable ``CEPH_CONF`` |
30 | - the command line argument ``-c`` | |
7c673cae FG |
31 | |
32 | Each stanza of the configuration file describes the key-value pairs that will be in | |
33 | effect for a particular subset of the daemons. The "global" stanza applies to | |
34 | everything. The "mon", "osd", and "mds" stanzas specify settings to take effect | |
35 | for all monitors, all OSDs, and all mds servers, respectively. A stanza of the | |
20effc67 | 36 | form ``mon.$name``, ``osd.$name``, or ``mds.$name`` gives settings for the monitor, OSD, or |
7c673cae FG |
37 | MDS of that name, respectively. Configuration values that appear later in the |
38 | file win over earlier ones. | |
39 | ||
40 | A sample configuration file can be found in src/sample.ceph.conf. | |
41 | ||
42 | ||
43 | Metavariables | |
44 | ============= | |
45 | ||
46 | The configuration system allows any configuration value to be | |
47 | substituted into another value using the ``$varname`` syntax, similar | |
48 | to how bash shell expansion works. | |
49 | ||
50 | A few additional special metavariables are also defined: | |
20effc67 | 51 | |
7c673cae FG |
52 | - $host: expands to the current hostname |
53 | - $type: expands to one of "mds", "osd", "mon", or "client" | |
54 | - $id: expands to the daemon identifier. For ``osd.0``, this would be ``0``; for ``mds.a``, it would be ``a``; for ``client.admin``, it would be ``admin``. | |
55 | - $num: same as $id | |
56 | - $name: expands to $type.$id | |
57 | ||
58 | ||
59 | Reading configuration values | |
60 | ==================================================== | |
61 | ||
62 | There are two ways for Ceph code to get configuration values. One way is to | |
20effc67 TL |
63 | read it directly from a variable named ``g_conf``, or equivalently, |
64 | ``g_ceph_ctx->_conf``. The other is to register an observer that will be called | |
65 | every time the relevant configuration values change. This observer will be | |
7c673cae FG |
66 | called soon after the initial configuration is read, and every time after that |
67 | when one of the relevant values changes. Each observer tracks a set of keys | |
68 | and is invoked only when one of the relevant keys changes. | |
69 | ||
20effc67 | 70 | The interface to implement is found in ``common/config_obs.h``. |
7c673cae FG |
71 | |
72 | The observer method should be preferred in new code because | |
20effc67 | 73 | |
7c673cae FG |
74 | - It is more flexible, allowing the code to do whatever reinitialization needs |
75 | to be done to implement the new configuration value. | |
76 | - It is the only way to create a std::string configuration variable that can | |
77 | be changed by injectargs. | |
78 | - Even for int-valued configuration options, changing the values in one thread | |
79 | while another thread is reading them can lead to subtle and | |
80 | impossible-to-diagnose bugs. | |
81 | ||
20effc67 TL |
82 | For these reasons, reading directly from ``g_conf`` should be considered deprecated |
83 | and not done in new code. Do not ever alter ``g_conf``. | |
7c673cae FG |
84 | |
85 | Changing configuration values | |
86 | ==================================================== | |
87 | ||
11fdf7f2 TL |
88 | Configuration values can be changed by calling ``g_conf()->set_val``. After changing |
89 | the configuration, you should call ``g_conf()->apply_changes`` to re-run all the | |
7c673cae | 90 | affected configuration observers. For convenience, you can call |
11fdf7f2 | 91 | ``g_conf()->set_val_or_die`` to make a configuration change which you think should |
7c673cae FG |
92 | never fail. |
93 | ||
20effc67 | 94 | ``injectargs``, ``parse_argv``, and ``parse_env`` are three other functions which modify |
7c673cae FG |
95 | the configuration. Just like with set_val, you should call apply_changes after |
96 | calling these functions to make sure your changes get applied. | |
c07f9fc5 FG |
97 | |
98 | ||
99 | Defining config options | |
100 | ======================= | |
101 | ||
20effc67 TL |
102 | Config options are defined in ``common/options/*.yaml.in``. The options are categorized |
103 | by their consumers. If an option is only used by ceph-osd, it should go to | |
104 | ``osd.yaml.in``. All the ``.yaml.in`` files are translated into ``.cc`` and ``.h`` files | |
105 | at build time by ``y2c.py``. | |
106 | ||
107 | Each option is represented using a YAML mapping (dictionary). A typical option looks like | |
108 | ||
109 | .. code-block:: yaml | |
110 | ||
111 | - name: public_addr | |
112 | type: addr | |
113 | level: basic | |
114 | desc: public-facing address to bind to | |
115 | long_desc: The IP address for the public (front-side) network. | |
116 | Set for each daemon. | |
117 | services: | |
118 | - mon | |
119 | - mds | |
120 | - osd | |
121 | - mgr | |
122 | flags: | |
123 | - startup | |
124 | with_legacy: true | |
125 | ||
126 | In which, following keys are allowed: | |
127 | ||
128 | level | |
129 | ----- | |
130 | ||
131 | The ``level`` property of an option is an indicator for the probability the | |
132 | option is adjusted by an operator or a developer: | |
133 | ||
134 | .. describe:: basic | |
135 | ||
136 | for basic config options that a normal operator is likely to adjust. | |
137 | ||
138 | .. describe:: advanced | |
139 | ||
140 | for options that an operator *can* adjust, but should not touch unless they | |
141 | understand what they are doing. Adjusting advanced options poorly can lead to | |
142 | problems (performance or even data loss) if done incorrectly. | |
143 | ||
144 | .. describe:: dev | |
145 | ||
146 | for options in place for use by developers only, either for testing purposes, | |
147 | or to describe constants that no user should adjust but we prefer not to compile | |
148 | into the code. | |
149 | ||
150 | ``desc``, ``long_desc`` and ``fmt_desc`` | |
151 | ---------------------------------------- | |
152 | ||
153 | .. describe:: desc | |
154 | ||
155 | Short description of the option. Sentence fragment. e.g. | |
156 | ||
157 | .. code-block:: yaml | |
c07f9fc5 | 158 | |
20effc67 | 159 | desc: Default checksum algorithm to use |
c07f9fc5 | 160 | |
20effc67 | 161 | .. describe:: long_desc |
c07f9fc5 | 162 | |
20effc67 TL |
163 | The long description is complete sentences, perhaps even multiple |
164 | paragraphs, and may include other detailed information or notes. e.g. | |
c07f9fc5 | 165 | |
20effc67 | 166 | .. code-block:: yaml |
c07f9fc5 | 167 | |
20effc67 TL |
168 | long_desc: crc32c, xxhash32, and xxhash64 are available. The _16 and _8 variants use |
169 | only a subset of the bits for more compact (but less reliable) checksumming. | |
c07f9fc5 | 170 | |
20effc67 | 171 | .. describe:: fmt_desc |
c07f9fc5 | 172 | |
20effc67 TL |
173 | The description formatted using reStructuredText. This property is |
174 | only used by the ``confval`` directive to render an option in the | |
175 | document. e.g.: | |
c07f9fc5 | 176 | |
20effc67 TL |
177 | .. code-block:: yaml |
178 | ||
179 | fmt_desc: The interval for "deep" scrubbing (fully reading all data). The | |
180 | ``osd_scrub_load_threshold`` does not affect this setting. | |
c07f9fc5 FG |
181 | |
182 | Default values | |
183 | -------------- | |
184 | ||
185 | There is a default value for every config option. In some cases, there may | |
186 | also be a *daemon default* that only applies to code that declares itself | |
20effc67 TL |
187 | as a daemon (in this case, the regular default only applies to non-daemons). Like: |
188 | ||
189 | .. code-block:: yaml | |
190 | ||
191 | default: crc32c | |
c07f9fc5 | 192 | |
20effc67 TL |
193 | Some literal postfixes are allowed when options with type of ``float``, ``size`` |
194 | and ``secs``, like: | |
c07f9fc5 | 195 | |
20effc67 | 196 | .. code-block:: yaml |
c07f9fc5 | 197 | |
20effc67 TL |
198 | - name: mon_scrub_interval |
199 | type: secs | |
200 | default: 1_day | |
201 | - name: osd_journal_size | |
202 | type: size | |
203 | default: 5_K | |
204 | ||
205 | For better readability, it is encouraged to use these literal postfixes when | |
206 | adding or updating the default value for an option. | |
c07f9fc5 FG |
207 | |
208 | Service | |
209 | ------- | |
210 | ||
211 | Service is a component name, like "common", "osd", "rgw", "mds", etc. It may | |
20effc67 TL |
212 | be a list of components, like: |
213 | ||
214 | .. code-block:: yaml | |
c07f9fc5 | 215 | |
20effc67 TL |
216 | services: |
217 | - mon | |
218 | - mds | |
219 | - osd | |
220 | - mgr | |
c07f9fc5 | 221 | |
20effc67 TL |
222 | For example, the rocksdb options affect both the osd and mon. If an option is put |
223 | into a service specific ``.yaml.in`` file, the corresponding service is added to | |
224 | its ``services`` property automatically. For instance, ``osd_scrub_begin_hour`` | |
225 | option is located in ``osd.yaml.in``, even its ``services`` is not specified | |
226 | explicitly in this file, this property still contains ``osd``. | |
c07f9fc5 FG |
227 | |
228 | Tags | |
229 | ---- | |
230 | ||
20effc67 | 231 | Tags identify options across services that relate in some way. For example: |
c07f9fc5 | 232 | |
20effc67 TL |
233 | network |
234 | options affecting network configuration | |
235 | mkfs | |
236 | options that only matter at mkfs time | |
237 | ||
238 | Like: | |
239 | ||
240 | .. code-block:: yaml | |
241 | ||
242 | tags: | |
243 | - network | |
c07f9fc5 FG |
244 | |
245 | Enums | |
246 | ----- | |
247 | ||
20effc67 TL |
248 | For options with a defined set of allowed values: |
249 | ||
250 | .. code-block:: yaml | |
c07f9fc5 | 251 | |
20effc67 TL |
252 | enum_values: |
253 | - none | |
254 | - crc32c | |
255 | - crc32c_16 | |
256 | - crc32c_8 | |
257 | - xxhash32 | |
258 | - xxhash64 | |
11fdf7f2 TL |
259 | |
260 | Flags | |
261 | ----- | |
262 | ||
20effc67 TL |
263 | .. describe:: runtime |
264 | ||
265 | the value can be updated at runtime | |
266 | ||
267 | .. describe:: no_mon_update | |
268 | ||
269 | Daemons/clients do not pull this value from the monitor config database. We | |
270 | disallow setting this option via ``ceph config set ...``. This option should | |
271 | be configured via ``ceph.conf`` or via the command line. | |
272 | ||
273 | .. describe:: startup | |
274 | ||
275 | option takes effect only during daemon startup | |
276 | ||
277 | .. describe:: cluster_create | |
278 | ||
279 | option only affects cluster creation | |
280 | ||
281 | .. describe:: create | |
282 | ||
283 | option only affects daemon creation |