]>
Commit | Line | Data |
---|---|---|
11fdf7f2 | 1 | .. _mgr-dashboard: |
31f18b77 | 2 | |
11fdf7f2 TL |
3 | Ceph Dashboard |
4 | ============== | |
5 | ||
6 | Overview | |
7 | -------- | |
8 | ||
9 | The Ceph Dashboard is a built-in web-based Ceph management and monitoring | |
10 | application to administer various aspects and objects of the cluster. It is | |
11 | implemented as a :ref:`ceph-manager-daemon` module. | |
12 | ||
13 | The original Ceph Dashboard that was shipped with Ceph Luminous started | |
14 | out as a simple read-only view into various run-time information and performance | |
15 | data of a Ceph cluster. It used a very simple architecture to achieve the | |
16 | original goal. However, there was a growing demand for adding more web-based | |
17 | management capabilities, to make it easier to administer Ceph for users that | |
18 | prefer a WebUI over using the command line. | |
19 | ||
20 | The new :term:`Ceph Dashboard` module is a replacement of the previous one and | |
21 | adds a built-in web based monitoring and administration application to the Ceph | |
9f95a23c | 22 | Manager. The architecture and functionality of this new module is derived from |
11fdf7f2 TL |
23 | and inspired by the `openATTIC Ceph management and monitoring tool |
24 | <https://openattic.org/>`_. The development is actively driven by the team | |
25 | behind openATTIC at `SUSE <https://www.suse.com/>`_, with a lot of support from | |
26 | companies like `Red Hat <https://redhat.com/>`_ and other members of the Ceph | |
27 | community. | |
28 | ||
29 | The dashboard module's backend code uses the CherryPy framework and a custom | |
30 | REST API implementation. The WebUI implementation is based on | |
31 | Angular/TypeScript, merging both functionality from the original dashboard as | |
32 | well as adding new functionality originally developed for the standalone version | |
33 | of openATTIC. The Ceph Dashboard module is implemented as a web | |
34 | application that visualizes information and statistics about the Ceph cluster | |
35 | using a web server hosted by ``ceph-mgr``. | |
36 | ||
37 | Feature Overview | |
38 | ^^^^^^^^^^^^^^^^ | |
39 | ||
40 | The dashboard provides the following features: | |
41 | ||
42 | * **Multi-User and Role Management**: The dashboard supports multiple user | |
43 | accounts with different permissions (roles). The user accounts and roles | |
9f95a23c TL |
44 | can be modified on both the command line and via the WebUI. The dashboard |
45 | supports various methods to enhance password security, e.g. by enforcing | |
46 | configurable password complexity rules, forcing users to change their password | |
47 | after the first login or after a configurable time period. See | |
48 | :ref:`dashboard-user-role-management` for details. | |
11fdf7f2 TL |
49 | * **Single Sign-On (SSO)**: the dashboard supports authentication |
50 | via an external identity provider using the SAML 2.0 protocol. See | |
51 | :ref:`dashboard-sso-support` for details. | |
52 | * **SSL/TLS support**: All HTTP communication between the web browser and the | |
53 | dashboard is secured via SSL. A self-signed certificate can be created with | |
54 | a built-in command, but it's also possible to import custom certificates | |
55 | signed and issued by a CA. See :ref:`dashboard-ssl-tls-support` for details. | |
56 | * **Auditing**: the dashboard backend can be configured to log all PUT, POST | |
57 | and DELETE API requests in the Ceph audit log. See :ref:`dashboard-auditing` | |
58 | for instructions on how to enable this feature. | |
494da23a TL |
59 | * **Internationalization (I18N)**: the dashboard can be used in different |
60 | languages that can be selected at run-time. | |
11fdf7f2 TL |
61 | |
62 | Currently, Ceph Dashboard is capable of monitoring and managing the following | |
63 | aspects of your Ceph cluster: | |
64 | ||
494da23a | 65 | * **Overall cluster health**: Display overall cluster status, performance |
11fdf7f2 TL |
66 | and capacity metrics. |
67 | * **Embedded Grafana Dashboards**: Ceph Dashboard is capable of embedding | |
e306af50 TL |
68 | `Grafana`_ dashboards in many locations, to display additional information |
69 | and performance metrics gathered by the :ref:`mgr-prometheus`. See | |
70 | :ref:`dashboard-grafana` for details on how to configure this functionality. | |
494da23a TL |
71 | * **Cluster logs**: Display the latest updates to the cluster's event and |
72 | audit log files. Log entries can be filtered by priority, date or keyword. | |
73 | * **Hosts**: Display a list of all hosts associated to the cluster, which | |
9f95a23c TL |
74 | disks are attached, which services are running and which version of Ceph is |
75 | installed. | |
494da23a | 76 | * **Performance counters**: Display detailed service-specific statistics for |
11fdf7f2 | 77 | each running service. |
494da23a | 78 | * **Monitors**: List all MONs, their quorum status, open sessions. |
9f95a23c TL |
79 | * **Monitoring**: Enable creation, re-creation, editing and expiration of |
80 | Prometheus' silences, list the alerting configuration of Prometheus and all | |
81 | configured and firing alerts. Show notifications for firing alerts. | |
494da23a | 82 | * **Configuration Editor**: Display all available configuration options, |
11fdf7f2 | 83 | their description, type and default values and edit the current values. |
494da23a | 84 | * **Pools**: List all Ceph pools and their details (e.g. applications, |
9f95a23c TL |
85 | pg-autoscaling, placement groups, replication size, EC profile, CRUSH |
86 | rulesets, quotas etc.) | |
494da23a TL |
87 | * **OSDs**: List all OSDs, their status and usage statistics as well as |
88 | detailed information like attributes (OSD map), metadata, performance | |
89 | counters and usage histograms for read/write operations. Mark OSDs | |
90 | up/down/out, purge and reweight OSDs, perform scrub operations, modify | |
91 | various scrub-related configuration options, select different profiles to | |
9f95a23c TL |
92 | adjust the level of backfilling activity. List all disks associated with an |
93 | OSD. Set and change the device class of an OSD, display and sort OSDs by | |
94 | device class. Deploy new OSDs on new disks/hosts. | |
95 | * **Device management**: List all hosts known by the orchestrator. List all | |
96 | disks and their properties attached to a node. Display disk health information | |
97 | (health prediction and SMART data). Blink enclosure LEDs. | |
494da23a | 98 | * **iSCSI**: List all hosts that run the TCMU runner service, display all |
11fdf7f2 | 99 | images and their performance characteristics (read/write ops, traffic). |
9f95a23c TL |
100 | Create, modify and delete iSCSI targets (via ``ceph-iscsi``). Display the |
101 | iSCSI gateway status on the landing page and info about active initiators. | |
102 | See :ref:`dashboard-iscsi-management` for instructions on how to configure | |
103 | this feature. | |
494da23a | 104 | * **RBD**: List all RBD images and their properties (size, objects, features). |
9f95a23c TL |
105 | Create, copy, modify and delete RBD images (incl. snapshots) and manage RBD |
106 | namespaces. Define various I/O or bandwidth limitation settings on a global, | |
107 | per-pool or per-image level. Create, delete and rollback snapshots of selected | |
108 | images, protect/unprotect these snapshots against modification. Copy or clone | |
109 | snapshots, flatten cloned images. | |
11fdf7f2 TL |
110 | * **RBD mirroring**: Enable and configure RBD mirroring to a remote Ceph server. |
111 | Lists all active sync daemons and their status, pools and RBD images including | |
112 | their synchronization state. | |
9f95a23c TL |
113 | * **CephFS**: List all active file system clients and associated pools, |
114 | including their usage statistics. Evict active CephFS clients. Manage CephFS | |
115 | quotas and snapshots. Browse a CephFS directory structure. | |
494da23a | 116 | * **Object Gateway**: List all active object gateways and their performance |
11fdf7f2 TL |
117 | counters. Display and manage (add/edit/delete) object gateway users and their |
118 | details (e.g. quotas) as well as the users' buckets and their details (e.g. | |
9f95a23c TL |
119 | placement targets, owner, quotas, versioning, multi-factor authentication). |
120 | See :ref:`dashboard-enabling-object-gateway` for configuration instructions. | |
121 | * **NFS**: Manage NFS exports of CephFS file systems and RGW S3 buckets via NFS | |
11fdf7f2 TL |
122 | Ganesha. See :ref:`dashboard-nfs-ganesha-management` for details on how to |
123 | enable this functionality. | |
124 | * **Ceph Manager Modules**: Enable and disable all Ceph Manager modules, change | |
125 | the module-specific configuration settings. | |
126 | ||
127 | ||
128 | Supported Browsers | |
129 | ^^^^^^^^^^^^^^^^^^ | |
130 | ||
131 | Ceph Dashboard is primarily tested and developed using the following web | |
132 | browsers: | |
133 | ||
9f95a23c TL |
134 | +-----------------------------------------------+----------+ |
135 | | Browser | Versions | | |
136 | +===============================================+==========+ | |
137 | | `Chrome <https://www.google.com/chrome/>`_ | 68+ | | |
138 | +-----------------------------------------------+----------+ | |
139 | | `Firefox <https://www.mozilla.org/firefox/>`_ | 61+ | | |
140 | +-----------------------------------------------+----------+ | |
11fdf7f2 TL |
141 | |
142 | While Ceph Dashboard might work in older browsers, we cannot guarantee it and | |
143 | recommend you to update your browser to the latest version. | |
224ce89b WB |
144 | |
145 | Enabling | |
146 | -------- | |
147 | ||
11fdf7f2 TL |
148 | If you have installed ``ceph-mgr-dashboard`` from distribution packages, the |
149 | package management system should have taken care of installing all the required | |
150 | dependencies. | |
151 | ||
152 | If you're installing Ceph from source and want to start the dashboard from your | |
153 | development environment, please see the files ``README.rst`` and ``HACKING.rst`` | |
154 | in directory ``src/pybind/mgr/dashboard`` of the source code. | |
224ce89b | 155 | |
11fdf7f2 TL |
156 | Within a running Ceph cluster, the Ceph Dashboard is enabled with:: |
157 | ||
158 | $ ceph mgr module enable dashboard | |
224ce89b WB |
159 | |
160 | Configuration | |
161 | ------------- | |
162 | ||
11fdf7f2 TL |
163 | .. _dashboard-ssl-tls-support: |
164 | ||
165 | SSL/TLS Support | |
166 | ^^^^^^^^^^^^^^^ | |
167 | ||
168 | All HTTP connections to the dashboard are secured with SSL/TLS by default. | |
169 | ||
170 | To get the dashboard up and running quickly, you can generate and install a | |
171 | self-signed certificate using the following built-in command:: | |
172 | ||
173 | $ ceph dashboard create-self-signed-cert | |
174 | ||
175 | Note that most web browsers will complain about such self-signed certificates | |
176 | and require explicit confirmation before establishing a secure connection to the | |
177 | dashboard. | |
178 | ||
179 | To properly secure a deployment and to remove the certificate warning, a | |
180 | certificate that is issued by a certificate authority (CA) should be used. | |
181 | ||
182 | For example, a key pair can be generated with a command similar to:: | |
183 | ||
184 | $ openssl req -new -nodes -x509 \ | |
185 | -subj "/O=IT/CN=ceph-mgr-dashboard" -days 3650 \ | |
186 | -keyout dashboard.key -out dashboard.crt -extensions v3_ca | |
187 | ||
188 | The ``dashboard.crt`` file should then be signed by a CA. Once that is done, you | |
189 | can enable it for all Ceph manager instances by running the following commands:: | |
190 | ||
494da23a TL |
191 | $ ceph dashboard set-ssl-certificate -i dashboard.crt |
192 | $ ceph dashboard set-ssl-certificate-key -i dashboard.key | |
11fdf7f2 TL |
193 | |
194 | If different certificates are desired for each manager instance for some reason, | |
195 | the name of the instance can be included as follows (where ``$name`` is the name | |
196 | of the ``ceph-mgr`` instance, usually the hostname):: | |
197 | ||
494da23a TL |
198 | $ ceph dashboard set-ssl-certificate $name -i dashboard.crt |
199 | $ ceph dashboard set-ssl-certificate-key $name -i dashboard.key | |
224ce89b | 200 | |
11fdf7f2 | 201 | SSL can also be disabled by setting this configuration value:: |
31f18b77 | 202 | |
11fdf7f2 | 203 | $ ceph config set mgr mgr/dashboard/ssl false |
31f18b77 | 204 | |
11fdf7f2 TL |
205 | This might be useful if the dashboard will be running behind a proxy which does |
206 | not support SSL for its upstream servers or other situations where SSL is not | |
9f95a23c | 207 | wanted or required. See :ref:`dashboard-proxy-configuration` for more details. |
224ce89b | 208 | |
11fdf7f2 | 209 | .. warning:: |
224ce89b | 210 | |
11fdf7f2 TL |
211 | Use caution when disabling SSL as usernames and passwords will be sent to the |
212 | dashboard unencrypted. | |
224ce89b | 213 | |
11fdf7f2 TL |
214 | |
215 | .. note:: | |
216 | ||
217 | You need to restart the Ceph manager processes manually after changing the SSL | |
218 | certificate and key. This can be accomplished by either running ``ceph mgr | |
219 | fail mgr`` or by disabling and re-enabling the dashboard module (which also | |
220 | triggers the manager to respawn itself):: | |
221 | ||
222 | $ ceph mgr module disable dashboard | |
223 | $ ceph mgr module enable dashboard | |
224 | ||
225 | Host Name and Port | |
226 | ^^^^^^^^^^^^^^^^^^ | |
227 | ||
228 | Like most web applications, dashboard binds to a TCP/IP address and TCP port. | |
229 | ||
230 | By default, the ``ceph-mgr`` daemon hosting the dashboard (i.e., the currently | |
231 | active manager) will bind to TCP port 8443 or 8080 when SSL is disabled. | |
232 | ||
233 | If no specific address has been configured, the web app will bind to ``::``, | |
224ce89b WB |
234 | which corresponds to all available IPv4 and IPv6 addresses. |
235 | ||
11fdf7f2 TL |
236 | These defaults can be changed via the configuration key facility on a |
237 | cluster-wide level (so they apply to all manager instances) as follows:: | |
238 | ||
239 | $ ceph config set mgr mgr/dashboard/server_addr $IP | |
240 | $ ceph config set mgr mgr/dashboard/server_port $PORT | |
241 | $ ceph config set mgr mgr/dashboard/ssl_server_port $PORT | |
242 | ||
243 | Since each ``ceph-mgr`` hosts its own instance of dashboard, it may also be | |
244 | necessary to configure them separately. The IP address and port for a specific | |
245 | manager instance can be changed with the following commands:: | |
246 | ||
247 | $ ceph config set mgr mgr/dashboard/$name/server_addr $IP | |
248 | $ ceph config set mgr mgr/dashboard/$name/server_port $PORT | |
249 | $ ceph config set mgr mgr/dashboard/$name/ssl_server_port $PORT | |
250 | ||
251 | Replace ``$name`` with the ID of the ceph-mgr instance hosting the dashboard web | |
252 | app. | |
253 | ||
254 | .. note:: | |
255 | ||
256 | The command ``ceph mgr services`` will show you all endpoints that are | |
257 | currently configured. Look for the ``dashboard`` key to obtain the URL for | |
258 | accessing the dashboard. | |
259 | ||
260 | Username and Password | |
261 | ^^^^^^^^^^^^^^^^^^^^^ | |
262 | ||
263 | In order to be able to log in, you need to create a user account and associate | |
264 | it with at least one role. We provide a set of predefined *system roles* that | |
265 | you can use. For more details please refer to the `User and Role Management`_ | |
266 | section. | |
267 | ||
268 | To create a user with the administrator role you can use the following | |
269 | commands:: | |
270 | ||
271 | $ ceph dashboard ac-user-create <username> <password> administrator | |
272 | ||
adb31ebb TL |
273 | Account Lock-out |
274 | ^^^^^^^^^^^^^^^^ | |
275 | ||
276 | It disables a user account if a user repeatedly enters the wrong credentials | |
277 | for multiple times. It is enabled by default to prevent brute-force or dictionary | |
278 | attacks. The user can get or set the default number of lock-out attempts using | |
279 | these commands respectively:: | |
280 | ||
281 | $ ceph dashboard get-account-lockout-attempts | |
282 | $ ceph dashboard set-account-lockout-attempts <value:int> | |
283 | ||
284 | .. warning:: | |
285 | ||
286 | This feature can be disabled by setting the default number of lock-out attempts to 0. | |
287 | However, by disabling this feature, the account is more vulnerable to brute-force or | |
288 | dictionary based attacks. This can be disabled by:: | |
289 | ||
290 | $ ceph dashboard set-account-lockout-attempts 0 | |
291 | ||
292 | Enable a Locked User | |
293 | ^^^^^^^^^^^^^^^^^^^^ | |
294 | ||
295 | If a user account is disabled as a result of multiple invalid login attempts, then | |
296 | it needs to be manually enabled by the administrator. This can be done by the following | |
297 | command:: | |
298 | ||
299 | $ ceph dashboard ac-user-enable <username> | |
300 | ||
9f95a23c TL |
301 | Accessing the Dashboard |
302 | ^^^^^^^^^^^^^^^^^^^^^^^ | |
303 | ||
304 | You can now access the dashboard using your (JavaScript-enabled) web browser, by | |
305 | pointing it to any of the host names or IP addresses and the selected TCP port | |
306 | where a manager instance is running: e.g., ``http(s)://<$IP>:<$PORT>/``. | |
307 | ||
308 | You should then be greeted by the dashboard login page, requesting your | |
309 | previously defined username and password. | |
310 | ||
11fdf7f2 TL |
311 | .. _dashboard-enabling-object-gateway: |
312 | ||
313 | Enabling the Object Gateway Management Frontend | |
314 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
315 | ||
316 | To use the Object Gateway management functionality of the dashboard, you will | |
317 | need to provide the login credentials of a user with the ``system`` flag | |
318 | enabled. | |
319 | ||
320 | If you do not have a user which shall be used for providing those credentials, | |
321 | you will also need to create one:: | |
322 | ||
323 | $ radosgw-admin user create --uid=<user_id> --display-name=<display_name> \ | |
324 | --system | |
325 | ||
326 | Take note of the keys ``access_key`` and ``secret_key`` in the output of this | |
327 | command. | |
328 | ||
329 | The credentials of an existing user can also be obtained by using | |
330 | `radosgw-admin`:: | |
331 | ||
332 | $ radosgw-admin user info --uid=<user_id> | |
333 | ||
334 | Finally, provide the credentials to the dashboard:: | |
335 | ||
336 | $ ceph dashboard set-rgw-api-access-key <access_key> | |
337 | $ ceph dashboard set-rgw-api-secret-key <secret_key> | |
338 | ||
81eedcae TL |
339 | In a typical default configuration with a single RGW endpoint, this is all you |
340 | have to do to get the Object Gateway management functionality working. The | |
341 | dashboard will try to automatically determine the host and port of the Object | |
342 | Gateway by obtaining this information from the Ceph Manager's service map. | |
11fdf7f2 TL |
343 | |
344 | If multiple zones are used, it will automatically determine the host within the | |
345 | master zone group and master zone. This should be sufficient for most setups, | |
346 | but in some circumstances you might want to set the host and port manually:: | |
347 | ||
348 | $ ceph dashboard set-rgw-api-host <host> | |
349 | $ ceph dashboard set-rgw-api-port <port> | |
350 | ||
351 | In addition to the settings mentioned so far, the following settings do also | |
352 | exist and you may find yourself in the situation that you have to use them:: | |
353 | ||
354 | $ ceph dashboard set-rgw-api-scheme <scheme> # http or https | |
355 | $ ceph dashboard set-rgw-api-admin-resource <admin_resource> | |
356 | $ ceph dashboard set-rgw-api-user-id <user_id> | |
357 | ||
358 | If you are using a self-signed certificate in your Object Gateway setup, then | |
359 | you should disable certificate verification in the dashboard to avoid refused | |
360 | connections, e.g. caused by certificates signed by unknown CA or not matching | |
361 | the host name:: | |
362 | ||
363 | $ ceph dashboard set-rgw-api-ssl-verify False | |
364 | ||
365 | If the Object Gateway takes too long to process requests and the dashboard runs | |
366 | into timeouts, then you can set the timeout value to your needs:: | |
367 | ||
368 | $ ceph dashboard set-rest-requests-timeout <seconds> | |
369 | ||
370 | The default value is 45 seconds. | |
371 | ||
372 | .. _dashboard-iscsi-management: | |
373 | ||
374 | Enabling iSCSI Management | |
375 | ^^^^^^^^^^^^^^^^^^^^^^^^^ | |
376 | ||
81eedcae TL |
377 | The Ceph Dashboard can manage iSCSI targets using the REST API provided by the |
378 | `rbd-target-api` service of the :ref:`ceph-iscsi`. Please make sure that it's | |
379 | installed and enabled on the iSCSI gateways. | |
380 | ||
381 | .. note:: | |
9f95a23c | 382 | |
81eedcae TL |
383 | The iSCSI management functionality of Ceph Dashboard depends on the latest |
384 | version 3 of the `ceph-iscsi <https://github.com/ceph/ceph-iscsi>`_ project. | |
385 | Make sure that your operating system provides the correct version, otherwise | |
386 | the dashboard won't enable the management features. | |
11fdf7f2 TL |
387 | |
388 | If ceph-iscsi REST API is configured in HTTPS mode and its using a self-signed | |
81eedcae | 389 | certificate, then you need to configure the dashboard to avoid SSL certificate |
11fdf7f2 TL |
390 | verification when accessing ceph-iscsi API. |
391 | ||
9f95a23c | 392 | To disable API SSL verification run the following command:: |
11fdf7f2 | 393 | |
9f95a23c | 394 | $ ceph dashboard set-iscsi-api-ssl-verification false |
11fdf7f2 TL |
395 | |
396 | The available iSCSI gateways must be defined using the following commands:: | |
397 | ||
9f95a23c TL |
398 | $ ceph dashboard iscsi-gateway-list |
399 | $ ceph dashboard iscsi-gateway-add <scheme>://<username>:<password>@<host>[:port] | |
400 | $ ceph dashboard iscsi-gateway-rm <gateway_name> | |
11fdf7f2 TL |
401 | |
402 | ||
403 | .. _dashboard-grafana: | |
404 | ||
405 | Enabling the Embedding of Grafana Dashboards | |
406 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
407 | ||
e306af50 TL |
408 | `Grafana`_ requires data from `Prometheus <https://prometheus.io/>`_. Although |
409 | Grafana can use other data sources, the Grafana dashboards we provide contain | |
410 | queries that are specific to Prometheus. Our Grafana dashboards therefore | |
411 | require Prometheus as the data source. The Ceph :ref:`mgr-prometheus` also only | |
412 | exports its data in the Prometheus' common format. The Grafana dashboards rely | |
413 | on metric names from the Prometheus module and `Node exporter | |
414 | <https://prometheus.io/docs/guides/node-exporter/>`_. The Node exporter is a | |
415 | separate application that provides machine metrics. | |
416 | ||
417 | .. note:: | |
418 | ||
419 | Prometheus' security model presumes that untrusted users have access to the | |
420 | Prometheus HTTP endpoint and logs. Untrusted users have access to all the | |
421 | (meta)data Prometheus collects that is contained in the database, plus a | |
422 | variety of operational and debugging information. | |
423 | ||
424 | However, Prometheus' HTTP API is limited to read-only operations. | |
425 | Configurations can *not* be changed using the API and secrets are not | |
426 | exposed. Moreover, Prometheus has some built-in measures to mitigate the | |
427 | impact of denial of service attacks. | |
428 | ||
429 | Please see `Prometheus' Security model | |
430 | <https://prometheus.io/docs/operating/security/>` for more detailed | |
431 | information. | |
432 | ||
11fdf7f2 TL |
433 | Grafana and Prometheus are likely going to be bundled and installed by some |
434 | orchestration tools along Ceph in the near future, but currently, you will have | |
435 | to install and configure both manually. After you have installed Prometheus and | |
436 | Grafana on your preferred hosts, proceed with the following steps. | |
437 | ||
9f95a23c | 438 | 1. Enable the Ceph Exporter which comes as Ceph Manager module by running:: |
11fdf7f2 TL |
439 | |
440 | $ ceph mgr module enable prometheus | |
441 | ||
494da23a | 442 | More details can be found in the documentation of the :ref:`mgr-prometheus`. |
11fdf7f2 | 443 | |
9f95a23c | 444 | 2. Add the corresponding scrape configuration to Prometheus. This may look |
11fdf7f2 TL |
445 | like:: |
446 | ||
9f95a23c TL |
447 | global: |
448 | scrape_interval: 5s | |
449 | ||
450 | scrape_configs: | |
451 | - job_name: 'prometheus' | |
452 | static_configs: | |
453 | - targets: ['localhost:9090'] | |
454 | - job_name: 'ceph' | |
455 | static_configs: | |
456 | - targets: ['localhost:9283'] | |
457 | - job_name: 'node-exporter' | |
458 | static_configs: | |
459 | - targets: ['localhost:9100'] | |
11fdf7f2 | 460 | |
9f95a23c | 461 | 3. Add Prometheus as data source to Grafana |
11fdf7f2 | 462 | |
9f95a23c | 463 | 4. Install the `vonage-status-panel and grafana-piechart-panel` plugins using:: |
11fdf7f2 | 464 | |
9f95a23c TL |
465 | grafana-cli plugins install vonage-status-panel |
466 | grafana-cli plugins install grafana-piechart-panel | |
11fdf7f2 | 467 | |
9f95a23c | 468 | 5. Add the Dashboards to Grafana: |
11fdf7f2 | 469 | |
9f95a23c TL |
470 | Dashboards can be added to Grafana by importing dashboard jsons. |
471 | Following command can be used for downloading json files:: | |
11fdf7f2 | 472 | |
9f95a23c | 473 | wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/<Dashboard-name>.json |
11fdf7f2 | 474 | |
9f95a23c TL |
475 | You can find all the dashboard jsons `here <https://github.com/ceph/ceph/tree/ |
476 | master/monitoring/grafana/dashboards>`_ . | |
11fdf7f2 | 477 | |
9f95a23c | 478 | For Example, for ceph-cluster overview you can use:: |
11fdf7f2 | 479 | |
9f95a23c | 480 | wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/ceph-cluster.json |
11fdf7f2 | 481 | |
9f95a23c | 482 | 6. Configure Grafana in `/etc/grafana/grafana.ini` to adapt anonymous mode:: |
11fdf7f2 | 483 | |
9f95a23c TL |
484 | [auth.anonymous] |
485 | enabled = true | |
486 | org_name = Main Org. | |
487 | org_role = Viewer | |
11fdf7f2 | 488 | |
9f95a23c TL |
489 | In newer versions of Grafana (starting with 6.2.0-beta1) a new setting named |
490 | ``allow_embedding`` has been introduced. This setting needs to be explicitly | |
491 | set to ``true`` for the Grafana integration in Ceph Dashboard to work, as its | |
492 | default is ``false``. | |
493 | ||
494 | :: | |
495 | ||
496 | [security] | |
497 | allow_embedding = true | |
11fdf7f2 | 498 | |
f91f0fd5 TL |
499 | Enabling RBD-Image monitoring |
500 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
501 | ||
502 | Due to performance reasons, monitoring of RBD images is disabled by default. For | |
503 | more information please see :ref:`prometheus-rbd-io-statistics`. If disabled, | |
504 | the overview and details dashboards will stay empty in Grafana and the metrics | |
505 | will not be visible in Prometheus. | |
506 | ||
11fdf7f2 TL |
507 | After you have set up Grafana and Prometheus, you will need to configure the |
508 | connection information that the Ceph Dashboard will use to access Grafana. | |
509 | ||
adb31ebb TL |
510 | You need to tell the dashboard on which URL the Grafana instance is |
511 | running/deployed:: | |
11fdf7f2 TL |
512 | |
513 | $ ceph dashboard set-grafana-api-url <grafana-server-url> # default: '' | |
514 | ||
515 | The format of url is : `<protocol>:<IP-address>:<port>` | |
516 | ||
517 | .. note:: | |
9f95a23c | 518 | |
11fdf7f2 TL |
519 | Ceph Dashboard embeds the Grafana dashboards via ``iframe`` HTML elements. |
520 | If Grafana is configured without SSL/TLS support, most browsers will block the | |
521 | embedding of insecure content into a secured web page, if the SSL support in | |
522 | the dashboard has been enabled (which is the default configuration). If you | |
523 | can't see the embedded Grafana dashboards after enabling them as outlined | |
524 | above, check your browser's documentation on how to unblock mixed content. | |
525 | Alternatively, consider enabling SSL/TLS support in Grafana. | |
494da23a | 526 | |
92f5a8d4 TL |
527 | If you are using a self-signed certificate in your Grafana setup, then you should |
528 | disable certificate verification in the dashboard to avoid refused connections, | |
529 | e.g. caused by certificates signed by unknown CA or not matching the host name:: | |
530 | ||
531 | $ ceph dashboard set-grafana-api-ssl-verify False | |
532 | ||
11fdf7f2 TL |
533 | You can directly access Grafana Instance as well to monitor your cluster. |
534 | ||
adb31ebb TL |
535 | Alternative URL for Browsers |
536 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
537 | ||
538 | The Ceph Dashboard backend requires the Grafana URL to be able to verify the | |
539 | existence of Grafana Dashboards before the frontend even loads them. Due to the | |
540 | nature of how Grafana is implemented in Ceph Dashboard, this means that two | |
541 | working connections are required in order to be able to see Grafana graphs in | |
542 | Ceph Dashboard: | |
543 | ||
544 | - The backend (Ceph Mgr module) needs to verify the existence of the requested | |
545 | graph. If this request succeeds, it lets the frontend know that it can safely | |
546 | access Grafana. | |
547 | - The frontend then requests the Grafana graphs directly from the user's | |
548 | browser using an iframe. The Grafana instance is accessed directly without any | |
549 | detour through Ceph Dashboard. | |
550 | ||
551 | Now, it might be the case that your environment makes it difficult for the | |
552 | user's browser to directly access the URL configured in Ceph Dashboard. To solve | |
553 | this issue, a separate URL can be configured which will solely be used to tell | |
554 | the frontend (the user's browser) which URL it should use to access Grafana. | |
555 | This setting won't ever be changed automatically, unlike the GRAFANA_API_URL | |
556 | which is set by :ref:`cephadm` (only if cephadm is used to deploy monitoring | |
557 | services). | |
558 | ||
559 | To change the URL that is returned to the frontend issue the following command:: | |
560 | ||
561 | $ ceph dashboard set-grafana-frontend-api-url <grafana-server-url> | |
562 | ||
563 | If no value is set for that option, it will simply fall back to the value of the | |
564 | GRAFANA_API_URL option. If set, it will instruct the browser to use this URL to | |
565 | access Grafana. | |
566 | ||
11fdf7f2 TL |
567 | .. _dashboard-sso-support: |
568 | ||
569 | Enabling Single Sign-On (SSO) | |
570 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
571 | ||
572 | The Ceph Dashboard supports external authentication of users via the | |
573 | `SAML 2.0 <https://en.wikipedia.org/wiki/SAML_2.0>`_ protocol. You need to create | |
574 | the user accounts and associate them with the desired roles first, as authorization | |
575 | is still performed by the Dashboard. However, the authentication process can be | |
576 | performed by an existing Identity Provider (IdP). | |
577 | ||
578 | .. note:: | |
9f95a23c | 579 | |
11fdf7f2 TL |
580 | Ceph Dashboard SSO support relies on onelogin's |
581 | `python-saml <https://pypi.org/project/python-saml/>`_ library. | |
582 | Please ensure that this library is installed on your system, either by using | |
583 | your distribution's package management or via Python's `pip` installer. | |
584 | ||
585 | To configure SSO on Ceph Dashboard, you should use the following command:: | |
586 | ||
587 | $ ceph dashboard sso setup saml2 <ceph_dashboard_base_url> <idp_metadata> {<idp_username_attribute>} {<idp_entity_id>} {<sp_x_509_cert>} {<sp_private_key>} | |
588 | ||
589 | Parameters: | |
590 | ||
591 | * **<ceph_dashboard_base_url>**: Base URL where Ceph Dashboard is accessible (e.g., `https://cephdashboard.local`) | |
9f95a23c | 592 | * **<idp_metadata>**: URL to remote (`http://`, `https://`) or local (`file://`) path or content of the IdP metadata XML (e.g., `https://myidp/metadata`, `file:///home/myuser/metadata.xml`). |
11fdf7f2 TL |
593 | * **<idp_username_attribute>** *(optional)*: Attribute that should be used to get the username from the authentication response. Defaults to `uid`. |
594 | * **<idp_entity_id>** *(optional)*: Use this when more than one entity id exists on the IdP metadata. | |
9f95a23c | 595 | * **<sp_x_509_cert> / <sp_private_key>** *(optional)*: File path of the certificate that should be used by Ceph Dashboard (Service Provider) for signing and encryption. |
11fdf7f2 TL |
596 | |
597 | .. note:: | |
9f95a23c | 598 | |
11fdf7f2 TL |
599 | The issuer value of SAML requests will follow this pattern: **<ceph_dashboard_base_url>**/auth/saml2/metadata |
600 | ||
601 | To display the current SAML 2.0 configuration, use the following command:: | |
602 | ||
603 | $ ceph dashboard sso show saml2 | |
604 | ||
605 | .. note:: | |
9f95a23c | 606 | |
11fdf7f2 TL |
607 | For more information about `onelogin_settings`, please check the `onelogin documentation <https://github.com/onelogin/python-saml>`_. |
608 | ||
609 | To disable SSO:: | |
610 | ||
611 | $ ceph dashboard sso disable | |
612 | ||
613 | To check if SSO is enabled:: | |
614 | ||
615 | $ ceph dashboard sso status | |
616 | ||
617 | To enable SSO:: | |
618 | ||
619 | $ ceph dashboard sso enable saml2 | |
620 | ||
9f95a23c TL |
621 | .. _dashboard-alerting: |
622 | ||
11fdf7f2 TL |
623 | Enabling Prometheus Alerting |
624 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
625 | ||
626 | Using Prometheus for monitoring, you have to define `alerting rules | |
627 | <https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules>`_. | |
628 | To manage them you need to use the `Alertmanager | |
629 | <https://prometheus.io/docs/alerting/alertmanager>`_. | |
630 | If you are not using the Alertmanager yet, please `install it | |
631 | <https://github.com/prometheus/alertmanager#install>`_ as it's mandatory in | |
632 | order to receive and manage alerts from Prometheus. | |
633 | ||
634 | The Alertmanager capabilities can be consumed by the dashboard in three different | |
635 | ways: | |
636 | ||
637 | #. Use the notification receiver of the dashboard. | |
638 | ||
639 | #. Use the Prometheus Alertmanager API. | |
640 | ||
641 | #. Use both sources simultaneously. | |
642 | ||
643 | All three methods are going to notify you about alerts. You won't be notified | |
494da23a TL |
644 | twice if you use both sources, but you need to consume at least the Alertmanager API |
645 | in order to manage silences. | |
11fdf7f2 | 646 | |
9f95a23c | 647 | 1. Use the notification receiver of the dashboard |
11fdf7f2 | 648 | |
9f95a23c TL |
649 | This allows you to get notifications as `configured |
650 | <https://prometheus.io/docs/alerting/configuration/>`_ from the Alertmanager. | |
651 | You will get notified inside the dashboard once a notification is send out, | |
652 | but you are not able to manage alerts. | |
11fdf7f2 | 653 | |
9f95a23c TL |
654 | Add the dashboard receiver and the new route to your Alertmanager |
655 | configuration. This should look like:: | |
11fdf7f2 | 656 | |
9f95a23c TL |
657 | route: |
658 | receiver: 'ceph-dashboard' | |
659 | ... | |
660 | receivers: | |
661 | - name: 'ceph-dashboard' | |
662 | webhook_configs: | |
663 | - url: '<url-to-dashboard>/api/prometheus_receiver' | |
11fdf7f2 | 664 | |
494da23a | 665 | |
9f95a23c TL |
666 | Please make sure that the Alertmanager considers your SSL certificate in terms |
667 | of the dashboard as valid. For more information about the correct | |
668 | configuration checkout the `<http_config> documentation | |
669 | <https://prometheus.io/docs/alerting/configuration/#%3Chttp_config%3E>`_. | |
11fdf7f2 | 670 | |
9f95a23c | 671 | 2. Use the API of Prometheus and the Alertmanager |
11fdf7f2 | 672 | |
9f95a23c TL |
673 | This allows you to manage alerts and silences. This will enable the "Active |
674 | Alerts", "All Alerts" as well as the "Silences" tabs in the "Monitoring" | |
675 | section of the "Cluster" menu entry. | |
494da23a | 676 | |
9f95a23c TL |
677 | Alerts can be sorted by name, job, severity, state and start time. |
678 | Unfortunately it's not possible to know when an alert was sent out through a | |
679 | notification by the Alertmanager based on your configuration, that's why the | |
680 | dashboard will notify the user on any visible change to an alert and will | |
681 | notify the changed alert. | |
494da23a | 682 | |
9f95a23c TL |
683 | Silences can be sorted by id, creator, status, start, updated and end time. |
684 | Silences can be created in various ways, it's also possible to expire them. | |
494da23a | 685 | |
9f95a23c | 686 | #. Create from scratch |
494da23a | 687 | |
9f95a23c | 688 | #. Based on a selected alert |
11fdf7f2 | 689 | |
9f95a23c | 690 | #. Recreate from expired silence |
11fdf7f2 | 691 | |
9f95a23c | 692 | #. Update a silence (which will recreate and expire it (default Alertmanager behaviour)) |
11fdf7f2 | 693 | |
9f95a23c | 694 | To use it, specify the host and port of the Alertmanager server:: |
11fdf7f2 | 695 | |
9f95a23c | 696 | $ ceph dashboard set-alertmanager-api-host <alertmanager-host:port> # default: '' |
11fdf7f2 | 697 | |
9f95a23c | 698 | For example:: |
494da23a | 699 | |
9f95a23c | 700 | $ ceph dashboard set-alertmanager-api-host 'http://localhost:9093' |
494da23a | 701 | |
9f95a23c TL |
702 | To be able to see all configured alerts, you will need to configure the URL to |
703 | the Prometheus API. Using this API, the UI will also help you in verifying | |
704 | that a new silence will match a corresponding alert. | |
494da23a | 705 | |
9f95a23c | 706 | :: |
494da23a | 707 | |
9f95a23c | 708 | $ ceph dashboard set-prometheus-api-host <prometheus-host:port> # default: '' |
11fdf7f2 | 709 | |
9f95a23c | 710 | For example:: |
11fdf7f2 | 711 | |
9f95a23c | 712 | $ ceph dashboard set-prometheus-api-host 'http://localhost:9090' |
11fdf7f2 | 713 | |
9f95a23c | 714 | After setting up the hosts, you have to refresh the dashboard in your browser window. |
11fdf7f2 | 715 | |
9f95a23c | 716 | 3. Use both methods |
11fdf7f2 | 717 | |
9f95a23c TL |
718 | The different behaviors of both methods are configured in a way that they |
719 | should not disturb each other through annoying duplicated notifications | |
720 | popping up. | |
11fdf7f2 TL |
721 | |
722 | .. _dashboard-user-role-management: | |
723 | ||
724 | User and Role Management | |
725 | ------------------------ | |
726 | ||
9f95a23c TL |
727 | Password Policy |
728 | ^^^^^^^^^^^^^^^ | |
729 | ||
730 | By default the password policy feature is enabled including the following | |
731 | checks: | |
732 | ||
733 | - Is the password longer than N characters? | |
734 | - Are the old and new password the same? | |
735 | ||
736 | The password policy feature can be switched on or off completely:: | |
737 | ||
738 | $ ceph dashboard set-pwd-policy-enabled <true|false> | |
739 | ||
740 | The following individual checks can be switched on or off:: | |
741 | ||
742 | $ ceph dashboard set-pwd-policy-check-length-enabled <true|false> | |
743 | $ ceph dashboard set-pwd-policy-check-oldpwd-enabled <true|false> | |
744 | $ ceph dashboard set-pwd-policy-check-username-enabled <true|false> | |
745 | $ ceph dashboard set-pwd-policy-check-exclusion-list-enabled <true|false> | |
746 | $ ceph dashboard set-pwd-policy-check-complexity-enabled <true|false> | |
747 | $ ceph dashboard set-pwd-policy-check-sequential-chars-enabled <true|false> | |
748 | $ ceph dashboard set-pwd-policy-check-repetitive-chars-enabled <true|false> | |
749 | ||
750 | Additionally the following options are available to configure the password | |
751 | policy behaviour. | |
752 | ||
753 | - The minimum password length (defaults to 8):: | |
754 | ||
755 | $ ceph dashboard set-pwd-policy-min-length <N> | |
756 | ||
757 | - The minimum password complexity (defaults to 10):: | |
758 | ||
759 | $ ceph dashboard set-pwd-policy-min-complexity <N> | |
760 | ||
761 | The password complexity is calculated by classifying each character in | |
762 | the password. The complexity count starts by 0. A character is rated by | |
763 | the following rules in the given order. | |
764 | ||
765 | - Increase by 1 if the character is a digit. | |
766 | - Increase by 1 if the character is a lower case ASCII character. | |
767 | - Increase by 2 if the character is an upper case ASCII character. | |
768 | - Increase by 3 if the character is a special character like ``!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~``. | |
769 | - Increase by 5 if the character has not been classified by one of the previous rules. | |
770 | ||
771 | - A list of comma separated words that are not allowed to be used in a | |
772 | password:: | |
773 | ||
774 | $ ceph dashboard set-pwd-policy-exclusion-list <word>[,...] | |
775 | ||
776 | ||
11fdf7f2 TL |
777 | User Accounts |
778 | ^^^^^^^^^^^^^ | |
779 | ||
780 | Ceph Dashboard supports managing multiple user accounts. Each user account | |
781 | consists of a username, a password (stored in encrypted form using ``bcrypt``), | |
782 | an optional name, and an optional email address. | |
783 | ||
9f95a23c TL |
784 | If a new user is created via Web UI, it is possible to set an option that this |
785 | user must assign a new password when they log in for the first time. | |
786 | ||
11fdf7f2 TL |
787 | User accounts are stored in MON's configuration database, and are globally |
788 | shared across all ceph-mgr instances. | |
789 | ||
790 | We provide a set of CLI commands to manage user accounts: | |
791 | ||
792 | - *Show User(s)*:: | |
793 | ||
794 | $ ceph dashboard ac-user-show [<username>] | |
795 | ||
796 | - *Create User*:: | |
797 | ||
9f95a23c TL |
798 | $ ceph dashboard ac-user-create [--enabled] [--force-password] [--pwd_update_required] <username> [<password>] [<rolename>] [<name>] [<email>] [<pwd_expiration_date>] |
799 | ||
800 | To bypass the password policy checks use the `force-password` option. | |
801 | Use the option `pwd_update_required` so that a newly created user has | |
802 | to change their password after the first login. | |
11fdf7f2 TL |
803 | |
804 | - *Delete User*:: | |
805 | ||
806 | $ ceph dashboard ac-user-delete <username> | |
807 | ||
808 | - *Change Password*:: | |
809 | ||
9f95a23c TL |
810 | $ ceph dashboard ac-user-set-password [--force-password] <username> <password> |
811 | ||
812 | - *Change Password Hash*:: | |
813 | ||
814 | $ ceph dashboard ac-user-set-password-hash <username> <hash> | |
815 | ||
816 | The hash must be a bcrypt hash and salt, e.g. ``$2b$12$Pt3Vq/rDt2y9glTPSV.VFegiLkQeIpddtkhoFetNApYmIJOY8gau2``. | |
817 | This can be used to import users from an external database. | |
11fdf7f2 TL |
818 | |
819 | - *Modify User (name, and email)*:: | |
820 | ||
821 | $ ceph dashboard ac-user-set-info <username> <name> <email> | |
822 | ||
9f95a23c TL |
823 | - *Disable User*:: |
824 | ||
825 | $ ceph dashboard ac-user-disable <username> | |
826 | ||
827 | - *Enable User*:: | |
828 | ||
829 | $ ceph dashboard ac-user-enable <username> | |
11fdf7f2 TL |
830 | |
831 | User Roles and Permissions | |
832 | ^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
833 | ||
834 | User accounts are also associated with a set of roles that define which | |
835 | dashboard functionality can be accessed by the user. | |
836 | ||
837 | The Dashboard functionality/modules are grouped within a *security scope*. | |
838 | Security scopes are predefined and static. The current available security | |
839 | scopes are: | |
840 | ||
841 | - **hosts**: includes all features related to the ``Hosts`` menu | |
842 | entry. | |
843 | - **config-opt**: includes all features related to management of Ceph | |
844 | configuration options. | |
845 | - **pool**: includes all features related to pool management. | |
846 | - **osd**: includes all features related to OSD management. | |
847 | - **monitor**: includes all features related to Monitor management. | |
848 | - **rbd-image**: includes all features related to RBD image | |
849 | management. | |
850 | - **rbd-mirroring**: includes all features related to RBD-Mirroring | |
851 | management. | |
852 | - **iscsi**: includes all features related to iSCSI management. | |
853 | - **rgw**: includes all features related to Rados Gateway management. | |
854 | - **cephfs**: includes all features related to CephFS management. | |
855 | - **manager**: include all features related to Ceph Manager | |
856 | management. | |
857 | - **log**: include all features related to Ceph logs management. | |
858 | - **grafana**: include all features related to Grafana proxy. | |
859 | - **prometheus**: include all features related to Prometheus alert management. | |
860 | - **dashboard-settings**: allows to change dashboard settings. | |
861 | ||
862 | A *role* specifies a set of mappings between a *security scope* and a set of | |
863 | *permissions*. There are four types of permissions: | |
864 | ||
865 | - **read** | |
866 | - **create** | |
867 | - **update** | |
868 | - **delete** | |
869 | ||
870 | See below for an example of a role specification based on a Python dictionary:: | |
871 | ||
872 | # example of a role | |
873 | { | |
874 | 'role': 'my_new_role', | |
875 | 'description': 'My new role', | |
876 | 'scopes_permissions': { | |
877 | 'pool': ['read', 'create'], | |
878 | 'rbd-image': ['read', 'create', 'update', 'delete'] | |
879 | } | |
880 | } | |
881 | ||
882 | The above role dictates that a user has *read* and *create* permissions for | |
883 | features related to pool management, and has full permissions for | |
884 | features related to RBD image management. | |
885 | ||
886 | The Dashboard already provides a set of predefined roles that we call | |
887 | *system roles*, and can be used right away in a fresh Ceph Dashboard | |
888 | installation. | |
889 | ||
890 | The list of system roles are: | |
891 | ||
892 | - **administrator**: provides full permissions for all security scopes. | |
893 | - **read-only**: provides *read* permission for all security scopes except | |
894 | the dashboard settings. | |
895 | - **block-manager**: provides full permissions for *rbd-image*, | |
896 | *rbd-mirroring*, and *iscsi* scopes. | |
897 | - **rgw-manager**: provides full permissions for the *rgw* scope | |
898 | - **cluster-manager**: provides full permissions for the *hosts*, *osd*, | |
899 | *monitor*, *manager*, and *config-opt* scopes. | |
900 | - **pool-manager**: provides full permissions for the *pool* scope. | |
901 | - **cephfs-manager**: provides full permissions for the *cephfs* scope. | |
902 | ||
903 | The list of currently available roles can be retrieved by the following | |
904 | command:: | |
905 | ||
906 | $ ceph dashboard ac-role-show [<rolename>] | |
907 | ||
908 | It is also possible to create new roles using CLI commands. The available | |
909 | commands to manage roles are the following: | |
910 | ||
911 | - *Create Role*:: | |
912 | ||
913 | $ ceph dashboard ac-role-create <rolename> [<description>] | |
914 | ||
915 | - *Delete Role*:: | |
916 | ||
917 | $ ceph dashboard ac-role-delete <rolename> | |
918 | ||
919 | - *Add Scope Permissions to Role*:: | |
920 | ||
921 | $ ceph dashboard ac-role-add-scope-perms <rolename> <scopename> <permission> [<permission>...] | |
922 | ||
923 | - *Delete Scope Permission from Role*:: | |
924 | ||
9f95a23c | 925 | $ ceph dashboard ac-role-del-scope-perms <rolename> <scopename> |
11fdf7f2 TL |
926 | |
927 | To associate roles to users, the following CLI commands are available: | |
928 | ||
929 | - *Set User Roles*:: | |
930 | ||
931 | $ ceph dashboard ac-user-set-roles <username> <rolename> [<rolename>...] | |
932 | ||
933 | - *Add Roles To User*:: | |
934 | ||
935 | $ ceph dashboard ac-user-add-roles <username> <rolename> [<rolename>...] | |
936 | ||
937 | - *Delete Roles from User*:: | |
938 | ||
939 | $ ceph dashboard ac-user-del-roles <username> <rolename> [<rolename>...] | |
940 | ||
941 | ||
942 | Example of User and Custom Role Creation | |
943 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
944 | ||
945 | In this section we show a full example of the commands that need to be used | |
946 | in order to create a user account, that should be able to manage RBD images, | |
947 | view and create Ceph pools, and have read-only access to any other scopes. | |
948 | ||
949 | 1. *Create the user*:: | |
950 | ||
951 | $ ceph dashboard ac-user-create bob mypassword | |
952 | ||
953 | 2. *Create role and specify scope permissions*:: | |
954 | ||
955 | $ ceph dashboard ac-role-create rbd/pool-manager | |
956 | $ ceph dashboard ac-role-add-scope-perms rbd/pool-manager rbd-image read create update delete | |
957 | $ ceph dashboard ac-role-add-scope-perms rbd/pool-manager pool read create | |
958 | ||
959 | 3. *Associate roles to user*:: | |
960 | ||
961 | $ ceph dashboard ac-user-set-roles bob rbd/pool-manager read-only | |
962 | ||
9f95a23c | 963 | .. _dashboard-proxy-configuration: |
11fdf7f2 | 964 | |
81eedcae TL |
965 | Proxy Configuration |
966 | ------------------- | |
967 | ||
968 | In a Ceph cluster with multiple ceph-mgr instances, only the dashboard running | |
969 | on the currently active ceph-mgr daemon will serve incoming requests. Accessing | |
970 | the dashboard's TCP port on any of the other ceph-mgr instances that are | |
971 | currently on standby will perform a HTTP redirect (303) to the currently active | |
972 | manager's dashboard URL. This way, you can point your browser to any of the | |
973 | ceph-mgr instances in order to access the dashboard. | |
974 | ||
975 | If you want to establish a fixed URL to reach the dashboard or if you don't want | |
976 | to allow direct connections to the manager nodes, you could set up a proxy that | |
977 | automatically forwards incoming requests to the currently active ceph-mgr | |
978 | instance. | |
979 | ||
81eedcae TL |
980 | Configuring a URL Prefix |
981 | ^^^^^^^^^^^^^^^^^^^^^^^^ | |
3efd9988 | 982 | |
b32b8144 | 983 | If you are accessing the dashboard via a reverse proxy configuration, |
81eedcae | 984 | you may wish to service it under a URL prefix. To get the dashboard |
b32b8144 FG |
985 | to use hyperlinks that include your prefix, you can set the |
986 | ``url_prefix`` setting: | |
3efd9988 | 987 | |
b32b8144 | 988 | :: |
3efd9988 | 989 | |
11fdf7f2 | 990 | ceph config set mgr mgr/dashboard/url_prefix $PREFIX |
3efd9988 | 991 | |
b32b8144 | 992 | so you can access the dashboard at ``http://$IP:$PORT/$PREFIX/``. |
31f18b77 | 993 | |
eafe8130 TL |
994 | Disable the redirection |
995 | ^^^^^^^^^^^^^^^^^^^^^^^ | |
996 | ||
997 | If the dashboard is behind a load-balancing proxy like `HAProxy <https://www.haproxy.org/>`_ | |
998 | you might want to disable the redirection behaviour to prevent situations that | |
999 | internal (unresolvable) URL's are published to the frontend client. Use the | |
1000 | following command to get the dashboard to respond with a HTTP error (500 by default) | |
1001 | instead of redirecting to the active dashboard:: | |
1002 | ||
1003 | $ ceph config set mgr mgr/dashboard/standby_behaviour "error" | |
1004 | ||
1005 | To reset the setting to the default redirection behaviour, use the following command:: | |
1006 | ||
1007 | $ ceph config set mgr mgr/dashboard/standby_behaviour "redirect" | |
1008 | ||
1009 | Configure the error status code | |
1010 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1011 | ||
1012 | When the redirection behaviour is disabled, then you want to customize the HTTP status | |
1013 | code of standby dashboards. To do so you need to run the command:: | |
1014 | ||
1015 | $ ceph config set mgr mgr/dashboard/standby_error_status_code 503 | |
1016 | ||
1017 | HAProxy example configuration | |
1018 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1019 | ||
1020 | Below you will find an example configuration for SSL/TLS pass through using | |
1021 | `HAProxy <https://www.haproxy.org/>`_. | |
1022 | ||
1023 | Please note that the configuration works under the following conditions. | |
1024 | If the dashboard fails over, the front-end client might receive a HTTP redirect | |
1025 | (303) response and will be redirected to an unresolvable host. This happens when | |
1026 | the failover occurs during two HAProxy health checks. In this situation the | |
1027 | previously active dashboard node will now respond with a 303 which points to | |
1028 | the new active node. To prevent that situation you should consider to disable | |
1029 | the redirection behaviour on standby nodes. | |
1030 | ||
1031 | :: | |
1032 | ||
1033 | defaults | |
1034 | log global | |
1035 | option log-health-checks | |
1036 | timeout connect 5s | |
1037 | timeout client 50s | |
1038 | timeout server 450s | |
1039 | ||
1040 | frontend dashboard_front | |
1041 | mode http | |
1042 | bind *:80 | |
1043 | option httplog | |
1044 | redirect scheme https code 301 if !{ ssl_fc } | |
1045 | ||
1046 | frontend dashboard_front_ssl | |
1047 | mode tcp | |
1048 | bind *:443 | |
1049 | option tcplog | |
1050 | default_backend dashboard_back_ssl | |
1051 | ||
1052 | backend dashboard_back_ssl | |
1053 | mode tcp | |
1054 | option httpchk GET / | |
1055 | http-check expect status 200 | |
1056 | server x <HOST>:<PORT> check-ssl check verify none | |
1057 | server y <HOST>:<PORT> check-ssl check verify none | |
1058 | server z <HOST>:<PORT> check-ssl check verify none | |
81eedcae | 1059 | |
11fdf7f2 TL |
1060 | .. _dashboard-auditing: |
1061 | ||
1062 | Auditing API Requests | |
1063 | --------------------- | |
1064 | ||
1065 | The REST API is capable of logging PUT, POST and DELETE requests to the Ceph | |
1066 | audit log. This feature is disabled by default, but can be enabled with the | |
1067 | following command:: | |
1068 | ||
1069 | $ ceph dashboard set-audit-api-enabled <true|false> | |
1070 | ||
1071 | If enabled, the following parameters are logged per each request: | |
1072 | ||
1073 | * from - The origin of the request, e.g. https://[::1]:44410 | |
1074 | * path - The REST API path, e.g. /api/auth | |
1075 | * method - e.g. PUT, POST or DELETE | |
1076 | * user - The name of the user, otherwise 'None' | |
1077 | ||
1078 | The logging of the request payload (the arguments and their values) is enabled | |
1079 | by default. Execute the following command to disable this behaviour:: | |
1080 | ||
1081 | $ ceph dashboard set-audit-api-log-payload <true|false> | |
1082 | ||
1083 | A log entry may look like this:: | |
1084 | ||
1085 | 2018-10-22 15:27:01.302514 mgr.x [INF] [DASHBOARD] from='https://[::ffff:127.0.0.1]:37022' path='/api/rgw/user/klaus' method='PUT' user='admin' params='{"max_buckets": "1000", "display_name": "Klaus Mustermann", "uid": "klaus", "suspended": "0", "email": "klaus.mustermann@ceph.com"}' | |
1086 | ||
1087 | .. _dashboard-nfs-ganesha-management: | |
1088 | ||
1089 | NFS-Ganesha Management | |
1090 | ---------------------- | |
1091 | ||
1092 | Ceph Dashboard can manage `NFS Ganesha <http://nfs-ganesha.github.io/>`_ exports that use | |
1093 | CephFS or RadosGW as their backstore. | |
1094 | ||
1095 | To enable this feature in Ceph Dashboard there are some assumptions that need | |
1096 | to be met regarding the way NFS-Ganesha services are configured. | |
1097 | ||
1098 | The dashboard manages NFS-Ganesha config files stored in RADOS objects on the Ceph Cluster. | |
1099 | NFS-Ganesha must store part of their configuration in the Ceph cluster. | |
1100 | ||
1101 | These configuration files must follow some conventions. | |
11fdf7f2 TL |
1102 | Each export block must be stored in its own RADOS object named |
1103 | ``export-<id>``, where ``<id>`` must match the ``Export_ID`` attribute of the | |
1104 | export configuration. Then, for each NFS-Ganesha service daemon there should | |
1105 | exist a RADOS object named ``conf-<daemon_id>``, where ``<daemon_id>`` is an | |
1106 | arbitrary string that should uniquely identify the daemon instance (e.g., the | |
1107 | hostname where the daemon is running). | |
1108 | Each ``conf-<daemon_id>`` object contains the RADOS URLs to the exports that | |
1109 | the NFS-Ganesha daemon should serve. These URLs are of the form:: | |
1110 | ||
1111 | %url rados://<pool_name>[/<namespace>]/export-<id> | |
1112 | ||
1113 | Both the ``conf-<daemon_id>`` and ``export-<id>`` objects must be stored in the | |
1114 | same RADOS pool/namespace. | |
1115 | ||
1116 | ||
1117 | Configuring NFS-Ganesha in the Dashboard | |
1118 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1119 | ||
1120 | To enable the management of NFS-Ganesha exports in Ceph Dashboard, we only | |
1121 | need to tell the Dashboard, in which RADOS pool and namespace the | |
1122 | configuration objects are stored. Then, Ceph Dashboard can access the objects | |
1123 | by following the naming convention described above. | |
1124 | ||
1125 | The Dashboard command to configure the NFS-Ganesha configuration objects | |
1126 | location is:: | |
1127 | ||
1128 | $ ceph dashboard set-ganesha-clusters-rados-pool-namespace <pool_name>[/<namespace>] | |
1129 | ||
1130 | After running the above command, Ceph Dashboard is able to find the NFS-Ganesha | |
1131 | configuration objects and we can start manage the exports through the Web UI. | |
1132 | ||
1133 | ||
1134 | Support for Multiple NFS-Ganesha Clusters | |
1135 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1136 | ||
1137 | Ceph Dashboard also supports the management of NFS-Ganesha exports belonging | |
1138 | to different NFS-Ganesha clusters. An NFS-Ganesha cluster is a group of | |
1139 | NFS-Ganesha service daemons sharing the same exports. Different NFS-Ganesha | |
1140 | clusters are independent and don't share the exports configuration between each | |
1141 | other. | |
1142 | ||
1143 | Each NFS-Ganesha cluster should store its configuration objects in a | |
1144 | different RADOS pool/namespace to isolate the configuration from each other. | |
1145 | ||
1146 | To specify the locations of the configuration of each NFS-Ganesha cluster we | |
1147 | can use the same command as above but with a different value pattern:: | |
1148 | ||
1149 | $ ceph dashboard set-ganesha-clusters-rados-pool-namespace <cluster_id>:<pool_name>[/<namespace>](,<cluster_id>:<pool_name>[/<namespace>])* | |
1150 | ||
1151 | The ``<cluster_id>`` is an arbitrary string that should uniquely identify the | |
1152 | NFS-Ganesha cluster. | |
1153 | ||
1154 | When configuring the Ceph Dashboard with multiple NFS-Ganesha clusters, the | |
1155 | Web UI will automatically allow to choose to which cluster an export belongs. | |
1156 | ||
1157 | ||
f91f0fd5 TL |
1158 | Support for NFS-Ganesha Clusters Deployed by the Orchestrator |
1159 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1160 | ||
1161 | Ceph Dashboard can be used to manage NFS-Ganesha clusters deployed by the | |
1162 | Orchestrator. It can detect the clusters automatically. For more details | |
1163 | on deploying NFS-Ganesha clusters with the Orchestrator, please see :ref:`orchestrator-cli-stateless-services`. | |
1164 | Or particularly, see :ref:`deploy-cephadm-nfs-ganesha` for how to deploy | |
1165 | NFS-Ganesha clusters with the Cephadm backend. | |
1166 | ||
1167 | ||
11fdf7f2 TL |
1168 | Plug-ins |
1169 | -------- | |
1170 | ||
9f95a23c TL |
1171 | Dashboard Plug-ins extend the functionality of the dashboard in a modular |
1172 | and loosely coupled fashion. | |
11fdf7f2 | 1173 | |
e306af50 TL |
1174 | .. _Grafana: https://grafana.com/ |
1175 | ||
11fdf7f2 | 1176 | .. include:: dashboard_plugins/feature_toggles.inc.rst |
9f95a23c | 1177 | .. include:: dashboard_plugins/debug.inc.rst |
e306af50 TL |
1178 | |
1179 | ||
1180 | Troubleshooting the Dashboard | |
1181 | ----------------------------- | |
1182 | ||
1183 | Locating the Dashboard | |
1184 | ^^^^^^^^^^^^^^^^^^^^^^ | |
1185 | ||
1186 | If you are unsure of the location of the Ceph Dashboard, run the following command:: | |
1187 | ||
1188 | $ ceph mgr services | jq .dashboard | |
1189 | "https://host:port" | |
1190 | ||
1191 | The command returns the URL where the Ceph Dashboard is located: ``https://<host>:<port>/`` | |
1192 | ||
1193 | .. note:: | |
1194 | ||
1195 | Many Ceph command line tools return results in JSON format. You may have to install | |
1196 | the `jq <https://stedolan.github.io/jq>`_ command-line JSON processor utility on | |
1197 | your operating system beforehand. | |
1198 | ||
1199 | ||
1200 | Accessing the Dashboard | |
1201 | ^^^^^^^^^^^^^^^^^^^^^^^ | |
1202 | ||
1203 | If you are unable to access the Ceph Dashboard, run through the following | |
1204 | commands: | |
1205 | ||
1206 | #. Verify the Ceph Dashboard module is enabled:: | |
1207 | ||
1208 | $ ceph mgr module ls | jq .enabled_modules | |
1209 | ||
1210 | Ensure the Ceph Dashboard module is listed in the return value of the | |
1211 | command. Example snipped output from the command above:: | |
1212 | ||
1213 | [ | |
1214 | "dashboard", | |
1215 | "iostat", | |
1216 | "restful" | |
1217 | ] | |
1218 | ||
1219 | #. If it is not listed, activate the module with the following command:: | |
1220 | ||
1221 | $ ceph mgr module enable dashboard | |
1222 | ||
1223 | #. Check the Ceph Dashboard and/or mgr log file for any errors. The exact | |
1224 | location of the log files depends on the Ceph configuration. | |
1225 | ||
1226 | * Check if mgr log messages are written to a file by:: | |
1227 | ||
1228 | $ ceph config get mgr log_to_file | |
1229 | true | |
1230 | ||
1231 | * Get the location of the log file (it's ``/var/log/ceph/<cluster-name>-<daemon-name>.log`` | |
1232 | by default):: | |
1233 | ||
1234 | $ ceph config get mgr log_file | |
1235 | /var/log/ceph/$cluster-$name.log | |
1236 | ||
1237 | #. Ensure the SSL/TSL support is configured properly: | |
1238 | ||
1239 | * Check if the SSL/TSL support is enabled:: | |
1240 | ||
1241 | $ ceph config get mgr mgr/dashboard/ssl | |
1242 | ||
1243 | * If the command returns ``true``, verify a certificate exists by:: | |
1244 | ||
1245 | $ ceph config-key get mgr/dashboard/crt | |
1246 | ||
1247 | and:: | |
1248 | ||
1249 | $ ceph config-key get mgr/dashboard/key | |
1250 | ||
1251 | * If it doesn't, run the following command to generate a self-signed | |
1252 | certificate or follow the instructions outlined in | |
1253 | :ref:`dashboard-ssl-tls-support`:: | |
1254 | ||
1255 | $ ceph dashboard create-self-signed-cert | |
1256 | ||
1257 | ||
1258 | Trouble Logging into the Dashboard | |
1259 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1260 | ||
1261 | If you are unable to log into the Ceph Dashboard and you receive the following | |
1262 | error, run through the procedural checks below: | |
1263 | ||
1264 | .. image:: ../images/dashboard/invalid-credentials.png | |
1265 | :align: center | |
1266 | ||
1267 | #. Check that your user credentials are correct. If you are seeing the | |
1268 | notification message above when trying to log into the Ceph Dashboard, it | |
1269 | is likely you are using the wrong credentials. Double check your username | |
1270 | and password, and ensure the caps lock key is not enabled by accident. | |
1271 | ||
1272 | #. If your user credentials are correct, but you are experiencing the same | |
1273 | error, check that the user account exists:: | |
1274 | ||
1275 | $ ceph dashboard ac-user-show <username> | |
1276 | ||
1277 | This command returns your user data. If the user does not exist, it will | |
1278 | print:: | |
1279 | ||
1280 | $ Error ENOENT: User <username> does not exist | |
1281 | ||
1282 | #. Check if the user is enabled:: | |
1283 | ||
1284 | $ ceph dashboard ac-user-show <username> | jq .enabled | |
1285 | true | |
1286 | ||
1287 | Check if ``enabled`` is set to ``true`` for your user. If not the user is | |
1288 | not enabled, run:: | |
1289 | ||
1290 | $ ceph dashboard ac-user-enable <username> | |
1291 | ||
1292 | Please see :ref:`dashboard-user-role-management` for more information. | |
1293 | ||
1294 | ||
1295 | A Dashboard Feature is Not Working | |
1296 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1297 | ||
1298 | When an error occurs on the backend, you will usually receive an error | |
1299 | notification on the frontend. Run through the following scenarios to debug. | |
1300 | ||
1301 | #. Check the Ceph Dashboard/mgr logfile(s) for any errors. These can be | |
1302 | identified by searching for keywords, such as *500 Internal Server Error*, | |
1303 | followed by ``traceback``. The end of a traceback contains more details about | |
1304 | what exact error occurred. | |
1305 | #. Check your web browser's Javascript Console for any errors. | |
1306 | ||
1307 | ||
1308 | Ceph Dashboard Logs | |
1309 | ^^^^^^^^^^^^^^^^^^^ | |
1310 | ||
1311 | Dashboard Debug Flag | |
1312 | '''''''''''''''''''' | |
1313 | ||
1314 | With this flag enabled, traceback of errors are included in backend responses. | |
1315 | ||
1316 | To enable this flag via the Ceph Dashboard, navigate from *Cluster* to *Manager | |
1317 | modules*. Select *Dashboard module* and click the edit button. Click the | |
1318 | *debug* checkbox and update. | |
1319 | ||
1320 | To enable it via the CLI, run the following command:: | |
1321 | ||
1322 | $ ceph dashboard debug enable | |
1323 | ||
1324 | ||
1325 | Setting Logging Level of Dashboard Module | |
1326 | ''''''''''''''''''''''''''''''''''''''''' | |
1327 | ||
1328 | Setting the logging level to debug makes the log more verbose and helpful for | |
1329 | debugging. | |
1330 | ||
1331 | #. Increase the logging level of manager daemons:: | |
1332 | ||
1333 | $ ceph tell mgr config set debug_mgr 20 | |
1334 | ||
1335 | #. Adjust the logging level of the Ceph Dashboard module via the Dashboard or | |
1336 | CLI: | |
1337 | ||
1338 | * Navigate from *Cluster* to *Manager modules*. Select *Dashboard module* | |
1339 | and click the edit button. Modify the ``log_level`` configuration. | |
1340 | * To adjust it via the CLI, run the following command:: | |
1341 | ||
1342 | $ bin/ceph config set mgr mgr/dashboard/log_level debug |