The new :term:`Ceph Dashboard` module is a replacement of the previous one and
adds a built-in web based monitoring and administration application to the Ceph
-Manager. The architecture and functionality of this new plugin is derived from
+Manager. The architecture and functionality of this new module is derived from
and inspired by the `openATTIC Ceph management and monitoring tool
<https://openattic.org/>`_. The development is actively driven by the team
behind openATTIC at `SUSE <https://www.suse.com/>`_, with a lot of support from
* **Multi-User and Role Management**: The dashboard supports multiple user
accounts with different permissions (roles). The user accounts and roles
- can be modified on both the command line and via the WebUI.
- See :ref:`dashboard-user-role-management` for details.
+ can be modified on both the command line and via the WebUI. The dashboard
+ supports various methods to enhance password security, e.g. by enforcing
+ configurable password complexity rules, forcing users to change their password
+ after the first login or after a configurable time period. See
+ :ref:`dashboard-user-role-management` for details.
* **Single Sign-On (SSO)**: the dashboard supports authentication
via an external identity provider using the SAML 2.0 protocol. See
:ref:`dashboard-sso-support` for details.
* **Auditing**: the dashboard backend can be configured to log all PUT, POST
and DELETE API requests in the Ceph audit log. See :ref:`dashboard-auditing`
for instructions on how to enable this feature.
-* **Internationalization (I18N)**: use the dashboard in different languages.
+* **Internationalization (I18N)**: the dashboard can be used in different
+ languages that can be selected at run-time.
Currently, Ceph Dashboard is capable of monitoring and managing the following
aspects of your Ceph cluster:
-* **Overall cluster health**: Displays overall cluster status, performance
+* **Overall cluster health**: Display overall cluster status, performance
and capacity metrics.
* **Embedded Grafana Dashboards**: Ceph Dashboard is capable of embedding
- `Grafana <https://grafana.com>`_ dashboards in many locations, to display
- additional information and performance metrics gathered by the
- :ref:`mgr-prometheus`. See :ref:`dashboard-grafana` for details on how to
- configure this functionality.
-* **Cluster logs**: Display the latest updates to the cluster's event and audit
- log files.
-* **Hosts**: Provides a list of all hosts associated to the cluster, which
- services are running and which version of Ceph is installed.
-* **Performance counters**: Displays detailed service-specific statistics for
+ `Grafana`_ dashboards in many locations, to display additional information
+ and performance metrics gathered by the :ref:`mgr-prometheus`. See
+ :ref:`dashboard-grafana` for details on how to configure this functionality.
+* **Cluster logs**: Display the latest updates to the cluster's event and
+ audit log files. Log entries can be filtered by priority, date or keyword.
+* **Hosts**: Display a list of all hosts associated to the cluster, which
+ disks are attached, which services are running and which version of Ceph is
+ installed.
+* **Performance counters**: Display detailed service-specific statistics for
each running service.
-* **Monitors**: Lists all MONs, their quorum status, open sessions.
-* **Configuration Editor**: View all available configuration options,
+* **Monitors**: List all MONs, their quorum status, open sessions.
+* **Monitoring**: Enable creation, re-creation, editing and expiration of
+ Prometheus' silences, list the alerting configuration of Prometheus and all
+ configured and firing alerts. Show notifications for firing alerts.
+* **Configuration Editor**: Display all available configuration options,
their description, type and default values and edit the current values.
-* **Pools**: List all Ceph pools and their details (e.g. applications, placement
- groups, replication size, EC profile, CRUSH ruleset, etc.)
-* **OSDs**: Lists all OSDs, their status and usage statistics as well as
- detailed information like attributes (OSD map), metadata, performance counters
- and usage histograms for read/write operations. Mark OSDs as up/down/out,
- perform scrub operations. Select between different recovery profiles to adjust
- the level of backfilling activity.
-* **iSCSI**: Lists all hosts that run the TCMU runner service, displaying all
+* **Pools**: List all Ceph pools and their details (e.g. applications,
+ pg-autoscaling, placement groups, replication size, EC profile, CRUSH
+ rulesets, quotas etc.)
+* **OSDs**: List all OSDs, their status and usage statistics as well as
+ detailed information like attributes (OSD map), metadata, performance
+ counters and usage histograms for read/write operations. Mark OSDs
+ up/down/out, purge and reweight OSDs, perform scrub operations, modify
+ various scrub-related configuration options, select different profiles to
+ adjust the level of backfilling activity. List all disks associated with an
+ OSD. Set and change the device class of an OSD, display and sort OSDs by
+ device class. Deploy new OSDs on new disks/hosts.
+* **Device management**: List all hosts known by the orchestrator. List all
+ disks and their properties attached to a node. Display disk health information
+ (health prediction and SMART data). Blink enclosure LEDs.
+* **iSCSI**: List all hosts that run the TCMU runner service, display all
images and their performance characteristics (read/write ops, traffic).
- Create, modify and delete iSCSI targets (via ``ceph-iscsi``). See
- :ref:`dashboard-iscsi-management` for instructions on how to configure this
- feature.
-* **RBD**: Lists all RBD images and their properties (size, objects, features).
- Create, copy, modify and delete RBD images. Create, delete and rollback
- snapshots of selected images, protect/unprotect these snapshots against
- modification. Copy or clone snapshots, flatten cloned images.
+ Create, modify and delete iSCSI targets (via ``ceph-iscsi``). Display the
+ iSCSI gateway status on the landing page and info about active initiators.
+ See :ref:`dashboard-iscsi-management` for instructions on how to configure
+ this feature.
+* **RBD**: List all RBD images and their properties (size, objects, features).
+ Create, copy, modify and delete RBD images (incl. snapshots) and manage RBD
+ namespaces. Define various I/O or bandwidth limitation settings on a global,
+ per-pool or per-image level. Create, delete and rollback snapshots of selected
+ images, protect/unprotect these snapshots against modification. Copy or clone
+ snapshots, flatten cloned images.
* **RBD mirroring**: Enable and configure RBD mirroring to a remote Ceph server.
Lists all active sync daemons and their status, pools and RBD images including
their synchronization state.
-* **CephFS**: Lists all active filesystem clients and associated pools,
- including their usage statistics.
-* **Object Gateway**: Lists all active object gateways and their performance
+* **CephFS**: List all active file system clients and associated pools,
+ including their usage statistics. Evict active CephFS clients. Manage CephFS
+ quotas and snapshots. Browse a CephFS directory structure.
+* **Object Gateway**: List all active object gateways and their performance
counters. Display and manage (add/edit/delete) object gateway users and their
details (e.g. quotas) as well as the users' buckets and their details (e.g.
- owner, quotas). See :ref:`dashboard-enabling-object-gateway` for configuration
- instructions.
-* **NFS**: Manage NFS exports of CephFS filesystems and RGW S3 buckets via NFS
+ placement targets, owner, quotas, versioning, multi-factor authentication).
+ See :ref:`dashboard-enabling-object-gateway` for configuration instructions.
+* **NFS**: Manage NFS exports of CephFS file systems and RGW S3 buckets via NFS
Ganesha. See :ref:`dashboard-nfs-ganesha-management` for details on how to
enable this functionality.
* **Ceph Manager Modules**: Enable and disable all Ceph Manager modules, change
Ceph Dashboard is primarily tested and developed using the following web
browsers:
-+----------------------------------------------+----------+
-| Browser | Versions |
-+==============================================+==========+
-| `Chrome <https://www.google.com/chrome/>`_ | 68+ |
-+----------------------------------------------+----------+
-| `Firefox <http://www.mozilla.org/firefox/>`_ | 61+ |
-+----------------------------------------------+----------+
++-----------------------------------------------+----------+
+| Browser | Versions |
++===============================================+==========+
+| `Chrome <https://www.google.com/chrome/>`_ | 68+ |
++-----------------------------------------------+----------+
+| `Firefox <https://www.mozilla.org/firefox/>`_ | 61+ |
++-----------------------------------------------+----------+
While Ceph Dashboard might work in older browsers, we cannot guarantee it and
recommend you to update your browser to the latest version.
The ``dashboard.crt`` file should then be signed by a CA. Once that is done, you
can enable it for all Ceph manager instances by running the following commands::
- $ ceph config-key set mgr/dashboard/crt -i dashboard.crt
- $ ceph config-key set mgr/dashboard/key -i dashboard.key
+ $ ceph dashboard set-ssl-certificate -i dashboard.crt
+ $ ceph dashboard set-ssl-certificate-key -i dashboard.key
If different certificates are desired for each manager instance for some reason,
the name of the instance can be included as follows (where ``$name`` is the name
of the ``ceph-mgr`` instance, usually the hostname)::
- $ ceph config-key set mgr/dashboard/$name/crt -i dashboard.crt
- $ ceph config-key set mgr/dashboard/$name/key -i dashboard.key
+ $ ceph dashboard set-ssl-certificate $name -i dashboard.crt
+ $ ceph dashboard set-ssl-certificate-key $name -i dashboard.key
SSL can also be disabled by setting this configuration value::
This might be useful if the dashboard will be running behind a proxy which does
not support SSL for its upstream servers or other situations where SSL is not
-wanted or required.
+wanted or required. See :ref:`dashboard-proxy-configuration` for more details.
.. warning::
$ ceph dashboard ac-user-create <username> <password> administrator
+Account Lock-out
+^^^^^^^^^^^^^^^^
+
+It disables a user account if a user repeatedly enters the wrong credentials
+for multiple times. It is enabled by default to prevent brute-force or dictionary
+attacks. The user can get or set the default number of lock-out attempts using
+these commands respectively::
+
+ $ ceph dashboard get-account-lockout-attempts
+ $ ceph dashboard set-account-lockout-attempts <value:int>
+
+.. warning::
+
+ This feature can be disabled by setting the default number of lock-out attempts to 0.
+ However, by disabling this feature, the account is more vulnerable to brute-force or
+ dictionary based attacks. This can be disabled by::
+
+ $ ceph dashboard set-account-lockout-attempts 0
+
+Enable a Locked User
+^^^^^^^^^^^^^^^^^^^^
+
+If a user account is disabled as a result of multiple invalid login attempts, then
+it needs to be manually enabled by the administrator. This can be done by the following
+command::
+
+ $ ceph dashboard ac-user-enable <username>
+
+Accessing the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^^
+
+You can now access the dashboard using your (JavaScript-enabled) web browser, by
+pointing it to any of the host names or IP addresses and the selected TCP port
+where a manager instance is running: e.g., ``http(s)://<$IP>:<$PORT>/``.
+
+You should then be greeted by the dashboard login page, requesting your
+previously defined username and password.
+
.. _dashboard-enabling-object-gateway:
Enabling the Object Gateway Management Frontend
$ ceph dashboard set-rgw-api-access-key <access_key>
$ ceph dashboard set-rgw-api-secret-key <secret_key>
-This is all you have to do to get the Object Gateway management functionality
-working. The host and port of the Object Gateway are determined automatically.
+In a typical default configuration with a single RGW endpoint, this is all you
+have to do to get the Object Gateway management functionality working. The
+dashboard will try to automatically determine the host and port of the Object
+Gateway by obtaining this information from the Ceph Manager's service map.
If multiple zones are used, it will automatically determine the host within the
master zone group and master zone. This should be sufficient for most setups,
Enabling iSCSI Management
^^^^^^^^^^^^^^^^^^^^^^^^^
-The Ceph Dashboard can manage iSCSI targets using the REST API provided
-by the `rbd-target-api` service of the `ceph-iscsi <https://github.com/ceph/ceph-iscsi>`_
-project. Please make sure that it's installed and enabled on the iSCSI gateways.
+The Ceph Dashboard can manage iSCSI targets using the REST API provided by the
+`rbd-target-api` service of the :ref:`ceph-iscsi`. Please make sure that it's
+installed and enabled on the iSCSI gateways.
+
+.. note::
+
+ The iSCSI management functionality of Ceph Dashboard depends on the latest
+ version 3 of the `ceph-iscsi <https://github.com/ceph/ceph-iscsi>`_ project.
+ Make sure that your operating system provides the correct version, otherwise
+ the dashboard won't enable the management features.
If ceph-iscsi REST API is configured in HTTPS mode and its using a self-signed
-certificate, then we need to configure the dashboard to avoid SSL certificate
+certificate, then you need to configure the dashboard to avoid SSL certificate
verification when accessing ceph-iscsi API.
-To disable API SSL verification run the following commmand::
+To disable API SSL verification run the following command::
- $ ceph dashboard set-iscsi-api-ssl-verification false
+ $ ceph dashboard set-iscsi-api-ssl-verification false
The available iSCSI gateways must be defined using the following commands::
- $ ceph dashboard iscsi-gateway-list
-
$ ceph dashboard iscsi-gateway-add <scheme>://<username>:<password>@<host>[:port]
- $ ceph dashboard iscsi-gateway-rm <gateway_name>
+ $ ceph dashboard iscsi-gateway-list
+ $ ceph dashboard iscsi-gateway-add <scheme>://<username>:<password>@<host>[:port]
+ $ ceph dashboard iscsi-gateway-rm <gateway_name>
.. _dashboard-grafana:
Enabling the Embedding of Grafana Dashboards
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+`Grafana`_ requires data from `Prometheus <https://prometheus.io/>`_. Although
+Grafana can use other data sources, the Grafana dashboards we provide contain
+queries that are specific to Prometheus. Our Grafana dashboards therefore
+require Prometheus as the data source. The Ceph :ref:`mgr-prometheus` also only
+exports its data in the Prometheus' common format. The Grafana dashboards rely
+on metric names from the Prometheus module and `Node exporter
+<https://prometheus.io/docs/guides/node-exporter/>`_. The Node exporter is a
+separate application that provides machine metrics.
+
+.. note::
+
+ Prometheus' security model presumes that untrusted users have access to the
+ Prometheus HTTP endpoint and logs. Untrusted users have access to all the
+ (meta)data Prometheus collects that is contained in the database, plus a
+ variety of operational and debugging information.
+
+ However, Prometheus' HTTP API is limited to read-only operations.
+ Configurations can *not* be changed using the API and secrets are not
+ exposed. Moreover, Prometheus has some built-in measures to mitigate the
+ impact of denial of service attacks.
+
+ Please see `Prometheus' Security model
+ <https://prometheus.io/docs/operating/security/>` for more detailed
+ information.
+
Grafana and Prometheus are likely going to be bundled and installed by some
orchestration tools along Ceph in the near future, but currently, you will have
to install and configure both manually. After you have installed Prometheus and
Grafana on your preferred hosts, proceed with the following steps.
-#. Enable the Ceph Exporter which comes as Ceph Manager module by running::
+1. Enable the Ceph Exporter which comes as Ceph Manager module by running::
$ ceph mgr module enable prometheus
-More details can be found on the `documentation <http://docs.ceph.com/docs/master/
-mgr/prometheus/>`_ of the prometheus module.
+More details can be found in the documentation of the :ref:`mgr-prometheus`.
-#. Add the corresponding scrape configuration to Prometheus. This may look
+2. Add the corresponding scrape configuration to Prometheus. This may look
like::
- global:
- scrape_interval: 5s
+ global:
+ scrape_interval: 5s
+
+ scrape_configs:
+ - job_name: 'prometheus'
+ static_configs:
+ - targets: ['localhost:9090']
+ - job_name: 'ceph'
+ static_configs:
+ - targets: ['localhost:9283']
+ - job_name: 'node-exporter'
+ static_configs:
+ - targets: ['localhost:9100']
- scrape_configs:
- - job_name: 'prometheus'
- static_configs:
- - targets: ['localhost:9090']
- - job_name: 'ceph'
- static_configs:
- - targets: ['localhost:9283']
- - job_name: 'node-exporter'
- static_configs:
- - targets: ['localhost:9100']
+3. Add Prometheus as data source to Grafana
-#. Add Prometheus as data source to Grafana
+4. Install the `vonage-status-panel and grafana-piechart-panel` plugins using::
-#. Install the `vonage-status-panel and grafana-piechart-panel` plugins using::
+ grafana-cli plugins install vonage-status-panel
+ grafana-cli plugins install grafana-piechart-panel
- grafana-cli plugins install vonage-status-panel
- grafana-cli plugins install grafana-piechart-panel
+5. Add the Dashboards to Grafana:
-#. Add the Dashboards to Grafana:
+ Dashboards can be added to Grafana by importing dashboard jsons.
+ Following command can be used for downloading json files::
- Dashboards can be added to Grafana by importing dashboard jsons.
- Following command can be used for downloading json files::
+ wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/<Dashboard-name>.json
- wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/<Dashboard-name>.json
+ You can find all the dashboard jsons `here <https://github.com/ceph/ceph/tree/
+ master/monitoring/grafana/dashboards>`_ .
- You can find all the dashboard jsons `here <https://github.com/ceph/ceph/tree/
- master/monitoring/grafana/dashboards>`_ .
+ For Example, for ceph-cluster overview you can use::
- For Example, for ceph-cluster overview you can use::
+ wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/ceph-cluster.json
- wget https://raw.githubusercontent.com/ceph/ceph/master/monitoring/grafana/dashboards/ceph-cluster.json
+6. Configure Grafana in `/etc/grafana/grafana.ini` to adapt anonymous mode::
-#. Configure Grafana in `/etc/grafana/grafana.ini` to adapt anonymous mode::
+ [auth.anonymous]
+ enabled = true
+ org_name = Main Org.
+ org_role = Viewer
+
+ In newer versions of Grafana (starting with 6.2.0-beta1) a new setting named
+ ``allow_embedding`` has been introduced. This setting needs to be explicitly
+ set to ``true`` for the Grafana integration in Ceph Dashboard to work, as its
+ default is ``false``.
+
+ ::
+
+ [security]
+ allow_embedding = true
+
+Enabling RBD-Image monitoring
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- [auth.anonymous]
- enabled = true
- org_name = Main Org.
- org_role = Viewer
+Due to performance reasons, monitoring of RBD images is disabled by default. For
+more information please see :ref:`prometheus-rbd-io-statistics`. If disabled,
+the overview and details dashboards will stay empty in Grafana and the metrics
+will not be visible in Prometheus.
After you have set up Grafana and Prometheus, you will need to configure the
connection information that the Ceph Dashboard will use to access Grafana.
-You need to tell the dashboard on which url Grafana instance is running/deployed::
+You need to tell the dashboard on which URL the Grafana instance is
+running/deployed::
$ ceph dashboard set-grafana-api-url <grafana-server-url> # default: ''
The format of url is : `<protocol>:<IP-address>:<port>`
.. note::
+
Ceph Dashboard embeds the Grafana dashboards via ``iframe`` HTML elements.
If Grafana is configured without SSL/TLS support, most browsers will block the
embedding of insecure content into a secured web page, if the SSL support in
can't see the embedded Grafana dashboards after enabling them as outlined
above, check your browser's documentation on how to unblock mixed content.
Alternatively, consider enabling SSL/TLS support in Grafana.
-
+
+If you are using a self-signed certificate in your Grafana setup, then you should
+disable certificate verification in the dashboard to avoid refused connections,
+e.g. caused by certificates signed by unknown CA or not matching the host name::
+
+ $ ceph dashboard set-grafana-api-ssl-verify False
+
You can directly access Grafana Instance as well to monitor your cluster.
+Alternative URL for Browsers
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The Ceph Dashboard backend requires the Grafana URL to be able to verify the
+existence of Grafana Dashboards before the frontend even loads them. Due to the
+nature of how Grafana is implemented in Ceph Dashboard, this means that two
+working connections are required in order to be able to see Grafana graphs in
+Ceph Dashboard:
+
+- The backend (Ceph Mgr module) needs to verify the existence of the requested
+ graph. If this request succeeds, it lets the frontend know that it can safely
+ access Grafana.
+- The frontend then requests the Grafana graphs directly from the user's
+ browser using an iframe. The Grafana instance is accessed directly without any
+ detour through Ceph Dashboard.
+
+Now, it might be the case that your environment makes it difficult for the
+user's browser to directly access the URL configured in Ceph Dashboard. To solve
+this issue, a separate URL can be configured which will solely be used to tell
+the frontend (the user's browser) which URL it should use to access Grafana.
+This setting won't ever be changed automatically, unlike the GRAFANA_API_URL
+which is set by :ref:`cephadm` (only if cephadm is used to deploy monitoring
+services).
+
+To change the URL that is returned to the frontend issue the following command::
+
+ $ ceph dashboard set-grafana-frontend-api-url <grafana-server-url>
+
+If no value is set for that option, it will simply fall back to the value of the
+GRAFANA_API_URL option. If set, it will instruct the browser to use this URL to
+access Grafana.
+
.. _dashboard-sso-support:
Enabling Single Sign-On (SSO)
performed by an existing Identity Provider (IdP).
.. note::
+
Ceph Dashboard SSO support relies on onelogin's
`python-saml <https://pypi.org/project/python-saml/>`_ library.
Please ensure that this library is installed on your system, either by using
Parameters:
* **<ceph_dashboard_base_url>**: Base URL where Ceph Dashboard is accessible (e.g., `https://cephdashboard.local`)
-* **<idp_metadata>**: URL, file path or content of the IdP metadata XML (e.g., `https://myidp/metadata`)
+* **<idp_metadata>**: URL to remote (`http://`, `https://`) or local (`file://`) path or content of the IdP metadata XML (e.g., `https://myidp/metadata`, `file:///home/myuser/metadata.xml`).
* **<idp_username_attribute>** *(optional)*: Attribute that should be used to get the username from the authentication response. Defaults to `uid`.
* **<idp_entity_id>** *(optional)*: Use this when more than one entity id exists on the IdP metadata.
-* **<sp_x_509_cert> / <sp_private_key>** *(optional)*: File path or content of the certificate that should be used by Ceph Dashboard (Service Provider) for signing and encryption.
+* **<sp_x_509_cert> / <sp_private_key>** *(optional)*: File path of the certificate that should be used by Ceph Dashboard (Service Provider) for signing and encryption.
.. note::
+
The issuer value of SAML requests will follow this pattern: **<ceph_dashboard_base_url>**/auth/saml2/metadata
To display the current SAML 2.0 configuration, use the following command::
$ ceph dashboard sso show saml2
.. note::
+
For more information about `onelogin_settings`, please check the `onelogin documentation <https://github.com/onelogin/python-saml>`_.
To disable SSO::
$ ceph dashboard sso enable saml2
+.. _dashboard-alerting:
+
Enabling Prometheus Alerting
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#. Use both sources simultaneously.
All three methods are going to notify you about alerts. You won't be notified
-twice if you use both sources.
+twice if you use both sources, but you need to consume at least the Alertmanager API
+in order to manage silences.
-#. Use the notification receiver of the dashboard:
+1. Use the notification receiver of the dashboard
- This allows you to get notifications as `configured
- <https://prometheus.io/docs/alerting/configuration/>`_ from the Alertmanager.
- You will get notified inside the dashboard once a notification is send out,
- but you are not able to manage alerts.
+ This allows you to get notifications as `configured
+ <https://prometheus.io/docs/alerting/configuration/>`_ from the Alertmanager.
+ You will get notified inside the dashboard once a notification is send out,
+ but you are not able to manage alerts.
- Add the dashboard receiver and the new route to your Alertmanager configuration.
- This should look like::
+ Add the dashboard receiver and the new route to your Alertmanager
+ configuration. This should look like::
- route:
- receiver: 'ceph-dashboard'
- ...
- receivers:
- - name: 'ceph-dashboard'
- webhook_configs:
- - url: '<url-to-dashboard>/api/prometheus_receiver'
+ route:
+ receiver: 'ceph-dashboard'
+ ...
+ receivers:
+ - name: 'ceph-dashboard'
+ webhook_configs:
+ - url: '<url-to-dashboard>/api/prometheus_receiver'
- Please make sure that the Alertmanager considers your SSL certificate in terms
- of the dashboard as valid. For more information about the correct
- configuration checkout the `<http_config> documentation
- <https://prometheus.io/docs/alerting/configuration/#%3Chttp_config%3E>`_.
+ Please make sure that the Alertmanager considers your SSL certificate in terms
+ of the dashboard as valid. For more information about the correct
+ configuration checkout the `<http_config> documentation
+ <https://prometheus.io/docs/alerting/configuration/#%3Chttp_config%3E>`_.
-#. Use the API of the Prometheus Alertmanager
+2. Use the API of Prometheus and the Alertmanager
- This allows you to manage alerts. You will see all alerts, the Alertmanager
- currently knows of, in the alerts listing. It can be found in the *Cluster*
- submenu as *Alerts*. The alerts can be sorted by name, job, severity,
- state and start time. Unfortunately it's not possible to know when an alert
- was sent out through a notification by the Alertmanager based on your
- configuration, that's why the dashboard will notify the user on any visible
- change to an alert and will notify the changed alert.
+ This allows you to manage alerts and silences. This will enable the "Active
+ Alerts", "All Alerts" as well as the "Silences" tabs in the "Monitoring"
+ section of the "Cluster" menu entry.
- Currently it's not yet possible to silence an alert and expire an silenced
- alert, but this is work in progress and will be added in a future release.
+ Alerts can be sorted by name, job, severity, state and start time.
+ Unfortunately it's not possible to know when an alert was sent out through a
+ notification by the Alertmanager based on your configuration, that's why the
+ dashboard will notify the user on any visible change to an alert and will
+ notify the changed alert.
- To use it, specify the host and port of the Alertmanager server::
+ Silences can be sorted by id, creator, status, start, updated and end time.
+ Silences can be created in various ways, it's also possible to expire them.
- $ ceph dashboard set-alertmanager-api-host <alertmanager-host:port> # default: ''
+ #. Create from scratch
- For example::
+ #. Based on a selected alert
- $ ceph dashboard set-alertmanager-api-host 'http://localhost:9093'
+ #. Recreate from expired silence
+ #. Update a silence (which will recreate and expire it (default Alertmanager behaviour))
-#. Use both methods
+ To use it, specify the host and port of the Alertmanager server::
- The different behaviors of both methods are configured in a way that they
- should not disturb each other through annoying duplicated notifications
- popping up.
+ $ ceph dashboard set-alertmanager-api-host <alertmanager-host:port> # default: ''
-Accessing the dashboard
-^^^^^^^^^^^^^^^^^^^^^^^
+ For example::
-You can now access the dashboard using your (JavaScript-enabled) web browser, by
-pointing it to any of the host names or IP addresses and the selected TCP port
-where a manager instance is running: e.g., ``httpS://<$IP>:<$PORT>/``.
+ $ ceph dashboard set-alertmanager-api-host 'http://localhost:9093'
-You should then be greeted by the dashboard login page, requesting your
-previously defined username and password. Select the **Keep me logged in**
-checkbox if you want to skip the username/password request when accessing the
-dashboard in the future.
+ To be able to see all configured alerts, you will need to configure the URL to
+ the Prometheus API. Using this API, the UI will also help you in verifying
+ that a new silence will match a corresponding alert.
+
+ ::
+
+ $ ceph dashboard set-prometheus-api-host <prometheus-host:port> # default: ''
+
+ For example::
+
+ $ ceph dashboard set-prometheus-api-host 'http://localhost:9090'
+
+ After setting up the hosts, you have to refresh the dashboard in your browser window.
+
+3. Use both methods
+
+ The different behaviors of both methods are configured in a way that they
+ should not disturb each other through annoying duplicated notifications
+ popping up.
.. _dashboard-user-role-management:
User and Role Management
------------------------
+Password Policy
+^^^^^^^^^^^^^^^
+
+By default the password policy feature is enabled including the following
+checks:
+
+- Is the password longer than N characters?
+- Are the old and new password the same?
+
+The password policy feature can be switched on or off completely::
+
+ $ ceph dashboard set-pwd-policy-enabled <true|false>
+
+The following individual checks can be switched on or off::
+
+ $ ceph dashboard set-pwd-policy-check-length-enabled <true|false>
+ $ ceph dashboard set-pwd-policy-check-oldpwd-enabled <true|false>
+ $ ceph dashboard set-pwd-policy-check-username-enabled <true|false>
+ $ ceph dashboard set-pwd-policy-check-exclusion-list-enabled <true|false>
+ $ ceph dashboard set-pwd-policy-check-complexity-enabled <true|false>
+ $ ceph dashboard set-pwd-policy-check-sequential-chars-enabled <true|false>
+ $ ceph dashboard set-pwd-policy-check-repetitive-chars-enabled <true|false>
+
+Additionally the following options are available to configure the password
+policy behaviour.
+
+- The minimum password length (defaults to 8)::
+
+ $ ceph dashboard set-pwd-policy-min-length <N>
+
+- The minimum password complexity (defaults to 10)::
+
+ $ ceph dashboard set-pwd-policy-min-complexity <N>
+
+ The password complexity is calculated by classifying each character in
+ the password. The complexity count starts by 0. A character is rated by
+ the following rules in the given order.
+
+ - Increase by 1 if the character is a digit.
+ - Increase by 1 if the character is a lower case ASCII character.
+ - Increase by 2 if the character is an upper case ASCII character.
+ - Increase by 3 if the character is a special character like ``!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~``.
+ - Increase by 5 if the character has not been classified by one of the previous rules.
+
+- A list of comma separated words that are not allowed to be used in a
+ password::
+
+ $ ceph dashboard set-pwd-policy-exclusion-list <word>[,...]
+
+
User Accounts
^^^^^^^^^^^^^
consists of a username, a password (stored in encrypted form using ``bcrypt``),
an optional name, and an optional email address.
+If a new user is created via Web UI, it is possible to set an option that this
+user must assign a new password when they log in for the first time.
+
User accounts are stored in MON's configuration database, and are globally
shared across all ceph-mgr instances.
- *Create User*::
- $ ceph dashboard ac-user-create <username> [<password>] [<rolename>] [<name>] [<email>]
+ $ ceph dashboard ac-user-create [--enabled] [--force-password] [--pwd_update_required] <username> [<password>] [<rolename>] [<name>] [<email>] [<pwd_expiration_date>]
+
+ To bypass the password policy checks use the `force-password` option.
+ Use the option `pwd_update_required` so that a newly created user has
+ to change their password after the first login.
- *Delete User*::
- *Change Password*::
- $ ceph dashboard ac-user-set-password <username> <password>
+ $ ceph dashboard ac-user-set-password [--force-password] <username> <password>
+
+- *Change Password Hash*::
+
+ $ ceph dashboard ac-user-set-password-hash <username> <hash>
+
+ The hash must be a bcrypt hash and salt, e.g. ``$2b$12$Pt3Vq/rDt2y9glTPSV.VFegiLkQeIpddtkhoFetNApYmIJOY8gau2``.
+ This can be used to import users from an external database.
- *Modify User (name, and email)*::
$ ceph dashboard ac-user-set-info <username> <name> <email>
+- *Disable User*::
+
+ $ ceph dashboard ac-user-disable <username>
+
+- *Enable User*::
+
+ $ ceph dashboard ac-user-enable <username>
User Roles and Permissions
^^^^^^^^^^^^^^^^^^^^^^^^^^
- *Delete Scope Permission from Role*::
- $ ceph dashboard ac-role-del-perms <rolename> <scopename>
+ $ ceph dashboard ac-role-del-scope-perms <rolename> <scopename>
To associate roles to users, the following CLI commands are available:
$ ceph dashboard ac-user-set-roles bob rbd/pool-manager read-only
+.. _dashboard-proxy-configuration:
+
+Proxy Configuration
+-------------------
+
+In a Ceph cluster with multiple ceph-mgr instances, only the dashboard running
+on the currently active ceph-mgr daemon will serve incoming requests. Accessing
+the dashboard's TCP port on any of the other ceph-mgr instances that are
+currently on standby will perform a HTTP redirect (303) to the currently active
+manager's dashboard URL. This way, you can point your browser to any of the
+ceph-mgr instances in order to access the dashboard.
-Reverse Proxies
----------------
+If you want to establish a fixed URL to reach the dashboard or if you don't want
+to allow direct connections to the manager nodes, you could set up a proxy that
+automatically forwards incoming requests to the currently active ceph-mgr
+instance.
+
+Configuring a URL Prefix
+^^^^^^^^^^^^^^^^^^^^^^^^
If you are accessing the dashboard via a reverse proxy configuration,
-you may wish to service it under a URL prefix. To get the dashboard
+you may wish to service it under a URL prefix. To get the dashboard
to use hyperlinks that include your prefix, you can set the
``url_prefix`` setting:
so you can access the dashboard at ``http://$IP:$PORT/$PREFIX/``.
+Disable the redirection
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If the dashboard is behind a load-balancing proxy like `HAProxy <https://www.haproxy.org/>`_
+you might want to disable the redirection behaviour to prevent situations that
+internal (unresolvable) URL's are published to the frontend client. Use the
+following command to get the dashboard to respond with a HTTP error (500 by default)
+instead of redirecting to the active dashboard::
+
+ $ ceph config set mgr mgr/dashboard/standby_behaviour "error"
+
+To reset the setting to the default redirection behaviour, use the following command::
+
+ $ ceph config set mgr mgr/dashboard/standby_behaviour "redirect"
+
+Configure the error status code
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the redirection behaviour is disabled, then you want to customize the HTTP status
+code of standby dashboards. To do so you need to run the command::
+
+ $ ceph config set mgr mgr/dashboard/standby_error_status_code 503
+
+HAProxy example configuration
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Below you will find an example configuration for SSL/TLS pass through using
+`HAProxy <https://www.haproxy.org/>`_.
+
+Please note that the configuration works under the following conditions.
+If the dashboard fails over, the front-end client might receive a HTTP redirect
+(303) response and will be redirected to an unresolvable host. This happens when
+the failover occurs during two HAProxy health checks. In this situation the
+previously active dashboard node will now respond with a 303 which points to
+the new active node. To prevent that situation you should consider to disable
+the redirection behaviour on standby nodes.
+
+::
+
+ defaults
+ log global
+ option log-health-checks
+ timeout connect 5s
+ timeout client 50s
+ timeout server 450s
+
+ frontend dashboard_front
+ mode http
+ bind *:80
+ option httplog
+ redirect scheme https code 301 if !{ ssl_fc }
+
+ frontend dashboard_front_ssl
+ mode tcp
+ bind *:443
+ option tcplog
+ default_backend dashboard_back_ssl
+
+ backend dashboard_back_ssl
+ mode tcp
+ option httpchk GET /
+ http-check expect status 200
+ server x <HOST>:<PORT> check-ssl check verify none
+ server y <HOST>:<PORT> check-ssl check verify none
+ server z <HOST>:<PORT> check-ssl check verify none
+
.. _dashboard-auditing:
Auditing API Requests
NFS-Ganesha must store part of their configuration in the Ceph cluster.
These configuration files must follow some conventions.
-conventions.
Each export block must be stored in its own RADOS object named
``export-<id>``, where ``<id>`` must match the ``Export_ID`` attribute of the
export configuration. Then, for each NFS-Ganesha service daemon there should
Web UI will automatically allow to choose to which cluster an export belongs.
+Support for NFS-Ganesha Clusters Deployed by the Orchestrator
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Ceph Dashboard can be used to manage NFS-Ganesha clusters deployed by the
+Orchestrator. It can detect the clusters automatically. For more details
+on deploying NFS-Ganesha clusters with the Orchestrator, please see :ref:`orchestrator-cli-stateless-services`.
+Or particularly, see :ref:`deploy-cephadm-nfs-ganesha` for how to deploy
+NFS-Ganesha clusters with the Cephadm backend.
+
+
Plug-ins
--------
-Dashboard Plug-ins allow to extend the functionality of the dashboard in a modular
-and loosely coupled approach.
+Dashboard Plug-ins extend the functionality of the dashboard in a modular
+and loosely coupled fashion.
+
+.. _Grafana: https://grafana.com/
.. include:: dashboard_plugins/feature_toggles.inc.rst
+.. include:: dashboard_plugins/debug.inc.rst
+
+
+Troubleshooting the Dashboard
+-----------------------------
+
+Locating the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^
+
+If you are unsure of the location of the Ceph Dashboard, run the following command::
+
+ $ ceph mgr services | jq .dashboard
+ "https://host:port"
+
+The command returns the URL where the Ceph Dashboard is located: ``https://<host>:<port>/``
+
+.. note::
+
+ Many Ceph command line tools return results in JSON format. You may have to install
+ the `jq <https://stedolan.github.io/jq>`_ command-line JSON processor utility on
+ your operating system beforehand.
+
+
+Accessing the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^^
+
+If you are unable to access the Ceph Dashboard, run through the following
+commands:
+
+#. Verify the Ceph Dashboard module is enabled::
+
+ $ ceph mgr module ls | jq .enabled_modules
+
+ Ensure the Ceph Dashboard module is listed in the return value of the
+ command. Example snipped output from the command above::
+
+ [
+ "dashboard",
+ "iostat",
+ "restful"
+ ]
+
+#. If it is not listed, activate the module with the following command::
+
+ $ ceph mgr module enable dashboard
+
+#. Check the Ceph Dashboard and/or mgr log file for any errors. The exact
+ location of the log files depends on the Ceph configuration.
+
+ * Check if mgr log messages are written to a file by::
+
+ $ ceph config get mgr log_to_file
+ true
+
+ * Get the location of the log file (it's ``/var/log/ceph/<cluster-name>-<daemon-name>.log``
+ by default)::
+
+ $ ceph config get mgr log_file
+ /var/log/ceph/$cluster-$name.log
+
+#. Ensure the SSL/TSL support is configured properly:
+
+ * Check if the SSL/TSL support is enabled::
+
+ $ ceph config get mgr mgr/dashboard/ssl
+
+ * If the command returns ``true``, verify a certificate exists by::
+
+ $ ceph config-key get mgr/dashboard/crt
+
+ and::
+
+ $ ceph config-key get mgr/dashboard/key
+
+ * If it doesn't, run the following command to generate a self-signed
+ certificate or follow the instructions outlined in
+ :ref:`dashboard-ssl-tls-support`::
+
+ $ ceph dashboard create-self-signed-cert
+
+
+Trouble Logging into the Dashboard
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you are unable to log into the Ceph Dashboard and you receive the following
+error, run through the procedural checks below:
+
+.. image:: ../images/dashboard/invalid-credentials.png
+ :align: center
+
+#. Check that your user credentials are correct. If you are seeing the
+ notification message above when trying to log into the Ceph Dashboard, it
+ is likely you are using the wrong credentials. Double check your username
+ and password, and ensure the caps lock key is not enabled by accident.
+
+#. If your user credentials are correct, but you are experiencing the same
+ error, check that the user account exists::
+
+ $ ceph dashboard ac-user-show <username>
+
+ This command returns your user data. If the user does not exist, it will
+ print::
+
+ $ Error ENOENT: User <username> does not exist
+
+#. Check if the user is enabled::
+
+ $ ceph dashboard ac-user-show <username> | jq .enabled
+ true
+
+ Check if ``enabled`` is set to ``true`` for your user. If not the user is
+ not enabled, run::
+
+ $ ceph dashboard ac-user-enable <username>
+
+Please see :ref:`dashboard-user-role-management` for more information.
+
+
+A Dashboard Feature is Not Working
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When an error occurs on the backend, you will usually receive an error
+notification on the frontend. Run through the following scenarios to debug.
+
+#. Check the Ceph Dashboard/mgr logfile(s) for any errors. These can be
+ identified by searching for keywords, such as *500 Internal Server Error*,
+ followed by ``traceback``. The end of a traceback contains more details about
+ what exact error occurred.
+#. Check your web browser's Javascript Console for any errors.
+
+
+Ceph Dashboard Logs
+^^^^^^^^^^^^^^^^^^^
+
+Dashboard Debug Flag
+''''''''''''''''''''
+
+With this flag enabled, traceback of errors are included in backend responses.
+
+To enable this flag via the Ceph Dashboard, navigate from *Cluster* to *Manager
+modules*. Select *Dashboard module* and click the edit button. Click the
+*debug* checkbox and update.
+
+To enable it via the CLI, run the following command::
+
+ $ ceph dashboard debug enable
+
+
+Setting Logging Level of Dashboard Module
+'''''''''''''''''''''''''''''''''''''''''
+
+Setting the logging level to debug makes the log more verbose and helpful for
+debugging.
+
+#. Increase the logging level of manager daemons::
+
+ $ ceph tell mgr config set debug_mgr 20
+
+#. Adjust the logging level of the Ceph Dashboard module via the Dashboard or
+ CLI:
+
+ * Navigate from *Cluster* to *Manager modules*. Select *Dashboard module*
+ and click the edit button. Modify the ``log_level`` configuration.
+ * To adjust it via the CLI, run the following command::
+
+ $ bin/ceph config set mgr mgr/dashboard/log_level debug
projects / ceph.git / blobdiff