1 Ceph Dashboard Developer Documentation
2 ======================================
4 .. contents:: Table of Contents
9 Before you can start the dashboard from within a development environment, you
10 will need to generate the frontend code and either use a compiled and running
11 Ceph cluster (e.g. started by ``vstart.sh``) or the standalone development web
14 The build process is based on `Node.js <https://nodejs.org/>`_ and requires the
15 `Node Package Manager <https://www.npmjs.com/>`_ ``npm`` to be installed.
20 * Node 8.9.0 or higher
24 During Ceph's build we create a virtualenv with ``node`` and ``npm``
25 installed, which can be used as an alternative to installing node/npm in your
28 If you want to use the node installed in the virtualenv you just need to
29 activate the virtualenv before you run any npm commands. To activate it run
30 ``. build/src/pybind/mgr/dashboard/node-env/bin/activate``.
32 Once you finish, you can simply run ``deactivate`` and exit the virtualenv.
35 If you do not have the `Angular CLI <https://github.com/angular/angular-cli>`_
36 installed globally, then you need to execute ``ng`` commands with an
37 additional ``npm run`` before it.
42 Run ``npm install`` in directory ``src/pybind/mgr/dashboard/frontend`` to
43 install the required packages locally.
45 Setting up a Development Server
46 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
48 Create the ``proxy.conf.json`` file based on ``proxy.conf.json.sample``.
50 Run ``npm start`` for a dev server.
51 Navigate to ``http://localhost:4200/``. The app will automatically
52 reload if you change any of the source files.
57 Run ``ng generate component component-name`` to generate a new
58 component. You can also use
59 ``ng generate directive|pipe|service|class|guard|interface|enum|module``.
64 Run ``npm run build`` to build the project. The build artifacts will be
65 stored in the ``dist/`` directory. Use the ``-prod`` flag for a
66 production build. Navigate to ``https://localhost:8443``.
68 Build the Code Documentation
69 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
71 Run ``npm run doc-build`` to generate code docs in the ``documentation/``
72 directory. To make them accesible locally for a web browser, run
73 ``npm run doc-serve`` and they will become available at ``http://localhost:8444``.
74 With ``npm run compodoc -- <opts>`` you may
75 `fully configure it https://compodoc.app/guides/usage.html`_.
77 Code linting and formatting
78 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
80 We use the following tools to lint and format the code in all our TS, SCSS and
83 - `codelyzer <http://codelyzer.com/>`_
84 - `html-linter <https://github.com/chinchiheather/html-linter>`_
85 - `Prettier <https://prettier.io/>`_
86 - `TSLint <https://palantir.github.io/tslint/>`_
88 We added 2 npm scripts to help run these tools:
90 - ``npm run lint``, will check frontend files against all linters
91 - ``npm run fix``, will try to fix all the detected linting errors
96 To write unit tests most efficient we have a small collection of tools,
97 we use within test suites.
99 Those tools can be found under
100 ``src/pybind/mgr/dashboard/frontend/src/testing/``, especially take
101 a look at ``unit-test-helper.ts``.
103 There you will be able to find:
105 ``configureTestBed`` that replaces the initial ``TestBed``
106 methods. It takes the same arguments as ``TestBed.configureTestingModule``.
107 Using it will run your tests a lot faster in development, as it doesn't
108 recreate everything from scratch on every test. To use the default behaviour
109 pass ``true`` as the second argument.
111 ``PermissionHelper`` to help determine if
112 the correct actions are shown based on the current permissions and selection
115 ``FormHelper`` which makes testing a form a lot easier
116 with a few simple methods. It allows you to set a control or multiple
117 controls, expect if a control is valid or has an error or just do both with
118 one method. Additional you can expect a template element or multiple elements
119 to be visible in the rendered template.
124 Create ``unit-test-configuration.ts`` file based on
125 ``unit-test-configuration.ts.sample`` in directory
126 ``src/pybind/mgr/dashboard/frontend/src``.
128 Run ``npm run test`` to execute the unit tests via `Jest
129 <https://facebook.github.io/jest/>`_.
131 If you get errors on all tests, it could be because `Jest
132 <https://facebook.github.io/jest/>`_ or something else was updated.
133 There are a few ways how you can try to resolve this:
135 - Remove all modules with ``rm -rf dist node_modules`` and run ``npm install``
136 again in order to reinstall them
137 - Clear the cache of jest by running ``npx jest --clearCache``
139 Running End-to-End Tests
140 ~~~~~~~~~~~~~~~~~~~~~~~~
142 We use `Protractor <http://www.protractortest.org/>`__ to run our frontend e2e
145 Our ``run-frontend-e2e-tests.sh`` script will check if Chrome or Docker is
146 installed and run the tests if either is found.
148 Start all frontend e2e tests by running::
150 $ ./run-frontend-e2e-tests.sh
153 You can force the script to use a specific device with the ``-d`` flag::
155 $ ./run-frontend-e2e-tests.sh -d <chrome|docker>
158 If you want to run the tests outside the ceph environment, you will need to
159 manually define the dashboard url using ``-r``::
161 $ ./run-frontend-e2e-tests.sh -r <DASHBOARD_URL>
164 When using docker, as your device, you might need to run the script with sudo
167 Writing End-to-End Tests
168 ~~~~~~~~~~~~~~~~~~~~~~~~
170 When writing e2e tests you don't want to recompile every time from scratch to
171 try out if your test has succeeded. As usual you have your development server
172 open (``npm start``) which already has compiled all files. To attach
173 `Protractor <http://www.protractortest.org/>`__ to this process, instead of
174 spinning up it's own server, you can use ``npm run e2e -- --dev-server-target``
175 or just ``npm run e2e:dev`` which is equivalent.
180 To get more help on the Angular CLI use ``ng help`` or go check out the
182 README <https://github.com/angular/angular-cli/blob/master/README.md>`__.
184 Example of a Generator
185 ~~~~~~~~~~~~~~~~~~~~~~
189 # Create module 'Core'
190 src/app> ng generate module core -m=app --routing
192 # Create module 'Auth' under module 'Core'
193 src/app/core> ng generate module auth -m=core --routing
195 src/app> ng generate module core/auth -m=core --routing
197 # Create component 'Login' under module 'Auth'
198 src/app/core/auth> ng generate component login -m=core/auth
200 src/app> ng generate component core/auth/login -m=core/auth
202 Frontend Typescript Code Style Guide Recommendations
203 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
205 Group the imports based on its source and separate them with a blank
208 The source groups can be either from Angular, external or internal.
214 import { Component } from '@angular/core';
215 import { Router } from '@angular/router';
217 import { ToastrManager } from 'ngx-toastr';
219 import { Credentials } from '../../../shared/models/credentials.model';
220 import { HostService } from './services/host.service';
225 There are several components that can be reused on different pages.
226 This components are declared on the components module:
227 `src/pybind/mgr/dashboard/frontend/src/app/shared/components`.
232 This component should be used to provide additional information to the user.
239 Some <strong>helper</strong> html text
242 Terminology and wording
243 ~~~~~~~~~~~~~~~~~~~~~~~
245 Instead of using the Ceph component names, the approach
246 suggested is to use the logical/generic names (Block over RBD, Filesystem over
247 CephFS, Object over RGW). Nevertheless, as Ceph-Dashboard cannot completely hide
248 the Ceph internals, some Ceph-specific names might remain visible.
250 Regarding the wording for action labels and other textual elements (form titles,
251 buttons, etc.), the chosen approach is to follow `these guidelines
252 <https://www.patternfly.org/styles/terminology-and-wording/#terminology-and-wording-for-action-labels>`_.
253 As a rule of thumb, 'Create' and 'Delete' are the proper wording for most forms,
254 instead of 'Add' and 'Remove', unless some already created item is either added
255 or removed to/from a set of items (e.g.: 'Add permission' to a user vs. 'Create
258 In order to enforce the use of this wording, a service ``ActionLabelsI18n`` has
259 been created, which provides translated labels for use in UI elements.
264 Every vendor can customize the 'Ceph dashboard' to his needs. No matter if
265 logo, HTML-Template or TypeScript, every file inside the frontend folder can be
268 To replace files, open ``./frontend/angular.json`` and scroll to the section
269 ``fileReplacements`` inside the production configuration. Here you can add the
270 files you wish to brand. We recommend to place the branded version of a file in
271 the same directory as the original one and to add a ``.brand`` to the file
272 name, right in front of the file extension. A ``fileReplacement`` could for
273 example look like this:
278 "replace": "src/app/core/auth/login/login.component.html",
279 "with": "src/app/core/auth/login/login.component.brand.html"
282 To serve or build the branded user interface run:
284 $ npm run start -- --prod
288 $ npm run build -- --prod
290 Unfortunately it's currently not possible to use multiple configurations when
291 serving or building the UI at the same time. That means a configuration just
292 for the branding ``fileReplacements`` is not an option, because you want to use
293 the production configuration anyway
294 (https://github.com/angular/angular-cli/issues/10612).
295 Furthermore it's also not possible to use glob expressions for
296 ``fileReplacements``. As long as the feature hasn't been implemented, you have
297 to add the file replacements manually to the angular.json file
298 (https://github.com/angular/angular-cli/issues/12354).
300 Nevertheless you should stick to the suggested naming scheme because it makes
301 it easier for you to use glob expressions once it's supported in the future.
303 To change the variable defaults you can overwrite them in the file
304 ``./frontend/src/vendor.variables.scss``. Just reassign the variable you want
305 to change, for example ``$color-primary: teal;``
306 To overwrite or extend the default CSS, you can add your own styles in
307 ``./frontend/src/vendor.overrides.scss``.
312 How to extract messages from source code?
313 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
315 To extract the I18N messages from the templates and the TypeScript files just
316 run the following command in ``src/pybind/mgr/dashboard/frontend``::
320 This will extract all marked messages from the HTML templates first and then
321 add all marked strings from the TypeScript files to the translation template.
322 Since the extraction from TypeScript files is still not supported by Angular
323 itself, we are using the
324 `ngx-translator <https://github.com/ngx-translate/i18n-polyfill>`_ extractor to
325 parse the TypeScript files.
327 When the command ran successfully, it should have created or updated the file
328 ``src/locale/messages.xlf``.
330 The file isn't tracked by git, you can just use it to start with the
331 translation offline or add/update the resource files on transifex.
336 All our supported languages should be registered in both exports in
337 ``supported-languages.enum.ts`` and have a corresponding test in
338 ``language-selector.component.spec.ts``.
340 The ``SupportedLanguages`` enum will provide the list for the default language selection.
342 The ``languageBootstrapMapping`` variable will provide the
343 `language support <https://github.com/valor-software/ngx-bootstrap/tree/development/src/chronos/i18n>`_
344 for ngx-bootstrap components like the
345 `date picker <https://valor-software.com/ngx-bootstrap/#/datepicker#locales>`_.
350 To facilitate the translation process of the dashboard we are using a web tool
351 called `transifex <https://www.transifex.com/>`_.
353 If you wish to help translating to any language just go to our `transifex
354 project page <https://www.transifex.com/ceph/ceph-dashboard/>`_, join the
355 project and you can start translating immediately.
357 All translations will then be reviewed and later pushed upstream.
359 Updating translated messages
360 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
362 Any time there are new messages translated and reviewed in a specific language
363 we should update the translation file upstream.
365 To do that, we need to download the language xlf file from transifex and replace
366 the current one in the repository. Since Angular doesn't support missing
367 translations, we need to do an extra step and fill all the untranslated strings
368 with the source string.
370 Each language file should be placed in ``src/locale/messages.<locale-id>.xlf``.
371 For example, the path for german would be ``src/locale/messages.de-DE.xlf``.
372 ``<locale-id>`` should match the id previouisly inserted in
373 ``supported-languages.enum.ts``.
378 Strings need to start and end in the same line as the element:
388 <span i18n>Foo</span>
398 <span i18n>Foo bar baz.
401 Isolated interpolations should not be translated:
406 <span i18n>{{ foo }}</span>
409 <span>{{ foo }}</span>
411 Interpolations used in a sentence should be kept in the translation:
416 <span i18n>There are {{ x }} OSDs.</span>
418 Remove elements that are outside the context of the translation:
425 <span class="required"></span>
430 <ng-container i18n>Profile<ng-container>
431 <span class="required"></span>
434 Keep elements that affect the sentence:
439 <span i18n>Profile <b>foo</b> will be removed.</span>
444 The Python backend code of this module requires a number of Python modules to be
445 installed. They are listed in file ``requirements.txt``. Using `pip
446 <https://pypi.python.org/pypi/pip>`_ you may install all required dependencies
447 by issuing ``pip install -r requirements.txt`` in directory
448 ``src/pybind/mgr/dashboard``.
450 If you're using the `ceph-dev-docker development environment
451 <https://github.com/ricardoasmarques/ceph-dev-docker/>`_, simply run
452 ``./install_deps.sh`` from the toplevel directory to install them.
457 In dashboard we have two different kinds of backend tests:
459 1. Unit tests based on ``tox``
460 2. API tests based on Teuthology.
462 Unit tests based on tox
463 ~~~~~~~~~~~~~~~~~~~~~~~~
465 We included a ``tox`` configuration file that will run the unit tests under
466 Python 2 or 3, as well as linting tools to guarantee the uniformity of code.
468 You need to install ``tox`` and ``coverage`` before running it. To install the
469 packages in your system, either install it via your operating system's package
470 management tools, e.g. by running ``dnf install python-tox python-coverage`` on
473 Alternatively, you can use Python's native package installation method::
476 $ pip install coverage
478 To run the tests, run ``run-tox.sh`` in the dashboard directory (where
479 ``tox.ini`` is located)::
481 ## Run Python 2+3 tests+lint commands:
484 ## Run Python 3 tests+lint commands:
485 $ WITH_PYTHON2=OFF ./run-tox.sh
487 ## Run Python 3 arbitrary command (e.g. 1 single test):
488 $ WITH_PYTHON2=OFF ./run-tox.sh pytest tests/test_rgw_client.py::RgwClientTest::test_ssl_verify
490 You can also run tox instead of ``run-tox.sh``::
492 ## Run Python 3 tests command:
493 $ CEPH_BUILD_DIR=.tox tox -e py3-cov
495 ## Run Python 3 arbitrary command (e.g. 1 single test):
496 $ CEPH_BUILD_DIR=.tox tox -e py3-run pytest tests/test_rgw_client.py::RgwClientTest::test_ssl_verify
498 We also collect coverage information from the backend code when you run tests. You can check the
499 coverage information provided by the tox output, or by running the following
500 command after tox has finished successfully::
504 This command will create a directory ``htmlcov`` with an HTML representation of
505 the code coverage of the backend.
507 API tests based on Teuthology
508 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
510 How to run existing API tests:
511 To run the API tests against a real Ceph cluster, we leverage the Teuthology
512 framework. This has the advantage of catching bugs originated from changes in
513 the internal Ceph code.
515 Our ``run-backend-api-tests.sh`` script will start a ``vstart`` Ceph cluster
516 before running the Teuthology tests, and then it stops the cluster after the
517 tests are run. Of course this implies that you have built/compiled Ceph
520 Start all dashboard tests by running::
522 $ ./run-backend-api-tests.sh
524 Or, start one or multiple specific tests by specifying the test name::
526 $ ./run-backend-api-tests.sh tasks.mgr.dashboard.test_pool.PoolTest
528 Or, ``source`` the script and run the tests manually::
530 $ source run-backend-api-tests.sh
531 $ run_teuthology_tests [tests]...
534 How to write your own tests:
535 There are two possible ways to write your own API tests:
537 The first is by extending one of the existing test classes in the
538 ``qa/tasks/mgr/dashboard`` directory.
540 The second way is by adding your own API test module if you're creating a new
541 controller for example. To do so you'll just need to add the file containing
542 your new test class to the ``qa/tasks/mgr/dashboard`` directory and implement
545 .. note:: Don't forget to add the path of the newly created module to
546 ``modules`` section in ``qa/suites/rados/mgr/tasks/dashboard.yaml``.
548 Short example: Let's assume you created a new controller called
549 ``my_new_controller.py`` and the related test module
550 ``test_my_new_controller.py``. You'll need to add
551 ``tasks.mgr.dashboard.test_my_new_controller`` to the ``modules`` section in
552 the ``dashboard.yaml`` file.
554 Also, if you're removing test modules please keep in mind to remove the
555 related section. Otherwise the Teuthology test run will fail.
557 Please run your API tests on your dev environment (as explained above)
558 before submitting a pull request. Also make sure that a full QA run in
559 Teuthology/sepia lab (based on your changes) has completed successfully
560 before it gets merged. You don't need to schedule the QA run yourself, just
561 add the 'needs-qa' label to your pull request as soon as you think it's ready
562 for merging (e.g. make check was successful, the pull request is approved and
563 all comments have been addressed). One of the developers who has access to
564 Teuthology/the sepia lab will take care of it and report the result back to
568 How to add a new controller?
569 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
571 A controller is a Python class that extends from the ``BaseController`` class
572 and is decorated with either the ``@Controller``, ``@ApiController`` or
573 ``@UiApiController`` decorators. The Python class must be stored inside a Python
574 file located under the ``controllers`` directory. The Dashboard module will
575 automatically load your new controller upon start.
577 ``@ApiController`` and ``@UiApiController`` are both specializations of the
578 ``@Controller`` decorator.
580 The ``@ApiController`` should be used for controllers that provide an API-like
581 REST interface and the ``@UiApiController`` should be used for endpoints consumed
582 by the UI but that are not part of the 'public' API. For any other kinds of
583 controllers the ``@Controller`` decorator should be used.
585 A controller has a URL prefix path associated that is specified in the
586 controller decorator, and all endpoints exposed by the controller will share
587 the same URL prefix path.
589 A controller's endpoint is exposed by implementing a method on the controller
590 class decorated with the ``@Endpoint`` decorator.
592 For example create a file ``ping.py`` under ``controllers`` directory with the
595 .. code-block:: python
597 from ..tools import Controller, ApiController, UiApiController, BaseController, Endpoint
600 class Ping(BaseController):
603 return {'msg': "Hello"}
605 @ApiController('/ping')
606 class ApiPing(BaseController):
609 return {'msg': "Hello"}
611 @UiApiController('/ping')
612 class UiApiPing(BaseController):
615 return {'msg': "Hello"}
617 The ``hello`` endpoint of the ``Ping`` controller can be reached by the
618 following URL: https://mgr_hostname:8443/ping/hello using HTTP GET requests.
619 As you can see the controller URL path ``/ping`` is concatenated to the
620 method name ``hello`` to generate the endpoint's URL.
622 In the case of the ``ApiPing`` controller, the ``hello`` endpoint can be
623 reached by the following URL: https://mgr_hostname:8443/api/ping/hello using a
625 The API controller URL path ``/ping`` is prefixed by the ``/api`` path and then
626 concatenated to the method name ``hello`` to generate the endpoint's URL.
627 Internally, the ``@ApiController`` is actually calling the ``@Controller``
628 decorator by passing an additional decorator parameter called ``base_url``::
630 @ApiController('/ping') <=> @Controller('/ping', base_url="/api")
632 ``UiApiPing`` works in a similar way than the ``ApiPing``, but the URL will be
633 prefixed by ``/ui-api``: https://mgr_hostname:8443/ui-api/ping/hello. ``UiApiPing`` is
634 also a ``@Controller`` extension::
636 @UiApiController('/ping') <=> @Controller('/ping', base_url="/ui-api")
638 The ``@Endpoint`` decorator also supports many parameters to customize the
641 * ``method="GET"``: the HTTP method allowed to access this endpoint.
642 * ``path="/<method_name>"``: the URL path of the endpoint, excluding the
643 controller URL path prefix.
644 * ``path_params=[]``: list of method parameter names that correspond to URL
645 path parameters. Can only be used when ``method in ['POST', 'PUT']``.
646 * ``query_params=[]``: list of method parameter names that correspond to URL
648 * ``json_response=True``: indicates if the endpoint response should be
649 serialized in JSON format.
650 * ``proxy=False``: indicates if the endpoint should be used as a proxy.
652 An endpoint method may have parameters declared. Depending on the HTTP method
653 defined for the endpoint the method parameters might be considered either
654 path parameters, query parameters, or body parameters.
656 For ``GET`` and ``DELETE`` methods, the method's non-optional parameters are
657 considered path parameters by default. Optional parameters are considered
658 query parameters. By specifying the ``query_parameters`` in the endpoint
659 decorator it is possible to make a non-optional parameter to be a query
662 For ``POST`` and ``PUT`` methods, all method parameters are considered
663 body parameters by default. To override this default, one can use the
664 ``path_params`` and ``query_params`` to specify which method parameters are
665 path and query parameters respectivelly.
666 Body parameters are decoded from the request body, either from a form format, or
667 from a dictionary in JSON format.
669 Let's use an example to better understand the possible ways to customize an
672 .. code-block:: python
674 from ..tools import Controller, BaseController, Endpoint
677 class Ping(BaseController):
679 # URL: /ping/{key}?opt1=...&opt2=...
680 @Endpoint(path="/", query_params=['opt1'])
681 def index(self, key, opt1, opt2=None):
684 # URL: /ping/{key}?opt1=...&opt2=...
685 @Endpoint(query_params=['opt1'])
686 def __call__(self, key, opt1, opt2=None):
689 # URL: /ping/post/{key1}/{key2}
690 @Endpoint('POST', path_params=['key1', 'key2'])
691 def post(self, key1, key2, data1, data2=None):
695 In the above example we see how the ``path`` option can be used to override the
696 generated endpoint URL in order to not use the method's name in the URL. In the
697 ``index`` method we set the ``path`` to ``"/"`` to generate an endpoint that is
698 accessible by the root URL of the controller.
700 An alternative approach to generate an endpoint that is accessible through just
701 the controller's path URL is by using the ``__call__`` method, as we show in
704 From the third method you can see that the path parameters are collected from
705 the URL by parsing the list of values separated by slashes ``/`` that come
706 after the URL path ``/ping`` for ``index`` method case, and ``/ping/post`` for
707 the ``post`` method case.
709 Defining path parameters in endpoints's URLs using python methods's parameters
710 is very easy but it is still a bit strict with respect to the position of these
711 parameters in the URL structure.
712 Sometimes we may want to explicitly define a URL scheme that
713 contains path parameters mixed with static parts of the URL.
714 Our controller infrastructure also supports the declaration of URL paths with
715 explicit path parameters at both the controller level and method level.
717 Consider the following example:
719 .. code-block:: python
721 from ..tools import Controller, BaseController, Endpoint
723 @Controller('/ping/{node}/stats')
724 class Ping(BaseController):
726 # URL: /ping/{node}/stats/{date}/latency?unit=...
727 @Endpoint(path="/{date}/latency")
728 def latency(self, node, date, unit="ms"):
731 In this example we explicitly declare a path parameter ``{node}`` in the
732 controller URL path, and a path parameter ``{date}`` in the ``latency``
733 method. The endpoint for the ``latency`` method is then accessible through
734 the URL: https://mgr_hostname:8443/ping/{node}/stats/{date}/latency .
736 For a full set of examples on how to use the ``@Endpoint``
737 decorator please check the unit test file: ``tests/test_controllers.py``.
738 There you will find many examples of how to customize endpoint methods.
741 Implementing Proxy Controller
742 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
744 Sometimes you might need to relay some requests from the Dashboard frontend
745 directly to an external service.
746 For that purpose we provide a decorator called ``@Proxy``.
747 (As a concrete example, check the ``controllers/rgw.py`` file where we
748 implemented an RGW Admin Ops proxy.)
751 The ``@Proxy`` decorator is a wrapper of the ``@Endpoint`` decorator that
752 already customizes the endpoint for working as a proxy.
753 A proxy endpoint works by capturing the URL path that follows the controller
754 URL prefix path, and does not do any decoding of the request body.
758 .. code-block:: python
760 from ..tools import Controller, BaseController, Proxy
762 @Controller('/foo/proxy')
763 class FooServiceProxy(BaseController):
766 def proxy(self, path, **params):
767 # if requested URL is "/foo/proxy/access/service?opt=1"
768 # then path is "access/service" and params is {'opt': '1'}
772 How does the RESTController work?
773 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
775 We also provide a simple mechanism to create REST based controllers using the
776 ``RESTController`` class. Any class which inherits from ``RESTController`` will,
777 by default, return JSON.
779 The ``RESTController`` is basically an additional abstraction layer which eases
780 and unifies the work with collections. A collection is just an array of objects
781 with a specific type. ``RESTController`` enables some default mappings of
782 request types and given parameters to specific method names. This may sound
783 complicated at first, but it's fairly easy. Lets have look at the following
786 .. code-block:: python
789 from ..tools import ApiController, RESTController
791 @ApiController('ping')
792 class Ping(RESTController):
794 return {"msg": "Hello"}
797 return self.objects[id]
799 In this case, the ``list`` method is automatically used for all requests to
800 ``api/ping`` where no additional argument is given and where the request type
801 is ``GET``. If the request is given an additional argument, the ID in our
802 case, it won't map to ``list`` anymore but to ``get`` and return the element
803 with the given ID (assuming that ``self.objects`` has been filled before). The
804 same applies to other request types:
806 +--------------+------------+----------------+-------------+
807 | Request type | Arguments | Method | Status Code |
808 +==============+============+================+=============+
809 | GET | No | list | 200 |
810 +--------------+------------+----------------+-------------+
811 | PUT | No | bulk_set | 200 |
812 +--------------+------------+----------------+-------------+
813 | POST | No | create | 201 |
814 +--------------+------------+----------------+-------------+
815 | DELETE | No | bulk_delete | 204 |
816 +--------------+------------+----------------+-------------+
817 | GET | Yes | get | 200 |
818 +--------------+------------+----------------+-------------+
819 | PUT | Yes | set | 200 |
820 +--------------+------------+----------------+-------------+
821 | DELETE | Yes | delete | 204 |
822 +--------------+------------+----------------+-------------+
824 How to use a custom API endpoint in a RESTController?
825 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
827 If you don't have any access restriction you can use ``@Endpoint``. If you
828 have set a permission scope to restrict access to your endpoints,
829 ``@Endpoint`` will fail, as it doesn't know which permission property should be
830 used. To use a custom endpoint inside a restricted ``RESTController`` use
831 ``@RESTController.Collection`` instead. You can also choose
832 ``@RESTController.Resource`` if you have set a ``RESOURCE_ID`` in your
833 ``RESTController`` class.
835 .. code-block:: python
838 from ..tools import ApiController, RESTController
840 @ApiController('ping', Scope.Ping)
841 class Ping(RESTController):
844 @RESTController.Resource('GET')
845 def some_get_endpoint(self):
846 return {"msg": "Hello"}
848 @RESTController.Collection('POST')
849 def some_post_endpoint(self, **data):
852 Both decorators also support four parameters to customize the
855 * ``method="GET"``: the HTTP method allowed to access this endpoint.
856 * ``path="/<method_name>"``: the URL path of the endpoint, excluding the
857 controller URL path prefix.
858 * ``status=200``: set the HTTP status response code
859 * ``query_params=[]``: list of method parameter names that correspond to URL
862 How to restrict access to a controller?
863 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
865 All controllers require authentication by default.
866 If you require that the controller can be accessed without authentication,
867 then you can add the parameter ``secure=False`` to the controller decorator.
871 .. code-block:: python
874 from . import ApiController, RESTController
877 @ApiController('ping', secure=False)
878 class Ping(RESTController):
880 return {"msg": "Hello"}
883 How to access the manager module instance from a controller?
884 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
886 We provide the manager module instance as a global variable that can be
887 imported in any module. We also provide a logger instance in the same way.
891 .. code-block:: python
894 from .. import logger, mgr
895 from ..tools import ApiController, RESTController
898 @ApiController('servers')
899 class Servers(RESTController):
901 logger.debug('Listing available servers')
902 return {'servers': mgr.list_servers()}
905 How to write a unit test for a controller?
906 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
908 We provide a test helper class called ``ControllerTestCase`` to easily create
909 unit tests for your controller.
911 If we want to write a unit test for the above ``Ping`` controller, create a
912 ``test_ping.py`` file under the ``tests`` directory with the following code:
914 .. code-block:: python
916 from .helper import ControllerTestCase
917 from .controllers.ping import Ping
920 class PingTest(ControllerTestCase):
923 Ping._cp_config['tools.authenticate.on'] = False
924 cls.setup_controllers([Ping])
927 self._get("/api/ping")
928 self.assertStatus(200)
929 self.assertJsonBody({'msg': 'Hello'})
931 The ``ControllerTestCase`` class starts by initializing a CherryPy webserver.
932 Then it will call the ``setup_test()`` class method where we can explicitly
933 load the controllers that we want to test. In the above example we are only
934 loading the ``Ping`` controller. We can also disable authentication of a
935 controller at this stage, as depicted in the example.
938 How to listen for manager notifications in a controller?
939 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
941 The manager notifies the modules of several types of cluster events, such
942 as cluster logging event, etc...
944 Each module has a "global" handler function called ``notify`` that the manager
945 calls to notify the module. But this handler function must not block or spend
946 too much time processing the event notification.
947 For this reason we provide a notification queue that controllers can register
948 themselves with to receive cluster notifications.
950 The example below represents a controller that implements a very simple live
953 .. code-block:: python
955 from __future__ import absolute_import
961 from ..tools import ApiController, BaseController, NotificationQueue
964 @ApiController('livelog')
965 class LiveLog(BaseController):
966 log_buffer = collections.deque(maxlen=1000)
969 super(LiveLog, self).__init__()
970 NotificationQueue.register(self.log, 'clog')
972 def log(self, log_struct):
973 self.log_buffer.appendleft(log_struct)
977 ret = '<html><meta http-equiv="refresh" content="2" /><body>'
978 for l in self.log_buffer:
979 ret += "{}<br>".format(l)
980 ret += "</body></html>"
983 As you can see above, the ``NotificationQueue`` class provides a register
984 method that receives the function as its first argument, and receives the
985 "notification type" as the second argument.
986 You can omit the second argument of the ``register`` method, and in that case
987 you are registering to listen all notifications of any type.
989 Here is an list of notification types (these might change in the future) that
992 * ``clog``: cluster log notifications
993 * ``command``: notification when a command issued by ``MgrModule.send_command``
995 * ``perf_schema_update``: perf counters schema update
996 * ``mon_map``: monitor map update
997 * ``fs_map``: cephfs map update
998 * ``osd_map``: OSD map update
999 * ``service_map``: services (RGW, RBD-Mirror, etc.) map update
1000 * ``mon_status``: monitor status regular update
1001 * ``health``: health status regular update
1002 * ``pg_summary``: regular update of PG status information
1005 How to write a unit test when a controller accesses a Ceph module?
1006 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1008 Consider the following example that implements a controller that retrieves the
1009 list of RBD images of the ``rbd`` pool:
1011 .. code-block:: python
1015 from ..tools import ApiController, RESTController
1018 @ApiController('rbdimages')
1019 class RbdImages(RESTController):
1021 self.ioctx = mgr.rados.open_ioctx('rbd')
1022 self.rbd = rbd.RBD()
1025 return [{'name': n} for n in self.rbd.list(self.ioctx)]
1027 In the example above, we want to mock the return value of the ``rbd.list``
1028 function, so that we can test the JSON response of the controller.
1030 The unit test code will look like the following:
1032 .. code-block:: python
1035 from .helper import ControllerTestCase
1038 class RbdImagesTest(ControllerTestCase):
1039 @mock.patch('rbd.RBD.list')
1040 def test_list(self, rbd_list_mock):
1041 rbd_list_mock.return_value = ['img1', 'img2']
1042 self._get('/api/rbdimages')
1043 self.assertJsonBody([{'name': 'img1'}, {'name': 'img2'}])
1047 How to add a new configuration setting?
1048 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1050 If you need to store some configuration setting for a new feature, we already
1051 provide an easy mechanism for you to specify/use the new config setting.
1053 For instance, if you want to add a new configuration setting to hold the
1054 email address of the dashboard admin, just add a setting name as a class
1055 attribute to the ``Options`` class in the ``settings.py`` file::
1058 class Options(object):
1061 ADMIN_EMAIL_ADDRESS = ('admin@admin.com', str)
1063 The value of the class attribute is a pair composed by the default value for that
1064 setting, and the python type of the value.
1066 By declaring the ``ADMIN_EMAIL_ADDRESS`` class attribute, when you restart the
1067 dashboard module, you will automatically gain two additional CLI commands to
1068 get and set that setting::
1070 $ ceph dashboard get-admin-email-address
1071 $ ceph dashboard set-admin-email-address <value>
1073 To access, or modify the config setting value from your Python code, either
1074 inside a controller or anywhere else, you just need to import the ``Settings``
1075 class and access it like this:
1077 .. code-block:: python
1079 from settings import Settings
1082 tmp_var = Settings.ADMIN_EMAIL_ADDRESS
1085 Settings.ADMIN_EMAIL_ADDRESS = 'myemail@admin.com'
1087 The settings management implementation will make sure that if you change a
1088 setting value from the Python code you will see that change when accessing
1089 that setting from the CLI and vice-versa.
1092 How to run a controller read-write operation asynchronously?
1093 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1095 Some controllers might need to execute operations that alter the state of the
1096 Ceph cluster. These operations might take some time to execute and to maintain
1097 a good user experience in the Web UI, we need to run those operations
1098 asynchronously and return immediately to frontend some information that the
1099 operations are running in the background.
1101 To help in the development of the above scenario we added the support for
1102 asynchronous tasks. To trigger the execution of an asynchronous task we must
1103 use the following class method of the ``TaskManager`` class::
1105 from ..tools import TaskManager
1107 TaskManager.run(name, metadata, func, args, kwargs)
1109 * ``name`` is a string that can be used to group tasks. For instance
1110 for RBD image creation tasks we could specify ``"rbd/create"`` as the
1111 name, or similarly ``"rbd/remove"`` for RBD image removal tasks.
1113 * ``metadata`` is a dictionary where we can store key-value pairs that
1114 characterize the task. For instance, when creating a task for creating
1115 RBD images we can specify the metadata argument as
1116 ``{'pool_name': "rbd", image_name': "test-img"}``.
1118 * ``func`` is the python function that implements the operation code, which
1119 will be executed asynchronously.
1121 * ``args`` and ``kwargs`` are the positional and named arguments that will be
1122 passed to ``func`` when the task manager starts its execution.
1124 The ``TaskManager.run`` method triggers the asynchronous execution of function
1125 ``func`` and returns a ``Task`` object.
1126 The ``Task`` provides the public method ``Task.wait(timeout)``, which can be
1127 used to wait for the task to complete up to a timeout defined in seconds and
1128 provided as an argument. If no argument is provided the ``wait`` method
1129 blocks until the task is finished.
1131 The ``Task.wait`` is very useful for tasks that usually are fast to execute but
1132 that sometimes may take a long time to run.
1133 The return value of the ``Task.wait`` method is a pair ``(state, value)``
1134 where ``state`` is a string with following possible values:
1136 * ``VALUE_DONE = "done"``
1137 * ``VALUE_EXECUTING = "executing"``
1139 The ``value`` will store the result of the execution of function ``func`` if
1140 ``state == VALUE_DONE``. If ``state == VALUE_EXECUTING`` then
1143 The pair ``(name, metadata)`` should unequivocally identify the task being
1144 run, which means that if you try to trigger a new task that matches the same
1145 ``(name, metadata)`` pair of the currently running task, then the new task
1146 is not created and you get the task object of the current running task.
1148 For instance, consider the following example:
1150 .. code-block:: python
1152 task1 = TaskManager.run("dummy/task", {'attr': 2}, func)
1153 task2 = TaskManager.run("dummy/task", {'attr': 2}, func)
1155 If the second call to ``TaskManager.run`` executes while the first task is
1156 still executing then it will return the same task object:
1157 ``assert task1 == task2``.
1160 How to get the list of executing and finished asynchronous tasks?
1161 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1163 The list of executing and finished tasks is included in the ``Summary``
1164 controller, which is already polled every 5 seconds by the dashboard frontend.
1165 But we also provide a dedicated controller to get the same list of executing
1168 The ``Task`` controller exposes the ``/api/task`` endpoint that returns the
1169 list of executing and finished tasks. This endpoint accepts the ``name``
1170 parameter that accepts a glob expression as its value.
1171 For instance, an HTTP GET request of the URL ``/api/task?name=rbd/*``
1172 will return all executing and finished tasks which name starts with ``rbd/``.
1174 To prevent the finished tasks list from growing unbounded, we will always
1175 maintain the 10 most recent finished tasks, and the remaining older finished
1176 tasks will be removed when reaching a TTL of 1 minute. The TTL is calculated
1177 using the timestamp when the task finished its execution. After a minute, when
1178 the finished task information is retrieved, either by the summary controller or
1179 by the task controller, it is automatically deleted from the list and it will
1180 not be included in further task queries.
1182 Each executing task is represented by the following dictionary::
1185 'name': "name", # str
1186 'metadata': { }, # dict
1187 'begin_time': "2018-03-14T15:31:38.423605Z", # str (ISO 8601 format)
1188 'progress': 0 # int (percentage)
1191 Each finished task is represented by the following dictionary::
1194 'name': "name", # str
1195 'metadata': { }, # dict
1196 'begin_time': "2018-03-14T15:31:38.423605Z", # str (ISO 8601 format)
1197 'end_time': "2018-03-14T15:31:39.423605Z", # str (ISO 8601 format)
1198 'duration': 0.0, # float
1199 'progress': 0 # int (percentage)
1200 'success': True, # bool
1201 'ret_value': None, # object, populated only if 'success' == True
1202 'exception': None, # str, populated only if 'success' == False
1206 How to use asynchronous APIs with asynchronous tasks?
1207 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1209 The ``TaskManager.run`` method as described in a previous section, is well
1210 suited for calling blocking functions, as it runs the function inside a newly
1211 created thread. But sometimes we want to call some function of an API that is
1212 already asynchronous by nature.
1214 For these cases we want to avoid creating a new thread for just running a
1215 non-blocking function, and want to leverage the asynchronous nature of the
1216 function. The ``TaskManager.run`` is already prepared to be used with
1217 non-blocking functions by passing an object of the type ``TaskExecutor`` as an
1218 additional parameter called ``executor``. The full method signature of
1219 ``TaskManager.run``::
1221 TaskManager.run(name, metadata, func, args=None, kwargs=None, executor=None)
1224 The ``TaskExecutor`` class is responsible for code that executes a given task
1225 function, and defines three methods that can be overridden by
1228 def init(self, task)
1230 def finish(self, ret_value, exception)
1232 The ``init`` method is called before the running the task function, and
1233 receives the task object (of class ``Task``).
1235 The ``start`` method runs the task function. The default implementation is to
1236 run the task function in the current thread context.
1238 The ``finish`` method should be called when the task function finishes with
1239 either the ``ret_value`` populated with the result of the execution, or with
1240 an exception object in the case that execution raised an exception.
1242 To leverage the asynchronous nature of a non-blocking function, the developer
1243 should implement a custom executor by creating a subclass of the
1244 ``TaskExecutor`` class, and provide an instance of the custom executor class
1245 as the ``executor`` parameter of the ``TaskManager.run``.
1247 To better understand the expressive power of executors, we write a full example
1248 of use a custom executor to execute the ``MgrModule.send_command`` asynchronous
1251 .. code-block:: python
1254 from mgr_module import CommandResult
1256 from ..tools import ApiController, RESTController, NotificationQueue, \
1257 TaskManager, TaskExecutor
1260 class SendCommandExecutor(TaskExecutor):
1262 super(SendCommandExecutor, self).__init__()
1266 def init(self, task):
1267 super(SendCommandExecutor, self).init(task)
1269 # we need to listen for 'command' events to know when the command
1271 NotificationQueue.register(self._handler, 'command')
1273 # store the CommandResult object to retrieve the results
1274 self.result = self.task.fn_args[0]
1275 if len(self.task.fn_args) > 4:
1276 # the user specified a tag for the command, so let's use it
1277 self.tag = self.task.fn_args[4]
1279 # let's generate a unique tag for the command
1280 self.tag = 'send_command_{}'.format(id(self))
1281 self.task.fn_args.append(self.tag)
1283 def _handler(self, data):
1284 if data == self.tag:
1285 # the command has finished, notifying the task with the result
1286 self.finish(self.result.wait(), None)
1287 # deregister listener to avoid memory leaks
1288 NotificationQueue.deregister(self._handler, 'command')
1291 @ApiController('test')
1292 class Test(RESTController):
1294 def _run_task(self, osd_id):
1295 task = TaskManager.run("test/task", {}, mgr.send_command,
1296 [CommandResult(''), 'osd', osd_id,
1297 json.dumps({'prefix': 'perf histogram dump'})],
1298 executor=SendCommandExecutor())
1299 return task.wait(1.0)
1301 def get(self, osd_id):
1302 status, value = self._run_task(osd_id)
1303 return {'status': status, 'value': value}
1306 The above ``SendCommandExecutor`` executor class can be used for any call to
1307 ``MgrModule.send_command``. This means that we should need just one custom
1308 executor class implementation for each non-blocking API that we use in our
1311 The default executor, used when no executor object is passed to
1312 ``TaskManager.run``, is the ``ThreadedExecutor``. You can check its
1313 implementation in the ``tools.py`` file.
1316 How to update the execution progress of an asynchronous task?
1317 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1319 The asynchronous tasks infrastructure provides support for updating the
1320 execution progress of an executing task.
1321 The progress can be updated from within the code the task is executing, which
1322 usually is the place where we have the progress information available.
1324 To update the progress from within the task code, the ``TaskManager`` class
1325 provides a method to retrieve the current task object::
1327 TaskManager.current_task()
1329 The above method is only available when using the default executor
1330 ``ThreadedExecutor`` for executing the task.
1331 The ``current_task()`` method returns the current ``Task`` object. The
1332 ``Task`` object provides two public methods to update the execution progress
1333 value: the ``set_progress(percentage)``, and the ``inc_progress(delta)``
1336 The ``set_progress`` method receives as argument an integer value representing
1337 the absolute percentage that we want to set to the task.
1339 The ``inc_progress`` method receives as argument an integer value representing
1340 the delta we want to increment to the current execution progress percentage.
1342 Take the following example of a controller that triggers a new task and
1343 updates its progress:
1345 .. code-block:: python
1347 from __future__ import absolute_import
1351 from ..tools import TaskManager, ApiController, BaseController
1354 @ApiController('dummy_task')
1355 class DummyTask(BaseController):
1357 top = random.randrange(100)
1358 for i in range(top):
1359 TaskManager.current_task().set_progress(i*100/top)
1360 # or TaskManager.current_task().inc_progress(100/top)
1365 @cherrypy.tools.json_out()
1367 task = TaskManager.run("dummy/task", {}, self._dummy)
1368 return task.wait(5) # wait for five seconds
1371 How to deal with asynchronous tasks in the front-end?
1372 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1374 All executing and most recently finished asynchronous tasks are displayed on
1375 "Background-Tasks" and if finished on "Recent-Notifications" in the menu bar.
1376 For each task a operation name for three states (running, success and failure),
1377 a function that tells who is involved and error descriptions, if any, have to
1378 be provided. This can be achieved by appending
1379 ``TaskManagerMessageService.messages``. This has to be done to achieve
1380 consistency among all tasks and states.
1383 Ensures consistency among all tasks. It consists of three verbs for each
1384 different state f.e.
1385 ``{running: 'Creating', failure: 'create', success: 'Created'}``.
1387 #. Put running operations in present participle f.e. ``'Updating'``.
1388 #. Failed messages always start with ``'Failed to '`` and should be continued
1389 with the operation in present tense f.e. ``'update'``.
1390 #. Put successful operations in past tense f.e. ``'Updated'``.
1393 Ensures consistency among all messages of a task, it resembles who's
1394 involved by the operation. It's a function that returns a string which
1395 takes the metadata from the task to return f.e.
1396 ``"RBD 'somePool/someImage'"``.
1398 Both combined create the following messages:
1400 * Failure => ``"Failed to create RBD 'somePool/someImage'"``
1401 * Running => ``"Creating RBD 'somePool/someImage'"``
1402 * Success => ``"Created RBD 'somePool/someImage'"``
1404 For automatic task handling use ``TaskWrapperService.wrapTaskAroundCall``.
1406 If for some reason ``wrapTaskAroundCall`` is not working for you,
1407 you have to subscribe to your asynchronous task manually through
1408 ``TaskManagerService.subscribe``, and provide it with a callback,
1409 in case of a success to notify the user. A notification can
1410 be triggered with ``NotificationService.notifyTask``. It will use
1411 ``TaskManagerMessageService.messages`` to display a message based on the state
1414 Notifications of API errors are handled by ``ApiInterceptorService``.
1418 .. code-block:: javascript
1420 export class TaskManagerMessageService {
1423 // Messages for task 'rbd/create'
1424 'rbd/create': new TaskManagerMessage(
1426 ['create', 'Creating', 'Created'],
1428 (metadata) => `RBD '${metadata.pool_name}/${metadata.image_name}'`,
1430 // Error code and description
1431 '17': `Name is already used by RBD '${metadata.pool_name}/${
1432 metadata.image_name}'.`
1440 export class RBDFormComponent {
1443 const request = this.createRequest();
1444 // Subscribes to 'call' with submitted 'task' and handles notifications
1445 return this.taskWrapper.wrapTaskAroundCall({
1446 task: new FinishedTask('rbd/create', {
1447 pool_name: request.pool_name,
1448 image_name: request.name
1450 call: this.rbdService.create(request)
1457 REST API documentation
1458 ~~~~~~~~~~~~~~~~~~~~~~
1459 There is an automatically generated Swagger UI page for documentation of the REST
1460 API endpoints.However, by default it is not very detailed. There are two
1461 decorators that can be used to add more information:
1463 * ``@EndpointDoc()`` for documentation of endpoints. It has four optional arguments
1464 (explained below): ``description``, ``group``, ``parameters`` and``responses``.
1465 * ``@ControllerDoc()`` for documentation of controller or group associated with
1466 the endpoints. It only takes the two first arguments: ``description`` and``group``.
1469 ``description``: A a string with a short (1-2 sentences) description of the object.
1472 ``group``: By default, an endpoint is grouped together with other endpoints
1473 within the same controller class. ``group`` is a string that can be used to
1474 assign an endpoint or all endpoints in a class to another controller or a
1475 conceived group name.
1478 ``parameters``: A dict used to describe path, query or request body parameters.
1479 By default, all parameters for an endpoint are listed on the Swagger UI page,
1480 including information of whether the parameter is optional/required and default
1481 values. However, there will be no description of the parameter and the parameter
1482 type will only be displayed in some cases.
1483 When adding information, each parameters should be described as in the example
1484 below. Note that the parameter type should be expressed as a built-in python
1485 type and not as a string. Allowed values are ``str``, ``int``, ``bool``, ``float``.
1487 .. code-block:: python
1489 @EndpointDoc(parameters={'my_string': (str, 'Description of my_string')})
1491 For body parameters, more complex cases are possible. If the parameter is a
1492 dictionary, the type should be replaced with a ``dict`` containing its nested
1493 parameters. When describing nested parameters, the same format as other
1494 parameters is used. However, all nested parameters are set as required by default.
1495 If the nested parameter is optional this must be specified as for ``item2`` in
1496 the example below. If a nested parameters is set to optional, it is also
1497 possible to specify the default value (this will not be provided automatically
1498 for nested parameters).
1500 .. code-block:: python
1502 @EndpointDoc(parameters={
1504 'item1': (str, 'Description of item1'),
1505 'item2': (str, 'Description of item2', True), # item2 is optional
1506 'item3': (str, 'Description of item3', True, 'foo'), # item3 is optional with 'foo' as default value
1507 }, 'Description of my_dictionary')})
1509 If the parameter is a ``list`` of primitive types, the type should be
1510 surrounded with square brackets.
1512 .. code-block:: python
1514 @EndpointDoc(parameters={'my_list': ([int], 'Description of my_list')})
1516 If the parameter is a ``list`` with nested parameters, the nested parameters
1517 should be placed in a dictionary and surrounded with square brackets.
1519 .. code-block:: python
1521 @EndpointDoc(parameters={
1523 'list_item': (str, 'Description of list_item'),
1524 'list_item2': (str, 'Description of list_item2')
1525 }], 'Description of my_list')})
1528 ``responses``: A dict used for describing responses. Rules for describing
1529 responses are the same as for request body parameters, with one difference:
1530 responses also needs to be assigned to the related response code as in the
1533 .. code-block:: python
1535 @EndpointDoc(responses={
1536 '400':{'my_response': (str, 'Description of my_response')}
1539 Error Handling in Python
1540 ~~~~~~~~~~~~~~~~~~~~~~~~
1542 Good error handling is a key requirement in creating a good user experience
1543 and providing a good API.
1545 Dashboard code should not duplicate C++ code. Thus, if error handling in C++
1546 is sufficient to provide good feedback, a new wrapper to catch these errors
1547 is not necessary. On the other hand, input validation is the best place to
1548 catch errors and generate the best error messages. If required, generate
1549 errors as soon as possible.
1551 The backend provides few standard ways of returning errors.
1553 First, there is a generic Internal Server Error::
1557 "version": <cherrypy version, e.g. 13.1.0>,
1558 "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.",
1562 For errors generated by the backend, we provide a standard error
1567 "detail": str(e), # E.g. "[errno -42] <some error message>"
1568 "component": "rbd", # this can be null to represent a global error code
1569 "code": "3", # Or a error name, e.g. "code": "some_error_key"
1573 In case, the API Endpoints uses @ViewCache to temporarily cache results,
1574 the error looks like so::
1578 "detail": str(e), # E.g. "[errno -42] <some error message>"
1579 "component": "rbd", # this can be null to represent a global error code
1580 "code": "3", # Or a error name, e.g. "code": "some_error_key"
1581 'status': 3, # Indicating the @ViewCache error status
1584 In case, the API Endpoints uses a task the error looks like so::
1588 "detail": str(e), # E.g. "[errno -42] <some error message>"
1589 "component": "rbd", # this can be null to represent a global error code
1590 "code": "3", # Or a error name, e.g. "code": "some_error_key"
1591 "task": { # Information about the task itself
1598 Our WebUI should show errors generated by the API to the user. Especially
1599 field-related errors in wizards and dialogs or show non-intrusive notifications.
1601 Handling exceptions in Python should be an exception. In general, we
1602 should have few exception handlers in our project. Per default, propagate
1603 errors to the API, as it will take care of all exceptions anyway. In general,
1604 log the exception by adding ``logger.exception()`` with a description to the
1607 We need to distinguish between user errors from internal errors and
1608 programming errors. Using different exception types will ease the
1609 task for the API layer and for the user interface:
1611 Standard Python errors, like ``SystemError``, ``ValueError`` or ``KeyError``
1612 will end up as internal server errors in the API.
1614 In general, do not ``return`` error responses in the REST API. They will be
1615 returned by the error handler. Instead, raise the appropriate exception.
1620 New functionality can be provided by means of a plug-in architecture. Among the
1621 benefits this approach brings in, loosely coupled development is one of the most
1622 notable. As the Ceph Dashboard grows in feature richness, its code-base becomes
1623 more and more complex. The hook-based nature of a plug-in architecture allows to
1624 extend functionality in a controlled manner, and isolate the scope of the
1627 Ceph Dashboard relies on `Pluggy <https://pluggy.readthedocs.io>`_ to provide
1628 for plug-ing support. On top of pluggy, an interface-based approach has been
1629 implemented, with some safety checks (method override and abstract method
1632 In order to create a new plugin, the following steps are required:
1634 #. Add a new file under ``src/pybind/mgr/dashboard/plugins``.
1635 #. Import the ``PLUGIN_MANAGER`` instance and the ``Interfaces``.
1636 #. Create a class extending the desired interfaces. The plug-in library will check if all the methods of the interfaces have been properly overridden.
1637 #. Register the plugin in the ``PLUGIN_MANAGER`` instance.
1638 #. Import the plug-in from within the Ceph Dashboard ``module.py`` (currently no dynamic loading is implemented).
1640 The available interfaces are the following:
1642 - ``CanMgr``: provides the plug-in with access to the ``mgr`` instance under ``self.mgr``.
1643 - ``CanLog``: provides the plug-in with access to the Ceph Dashboard logger under ``self.log``.
1644 - ``Setupable``: requires overriding ``setup()`` hook. This method is run in the Ceph Dashboard ``serve()`` method, right after CherryPy has been configured, but before it is started. It's a placeholder for the plug-in initialization logic.
1645 - ``HasOptions``: requires overriding ``get_options()`` hook by returning a list of ``Options()``. The options returned here are added to the ``MODULE_OPTIONS``.
1646 - ``HasCommands``: requires overriding ``register_commands()`` hook by defining the commands the plug-in can handle and decorating them with ``@CLICommand`. The commands can be optionally returned, so that they can be invoked externally (which makes unit testing easier).
1647 - ``HasControllers``: requires overriding ``get_controllers()`` hook by defining and returning the controllers as usual.
1648 - ``FilterRequest.BeforeHandler``: requires overriding ``filter_request_before_handler()`` hook. This method receives a ``cherrypy.request`` object for processing. A usual implementation of this method will allow some requests to pass or will raise a ``cherrypy.HTTPError` based on the ``request`` metadata and other conditions.
1650 New interfaces and hooks should be added as soon as they are required to
1651 implement new functionality. The above list only comprises the hooks needed for
1652 the existing plugins.
1654 A sample plugin implementation would look like this:
1656 .. code-block:: python
1658 # src/pybind/mgr/dashboard/plugins/mute.py
1660 from . import PLUGIN_MANAGER as PM
1661 from . import interfaces as I
1663 from mgr_module import CLICommand, Option
1667 class Mute(I.CanMgr, I.CanLog, I.Setupable, I.HasOptions,
1668 I.HasCommands, I.FilterRequest.BeforeHandler,
1671 def get_options(self):
1672 return [Option('mute', default=False, type='bool')]
1676 self.mute = self.mgr.get_module_options('mute')
1679 def register_commands(self):
1680 @CLICommand("dashboard mute")
1683 self.mgr.set_module_options('mute', True)
1687 def filter_request_before_handler(self, request):
1689 raise cherrypy.HTTPError(500, "I'm muted :-x")
1692 def get_controllers(self):
1693 from ..controllers import ApiController, RESTController
1695 @ApiController('/mute')
1696 class MuteController(RESTController):
1700 return [FeatureTogglesEndpoint]