]>
Commit | Line | Data |
---|---|---|
9f95a23c TL |
1 | Testing - unit tests |
2 | ==================== | |
3 | ||
f67539c2 TL |
4 | The Ceph GitHub repository has two types of tests: unit tests (also called |
5 | ``make check`` tests) and integration tests. Strictly speaking, the | |
6 | ``make check`` tests are not "unit tests", but rather tests that can be run | |
7 | easily on a single build machine after compiling Ceph from source, whereas | |
8 | integration tests require package installation and multi-machine clusters to | |
9 | run. | |
10 | ||
11 | .. _make-check: | |
9f95a23c TL |
12 | |
13 | What does "make check" mean? | |
14 | ---------------------------- | |
15 | ||
f67539c2 TL |
16 | After compiling Ceph, the code can be run through a battery of tests. For |
17 | historical reasons, this is often referred to as ``make check`` even though | |
18 | the actual command used to run the tests is now ``ctest``. To be included in | |
19 | this group of tests, a test must: | |
9f95a23c TL |
20 | |
21 | * bind ports that do not conflict with other tests | |
22 | * not require root access | |
23 | * not require more than one machine to run | |
24 | * complete within a few minutes | |
25 | ||
f67539c2 TL |
26 | For the sake of simplicity, this class of tests is referred to as "make |
27 | check tests" or "unit tests". This is meant to distinguish these tests from | |
28 | the more complex "integration tests" that are run via the `teuthology | |
29 | framework`_. | |
9f95a23c TL |
30 | |
31 | While it is possible to run ``ctest`` directly, it can be tricky to correctly | |
f67539c2 TL |
32 | set up your environment for it. Fortunately, there is a script that makes it |
33 | easy to run the unit tests on your code. This script can be run from the | |
34 | top-level directory of the Ceph source tree by invoking: | |
35 | ||
36 | .. prompt:: bash $ | |
9f95a23c | 37 | |
f67539c2 | 38 | ./run-make-check.sh |
9f95a23c | 39 | |
f67539c2 TL |
40 | You will need a minimum of 8GB of RAM and 32GB of free drive space for this |
41 | command to complete successfully on x86_64 architectures; other architectures | |
42 | may have different requirements. Depending on your hardware, it can take from | |
43 | twenty minutes to three hours to complete. | |
9f95a23c | 44 | |
9f95a23c TL |
45 | |
46 | How unit tests are declared | |
47 | --------------------------- | |
48 | ||
f67539c2 TL |
49 | Unit tests are declared in the ``CMakeLists.txt`` file, which is found in the |
50 | ``./src`` directory. The ``add_ceph_test`` and ``add_ceph_unittest`` CMake | |
51 | functions are used to declare unit tests. ``add_ceph_test`` and | |
52 | ``add_ceph_unittest`` are themselves defined in | |
53 | ``./cmake/modules/AddCephTest.cmake``. | |
54 | ||
55 | Some unit tests are scripts and other unit tests are binaries that are | |
56 | compiled during the build process. | |
57 | ||
58 | * ``add_ceph_test`` function - used to declare unit test scripts | |
59 | * ``add_ceph_unittest`` function - used for unit test binaries | |
9f95a23c TL |
60 | |
61 | Unit testing of CLI tools | |
62 | ------------------------- | |
9f95a23c TL |
63 | Some of the CLI tools are tested using special files ending with the extension |
64 | ``.t`` and stored under ``./src/test/cli``. These tests are run using a tool | |
f67539c2 TL |
65 | called `cram`_ via a shell script called ``./src/test/run-cli-tests``. |
66 | `cram`_ tests that are not suitable for ``make check`` can also be run by | |
67 | teuthology using the `cram task`_. | |
9f95a23c TL |
68 | |
69 | .. _`cram`: https://bitheap.org/cram/ | |
70 | .. _`cram task`: https://github.com/ceph/ceph/blob/master/qa/tasks/cram.py | |
71 | ||
f67539c2 | 72 | Tox-based testing of Python modules |
9f95a23c | 73 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
f67539c2 TL |
74 | Some of the Python modules in Ceph use `tox <https://tox.readthedocs.io/en/latest/>`_ |
75 | to run their unit tests. | |
76 | ||
77 | Most of these Python modules can be found in the directory ``./src/pybind/``. | |
78 | ||
79 | Currently (December 2020) the following modules use **tox**: | |
80 | ||
81 | * Cephadm (``./src/cephadm/tox.ini``) | |
82 | * Ceph Manager Python API (``./src/pybind/mgr``) | |
83 | ||
84 | * ``./src/pybind/mgr/tox.ini`` | |
85 | ||
86 | * ``./src/pybind/mgr/dashboard/tox.ini`` | |
87 | ||
88 | * ``./src/pybind/tox.ini`` | |
89 | ||
90 | * Dashboard (``./src/pybind/mgr/dashboard``) | |
91 | * Python common (``./src/python-common/tox.ini``) | |
92 | * CephFS (``./src/tools/cephfs/tox.ini``) | |
93 | * ceph-volume | |
94 | ||
95 | * ``./src/ceph-volume/tox.ini`` | |
96 | ||
97 | * ``./src/ceph-volume/plugin/zfs/tox.ini`` | |
98 | ||
99 | * ``./src/ceph-volume/ceph_volume/tests/functional/batch/tox.ini`` | |
100 | ||
101 | * ``./src/ceph-volume/ceph_volume/tests/functional/simple/tox.ini`` | |
102 | ||
103 | * ``./src/ceph-volume/ceph_volume/tests/functional/lvm/tox.ini`` | |
104 | ||
105 | Configuring Tox environments and tasks | |
106 | """""""""""""""""""""""""""""""""""""" | |
107 | Most tox configurations support multiple environments and tasks. | |
108 | ||
109 | The list of environments and tasks that are supported is in the ``tox.ini`` | |
110 | file, under ``envlist``. For example, here are the first three lines of | |
111 | ``./src/cephadm/tox.ini``:: | |
112 | ||
113 | [tox] | |
114 | envlist = py3, mypy | |
115 | skipsdist=true | |
116 | ||
117 | In this example, the ``Python 3`` and ``mypy`` environments are specified. | |
9f95a23c | 118 | |
f67539c2 | 119 | The list of environments can be retrieved with the following command: |
9f95a23c | 120 | |
f67539c2 | 121 | .. prompt:: bash $ |
9f95a23c | 122 | |
f67539c2 | 123 | tox --list |
9f95a23c | 124 | |
f67539c2 | 125 | Or: |
9f95a23c | 126 | |
f67539c2 | 127 | .. prompt:: bash $ |
9f95a23c | 128 | |
f67539c2 | 129 | tox -l |
9f95a23c | 130 | |
f67539c2 TL |
131 | Running Tox |
132 | """"""""""" | |
133 | To run **tox**, just execute ``tox`` in the directory containing | |
134 | ``tox.ini``. If you do not specify any environments (for example, ``-e | |
135 | $env1,$env2``), then ``tox`` will run all environments. Jenkins will run | |
136 | ``tox`` by executing ``./src/script/run_tox.sh``. | |
9f95a23c | 137 | |
f67539c2 | 138 | Here are some examples from Ceph Dashboard that show how to specify different |
9f95a23c TL |
139 | environments and run options:: |
140 | ||
141 | ## Run Python 2+3 tests+lint commands: | |
142 | $ tox -e py27,py3,lint,check | |
143 | ||
144 | ## Run Python 3 tests+lint commands: | |
145 | $ tox -e py3,lint,check | |
146 | ||
f67539c2 | 147 | ## To run it as Jenkins would: |
9f95a23c TL |
148 | $ ../../../script/run_tox.sh --tox-env py3,lint,check |
149 | ||
150 | Manager core unit tests | |
151 | """"""""""""""""""""""" | |
152 | ||
153 | Currently only doctests_ inside ``mgr_util.py`` are run. | |
154 | ||
f67539c2 TL |
155 | To add more files to be tested inside the core of the manager, open the |
156 | ``tox.ini`` file and add the files to be tested at the end of the line that | |
157 | includes ``mgr_util.py``. | |
9f95a23c TL |
158 | |
159 | .. _doctests: https://docs.python.org/3/library/doctest.html | |
160 | ||
161 | Unit test caveats | |
162 | ----------------- | |
163 | ||
f67539c2 TL |
164 | #. Unlike the various Ceph daemons and ``ceph-fuse``, the unit tests are |
165 | linked against the default memory allocator (glibc) unless they are | |
166 | explicitly linked against something else. This enables tools such as | |
167 | **valgrind** to be used in the tests. | |
9f95a23c | 168 | |
39ae355f TL |
169 | #. Google Test unit testing library hides the client output from the shell. |
170 | In order to debug the client after setting the desired debug level | |
171 | (e.g ``ceph config set client debug_rbd 20``), the debug log file can | |
172 | be found at ``build/out/client.admin.<pid>.log``. | |
173 | This can also be handy when examining teuthology failed unit test | |
174 | jobs, the job's debug level can be set at the relevant yaml file. | |
175 | ||
9f95a23c TL |
176 | .. _make check: |
177 | .. _teuthology framework: https://github.com/ceph/teuthology |