]>
Commit | Line | Data |
---|---|---|
9f95a23c TL |
1 | Running Integration Tests using Teuthology |
2 | ========================================== | |
3 | ||
4 | Getting binaries | |
5 | ---------------- | |
6 | To run integration tests using teuthology, you need to have Ceph binaries | |
7 | built for your branch. Follow these steps to initiate the build process - | |
8 | ||
9 | #. Push the branch to `ceph-ci`_ repository. This triggers the process of | |
10 | building the binaries. | |
11 | ||
12 | #. To confirm that the build process has been initiated, spot the branch name | |
13 | at `Shaman`_. Little after the build process has been initiated, the single | |
14 | entry with your branch name would multiply, each new entry for a different | |
15 | combination of distro and flavour. | |
16 | ||
17 | #. Wait until the packages are built and uploaded, and the repository offering | |
18 | them are created. This is marked by colouring the entries for the branch | |
19 | name green. Preferably, wait until each entry is coloured green. Usually, | |
20 | it takes around 2-3 hours depending on the availability of the machines. | |
21 | ||
22 | .. note:: Branch to be pushed on ceph-ci can be any branch, it shouldn't | |
23 | necessarily be a PR branch. | |
24 | ||
25 | .. note:: In case you are pushing master or any other standard branch, check | |
26 | `Shaman`_ beforehand since it already might have builds ready for it. | |
27 | ||
28 | Triggering Tests | |
29 | ---------------- | |
30 | After building is complete, proceed to trigger tests - | |
31 | ||
32 | #. Log in to the teuthology machine:: | |
33 | ||
34 | ssh <username>@teuthology.front.sepia.ceph.com | |
35 | ||
36 | This would require Sepia lab access. To know how to request it, see: https://ceph.github.io/sepia/adding_users/ | |
37 | ||
38 | #. Next, get teuthology installed. Run the first set of commands in | |
39 | `Running Your First Test`_ for that. After that, activate the virtual | |
40 | environment in which teuthology is installed. | |
41 | ||
42 | #. Run the ``teuthology-suite`` command:: | |
43 | ||
44 | teuthology-suite -v -m smithi -c wip-devname-feature-x -s fs -p 110 --filter "cephfs-shell" | |
45 | ||
46 | Following are the options used in above command with their meanings - | |
47 | -v verbose | |
48 | -m machine name | |
49 | -c branch name, the branch that was pushed on ceph-ci | |
50 | -s test-suite name | |
51 | -p higher the number, lower the priority of the job | |
52 | --filter filter tests in given suite that needs to run, the arg to | |
53 | filter should be the test you want to run | |
54 | ||
55 | .. note:: The priority number present in the command above is just a | |
56 | placeholder. It might be highly inappropriate for the jobs you may want to | |
57 | trigger. See `Testing Priority`_ section to pick a priority number. | |
58 | ||
59 | .. note:: Don't skip passing a priority number, the default value is 1000 | |
60 | which way too high; the job probably might never run. | |
61 | ||
62 | #. Wait for the tests to run. ``teuthology-suite`` prints a link to the | |
63 | `Pulpito`_ page created for the tests triggered. | |
64 | ||
65 | Other frequently used/useful options are ``-d`` (or ``--distro``), | |
66 | ``--distroversion``, ``--filter-out``, ``--timeout``, ``flavor``, ``-rerun``. | |
67 | Run ``teuthology-suite --help`` to read description of these and every other | |
68 | options available. | |
69 | ||
70 | About Suites and Filters | |
71 | ------------------------ | |
72 | See `Suites Inventory`_ for a list of suites of integration tests present | |
73 | right now. Alternatively, each directory under ``qa/suites`` in Ceph | |
74 | repository is an integration test suite, so looking within that directory | |
75 | to decide an appropriate argument for ``-s`` also works. | |
76 | ||
77 | For picking an argument for ``--filter``, look within | |
78 | ``qa/suites/<suite-name>/<subsuite-name>/tasks`` to get keywords for filtering | |
79 | tests. Each YAML file in there can trigger a bunch of tests; using the name of | |
80 | the file, without the extension part of the file name, as an argument to the | |
81 | ``--filter`` will trigger those tests. For example, the sample command above | |
82 | uses ``cephfs-shell`` since there's a file named ``cephfs-shell.yaml`` in | |
83 | ``qa/suites/fs/basic_functional/tasks/``. In case, the file name doesn't hint | |
84 | what bunch of tests it would trigger, look at the contents of the file for | |
85 | ``modules`` attribute. For ``cephfs-shell.yaml`` the ``modules`` attribute | |
86 | is ``tasks.cephfs.test_cephfs_shell`` which means it'll trigger all tests in | |
87 | ``qa/tasks/cephfs/test_cephfs_shell.py``. | |
88 | ||
89 | Killing Tests | |
90 | ------------- | |
91 | Sometimes a teuthology job might not complete running for several minutes or | |
92 | even hours after tests that were trigged have completed running and other | |
93 | times wrong set of tests can be triggered is filter wasn't chosen carefully. | |
94 | To save resource it's better to termniate such a job. Following is the command | |
95 | to terminate a job:: | |
96 | ||
97 | teuthology-kill -r teuthology-2019-12-10_05:00:03-smoke-master-testing-basic-smithi | |
98 | ||
99 | Let's call the the argument passed to ``-r`` as test ID. It can be found | |
100 | easily in the link to the Pulpito page for the tests you triggered. For | |
101 | example, for the above test ID, the link is - http://pulpito.front.sepia.ceph.com/teuthology-2019-12-10_05:00:03-smoke-master-testing-basic-smithi/ | |
102 | ||
103 | Re-running Tests | |
104 | ---------------- | |
105 | Pass ``--rerun`` option, with test ID as an argument to it, to | |
106 | ``teuthology-suite`` command:: | |
107 | ||
108 | teuthology-suite -v -m smithi -c wip-rishabh-fs-test_cephfs_shell-fix -p 50 --rerun teuthology-2019-12-10_05:00:03-smoke-master-testing-basic-smithi | |
109 | ||
110 | The meaning of rest of the options is already covered in `Triggering Tests` | |
111 | section. | |
112 | ||
113 | Teuthology Archives | |
114 | ------------------- | |
115 | Once the tests have finished running, the log for the job can be obtained by | |
116 | clicking on job ID at the Pulpito page for your tests. It's more convenient to | |
117 | download the log and then view it rather than viewing it in an internet | |
118 | browser since these logs can easily be upto size of 1 GB. What's much more | |
119 | easier is to log in to the teuthology machine again | |
120 | (``teuthology.front.sepia.ceph.com``), and access the following path:: | |
121 | ||
122 | /ceph/teuthology-archive/<test-id>/<job-id>/teuthology.log | |
123 | ||
124 | For example, for above test ID path is:: | |
125 | ||
126 | /ceph/teuthology-archive/teuthology-2019-12-10_05:00:03-smoke-master-testing-basic-smithi/4588482/teuthology.log | |
127 | ||
128 | This way the log remotely can be viewed remotely without having to wait too | |
129 | much. | |
130 | ||
131 | Naming the ceph-ci branch | |
132 | ------------------------- | |
133 | There are no hard conventions (except for the case of stable branch; see | |
134 | next paragraph) for how the branch pushed on ceph-ci is named. But, to make | |
135 | builds and tests easily identitifiable on Shaman and Pulpito respectively, | |
136 | prepend it with your name. For example branch ``feature-x`` can be named | |
137 | ``wip-yourname-feature-x`` while pushing on ceph-ci. | |
138 | ||
139 | In case you are using one of the stable branches (e.g. nautilis, mimic, | |
140 | etc.), include the name of that stable branch in your ceph-ci branch name. | |
141 | For example, ``feature-x`` PR branch should be named as | |
142 | ``wip-feature-x-nautilus``. *This is not just a matter of convention but this, | |
143 | more essentially, builds your branch in the correct environment.* | |
144 | ||
145 | Delete the branch from ceph-ci, once it's not required anymore. If you are | |
146 | logged in at GitHub, all your branches on ceph-ci can be easily found here - | |
147 | https://github.com/ceph/ceph-ci/branches. | |
148 | ||
149 | .. _ceph-ci: https://github.com/ceph/ceph-ci | |
150 | .. _Pulpito: http://pulpito.front.sepia.ceph.com/ | |
151 | .. _Running Your First Test: ../running-tests-locally/#running-your-first-test | |
152 | .. _Shaman: https://shaman.ceph.com/builds/ceph/ | |
153 | .. _Suites Inventory: ../tests-integration-tests/#suites-inventory | |
154 | .. _Testing Priority: ../tests-integration-tests/#testing-priority |