]>
Commit | Line | Data |
---|---|---|
6bb24f4d BB |
1 | .\" |
2 | .\" This file and its contents are supplied under the terms of the | |
3 | .\" Common Development and Distribution License ("CDDL"), version 1.0. | |
4 | .\" You may only use this file in accordance with the terms of version | |
5 | .\" 1.0 of the CDDL. | |
6 | .\" | |
7 | .\" A full copy of the text of the CDDL should have accompanied this | |
8 | .\" source. A copy of the CDDL is also available via the Internet at | |
9 | .\" http://www.illumos.org/license/CDDL. | |
10 | .\" | |
6bb24f4d BB |
11 | .\" Copyright (c) 2012 by Delphix. All rights reserved. |
12 | .\" | |
102c91b4 | 13 | .Dd May 26, 2021 |
4910903c AZ |
14 | .Dt RUN 1 |
15 | .Os | |
16 | . | |
17 | .Sh NAME | |
18 | .Nm run | |
19 | .Nd find, execute, and log the results of tests | |
20 | .Sh SYNOPSIS | |
21 | .Nm | |
22 | .Op Fl dgq | |
23 | .Op Fl o Ar outputdir | |
24 | .Op Fl pP Ar script | |
25 | .Op Fl t seconds | |
26 | .Op Fl uxX Ar username | |
27 | .Ar pathname Ns No … | |
28 | .Pp | |
29 | .Nm | |
30 | .Fl w Ar runfile | |
31 | .Op Fl gq | |
32 | .Op Fl o Ar outputdir | |
33 | .Op Fl pP Ar script | |
34 | .Op Fl t seconds | |
35 | .Op Fl uxX Ar username | |
36 | .Ar pathname Ns No … | |
37 | .Pp | |
38 | .Nm | |
39 | .Fl c Ar runfile | |
40 | .Op Fl dq | |
41 | .Pp | |
42 | .Nm | |
43 | .Op Fl h | |
44 | . | |
45 | .Sh DESCRIPTION | |
46 | .Nm | |
102c91b4 AZ |
47 | command has three basic modes of operation. |
48 | With neither | |
4910903c AZ |
49 | .Fl c |
50 | nor | |
51 | .Fl w , | |
52 | .Nm | |
53 | processes the arguments provided on | |
102c91b4 AZ |
54 | the command line, adding them to the list for this run. |
55 | If a specified | |
4910903c | 56 | .Ar pathname |
102c91b4 AZ |
57 | is an executable file, it is added as a test. |
58 | If a specified | |
4910903c AZ |
59 | .Ar pathname |
60 | is a directory, the behavior depends upon the presence of | |
61 | .Fl g . | |
62 | If | |
63 | .Fl g | |
102c91b4 AZ |
64 | is specified, the directory is treated as a test group. |
65 | See the section on | |
4910903c | 66 | .Sy Test Groups |
102c91b4 AZ |
67 | below. |
68 | Without | |
4910903c AZ |
69 | .Fl g , |
70 | .Nm | |
71 | simply descends into the directory looking for executable files. | |
72 | The tests are then executed, and the results are logged. | |
73 | .Pp | |
74 | With | |
75 | .Fl w , | |
76 | .Nm | |
77 | finds tests in the manner described above. | |
6bb24f4d | 78 | Rather than executing the tests and logging the results, the test configuration |
4910903c AZ |
79 | is stored in a |
80 | .Ar runfile , | |
81 | which can be used in future invocations, or edited | |
102c91b4 AZ |
82 | to modify which tests are executed and which options are applied. |
83 | Options included on the command line with | |
4910903c AZ |
84 | .Fl w |
85 | become defaults in the | |
86 | .Ar runfile . | |
87 | .Pp | |
88 | With | |
89 | .Fl c , | |
90 | .Nm | |
91 | parses a | |
92 | .Ar runfile , | |
93 | which can specify a series of tests and test groups to be executed. | |
94 | The tests are then executed, and the results are logged. | |
95 | . | |
96 | .Ss Test Groups | |
6bb24f4d | 97 | A test group is comprised of a set of executable files, all of which exist in |
102c91b4 AZ |
98 | one directory. |
99 | The options specified on the command line or in a | |
4910903c | 100 | .Ar runfile |
102c91b4 | 101 | apply to individual tests in the group. |
b46be903 DS |
102 | The exception is options pertaining to pre and post scripts, which act on all |
103 | tests as a group. | |
102c91b4 AZ |
104 | Rather than running before and after each test, |
105 | these scripts are run only once each at the start and end of the test group. | |
4910903c | 106 | .Ss Test Execution |
6bb24f4d | 107 | The specified tests run serially, and are typically assigned results according |
102c91b4 AZ |
108 | to exit values. |
109 | Tests that exit zero and non-zero are marked | |
4910903c AZ |
110 | .Sy PASS |
111 | and | |
112 | .Sy FAIL , | |
102c91b4 AZ |
113 | respectively. |
114 | When a pre script fails for a test group, only the post script is executed, | |
115 | and the remaining tests are marked | |
4910903c AZ |
116 | .Sy SKIPPED . |
117 | Any test that exceeds | |
118 | its | |
119 | .Ar timeout | |
120 | is terminated, and marked | |
121 | .Sy KILLED . | |
122 | .Pp | |
123 | By default, tests are executed with the credentials of the | |
102c91b4 | 124 | .Nm |
4910903c AZ |
125 | script. |
126 | Executing tests with other credentials is done via | |
127 | .Xr sudo 1m , | |
128 | which must | |
102c91b4 AZ |
129 | be configured to allow execution without prompting for a password. |
130 | Environment variables from the calling shell are available to individual tests. | |
131 | During test execution, the working directory is changed to | |
4910903c AZ |
132 | .Ar outputdir . |
133 | . | |
134 | .Ss Output Logging | |
135 | By default, | |
136 | .Nm | |
137 | will print one line on standard output at the conclusion | |
102c91b4 AZ |
138 | of each test indicating the test name, result and elapsed time. |
139 | Additionally, for each invocation of | |
4910903c | 140 | .Nm , |
102c91b4 AZ |
141 | a directory is created using the ISO 8601 date format. |
142 | Within this directory is a file named | |
4910903c AZ |
143 | .Sy log |
144 | containing all the | |
102c91b4 AZ |
145 | test output with timestamps, and a directory for each test. |
146 | Within the test directories, there is one file each for standard output, | |
147 | standard error and merged output. | |
148 | The default location for the | |
4910903c AZ |
149 | .Ar outputdir |
150 | is | |
151 | .Pa /var/tmp/test_results . | |
152 | .Ss "Runfiles" | |
153 | The | |
154 | .Ar runfile | |
155 | is an INI-style configuration file that describes a test run. | |
156 | The file has one section named | |
157 | .Sy DEFAULT , | |
158 | which contains configuration option | |
159 | names and their values in | |
160 | .Sy name No = Ar value | |
102c91b4 AZ |
161 | format. |
162 | The values in this section apply to all the subsequent sections, | |
163 | unless they are also specified there, in which case the default is overridden. | |
164 | The remaining section names are the absolute pathnames of files and directories, | |
165 | describing tests and test groups respectively. | |
166 | The legal option names are: | |
4910903c AZ |
167 | .Bl -tag -width "tests = ['filename', …]" |
168 | .It Sy outputdir No = Ar pathname | |
6bb24f4d | 169 | The name of the directory that holds test logs. |
4910903c AZ |
170 | .It Sy pre No = Ar script |
171 | Run | |
172 | .Ar script | |
173 | prior to the test or test group. | |
174 | .It Sy pre_user No = Ar username | |
175 | Execute the pre script as | |
176 | .Ar username . | |
177 | .It Sy post No = Ar script | |
178 | Run | |
179 | .Ar script | |
180 | after the test or test group. | |
181 | .It Sy post_user No = Ar username | |
182 | Execute the post script as | |
183 | .Ar username . | |
184 | .It Sy quiet No = Sy True Ns | Ns Sy False | |
185 | If | |
186 | .Sy True , | |
187 | only the results summary is printed to standard out. | |
102c91b4 | 188 | .It Sy tests No = [ Ns Ar 'filename' , No … ] |
4910903c AZ |
189 | Specify a list of |
190 | .Ar filenames | |
102c91b4 AZ |
191 | for this test group. |
192 | Only the basename of the absolute path is required. | |
193 | This option is only valid for test groups, and each | |
4910903c AZ |
194 | .Ar filename |
195 | must be single quoted. | |
196 | .It Sy timeout No = Ar n | |
197 | A timeout value of | |
198 | .Ar n | |
199 | seconds. | |
200 | .It Sy user No = Ar username | |
201 | Execute the test or test group as | |
202 | .Ar username . | |
203 | .El | |
204 | . | |
205 | .Sh OPTIONS | |
206 | .Bl -tag -width "-o outputdir" | |
207 | .It Fl c Ar runfile | |
208 | Specify a | |
209 | .Ar runfile | |
210 | to be consumed by the run command. | |
211 | .It Fl d | |
102c91b4 AZ |
212 | Dry run mode. |
213 | Execute no tests, but print a description of each test that would have been run. | |
78fad47c DS |
214 | .It Fl m |
215 | Enable kmemleak reporting (Linux only) | |
4910903c | 216 | .It Fl g |
6bb24f4d | 217 | Create test groups from any directories found while searching for tests. |
4910903c | 218 | .It Fl o Ar outputdir |
6bb24f4d | 219 | Specify the directory in which to write test results. |
4910903c AZ |
220 | .It Fl p Ar script |
221 | Run | |
222 | .Ar script | |
223 | prior to any test or test group. | |
224 | .It Fl P Ar script | |
225 | Run | |
226 | .Ar script | |
227 | after any test or test group. | |
228 | .It Fl q | |
a57c82fc | 229 | Print only the results summary to the standard output. |
4910903c AZ |
230 | .It Fl s Ar script |
231 | Run | |
102c91b4 AZ |
232 | .Ar script |
233 | as a failsafe after any test is killed. | |
4910903c AZ |
234 | .It Fl S Ar username |
235 | Execute the failsafe script as | |
236 | .Ar username . | |
237 | .It Fl t Ar n | |
238 | Specify a timeout value of | |
102c91b4 AZ |
239 | .Ar n |
240 | seconds per test. | |
4910903c AZ |
241 | .It Fl u Ar username |
242 | Execute tests or test groups as | |
243 | .Ar username . | |
244 | .It Fl w Ar runfile | |
245 | Specify the name of the | |
246 | .Ar runfile | |
247 | to create. | |
248 | .It Fl x Ar username | |
249 | Execute the pre script as | |
250 | .Ar username . | |
251 | .It Fl X Ar username | |
252 | Execute the post script as | |
253 | .Ar username . | |
254 | .El | |
255 | . | |
256 | .Sh EXAMPLES | |
257 | .Bl -tag -width "-h" | |
b46be903 | 258 | .It Sy Example 1 : No Running ad-hoc tests . |
4910903c AZ |
259 | This example demonstrates the simplest invocation of |
260 | .Nm . | |
261 | .Bd -literal | |
262 | .No % Nm run Ar my-tests | |
6bb24f4d BB |
263 | Test: /home/jkennedy/my-tests/test-01 [00:02] [PASS] |
264 | Test: /home/jkennedy/my-tests/test-02 [00:04] [PASS] | |
265 | Test: /home/jkennedy/my-tests/test-03 [00:01] [PASS] | |
266 | ||
267 | Results Summary | |
268 | PASS 3 | |
269 | ||
270 | Running Time: 00:00:07 | |
271 | Percent passed: 100.0% | |
272 | Log directory: /var/tmp/test_results/20120923T180654 | |
4910903c | 273 | .Ed |
b46be903 | 274 | .It Sy Example 2 : No Creating a Ar runfile No for future use . |
4910903c AZ |
275 | This example demonstrates creating a |
276 | .Ar runfile | |
277 | with non-default options. | |
278 | .Bd -literal | |
279 | .No % Nm run Fl p Ar setup Fl x Ar root Fl g Fl w Ar new-tests.run Ar new-tests | |
280 | .No % Nm cat Pa new-tests.run | |
6bb24f4d BB |
281 | [DEFAULT] |
282 | pre = setup | |
283 | post_user = | |
284 | quiet = False | |
285 | user = | |
286 | timeout = 60 | |
287 | post = | |
288 | pre_user = root | |
289 | outputdir = /var/tmp/test_results | |
290 | ||
291 | [/home/jkennedy/new-tests] | |
292 | tests = ['test-01', 'test-02', 'test-03'] | |
4910903c AZ |
293 | .Ed |
294 | .El | |
295 | . | |
296 | .Sh SEE ALSO | |
297 | .Xr sudo 1m |