]>
Commit | Line | Data |
---|---|---|
85aaf69f | 1 | % Testing |
1a4d82fc JJ |
2 | |
3 | > Program testing can be a very effective way to show the presence of bugs, but | |
c34b1796 | 4 | > it is hopelessly inadequate for showing their absence. |
1a4d82fc JJ |
5 | > |
6 | > Edsger W. Dijkstra, "The Humble Programmer" (1972) | |
7 | ||
8 | Let's talk about how to test Rust code. What we will not be talking about is | |
9 | the right way to test Rust code. There are many schools of thought regarding | |
10 | the right and wrong way to write tests. All of these approaches use the same | |
11 | basic tools, and so we'll show you the syntax for using them. | |
12 | ||
13 | # The `test` attribute | |
14 | ||
15 | At its simplest, a test in Rust is a function that's annotated with the `test` | |
16 | attribute. Let's make a new project with Cargo called `adder`: | |
17 | ||
18 | ```bash | |
19 | $ cargo new adder | |
20 | $ cd adder | |
21 | ``` | |
22 | ||
23 | Cargo will automatically generate a simple test when you make a new project. | |
24 | Here's the contents of `src/lib.rs`: | |
25 | ||
26 | ```rust | |
27 | #[test] | |
28 | fn it_works() { | |
29 | } | |
30 | ``` | |
31 | ||
32 | Note the `#[test]`. This attribute indicates that this is a test function. It | |
33 | currently has no body. That's good enough to pass! We can run the tests with | |
34 | `cargo test`: | |
35 | ||
36 | ```bash | |
37 | $ cargo test | |
38 | Compiling adder v0.0.1 (file:///home/you/projects/adder) | |
39 | Running target/adder-91b3e234d4ed382a | |
40 | ||
41 | running 1 test | |
42 | test it_works ... ok | |
43 | ||
44 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
45 | ||
46 | Doc-tests adder | |
47 | ||
48 | running 0 tests | |
49 | ||
50 | test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured | |
51 | ``` | |
52 | ||
53 | Cargo compiled and ran our tests. There are two sets of output here: one | |
54 | for the test we wrote, and another for documentation tests. We'll talk about | |
55 | those later. For now, see this line: | |
56 | ||
57 | ```text | |
58 | test it_works ... ok | |
59 | ``` | |
60 | ||
61 | Note the `it_works`. This comes from the name of our function: | |
62 | ||
63 | ```rust | |
64 | fn it_works() { | |
65 | # } | |
66 | ``` | |
67 | ||
68 | We also get a summary line: | |
69 | ||
70 | ```text | |
71 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
72 | ``` | |
73 | ||
74 | So why does our do-nothing test pass? Any test which doesn't `panic!` passes, | |
75 | and any test that does `panic!` fails. Let's make our test fail: | |
76 | ||
77 | ```rust | |
78 | #[test] | |
79 | fn it_works() { | |
80 | assert!(false); | |
81 | } | |
82 | ``` | |
83 | ||
84 | `assert!` is a macro provided by Rust which takes one argument: if the argument | |
85 | is `true`, nothing happens. If the argument is false, it `panic!`s. Let's run | |
86 | our tests again: | |
87 | ||
88 | ```bash | |
89 | $ cargo test | |
90 | Compiling adder v0.0.1 (file:///home/you/projects/adder) | |
91 | Running target/adder-91b3e234d4ed382a | |
92 | ||
93 | running 1 test | |
94 | test it_works ... FAILED | |
95 | ||
96 | failures: | |
97 | ||
98 | ---- it_works stdout ---- | |
85aaf69f | 99 | thread 'it_works' panicked at 'assertion failed: false', /home/steve/tmp/adder/src/lib.rs:3 |
1a4d82fc JJ |
100 | |
101 | ||
102 | ||
103 | failures: | |
104 | it_works | |
105 | ||
106 | test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured | |
107 | ||
85aaf69f | 108 | thread '<main>' panicked at 'Some tests failed', /home/steve/src/rust/src/libtest/lib.rs:247 |
1a4d82fc JJ |
109 | ``` |
110 | ||
111 | Rust indicates that our test failed: | |
112 | ||
113 | ```text | |
114 | test it_works ... FAILED | |
115 | ``` | |
116 | ||
117 | And that's reflected in the summary line: | |
118 | ||
119 | ```text | |
120 | test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured | |
121 | ``` | |
122 | ||
123 | We also get a non-zero status code: | |
124 | ||
125 | ```bash | |
126 | $ echo $? | |
127 | 101 | |
128 | ``` | |
129 | ||
130 | This is useful if you want to integrate `cargo test` into other tooling. | |
131 | ||
c34b1796 | 132 | We can invert our test's failure with another attribute: `should_panic`: |
1a4d82fc JJ |
133 | |
134 | ```rust | |
135 | #[test] | |
c34b1796 | 136 | #[should_panic] |
1a4d82fc JJ |
137 | fn it_works() { |
138 | assert!(false); | |
139 | } | |
140 | ``` | |
141 | ||
142 | This test will now succeed if we `panic!` and fail if we complete. Let's try it: | |
143 | ||
144 | ```bash | |
145 | $ cargo test | |
146 | Compiling adder v0.0.1 (file:///home/you/projects/adder) | |
147 | Running target/adder-91b3e234d4ed382a | |
148 | ||
149 | running 1 test | |
150 | test it_works ... ok | |
151 | ||
152 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
153 | ||
154 | Doc-tests adder | |
155 | ||
156 | running 0 tests | |
157 | ||
158 | test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured | |
159 | ``` | |
160 | ||
161 | Rust provides another macro, `assert_eq!`, that compares two arguments for | |
162 | equality: | |
163 | ||
164 | ```rust | |
165 | #[test] | |
c34b1796 | 166 | #[should_panic] |
1a4d82fc JJ |
167 | fn it_works() { |
168 | assert_eq!("Hello", "world"); | |
169 | } | |
170 | ``` | |
171 | ||
c34b1796 | 172 | Does this test pass or fail? Because of the `should_panic` attribute, it |
1a4d82fc JJ |
173 | passes: |
174 | ||
175 | ```bash | |
176 | $ cargo test | |
177 | Compiling adder v0.0.1 (file:///home/you/projects/adder) | |
178 | Running target/adder-91b3e234d4ed382a | |
179 | ||
180 | running 1 test | |
181 | test it_works ... ok | |
182 | ||
183 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
184 | ||
185 | Doc-tests adder | |
186 | ||
187 | running 0 tests | |
188 | ||
189 | test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured | |
190 | ``` | |
191 | ||
c34b1796 | 192 | `should_panic` tests can be fragile, as it's hard to guarantee that the test |
1a4d82fc | 193 | didn't fail for an unexpected reason. To help with this, an optional `expected` |
c34b1796 | 194 | parameter can be added to the `should_panic` attribute. The test harness will |
1a4d82fc JJ |
195 | make sure that the failure message contains the provided text. A safer version |
196 | of the example above would be: | |
197 | ||
62682a34 | 198 | ```rust |
1a4d82fc | 199 | #[test] |
c34b1796 | 200 | #[should_panic(expected = "assertion failed")] |
1a4d82fc JJ |
201 | fn it_works() { |
202 | assert_eq!("Hello", "world"); | |
203 | } | |
204 | ``` | |
205 | ||
206 | That's all there is to the basics! Let's write one 'real' test: | |
207 | ||
62682a34 | 208 | ```rust,ignore |
1a4d82fc JJ |
209 | pub fn add_two(a: i32) -> i32 { |
210 | a + 2 | |
211 | } | |
212 | ||
213 | #[test] | |
214 | fn it_works() { | |
215 | assert_eq!(4, add_two(2)); | |
216 | } | |
217 | ``` | |
218 | ||
219 | This is a very common use of `assert_eq!`: call some function with | |
220 | some known arguments and compare it to the expected output. | |
221 | ||
bd371182 | 222 | # The `tests` module |
1a4d82fc JJ |
223 | |
224 | There is one way in which our existing example is not idiomatic: it's | |
bd371182 | 225 | missing the `tests` module. The idiomatic way of writing our example |
1a4d82fc JJ |
226 | looks like this: |
227 | ||
62682a34 | 228 | ```rust,ignore |
1a4d82fc JJ |
229 | pub fn add_two(a: i32) -> i32 { |
230 | a + 2 | |
231 | } | |
232 | ||
233 | #[cfg(test)] | |
bd371182 | 234 | mod tests { |
1a4d82fc JJ |
235 | use super::add_two; |
236 | ||
237 | #[test] | |
238 | fn it_works() { | |
239 | assert_eq!(4, add_two(2)); | |
240 | } | |
241 | } | |
242 | ``` | |
243 | ||
bd371182 | 244 | There's a few changes here. The first is the introduction of a `mod tests` with |
1a4d82fc JJ |
245 | a `cfg` attribute. The module allows us to group all of our tests together, and |
246 | to also define helper functions if needed, that don't become a part of the rest | |
247 | of our crate. The `cfg` attribute only compiles our test code if we're | |
248 | currently trying to run the tests. This can save compile time, and also ensures | |
249 | that our tests are entirely left out of a normal build. | |
250 | ||
251 | The second change is the `use` declaration. Because we're in an inner module, | |
252 | we need to bring our test function into scope. This can be annoying if you have | |
253 | a large module, and so this is a common use of the `glob` feature. Let's change | |
254 | our `src/lib.rs` to make use of it: | |
255 | ||
62682a34 | 256 | ```rust,ignore |
1a4d82fc JJ |
257 | |
258 | pub fn add_two(a: i32) -> i32 { | |
259 | a + 2 | |
260 | } | |
261 | ||
262 | #[cfg(test)] | |
bd371182 | 263 | mod tests { |
1a4d82fc JJ |
264 | use super::*; |
265 | ||
266 | #[test] | |
267 | fn it_works() { | |
268 | assert_eq!(4, add_two(2)); | |
269 | } | |
270 | } | |
271 | ``` | |
272 | ||
85aaf69f | 273 | Note the different `use` line. Now we run our tests: |
1a4d82fc JJ |
274 | |
275 | ```bash | |
276 | $ cargo test | |
277 | Updating registry `https://github.com/rust-lang/crates.io-index` | |
278 | Compiling adder v0.0.1 (file:///home/you/projects/adder) | |
279 | Running target/adder-91b3e234d4ed382a | |
280 | ||
281 | running 1 test | |
bd371182 | 282 | test tests::it_works ... ok |
1a4d82fc JJ |
283 | |
284 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
285 | ||
286 | Doc-tests adder | |
287 | ||
288 | running 0 tests | |
289 | ||
290 | test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured | |
291 | ``` | |
292 | ||
293 | It works! | |
294 | ||
bd371182 | 295 | The current convention is to use the `tests` module to hold your "unit-style" |
1a4d82fc | 296 | tests. Anything that just tests one small bit of functionality makes sense to |
85aaf69f | 297 | go here. But what about "integration-style" tests instead? For that, we have |
1a4d82fc JJ |
298 | the `tests` directory |
299 | ||
300 | # The `tests` directory | |
301 | ||
302 | To write an integration test, let's make a `tests` directory, and | |
303 | put a `tests/lib.rs` file inside, with this as its contents: | |
304 | ||
62682a34 | 305 | ```rust,ignore |
1a4d82fc JJ |
306 | extern crate adder; |
307 | ||
308 | #[test] | |
309 | fn it_works() { | |
85aaf69f | 310 | assert_eq!(4, adder::add_two(2)); |
c34b1796 | 311 | } |
1a4d82fc JJ |
312 | ``` |
313 | ||
314 | This looks similar to our previous tests, but slightly different. We now have | |
315 | an `extern crate adder` at the top. This is because the tests in the `tests` | |
316 | directory are an entirely separate crate, and so we need to import our library. | |
317 | This is also why `tests` is a suitable place to write integration-style tests: | |
318 | they use the library like any other consumer of it would. | |
319 | ||
320 | Let's run them: | |
321 | ||
322 | ```bash | |
323 | $ cargo test | |
324 | Compiling adder v0.0.1 (file:///home/you/projects/adder) | |
325 | Running target/adder-91b3e234d4ed382a | |
326 | ||
327 | running 1 test | |
bd371182 | 328 | test tests::it_works ... ok |
1a4d82fc JJ |
329 | |
330 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
331 | ||
332 | Running target/lib-c18e7d3494509e74 | |
333 | ||
334 | running 1 test | |
335 | test it_works ... ok | |
336 | ||
337 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
338 | ||
339 | Doc-tests adder | |
340 | ||
341 | running 0 tests | |
342 | ||
343 | test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured | |
344 | ``` | |
345 | ||
346 | Now we have three sections: our previous test is also run, as well as our new | |
347 | one. | |
348 | ||
bd371182 | 349 | That's all there is to the `tests` directory. The `tests` module isn't needed |
1a4d82fc JJ |
350 | here, since the whole thing is focused on tests. |
351 | ||
352 | Let's finally check out that third section: documentation tests. | |
353 | ||
354 | # Documentation tests | |
355 | ||
356 | Nothing is better than documentation with examples. Nothing is worse than | |
357 | examples that don't actually work, because the code has changed since the | |
358 | documentation has been written. To this end, Rust supports automatically | |
359 | running examples in your documentation. Here's a fleshed-out `src/lib.rs` | |
360 | with examples: | |
361 | ||
62682a34 | 362 | ```rust,ignore |
1a4d82fc JJ |
363 | //! The `adder` crate provides functions that add numbers to other numbers. |
364 | //! | |
365 | //! # Examples | |
366 | //! | |
367 | //! ``` | |
368 | //! assert_eq!(4, adder::add_two(2)); | |
369 | //! ``` | |
370 | ||
1a4d82fc JJ |
371 | /// This function adds two to its argument. |
372 | /// | |
373 | /// # Examples | |
374 | /// | |
375 | /// ``` | |
376 | /// use adder::add_two; | |
377 | /// | |
378 | /// assert_eq!(4, add_two(2)); | |
379 | /// ``` | |
380 | pub fn add_two(a: i32) -> i32 { | |
381 | a + 2 | |
382 | } | |
383 | ||
384 | #[cfg(test)] | |
bd371182 | 385 | mod tests { |
1a4d82fc JJ |
386 | use super::*; |
387 | ||
388 | #[test] | |
389 | fn it_works() { | |
390 | assert_eq!(4, add_two(2)); | |
391 | } | |
392 | } | |
393 | ``` | |
394 | ||
395 | Note the module-level documentation with `//!` and the function-level | |
396 | documentation with `///`. Rust's documentation supports Markdown in comments, | |
397 | and so triple graves mark code blocks. It is conventional to include the | |
398 | `# Examples` section, exactly like that, with examples following. | |
399 | ||
400 | Let's run the tests again: | |
401 | ||
402 | ```bash | |
403 | $ cargo test | |
404 | Compiling adder v0.0.1 (file:///home/steve/tmp/adder) | |
405 | Running target/adder-91b3e234d4ed382a | |
406 | ||
407 | running 1 test | |
bd371182 | 408 | test tests::it_works ... ok |
1a4d82fc JJ |
409 | |
410 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
411 | ||
412 | Running target/lib-c18e7d3494509e74 | |
413 | ||
414 | running 1 test | |
415 | test it_works ... ok | |
416 | ||
417 | test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured | |
418 | ||
419 | Doc-tests adder | |
420 | ||
421 | running 2 tests | |
422 | test add_two_0 ... ok | |
423 | test _0 ... ok | |
424 | ||
425 | test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured | |
426 | ``` | |
427 | ||
428 | Now we have all three kinds of tests running! Note the names of the | |
429 | documentation tests: the `_0` is generated for the module test, and `add_two_0` | |
430 | for the function test. These will auto increment with names like `add_two_1` as | |
431 | you add more examples. | |
432 |