3 > Program testing can be a very effective way to show the presence of bugs, but
4 > it is hopelessly inadequate for showing their absence.
6 > Edsger W. Dijkstra, "The Humble Programmer" (1972)
8 Let's talk about how to test Rust code. What we will not be talking about is
9 the right way to test Rust code. There are many schools of thought regarding
10 the right and wrong way to write tests. All of these approaches use the same
11 basic tools, and so we'll show you the syntax for using them.
13 # The `test` attribute
15 At its simplest, a test in Rust is a function that's annotated with the `test`
16 attribute. Let's make a new project with Cargo called `adder`:
23 Cargo will automatically generate a simple test when you make a new project.
24 Here's the contents of `src/lib.rs`:
32 Note the `#[test]`. This attribute indicates that this is a test function. It
33 currently has no body. That's good enough to pass! We can run the tests with
38 Compiling adder v0.0.1 (file:///home/you/projects/adder)
39 Running target/adder-91b3e234d4ed382a
44 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
50 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
53 Cargo compiled and ran our tests. There are two sets of output here: one
54 for the test we wrote, and another for documentation tests. We'll talk about
55 those later. For now, see this line:
61 Note the `it_works`. This comes from the name of our function:
68 We also get a summary line:
71 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
74 So why does our do-nothing test pass? Any test which doesn't `panic!` passes,
75 and any test that does `panic!` fails. Let's make our test fail:
84 `assert!` is a macro provided by Rust which takes one argument: if the argument
85 is `true`, nothing happens. If the argument is `false`, it `panic!`s. Let's run
90 Compiling adder v0.0.1 (file:///home/you/projects/adder)
91 Running target/adder-91b3e234d4ed382a
94 test it_works ... FAILED
98 ---- it_works stdout ----
99 thread 'it_works' panicked at 'assertion failed: false', /home/steve/tmp/adder/src/lib.rs:3
106 test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
108 thread '<main>' panicked at 'Some tests failed', /home/steve/src/rust/src/libtest/lib.rs:247
111 Rust indicates that our test failed:
114 test it_works ... FAILED
117 And that's reflected in the summary line:
120 test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
123 We also get a non-zero status code. We can use `$?` on OS X and Linux:
130 On Windows, if you’re using `cmd`:
136 And if you’re using PowerShell:
139 > echo $LASTEXITCODE # the code itself
140 > echo $? # a boolean, fail or succeed
143 This is useful if you want to integrate `cargo test` into other tooling.
145 We can invert our test's failure with another attribute: `should_panic`:
155 This test will now succeed if we `panic!` and fail if we complete. Let's try it:
159 Compiling adder v0.0.1 (file:///home/you/projects/adder)
160 Running target/adder-91b3e234d4ed382a
165 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
171 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
174 Rust provides another macro, `assert_eq!`, that compares two arguments for
181 assert_eq!("Hello", "world");
185 Does this test pass or fail? Because of the `should_panic` attribute, it
190 Compiling adder v0.0.1 (file:///home/you/projects/adder)
191 Running target/adder-91b3e234d4ed382a
196 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
202 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
205 `should_panic` tests can be fragile, as it's hard to guarantee that the test
206 didn't fail for an unexpected reason. To help with this, an optional `expected`
207 parameter can be added to the `should_panic` attribute. The test harness will
208 make sure that the failure message contains the provided text. A safer version
209 of the example above would be:
213 #[should_panic(expected = "assertion failed")]
215 assert_eq!("Hello", "world");
219 That's all there is to the basics! Let's write one 'real' test:
222 pub fn add_two(a: i32) -> i32 {
228 assert_eq!(4, add_two(2));
232 This is a very common use of `assert_eq!`: call some function with
233 some known arguments and compare it to the expected output.
235 # The `ignore` attribute
237 Sometimes a few specific tests can be very time-consuming to execute. These
238 can be disabled by default by using the `ignore` attribute:
243 assert_eq!(4, add_two(2));
248 fn expensive_test() {
249 // code that takes an hour to run
253 Now we run our tests and see that `it_works` is run, but `expensive_test` is
258 Compiling adder v0.0.1 (file:///home/you/projects/adder)
259 Running target/adder-91b3e234d4ed382a
262 test expensive_test ... ignored
265 test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured
271 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
274 The expensive tests can be run explicitly using `cargo test -- --ignored`:
277 $ cargo test -- --ignored
278 Running target/adder-91b3e234d4ed382a
281 test expensive_test ... ok
283 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
289 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
292 The `--ignored` argument is an argument to the test binary, and not to Cargo,
293 which is why the command is `cargo test -- --ignored`.
297 There is one way in which our existing example is not idiomatic: it's
298 missing the `tests` module. The idiomatic way of writing our example
302 pub fn add_two(a: i32) -> i32 {
312 assert_eq!(4, add_two(2));
317 There's a few changes here. The first is the introduction of a `mod tests` with
318 a `cfg` attribute. The module allows us to group all of our tests together, and
319 to also define helper functions if needed, that don't become a part of the rest
320 of our crate. The `cfg` attribute only compiles our test code if we're
321 currently trying to run the tests. This can save compile time, and also ensures
322 that our tests are entirely left out of a normal build.
324 The second change is the `use` declaration. Because we're in an inner module,
325 we need to bring our test function into scope. This can be annoying if you have
326 a large module, and so this is a common use of globs. Let's change our
327 `src/lib.rs` to make use of it:
330 pub fn add_two(a: i32) -> i32 {
340 assert_eq!(4, add_two(2));
345 Note the different `use` line. Now we run our tests:
349 Updating registry `https://github.com/rust-lang/crates.io-index`
350 Compiling adder v0.0.1 (file:///home/you/projects/adder)
351 Running target/adder-91b3e234d4ed382a
354 test tests::it_works ... ok
356 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
362 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
367 The current convention is to use the `tests` module to hold your "unit-style"
368 tests. Anything that tests one small bit of functionality makes sense to
369 go here. But what about "integration-style" tests instead? For that, we have
370 the `tests` directory.
372 # The `tests` directory
374 To write an integration test, let's make a `tests` directory, and
375 put a `tests/lib.rs` file inside, with this as its contents:
382 assert_eq!(4, adder::add_two(2));
386 This looks similar to our previous tests, but slightly different. We now have
387 an `extern crate adder` at the top. This is because the tests in the `tests`
388 directory are an entirely separate crate, and so we need to import our library.
389 This is also why `tests` is a suitable place to write integration-style tests:
390 they use the library like any other consumer of it would.
396 Compiling adder v0.0.1 (file:///home/you/projects/adder)
397 Running target/adder-91b3e234d4ed382a
400 test tests::it_works ... ok
402 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
404 Running target/lib-c18e7d3494509e74
409 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
415 test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured
418 Now we have three sections: our previous test is also run, as well as our new
421 That's all there is to the `tests` directory. The `tests` module isn't needed
422 here, since the whole thing is focused on tests.
424 Let's finally check out that third section: documentation tests.
426 # Documentation tests
428 Nothing is better than documentation with examples. Nothing is worse than
429 examples that don't actually work, because the code has changed since the
430 documentation has been written. To this end, Rust supports automatically
431 running examples in your documentation (**note:** this only works in library
432 crates, not binary crates). Here's a fleshed-out `src/lib.rs` with examples:
435 //! The `adder` crate provides functions that add numbers to other numbers.
440 //! assert_eq!(4, adder::add_two(2));
443 /// This function adds two to its argument.
448 /// use adder::add_two;
450 /// assert_eq!(4, add_two(2));
452 pub fn add_two(a: i32) -> i32 {
462 assert_eq!(4, add_two(2));
467 Note the module-level documentation with `//!` and the function-level
468 documentation with `///`. Rust's documentation supports Markdown in comments,
469 and so triple graves mark code blocks. It is conventional to include the
470 `# Examples` section, exactly like that, with examples following.
472 Let's run the tests again:
476 Compiling adder v0.0.1 (file:///home/steve/tmp/adder)
477 Running target/adder-91b3e234d4ed382a
480 test tests::it_works ... ok
482 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
484 Running target/lib-c18e7d3494509e74
489 test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
494 test add_two_0 ... ok
497 test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured
500 Now we have all three kinds of tests running! Note the names of the
501 documentation tests: the `_0` is generated for the module test, and `add_two_0`
502 for the function test. These will auto increment with names like `add_two_1` as
503 you add more examples.
505 We haven’t covered all of the details with writing documentation tests. For more,
506 please see the [Documentation chapter](documentation.html).
508 One final note: documentation tests *cannot* be run on binary crates.
509 To see more on file arrangement see the [Crates and
510 Modules](crates-and-modules.html) section.