src/doc/book/src/ch11-03-test-organization.md

   1 ## Test Organization
   2
   3 As mentioned at the start of the chapter, testing is a complex discipline, and
   4 different people use different terminology and organization. The Rust community
   5 thinks about tests in terms of two main categories: *unit tests* and
   6 *integration tests*. Unit tests are small and more focused, testing one module
   7 in isolation at a time, and can test private interfaces. Integration tests are
   8 entirely external to your library and use your code in the same way any other
   9 external code would, using only the public interface and potentially exercising
  10 multiple modules per test.
  11
  12 Writing both kinds of tests is important to ensure that the pieces of your
  13 library are doing what you expect them to, separately and together.
  14
  15 ### Unit Tests
  16
  17 The purpose of unit tests is to test each unit of code in isolation from the
  18 rest of the code to quickly pinpoint where code is and isn’t working as
  19 expected. You’ll put unit tests in the *src* directory in each file with the
  20 code that they’re testing. The convention is to create a module named `tests`
  21 in each file to contain the test functions and to annotate the module with
  22 `cfg(test)`.
  23
  24 #### The Tests Module and `#[cfg(test)]`
  25
  26 The `#[cfg(test)]` annotation on the tests module tells Rust to compile and run
  27 the test code only when you run `cargo test`, not when you run `cargo build`.
  28 This saves compile time when you only want to build the library and saves space
  29 in the resulting compiled artifact because the tests are not included. You’ll
  30 see that because integration tests go in a different directory, they don’t need
  31 the `#[cfg(test)]` annotation. However, because unit tests go in the same files
  32 as the code, you’ll use `#[cfg(test)]` to specify that they shouldn’t be
  33 included in the compiled result.
  34
  35 Recall that when we generated the new `adder` project in the first section of
  36 this chapter, Cargo generated this code for us:
  37
  38 <span class="filename">Filename: src/lib.rs</span>
  39
  40 ```rust
  41 {{#rustdoc_include ../listings/ch11-writing-automated-tests/listing-11-01/src/lib.rs:here}}
  42 ```
  43
  44 This code is the automatically generated test module. The attribute `cfg`
  45 stands for *configuration* and tells Rust that the following item should only
  46 be included given a certain configuration option. In this case, the
  47 configuration option is `test`, which is provided by Rust for compiling and
  48 running tests. By using the `cfg` attribute, Cargo compiles our test code only
  49 if we actively run the tests with `cargo test`. This includes any helper
  50 functions that might be within this module, in addition to the functions
  51 annotated with `#[test]`.
  52
  53 #### Testing Private Functions
  54
  55 There’s debate within the testing community about whether or not private
  56 functions should be tested directly, and other languages make it difficult or
  57 impossible to test private functions. Regardless of which testing ideology you
  58 adhere to, Rust’s privacy rules do allow you to test private functions.
  59 Consider the code in Listing 11-12 with the private function `internal_adder`.
  60
  61 <span class="filename">Filename: src/lib.rs</span>
  62
  63 ```rust
  64 {{#rustdoc_include ../listings/ch11-writing-automated-tests/listing-11-12/src/lib.rs:here}}
  65 ```
  66
  67 <span class="caption">Listing 11-12: Testing a private function</span>
  68
  69 Note that the `internal_adder` function is not marked as `pub`, but because
  70 tests are just Rust code and the `tests` module is just another module, you can
  71 bring `internal_adder` into a test’s scope and call it. If you don’t think
  72 private functions should be tested, there’s nothing in Rust that will compel
  73 you to do so.
  74
  75 ### Integration Tests
  76
  77 In Rust, integration tests are entirely external to your library. They use your
  78 library in the same way any other code would, which means they can only call
  79 functions that are part of your library’s public API. Their purpose is to test
  80 whether many parts of your library work together correctly. Units of code that
  81 work correctly on their own could have problems when integrated, so test
  82 coverage of the integrated code is important as well. To create integration
  83 tests, you first need a *tests* directory.
  84
  85 #### The *tests* Directory
  86
  87 We create a *tests* directory at the top level of our project directory, next
  88 to *src*. Cargo knows to look for integration test files in this directory. We
  89 can then make as many test files as we want to in this directory, and Cargo
  90 will compile each of the files as an individual crate.
  91
  92 Let’s create an integration test. With the code in Listing 11-12 still in the
  93 *src/lib.rs* file, make a *tests* directory, create a new file named
  94 *tests/integration_test.rs*, and enter the code in Listing 11-13.
  95
  96 <span class="filename">Filename: tests/integration_test.rs</span>
  97
  98 ```rust,ignore
  99 {{#rustdoc_include ../listings/ch11-writing-automated-tests/listing-11-13/tests/integration_test.rs}}
 100 ```
 101
 102 <span class="caption">Listing 11-13: An integration test of a function in the
 103 `adder` crate</span>
 104
 105 We’ve added `use adder` at the top of the code, which we didn’t need in the
 106 unit tests. The reason is that each file in the `tests` directory is a separate
 107 crate, so we need to bring our library into each test crate’s scope.
 108
 109 We don’t need to annotate any code in *tests/integration_test.rs* with
 110 `#[cfg(test)]`. Cargo treats the `tests` directory specially and compiles files
 111 in this directory only when we run `cargo test`. Run `cargo test` now:
 112
 113 ```console
 114 {{#include ../listings/ch11-writing-automated-tests/listing-11-13/output.txt}}
 115 ```
 116
 117 The three sections of output include the unit tests, the integration test, and
 118 the doc tests. The first section for the unit tests is the same as we’ve been
 119 seeing: one line for each unit test (one named `internal` that we added in
 120 Listing 11-12) and then a summary line for the unit tests.
 121
 122 The integration tests section starts with the line `Running
 123 target/debug/deps/integration_test-ce99bcc2479f4607` (the hash at the end of
 124 your output will be different). Next, there is a line for each test function in
 125 that integration test and a summary line for the results of the integration
 126 test just before the `Doc-tests adder` section starts.
 127
 128 Similarly to how adding more unit test functions adds more result lines to the
 129 unit tests section, adding more test functions to the integration test file
 130 adds more result lines to this integration test file’s section. Each
 131 integration test file has its own section, so if we add more files in the
 132 *tests* directory, there will be more integration test sections.
 133
 134 We can still run a particular integration test function by specifying the test
 135 function’s name as an argument to `cargo test`. To run all the tests in a
 136 particular integration test file, use the `--test` argument of `cargo test`
 137 followed by the name of the file:
 138
 139 ```console
 140 {{#include ../listings/ch11-writing-automated-tests/output-only-05-single-integration/output.txt}}
 141 ```
 142
 143 This command runs only the tests in the *tests/integration_test.rs* file.
 144
 145 #### Submodules in Integration Tests
 146
 147 As you add more integration tests, you might want to make more than one file in
 148 the *tests* directory to help organize them; for example, you can group the
 149 test functions by the functionality they’re testing. As mentioned earlier, each
 150 file in the *tests* directory is compiled as its own separate crate.
 151
 152 Treating each integration test file as its own crate is useful to create
 153 separate scopes that are more like the way end users will be using your crate.
 154 However, this means files in the *tests* directory don’t share the same
 155 behavior as files in *src* do, as you learned in Chapter 7 regarding how to
 156 separate code into modules and files.
 157
 158 The different behavior of files in the *tests* directory is most noticeable
 159 when you have a set of helper functions that would be useful in multiple
 160 integration test files and you try to follow the steps in the [“Separating
 161 Modules into Different Files”][separating-modules-into-files]<!-- ignore -->
 162 section of Chapter 7 to extract them into a common module. For example, if we
 163 create *tests/common.rs* and place a function named `setup` in it, we can add
 164 some code to `setup` that we want to call from multiple test functions in
 165 multiple test files:
 166
 167 <span class="filename">Filename: tests/common.rs</span>
 168
 169 ```rust
 170 {{#rustdoc_include ../listings/ch11-writing-automated-tests/no-listing-12-shared-test-code-problem/tests/common.rs}}
 171 ```
 172
 173 When we run the tests again, we’ll see a new section in the test output for the
 174 *common.rs* file, even though this file doesn’t contain any test functions nor
 175 did we call the `setup` function from anywhere:
 176
 177 ```console
 178 {{#include ../listings/ch11-writing-automated-tests/no-listing-12-shared-test-code-problem/output.txt}}
 179 ```
 180
 181 Having `common` appear in the test results with `running 0 tests` displayed for
 182 it is not what we wanted. We just wanted to share some code with the other
 183 integration test files.
 184
 185 To avoid having `common` appear in the test output, instead of creating
 186 *tests/common.rs*, we’ll create *tests/common/mod.rs*. This is an alternate
 187 naming convention that Rust also understands. Naming the file this way tells
 188 Rust not to treat the `common` module as an integration test file. When we move
 189 the `setup` function code into *tests/common/mod.rs* and delete the
 190 *tests/common.rs* file, the section in the test output will no longer appear.
 191 Files in subdirectories of the *tests* directory don’t get compiled as separate
 192 crates or have sections in the test output.
 193
 194 After we’ve created *tests/common/mod.rs*, we can use it from any of the
 195 integration test files as a module. Here’s an example of calling the `setup`
 196 function from the `it_adds_two` test in *tests/integration_test.rs*:
 197
 198 <span class="filename">Filename: tests/integration_test.rs</span>
 199
 200 ```rust,ignore
 201 {{#rustdoc_include ../listings/ch11-writing-automated-tests/no-listing-13-fix-shared-test-code-problem/tests/integration_test.rs}}
 202 ```
 203
 204 Note that the `mod common;` declaration is the same as the module declaration
 205 we demonstrated in Listing 7-21. Then in the test function, we can call the
 206 `common::setup()` function.
 207
 208 #### Integration Tests for Binary Crates
 209
 210 If our project is a binary crate that only contains a *src/main.rs* file and
 211 doesn’t have a *src/lib.rs* file, we can’t create integration tests in the
 212 *tests* directory and bring functions defined in the *src/main.rs* file into
 213 scope with a `use` statement. Only library crates expose functions that other
 214 crates can use; binary crates are meant to be run on their own.
 215
 216 This is one of the reasons Rust projects that provide a binary have a
 217 straightforward *src/main.rs* file that calls logic that lives in the
 218 *src/lib.rs* file. Using that structure, integration tests *can* test the
 219 library crate with `use` to make the important functionality available.
 220 If the important functionality works, the small amount of code in the
 221 *src/main.rs* file will work as well, and that small amount of code doesn’t
 222 need to be tested.
 223
 224 ## Summary
 225
 226 Rust’s testing features provide a way to specify how code should function to
 227 ensure it continues to work as you expect, even as you make changes. Unit tests
 228 exercise different parts of a library separately and can test private
 229 implementation details. Integration tests check that many parts of the library
 230 work together correctly, and they use the library’s public API to test the code
 231 in the same way external code will use it. Even though Rust’s type system and
 232 ownership rules help prevent some kinds of bugs, tests are still important to
 233 reduce logic bugs having to do with how your code is expected to behave.
 234
 235 Let’s combine the knowledge you learned in this chapter and in previous
 236 chapters to work on a project!
 237
 238 [separating-modules-into-files]:
 239 ch07-05-separating-modules-into-different-files.html