src/doc/book/src/ch11-03-test-organization.md

   1 ## Test Organization
   2
   3 As mentioned at the start of the chapter, testing is a complex discipline, and
   4 different people use different terminology and organization. The Rust community
   5 thinks about tests in terms of two main categories: unit tests and integration
   6 tests. *Unit tests* are small and more focused, testing one module in isolation
   7 at a time, and can test private interfaces. *Integration tests* are entirely
   8 external to your library and use your code in the same way any other external
   9 code would, using only the public interface and potentially exercising multiple
  10 modules per test.
  11
  12 Writing both kinds of tests is important to ensure that the pieces of your
  13 library are doing what you expect them to, separately and together.
  14
  15 ### Unit Tests
  16
  17 The purpose of unit tests is to test each unit of code in isolation from the
  18 rest of the code to quickly pinpoint where code is and isn’t working as
  19 expected. You’ll put unit tests in the *src* directory in each file with the
  20 code that they’re testing. The convention is to create a module named `tests`
  21 in each file to contain the test functions and to annotate the module with
  22 `cfg(test)`.
  23
  24 #### The Tests Module and `#[cfg(test)]`
  25
  26 The `#[cfg(test)]` annotation on the tests module tells Rust to compile and run
  27 the test code only when you run `cargo test`, not when you run `cargo build`.
  28 This saves compile time when you only want to build the library and saves space
  29 in the resulting compiled artifact because the tests are not included. You’ll
  30 see that because integration tests go in a different directory, they don’t need
  31 the `#[cfg(test)]` annotation. However, because unit tests go in the same files
  32 as the code, you’ll use `#[cfg(test)]` to specify that they shouldn’t be
  33 included in the compiled result.
  34
  35 Recall that when we generated the new `adder` project in the first section of
  36 this chapter, Cargo generated this code for us:
  37
  38 <span class="filename">Filename: src/lib.rs</span>
  39
  40 ```rust,noplayground
  41 {{#rustdoc_include ../listings/ch11-writing-automated-tests/listing-11-01/src/lib.rs}}
  42 ```
  43
  44 This code is the automatically generated test module. The attribute `cfg`
  45 stands for *configuration* and tells Rust that the following item should only
  46 be included given a certain configuration option. In this case, the
  47 configuration option is `test`, which is provided by Rust for compiling and
  48 running tests. By using the `cfg` attribute, Cargo compiles our test code only
  49 if we actively run the tests with `cargo test`. This includes any helper
  50 functions that might be within this module, in addition to the functions
  51 annotated with `#[test]`.
  52
  53 #### Testing Private Functions
  54
  55 There’s debate within the testing community about whether or not private
  56 functions should be tested directly, and other languages make it difficult or
  57 impossible to test private functions. Regardless of which testing ideology you
  58 adhere to, Rust’s privacy rules do allow you to test private functions.
  59 Consider the code in Listing 11-12 with the private function `internal_adder`.
  60
  61 <span class="filename">Filename: src/lib.rs</span>
  62
  63 ```rust,noplayground
  64 {{#rustdoc_include ../listings/ch11-writing-automated-tests/listing-11-12/src/lib.rs}}
  65 ```
  66
  67 <span class="caption">Listing 11-12: Testing a private function</span>
  68
  69 Note that the `internal_adder` function is not marked as `pub`. Tests are just
  70 Rust code, and the `tests` module is just another module. As we discussed in
  71 the [“Paths for Referring to an Item in the Module Tree”][paths]<!-- ignore -->
  72 section, items in child modules can use the items in their ancestor modules. In
  73 this test, we bring all of the `test` module’s parent’s items into scope with
  74 `use super::*`, and then the test can call `internal_adder`. If you don’t think
  75 private functions should be tested, there’s nothing in Rust that will compel
  76 you to do so.
  77
  78 ### Integration Tests
  79
  80 In Rust, integration tests are entirely external to your library. They use your
  81 library in the same way any other code would, which means they can only call
  82 functions that are part of your library’s public API. Their purpose is to test
  83 whether many parts of your library work together correctly. Units of code that
  84 work correctly on their own could have problems when integrated, so test
  85 coverage of the integrated code is important as well. To create integration
  86 tests, you first need a *tests* directory.
  87
  88 #### The *tests* Directory
  89
  90 We create a *tests* directory at the top level of our project directory, next
  91 to *src*. Cargo knows to look for integration test files in this directory. We
  92 can then make as many test files as we want, and Cargo will compile each of the
  93 files as an individual crate.
  94
  95 Let’s create an integration test. With the code in Listing 11-12 still in the
  96 *src/lib.rs* file, make a *tests* directory, and create a new file named
  97 *tests/integration_test.rs*. Your directory structure should look like this:
  98
  99 ```text
 100 adder
 101 ├── Cargo.lock
 102 ├── Cargo.toml
 103 ├── src
 104 │   └── lib.rs
 105 └── tests
 106     └── integration_test.rs
 107 ```
 108
 109 Enter the code in Listing 11-13 into the *tests/integration_test.rs* file:
 110
 111 <span class="filename">Filename: tests/integration_test.rs</span>
 112
 113 ```rust,ignore
 114 {{#rustdoc_include ../listings/ch11-writing-automated-tests/listing-11-13/tests/integration_test.rs}}
 115 ```
 116
 117 <span class="caption">Listing 11-13: An integration test of a function in the
 118 `adder` crate</span>
 119
 120 Each file in the `tests` directory is a separate crate, so we need to bring our
 121 library into each test crate’s scope. For that reason we add `use adder` at the
 122 top of the code, which we didn’t need in the unit tests.
 123
 124 We don’t need to annotate any code in *tests/integration_test.rs* with
 125 `#[cfg(test)]`. Cargo treats the `tests` directory specially and compiles files
 126 in this directory only when we run `cargo test`. Run `cargo test` now:
 127
 128 ```console
 129 {{#include ../listings/ch11-writing-automated-tests/listing-11-13/output.txt}}
 130 ```
 131
 132 The three sections of output include the unit tests, the integration test, and
 133 the doc tests. Note that if any test in a section fails, the following sections
 134 will not be run. For example, if a unit test fails, there won’t be any output
 135 for integration and doc tests because those tests will only be run if all unit
 136 tests are passing.
 137
 138 The first section for the unit tests is the same as we’ve been seeing: one line
 139 for each unit test (one named `internal` that we added in Listing 11-12) and
 140 then a summary line for the unit tests.
 141
 142 The integration tests section starts with the line `Running
 143 tests/integration_test.rs`. Next, there is a line for each test function in
 144 that integration test and a summary line for the results of the integration
 145 test just before the `Doc-tests adder` section starts.
 146
 147 Each integration test file has its own section, so if we add more files in the
 148 *tests* directory, there will be more integration test sections.
 149
 150 We can still run a particular integration test function by specifying the test
 151 function’s name as an argument to `cargo test`. To run all the tests in a
 152 particular integration test file, use the `--test` argument of `cargo test`
 153 followed by the name of the file:
 154
 155 ```console
 156 {{#include ../listings/ch11-writing-automated-tests/output-only-05-single-integration/output.txt}}
 157 ```
 158
 159 This command runs only the tests in the *tests/integration_test.rs* file.
 160
 161 #### Submodules in Integration Tests
 162
 163 As you add more integration tests, you might want to make more files in the
 164 *tests* directory to help organize them; for example, you can group the test
 165 functions by the functionality they’re testing. As mentioned earlier, each file
 166 in the *tests* directory is compiled as its own separate crate, which is useful
 167 for creating separate scopes to more closely imitate the way end users will be
 168 using your crate. However, this means files in the *tests* directory don’t
 169 share the same behavior as files in *src* do, as you learned in Chapter 7
 170 regarding how to separate code into modules and files.
 171
 172 The different behavior of *tests* directory files is most noticeable when you
 173 have a set of helper functions to use in multiple integration test files and
 174 you try to follow the steps in the [“Separating Modules into Different
 175 Files”][separating-modules-into-files]<!-- ignore --> section of Chapter 7 to
 176 extract them into a common module. For example, if we create *tests/common.rs*
 177 and place a function named `setup` in it, we can add some code to `setup` that
 178 we want to call from multiple test functions in multiple test files:
 179
 180 <span class="filename">Filename: tests/common.rs</span>
 181
 182 ```rust,noplayground
 183 {{#rustdoc_include ../listings/ch11-writing-automated-tests/no-listing-12-shared-test-code-problem/tests/common.rs}}
 184 ```
 185
 186 When we run the tests again, we’ll see a new section in the test output for the
 187 *common.rs* file, even though this file doesn’t contain any test functions nor
 188 did we call the `setup` function from anywhere:
 189
 190 ```console
 191 {{#include ../listings/ch11-writing-automated-tests/no-listing-12-shared-test-code-problem/output.txt}}
 192 ```
 193
 194 Having `common` appear in the test results with `running 0 tests` displayed for
 195 it is not what we wanted. We just wanted to share some code with the other
 196 integration test files.
 197
 198 To avoid having `common` appear in the test output, instead of creating
 199 *tests/common.rs*, we’ll create *tests/common/mod.rs*. The project directory
 200 now looks like this:
 201
 202 ```text
 203 ├── Cargo.lock
 204 ├── Cargo.toml
 205 ├── src
 206 │   └── lib.rs
 207 └── tests
 208     ├── common
 209     │   └── mod.rs
 210     └── integration_test.rs
 211 ```
 212
 213 This is the older naming convention that Rust also understands that we
 214 mentioned in the [“Alternate File Paths”][alt-paths]<!-- ignore --> section of
 215 Chapter 7. Naming the file this way tells Rust not to treat the `common` module
 216 as an integration test file. When we move the `setup` function code into
 217 *tests/common/mod.rs* and delete the *tests/common.rs* file, the section in the
 218 test output will no longer appear. Files in subdirectories of the *tests*
 219 directory don’t get compiled as separate crates or have sections in the test
 220 output.
 221
 222 After we’ve created *tests/common/mod.rs*, we can use it from any of the
 223 integration test files as a module. Here’s an example of calling the `setup`
 224 function from the `it_adds_two` test in *tests/integration_test.rs*:
 225
 226 <span class="filename">Filename: tests/integration_test.rs</span>
 227
 228 ```rust,ignore
 229 {{#rustdoc_include ../listings/ch11-writing-automated-tests/no-listing-13-fix-shared-test-code-problem/tests/integration_test.rs}}
 230 ```
 231
 232 Note that the `mod common;` declaration is the same as the module declaration
 233 we demonstrated in Listing 7-21. Then in the test function, we can call the
 234 `common::setup()` function.
 235
 236 #### Integration Tests for Binary Crates
 237
 238 If our project is a binary crate that only contains a *src/main.rs* file and
 239 doesn’t have a *src/lib.rs* file, we can’t create integration tests in the
 240 *tests* directory and bring functions defined in the *src/main.rs* file into
 241 scope with a `use` statement. Only library crates expose functions that other
 242 crates can use; binary crates are meant to be run on their own.
 243
 244 This is one of the reasons Rust projects that provide a binary have a
 245 straightforward *src/main.rs* file that calls logic that lives in the
 246 *src/lib.rs* file. Using that structure, integration tests *can* test the
 247 library crate with `use` to make the important functionality available.
 248 If the important functionality works, the small amount of code in the
 249 *src/main.rs* file will work as well, and that small amount of code doesn’t
 250 need to be tested.
 251
 252 ## Summary
 253
 254 Rust’s testing features provide a way to specify how code should function to
 255 ensure it continues to work as you expect, even as you make changes. Unit tests
 256 exercise different parts of a library separately and can test private
 257 implementation details. Integration tests check that many parts of the library
 258 work together correctly, and they use the library’s public API to test the code
 259 in the same way external code will use it. Even though Rust’s type system and
 260 ownership rules help prevent some kinds of bugs, tests are still important to
 261 reduce logic bugs having to do with how your code is expected to behave.
 262
 263 Let’s combine the knowledge you learned in this chapter and in previous
 264 chapters to work on a project!
 265
 266 [paths]: ch07-03-paths-for-referring-to-an-item-in-the-module-tree.html
 267 [separating-modules-into-files]:
 268 ch07-05-separating-modules-into-different-files.html
 269 [alt-paths]: ch07-05-separating-modules-into-different-files.html#alternate-file-paths