src/doc/rustdoc/src/documentation-tests.md

   1 # Documentation tests
   2
   3 `rustdoc` supports executing your documentation examples as tests. This makes sure
   4 that examples within your documentation are up to date and working.
   5
   6 The basic idea is this:
   7
   8 ```ignore
   9 /// # Examples
  10 ///
  11 /// ```
  12 /// let x = 5;
  13 /// ```
  14 ```
  15
  16 The triple backticks start and end code blocks. If this were in a file named `foo.rs`,
  17 running `rustdoc --test foo.rs` will extract this example, and then run it as a test.
  18
  19 Please note that by default, if no language is set for the block code, `rustdoc`
  20 assumes it is `Rust` code. So the following:
  21
  22 ``````markdown
  23 ```rust
  24 let x = 5;
  25 ```
  26 ``````
  27
  28 is strictly equivalent to:
  29
  30 ``````markdown
  31 ```
  32 let x = 5;
  33 ```
  34 ``````
  35
  36 There's some subtlety though! Read on for more details.
  37
  38 ## Passing or failing a doctest
  39
  40 Like regular unit tests, regular doctests are considered to "pass"
  41 if they compile and run without panicking.
  42 So if you want to demonstrate that some computation gives a certain result,
  43 the `assert!` family of macros works the same as other Rust code:
  44
  45 ```rust
  46 let foo = "foo";
  47
  48 assert_eq!(foo, "foo");
  49 ```
  50
  51 This way, if the computation ever returns something different,
  52 the code panics and the doctest fails.
  53
  54 ## Pre-processing examples
  55
  56 In the example above, you'll note something strange: there's no `main`
  57 function! Forcing you to write `main` for every example, no matter how small,
  58 adds friction. So `rustdoc` processes your examples slightly before
  59 running them. Here's the full algorithm rustdoc uses to preprocess examples:
  60
  61 1. Some common `allow` attributes are inserted, including
  62    `unused_variables`, `unused_assignments`, `unused_mut`,
  63    `unused_attributes`, and `dead_code`. Small examples often trigger
  64    these lints.
  65 2. Any attributes specified with `#![doc(test(attr(...)))]` are added.
  66 3. Any leading `#![foo]` attributes are left intact as crate attributes.
  67 4. If the example does not contain `extern crate`, and
  68    `#![doc(test(no_crate_inject))]` was not specified, then `extern crate
  69    <mycrate>;` is inserted (note the lack of `#[macro_use]`).
  70 5. Finally, if the example does not contain `fn main`, the remainder of the
  71    text is wrapped in `fn main() { your_code }`.
  72
  73 For more about that caveat in rule 4, see "Documenting Macros" below.
  74
  75 ## Hiding portions of the example
  76
  77 Sometimes, you need some setup code, or other things that would distract
  78 from your example, but are important to make the tests work. Consider
  79 an example block that looks like this:
  80
  81 ```text
  82 /// /// Some documentation.
  83 /// # fn foo() {} // this function will be hidden
  84 /// println!("Hello, World!");
  85 ```
  86
  87 It will render like this:
  88
  89 ```rust
  90 /// Some documentation.
  91 # fn foo() {}
  92 println!("Hello, World!");
  93 ```
  94
  95 Yes, that's right: you can add lines that start with `# `, and they will
  96 be hidden from the output, but will be used when compiling your code. You
  97 can use this to your advantage. In this case, documentation comments need
  98 to apply to some kind of function, so if I want to show you just a
  99 documentation comment, I need to add a little function definition below
 100 it. At the same time, it's only there to satisfy the compiler, so hiding
 101 it makes the example more clear. You can use this technique to explain
 102 longer examples in detail, while still preserving the testability of your
 103 documentation.
 104
 105 For example, imagine that we wanted to document this code:
 106
 107 ```rust
 108 let x = 5;
 109 let y = 6;
 110 println!("{}", x + y);
 111 ```
 112
 113 We might want the documentation to end up looking like this:
 114
 115 > First, we set `x` to five:
 116 >
 117 > ```rust
 118 > let x = 5;
 119 > # let y = 6;
 120 > # println!("{}", x + y);
 121 > ```
 122 >
 123 > Next, we set `y` to six:
 124 >
 125 > ```rust
 126 > # let x = 5;
 127 > let y = 6;
 128 > # println!("{}", x + y);
 129 > ```
 130 >
 131 > Finally, we print the sum of `x` and `y`:
 132 >
 133 > ```rust
 134 > # let x = 5;
 135 > # let y = 6;
 136 > println!("{}", x + y);
 137 > ```
 138
 139 To keep each code block testable, we want the whole program in each block, but
 140 we don't want the reader to see every line every time.  Here's what we put in
 141 our source code:
 142
 143 ``````markdown
 144 First, we set `x` to five:
 145
 146 ```
 147 let x = 5;
 148 # let y = 6;
 149 # println!("{}", x + y);
 150 ```
 151
 152 Next, we set `y` to six:
 153
 154 ```
 155 # let x = 5;
 156 let y = 6;
 157 # println!("{}", x + y);
 158 ```
 159
 160 Finally, we print the sum of `x` and `y`:
 161
 162 ```
 163 # let x = 5;
 164 # let y = 6;
 165 println!("{}", x + y);
 166 ```
 167 ``````
 168
 169 By repeating all parts of the example, you can ensure that your example still
 170 compiles, while only showing the parts that are relevant to that part of your
 171 explanation.
 172
 173 The `#`-hiding of lines can be prevented by using two consecutive hashes
 174 `##`. This only needs to be done with the first `#` which would've
 175 otherwise caused hiding. If we have a string literal like the following,
 176 which has a line that starts with a `#`:
 177
 178 ```rust
 179 let s = "foo
 180 ## bar # baz";
 181 ```
 182
 183 We can document it by escaping the initial `#`:
 184
 185 ```text
 186 /// let s = "foo
 187 /// ## bar # baz";
 188 ```
 189
 190
 191 ## Using `?` in doc tests
 192
 193 When writing an example, it is rarely useful to include a complete error
 194 handling, as it would add significant amounts of boilerplate code. Instead, you
 195 may want the following:
 196
 197 ```ignore
 198 /// ```
 199 /// use std::io;
 200 /// let mut input = String::new();
 201 /// io::stdin().read_line(&mut input)?;
 202 /// ```
 203 ```
 204
 205 The problem is that `?` returns a `Result<T, E>` and test functions don't
 206 return anything, so this will give a mismatched types error.
 207
 208 You can get around this limitation by manually adding a `main` that returns
 209 `Result<T, E>`, because `Result<T, E>` implements the `Termination` trait:
 210
 211 ```ignore
 212 /// A doc test using ?
 213 ///
 214 /// ```
 215 /// use std::io;
 216 ///
 217 /// fn main() -> io::Result<()> {
 218 ///     let mut input = String::new();
 219 ///     io::stdin().read_line(&mut input)?;
 220 ///     Ok(())
 221 /// }
 222 /// ```
 223 ```
 224
 225 Together with the `# ` from the section above, you arrive at a solution that
 226 appears to the reader as the initial idea but works with doc tests:
 227
 228 ```ignore
 229 /// ```
 230 /// use std::io;
 231 /// # fn main() -> io::Result<()> {
 232 /// let mut input = String::new();
 233 /// io::stdin().read_line(&mut input)?;
 234 /// # Ok(())
 235 /// # }
 236 /// ```
 237 ```
 238
 239 As of version 1.34.0, one can also omit the `fn main()`, but you will have to
 240 disambiguate the error type:
 241
 242 ```ignore
 243 /// ```
 244 /// use std::io;
 245 /// let mut input = String::new();
 246 /// io::stdin().read_line(&mut input)?;
 247 /// # Ok::<(), io::Error>(())
 248 /// ```
 249 ```
 250
 251 This is an unfortunate consequence of the `?` operator adding an implicit
 252 conversion, so type inference fails because the type is not unique. Please note
 253 that you must write the `(())` in one sequence without intermediate whitespace
 254 so that rustdoc understands you want an implicit `Result`-returning function.
 255
 256 ## Documenting macros
 257
 258 Here’s an example of documenting a macro:
 259
 260 ```rust
 261 /// Panic with a given message unless an expression evaluates to true.
 262 ///
 263 /// # Examples
 264 ///
 265 /// ```
 266 /// # #[macro_use] extern crate foo;
 267 /// # fn main() {
 268 /// panic_unless!(1 + 1 == 2, “Math is broken.”);
 269 /// # }
 270 /// ```
 271 ///
 272 /// ```should_panic
 273 /// # #[macro_use] extern crate foo;
 274 /// # fn main() {
 275 /// panic_unless!(true == false, “I’m broken.”);
 276 /// # }
 277 /// ```
 278 #[macro_export]
 279 macro_rules! panic_unless {
 280     ($condition:expr, $($rest:expr),+) => ({ if ! $condition { panic!($($rest),+); } });
 281 }
 282 # fn main() {}
 283 ```
 284
 285 You’ll note three things: we need to add our own `extern crate` line, so that
 286 we can add the `#[macro_use]` attribute. Second, we’ll need to add our own
 287 `main()` as well (for reasons discussed above). Finally, a judicious use of
 288 `#` to comment out those two things, so they don’t show up in the output.
 289
 290 ## Attributes
 291
 292 There are a few annotations that are useful to help `rustdoc` do the right
 293 thing when testing your code:
 294
 295 ```rust
 296 /// ```ignore
 297 /// fn foo() {
 298 /// ```
 299 # fn foo() {}
 300 ```
 301
 302 The `ignore` directive tells Rust to ignore your code. This is almost never
 303 what you want, as it's the most generic. Instead, consider annotating it
 304 with `text` if it's not code, or using `#`s to get a working example that
 305 only shows the part you care about.
 306
 307 ```rust
 308 /// ```should_panic
 309 /// assert!(false);
 310 /// ```
 311 # fn foo() {}
 312 ```
 313
 314 `should_panic` tells `rustdoc` that the code should compile correctly, but
 315 not actually pass as a test.
 316
 317 ```rust
 318 /// ```no_run
 319 /// loop {
 320 ///     println!("Hello, world");
 321 /// }
 322 /// ```
 323 # fn foo() {}
 324 ```
 325
 326 The `no_run` attribute will compile your code, but not run it. This is
 327 important for examples such as "Here's how to retrieve a web page,"
 328 which you would want to ensure compiles, but might be run in a test
 329 environment that has no network access.
 330
 331 ```text
 332 /// ```compile_fail
 333 /// let x = 5;
 334 /// x += 2; // shouldn't compile!
 335 /// ```
 336 ```
 337
 338 `compile_fail` tells `rustdoc` that the compilation should fail. If it
 339 compiles, then the test will fail. However please note that code failing
 340 with the current Rust release may work in a future release, as new features
 341 are added.
 342
 343 ```text
 344 /// Only runs on the 2018 edition.
 345 ///
 346 /// ```edition2018
 347 /// let result: Result<i32, ParseIntError> = try {
 348 ///     "1".parse::<i32>()?
 349 ///         + "2".parse::<i32>()?
 350 ///         + "3".parse::<i32>()?
 351 /// };
 352 /// ```
 353 ```
 354
 355 `edition2018` tells `rustdoc` that the code sample should be compiled using
 356 the 2018 edition of Rust. Similarly, you can specify `edition2015` to compile
 357 the code with the 2015 edition.
 358
 359 ## Syntax reference
 360
 361 The *exact* syntax for code blocks, including the edge cases, can be found
 362 in the [Fenced Code Blocks](https://spec.commonmark.org/0.28/#fenced-code-blocks)
 363 section of the CommonMark specification.
 364
 365 Rustdoc also accepts *indented* code blocks as an alternative to fenced
 366 code blocks: instead of surrounding your code with three backticks, you
 367 can indent each line by four or more spaces.
 368
 369 ``````markdown
 370     let foo = "foo";
 371     assert_eq!(foo, "foo");
 372 ``````
 373
 374 These, too, are documented in the CommonMark specification, in the
 375 [Indented Code Blocks](https://spec.commonmark.org/0.28/#indented-code-blocks)
 376 section.
 377
 378 However, it's preferable to use fenced code blocks over indented code blocks.
 379 Not only are fenced code blocks considered more idiomatic for Rust code,
 380 but there is no way to use directives such as `ignore` or `should_panic` with
 381 indented code blocks.
 382
 383 ### Include items only when collecting doctests
 384
 385 Rustdoc's documentation tests can do some things that regular unit tests can't, so it can
 386 sometimes be useful to extend your doctests with samples that wouldn't otherwise need to be in
 387 documentation. To this end, Rustdoc allows you to have certain items only appear when it's
 388 collecting doctests, so you can utilize doctest functionality without forcing the test to appear in
 389 docs, or to find an arbitrary private item to include it on.
 390
 391 When compiling a crate for use in doctests (with `--test` option), rustdoc will set `cfg(doctest)`.
 392 Note that they will still link against only the public items of your crate; if you need to test
 393 private items, you need to write a unit test.
 394
 395 In this example, we're adding doctests that we know won't compile, to verify that our struct can
 396 only take in valid data:
 397
 398 ```rust
 399 /// We have a struct here. Remember it doesn't accept negative numbers!
 400 pub struct MyStruct(pub usize);
 401
 402 /// ```compile_fail
 403 /// let x = my_crate::MyStruct(-5);
 404 /// ```
 405 #[cfg(doctest)]
 406 pub struct MyStructOnlyTakesUsize;
 407 ```
 408
 409 Note that the struct `MyStructOnlyTakesUsize` here isn't actually part of your public crate
 410 API. The use of `#[cfg(doctest)]` makes sure that this struct only exists while rustdoc is
 411 collecting doctests. This means that its doctest is executed when `--test` is passed to rustdoc,
 412 but is hidden from the public documentation.
 413
 414 Another possible use of `cfg(doctest)` is to test doctests that are included in your README file
 415 without including it in your main documentation. For example, you could write this into your
 416 `lib.rs` to test your README as part of your doctests:
 417
 418 ```rust,ignore
 419 #![feature(extern_doc)]
 420
 421 #[doc(include="../README.md")]
 422 #[cfg(doctest)]
 423 pub struct ReadmeDoctests;
 424 ```
 425
 426 This will include your README as documentation on the hidden struct `ReadmeDoctests`, which will
 427 then be tested alongside the rest of your doctests.