src/doc/rustc-dev-guide/src/rustdoc.md

   1 # Rustdoc overview
   2
   3 `rustdoc` uses `rustc` internals (and, of course, the standard library), so you
   4 will have to build the compiler and `std` once before you can build `rustdoc`.
   5
   6 `rustdoc` lives in-tree with the
   7 compiler and standard library. This chapter is about how it works.
   8 For information about Rustdoc's features and how to use them, see
   9 the [Rustdoc book](https://doc.rust-lang.org/nightly/rustdoc/).
  10 For more details about how rustdoc works, see the
  11 ["Rustdoc internals" chapter][Rustdoc internals].
  12
  13 [Rustdoc internals]: ./rustdoc-internals.md
  14
  15 Rustdoc is implemented entirely within the crate [`librustdoc`][rd]. It runs
  16 the compiler up to the point where we have an internal representation of a
  17 crate (HIR) and the ability to run some queries about the types of items. [HIR]
  18 and [queries] are discussed in the linked chapters.
  19
  20 [HIR]: ./hir.md
  21 [queries]: ./query.md
  22 [rd]: https://github.com/rust-lang/rust/tree/master/src/librustdoc
  23
  24 `librustdoc` performs two major steps after that to render a set of
  25 documentation:
  26
  27 * "Clean" the AST into a form that's more suited to creating documentation (and
  28   slightly more resistant to churn in the compiler).
  29 * Use this cleaned AST to render a crate's documentation, one page at a time.
  30
  31 Naturally, there's more than just this, and those descriptions simplify out
  32 lots of details, but that's the high-level overview.
  33
  34 (Side note: `librustdoc` is a library crate! The `rustdoc` binary is created
  35 using the project in [`src/tools/rustdoc`][bin]. Note that literally all that
  36 does is call the `main()` that's in this crate's `lib.rs`, though.)
  37
  38 [bin]: https://github.com/rust-lang/rust/tree/master/src/tools/rustdoc
  39
  40 ## Cheat sheet
  41
  42 * Run `./x.py setup tools` before getting started. This will configure `x.py`
  43   with nice settings for developing rustdoc and other tools, including
  44   downloading a copy of rustc rather than building it.
  45 * Use `./x.py check src/tools/rustdoc` to quickly check for compile errors.
  46 * Use `./x.py build` to make a usable
  47   rustdoc you can run on other projects.
  48   * Add `library/test` to be able to use `rustdoc --test`.
  49   * Run `rustup toolchain link stage2 build/$TARGET/stage2` to add a
  50     custom toolchain called `stage2` to your rustup environment. After
  51     running that, `cargo +stage2 doc` in any directory will build with
  52     your locally-compiled rustdoc.
  53 * Use `./x.py doc library` to use this rustdoc to generate the
  54   standard library docs.
  55   * The completed docs will be available in `build/$TARGET/doc` (under `core`, `alloc`, and `std`).
  56   * If you want to copy those docs to a webserver, copy all of
  57     `build/$TARGET/doc`, since that's where the CSS, JS, fonts, and landing
  58     page are.
  59 * Use `./x.py test src/test/rustdoc*` to run the tests using a stage1
  60   rustdoc.
  61   * See [Rustdoc internals] for more information about tests.
  62
  63 ## Code structure
  64
  65 * All paths in this section are relative to `src/librustdoc` in the rust-lang/rust repository.
  66 * Most of the HTML printing code is in `html/format.rs` and `html/render/mod.rs`.
  67   It's in a bunch of `fmt::Display` implementations and supplementary
  68   functions.
  69 * The types that got `Display` impls above are defined in `clean/mod.rs`, right
  70   next to the custom `Clean` trait used to process them out of the rustc HIR.
  71 * The bits specific to using rustdoc as a test harness are in
  72   `doctest.rs`.
  73 * The Markdown renderer is loaded up in `html/markdown.rs`, including functions
  74   for extracting doctests from a given block of Markdown.
  75 * The tests on the structure of rustdoc HTML output are located in `src/test/rustdoc`, where
  76   they're handled by the test runner of rustbuild and the supplementary script
  77   `src/etc/htmldocck.py`.
  78
  79 ## Tests
  80
  81 * All paths in this section are relative to `src/test` in the rust-lang/rust repository.
  82 * Tests on search index generation are located in `rustdoc-js`, as a
  83   series of JavaScript files that encode queries on the standard library search
  84   index and expected results.
  85 * Tests on the "UI" of rustdoc (the terminal output it produces when run) are in
  86   `rustdoc-ui`
  87 * Tests on the "GUI" of rustdoc (the HTML, JS, and CSS as rendered in a browser)
  88   are in `rustdoc-gui`. These use a [NodeJS tool called
  89   browser-UI-test](https://github.com/GuillaumeGomez/browser-UI-test/) that uses
  90   puppeteer to run tests in a headless browser and check rendering and
  91   interactivity.
  92
  93 ## Constraints
  94
  95 We try to make rustdoc work reasonably well with JavaScript disabled, and when
  96 browsing local files. We support
  97 [these browsers](https://rust-lang.github.io/rfcs/1985-tiered-browser-support.html#supported-browsers).
  98
  99 Supporting local files (`file:///` URLs) brings some surprising restrictions.
 100 Certain browser features that require secure origins, like `localStorage` and
 101 Service Workers, don't work reliably. We can still use such features but we
 102 should make sure pages are still usable without them.
 103
 104 ## Multiple runs, same output directory
 105
 106 Rustdoc can be run multiple times for varying inputs, with its output set to the
 107 same directory. That's how cargo produces documentation for dependencies of the
 108 current crate. It can also be done manually if a user wants a big
 109 documentation bundle with all of the docs they care about.
 110
 111 HTML is generated independently for each crate, but there is some cross-crate
 112 information that we update as we add crates to the output directory:
 113
 114  - `crates<SUFFIX>.js` holds a list of all crates in the output directory.
 115  - `search-index<SUFFIX>.js` holds a list of all searchable items.
 116  - For each trait, there is a file under `implementors/.../trait.TraitName.js`
 117    containing a list of implementors of that trait. The implementors may be in
 118    different crates than the trait, and the JS file is updated as we discover
 119    new ones.
 120
 121 ## Use cases
 122
 123 There are a few major use cases for rustdoc that you should keep in mind when
 124 working on it:
 125
 126 ### Standard library docs
 127
 128 These are published at <https://doc.rust-lang.org/std> as part of the Rust release
 129 process. Stable releases are also uploaded to specific versioned URLs like
 130 <https://doc.rust-lang.org/1.57.0/std/>. Beta and nightly docs are published to
 131 <https://doc.rust-lang.org/beta/std/> and <https://doc.rust-lang.org/nightly/std/>.
 132 The docs are uploaded with the [promote-release
 133 tool](https://github.com/rust-lang/promote-release) and served from S3 with
 134 CloudFront.
 135
 136 The standard library docs contain five crates: alloc, core, proc_macro, std, and
 137 test.
 138
 139 ### docs.rs
 140
 141 When crates are published to crates.io, docs.rs automatically builds
 142 and publishes their documentation, for instance at
 143 <https://docs.rs/serde/latest/serde/>. It always builds with the current nightly
 144 rustdoc, so any changes you land in rustdoc are "insta-stable" in that they will
 145 have an immediate public effect on docs.rs. Old documentation is not rebuilt, so
 146 you will see some variation in UI when browsing old releases in docs.rs. Crate
 147 authors can request rebuilds, which will be run with the latest rustdoc.
 148
 149 Docs.rs performs some transformations on rustdoc's output in order to save
 150 storage and display a navigation bar at the top. In particular, certain static
 151 files (like main.js and rustdoc.css may be shared across multiple invocations
 152 of the same version of rustdoc. Others, like crates.js and sidebar-items.js, are
 153 different for different invocations. Still others, like fonts, will never
 154 change. These categories are distinguished using the `SharedResource` enum in
 155 `src/librustdoc/html/render/write_shared.rs`
 156
 157 Documentation on docs.rs is always generated for a single crate at a time, so
 158 the search and sidebar functionality don't include dependencies of the current
 159 crate.
 160
 161 ### Locally generated docs
 162
 163 Crate authors can run `cargo doc --open` in crates they have checked
 164 out locally to see the docs. This is useful to check that the docs they
 165 are writing are useful and display correctly. It can also be useful for
 166 people to view documentation on crates they aren't authors of, but want to
 167 use. In both cases, people may use `--document-private-items` Cargo flag to
 168 see private methods, fields, and so on, which are normally not displayed.
 169
 170 By default `cargo doc` will generate documentation for a crate and all of its
 171 dependencies. That can result in a very large documentation bundle, with a large
 172 (and slow) search corpus. The Cargo flag `--no-deps` inhibits that behavior and
 173 generates docs for just the crate.
 174
 175 ### Self-hosted project docs
 176
 177 Some projects like to host their own documentation. For example:
 178 <https://docs.serde.rs/>. This is easy to do by locally generating docs, and
 179 simply copying them to a web server. Rustdoc's HTML output can be extensively
 180 customized by flags. Users can add a theme, set the default theme, and inject
 181 arbitrary HTML. See `rustdoc --help` for details.