src/doc/rustc-guide/src/building/bootstrapping.md

   1 # Bootstrapping the Compiler
   2
   3 This subchapter is about the bootstrapping process.
   4
   5 When running `x.py` you will see output such as:
   6
   7 ```txt
   8 Building stage0 std artifacts
   9 Copying stage0 std from stage0
  10 Building stage0 compiler artifacts
  11 Copying stage0 rustc from stage0
  12 Building LLVM for x86_64-apple-darwin
  13 Building stage0 codegen artifacts
  14 Assembling stage1 compiler
  15 Building stage1 std artifacts
  16 Copying stage1 std from stage1
  17 Building stage1 compiler artifacts
  18 Copying stage1 rustc from stage1
  19 Building stage1 codegen artifacts
  20 Assembling stage2 compiler
  21 Uplifting stage1 std
  22 Copying stage2 std from stage1
  23 Generating unstable book md files
  24 Building stage0 tool unstable-book-gen
  25 Building stage0 tool rustbook
  26 Documenting standalone
  27 Building rustdoc for stage2
  28 Documenting book redirect pages
  29 Documenting stage2 std
  30 Building rustdoc for stage1
  31 Documenting stage2 whitelisted compiler
  32 Documenting stage2 compiler
  33 Documenting stage2 rustdoc
  34 Documenting error index
  35 Uplifting stage1 rustc
  36 Copying stage2 rustc from stage1
  37 Building stage2 tool error_index_generator
  38 ```
  39
  40 A deeper look into `x.py`'s phases can be seen here:
  41
  42 <img alt="A diagram of the rustc compilation phases" src="../img/rustc_stages.svg" class="center" />
  43
  44 Keep in mind this diagram is a simplification, i.e. `rustdoc` can be built at
  45 different stages, the process is a bit different when passing flags such as
  46 `--keep-stage`, or if there are non-host targets.
  47
  48 The following tables indicate the outputs of various stage actions:
  49
  50 | Stage 0 Action                                            | Output                                       |
  51 |-----------------------------------------------------------|----------------------------------------------|
  52 | `beta` extracted                                          | `build/HOST/stage0`                          |
  53 | `stage0` builds `bootstrap`                               | `build/bootstrap`                            |
  54 | `stage0` builds `libstd`                                  | `build/HOST/stage0-std/TARGET`               |
  55 | copy `stage0-std` (HOST only)                             | `build/HOST/stage0-sysroot/lib/rustlib/HOST` |
  56 | `stage0` builds `rustc` with `stage0-sysroot`             | `build/HOST/stage0-rustc/HOST`               |
  57 | copy `stage0-rustc (except executable)`                   | `build/HOST/stage0-sysroot/lib/rustlib/HOST` |
  58 | build `llvm`                                              | `build/HOST/llvm`                            |
  59 | `stage0` builds `codegen` with `stage0-sysroot`           | `build/HOST/stage0-codegen/HOST`             |
  60 | `stage0` builds `rustdoc` with `stage0-sysroot`           | `build/HOST/stage0-tools/HOST`               |
  61
  62 `--stage=0` stops here.
  63
  64 | Stage 1 Action                                      | Output                                |
  65 |-----------------------------------------------------|---------------------------------------|
  66 | copy (uplift) `stage0-rustc` executable to `stage1` | `build/HOST/stage1/bin`               |
  67 | copy (uplift) `stage0-codegen` to `stage1`          | `build/HOST/stage1/lib`               |
  68 | copy (uplift) `stage0-sysroot` to `stage1`          | `build/HOST/stage1/lib`               |
  69 | `stage1` builds `libstd`                            | `build/HOST/stage1-std/TARGET`        |
  70 | copy `stage1-std` (HOST only)                       | `build/HOST/stage1/lib/rustlib/HOST`  |
  71 | `stage1` builds `rustc`                             | `build/HOST/stage1-rustc/HOST`        |
  72 | copy `stage1-rustc` (except executable)             | `build/HOST/stage1/lib/rustlib/HOST`  |
  73 | `stage1` builds `codegen`                           | `build/HOST/stage1-codegen/HOST`      |
  74
  75 `--stage=1` stops here.
  76
  77 | Stage 2 Action                            | Output                                                          |
  78 |-------------------------------------------|-----------------------------------------------------------------|
  79 | copy (uplift) `stage1-rustc` executable   | `build/HOST/stage2/bin`                                         |
  80 | copy (uplift) `stage1-sysroot`            | `build/HOST/stage2/lib and build/HOST/stage2/lib/rustlib/HOST`  |
  81 | `stage2` builds `libstd` (except HOST?)   | `build/HOST/stage2-std/TARGET`                                  |
  82 | copy `stage2-std` (not HOST targets)      | `build/HOST/stage2/lib/rustlib/TARGET`                          |
  83 | `stage2` builds `rustdoc`                 | `build/HOST/stage2-tools/HOST`                                  |
  84 | copy `rustdoc`                            | `build/HOST/stage2/bin`                                         |
  85
  86 `--stage=2` stops here.
  87
  88 Note that the convention `x.py` uses is that:
  89 - A "stage N artifact" is an artifact that is _produced_ by the stage N compiler.
  90 - The "stage (N+1) compiler" is assembled from "stage N artifacts".
  91 - A `--stage N` flag means build _with_ stage N.
  92
  93 In short, _stage 0 uses the stage0 compiler to create stage0 artifacts which
  94 will later be uplifted to stage1_.
  95
  96 Every time any of the main artifacts (`std` and `rustc`) are compiled, two
  97 steps are performed.
  98 When `std` is compiled by a stage N compiler, that `std` will be linked to
  99 programs built by the stage N compiler (including `rustc` built later
 100 on). It will also be used by the stage (N+1) compiler to link against itself.
 101 This is somewhat intuitive if one thinks of the stage (N+1) compiler as "just"
 102 another program we are building with the stage N compiler. In some ways, `rustc`
 103 (the binary, not the `rustbuild` step) could be thought of as one of the few
 104 `no_core` binaries out there.
 105
 106 So "stage0 std artifacts" are in fact the output of the downloaded stage0
 107 compiler, and are going to be used for anything built by the stage0 compiler:
 108 e.g. `rustc` artifacts. When it announces that it is "building stage1
 109 std artifacts" it has moved on to the next bootstrapping phase. This pattern
 110 continues in latter stages.
 111
 112 Also note that building host `std` and target `std` are different based on the
 113 stage (e.g. see in the table how stage2 only builds non-host `std` targets.
 114 This is because during stage2, the host `std` is uplifted from the "stage 1"
 115 `std` -- specifically, when "Building stage 1 artifacts" is announced, it is
 116 later copied into stage2 as well (both the compiler's `libdir` and the
 117 `sysroot`).
 118
 119 This `std` is pretty much necessary for any useful work with the compiler.
 120 Specifically, it's used as the `std` for programs compiled by the newly compiled
 121 compiler (so when you compile `fn main() { }` it is linked to the last `std`
 122 compiled with `x.py build --stage 1 src/libstd`).
 123
 124 The `rustc` generated by the stage0 compiler is linked to the freshly-built
 125 `libstd`, which means that for the most part only `std` needs to be cfg-gated,
 126 so that `rustc` can use featured added to std immediately after their addition,
 127 without need for them to get into the downloaded beta. The `libstd` built by the
 128 `stage1/bin/rustc` compiler, also known as "stage1 std artifacts", is not
 129 necessarily ABI-compatible with that compiler.
 130 That is, the `rustc` binary most likely could not use this `std` itself.
 131 It is however ABI-compatible with any programs that the `stage1/bin/rustc`
 132 binary builds (including itself), so in that sense they're paired.
 133
 134 This is also where `--keep-stage 1 src/libstd` comes into play. Since most
 135 changes to the compiler don't actually change the ABI, once you've produced a
 136 `libstd` in stage 1, you can probably just reuse it with a different compiler.
 137 If the ABI hasn't changed, you're good to go, no need to spend the time
 138 recompiling that `std`.
 139 `--keep-stage` simply assumes the previous compile is fine and copies those
 140 artifacts into the appropriate place, skipping the cargo invocation.
 141
 142 The reason we first build `std`, then `rustc`, is largely just
 143 because we want to minimize `cfg(stage0)` in the code for `rustc`.
 144 Currently `rustc` is always linked against a "new" `std` so it doesn't
 145 ever need to be concerned with differences in std; it can assume that the std is
 146 as fresh as possible.
 147
 148 The reason we need to build it twice is because of ABI compatibility.
 149 The beta compiler has it's own ABI, and then the `stage1/bin/rustc` compiler
 150 will produce programs/libraries with the new ABI.
 151 We used to build three times, but because we assume that the ABI is constant
 152 within a codebase, we presume that the libraries produced by the "stage2"
 153 compiler (produced by the `stage1/bin/rustc` compiler) is ABI-compatible with
 154 the `stage1/bin/rustc` compiler's produced libraries.
 155 What this means is that we can skip that final compilation -- and simply use the
 156 same libraries as the `stage2/bin/rustc` compiler uses itself for programs it
 157 links against.
 158
 159 This `stage2/bin/rustc` compiler is shipped to end-users, along with the
 160 `stage 1 {std,rustc}` artifacts.
 161