]>
Commit | Line | Data |
---|---|---|
60c5eb7d XL |
1 | # How to Build and Run the Compiler |
2 | ||
3 | The compiler is built using a tool called `x.py`. You will need to | |
4 | have Python installed to run it. But before we get to that, if you're going to | |
5 | be hacking on `rustc`, you'll want to tweak the configuration of the compiler. | |
6 | The default configuration is oriented towards running the compiler as a user, | |
7 | not a developer. | |
8 | ||
6a06907d XL |
9 | For instructions on how to install Python and other prerequisites, |
10 | see [the next page](./prerequisites.md). | |
11 | ||
12 | ## Get the source code | |
13 | ||
a2a8927a XL |
14 | The main repository is [`rust-lang/rust`][repo]. This contains the compiler, |
15 | the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc), | |
16 | and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc). | |
17 | ||
18 | [repo]: https://github.com/rust-lang/rust | |
19 | ||
6a06907d XL |
20 | The very first step to work on `rustc` is to clone the repository: |
21 | ||
22 | ```bash | |
23 | git clone https://github.com/rust-lang/rust.git | |
24 | cd rust | |
25 | ``` | |
26 | ||
a2a8927a XL |
27 | There are also submodules for things like LLVM, `clippy`, `miri`, etc. The |
28 | build tool will automatically clone and sync these for you. But if you want to, | |
29 | you can do the following: | |
30 | ||
31 | ```sh | |
32 | # first time | |
33 | git submodule update --init --recursive | |
34 | ||
35 | # subsequent times (to pull new commits) | |
36 | git submodule update | |
37 | ``` | |
38 | ||
6a06907d | 39 | ## Create a `config.toml` |
60c5eb7d | 40 | |
a2a8927a XL |
41 | To start, run `./x.py setup`. This will do some initialization and create a |
42 | `config.toml` for you with reasonable defaults. These defaults are specified | |
43 | indirectly via the `profile` setting, which points to one of the TOML files in | |
44 | `src/bootstrap/defaults.` | |
60c5eb7d | 45 | |
a2a8927a XL |
46 | Alternatively, you can write `config.toml` by hand. See `config.toml.example` |
47 | for all the available settings and explanations of them. The following settings | |
48 | are of particular interest, and `config.toml.example` has full explanations. | |
49 | ||
50 | You may want to change some of the following settings (and possibly others, such as | |
cdc7bbd5 | 51 | `llvm.ccache`): |
60c5eb7d XL |
52 | |
53 | ```toml | |
54 | [llvm] | |
cdc7bbd5 | 55 | # Whether to use Rust CI built LLVM instead of locally building it. |
a2a8927a XL |
56 | download-ci-llvm = true # Download a pre-built LLVM? |
57 | assertions = true # LLVM assertions on? | |
58 | ccache = "/path/to/ccache" # Use ccache when building LLVM? | |
60c5eb7d XL |
59 | |
60 | [rust] | |
a2a8927a XL |
61 | debug-logging = true # Leave debug! and trace! calls in rustc? |
62 | incremental = true # Build rustc with incremental compilation? | |
60c5eb7d XL |
63 | ``` |
64 | ||
a2a8927a XL |
65 | If you set `download-ci-llvm = true`, in some circumstances, such as when |
66 | updating the version of LLVM used by `rustc`, you may want to temporarily | |
67 | disable this feature. See the ["Updating LLVM" section] for more. | |
68 | ||
69 | ["Updating LLVM" section]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html?highlight=download-ci-llvm#feature-updates | |
70 | ||
cdc7bbd5 XL |
71 | If you have already built `rustc` and you change settings related to LLVM, then you may have to |
72 | execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py | |
73 | clean` will not cause a rebuild of LLVM. | |
60c5eb7d XL |
74 | |
75 | ## What is `x.py`? | |
76 | ||
77 | `x.py` is the script used to orchestrate the tooling in the `rustc` repository. | |
78 | It is the script that can build docs, run tests, and compile `rustc`. | |
79 | It is the now preferred way to build `rustc` and it replaces the old makefiles | |
80 | from before. Below are the different ways to utilize `x.py` in order to | |
81 | effectively deal with the repo for various common tasks. | |
82 | ||
83 | This chapter focuses on the basics to be productive, but | |
84 | if you want to learn more about `x.py`, read its README.md | |
85 | [here](https://github.com/rust-lang/rust/blob/master/src/bootstrap/README.md). | |
6a06907d XL |
86 | To read more about the bootstrap process and why `x.py` is necessary, |
87 | [read this chapter][bootstrap]. | |
88 | ||
89 | ### Running `x.py` slightly more conveniently | |
90 | ||
91 | There is a binary that wraps `x.py` called `x` in `src/tools/x`. All it does is | |
92 | run `x.py`, but it can be installed system-wide and run from any subdirectory | |
93 | of a checkout. It also looks up the appropriate version of `python` to use. | |
60c5eb7d | 94 | |
6a06907d | 95 | You can install it with `cargo install --path src/tools/x`. |
60c5eb7d XL |
96 | |
97 | [bootstrap]: ./bootstrapping.md | |
98 | ||
99 | ## Building the Compiler | |
100 | ||
6a06907d XL |
101 | To build a compiler, run `./x.py build`. This will build up to the stage1 compiler, |
102 | including `rustdoc`, producing a usable compiler toolchain from the source | |
103 | code you have checked out. | |
104 | ||
105 | Note that building will require a relatively large amount of storage space. | |
106 | You may want to have upwards of 10 or 15 gigabytes available to build the compiler. | |
60c5eb7d XL |
107 | |
108 | There are many flags you can pass to the build command of `x.py` that can be | |
109 | beneficial to cutting down compile times or fitting other things you might | |
110 | need to change. They are: | |
111 | ||
112 | ```txt | |
113 | Options: | |
114 | -v, --verbose use verbose output (-vv for very verbose) | |
115 | -i, --incremental use incremental compilation | |
116 | --config FILE TOML configuration file for build | |
117 | --build BUILD build target of the stage0 compiler | |
118 | --host HOST host targets to build | |
119 | --target TARGET target targets to build | |
120 | --on-fail CMD command to run on failure | |
121 | --stage N stage to build | |
122 | --keep-stage N stage to keep without recompiling | |
5099ac24 | 123 | --src DIR path to the root of the Rust checkout |
60c5eb7d XL |
124 | -j, --jobs JOBS number of jobs to run in parallel |
125 | -h, --help print this help message | |
126 | ``` | |
127 | ||
5099ac24 FG |
128 | For hacking, often building the stage 1 compiler is enough, which saves a lot |
129 | of time. But for final testing and release, the stage 2 compiler is used. | |
60c5eb7d | 130 | |
5099ac24 | 131 | `./x.py check` is really fast to build the Rust compiler. |
60c5eb7d XL |
132 | It is, in particular, very useful when you're doing some kind of |
133 | "type-based refactoring", like renaming a method, or changing the | |
134 | signature of some function. | |
135 | ||
6a06907d | 136 | Once you've created a `config.toml`, you are now ready to run |
60c5eb7d XL |
137 | `x.py`. There are a lot of options here, but let's start with what is |
138 | probably the best "go to" command for building a local rust: | |
139 | ||
140 | ```bash | |
6a06907d | 141 | ./x.py build -i library/std |
60c5eb7d XL |
142 | ``` |
143 | ||
6a06907d | 144 | This may *look* like it only builds `std`, but that is not the case. |
60c5eb7d XL |
145 | What this command does is the following: |
146 | ||
6a06907d XL |
147 | - Build `std` using the stage0 compiler (using incremental) |
148 | - Build `rustc` using the stage0 compiler (using incremental) | |
60c5eb7d | 149 | - This produces the stage1 compiler |
6a06907d | 150 | - Build `std` using the stage1 compiler (cannot use incremental) |
60c5eb7d XL |
151 | |
152 | This final product (stage1 compiler + libs built using that compiler) | |
5099ac24 | 153 | is what you need to build other Rust programs (unless you use `#![no_std]` or |
60c5eb7d XL |
154 | `#![no_core]`). |
155 | ||
156 | The command includes the `-i` switch which enables incremental compilation. | |
157 | This will be used to speed up the first two steps of the process: | |
158 | in particular, if you make a small change, we ought to be able to use your old | |
159 | results to make producing the stage1 **compiler** faster. | |
160 | ||
161 | Unfortunately, incremental cannot be used to speed up making the | |
162 | stage1 libraries. This is because incremental only works when you run | |
163 | the *same compiler* twice in a row. In this case, we are building a | |
164 | *new stage1 compiler* every time. Therefore, the old incremental | |
165 | results may not apply. **As a result, you will probably find that | |
6a06907d | 166 | building the stage1 `std` is a bottleneck for you** -- but fear not, |
60c5eb7d XL |
167 | there is a (hacky) workaround. See [the section on "recommended |
168 | workflows"](./suggested.md) below. | |
169 | ||
170 | Note that this whole command just gives you a subset of the full `rustc` | |
5099ac24 | 171 | build. The **full** `rustc` build (what you get with `./x.py build |
6a06907d | 172 | --stage 2 compiler/rustc`) has quite a few more steps: |
60c5eb7d | 173 | |
6a06907d | 174 | - Build `rustc` with the stage1 compiler. |
60c5eb7d | 175 | - The resulting compiler here is called the "stage2" compiler. |
6a06907d | 176 | - Build `std` with stage2 compiler. |
ba9703b0 | 177 | - Build `librustdoc` and a bunch of other things with the stage2 compiler. |
60c5eb7d | 178 | |
5099ac24 | 179 | You almost never need to do this. |
60c5eb7d XL |
180 | |
181 | ## Build specific components | |
182 | ||
5099ac24 FG |
183 | If you are working on the standard library, you probably don't need to build |
184 | the compiler unless you are planning to use a recently added nightly feature. | |
185 | Instead, you can just build using the bootstrap compiler. | |
60c5eb7d XL |
186 | |
187 | ```bash | |
5099ac24 | 188 | ./x.py build --stage 0 library/std |
60c5eb7d XL |
189 | ``` |
190 | ||
191 | Sometimes you might just want to test if the part you’re working on can | |
192 | compile. Using these commands you can test that it compiles before doing | |
193 | a bigger build to make sure it works with the compiler. As shown before | |
6a06907d | 194 | you can also pass flags at the end such as `--stage`. |
60c5eb7d XL |
195 | |
196 | ## Creating a rustup toolchain | |
197 | ||
198 | Once you have successfully built `rustc`, you will have created a bunch | |
199 | of files in your `build` directory. In order to actually run the | |
200 | resulting `rustc`, we recommend creating rustup toolchains. The first | |
201 | one will run the stage1 compiler (which we built above). The second | |
202 | will execute the stage2 compiler (which we did not build, but which | |
203 | you will likely need to build at some point; for example, if you want | |
204 | to run the entire test suite). | |
205 | ||
206 | ```bash | |
207 | rustup toolchain link stage1 build/<host-triple>/stage1 | |
208 | rustup toolchain link stage2 build/<host-triple>/stage2 | |
209 | ``` | |
210 | ||
211 | The `<host-triple>` would typically be one of the following: | |
212 | ||
213 | - Linux: `x86_64-unknown-linux-gnu` | |
5e7ed085 | 214 | - Mac: `x86_64-apple-darwin` or `aarch64-apple-darwin` |
60c5eb7d XL |
215 | - Windows: `x86_64-pc-windows-msvc` |
216 | ||
217 | Now you can run the `rustc` you built with. If you run with `-vV`, you | |
218 | should see a version number ending in `-dev`, indicating a build from | |
219 | your local environment: | |
220 | ||
221 | ```bash | |
222 | $ rustc +stage1 -vV | |
6a06907d | 223 | rustc 1.48.0-dev |
60c5eb7d XL |
224 | binary: rustc |
225 | commit-hash: unknown | |
226 | commit-date: unknown | |
227 | host: x86_64-unknown-linux-gnu | |
6a06907d XL |
228 | release: 1.48.0-dev |
229 | LLVM version: 11.0 | |
60c5eb7d | 230 | ``` |
a2a8927a XL |
231 | |
232 | The rustup toolchain points to the specified toolchain compiled in your `build` directory, | |
233 | so the rustup toolchain will be updated whenever `x.py build` or `x.py test` are run for | |
234 | that toolchain/stage. | |
235 | ||
923072b8 FG |
236 | **Note:** the toolchain we've built does not include `cargo`. In this case, `rustup` will |
237 | fall back to using `cargo` from the installed `nightly`, `beta`, or `stable` toolchain | |
238 | (in that order). If you need to use unstable `cargo` flags, be sure to run | |
239 | `rustup install nightly` if you haven't already. See the | |
240 | [rustup documentation on custom toolchains](https://rust-lang.github.io/rustup/concepts/toolchains.html#custom-toolchains). | |
241 | ||
60c5eb7d XL |
242 | ## Other `x.py` commands |
243 | ||
244 | Here are a few other useful `x.py` commands. We'll cover some of them in detail | |
245 | in other sections: | |
246 | ||
247 | - Building things: | |
6a06907d XL |
248 | - `./x.py build` – builds everything using the stage 1 compiler, |
249 | not just up to `std` | |
5099ac24 FG |
250 | - `./x.py build --stage 2` – builds everything with the stage 2 compiler including |
251 | `rustdoc` (which doesn't take too long) | |
60c5eb7d XL |
252 | - Running tests (see the [section on running tests](../tests/running.html) for |
253 | more details): | |
6a06907d XL |
254 | - `./x.py test library/std` – runs the `#[test]` tests from `std` |
255 | - `./x.py test src/test/ui` – runs the `ui` test suite | |
256 | - `./x.py test src/test/ui/const-generics` - runs all the tests in | |
60c5eb7d | 257 | the `const-generics/` subdirectory of the `ui` test suite |
6a06907d | 258 | - `./x.py test src/test/ui/const-generics/const-types.rs` - runs |
60c5eb7d XL |
259 | the single test `const-types.rs` from the `ui` test suite |
260 | ||
261 | ### Cleaning out build directories | |
262 | ||
263 | Sometimes you need to start fresh, but this is normally not the case. | |
ba9703b0 | 264 | If you need to run this then `rustbuild` is most likely not acting right and |
60c5eb7d XL |
265 | you should file a bug as to what is going wrong. If you do need to clean |
266 | everything up then you only need to run one command! | |
267 | ||
268 | ```bash | |
269 | ./x.py clean | |
270 | ``` | |
6a06907d XL |
271 | |
272 | `rm -rf build` works too, but then you have to rebuild LLVM, which can take | |
273 | a long time even on fast computers. |