]>
Commit | Line | Data |
---|---|---|
60c5eb7d XL |
1 | # How to Build and Run the Compiler |
2 | ||
3 | The compiler is built using a tool called `x.py`. You will need to | |
064997fb | 4 | have Python installed to run it. |
60c5eb7d | 5 | |
6a06907d XL |
6 | For instructions on how to install Python and other prerequisites, |
7 | see [the next page](./prerequisites.md). | |
8 | ||
9 | ## Get the source code | |
10 | ||
a2a8927a XL |
11 | The main repository is [`rust-lang/rust`][repo]. This contains the compiler, |
12 | the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc), | |
13 | and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc). | |
14 | ||
15 | [repo]: https://github.com/rust-lang/rust | |
16 | ||
6a06907d XL |
17 | The very first step to work on `rustc` is to clone the repository: |
18 | ||
19 | ```bash | |
20 | git clone https://github.com/rust-lang/rust.git | |
21 | cd rust | |
22 | ``` | |
23 | ||
a2a8927a XL |
24 | There are also submodules for things like LLVM, `clippy`, `miri`, etc. The |
25 | build tool will automatically clone and sync these for you. But if you want to, | |
26 | you can do the following: | |
27 | ||
28 | ```sh | |
29 | # first time | |
30 | git submodule update --init --recursive | |
31 | ||
32 | # subsequent times (to pull new commits) | |
33 | git submodule update | |
34 | ``` | |
35 | ||
6a06907d | 36 | ## Create a `config.toml` |
60c5eb7d | 37 | |
a2a8927a XL |
38 | To start, run `./x.py setup`. This will do some initialization and create a |
39 | `config.toml` for you with reasonable defaults. These defaults are specified | |
40 | indirectly via the `profile` setting, which points to one of the TOML files in | |
41 | `src/bootstrap/defaults.` | |
60c5eb7d | 42 | |
a2a8927a XL |
43 | Alternatively, you can write `config.toml` by hand. See `config.toml.example` |
44 | for all the available settings and explanations of them. The following settings | |
45 | are of particular interest, and `config.toml.example` has full explanations. | |
46 | ||
47 | You may want to change some of the following settings (and possibly others, such as | |
cdc7bbd5 | 48 | `llvm.ccache`): |
60c5eb7d XL |
49 | |
50 | ```toml | |
51 | [llvm] | |
cdc7bbd5 | 52 | # Whether to use Rust CI built LLVM instead of locally building it. |
a2a8927a XL |
53 | download-ci-llvm = true # Download a pre-built LLVM? |
54 | assertions = true # LLVM assertions on? | |
55 | ccache = "/path/to/ccache" # Use ccache when building LLVM? | |
60c5eb7d XL |
56 | |
57 | [rust] | |
a2a8927a XL |
58 | debug-logging = true # Leave debug! and trace! calls in rustc? |
59 | incremental = true # Build rustc with incremental compilation? | |
60c5eb7d XL |
60 | ``` |
61 | ||
a2a8927a XL |
62 | If you set `download-ci-llvm = true`, in some circumstances, such as when |
63 | updating the version of LLVM used by `rustc`, you may want to temporarily | |
64 | disable this feature. See the ["Updating LLVM" section] for more. | |
65 | ||
064997fb | 66 | ["Updating LLVM" section]: ../backend/updating-llvm.md#feature-updates |
a2a8927a | 67 | |
cdc7bbd5 XL |
68 | If you have already built `rustc` and you change settings related to LLVM, then you may have to |
69 | execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py | |
70 | clean` will not cause a rebuild of LLVM. | |
60c5eb7d XL |
71 | |
72 | ## What is `x.py`? | |
73 | ||
74 | `x.py` is the script used to orchestrate the tooling in the `rustc` repository. | |
75 | It is the script that can build docs, run tests, and compile `rustc`. | |
76 | It is the now preferred way to build `rustc` and it replaces the old makefiles | |
77 | from before. Below are the different ways to utilize `x.py` in order to | |
78 | effectively deal with the repo for various common tasks. | |
79 | ||
80 | This chapter focuses on the basics to be productive, but | |
81 | if you want to learn more about `x.py`, read its README.md | |
82 | [here](https://github.com/rust-lang/rust/blob/master/src/bootstrap/README.md). | |
6a06907d XL |
83 | To read more about the bootstrap process and why `x.py` is necessary, |
84 | [read this chapter][bootstrap]. | |
85 | ||
86 | ### Running `x.py` slightly more conveniently | |
87 | ||
88 | There is a binary that wraps `x.py` called `x` in `src/tools/x`. All it does is | |
89 | run `x.py`, but it can be installed system-wide and run from any subdirectory | |
90 | of a checkout. It also looks up the appropriate version of `python` to use. | |
60c5eb7d | 91 | |
6a06907d | 92 | You can install it with `cargo install --path src/tools/x`. |
60c5eb7d XL |
93 | |
94 | [bootstrap]: ./bootstrapping.md | |
95 | ||
96 | ## Building the Compiler | |
97 | ||
6a06907d XL |
98 | To build a compiler, run `./x.py build`. This will build up to the stage1 compiler, |
99 | including `rustdoc`, producing a usable compiler toolchain from the source | |
100 | code you have checked out. | |
101 | ||
102 | Note that building will require a relatively large amount of storage space. | |
103 | You may want to have upwards of 10 or 15 gigabytes available to build the compiler. | |
60c5eb7d XL |
104 | |
105 | There are many flags you can pass to the build command of `x.py` that can be | |
106 | beneficial to cutting down compile times or fitting other things you might | |
107 | need to change. They are: | |
108 | ||
109 | ```txt | |
110 | Options: | |
111 | -v, --verbose use verbose output (-vv for very verbose) | |
112 | -i, --incremental use incremental compilation | |
113 | --config FILE TOML configuration file for build | |
114 | --build BUILD build target of the stage0 compiler | |
115 | --host HOST host targets to build | |
116 | --target TARGET target targets to build | |
117 | --on-fail CMD command to run on failure | |
118 | --stage N stage to build | |
119 | --keep-stage N stage to keep without recompiling | |
5099ac24 | 120 | --src DIR path to the root of the Rust checkout |
60c5eb7d XL |
121 | -j, --jobs JOBS number of jobs to run in parallel |
122 | -h, --help print this help message | |
123 | ``` | |
124 | ||
5099ac24 FG |
125 | For hacking, often building the stage 1 compiler is enough, which saves a lot |
126 | of time. But for final testing and release, the stage 2 compiler is used. | |
60c5eb7d | 127 | |
5099ac24 | 128 | `./x.py check` is really fast to build the Rust compiler. |
60c5eb7d XL |
129 | It is, in particular, very useful when you're doing some kind of |
130 | "type-based refactoring", like renaming a method, or changing the | |
131 | signature of some function. | |
132 | ||
6a06907d | 133 | Once you've created a `config.toml`, you are now ready to run |
60c5eb7d XL |
134 | `x.py`. There are a lot of options here, but let's start with what is |
135 | probably the best "go to" command for building a local rust: | |
136 | ||
137 | ```bash | |
064997fb | 138 | ./x.py build library |
60c5eb7d XL |
139 | ``` |
140 | ||
064997fb | 141 | This may *look* like it only builds the standard library, but that is not the case. |
60c5eb7d XL |
142 | What this command does is the following: |
143 | ||
064997fb FG |
144 | - Build `std` using the stage0 compiler |
145 | - Build `rustc` using the stage0 compiler | |
60c5eb7d | 146 | - This produces the stage1 compiler |
064997fb | 147 | - Build `std` using the stage1 compiler |
60c5eb7d XL |
148 | |
149 | This final product (stage1 compiler + libs built using that compiler) | |
5099ac24 | 150 | is what you need to build other Rust programs (unless you use `#![no_std]` or |
60c5eb7d XL |
151 | `#![no_core]`). |
152 | ||
064997fb FG |
153 | You will probably find that building the stage1 `std` is a bottleneck for you** -- but fear not, |
154 | there is a (hacky) workaround. See [the section on "recommended workflows"](./suggested.md) below. | |
60c5eb7d XL |
155 | |
156 | Note that this whole command just gives you a subset of the full `rustc` | |
5099ac24 | 157 | build. The **full** `rustc` build (what you get with `./x.py build |
6a06907d | 158 | --stage 2 compiler/rustc`) has quite a few more steps: |
60c5eb7d | 159 | |
6a06907d | 160 | - Build `rustc` with the stage1 compiler. |
60c5eb7d | 161 | - The resulting compiler here is called the "stage2" compiler. |
6a06907d | 162 | - Build `std` with stage2 compiler. |
ba9703b0 | 163 | - Build `librustdoc` and a bunch of other things with the stage2 compiler. |
60c5eb7d | 164 | |
5099ac24 | 165 | You almost never need to do this. |
60c5eb7d XL |
166 | |
167 | ## Build specific components | |
168 | ||
5099ac24 FG |
169 | If you are working on the standard library, you probably don't need to build |
170 | the compiler unless you are planning to use a recently added nightly feature. | |
171 | Instead, you can just build using the bootstrap compiler. | |
60c5eb7d XL |
172 | |
173 | ```bash | |
064997fb | 174 | ./x.py build --stage 0 library |
60c5eb7d XL |
175 | ``` |
176 | ||
60c5eb7d XL |
177 | ## Creating a rustup toolchain |
178 | ||
179 | Once you have successfully built `rustc`, you will have created a bunch | |
180 | of files in your `build` directory. In order to actually run the | |
181 | resulting `rustc`, we recommend creating rustup toolchains. The first | |
182 | one will run the stage1 compiler (which we built above). The second | |
183 | will execute the stage2 compiler (which we did not build, but which | |
184 | you will likely need to build at some point; for example, if you want | |
185 | to run the entire test suite). | |
186 | ||
187 | ```bash | |
188 | rustup toolchain link stage1 build/<host-triple>/stage1 | |
189 | rustup toolchain link stage2 build/<host-triple>/stage2 | |
190 | ``` | |
191 | ||
192 | The `<host-triple>` would typically be one of the following: | |
193 | ||
194 | - Linux: `x86_64-unknown-linux-gnu` | |
5e7ed085 | 195 | - Mac: `x86_64-apple-darwin` or `aarch64-apple-darwin` |
60c5eb7d XL |
196 | - Windows: `x86_64-pc-windows-msvc` |
197 | ||
198 | Now you can run the `rustc` you built with. If you run with `-vV`, you | |
199 | should see a version number ending in `-dev`, indicating a build from | |
200 | your local environment: | |
201 | ||
202 | ```bash | |
203 | $ rustc +stage1 -vV | |
6a06907d | 204 | rustc 1.48.0-dev |
60c5eb7d XL |
205 | binary: rustc |
206 | commit-hash: unknown | |
207 | commit-date: unknown | |
208 | host: x86_64-unknown-linux-gnu | |
6a06907d XL |
209 | release: 1.48.0-dev |
210 | LLVM version: 11.0 | |
60c5eb7d | 211 | ``` |
a2a8927a XL |
212 | |
213 | The rustup toolchain points to the specified toolchain compiled in your `build` directory, | |
214 | so the rustup toolchain will be updated whenever `x.py build` or `x.py test` are run for | |
215 | that toolchain/stage. | |
216 | ||
923072b8 FG |
217 | **Note:** the toolchain we've built does not include `cargo`. In this case, `rustup` will |
218 | fall back to using `cargo` from the installed `nightly`, `beta`, or `stable` toolchain | |
219 | (in that order). If you need to use unstable `cargo` flags, be sure to run | |
220 | `rustup install nightly` if you haven't already. See the | |
221 | [rustup documentation on custom toolchains](https://rust-lang.github.io/rustup/concepts/toolchains.html#custom-toolchains). | |
222 | ||
064997fb FG |
223 | ## Building targets for cross-compilation |
224 | ||
225 | To produce a compiler that can cross-compile for other targets, | |
226 | pass any number of `target` flags to `x.py build`. | |
227 | For example, if your host platform is `x86_64-unknown-linux-gnu` | |
228 | and your cross-compilation target is `wasm32-wasi`, you can build with: | |
229 | ||
230 | ```bash | |
231 | ./x.py build --target x86_64-unknown-linux-gnu --target wasm32-wasi | |
232 | ``` | |
233 | ||
234 | Note that if you want the resulting compiler to be able to build crates that | |
235 | involve proc macros or build scripts, you must be sure to explicitly build target support for the | |
236 | host platform (in this case, `x86_64-unknown-linux-gnu`). | |
237 | ||
238 | If you want to always build for other targets without needing to pass flags to `x.py build`, | |
239 | you can configure this in the `[build]` section of your `config.toml` like so: | |
240 | ||
241 | ```toml | |
242 | [build] | |
243 | target = ["x86_64-unknown-linux-gnu", "wasm32-wasi"] | |
244 | ``` | |
245 | ||
246 | Note that building for some targets requires having external dependencies installed | |
247 | (e.g. building musl targets requires a local copy of musl). | |
248 | Any target-specific configuration (e.g. the path to a local copy of musl) | |
249 | will need to be provided by your `config.toml`. | |
250 | Please see `config.toml.example` for information on target-specific configuration keys. | |
251 | ||
252 | For examples of the complete configuration necessary to build a target, please visit | |
253 | [the rustc book](https://doc.rust-lang.org/rustc/platform-support.html), | |
254 | select any target under the "Platform Support" heading on the left, | |
255 | and see the section related to building a compiler for that target. | |
256 | For targets without a corresponding page in the rustc book, | |
257 | it may be useful to [inspect the Dockerfiles](../tests/docker.md) | |
258 | that the Rust infrastructure itself uses to set up and configure cross-compilation. | |
259 | ||
260 | If you have followed the directions from the prior section on creating a rustup toolchain, | |
261 | then once you have built your compiler you will be able to use it to cross-compile like so: | |
262 | ||
263 | ```bash | |
264 | cargo +stage1 build --target wasm32-wasi | |
265 | ``` | |
266 | ||
60c5eb7d XL |
267 | ## Other `x.py` commands |
268 | ||
269 | Here are a few other useful `x.py` commands. We'll cover some of them in detail | |
270 | in other sections: | |
271 | ||
272 | - Building things: | |
6a06907d XL |
273 | - `./x.py build` – builds everything using the stage 1 compiler, |
274 | not just up to `std` | |
5099ac24 FG |
275 | - `./x.py build --stage 2` – builds everything with the stage 2 compiler including |
276 | `rustdoc` (which doesn't take too long) | |
60c5eb7d XL |
277 | - Running tests (see the [section on running tests](../tests/running.html) for |
278 | more details): | |
064997fb | 279 | - `./x.py test library/std` – runs the unit tests and integration tests from `std` |
6a06907d XL |
280 | - `./x.py test src/test/ui` – runs the `ui` test suite |
281 | - `./x.py test src/test/ui/const-generics` - runs all the tests in | |
60c5eb7d | 282 | the `const-generics/` subdirectory of the `ui` test suite |
6a06907d | 283 | - `./x.py test src/test/ui/const-generics/const-types.rs` - runs |
60c5eb7d XL |
284 | the single test `const-types.rs` from the `ui` test suite |
285 | ||
286 | ### Cleaning out build directories | |
287 | ||
288 | Sometimes you need to start fresh, but this is normally not the case. | |
ba9703b0 | 289 | If you need to run this then `rustbuild` is most likely not acting right and |
60c5eb7d XL |
290 | you should file a bug as to what is going wrong. If you do need to clean |
291 | everything up then you only need to run one command! | |
292 | ||
293 | ```bash | |
294 | ./x.py clean | |
295 | ``` | |
6a06907d XL |
296 | |
297 | `rm -rf build` works too, but then you have to rebuild LLVM, which can take | |
298 | a long time even on fast computers. |