]>
Commit | Line | Data |
---|---|---|
5bcae85e | 1 | //! Implementation of rustbuild, the Rust build system. |
a7813a04 | 2 | //! |
5bcae85e SL |
3 | //! This module, and its descendants, are the implementation of the Rust build |
4 | //! system. Most of this build system is backed by Cargo but the outer layer | |
5 | //! here serves as the ability to orchestrate calling Cargo, sequencing Cargo | |
476ff2be | 6 | //! builds, building artifacts like LLVM, etc. The goals of rustbuild are: |
5bcae85e | 7 | //! |
476ff2be SL |
8 | //! * To be an easily understandable, easily extensible, and maintainable build |
9 | //! system. | |
10 | //! * Leverage standard tools in the Rust ecosystem to build the compiler, aka | |
11 | //! crates.io and Cargo. | |
12 | //! * A standard interface to build across all platforms, including MSVC | |
13 | //! | |
14 | //! ## Architecture | |
15 | //! | |
3b2f2976 XL |
16 | //! The build system defers most of the complicated logic managing invocations |
17 | //! of rustc and rustdoc to Cargo itself. However, moving through various stages | |
18 | //! and copying artifacts is still necessary for it to do. Each time rustbuild | |
19 | //! is invoked, it will iterate through the list of predefined steps and execute | |
20 | //! each serially in turn if it matches the paths passed or is a default rule. | |
21 | //! For each step rustbuild relies on the step internally being incremental and | |
476ff2be SL |
22 | //! parallel. Note, though, that the `-j` parameter to rustbuild gets forwarded |
23 | //! to appropriate test harnesses and such. | |
24 | //! | |
25 | //! Most of the "meaty" steps that matter are backed by Cargo, which does indeed | |
26 | //! have its own parallelism and incremental management. Later steps, like | |
27 | //! tests, aren't incremental and simply run the entire suite currently. | |
3b2f2976 XL |
28 | //! However, compiletest itself tries to avoid running tests when the artifacts |
29 | //! that are involved (mainly the compiler) haven't changed. | |
476ff2be | 30 | //! |
0731742a | 31 | //! When you execute `x.py build`, the steps executed are: |
476ff2be SL |
32 | //! |
33 | //! * First, the python script is run. This will automatically download the | |
3b2f2976 | 34 | //! stage0 rustc and cargo according to `src/stage0.txt`, or use the cached |
476ff2be SL |
35 | //! versions if they're available. These are then used to compile rustbuild |
36 | //! itself (using Cargo). Finally, control is then transferred to rustbuild. | |
37 | //! | |
38 | //! * Rustbuild takes over, performs sanity checks, probes the environment, | |
3b2f2976 XL |
39 | //! reads configuration, and starts executing steps as it reads the command |
40 | //! line arguments (paths) or going through the default rules. | |
476ff2be | 41 | //! |
3b2f2976 XL |
42 | //! The build output will be something like the following: |
43 | //! | |
44 | //! Building stage0 std artifacts | |
45 | //! Copying stage0 std | |
46 | //! Building stage0 test artifacts | |
47 | //! Copying stage0 test | |
48 | //! Building stage0 compiler artifacts | |
49 | //! Copying stage0 rustc | |
50 | //! Assembling stage1 compiler | |
51 | //! Building stage1 std artifacts | |
52 | //! Copying stage1 std | |
53 | //! Building stage1 test artifacts | |
54 | //! Copying stage1 test | |
55 | //! Building stage1 compiler artifacts | |
56 | //! Copying stage1 rustc | |
57 | //! Assembling stage2 compiler | |
58 | //! Uplifting stage1 std | |
59 | //! Uplifting stage1 test | |
60 | //! Uplifting stage1 rustc | |
61 | //! | |
62 | //! Let's disect that a little: | |
63 | //! | |
64 | //! ## Building stage0 {std,test,compiler} artifacts | |
65 | //! | |
66 | //! These steps use the provided (downloaded, usually) compiler to compile the | |
67 | //! local Rust source into libraries we can use. | |
68 | //! | |
69 | //! ## Copying stage0 {std,test,rustc} | |
70 | //! | |
71 | //! This copies the build output from Cargo into | |
9fa01778 | 72 | //! `build/$HOST/stage0-sysroot/lib/rustlib/$ARCH/lib`. FIXME: this step's |
3b2f2976 XL |
73 | //! documentation should be expanded -- the information already here may be |
74 | //! incorrect. | |
75 | //! | |
76 | //! ## Assembling stage1 compiler | |
77 | //! | |
78 | //! This copies the libraries we built in "building stage0 ... artifacts" into | |
79 | //! the stage1 compiler's lib directory. These are the host libraries that the | |
80 | //! compiler itself uses to run. These aren't actually used by artifacts the new | |
81 | //! compiler generates. This step also copies the rustc and rustdoc binaries we | |
82 | //! generated into build/$HOST/stage/bin. | |
83 | //! | |
84 | //! The stage1/bin/rustc is a fully functional compiler, but it doesn't yet have | |
85 | //! any libraries to link built binaries or libraries to. The next 3 steps will | |
86 | //! provide those libraries for it; they are mostly equivalent to constructing | |
87 | //! the stage1/bin compiler so we don't go through them individually. | |
88 | //! | |
89 | //! ## Uplifting stage1 {std,test,rustc} | |
90 | //! | |
91 | //! This step copies the libraries from the stage1 compiler sysroot into the | |
92 | //! stage2 compiler. This is done to avoid rebuilding the compiler; libraries | |
93 | //! we'd build in this step should be identical (in function, if not necessarily | |
94 | //! identical on disk) so there's no need to recompile the compiler again. Note | |
95 | //! that if you want to, you can enable the full-bootstrap option to change this | |
96 | //! behavior. | |
476ff2be SL |
97 | //! |
98 | //! Each step is driven by a separate Cargo project and rustbuild orchestrates | |
99 | //! copying files between steps and otherwise preparing for Cargo to run. | |
100 | //! | |
101 | //! ## Further information | |
102 | //! | |
103 | //! More documentation can be found in each respective module below, and you can | |
104 | //! also check out the `src/bootstrap/README.md` file for more information. | |
5bcae85e | 105 | |
2c00a5a8 | 106 | #![feature(core_intrinsics)] |
8faf50e0 | 107 | #![feature(drain_filter)] |
32a655c1 | 108 | |
ff7c6d11 | 109 | use std::cell::{RefCell, Cell}; |
3b2f2976 | 110 | use std::collections::{HashSet, HashMap}; |
7453a54e | 111 | use std::env; |
83c7162d | 112 | use std::fs::{self, OpenOptions, File}; |
9fa01778 | 113 | use std::io::{Seek, SeekFrom, Write, Read}; |
7cac9316 | 114 | use std::path::{PathBuf, Path}; |
ea8adc8c | 115 | use std::process::{self, Command}; |
3b2f2976 | 116 | use std::slice; |
83c7162d | 117 | use std::str; |
5bcae85e | 118 | |
b7449926 XL |
119 | #[cfg(unix)] |
120 | use std::os::unix::fs::symlink as symlink_file; | |
121 | #[cfg(windows)] | |
122 | use std::os::windows::fs::symlink_file; | |
123 | ||
48663c56 | 124 | use build_helper::{ |
416331ca | 125 | mtime, output, run, run_suppressed, t, try_run, try_run_suppressed, |
48663c56 | 126 | }; |
83c7162d | 127 | use filetime::FileTime; |
5bcae85e | 128 | |
416331ca | 129 | use crate::util::{exe, libdir, CiEnv}; |
5bcae85e | 130 | |
ea8adc8c | 131 | mod cc_detect; |
5bcae85e SL |
132 | mod channel; |
133 | mod check; | |
2c00a5a8 | 134 | mod test; |
5bcae85e SL |
135 | mod clean; |
136 | mod compile; | |
c30ab7b3 | 137 | mod metadata; |
5bcae85e SL |
138 | mod config; |
139 | mod dist; | |
140 | mod doc; | |
141 | mod flags; | |
c30ab7b3 | 142 | mod install; |
5bcae85e SL |
143 | mod native; |
144 | mod sanity; | |
5bcae85e | 145 | pub mod util; |
3b2f2976 XL |
146 | mod builder; |
147 | mod cache; | |
148 | mod tool; | |
ea8adc8c | 149 | mod toolstate; |
5bcae85e SL |
150 | |
151 | #[cfg(windows)] | |
152 | mod job; | |
153 | ||
8faf50e0 | 154 | #[cfg(all(unix, not(target_os = "haiku")))] |
7cac9316 | 155 | mod job { |
0731742a | 156 | pub unsafe fn setup(build: &mut crate::Build) { |
7cac9316 XL |
157 | if build.config.low_priority { |
158 | libc::setpriority(libc::PRIO_PGRP as _, 0, 10); | |
159 | } | |
160 | } | |
161 | } | |
162 | ||
e74abb32 | 163 | #[cfg(any(target_os = "haiku", target_os = "hermit", not(any(unix, windows))))] |
5bcae85e | 164 | mod job { |
0731742a | 165 | pub unsafe fn setup(_build: &mut crate::Build) { |
7cac9316 | 166 | } |
5bcae85e SL |
167 | } |
168 | ||
0731742a XL |
169 | pub use crate::config::Config; |
170 | use crate::flags::Subcommand; | |
171 | use crate::cache::{Interned, INTERNER}; | |
5bcae85e | 172 | |
8faf50e0 XL |
173 | const LLVM_TOOLS: &[&str] = &[ |
174 | "llvm-nm", // used to inspect binaries; it shows symbol names, their sizes and visibility | |
175 | "llvm-objcopy", // used to transform ELFs into binary format which flashing tools consume | |
176 | "llvm-objdump", // used to disassemble programs | |
177 | "llvm-profdata", // used to inspect and merge files generated by profiles | |
b7449926 | 178 | "llvm-readobj", // used to get information from ELFs/objects that the other tools don't provide |
8faf50e0 XL |
179 | "llvm-size", // used to prints the size of the linker sections of a program |
180 | "llvm-strip", // used to discard symbols from binary files to reduce their size | |
532ac7d7 | 181 | "llvm-ar" // used for creating and modifying archive files |
8faf50e0 XL |
182 | ]; |
183 | ||
5bcae85e SL |
184 | /// A structure representing a Rust compiler. |
185 | /// | |
186 | /// Each compiler has a `stage` that it is associated with and a `host` that | |
187 | /// corresponds to the platform the compiler runs on. This structure is used as | |
188 | /// a parameter to many methods below. | |
83c7162d | 189 | #[derive(Eq, PartialOrd, Ord, PartialEq, Clone, Copy, Hash, Debug)] |
3b2f2976 | 190 | pub struct Compiler { |
5bcae85e | 191 | stage: u32, |
3b2f2976 | 192 | host: Interned<String>, |
5bcae85e SL |
193 | } |
194 | ||
83c7162d XL |
195 | #[derive(PartialEq, Eq, Copy, Clone, Debug)] |
196 | pub enum DocTests { | |
dc9dc135 | 197 | /// Run normal tests and doc tests (default). |
83c7162d | 198 | Yes, |
dc9dc135 | 199 | /// Do not run any doc tests. |
83c7162d | 200 | No, |
dc9dc135 | 201 | /// Only run doc tests. |
83c7162d XL |
202 | Only, |
203 | } | |
204 | ||
b7449926 XL |
205 | pub enum GitRepo { |
206 | Rustc, | |
207 | Llvm, | |
208 | } | |
209 | ||
5bcae85e SL |
210 | /// Global configuration for the build system. |
211 | /// | |
212 | /// This structure transitively contains all configuration for the build system. | |
213 | /// All filesystem-encoded configuration is in `config`, all flags are in | |
214 | /// `flags`, and then parsed or probed information is listed in the keys below. | |
215 | /// | |
216 | /// This structure is a parameter of almost all methods in the build system, | |
217 | /// although most functions are implemented as free functions rather than | |
218 | /// methods specifically on this structure itself (to make it easier to | |
219 | /// organize). | |
220 | pub struct Build { | |
dc9dc135 | 221 | /// User-specified configuration from `config.toml`. |
5bcae85e SL |
222 | config: Config, |
223 | ||
dc9dc135 | 224 | // Properties derived from the above configuration |
5bcae85e SL |
225 | src: PathBuf, |
226 | out: PathBuf, | |
8bb4bdeb XL |
227 | rust_info: channel::GitInfo, |
228 | cargo_info: channel::GitInfo, | |
cc61c64b | 229 | rls_info: channel::GitInfo, |
8faf50e0 | 230 | clippy_info: channel::GitInfo, |
0731742a | 231 | miri_info: channel::GitInfo, |
abe05a73 | 232 | rustfmt_info: channel::GitInfo, |
9fa01778 | 233 | in_tree_llvm_info: channel::GitInfo, |
5bcae85e | 234 | local_rebuild: bool, |
041b39d2 | 235 | fail_fast: bool, |
83c7162d | 236 | doc_tests: DocTests, |
041b39d2 XL |
237 | verbosity: usize, |
238 | ||
dc9dc135 | 239 | // Targets for which to build |
3b2f2976 XL |
240 | build: Interned<String>, |
241 | hosts: Vec<Interned<String>>, | |
242 | targets: Vec<Interned<String>>, | |
041b39d2 | 243 | |
dc9dc135 | 244 | // Stage 0 (downloaded) compiler and cargo or their local rust equivalents |
041b39d2 XL |
245 | initial_rustc: PathBuf, |
246 | initial_cargo: PathBuf, | |
5bcae85e | 247 | |
5bcae85e | 248 | // Runtime state filled in later on |
abe05a73 XL |
249 | // C/C++ compilers and archiver for all targets |
250 | cc: HashMap<Interned<String>, cc::Tool>, | |
ea8adc8c | 251 | cxx: HashMap<Interned<String>, cc::Tool>, |
abe05a73 | 252 | ar: HashMap<Interned<String>, PathBuf>, |
b7449926 | 253 | ranlib: HashMap<Interned<String>, PathBuf>, |
dc9dc135 | 254 | // Miscellaneous |
3b2f2976 | 255 | crates: HashMap<Interned<String>, Crate>, |
476ff2be | 256 | is_sudo: bool, |
7cac9316 | 257 | ci_env: CiEnv, |
ea8adc8c | 258 | delayed_failures: RefCell<Vec<String>>, |
ff7c6d11 | 259 | prerelease_version: Cell<Option<u32>>, |
0531ce1d XL |
260 | tool_artifacts: RefCell<HashMap< |
261 | Interned<String>, | |
262 | HashMap<String, (&'static str, PathBuf, Vec<String>)> | |
263 | >>, | |
c30ab7b3 SL |
264 | } |
265 | ||
266 | #[derive(Debug)] | |
267 | struct Crate { | |
3b2f2976 | 268 | name: Interned<String>, |
94b46f34 XL |
269 | deps: HashSet<Interned<String>>, |
270 | id: String, | |
c30ab7b3 | 271 | path: PathBuf, |
5bcae85e SL |
272 | } |
273 | ||
2c00a5a8 XL |
274 | impl Crate { |
275 | fn is_local(&self, build: &Build) -> bool { | |
276 | self.path.starts_with(&build.config.src) && | |
277 | !self.path.to_string_lossy().ends_with("_shim") | |
278 | } | |
279 | ||
280 | fn local_path(&self, build: &Build) -> PathBuf { | |
281 | assert!(self.is_local(build)); | |
282 | self.path.strip_prefix(&build.config.src).unwrap().into() | |
283 | } | |
284 | } | |
285 | ||
5bcae85e SL |
286 | /// The various "modes" of invoking Cargo. |
287 | /// | |
288 | /// These entries currently correspond to the various output directories of the | |
289 | /// build system, with each mod generating output in a different directory. | |
83c7162d | 290 | #[derive(Debug, Hash, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] |
5bcae85e | 291 | pub enum Mode { |
041b39d2 | 292 | /// Build the standard library, placing output in the "stageN-std" directory. |
94b46f34 | 293 | Std, |
5bcae85e | 294 | |
94b46f34 XL |
295 | /// Build librustc, and compiler libraries, placing output in the "stageN-rustc" directory. |
296 | Rustc, | |
5bcae85e | 297 | |
94b46f34 XL |
298 | /// Build codegen libraries, placing output in the "stageN-codegen" directory |
299 | Codegen, | |
300 | ||
8faf50e0 XL |
301 | /// Build some tools, placing output in the "stageN-tools" directory. The |
302 | /// "other" here is for miscellaneous sets of tools that are built using the | |
303 | /// bootstrap compiler in its entirety (target libraries and all). | |
304 | /// Typically these tools compile with stable Rust. | |
305 | ToolBootstrap, | |
306 | ||
307 | /// Compile a tool which uses all libraries we compile (up to rustc). | |
308 | /// Doesn't use the stage0 compiler libraries like "other", and includes | |
309 | /// tools like rustdoc, cargo, rls, etc. | |
94b46f34 | 310 | ToolStd, |
94b46f34 XL |
311 | ToolRustc, |
312 | } | |
313 | ||
314 | impl Mode { | |
315 | pub fn is_tool(&self) -> bool { | |
316 | match self { | |
8faf50e0 | 317 | Mode::ToolBootstrap | Mode::ToolRustc | Mode::ToolStd => true, |
94b46f34 XL |
318 | _ => false |
319 | } | |
320 | } | |
5bcae85e SL |
321 | } |
322 | ||
323 | impl Build { | |
324 | /// Creates a new set of build configuration from the `flags` on the command | |
325 | /// line and the filesystem `config`. | |
326 | /// | |
327 | /// By default all build output will be placed in the current directory. | |
3b2f2976 | 328 | pub fn new(config: Config) -> Build { |
3b2f2976 | 329 | let src = config.src.clone(); |
83c7162d | 330 | let out = config.out.clone(); |
5bcae85e | 331 | |
476ff2be SL |
332 | let is_sudo = match env::var_os("SUDO_USER") { |
333 | Some(sudo_user) => { | |
334 | match env::var_os("USER") { | |
335 | Some(user) => user != sudo_user, | |
336 | None => false, | |
337 | } | |
338 | } | |
339 | None => false, | |
340 | }; | |
532ac7d7 XL |
341 | |
342 | let ignore_git = config.ignore_git; | |
343 | let rust_info = channel::GitInfo::new(ignore_git, &src); | |
344 | let cargo_info = channel::GitInfo::new(ignore_git, &src.join("src/tools/cargo")); | |
345 | let rls_info = channel::GitInfo::new(ignore_git, &src.join("src/tools/rls")); | |
346 | let clippy_info = channel::GitInfo::new(ignore_git, &src.join("src/tools/clippy")); | |
347 | let miri_info = channel::GitInfo::new(ignore_git, &src.join("src/tools/miri")); | |
348 | let rustfmt_info = channel::GitInfo::new(ignore_git, &src.join("src/tools/rustfmt")); | |
349 | ||
350 | // we always try to use git for LLVM builds | |
351 | let in_tree_llvm_info = channel::GitInfo::new(false, &src.join("src/llvm-project")); | |
476ff2be | 352 | |
83c7162d | 353 | let mut build = Build { |
041b39d2 XL |
354 | initial_rustc: config.initial_rustc.clone(), |
355 | initial_cargo: config.initial_cargo.clone(), | |
356 | local_rebuild: config.local_rebuild, | |
3b2f2976 | 357 | fail_fast: config.cmd.fail_fast(), |
0531ce1d | 358 | doc_tests: config.cmd.doc_tests(), |
3b2f2976 | 359 | verbosity: config.verbose, |
041b39d2 | 360 | |
3b2f2976 XL |
361 | build: config.build, |
362 | hosts: config.hosts.clone(), | |
363 | targets: config.targets.clone(), | |
041b39d2 | 364 | |
3b2f2976 XL |
365 | config, |
366 | src, | |
367 | out, | |
5bcae85e | 368 | |
3b2f2976 XL |
369 | rust_info, |
370 | cargo_info, | |
371 | rls_info, | |
8faf50e0 | 372 | clippy_info, |
0731742a | 373 | miri_info, |
abe05a73 | 374 | rustfmt_info, |
9fa01778 | 375 | in_tree_llvm_info, |
5bcae85e SL |
376 | cc: HashMap::new(), |
377 | cxx: HashMap::new(), | |
abe05a73 | 378 | ar: HashMap::new(), |
b7449926 | 379 | ranlib: HashMap::new(), |
c30ab7b3 | 380 | crates: HashMap::new(), |
3b2f2976 | 381 | is_sudo, |
7cac9316 | 382 | ci_env: CiEnv::current(), |
ea8adc8c | 383 | delayed_failures: RefCell::new(Vec::new()), |
ff7c6d11 | 384 | prerelease_version: Cell::new(None), |
0531ce1d | 385 | tool_artifacts: Default::default(), |
83c7162d XL |
386 | }; |
387 | ||
388 | build.verbose("finding compilers"); | |
389 | cc_detect::find(&mut build); | |
390 | build.verbose("running sanity check"); | |
391 | sanity::check(&mut build); | |
392 | ||
393 | // If local-rust is the same major.minor as the current version, then force a | |
394 | // local-rebuild | |
395 | let local_version_verbose = output( | |
396 | Command::new(&build.initial_rustc).arg("--version").arg("--verbose")); | |
397 | let local_release = local_version_verbose | |
398 | .lines().filter(|x| x.starts_with("release:")) | |
0731742a | 399 | .next().unwrap().trim_start_matches("release:").trim(); |
83c7162d XL |
400 | let my_version = channel::CFG_RELEASE_NUM; |
401 | if local_release.split('.').take(2).eq(my_version.split('.').take(2)) { | |
402 | build.verbose(&format!("auto-detected local-rebuild {}", local_release)); | |
403 | build.local_rebuild = true; | |
5bcae85e | 404 | } |
83c7162d XL |
405 | |
406 | build.verbose("learning about cargo"); | |
407 | metadata::build(&mut build); | |
408 | ||
409 | build | |
5bcae85e SL |
410 | } |
411 | ||
3b2f2976 | 412 | pub fn build_triple(&self) -> &[Interned<String>] { |
041b39d2 | 413 | unsafe { |
3b2f2976 | 414 | slice::from_raw_parts(&self.build, 1) |
041b39d2 XL |
415 | } |
416 | } | |
417 | ||
5bcae85e SL |
418 | /// Executes the entire build, as configured by the flags and configuration. |
419 | pub fn build(&mut self) { | |
5bcae85e | 420 | unsafe { |
7cac9316 | 421 | job::setup(self); |
5bcae85e SL |
422 | } |
423 | ||
ea8adc8c XL |
424 | if let Subcommand::Clean { all } = self.config.cmd { |
425 | return clean::clean(self, all); | |
5bcae85e SL |
426 | } |
427 | ||
83c7162d XL |
428 | { |
429 | let builder = builder::Builder::new(&self); | |
430 | if let Some(path) = builder.paths.get(0) { | |
431 | if path == Path::new("nonexistent/path/to/trigger/cargo/metadata") { | |
432 | return; | |
433 | } | |
434 | } | |
5bcae85e | 435 | } |
5bcae85e | 436 | |
83c7162d XL |
437 | if !self.config.dry_run { |
438 | { | |
439 | self.config.dry_run = true; | |
440 | let builder = builder::Builder::new(&self); | |
441 | builder.execute_cli(); | |
442 | } | |
443 | self.config.dry_run = false; | |
444 | let builder = builder::Builder::new(&self); | |
445 | builder.execute_cli(); | |
446 | } else { | |
447 | let builder = builder::Builder::new(&self); | |
448 | let _ = builder.execute_cli(); | |
449 | } | |
ea8adc8c XL |
450 | |
451 | // Check for postponed failures from `test --no-fail-fast`. | |
452 | let failures = self.delayed_failures.borrow(); | |
453 | if failures.len() > 0 { | |
454 | println!("\n{} command(s) did not execute successfully:\n", failures.len()); | |
455 | for failure in failures.iter() { | |
456 | println!(" - {}\n", failure); | |
457 | } | |
458 | process::exit(1); | |
459 | } | |
5bcae85e SL |
460 | } |
461 | ||
5bcae85e SL |
462 | /// Clear out `dir` if `input` is newer. |
463 | /// | |
464 | /// After this executes, it will also ensure that `dir` exists. | |
abe05a73 | 465 | fn clear_if_dirty(&self, dir: &Path, input: &Path) -> bool { |
5bcae85e | 466 | let stamp = dir.join(".stamp"); |
abe05a73 | 467 | let mut cleared = false; |
5bcae85e SL |
468 | if mtime(&stamp) < mtime(input) { |
469 | self.verbose(&format!("Dirty - {}", dir.display())); | |
470 | let _ = fs::remove_dir_all(dir); | |
abe05a73 | 471 | cleared = true; |
9e0c209e | 472 | } else if stamp.exists() { |
abe05a73 | 473 | return cleared; |
5bcae85e SL |
474 | } |
475 | t!(fs::create_dir_all(dir)); | |
476 | t!(File::create(stamp)); | |
abe05a73 | 477 | cleared |
5bcae85e SL |
478 | } |
479 | ||
9fa01778 | 480 | /// Gets the space-separated set of activated features for the standard |
5bcae85e SL |
481 | /// library. |
482 | fn std_features(&self) -> String { | |
476ff2be | 483 | let mut features = "panic-unwind".to_string(); |
8bb4bdeb | 484 | |
532ac7d7 XL |
485 | if self.config.llvm_libunwind { |
486 | features.push_str(" llvm-libunwind"); | |
487 | } | |
5bcae85e SL |
488 | if self.config.backtrace { |
489 | features.push_str(" backtrace"); | |
490 | } | |
041b39d2 XL |
491 | if self.config.profiler { |
492 | features.push_str(" profiler"); | |
493 | } | |
494 | features | |
5bcae85e SL |
495 | } |
496 | ||
9fa01778 | 497 | /// Gets the space-separated set of activated features for the compiler. |
5bcae85e SL |
498 | fn rustc_features(&self) -> String { |
499 | let mut features = String::new(); | |
a1dfa0c6 XL |
500 | if self.config.jemalloc { |
501 | features.push_str("jemalloc"); | |
5bcae85e | 502 | } |
60c5eb7d XL |
503 | if self.config.llvm_enabled() { |
504 | features.push_str(" llvm"); | |
505 | } | |
041b39d2 | 506 | features |
5bcae85e SL |
507 | } |
508 | ||
509 | /// Component directory that Cargo will produce output into (e.g. | |
510 | /// release/debug) | |
511 | fn cargo_dir(&self) -> &'static str { | |
512 | if self.config.rust_optimize {"release"} else {"debug"} | |
513 | } | |
514 | ||
abe05a73 XL |
515 | fn tools_dir(&self, compiler: Compiler) -> PathBuf { |
516 | let out = self.out.join(&*compiler.host).join(format!("stage{}-tools-bin", compiler.stage)); | |
517 | t!(fs::create_dir_all(&out)); | |
518 | out | |
519 | } | |
520 | ||
5bcae85e SL |
521 | /// Returns the root directory for all output generated in a particular |
522 | /// stage when running with a particular host compiler. | |
523 | /// | |
524 | /// The mode indicates what the root directory is for. | |
3b2f2976 | 525 | fn stage_out(&self, compiler: Compiler, mode: Mode) -> PathBuf { |
5bcae85e | 526 | let suffix = match mode { |
94b46f34 | 527 | Mode::Std => "-std", |
94b46f34 | 528 | Mode::Rustc => "-rustc", |
b7449926 | 529 | Mode::Codegen => "-codegen", |
8faf50e0 | 530 | Mode::ToolBootstrap => "-bootstrap-tools", |
e1599b0c | 531 | Mode::ToolStd | Mode::ToolRustc => "-tools", |
5bcae85e | 532 | }; |
3b2f2976 | 533 | self.out.join(&*compiler.host) |
5bcae85e SL |
534 | .join(format!("stage{}{}", compiler.stage, suffix)) |
535 | } | |
536 | ||
537 | /// Returns the root output directory for all Cargo output in a given stage, | |
3b2f2976 | 538 | /// running a particular compiler, whether or not we're building the |
5bcae85e SL |
539 | /// standard library, and targeting the specified architecture. |
540 | fn cargo_out(&self, | |
3b2f2976 | 541 | compiler: Compiler, |
5bcae85e | 542 | mode: Mode, |
3b2f2976 XL |
543 | target: Interned<String>) -> PathBuf { |
544 | self.stage_out(compiler, mode).join(&*target).join(self.cargo_dir()) | |
5bcae85e SL |
545 | } |
546 | ||
547 | /// Root output directory for LLVM compiled for `target` | |
548 | /// | |
549 | /// Note that if LLVM is configured externally then the directory returned | |
550 | /// will likely be empty. | |
3b2f2976 XL |
551 | fn llvm_out(&self, target: Interned<String>) -> PathBuf { |
552 | self.out.join(&*target).join("llvm") | |
5bcae85e SL |
553 | } |
554 | ||
0531ce1d XL |
555 | fn lld_out(&self, target: Interned<String>) -> PathBuf { |
556 | self.out.join(&*target).join("lld") | |
557 | } | |
558 | ||
c30ab7b3 | 559 | /// Output directory for all documentation for a target |
3b2f2976 XL |
560 | fn doc_out(&self, target: Interned<String>) -> PathBuf { |
561 | self.out.join(&*target).join("doc") | |
c30ab7b3 SL |
562 | } |
563 | ||
0531ce1d XL |
564 | /// Output directory for all documentation for a target |
565 | fn compiler_doc_out(&self, target: Interned<String>) -> PathBuf { | |
566 | self.out.join(&*target).join("compiler-doc") | |
567 | } | |
568 | ||
041b39d2 | 569 | /// Output directory for some generated md crate documentation for a target (temporary) |
3b2f2976 XL |
570 | fn md_doc_out(&self, target: Interned<String>) -> Interned<PathBuf> { |
571 | INTERNER.intern_path(self.out.join(&*target).join("md-doc")) | |
041b39d2 XL |
572 | } |
573 | ||
8bb4bdeb XL |
574 | /// Output directory for all crate documentation for a target (temporary) |
575 | /// | |
576 | /// The artifacts here are then copied into `doc_out` above. | |
3b2f2976 XL |
577 | fn crate_doc_out(&self, target: Interned<String>) -> PathBuf { |
578 | self.out.join(&*target).join("crate-docs") | |
8bb4bdeb XL |
579 | } |
580 | ||
9fa01778 | 581 | /// Returns `true` if no custom `llvm-config` is set for the specified target. |
5bcae85e SL |
582 | /// |
583 | /// If no custom `llvm-config` was specified then Rust's llvm will be used. | |
3b2f2976 XL |
584 | fn is_rust_llvm(&self, target: Interned<String>) -> bool { |
585 | match self.config.target_config.get(&target) { | |
5bcae85e SL |
586 | Some(ref c) => c.llvm_config.is_none(), |
587 | None => true | |
588 | } | |
589 | } | |
590 | ||
5bcae85e | 591 | /// Returns the path to `FileCheck` binary for the specified target |
3b2f2976 XL |
592 | fn llvm_filecheck(&self, target: Interned<String>) -> PathBuf { |
593 | let target_config = self.config.target_config.get(&target); | |
0bf4aa26 XL |
594 | if let Some(s) = target_config.and_then(|c| c.llvm_filecheck.as_ref()) { |
595 | s.to_path_buf() | |
596 | } else if let Some(s) = target_config.and_then(|c| c.llvm_config.as_ref()) { | |
32a655c1 | 597 | let llvm_bindir = output(Command::new(s).arg("--bindir")); |
0bf4aa26 XL |
598 | let filecheck = Path::new(llvm_bindir.trim()).join(exe("FileCheck", &*target)); |
599 | if filecheck.exists() { | |
600 | filecheck | |
601 | } else { | |
602 | // On Fedora the system LLVM installs FileCheck in the | |
603 | // llvm subdirectory of the libdir. | |
604 | let llvm_libdir = output(Command::new(s).arg("--libdir")); | |
605 | let lib_filecheck = Path::new(llvm_libdir.trim()) | |
606 | .join("llvm").join(exe("FileCheck", &*target)); | |
607 | if lib_filecheck.exists() { | |
608 | lib_filecheck | |
609 | } else { | |
610 | // Return the most normal file name, even though | |
611 | // it doesn't exist, so that any error message | |
612 | // refers to that. | |
613 | filecheck | |
614 | } | |
615 | } | |
5bcae85e | 616 | } else { |
3b2f2976 | 617 | let base = self.llvm_out(self.config.build).join("build"); |
94b46f34 XL |
618 | let base = if !self.config.ninja && self.config.build.contains("msvc") { |
619 | if self.config.llvm_optimize { | |
620 | if self.config.llvm_release_debuginfo { | |
621 | base.join("RelWithDebInfo") | |
622 | } else { | |
623 | base.join("Release") | |
624 | } | |
625 | } else { | |
626 | base.join("Debug") | |
627 | } | |
5bcae85e | 628 | } else { |
94b46f34 XL |
629 | base |
630 | }; | |
631 | base.join("bin").join(exe("FileCheck", &*target)) | |
5bcae85e SL |
632 | } |
633 | } | |
634 | ||
8bb4bdeb | 635 | /// Directory for libraries built from C/C++ code and shared between stages. |
3b2f2976 XL |
636 | fn native_dir(&self, target: Interned<String>) -> PathBuf { |
637 | self.out.join(&*target).join("native") | |
8bb4bdeb XL |
638 | } |
639 | ||
5bcae85e SL |
640 | /// Root output directory for rust_test_helpers library compiled for |
641 | /// `target` | |
3b2f2976 | 642 | fn test_helpers_out(&self, target: Interned<String>) -> PathBuf { |
8bb4bdeb | 643 | self.native_dir(target).join("rust-test-helpers") |
5bcae85e SL |
644 | } |
645 | ||
476ff2be SL |
646 | /// Adds the `RUST_TEST_THREADS` env var if necessary |
647 | fn add_rust_test_threads(&self, cmd: &mut Command) { | |
648 | if env::var_os("RUST_TEST_THREADS").is_none() { | |
649 | cmd.env("RUST_TEST_THREADS", self.jobs().to_string()); | |
650 | } | |
5bcae85e SL |
651 | } |
652 | ||
5bcae85e SL |
653 | /// Returns the libdir of the snapshot compiler. |
654 | fn rustc_snapshot_libdir(&self) -> PathBuf { | |
8faf50e0 XL |
655 | self.rustc_snapshot_sysroot().join(libdir(&self.config.build)) |
656 | } | |
657 | ||
658 | /// Returns the sysroot of the snapshot compiler. | |
659 | fn rustc_snapshot_sysroot(&self) -> &Path { | |
041b39d2 | 660 | self.initial_rustc.parent().unwrap().parent().unwrap() |
5bcae85e SL |
661 | } |
662 | ||
663 | /// Runs a command, printing out nice contextual information if it fails. | |
664 | fn run(&self, cmd: &mut Command) { | |
83c7162d | 665 | if self.config.dry_run { return; } |
ff7c6d11 | 666 | self.verbose(&format!("running: {:?}", cmd)); |
416331ca | 667 | run(cmd) |
5bcae85e SL |
668 | } |
669 | ||
8bb4bdeb XL |
670 | /// Runs a command, printing out nice contextual information if it fails. |
671 | fn run_quiet(&self, cmd: &mut Command) { | |
83c7162d | 672 | if self.config.dry_run { return; } |
8bb4bdeb | 673 | self.verbose(&format!("running: {:?}", cmd)); |
ff7c6d11 | 674 | run_suppressed(cmd) |
8bb4bdeb XL |
675 | } |
676 | ||
ff7c6d11 XL |
677 | /// Runs a command, printing out nice contextual information if it fails. |
678 | /// Exits if the command failed to execute at all, otherwise returns its | |
679 | /// `status.success()`. | |
680 | fn try_run(&self, cmd: &mut Command) -> bool { | |
83c7162d | 681 | if self.config.dry_run { return true; } |
7cac9316 | 682 | self.verbose(&format!("running: {:?}", cmd)); |
416331ca | 683 | try_run(cmd) |
7cac9316 XL |
684 | } |
685 | ||
686 | /// Runs a command, printing out nice contextual information if it fails. | |
687 | /// Exits if the command failed to execute at all, otherwise returns its | |
688 | /// `status.success()`. | |
689 | fn try_run_quiet(&self, cmd: &mut Command) -> bool { | |
83c7162d | 690 | if self.config.dry_run { return true; } |
7cac9316 | 691 | self.verbose(&format!("running: {:?}", cmd)); |
ff7c6d11 | 692 | try_run_suppressed(cmd) |
7cac9316 XL |
693 | } |
694 | ||
041b39d2 XL |
695 | pub fn is_verbose(&self) -> bool { |
696 | self.verbosity > 0 | |
697 | } | |
698 | ||
5bcae85e SL |
699 | /// Prints a message if this build is configured in verbose mode. |
700 | fn verbose(&self, msg: &str) { | |
041b39d2 | 701 | if self.is_verbose() { |
5bcae85e SL |
702 | println!("{}", msg); |
703 | } | |
704 | } | |
705 | ||
532ac7d7 XL |
706 | pub fn is_verbose_than(&self, level: usize) -> bool { |
707 | self.verbosity > level | |
708 | } | |
709 | ||
710 | /// Prints a message if this build is configured in more verbose mode than `level`. | |
711 | fn verbose_than(&self, level: usize, msg: &str) { | |
712 | if self.is_verbose_than(level) { | |
713 | println!("{}", msg); | |
714 | } | |
715 | } | |
716 | ||
83c7162d XL |
717 | fn info(&self, msg: &str) { |
718 | if self.config.dry_run { return; } | |
719 | println!("{}", msg); | |
720 | } | |
721 | ||
5bcae85e SL |
722 | /// Returns the number of parallel jobs that have been configured for this |
723 | /// build. | |
724 | fn jobs(&self) -> u32 { | |
3b2f2976 | 725 | self.config.jobs.unwrap_or_else(|| num_cpus::get() as u32) |
5bcae85e SL |
726 | } |
727 | ||
b7449926 XL |
728 | fn debuginfo_map(&self, which: GitRepo) -> Option<String> { |
729 | if !self.config.rust_remap_debuginfo { | |
730 | return None | |
731 | } | |
732 | ||
733 | let path = match which { | |
734 | GitRepo::Rustc => { | |
0bf4aa26 | 735 | let sha = self.rust_sha().unwrap_or(channel::CFG_RELEASE_NUM); |
b7449926 XL |
736 | format!("/rustc/{}", sha) |
737 | } | |
a1dfa0c6 | 738 | GitRepo::Llvm => String::from("/rustc/llvm"), |
b7449926 XL |
739 | }; |
740 | Some(format!("{}={}", self.src.display(), path)) | |
741 | } | |
742 | ||
5bcae85e | 743 | /// Returns the path to the C compiler for the target specified. |
3b2f2976 | 744 | fn cc(&self, target: Interned<String>) -> &Path { |
abe05a73 | 745 | self.cc[&target].path() |
5bcae85e SL |
746 | } |
747 | ||
748 | /// Returns a list of flags to pass to the C compiler for the target | |
749 | /// specified. | |
b7449926 | 750 | fn cflags(&self, target: Interned<String>, which: GitRepo) -> Vec<String> { |
5bcae85e | 751 | // Filter out -O and /O (the optimization flags) that we picked up from |
ea8adc8c | 752 | // cc-rs because the build scripts will determine that for themselves. |
abe05a73 | 753 | let mut base = self.cc[&target].args().iter() |
5bcae85e SL |
754 | .map(|s| s.to_string_lossy().into_owned()) |
755 | .filter(|s| !s.starts_with("-O") && !s.starts_with("/O")) | |
a1dfa0c6 | 756 | .collect::<Vec<String>>(); |
5bcae85e | 757 | |
cc61c64b | 758 | // If we're compiling on macOS then we add a few unconditional flags |
5bcae85e SL |
759 | // indicating that we want libc++ (more filled out than libstdc++) and |
760 | // we want to compile for 10.7. This way we can ensure that | |
a1dfa0c6 | 761 | // LLVM/etc are all properly compiled. |
5bcae85e SL |
762 | if target.contains("apple-darwin") { |
763 | base.push("-stdlib=libc++".into()); | |
5bcae85e | 764 | } |
7cac9316 XL |
765 | |
766 | // Work around an apparently bad MinGW / GCC optimization, | |
767 | // See: http://lists.llvm.org/pipermail/cfe-dev/2016-December/051980.html | |
768 | // See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78936 | |
3b2f2976 | 769 | if &*target == "i686-pc-windows-gnu" { |
7cac9316 XL |
770 | base.push("-fno-omit-frame-pointer".into()); |
771 | } | |
b7449926 XL |
772 | |
773 | if let Some(map) = self.debuginfo_map(which) { | |
774 | let cc = self.cc(target); | |
775 | if cc.ends_with("clang") || cc.ends_with("gcc") { | |
a1dfa0c6 | 776 | base.push(format!("-fdebug-prefix-map={}", map)); |
b7449926 XL |
777 | } else if cc.ends_with("clang-cl.exe") { |
778 | base.push("-Xclang".into()); | |
a1dfa0c6 | 779 | base.push(format!("-fdebug-prefix-map={}", map)); |
b7449926 XL |
780 | } |
781 | } | |
041b39d2 | 782 | base |
5bcae85e SL |
783 | } |
784 | ||
785 | /// Returns the path to the `ar` archive utility for the target specified. | |
3b2f2976 | 786 | fn ar(&self, target: Interned<String>) -> Option<&Path> { |
abe05a73 | 787 | self.ar.get(&target).map(|p| &**p) |
5bcae85e SL |
788 | } |
789 | ||
b7449926 XL |
790 | /// Returns the path to the `ranlib` utility for the target specified. |
791 | fn ranlib(&self, target: Interned<String>) -> Option<&Path> { | |
792 | self.ranlib.get(&target).map(|p| &**p) | |
793 | } | |
794 | ||
041b39d2 | 795 | /// Returns the path to the C++ compiler for the target specified. |
3b2f2976 XL |
796 | fn cxx(&self, target: Interned<String>) -> Result<&Path, String> { |
797 | match self.cxx.get(&target) { | |
041b39d2 XL |
798 | Some(p) => Ok(p.path()), |
799 | None => Err(format!( | |
800 | "target `{}` is not configured as a host, only as a target", | |
801 | target)) | |
9e0c209e | 802 | } |
5bcae85e SL |
803 | } |
804 | ||
2c00a5a8 | 805 | /// Returns the path to the linker for the given target if it needs to be overridden. |
abe05a73 XL |
806 | fn linker(&self, target: Interned<String>) -> Option<&Path> { |
807 | if let Some(linker) = self.config.target_config.get(&target) | |
808 | .and_then(|c| c.linker.as_ref()) { | |
809 | Some(linker) | |
810 | } else if target != self.config.build && | |
60c5eb7d XL |
811 | util::use_host_linker(&target) && |
812 | !target.contains("msvc") { | |
abe05a73 XL |
813 | Some(self.cc(target)) |
814 | } else { | |
815 | None | |
5bcae85e | 816 | } |
85aaf69f | 817 | } |
9e0c209e | 818 | |
3b2f2976 XL |
819 | /// Returns if this target should statically link the C runtime, if specified |
820 | fn crt_static(&self, target: Interned<String>) -> Option<bool> { | |
821 | if target.contains("pc-windows-msvc") { | |
822 | Some(true) | |
823 | } else { | |
824 | self.config.target_config.get(&target) | |
825 | .and_then(|t| t.crt_static) | |
826 | } | |
827 | } | |
828 | ||
9e0c209e | 829 | /// Returns the "musl root" for this `target`, if defined |
3b2f2976 XL |
830 | fn musl_root(&self, target: Interned<String>) -> Option<&Path> { |
831 | self.config.target_config.get(&target) | |
c30ab7b3 | 832 | .and_then(|t| t.musl_root.as_ref()) |
9e0c209e SL |
833 | .or(self.config.musl_root.as_ref()) |
834 | .map(|p| &**p) | |
835 | } | |
476ff2be | 836 | |
532ac7d7 XL |
837 | /// Returns the sysroot for the wasi target, if defined |
838 | fn wasi_root(&self, target: Interned<String>) -> Option<&Path> { | |
839 | self.config.target_config.get(&target) | |
840 | .and_then(|t| t.wasi_root.as_ref()) | |
841 | .map(|p| &**p) | |
842 | } | |
843 | ||
9fa01778 | 844 | /// Returns `true` if this is a no-std `target`, if defined |
83c7162d XL |
845 | fn no_std(&self, target: Interned<String>) -> Option<bool> { |
846 | self.config.target_config.get(&target) | |
847 | .map(|t| t.no_std) | |
848 | } | |
849 | ||
9fa01778 | 850 | /// Returns `true` if the target will be tested using the `remote-test-client` |
7cac9316 | 851 | /// and `remote-test-server` binaries. |
3b2f2976 XL |
852 | fn remote_tested(&self, target: Interned<String>) -> bool { |
853 | self.qemu_rootfs(target).is_some() || target.contains("android") || | |
854 | env::var_os("TEST_DEVICE_ADDR").is_some() | |
7cac9316 XL |
855 | } |
856 | ||
8bb4bdeb XL |
857 | /// Returns the root of the "rootfs" image that this target will be using, |
858 | /// if one was configured. | |
859 | /// | |
860 | /// If `Some` is returned then that means that tests for this target are | |
861 | /// emulated with QEMU and binaries will need to be shipped to the emulator. | |
3b2f2976 XL |
862 | fn qemu_rootfs(&self, target: Interned<String>) -> Option<&Path> { |
863 | self.config.target_config.get(&target) | |
8bb4bdeb XL |
864 | .and_then(|t| t.qemu_rootfs.as_ref()) |
865 | .map(|p| &**p) | |
866 | } | |
867 | ||
476ff2be SL |
868 | /// Path to the python interpreter to use |
869 | fn python(&self) -> &Path { | |
870 | self.config.python.as_ref().unwrap() | |
871 | } | |
32a655c1 | 872 | |
ff7c6d11 XL |
873 | /// Temporary directory that extended error information is emitted to. |
874 | fn extended_error_dir(&self) -> PathBuf { | |
875 | self.out.join("tmp/extended-error-metadata") | |
876 | } | |
877 | ||
32a655c1 SL |
878 | /// Tests whether the `compiler` compiling for `target` should be forced to |
879 | /// use a stage1 compiler instead. | |
880 | /// | |
881 | /// Currently, by default, the build system does not perform a "full | |
882 | /// bootstrap" by default where we compile the compiler three times. | |
883 | /// Instead, we compile the compiler two times. The final stage (stage2) | |
884 | /// just copies the libraries from the previous stage, which is what this | |
885 | /// method detects. | |
886 | /// | |
887 | /// Here we return `true` if: | |
888 | /// | |
889 | /// * The build isn't performing a full bootstrap | |
890 | /// * The `compiler` is in the final stage, 2 | |
891 | /// * We're not cross-compiling, so the artifacts are already available in | |
892 | /// stage1 | |
893 | /// | |
894 | /// When all of these conditions are met the build will lift artifacts from | |
895 | /// the previous stage forward. | |
3b2f2976 | 896 | fn force_use_stage1(&self, compiler: Compiler, target: Interned<String>) -> bool { |
32a655c1 SL |
897 | !self.config.full_bootstrap && |
898 | compiler.stage >= 2 && | |
ea8adc8c | 899 | (self.hosts.iter().any(|h| *h == target) || target == self.build) |
32a655c1 | 900 | } |
8bb4bdeb | 901 | |
8bb4bdeb XL |
902 | /// Given `num` in the form "a.b.c" return a "release string" which |
903 | /// describes the release version number. | |
904 | /// | |
905 | /// For example on nightly this returns "a.b.c-nightly", on beta it returns | |
906 | /// "a.b.c-beta.1" and on stable it just returns "a.b.c". | |
907 | fn release(&self, num: &str) -> String { | |
908 | match &self.config.channel[..] { | |
909 | "stable" => num.to_string(), | |
ff7c6d11 XL |
910 | "beta" => if self.rust_info.is_git() { |
911 | format!("{}-beta.{}", num, self.beta_prerelease_version()) | |
912 | } else { | |
913 | format!("{}-beta", num) | |
914 | }, | |
8bb4bdeb XL |
915 | "nightly" => format!("{}-nightly", num), |
916 | _ => format!("{}-dev", num), | |
917 | } | |
918 | } | |
919 | ||
ff7c6d11 XL |
920 | fn beta_prerelease_version(&self) -> u32 { |
921 | if let Some(s) = self.prerelease_version.get() { | |
922 | return s | |
923 | } | |
924 | ||
925 | let beta = output( | |
926 | Command::new("git") | |
927 | .arg("ls-remote") | |
928 | .arg("origin") | |
929 | .arg("beta") | |
930 | .current_dir(&self.src) | |
931 | ); | |
932 | let beta = beta.trim().split_whitespace().next().unwrap(); | |
933 | let master = output( | |
934 | Command::new("git") | |
935 | .arg("ls-remote") | |
936 | .arg("origin") | |
937 | .arg("master") | |
938 | .current_dir(&self.src) | |
939 | ); | |
940 | let master = master.trim().split_whitespace().next().unwrap(); | |
941 | ||
942 | // Figure out where the current beta branch started. | |
943 | let base = output( | |
944 | Command::new("git") | |
945 | .arg("merge-base") | |
946 | .arg(beta) | |
947 | .arg(master) | |
948 | .current_dir(&self.src), | |
949 | ); | |
950 | let base = base.trim(); | |
951 | ||
952 | // Next figure out how many merge commits happened since we branched off | |
953 | // beta. That's our beta number! | |
954 | let count = output( | |
955 | Command::new("git") | |
956 | .arg("rev-list") | |
957 | .arg("--count") | |
958 | .arg("--merges") | |
959 | .arg(format!("{}...HEAD", base)) | |
960 | .current_dir(&self.src), | |
961 | ); | |
962 | let n = count.trim().parse().unwrap(); | |
963 | self.prerelease_version.set(Some(n)); | |
2c00a5a8 | 964 | n |
ff7c6d11 XL |
965 | } |
966 | ||
8bb4bdeb XL |
967 | /// Returns the value of `release` above for Rust itself. |
968 | fn rust_release(&self) -> String { | |
969 | self.release(channel::CFG_RELEASE_NUM) | |
970 | } | |
971 | ||
972 | /// Returns the "package version" for a component given the `num` release | |
973 | /// number. | |
974 | /// | |
975 | /// The package version is typically what shows up in the names of tarballs. | |
976 | /// For channels like beta/nightly it's just the channel name, otherwise | |
977 | /// it's the `num` provided. | |
978 | fn package_vers(&self, num: &str) -> String { | |
979 | match &self.config.channel[..] { | |
980 | "stable" => num.to_string(), | |
981 | "beta" => "beta".to_string(), | |
982 | "nightly" => "nightly".to_string(), | |
983 | _ => format!("{}-dev", num), | |
984 | } | |
985 | } | |
986 | ||
987 | /// Returns the value of `package_vers` above for Rust itself. | |
988 | fn rust_package_vers(&self) -> String { | |
989 | self.package_vers(channel::CFG_RELEASE_NUM) | |
990 | } | |
991 | ||
992 | /// Returns the value of `package_vers` above for Cargo | |
993 | fn cargo_package_vers(&self) -> String { | |
cc61c64b | 994 | self.package_vers(&self.release_num("cargo")) |
8bb4bdeb XL |
995 | } |
996 | ||
7cac9316 XL |
997 | /// Returns the value of `package_vers` above for rls |
998 | fn rls_package_vers(&self) -> String { | |
999 | self.package_vers(&self.release_num("rls")) | |
1000 | } | |
1001 | ||
8faf50e0 XL |
1002 | /// Returns the value of `package_vers` above for clippy |
1003 | fn clippy_package_vers(&self) -> String { | |
1004 | self.package_vers(&self.release_num("clippy")) | |
1005 | } | |
1006 | ||
0731742a XL |
1007 | /// Returns the value of `package_vers` above for miri |
1008 | fn miri_package_vers(&self) -> String { | |
1009 | self.package_vers(&self.release_num("miri")) | |
1010 | } | |
1011 | ||
abe05a73 XL |
1012 | /// Returns the value of `package_vers` above for rustfmt |
1013 | fn rustfmt_package_vers(&self) -> String { | |
1014 | self.package_vers(&self.release_num("rustfmt")) | |
1015 | } | |
1016 | ||
8faf50e0 | 1017 | fn llvm_tools_package_vers(&self) -> String { |
48663c56 | 1018 | self.package_vers(channel::CFG_RELEASE_NUM) |
8faf50e0 XL |
1019 | } |
1020 | ||
1021 | fn llvm_tools_vers(&self) -> String { | |
1022 | self.rust_version() | |
1023 | } | |
1024 | ||
b7449926 | 1025 | fn lldb_package_vers(&self) -> String { |
48663c56 | 1026 | self.package_vers(channel::CFG_RELEASE_NUM) |
b7449926 XL |
1027 | } |
1028 | ||
1029 | fn lldb_vers(&self) -> String { | |
1030 | self.rust_version() | |
1031 | } | |
1032 | ||
1033 | fn llvm_link_tools_dynamically(&self, target: Interned<String>) -> bool { | |
1034 | (target.contains("linux-gnu") || target.contains("apple-darwin")) | |
1035 | } | |
1036 | ||
8bb4bdeb XL |
1037 | /// Returns the `version` string associated with this compiler for Rust |
1038 | /// itself. | |
1039 | /// | |
1040 | /// Note that this is a descriptive string which includes the commit date, | |
1041 | /// sha, version, etc. | |
1042 | fn rust_version(&self) -> String { | |
1043 | self.rust_info.version(self, channel::CFG_RELEASE_NUM) | |
1044 | } | |
1045 | ||
9fa01778 | 1046 | /// Returns the full commit hash. |
ea8adc8c XL |
1047 | fn rust_sha(&self) -> Option<&str> { |
1048 | self.rust_info.sha() | |
1049 | } | |
1050 | ||
cc61c64b XL |
1051 | /// Returns the `a.b.c` version that the given package is at. |
1052 | fn release_num(&self, package: &str) -> String { | |
7cac9316 | 1053 | let toml_file_name = self.src.join(&format!("src/tools/{}/Cargo.toml", package)); |
0731742a | 1054 | let toml = t!(fs::read_to_string(&toml_file_name)); |
8bb4bdeb XL |
1055 | for line in toml.lines() { |
1056 | let prefix = "version = \""; | |
1057 | let suffix = "\""; | |
1058 | if line.starts_with(prefix) && line.ends_with(suffix) { | |
1059 | return line[prefix.len()..line.len() - suffix.len()].to_string() | |
1060 | } | |
1061 | } | |
1062 | ||
cc61c64b | 1063 | panic!("failed to find version in {}'s Cargo.toml", package) |
8bb4bdeb XL |
1064 | } |
1065 | ||
9fa01778 | 1066 | /// Returns `true` if unstable features should be enabled for the compiler |
8bb4bdeb XL |
1067 | /// we're building. |
1068 | fn unstable_features(&self) -> bool { | |
1069 | match &self.config.channel[..] { | |
1070 | "stable" | "beta" => false, | |
1071 | "nightly" | _ => true, | |
1072 | } | |
1073 | } | |
7cac9316 | 1074 | |
2c00a5a8 | 1075 | fn in_tree_crates(&self, root: &str) -> Vec<&Crate> { |
3b2f2976 | 1076 | let mut ret = Vec::new(); |
2c00a5a8 | 1077 | let mut list = vec![INTERNER.intern_str(root)]; |
3b2f2976 XL |
1078 | let mut visited = HashSet::new(); |
1079 | while let Some(krate) = list.pop() { | |
1080 | let krate = &self.crates[&krate]; | |
2c00a5a8 XL |
1081 | if krate.is_local(self) { |
1082 | ret.push(krate); | |
0731742a XL |
1083 | } |
1084 | for dep in &krate.deps { | |
1085 | if visited.insert(dep) && dep != "build_helper" { | |
1086 | list.push(*dep); | |
3b2f2976 XL |
1087 | } |
1088 | } | |
1089 | } | |
1090 | ret | |
1091 | } | |
83c7162d | 1092 | |
532ac7d7 | 1093 | fn read_stamp_file(&self, stamp: &Path) -> Vec<(PathBuf, bool)> { |
83c7162d XL |
1094 | if self.config.dry_run { |
1095 | return Vec::new(); | |
1096 | } | |
1097 | ||
1098 | let mut paths = Vec::new(); | |
e74abb32 | 1099 | let contents = t!(fs::read(stamp), &stamp); |
83c7162d XL |
1100 | // This is the method we use for extracting paths from the stamp file passed to us. See |
1101 | // run_cargo for more information (in compile.rs). | |
1102 | for part in contents.split(|b| *b == 0) { | |
1103 | if part.is_empty() { | |
1104 | continue | |
1105 | } | |
532ac7d7 XL |
1106 | let host = part[0] as char == 'h'; |
1107 | let path = PathBuf::from(t!(str::from_utf8(&part[1..]))); | |
1108 | paths.push((path, host)); | |
83c7162d XL |
1109 | } |
1110 | paths | |
1111 | } | |
1112 | ||
1113 | /// Copies a file from `src` to `dst` | |
1114 | pub fn copy(&self, src: &Path, dst: &Path) { | |
1115 | if self.config.dry_run { return; } | |
532ac7d7 | 1116 | self.verbose_than(1, &format!("Copy {:?} to {:?}", src, dst)); |
e74abb32 | 1117 | if src == dst { return; } |
83c7162d | 1118 | let _ = fs::remove_file(&dst); |
b7449926 XL |
1119 | let metadata = t!(src.symlink_metadata()); |
1120 | if metadata.file_type().is_symlink() { | |
1121 | let link = t!(fs::read_link(src)); | |
1122 | t!(symlink_file(link, dst)); | |
1123 | } else if let Ok(()) = fs::hard_link(src, dst) { | |
1124 | // Attempt to "easy copy" by creating a hard link | |
1125 | // (symlinks don't work on windows), but if that fails | |
1126 | // just fall back to a slow `copy` operation. | |
1127 | } else { | |
1128 | if let Err(e) = fs::copy(src, dst) { | |
1129 | panic!("failed to copy `{}` to `{}`: {}", src.display(), | |
1130 | dst.display(), e) | |
1131 | } | |
1132 | t!(fs::set_permissions(dst, metadata.permissions())); | |
1133 | let atime = FileTime::from_last_access_time(&metadata); | |
1134 | let mtime = FileTime::from_last_modification_time(&metadata); | |
1135 | t!(filetime::set_file_times(dst, atime, mtime)); | |
83c7162d | 1136 | } |
83c7162d XL |
1137 | } |
1138 | ||
1139 | /// Search-and-replaces within a file. (Not maximally efficiently: allocates a | |
1140 | /// new string for each replacement.) | |
1141 | pub fn replace_in_file(&self, path: &Path, replacements: &[(&str, &str)]) { | |
1142 | if self.config.dry_run { return; } | |
1143 | let mut contents = String::new(); | |
1144 | let mut file = t!(OpenOptions::new().read(true).write(true).open(path)); | |
1145 | t!(file.read_to_string(&mut contents)); | |
1146 | for &(target, replacement) in replacements { | |
1147 | contents = contents.replace(target, replacement); | |
1148 | } | |
1149 | t!(file.seek(SeekFrom::Start(0))); | |
1150 | t!(file.set_len(0)); | |
1151 | t!(file.write_all(contents.as_bytes())); | |
1152 | } | |
1153 | ||
1154 | /// Copies the `src` directory recursively to `dst`. Both are assumed to exist | |
1155 | /// when this function is called. | |
1156 | pub fn cp_r(&self, src: &Path, dst: &Path) { | |
1157 | if self.config.dry_run { return; } | |
dc9dc135 | 1158 | for f in self.read_dir(src) { |
83c7162d XL |
1159 | let path = f.path(); |
1160 | let name = path.file_name().unwrap(); | |
1161 | let dst = dst.join(name); | |
1162 | if t!(f.file_type()).is_dir() { | |
1163 | t!(fs::create_dir_all(&dst)); | |
1164 | self.cp_r(&path, &dst); | |
1165 | } else { | |
1166 | let _ = fs::remove_file(&dst); | |
1167 | self.copy(&path, &dst); | |
1168 | } | |
1169 | } | |
1170 | } | |
1171 | ||
1172 | /// Copies the `src` directory recursively to `dst`. Both are assumed to exist | |
1173 | /// when this function is called. Unwanted files or directories can be skipped | |
1174 | /// by returning `false` from the filter function. | |
8faf50e0 | 1175 | pub fn cp_filtered(&self, src: &Path, dst: &Path, filter: &dyn Fn(&Path) -> bool) { |
83c7162d XL |
1176 | // Immediately recurse with an empty relative path |
1177 | self.recurse_(src, dst, Path::new(""), filter) | |
1178 | } | |
1179 | ||
1180 | // Inner function does the actual work | |
8faf50e0 | 1181 | fn recurse_(&self, src: &Path, dst: &Path, relative: &Path, filter: &dyn Fn(&Path) -> bool) { |
83c7162d XL |
1182 | for f in self.read_dir(src) { |
1183 | let path = f.path(); | |
1184 | let name = path.file_name().unwrap(); | |
1185 | let dst = dst.join(name); | |
1186 | let relative = relative.join(name); | |
1187 | // Only copy file or directory if the filter function returns true | |
1188 | if filter(&relative) { | |
1189 | if t!(f.file_type()).is_dir() { | |
1190 | let _ = fs::remove_dir_all(&dst); | |
1191 | self.create_dir(&dst); | |
1192 | self.recurse_(&path, &dst, &relative, filter); | |
1193 | } else { | |
1194 | let _ = fs::remove_file(&dst); | |
1195 | self.copy(&path, &dst); | |
1196 | } | |
1197 | } | |
1198 | } | |
1199 | } | |
1200 | ||
1201 | fn copy_to_folder(&self, src: &Path, dest_folder: &Path) { | |
1202 | let file_name = src.file_name().unwrap(); | |
1203 | let dest = dest_folder.join(file_name); | |
1204 | self.copy(src, &dest); | |
1205 | } | |
1206 | ||
1207 | fn install(&self, src: &Path, dstdir: &Path, perms: u32) { | |
1208 | if self.config.dry_run { return; } | |
1209 | let dst = dstdir.join(src.file_name().unwrap()); | |
532ac7d7 | 1210 | self.verbose_than(1, &format!("Install {:?} to {:?}", src, dst)); |
83c7162d XL |
1211 | t!(fs::create_dir_all(dstdir)); |
1212 | drop(fs::remove_file(&dst)); | |
1213 | { | |
0bf4aa26 XL |
1214 | if !src.exists() { |
1215 | panic!("Error: File \"{}\" not found!", src.display()); | |
1216 | } | |
9fa01778 XL |
1217 | let metadata = t!(src.symlink_metadata()); |
1218 | if let Err(e) = fs::copy(&src, &dst) { | |
1219 | panic!("failed to copy `{}` to `{}`: {}", src.display(), | |
1220 | dst.display(), e) | |
1221 | } | |
1222 | t!(fs::set_permissions(&dst, metadata.permissions())); | |
1223 | let atime = FileTime::from_last_access_time(&metadata); | |
1224 | let mtime = FileTime::from_last_modification_time(&metadata); | |
1225 | t!(filetime::set_file_times(&dst, atime, mtime)); | |
83c7162d XL |
1226 | } |
1227 | chmod(&dst, perms); | |
1228 | } | |
1229 | ||
1230 | fn create(&self, path: &Path, s: &str) { | |
1231 | if self.config.dry_run { return; } | |
1232 | t!(fs::write(path, s)); | |
1233 | } | |
1234 | ||
1235 | fn read(&self, path: &Path) -> String { | |
1236 | if self.config.dry_run { return String::new(); } | |
1237 | t!(fs::read_to_string(path)) | |
1238 | } | |
1239 | ||
1240 | fn create_dir(&self, dir: &Path) { | |
1241 | if self.config.dry_run { return; } | |
1242 | t!(fs::create_dir_all(dir)) | |
1243 | } | |
1244 | ||
1245 | fn remove_dir(&self, dir: &Path) { | |
1246 | if self.config.dry_run { return; } | |
1247 | t!(fs::remove_dir_all(dir)) | |
1248 | } | |
1249 | ||
1250 | fn read_dir(&self, dir: &Path) -> impl Iterator<Item=fs::DirEntry> { | |
1251 | let iter = match fs::read_dir(dir) { | |
1252 | Ok(v) => v, | |
1253 | Err(_) if self.config.dry_run => return vec![].into_iter(), | |
1254 | Err(err) => panic!("could not read dir {:?}: {:?}", dir, err), | |
1255 | }; | |
1256 | iter.map(|e| t!(e)).collect::<Vec<_>>().into_iter() | |
1257 | } | |
1258 | ||
1259 | fn remove(&self, f: &Path) { | |
1260 | if self.config.dry_run { return; } | |
1261 | fs::remove_file(f).unwrap_or_else(|_| panic!("failed to remove {:?}", f)); | |
1262 | } | |
85aaf69f SL |
1263 | } |
1264 | ||
83c7162d XL |
1265 | #[cfg(unix)] |
1266 | fn chmod(path: &Path, perms: u32) { | |
1267 | use std::os::unix::fs::*; | |
1268 | t!(fs::set_permissions(path, fs::Permissions::from_mode(perms))); | |
1269 | } | |
1270 | #[cfg(windows)] | |
1271 | fn chmod(_path: &Path, _perms: u32) {} | |
1272 | ||
1273 | ||
416331ca | 1274 | impl Compiler { |
3b2f2976 XL |
1275 | pub fn with_stage(mut self, stage: u32) -> Compiler { |
1276 | self.stage = stage; | |
1277 | self | |
5bcae85e SL |
1278 | } |
1279 | ||
9fa01778 | 1280 | /// Returns `true` if this is a snapshot compiler for `build`'s configuration |
3b2f2976 | 1281 | pub fn is_snapshot(&self, build: &Build) -> bool { |
041b39d2 | 1282 | self.stage == 0 && self.host == build.build |
5bcae85e | 1283 | } |
32a655c1 SL |
1284 | |
1285 | /// Returns if this compiler should be treated as a final stage one in the | |
1286 | /// current build session. | |
1287 | /// This takes into account whether we're performing a full bootstrap or | |
1288 | /// not; don't directly compare the stage with `2`! | |
3b2f2976 | 1289 | pub fn is_final_stage(&self, build: &Build) -> bool { |
32a655c1 SL |
1290 | let final_stage = if build.config.full_bootstrap { 2 } else { 1 }; |
1291 | self.stage >= final_stage | |
1292 | } | |
223e47cc | 1293 | } |
e1599b0c XL |
1294 | |
1295 | fn envify(s: &str) -> String { | |
1296 | s.chars() | |
1297 | .map(|c| match c { | |
1298 | '-' => '_', | |
1299 | c => c, | |
1300 | }) | |
1301 | .flat_map(|c| c.to_uppercase()) | |
1302 | .collect() | |
1303 | } |