]>
Commit | Line | Data |
---|---|---|
85aaf69f | 1 | // Copyright 2012-2015 The Rust Project Developers. See the COPYRIGHT |
223e47cc LB |
2 | // file at the top-level directory of this distribution and at |
3 | // http://rust-lang.org/COPYRIGHT. | |
4 | // | |
5 | // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or | |
6 | // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license | |
7 | // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your | |
8 | // option. This file may not be copied, modified, or distributed | |
9 | // except according to those terms. | |
10 | ||
223e47cc | 11 | //! Finds crate binaries and loads their metadata |
1a4d82fc JJ |
12 | //! |
13 | //! Might I be the first to welcome you to a world of platform differences, | |
14 | //! version requirements, dependency graphs, conflicting desires, and fun! This | |
15 | //! is the major guts (along with metadata::creader) of the compiler for loading | |
16 | //! crates and resolving dependencies. Let's take a tour! | |
17 | //! | |
18 | //! # The problem | |
19 | //! | |
20 | //! Each invocation of the compiler is immediately concerned with one primary | |
21 | //! problem, to connect a set of crates to resolved crates on the filesystem. | |
22 | //! Concretely speaking, the compiler follows roughly these steps to get here: | |
23 | //! | |
24 | //! 1. Discover a set of `extern crate` statements. | |
25 | //! 2. Transform these directives into crate names. If the directive does not | |
26 | //! have an explicit name, then the identifier is the name. | |
27 | //! 3. For each of these crate names, find a corresponding crate on the | |
28 | //! filesystem. | |
29 | //! | |
30 | //! Sounds easy, right? Let's walk into some of the nuances. | |
31 | //! | |
32 | //! ## Transitive Dependencies | |
33 | //! | |
34 | //! Let's say we've got three crates: A, B, and C. A depends on B, and B depends | |
35 | //! on C. When we're compiling A, we primarily need to find and locate B, but we | |
36 | //! also end up needing to find and locate C as well. | |
37 | //! | |
38 | //! The reason for this is that any of B's types could be composed of C's types, | |
39 | //! any function in B could return a type from C, etc. To be able to guarantee | |
40 | //! that we can always typecheck/translate any function, we have to have | |
41 | //! complete knowledge of the whole ecosystem, not just our immediate | |
42 | //! dependencies. | |
43 | //! | |
44 | //! So now as part of the "find a corresponding crate on the filesystem" step | |
45 | //! above, this involves also finding all crates for *all upstream | |
46 | //! dependencies*. This includes all dependencies transitively. | |
47 | //! | |
48 | //! ## Rlibs and Dylibs | |
49 | //! | |
50 | //! The compiler has two forms of intermediate dependencies. These are dubbed | |
51 | //! rlibs and dylibs for the static and dynamic variants, respectively. An rlib | |
52 | //! is a rustc-defined file format (currently just an ar archive) while a dylib | |
53 | //! is a platform-defined dynamic library. Each library has a metadata somewhere | |
54 | //! inside of it. | |
55 | //! | |
56 | //! When translating a crate name to a crate on the filesystem, we all of a | |
57 | //! sudden need to take into account both rlibs and dylibs! Linkage later on may | |
58 | //! use either one of these files, as each has their pros/cons. The job of crate | |
59 | //! loading is to discover what's possible by finding all candidates. | |
60 | //! | |
61 | //! Most parts of this loading systems keep the dylib/rlib as just separate | |
62 | //! variables. | |
63 | //! | |
64 | //! ## Where to look? | |
65 | //! | |
66 | //! We can't exactly scan your whole hard drive when looking for dependencies, | |
67 | //! so we need to places to look. Currently the compiler will implicitly add the | |
68 | //! target lib search path ($prefix/lib/rustlib/$target/lib) to any compilation, | |
69 | //! and otherwise all -L flags are added to the search paths. | |
70 | //! | |
71 | //! ## What criterion to select on? | |
72 | //! | |
73 | //! This a pretty tricky area of loading crates. Given a file, how do we know | |
74 | //! whether it's the right crate? Currently, the rules look along these lines: | |
75 | //! | |
76 | //! 1. Does the filename match an rlib/dylib pattern? That is to say, does the | |
77 | //! filename have the right prefix/suffix? | |
78 | //! 2. Does the filename have the right prefix for the crate name being queried? | |
79 | //! This is filtering for files like `libfoo*.rlib` and such. | |
80 | //! 3. Is the file an actual rust library? This is done by loading the metadata | |
81 | //! from the library and making sure it's actually there. | |
82 | //! 4. Does the name in the metadata agree with the name of the library? | |
83 | //! 5. Does the target in the metadata agree with the current target? | |
84 | //! 6. Does the SVH match? (more on this later) | |
85 | //! | |
86 | //! If the file answers `yes` to all these questions, then the file is | |
87 | //! considered as being *candidate* for being accepted. It is illegal to have | |
88 | //! more than two candidates as the compiler has no method by which to resolve | |
89 | //! this conflict. Additionally, rlib/dylib candidates are considered | |
90 | //! separately. | |
91 | //! | |
92 | //! After all this has happened, we have 1 or two files as candidates. These | |
93 | //! represent the rlib/dylib file found for a library, and they're returned as | |
94 | //! being found. | |
95 | //! | |
96 | //! ### What about versions? | |
97 | //! | |
98 | //! A lot of effort has been put forth to remove versioning from the compiler. | |
99 | //! There have been forays in the past to have versioning baked in, but it was | |
100 | //! largely always deemed insufficient to the point that it was recognized that | |
101 | //! it's probably something the compiler shouldn't do anyway due to its | |
102 | //! complicated nature and the state of the half-baked solutions. | |
103 | //! | |
104 | //! With a departure from versioning, the primary criterion for loading crates | |
105 | //! is just the name of a crate. If we stopped here, it would imply that you | |
106 | //! could never link two crates of the same name from different sources | |
107 | //! together, which is clearly a bad state to be in. | |
108 | //! | |
109 | //! To resolve this problem, we come to the next section! | |
110 | //! | |
111 | //! # Expert Mode | |
112 | //! | |
113 | //! A number of flags have been added to the compiler to solve the "version | |
114 | //! problem" in the previous section, as well as generally enabling more | |
115 | //! powerful usage of the crate loading system of the compiler. The goal of | |
116 | //! these flags and options are to enable third-party tools to drive the | |
117 | //! compiler with prior knowledge about how the world should look. | |
118 | //! | |
119 | //! ## The `--extern` flag | |
120 | //! | |
121 | //! The compiler accepts a flag of this form a number of times: | |
122 | //! | |
123 | //! ```text | |
124 | //! --extern crate-name=path/to/the/crate.rlib | |
125 | //! ``` | |
126 | //! | |
127 | //! This flag is basically the following letter to the compiler: | |
128 | //! | |
129 | //! > Dear rustc, | |
130 | //! > | |
131 | //! > When you are attempting to load the immediate dependency `crate-name`, I | |
9346a6ac | 132 | //! > would like you to assume that the library is located at |
1a4d82fc JJ |
133 | //! > `path/to/the/crate.rlib`, and look nowhere else. Also, please do not |
134 | //! > assume that the path I specified has the name `crate-name`. | |
135 | //! | |
136 | //! This flag basically overrides most matching logic except for validating that | |
137 | //! the file is indeed a rust library. The same `crate-name` can be specified | |
138 | //! twice to specify the rlib/dylib pair. | |
139 | //! | |
140 | //! ## Enabling "multiple versions" | |
141 | //! | |
142 | //! This basically boils down to the ability to specify arbitrary packages to | |
143 | //! the compiler. For example, if crate A wanted to use Bv1 and Bv2, then it | |
144 | //! would look something like: | |
145 | //! | |
146 | //! ```ignore | |
147 | //! extern crate b1; | |
148 | //! extern crate b2; | |
149 | //! | |
150 | //! fn main() {} | |
151 | //! ``` | |
152 | //! | |
153 | //! and the compiler would be invoked as: | |
154 | //! | |
155 | //! ```text | |
156 | //! rustc a.rs --extern b1=path/to/libb1.rlib --extern b2=path/to/libb2.rlib | |
157 | //! ``` | |
158 | //! | |
159 | //! In this scenario there are two crates named `b` and the compiler must be | |
160 | //! manually driven to be informed where each crate is. | |
161 | //! | |
162 | //! ## Frobbing symbols | |
163 | //! | |
164 | //! One of the immediate problems with linking the same library together twice | |
165 | //! in the same problem is dealing with duplicate symbols. The primary way to | |
166 | //! deal with this in rustc is to add hashes to the end of each symbol. | |
167 | //! | |
168 | //! In order to force hashes to change between versions of a library, if | |
169 | //! desired, the compiler exposes an option `-C metadata=foo`, which is used to | |
170 | //! initially seed each symbol hash. The string `foo` is prepended to each | |
171 | //! string-to-hash to ensure that symbols change over time. | |
172 | //! | |
173 | //! ## Loading transitive dependencies | |
174 | //! | |
175 | //! Dealing with same-named-but-distinct crates is not just a local problem, but | |
176 | //! one that also needs to be dealt with for transitive dependencies. Note that | |
177 | //! in the letter above `--extern` flags only apply to the *local* set of | |
178 | //! dependencies, not the upstream transitive dependencies. Consider this | |
179 | //! dependency graph: | |
180 | //! | |
181 | //! ```text | |
182 | //! A.1 A.2 | |
183 | //! | | | |
184 | //! | | | |
185 | //! B C | |
186 | //! \ / | |
187 | //! \ / | |
188 | //! D | |
189 | //! ``` | |
190 | //! | |
191 | //! In this scenario, when we compile `D`, we need to be able to distinctly | |
192 | //! resolve `A.1` and `A.2`, but an `--extern` flag cannot apply to these | |
193 | //! transitive dependencies. | |
194 | //! | |
195 | //! Note that the key idea here is that `B` and `C` are both *already compiled*. | |
196 | //! That is, they have already resolved their dependencies. Due to unrelated | |
197 | //! technical reasons, when a library is compiled, it is only compatible with | |
198 | //! the *exact same* version of the upstream libraries it was compiled against. | |
199 | //! We use the "Strict Version Hash" to identify the exact copy of an upstream | |
200 | //! library. | |
201 | //! | |
202 | //! With this knowledge, we know that `B` and `C` will depend on `A` with | |
203 | //! different SVH values, so we crawl the normal `-L` paths looking for | |
204 | //! `liba*.rlib` and filter based on the contained SVH. | |
205 | //! | |
206 | //! In the end, this ends up not needing `--extern` to specify upstream | |
207 | //! transitive dependencies. | |
208 | //! | |
209 | //! # Wrapping up | |
210 | //! | |
211 | //! That's the general overview of loading crates in the compiler, but it's by | |
212 | //! no means all of the necessary details. Take a look at the rest of | |
213 | //! metadata::loader or metadata::creader for all the juicy details! | |
223e47cc | 214 | |
92a42be0 | 215 | use cstore::{MetadataBlob, MetadataVec, MetadataArchive}; |
a7813a04 | 216 | use common::{metadata_encoding_version, rustc_version}; |
92a42be0 | 217 | use decoder; |
92a42be0 | 218 | |
54a0048b | 219 | use rustc::hir::svh::Svh; |
92a42be0 SL |
220 | use rustc::session::Session; |
221 | use rustc::session::filesearch::{FileSearch, FileMatches, FileDoesntMatch}; | |
222 | use rustc::session::search_paths::PathKind; | |
223 | use rustc::util::common; | |
224 | ||
225 | use rustc_llvm as llvm; | |
226 | use rustc_llvm::{False, ObjectFile, mk_section_iter}; | |
227 | use rustc_llvm::archive_ro::ArchiveRO; | |
3157f602 XL |
228 | use errors::DiagnosticBuilder; |
229 | use syntax_pos::Span; | |
85aaf69f | 230 | use rustc_back::target::Target; |
1a4d82fc | 231 | |
1a4d82fc | 232 | use std::cmp; |
85aaf69f | 233 | use std::collections::HashMap; |
a7813a04 | 234 | use std::fmt; |
d9579d0f | 235 | use std::fs; |
c34b1796 AL |
236 | use std::io; |
237 | use std::path::{Path, PathBuf}; | |
970d7e83 | 238 | use std::ptr; |
1a4d82fc | 239 | use std::slice; |
92a42be0 | 240 | use std::time::Instant; |
1a4d82fc JJ |
241 | |
242 | use flate; | |
243 | ||
244 | pub struct CrateMismatch { | |
c34b1796 | 245 | path: PathBuf, |
1a4d82fc | 246 | got: String, |
223e47cc LB |
247 | } |
248 | ||
1a4d82fc JJ |
249 | pub struct Context<'a> { |
250 | pub sess: &'a Session, | |
251 | pub span: Span, | |
252 | pub ident: &'a str, | |
253 | pub crate_name: &'a str, | |
254 | pub hash: Option<&'a Svh>, | |
85aaf69f SL |
255 | // points to either self.sess.target.target or self.sess.host, must match triple |
256 | pub target: &'a Target, | |
1a4d82fc JJ |
257 | pub triple: &'a str, |
258 | pub filesearch: FileSearch<'a>, | |
259 | pub root: &'a Option<CratePaths>, | |
260 | pub rejected_via_hash: Vec<CrateMismatch>, | |
261 | pub rejected_via_triple: Vec<CrateMismatch>, | |
85aaf69f | 262 | pub rejected_via_kind: Vec<CrateMismatch>, |
a7813a04 | 263 | pub rejected_via_version: Vec<CrateMismatch>, |
1a4d82fc | 264 | pub should_match_name: bool, |
223e47cc LB |
265 | } |
266 | ||
1a4d82fc | 267 | pub struct Library { |
c34b1796 AL |
268 | pub dylib: Option<(PathBuf, PathKind)>, |
269 | pub rlib: Option<(PathBuf, PathKind)>, | |
1a4d82fc | 270 | pub metadata: MetadataBlob, |
223e47cc LB |
271 | } |
272 | ||
1a4d82fc JJ |
273 | pub struct ArchiveMetadata { |
274 | _archive: ArchiveRO, | |
275 | // points into self._archive | |
276 | data: *const [u8], | |
223e47cc LB |
277 | } |
278 | ||
1a4d82fc JJ |
279 | pub struct CratePaths { |
280 | pub ident: String, | |
c34b1796 AL |
281 | pub dylib: Option<PathBuf>, |
282 | pub rlib: Option<PathBuf> | |
223e47cc LB |
283 | } |
284 | ||
c1a9b12d SL |
285 | pub const METADATA_FILENAME: &'static str = "rust.metadata.bin"; |
286 | ||
a7813a04 XL |
287 | #[derive(Copy, Clone, PartialEq)] |
288 | enum CrateFlavor { | |
289 | Rlib, | |
290 | Dylib | |
291 | } | |
292 | ||
293 | impl fmt::Display for CrateFlavor { | |
294 | fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { | |
295 | f.write_str(match *self { | |
296 | CrateFlavor::Rlib => "rlib", | |
297 | CrateFlavor::Dylib => "dylib" | |
298 | }) | |
299 | } | |
300 | } | |
301 | ||
1a4d82fc | 302 | impl CratePaths { |
c34b1796 | 303 | fn paths(&self) -> Vec<PathBuf> { |
1a4d82fc JJ |
304 | match (&self.dylib, &self.rlib) { |
305 | (&None, &None) => vec!(), | |
306 | (&Some(ref p), &None) | | |
307 | (&None, &Some(ref p)) => vec!(p.clone()), | |
308 | (&Some(ref p1), &Some(ref p2)) => vec!(p1.clone(), p2.clone()), | |
309 | } | |
310 | } | |
311 | } | |
312 | ||
313 | impl<'a> Context<'a> { | |
314 | pub fn maybe_load_library_crate(&mut self) -> Option<Library> { | |
315 | self.find_library_crate() | |
316 | } | |
317 | ||
318 | pub fn load_library_crate(&mut self) -> Library { | |
54a0048b | 319 | self.find_library_crate().unwrap_or_else(|| self.report_load_errs()) |
1a4d82fc JJ |
320 | } |
321 | ||
54a0048b | 322 | pub fn report_load_errs(&mut self) -> ! { |
b039eaaf SL |
323 | let add = match self.root { |
324 | &None => String::new(), | |
325 | &Some(ref r) => format!(" which `{}` depends on", | |
326 | r.ident) | |
327 | }; | |
9cc50fc6 SL |
328 | let mut err = if !self.rejected_via_hash.is_empty() { |
329 | struct_span_err!(self.sess, self.span, E0460, | |
330 | "found possibly newer version of crate `{}`{}", | |
331 | self.ident, add) | |
9346a6ac | 332 | } else if !self.rejected_via_triple.is_empty() { |
9cc50fc6 SL |
333 | struct_span_err!(self.sess, self.span, E0461, |
334 | "couldn't find crate `{}` with expected target triple {}{}", | |
335 | self.ident, self.triple, add) | |
9346a6ac | 336 | } else if !self.rejected_via_kind.is_empty() { |
9cc50fc6 SL |
337 | struct_span_err!(self.sess, self.span, E0462, |
338 | "found staticlib `{}` instead of rlib or dylib{}", | |
339 | self.ident, add) | |
a7813a04 XL |
340 | } else if !self.rejected_via_version.is_empty() { |
341 | struct_span_err!(self.sess, self.span, E0514, | |
342 | "found crate `{}` compiled by an incompatible version of rustc{}", | |
343 | self.ident, add) | |
1a4d82fc | 344 | } else { |
9cc50fc6 SL |
345 | struct_span_err!(self.sess, self.span, E0463, |
346 | "can't find crate for `{}`{}", | |
347 | self.ident, add) | |
348 | }; | |
1a4d82fc | 349 | |
9346a6ac | 350 | if !self.rejected_via_triple.is_empty() { |
1a4d82fc JJ |
351 | let mismatches = self.rejected_via_triple.iter(); |
352 | for (i, &CrateMismatch{ ref path, ref got }) in mismatches.enumerate() { | |
a7813a04 XL |
353 | err.note(&format!("crate `{}`, path #{}, triple {}: {}", |
354 | self.ident, i+1, got, path.display())); | |
1a4d82fc JJ |
355 | } |
356 | } | |
9346a6ac | 357 | if !self.rejected_via_hash.is_empty() { |
a7813a04 | 358 | err.note("perhaps that crate needs to be recompiled?"); |
1a4d82fc JJ |
359 | let mismatches = self.rejected_via_hash.iter(); |
360 | for (i, &CrateMismatch{ ref path, .. }) in mismatches.enumerate() { | |
a7813a04 XL |
361 | err.note(&format!("crate `{}` path #{}: {}", |
362 | self.ident, i+1, path.display())); | |
1a4d82fc JJ |
363 | } |
364 | match self.root { | |
365 | &None => {} | |
366 | &Some(ref r) => { | |
367 | for (i, path) in r.paths().iter().enumerate() { | |
a7813a04 XL |
368 | err.note(&format!("crate `{}` path #{}: {}", |
369 | r.ident, i+1, path.display())); | |
970d7e83 | 370 | } |
223e47cc | 371 | } |
223e47cc | 372 | } |
1a4d82fc | 373 | } |
9346a6ac | 374 | if !self.rejected_via_kind.is_empty() { |
a7813a04 | 375 | err.help("please recompile that crate using --crate-type lib"); |
85aaf69f SL |
376 | let mismatches = self.rejected_via_kind.iter(); |
377 | for (i, &CrateMismatch { ref path, .. }) in mismatches.enumerate() { | |
a7813a04 XL |
378 | err.note(&format!("crate `{}` path #{}: {}", |
379 | self.ident, i+1, path.display())); | |
380 | } | |
381 | } | |
382 | if !self.rejected_via_version.is_empty() { | |
383 | err.help(&format!("please recompile that crate using this compiler ({})", | |
384 | rustc_version())); | |
385 | let mismatches = self.rejected_via_version.iter(); | |
386 | for (i, &CrateMismatch { ref path, ref got }) in mismatches.enumerate() { | |
387 | err.note(&format!("crate `{}` path #{}: {} compiled by {:?}", | |
388 | self.ident, i+1, path.display(), got)); | |
85aaf69f SL |
389 | } |
390 | } | |
9cc50fc6 SL |
391 | |
392 | err.emit(); | |
1a4d82fc | 393 | self.sess.abort_if_errors(); |
54a0048b | 394 | unreachable!(); |
1a4d82fc JJ |
395 | } |
396 | ||
397 | fn find_library_crate(&mut self) -> Option<Library> { | |
398 | // If an SVH is specified, then this is a transitive dependency that | |
399 | // must be loaded via -L plus some filtering. | |
400 | if self.hash.is_none() { | |
401 | self.should_match_name = false; | |
85aaf69f | 402 | if let Some(s) = self.sess.opts.externs.get(self.crate_name) { |
5bcae85e | 403 | return self.find_commandline_library(s.iter()); |
1a4d82fc JJ |
404 | } |
405 | self.should_match_name = true; | |
406 | } | |
407 | ||
408 | let dypair = self.dylibname(); | |
7453a54e | 409 | let staticpair = self.staticlibname(); |
1a4d82fc JJ |
410 | |
411 | // want: crate_name.dir_part() + prefix + crate_name.file_part + "-" | |
412 | let dylib_prefix = format!("{}{}", dypair.0, self.crate_name); | |
413 | let rlib_prefix = format!("lib{}", self.crate_name); | |
7453a54e | 414 | let staticlib_prefix = format!("{}{}", staticpair.0, self.crate_name); |
1a4d82fc JJ |
415 | |
416 | let mut candidates = HashMap::new(); | |
85aaf69f | 417 | let mut staticlibs = vec!(); |
1a4d82fc JJ |
418 | |
419 | // First, find all possible candidate rlibs and dylibs purely based on | |
420 | // the name of the files themselves. We're trying to match against an | |
421 | // exact crate name and a possibly an exact hash. | |
422 | // | |
423 | // During this step, we can filter all found libraries based on the | |
424 | // name and id found in the crate id (we ignore the path portion for | |
425 | // filename matching), as well as the exact hash (if specified). If we | |
426 | // end up having many candidates, we must look at the metadata to | |
427 | // perform exact matches against hashes/crate ids. Note that opening up | |
428 | // the metadata is where we do an exact match against the full contents | |
429 | // of the crate id (path/name/id). | |
430 | // | |
431 | // The goal of this step is to look at as little metadata as possible. | |
85aaf69f | 432 | self.filesearch.search(|path, kind| { |
c34b1796 | 433 | let file = match path.file_name().and_then(|s| s.to_str()) { |
1a4d82fc JJ |
434 | None => return FileDoesntMatch, |
435 | Some(file) => file, | |
436 | }; | |
85aaf69f SL |
437 | let (hash, rlib) = if file.starts_with(&rlib_prefix[..]) && |
438 | file.ends_with(".rlib") { | |
439 | (&file[(rlib_prefix.len()) .. (file.len() - ".rlib".len())], | |
1a4d82fc | 440 | true) |
85aaf69f SL |
441 | } else if file.starts_with(&dylib_prefix) && |
442 | file.ends_with(&dypair.1) { | |
443 | (&file[(dylib_prefix.len()) .. (file.len() - dypair.1.len())], | |
1a4d82fc JJ |
444 | false) |
445 | } else { | |
85aaf69f | 446 | if file.starts_with(&staticlib_prefix[..]) && |
7453a54e | 447 | file.ends_with(&staticpair.1) { |
85aaf69f | 448 | staticlibs.push(CrateMismatch { |
c34b1796 | 449 | path: path.to_path_buf(), |
85aaf69f SL |
450 | got: "static".to_string() |
451 | }); | |
452 | } | |
1a4d82fc JJ |
453 | return FileDoesntMatch |
454 | }; | |
455 | info!("lib candidate: {}", path.display()); | |
456 | ||
457 | let hash_str = hash.to_string(); | |
c34b1796 AL |
458 | let slot = candidates.entry(hash_str) |
459 | .or_insert_with(|| (HashMap::new(), HashMap::new())); | |
1a4d82fc | 460 | let (ref mut rlibs, ref mut dylibs) = *slot; |
62682a34 SL |
461 | fs::canonicalize(path).map(|p| { |
462 | if rlib { | |
463 | rlibs.insert(p, kind); | |
464 | } else { | |
465 | dylibs.insert(p, kind); | |
466 | } | |
467 | FileMatches | |
468 | }).unwrap_or(FileDoesntMatch) | |
1a4d82fc | 469 | }); |
62682a34 | 470 | self.rejected_via_kind.extend(staticlibs); |
1a4d82fc JJ |
471 | |
472 | // We have now collected all known libraries into a set of candidates | |
473 | // keyed of the filename hash listed. For each filename, we also have a | |
474 | // list of rlibs/dylibs that apply. Here, we map each of these lists | |
475 | // (per hash), to a Library candidate for returning. | |
476 | // | |
477 | // A Library candidate is created if the metadata for the set of | |
478 | // libraries corresponds to the crate id and hash criteria that this | |
479 | // search is being performed for. | |
a7813a04 | 480 | let mut libraries = HashMap::new(); |
85aaf69f | 481 | for (_hash, (rlibs, dylibs)) in candidates { |
a7813a04 XL |
482 | let mut slot = None; |
483 | let rlib = self.extract_one(rlibs, CrateFlavor::Rlib, &mut slot); | |
484 | let dylib = self.extract_one(dylibs, CrateFlavor::Dylib, &mut slot); | |
485 | if let Some((h, m)) = slot { | |
486 | libraries.insert(h, Library { | |
487 | dylib: dylib, | |
488 | rlib: rlib, | |
489 | metadata: m, | |
490 | }); | |
1a4d82fc JJ |
491 | } |
492 | } | |
493 | ||
494 | // Having now translated all relevant found hashes into libraries, see | |
495 | // what we've got and figure out if we found multiple candidates for | |
496 | // libraries or not. | |
497 | match libraries.len() { | |
498 | 0 => None, | |
a7813a04 | 499 | 1 => Some(libraries.into_iter().next().unwrap().1), |
970d7e83 | 500 | _ => { |
9cc50fc6 SL |
501 | let mut err = struct_span_err!(self.sess, self.span, E0464, |
502 | "multiple matching crates for `{}`", | |
503 | self.crate_name); | |
504 | err.note("candidates:"); | |
a7813a04 | 505 | for (_, lib) in libraries { |
3157f602 XL |
506 | if let Some((ref p, _)) = lib.dylib { |
507 | err.note(&format!("path: {}", p.display())); | |
1a4d82fc | 508 | } |
3157f602 XL |
509 | if let Some((ref p, _)) = lib.rlib { |
510 | err.note(&format!("path: {}", p.display())); | |
1a4d82fc JJ |
511 | } |
512 | let data = lib.metadata.as_slice(); | |
513 | let name = decoder::get_crate_name(data); | |
9cc50fc6 | 514 | note_crate_name(&mut err, &name); |
1a4d82fc | 515 | } |
9cc50fc6 | 516 | err.emit(); |
970d7e83 LB |
517 | None |
518 | } | |
1a4d82fc JJ |
519 | } |
520 | } | |
521 | ||
522 | // Attempts to extract *one* library from the set `m`. If the set has no | |
523 | // elements, `None` is returned. If the set has more than one element, then | |
524 | // the errors and notes are emitted about the set of libraries. | |
525 | // | |
526 | // With only one library in the set, this function will extract it, and then | |
527 | // read the metadata from it if `*slot` is `None`. If the metadata couldn't | |
528 | // be read, it is assumed that the file isn't a valid rust library (no | |
529 | // errors are emitted). | |
a7813a04 XL |
530 | fn extract_one(&mut self, m: HashMap<PathBuf, PathKind>, flavor: CrateFlavor, |
531 | slot: &mut Option<(Svh, MetadataBlob)>) -> Option<(PathBuf, PathKind)> { | |
532 | let mut ret: Option<(PathBuf, PathKind)> = None; | |
85aaf69f | 533 | let mut error = 0; |
1a4d82fc JJ |
534 | |
535 | if slot.is_some() { | |
536 | // FIXME(#10786): for an optimization, we only read one of the | |
a7813a04 | 537 | // libraries' metadata sections. In theory we should |
1a4d82fc JJ |
538 | // read both, but reading dylib metadata is quite |
539 | // slow. | |
9346a6ac | 540 | if m.is_empty() { |
1a4d82fc JJ |
541 | return None |
542 | } else if m.len() == 1 { | |
543 | return Some(m.into_iter().next().unwrap()) | |
544 | } | |
545 | } | |
546 | ||
9cc50fc6 | 547 | let mut err: Option<DiagnosticBuilder> = None; |
85aaf69f | 548 | for (lib, kind) in m { |
1a4d82fc | 549 | info!("{} reading metadata from: {}", flavor, lib.display()); |
a7813a04 | 550 | let (hash, metadata) = match get_metadata_section(self.target, flavor, &lib) { |
1a4d82fc | 551 | Ok(blob) => { |
a7813a04 XL |
552 | if let Some(h) = self.crate_matches(blob.as_slice(), &lib) { |
553 | (h, blob) | |
1a4d82fc JJ |
554 | } else { |
555 | info!("metadata mismatch"); | |
556 | continue | |
557 | } | |
970d7e83 | 558 | } |
92a42be0 SL |
559 | Err(err) => { |
560 | info!("no metadata found: {}", err); | |
1a4d82fc JJ |
561 | continue |
562 | } | |
563 | }; | |
a7813a04 XL |
564 | // If we see multiple hashes, emit an error about duplicate candidates. |
565 | if slot.as_ref().map_or(false, |s| s.0 != hash) { | |
9cc50fc6 SL |
566 | let mut e = struct_span_err!(self.sess, self.span, E0465, |
567 | "multiple {} candidates for `{}` found", | |
568 | flavor, self.crate_name); | |
569 | e.span_note(self.span, | |
570 | &format!(r"candidate #1: {}", | |
571 | ret.as_ref().unwrap().0 | |
572 | .display())); | |
573 | if let Some(ref mut e) = err { | |
574 | e.emit(); | |
575 | } | |
576 | err = Some(e); | |
1a4d82fc | 577 | error = 1; |
a7813a04 | 578 | *slot = None; |
1a4d82fc JJ |
579 | } |
580 | if error > 0 { | |
581 | error += 1; | |
9cc50fc6 SL |
582 | err.as_mut().unwrap().span_note(self.span, |
583 | &format!(r"candidate #{}: {}", error, | |
584 | lib.display())); | |
1a4d82fc | 585 | continue |
970d7e83 | 586 | } |
a7813a04 | 587 | *slot = Some((hash, metadata)); |
85aaf69f | 588 | ret = Some((lib, kind)); |
1a4d82fc | 589 | } |
9cc50fc6 SL |
590 | |
591 | if error > 0 { | |
592 | err.unwrap().emit(); | |
593 | None | |
594 | } else { | |
595 | ret | |
596 | } | |
1a4d82fc JJ |
597 | } |
598 | ||
a7813a04 XL |
599 | fn crate_matches(&mut self, crate_data: &[u8], libpath: &Path) -> Option<Svh> { |
600 | let crate_rustc_version = decoder::crate_rustc_version(crate_data); | |
601 | if crate_rustc_version != Some(rustc_version()) { | |
602 | let message = crate_rustc_version.unwrap_or(format!("an unknown compiler")); | |
603 | info!("Rejecting via version: expected {} got {}", rustc_version(), message); | |
604 | self.rejected_via_version.push(CrateMismatch { | |
605 | path: libpath.to_path_buf(), | |
606 | got: message | |
607 | }); | |
608 | return None; | |
609 | } | |
610 | ||
1a4d82fc JJ |
611 | if self.should_match_name { |
612 | match decoder::maybe_get_crate_name(crate_data) { | |
613 | Some(ref name) if self.crate_name == *name => {} | |
a7813a04 | 614 | _ => { info!("Rejecting via crate name"); return None } |
1a4d82fc JJ |
615 | } |
616 | } | |
617 | let hash = match decoder::maybe_get_crate_hash(crate_data) { | |
a7813a04 XL |
618 | None => { info!("Rejecting via lack of crate hash"); return None; } |
619 | Some(h) => h, | |
1a4d82fc JJ |
620 | }; |
621 | ||
622 | let triple = match decoder::get_crate_triple(crate_data) { | |
a7813a04 | 623 | None => { debug!("triple not present"); return None } |
1a4d82fc JJ |
624 | Some(t) => t, |
625 | }; | |
626 | if triple != self.triple { | |
627 | info!("Rejecting via crate triple: expected {} got {}", self.triple, triple); | |
628 | self.rejected_via_triple.push(CrateMismatch { | |
c34b1796 | 629 | path: libpath.to_path_buf(), |
1a4d82fc JJ |
630 | got: triple.to_string() |
631 | }); | |
a7813a04 | 632 | return None; |
223e47cc | 633 | } |
223e47cc | 634 | |
a7813a04 XL |
635 | if let Some(myhash) = self.hash { |
636 | if *myhash != hash { | |
637 | info!("Rejecting via hash: expected {} got {}", *myhash, hash); | |
638 | self.rejected_via_hash.push(CrateMismatch { | |
639 | path: libpath.to_path_buf(), | |
640 | got: myhash.to_string() | |
641 | }); | |
642 | return None; | |
1a4d82fc | 643 | } |
223e47cc | 644 | } |
a7813a04 XL |
645 | |
646 | Some(hash) | |
223e47cc | 647 | } |
223e47cc | 648 | |
1a4d82fc JJ |
649 | |
650 | // Returns the corresponding (prefix, suffix) that files need to have for | |
651 | // dynamic libraries | |
652 | fn dylibname(&self) -> (String, String) { | |
85aaf69f | 653 | let t = &self.target; |
1a4d82fc | 654 | (t.options.dll_prefix.clone(), t.options.dll_suffix.clone()) |
223e47cc | 655 | } |
223e47cc | 656 | |
7453a54e SL |
657 | // Returns the corresponding (prefix, suffix) that files need to have for |
658 | // static libraries | |
659 | fn staticlibname(&self) -> (String, String) { | |
660 | let t = &self.target; | |
661 | (t.options.staticlib_prefix.clone(), t.options.staticlib_suffix.clone()) | |
662 | } | |
663 | ||
5bcae85e SL |
664 | fn find_commandline_library<'b, LOCS> (&mut self, locs: LOCS) -> Option<Library> |
665 | where LOCS: Iterator<Item=&'b String> | |
666 | { | |
1a4d82fc JJ |
667 | // First, filter out all libraries that look suspicious. We only accept |
668 | // files which actually exist that have the correct naming scheme for | |
669 | // rlibs/dylibs. | |
670 | let sess = self.sess; | |
671 | let dylibname = self.dylibname(); | |
85aaf69f SL |
672 | let mut rlibs = HashMap::new(); |
673 | let mut dylibs = HashMap::new(); | |
1a4d82fc | 674 | { |
5bcae85e | 675 | let locs = locs.map(|l| PathBuf::from(l)).filter(|loc| { |
1a4d82fc JJ |
676 | if !loc.exists() { |
677 | sess.err(&format!("extern location for {} does not exist: {}", | |
c34b1796 | 678 | self.crate_name, loc.display())); |
1a4d82fc JJ |
679 | return false; |
680 | } | |
c34b1796 | 681 | let file = match loc.file_name().and_then(|s| s.to_str()) { |
1a4d82fc JJ |
682 | Some(file) => file, |
683 | None => { | |
684 | sess.err(&format!("extern location for {} is not a file: {}", | |
c34b1796 | 685 | self.crate_name, loc.display())); |
1a4d82fc JJ |
686 | return false; |
687 | } | |
688 | }; | |
689 | if file.starts_with("lib") && file.ends_with(".rlib") { | |
690 | return true | |
691 | } else { | |
692 | let (ref prefix, ref suffix) = dylibname; | |
85aaf69f SL |
693 | if file.starts_with(&prefix[..]) && |
694 | file.ends_with(&suffix[..]) { | |
1a4d82fc JJ |
695 | return true |
696 | } | |
697 | } | |
9cc50fc6 SL |
698 | sess.struct_err(&format!("extern location for {} is of an unknown type: {}", |
699 | self.crate_name, loc.display())) | |
700 | .help(&format!("file name should be lib*.rlib or {}*.{}", | |
701 | dylibname.0, dylibname.1)) | |
702 | .emit(); | |
1a4d82fc JJ |
703 | false |
704 | }); | |
705 | ||
85aaf69f SL |
706 | // Now that we have an iterator of good candidates, make sure |
707 | // there's at most one rlib and at most one dylib. | |
1a4d82fc | 708 | for loc in locs { |
c34b1796 | 709 | if loc.file_name().unwrap().to_str().unwrap().ends_with(".rlib") { |
d9579d0f | 710 | rlibs.insert(fs::canonicalize(&loc).unwrap(), |
85aaf69f | 711 | PathKind::ExternFlag); |
1a4d82fc | 712 | } else { |
d9579d0f | 713 | dylibs.insert(fs::canonicalize(&loc).unwrap(), |
85aaf69f | 714 | PathKind::ExternFlag); |
1a4d82fc JJ |
715 | } |
716 | } | |
717 | }; | |
718 | ||
719 | // Extract the rlib/dylib pair. | |
a7813a04 XL |
720 | let mut slot = None; |
721 | let rlib = self.extract_one(rlibs, CrateFlavor::Rlib, &mut slot); | |
722 | let dylib = self.extract_one(dylibs, CrateFlavor::Dylib, &mut slot); | |
1a4d82fc JJ |
723 | |
724 | if rlib.is_none() && dylib.is_none() { return None } | |
a7813a04 XL |
725 | match slot { |
726 | Some((_, metadata)) => Some(Library { | |
1a4d82fc JJ |
727 | dylib: dylib, |
728 | rlib: rlib, | |
729 | metadata: metadata, | |
730 | }), | |
731 | None => None, | |
732 | } | |
223e47cc | 733 | } |
223e47cc LB |
734 | } |
735 | ||
9cc50fc6 SL |
736 | pub fn note_crate_name(err: &mut DiagnosticBuilder, name: &str) { |
737 | err.note(&format!("crate name: {}", name)); | |
1a4d82fc | 738 | } |
223e47cc | 739 | |
1a4d82fc JJ |
740 | impl ArchiveMetadata { |
741 | fn new(ar: ArchiveRO) -> Option<ArchiveMetadata> { | |
d9579d0f | 742 | let data = { |
7453a54e | 743 | let section = ar.iter().filter_map(|s| s.ok()).find(|sect| { |
d9579d0f AL |
744 | sect.name() == Some(METADATA_FILENAME) |
745 | }); | |
746 | match section { | |
747 | Some(s) => s.data() as *const [u8], | |
748 | None => { | |
749 | debug!("didn't find '{}' in the archive", METADATA_FILENAME); | |
750 | return None; | |
751 | } | |
1a4d82fc JJ |
752 | } |
753 | }; | |
223e47cc | 754 | |
1a4d82fc JJ |
755 | Some(ArchiveMetadata { |
756 | _archive: ar, | |
757 | data: data, | |
758 | }) | |
223e47cc | 759 | } |
1a4d82fc JJ |
760 | |
761 | pub fn as_slice<'a>(&'a self) -> &'a [u8] { unsafe { &*self.data } } | |
762 | } | |
763 | ||
a7813a04 XL |
764 | fn verify_decompressed_encoding_version(blob: &MetadataBlob, filename: &Path) |
765 | -> Result<(), String> | |
766 | { | |
767 | let data = blob.as_slice_raw(); | |
768 | if data.len() < 4+metadata_encoding_version.len() || | |
769 | !<[u8]>::eq(&data[..4], &[0, 0, 0, 0]) || | |
770 | &data[4..4+metadata_encoding_version.len()] != metadata_encoding_version | |
771 | { | |
772 | Err((format!("incompatible metadata version found: '{}'", | |
773 | filename.display()))) | |
774 | } else { | |
775 | Ok(()) | |
776 | } | |
777 | } | |
778 | ||
1a4d82fc | 779 | // Just a small wrapper to time how long reading metadata takes. |
a7813a04 | 780 | fn get_metadata_section(target: &Target, flavor: CrateFlavor, filename: &Path) |
62682a34 | 781 | -> Result<MetadataBlob, String> { |
92a42be0 | 782 | let start = Instant::now(); |
a7813a04 | 783 | let ret = get_metadata_section_imp(target, flavor, filename); |
92a42be0 SL |
784 | info!("reading {:?} => {:?}", filename.file_name().unwrap(), |
785 | start.elapsed()); | |
786 | return ret | |
223e47cc LB |
787 | } |
788 | ||
a7813a04 | 789 | fn get_metadata_section_imp(target: &Target, flavor: CrateFlavor, filename: &Path) |
62682a34 | 790 | -> Result<MetadataBlob, String> { |
1a4d82fc JJ |
791 | if !filename.exists() { |
792 | return Err(format!("no such file: '{}'", filename.display())); | |
793 | } | |
a7813a04 | 794 | if flavor == CrateFlavor::Rlib { |
1a4d82fc JJ |
795 | // Use ArchiveRO for speed here, it's backed by LLVM and uses mmap |
796 | // internally to read the file. We also avoid even using a memcpy by | |
797 | // just keeping the archive along while the metadata is in use. | |
798 | let archive = match ArchiveRO::open(filename) { | |
799 | Some(ar) => ar, | |
800 | None => { | |
801 | debug!("llvm didn't like `{}`", filename.display()); | |
802 | return Err(format!("failed to read rlib metadata: '{}'", | |
803 | filename.display())); | |
804 | } | |
805 | }; | |
806 | return match ArchiveMetadata::new(archive).map(|ar| MetadataArchive(ar)) { | |
c34b1796 AL |
807 | None => Err(format!("failed to read rlib metadata: '{}'", |
808 | filename.display())), | |
a7813a04 XL |
809 | Some(blob) => { |
810 | try!(verify_decompressed_encoding_version(&blob, filename)); | |
811 | Ok(blob) | |
812 | } | |
c34b1796 | 813 | }; |
1a4d82fc | 814 | } |
223e47cc | 815 | unsafe { |
c34b1796 | 816 | let buf = common::path2cstr(filename); |
1a4d82fc | 817 | let mb = llvm::LLVMRustCreateMemoryBufferWithContentsOfFile(buf.as_ptr()); |
c34b1796 | 818 | if mb as isize == 0 { |
1a4d82fc JJ |
819 | return Err(format!("error reading library: '{}'", |
820 | filename.display())) | |
821 | } | |
822 | let of = match ObjectFile::new(mb) { | |
823 | Some(of) => of, | |
824 | _ => { | |
825 | return Err((format!("provided path not an object file: '{}'", | |
826 | filename.display()))) | |
827 | } | |
223e47cc LB |
828 | }; |
829 | let si = mk_section_iter(of.llof); | |
830 | while llvm::LLVMIsSectionIteratorAtEnd(of.llof, si.llsi) == False { | |
1a4d82fc JJ |
831 | let mut name_buf = ptr::null(); |
832 | let name_len = llvm::LLVMRustGetSectionName(si.llsi, &mut name_buf); | |
85aaf69f | 833 | let name = slice::from_raw_parts(name_buf as *const u8, |
c34b1796 | 834 | name_len as usize).to_vec(); |
1a4d82fc JJ |
835 | let name = String::from_utf8(name).unwrap(); |
836 | debug!("get_metadata_section: name {}", name); | |
62682a34 | 837 | if read_meta_section_name(target) == name { |
223e47cc | 838 | let cbuf = llvm::LLVMGetSectionContents(si.llsi); |
c34b1796 | 839 | let csz = llvm::LLVMGetSectionSize(si.llsi) as usize; |
1a4d82fc | 840 | let cvbuf: *const u8 = cbuf as *const u8; |
a7813a04 | 841 | let vlen = metadata_encoding_version.len(); |
1a4d82fc | 842 | debug!("checking {} bytes of metadata-version stamp", |
970d7e83 | 843 | vlen); |
1a4d82fc | 844 | let minsz = cmp::min(vlen, csz); |
85aaf69f | 845 | let buf0 = slice::from_raw_parts(cvbuf, minsz); |
a7813a04 | 846 | let version_ok = buf0 == metadata_encoding_version; |
1a4d82fc JJ |
847 | if !version_ok { |
848 | return Err((format!("incompatible metadata version found: '{}'", | |
849 | filename.display()))); | |
970d7e83 | 850 | } |
223e47cc | 851 | |
c34b1796 | 852 | let cvbuf1 = cvbuf.offset(vlen as isize); |
1a4d82fc | 853 | debug!("inflating {} bytes of compressed metadata", |
970d7e83 | 854 | csz - vlen); |
85aaf69f | 855 | let bytes = slice::from_raw_parts(cvbuf1, csz - vlen); |
1a4d82fc | 856 | match flate::inflate_bytes(bytes) { |
a7813a04 XL |
857 | Ok(inflated) => { |
858 | let blob = MetadataVec(inflated); | |
859 | try!(verify_decompressed_encoding_version(&blob, filename)); | |
860 | return Ok(blob); | |
861 | } | |
c34b1796 | 862 | Err(_) => {} |
223e47cc LB |
863 | } |
864 | } | |
865 | llvm::LLVMMoveToNextSection(si.llsi); | |
866 | } | |
c34b1796 | 867 | Err(format!("metadata not found: '{}'", filename.display())) |
223e47cc LB |
868 | } |
869 | } | |
870 | ||
62682a34 | 871 | pub fn meta_section_name(target: &Target) -> &'static str { |
5bcae85e SL |
872 | // Historical note: |
873 | // | |
874 | // When using link.exe it was seen that the section name `.note.rustc` | |
875 | // was getting shortened to `.note.ru`, and according to the PE and COFF | |
876 | // specification: | |
877 | // | |
878 | // > Executable images do not use a string table and do not support | |
879 | // > section names longer than 8 characters | |
880 | // | |
881 | // https://msdn.microsoft.com/en-us/library/windows/hardware/gg463119.aspx | |
882 | // | |
883 | // As a result, we choose a slightly shorter name! As to why | |
884 | // `.note.rustc` works on MinGW, that's another good question... | |
885 | ||
62682a34 | 886 | if target.options.is_like_osx { |
5bcae85e | 887 | "__DATA,.rustc" |
1a4d82fc | 888 | } else { |
5bcae85e | 889 | ".rustc" |
223e47cc LB |
890 | } |
891 | } | |
892 | ||
5bcae85e SL |
893 | pub fn read_meta_section_name(_target: &Target) -> &'static str { |
894 | ".rustc" | |
970d7e83 LB |
895 | } |
896 | ||
223e47cc | 897 | // A diagnostic function for dumping crate metadata to an output stream |
62682a34 | 898 | pub fn list_file_metadata(target: &Target, path: &Path, |
c34b1796 | 899 | out: &mut io::Write) -> io::Result<()> { |
a7813a04 XL |
900 | let filename = path.file_name().unwrap().to_str().unwrap(); |
901 | let flavor = if filename.ends_with(".rlib") { CrateFlavor::Rlib } else { CrateFlavor::Dylib }; | |
902 | match get_metadata_section(target, flavor, path) { | |
1a4d82fc JJ |
903 | Ok(bytes) => decoder::list_crate_metadata(bytes.as_slice(), out), |
904 | Err(msg) => { | |
905 | write!(out, "{}\n", msg) | |
906 | } | |
223e47cc LB |
907 | } |
908 | } |