]>
Commit | Line | Data |
---|---|---|
ff7c6d11 | 1 | //! Mono Item Collection |
60c5eb7d | 2 | //! ==================== |
7453a54e | 3 | //! |
f035d41b | 4 | //! This module is responsible for discovering all items that will contribute |
7453a54e SL |
5 | //! to code generation of the crate. The important part here is that it not only |
6 | //! needs to find syntax-level items (functions, structs, etc) but also all | |
7 | //! their monomorphized instantiations. Every non-generic, non-const function | |
8 | //! maps to one LLVM artifact. Every generic function can produce | |
9 | //! from zero to N artifacts, depending on the sets of type arguments it | |
10 | //! is instantiated with. | |
11 | //! This also applies to generic items from other crates: A generic definition | |
12 | //! in crate X might produce monomorphizations that are compiled into crate Y. | |
13 | //! We also have to collect these here. | |
14 | //! | |
ff7c6d11 | 15 | //! The following kinds of "mono items" are handled here: |
7453a54e SL |
16 | //! |
17 | //! - Functions | |
18 | //! - Methods | |
19 | //! - Closures | |
20 | //! - Statics | |
21 | //! - Drop glue | |
22 | //! | |
23 | //! The following things also result in LLVM artifacts, but are not collected | |
24 | //! here, since we instantiate them locally on demand when needed in a given | |
25 | //! codegen unit: | |
26 | //! | |
27 | //! - Constants | |
064997fb | 28 | //! - VTables |
7453a54e SL |
29 | //! - Object Shims |
30 | //! | |
31 | //! | |
32 | //! General Algorithm | |
33 | //! ----------------- | |
34 | //! Let's define some terms first: | |
35 | //! | |
ff7c6d11 XL |
36 | //! - A "mono item" is something that results in a function or global in |
37 | //! the LLVM IR of a codegen unit. Mono items do not stand on their | |
38 | //! own, they can reference other mono items. For example, if function | |
39 | //! `foo()` calls function `bar()` then the mono item for `foo()` | |
40 | //! references the mono item for function `bar()`. In general, the | |
41 | //! definition for mono item A referencing a mono item B is that | |
7453a54e SL |
42 | //! the LLVM artifact produced for A references the LLVM artifact produced |
43 | //! for B. | |
44 | //! | |
ff7c6d11 XL |
45 | //! - Mono items and the references between them form a directed graph, |
46 | //! where the mono items are the nodes and references form the edges. | |
47 | //! Let's call this graph the "mono item graph". | |
7453a54e | 48 | //! |
ff7c6d11 | 49 | //! - The mono item graph for a program contains all mono items |
7453a54e SL |
50 | //! that are needed in order to produce the complete LLVM IR of the program. |
51 | //! | |
52 | //! The purpose of the algorithm implemented in this module is to build the | |
ff7c6d11 | 53 | //! mono item graph for the current crate. It runs in two phases: |
7453a54e SL |
54 | //! |
55 | //! 1. Discover the roots of the graph by traversing the HIR of the crate. | |
56 | //! 2. Starting from the roots, find neighboring nodes by inspecting the MIR | |
57 | //! representation of the item corresponding to a given node, until no more | |
58 | //! new nodes are found. | |
59 | //! | |
60 | //! ### Discovering roots | |
61 | //! | |
cdc7bbd5 | 62 | //! The roots of the mono item graph correspond to the public non-generic |
7453a54e | 63 | //! syntactic items in the source code. We find them by walking the HIR of the |
cdc7bbd5 XL |
64 | //! crate, and whenever we hit upon a public function, method, or static item, |
65 | //! we create a mono item consisting of the items DefId and, since we only | |
66 | //! consider non-generic items, an empty type-substitution set. (In eager | |
67 | //! collection mode, during incremental compilation, all non-generic functions | |
68 | //! are considered as roots, as well as when the `-Clink-dead-code` option is | |
69 | //! specified. Functions marked `#[no_mangle]` and functions called by inlinable | |
70 | //! functions also always act as roots.) | |
7453a54e SL |
71 | //! |
72 | //! ### Finding neighbor nodes | |
ff7c6d11 | 73 | //! Given a mono item node, we can discover neighbors by inspecting its |
7453a54e | 74 | //! MIR. We walk the MIR and any time we hit upon something that signifies a |
ff7c6d11 XL |
75 | //! reference to another mono item, we have found a neighbor. Since the |
76 | //! mono item we are currently at is always monomorphic, we also know the | |
7453a54e SL |
77 | //! concrete type arguments of its neighbors, and so all neighbors again will be |
78 | //! monomorphic. The specific forms a reference to a neighboring node can take | |
79 | //! in MIR are quite diverse. Here is an overview: | |
80 | //! | |
81 | //! #### Calling Functions/Methods | |
ff7c6d11 | 82 | //! The most obvious form of one mono item referencing another is a |
7453a54e SL |
83 | //! function or method call (represented by a CALL terminator in MIR). But |
84 | //! calls are not the only thing that might introduce a reference between two | |
ff7c6d11 | 85 | //! function mono items, and as we will see below, they are just a |
f035d41b | 86 | //! specialization of the form described next, and consequently will not get any |
7453a54e SL |
87 | //! special treatment in the algorithm. |
88 | //! | |
89 | //! #### Taking a reference to a function or method | |
90 | //! A function does not need to actually be called in order to be a neighbor of | |
91 | //! another function. It suffices to just take a reference in order to introduce | |
92 | //! an edge. Consider the following example: | |
93 | //! | |
04454e1e FG |
94 | //! ``` |
95 | //! # use core::fmt::Display; | |
7453a54e SL |
96 | //! fn print_val<T: Display>(x: T) { |
97 | //! println!("{}", x); | |
98 | //! } | |
99 | //! | |
04454e1e | 100 | //! fn call_fn(f: &dyn Fn(i32), x: i32) { |
7453a54e SL |
101 | //! f(x); |
102 | //! } | |
103 | //! | |
104 | //! fn main() { | |
105 | //! let print_i32 = print_val::<i32>; | |
106 | //! call_fn(&print_i32, 0); | |
107 | //! } | |
108 | //! ``` | |
109 | //! The MIR of none of these functions will contain an explicit call to | |
ff7c6d11 | 110 | //! `print_val::<i32>`. Nonetheless, in order to mono this program, we need |
7453a54e SL |
111 | //! an instance of this function. Thus, whenever we encounter a function or |
112 | //! method in operand position, we treat it as a neighbor of the current | |
ff7c6d11 | 113 | //! mono item. Calls are just a special case of that. |
7453a54e | 114 | //! |
7453a54e | 115 | //! #### Drop glue |
ff7c6d11 XL |
116 | //! Drop glue mono items are introduced by MIR drop-statements. The |
117 | //! generated mono item will again have drop-glue item neighbors if the | |
7453a54e SL |
118 | //! type to be dropped contains nested values that also need to be dropped. It |
119 | //! might also have a function item neighbor for the explicit `Drop::drop` | |
120 | //! implementation of its type. | |
121 | //! | |
122 | //! #### Unsizing Casts | |
123 | //! A subtle way of introducing neighbor edges is by casting to a trait object. | |
124 | //! Since the resulting fat-pointer contains a reference to a vtable, we need to | |
f2b60f7d | 125 | //! instantiate all object-safe methods of the trait, as we need to store |
7453a54e SL |
126 | //! pointers to these functions even if they never get called anywhere. This can |
127 | //! be seen as a special case of taking a function reference. | |
128 | //! | |
129 | //! #### Boxes | |
130 | //! Since `Box` expression have special compiler support, no explicit calls to | |
0731742a | 131 | //! `exchange_malloc()` and `box_free()` may show up in MIR, even if the |
7453a54e SL |
132 | //! compiler will generate them. We have to observe `Rvalue::Box` expressions |
133 | //! and Box-typed drop-statements for that purpose. | |
134 | //! | |
135 | //! | |
136 | //! Interaction with Cross-Crate Inlining | |
137 | //! ------------------------------------- | |
138 | //! The binary of a crate will not only contain machine code for the items | |
139 | //! defined in the source code of that crate. It will also contain monomorphic | |
140 | //! instantiations of any extern generic functions and of functions marked with | |
ff7c6d11 XL |
141 | //! `#[inline]`. |
142 | //! The collection algorithm handles this more or less mono. If it is | |
143 | //! about to create a mono item for something with an external `DefId`, | |
7453a54e | 144 | //! it will take a look if the MIR for that item is available, and if so just |
9e0c209e | 145 | //! proceed normally. If the MIR is not available, it assumes that the item is |
7453a54e SL |
146 | //! just linked to and no node is created; which is exactly what we want, since |
147 | //! no machine code should be generated in the current crate for such an item. | |
148 | //! | |
149 | //! Eager and Lazy Collection Mode | |
150 | //! ------------------------------ | |
ff7c6d11 | 151 | //! Mono item collection can be performed in one of two modes: |
7453a54e SL |
152 | //! |
153 | //! - Lazy mode means that items will only be instantiated when actually | |
154 | //! referenced. The goal is to produce the least amount of machine code | |
155 | //! possible. | |
156 | //! | |
157 | //! - Eager mode is meant to be used in conjunction with incremental compilation | |
ff7c6d11 | 158 | //! where a stable set of mono items is more important than a minimal |
7453a54e | 159 | //! one. Thus, eager mode will instantiate drop-glue for every drop-able type |
f035d41b | 160 | //! in the crate, even if no drop call for that type exists (yet). It will |
7453a54e SL |
161 | //! also instantiate default implementations of trait methods, something that |
162 | //! otherwise is only done on demand. | |
163 | //! | |
164 | //! | |
165 | //! Open Issues | |
166 | //! ----------- | |
167 | //! Some things are not yet fully implemented in the current version of this | |
168 | //! module. | |
169 | //! | |
7453a54e | 170 | //! ### Const Fns |
ff7c6d11 | 171 | //! Ideally, no mono item should be generated for const fns unless there |
7453a54e | 172 | //! is a call to them that cannot be evaluated at compile time. At the moment |
ff7c6d11 | 173 | //! this is not implemented however: a mono item will be produced |
7453a54e SL |
174 | //! regardless of whether it is actually needed or not. |
175 | ||
dfeec247 | 176 | use rustc_data_structures::fx::{FxHashMap, FxHashSet}; |
353b0b11 | 177 | use rustc_data_structures::sync::{par_for_each_in, MTLock, MTLockRef}; |
dfeec247 | 178 | use rustc_hir as hir; |
04454e1e FG |
179 | use rustc_hir::def::DefKind; |
180 | use rustc_hir::def_id::{DefId, DefIdMap, LocalDefId}; | |
3dfed10e | 181 | use rustc_hir::lang_items::LangItem; |
e74abb32 | 182 | use rustc_index::bit_set::GrowableBitSet; |
ba9703b0 XL |
183 | use rustc_middle::mir::interpret::{AllocId, ConstValue}; |
184 | use rustc_middle::mir::interpret::{ErrorHandled, GlobalAlloc, Scalar}; | |
185 | use rustc_middle::mir::mono::{InstantiationMode, MonoItem}; | |
186 | use rustc_middle::mir::visit::Visitor as MirVisitor; | |
187 | use rustc_middle::mir::{self, Local, Location}; | |
188 | use rustc_middle::ty::adjustment::{CustomCoerceUnsized, PointerCast}; | |
17df50a5 | 189 | use rustc_middle::ty::print::with_no_trimmed_paths; |
487cf647 | 190 | use rustc_middle::ty::query::TyCtxtAt; |
ba9703b0 | 191 | use rustc_middle::ty::subst::{GenericArgKind, InternalSubsts}; |
064997fb | 192 | use rustc_middle::ty::{ |
353b0b11 FG |
193 | self, GenericParamDefKind, Instance, InstanceDef, Ty, TyCtxt, TypeFoldable, TypeVisitableExt, |
194 | VtblEntry, | |
064997fb | 195 | }; |
cdc7bbd5 | 196 | use rustc_middle::{middle::codegen_fn_attrs::CodegenFnAttrFlags, mir::visit::TyContext}; |
ba9703b0 | 197 | use rustc_session::config::EntryFnType; |
cdc7bbd5 | 198 | use rustc_session::lint::builtin::LARGE_ASSIGNMENTS; |
136023e0 | 199 | use rustc_session::Limit; |
f035d41b | 200 | use rustc_span::source_map::{dummy_spanned, respan, Span, Spanned, DUMMY_SP}; |
cdc7bbd5 | 201 | use rustc_target::abi::Size; |
29967ef6 | 202 | use std::ops::Range; |
1b1a35ee | 203 | use std::path::PathBuf; |
532ac7d7 | 204 | |
9ffffee4 FG |
205 | use crate::errors::{ |
206 | EncounteredErrorWhileInstantiating, LargeAssignmentsLint, RecursionLimit, TypeLengthLimit, | |
207 | }; | |
f2b60f7d | 208 | |
e74abb32 | 209 | #[derive(PartialEq)] |
ff7c6d11 | 210 | pub enum MonoItemCollectionMode { |
7453a54e | 211 | Eager, |
dfeec247 | 212 | Lazy, |
7453a54e SL |
213 | } |
214 | ||
ff7c6d11 | 215 | /// Maps every mono item to all mono items it references in its |
a7813a04 XL |
216 | /// body. |
217 | pub struct InliningMap<'tcx> { | |
ff7c6d11 | 218 | // Maps a source mono item to the range of mono items |
3b2f2976 | 219 | // accessed by it. |
29967ef6 XL |
220 | // The range selects elements within the `targets` vecs. |
221 | index: FxHashMap<MonoItem<'tcx>, Range<usize>>, | |
ff7c6d11 | 222 | targets: Vec<MonoItem<'tcx>>, |
3b2f2976 | 223 | |
ff7c6d11 XL |
224 | // Contains one bit per mono item in the `targets` field. That bit |
225 | // is true if that mono item needs to be inlined into every CGU. | |
0bf4aa26 | 226 | inlines: GrowableBitSet<usize>, |
7453a54e SL |
227 | } |
228 | ||
923072b8 FG |
229 | /// Struct to store mono items in each collecting and if they should |
230 | /// be inlined. We call `instantiation_mode` to get their inlining | |
231 | /// status when inserting new elements, which avoids calling it in | |
232 | /// `inlining_map.lock_mut()`. See the `collect_items_rec` implementation | |
233 | /// below. | |
234 | struct MonoItems<'tcx> { | |
235 | // If this is false, we do not need to compute whether items | |
236 | // will need to be inlined. | |
237 | compute_inlining: bool, | |
238 | ||
239 | // The TyCtxt used to determine whether the a item should | |
240 | // be inlined. | |
241 | tcx: TyCtxt<'tcx>, | |
242 | ||
243 | // The collected mono items. The bool field in each element | |
244 | // indicates whether this element should be inlined. | |
245 | items: Vec<(Spanned<MonoItem<'tcx>>, bool /*inlined*/)>, | |
246 | } | |
247 | ||
248 | impl<'tcx> MonoItems<'tcx> { | |
249 | #[inline] | |
250 | fn push(&mut self, item: Spanned<MonoItem<'tcx>>) { | |
251 | self.extend([item]); | |
252 | } | |
253 | ||
254 | #[inline] | |
255 | fn extend<T: IntoIterator<Item = Spanned<MonoItem<'tcx>>>>(&mut self, iter: T) { | |
256 | self.items.extend(iter.into_iter().map(|mono_item| { | |
257 | let inlined = if !self.compute_inlining { | |
258 | false | |
259 | } else { | |
260 | mono_item.node.instantiation_mode(self.tcx) == InstantiationMode::LocalCopy | |
261 | }; | |
262 | (mono_item, inlined) | |
263 | })) | |
264 | } | |
265 | } | |
266 | ||
a7813a04 | 267 | impl<'tcx> InliningMap<'tcx> { |
a7813a04 XL |
268 | fn new() -> InliningMap<'tcx> { |
269 | InliningMap { | |
0bf4aa26 | 270 | index: FxHashMap::default(), |
a7813a04 | 271 | targets: Vec::new(), |
0bf4aa26 | 272 | inlines: GrowableBitSet::with_capacity(1024), |
a7813a04 XL |
273 | } |
274 | } | |
275 | ||
923072b8 FG |
276 | fn record_accesses<'a>( |
277 | &mut self, | |
278 | source: MonoItem<'tcx>, | |
279 | new_targets: &'a [(Spanned<MonoItem<'tcx>>, bool)], | |
280 | ) where | |
281 | 'tcx: 'a, | |
282 | { | |
a7813a04 | 283 | let start_index = self.targets.len(); |
3b2f2976 XL |
284 | let new_items_count = new_targets.len(); |
285 | let new_items_count_total = new_items_count + self.targets.len(); | |
286 | ||
287 | self.targets.reserve(new_items_count); | |
0bf4aa26 | 288 | self.inlines.ensure(new_items_count_total); |
3b2f2976 | 289 | |
923072b8 FG |
290 | for (i, (Spanned { node: mono_item, .. }, inlined)) in new_targets.into_iter().enumerate() { |
291 | self.targets.push(*mono_item); | |
292 | if *inlined { | |
3b2f2976 XL |
293 | self.inlines.insert(i + start_index); |
294 | } | |
295 | } | |
296 | ||
a7813a04 | 297 | let end_index = self.targets.len(); |
29967ef6 | 298 | assert!(self.index.insert(source, start_index..end_index).is_none()); |
a7813a04 XL |
299 | } |
300 | ||
487cf647 FG |
301 | /// Internally iterate over all items referenced by `source` which will be |
302 | /// made available for inlining. | |
ff7c6d11 | 303 | pub fn with_inlining_candidates<F>(&self, source: MonoItem<'tcx>, mut f: F) |
dfeec247 XL |
304 | where |
305 | F: FnMut(MonoItem<'tcx>), | |
3b2f2976 | 306 | { |
29967ef6 XL |
307 | if let Some(range) = self.index.get(&source) { |
308 | for (i, candidate) in self.targets[range.clone()].iter().enumerate() { | |
309 | if self.inlines.contains(range.start + i) { | |
3b2f2976 XL |
310 | f(*candidate); |
311 | } | |
7453a54e | 312 | } |
a7813a04 | 313 | } |
7453a54e | 314 | } |
3b2f2976 | 315 | |
487cf647 | 316 | /// Internally iterate over all items and the things each accesses. |
3b2f2976 | 317 | pub fn iter_accesses<F>(&self, mut f: F) |
dfeec247 XL |
318 | where |
319 | F: FnMut(MonoItem<'tcx>, &[MonoItem<'tcx>]), | |
3b2f2976 | 320 | { |
29967ef6 XL |
321 | for (&accessor, range) in &self.index { |
322 | f(accessor, &self.targets[range.clone()]) | |
3b2f2976 XL |
323 | } |
324 | } | |
7453a54e SL |
325 | } |
326 | ||
923072b8 | 327 | #[instrument(skip(tcx, mode), level = "debug")] |
416331ca XL |
328 | pub fn collect_crate_mono_items( |
329 | tcx: TyCtxt<'_>, | |
dc9dc135 | 330 | mode: MonoItemCollectionMode, |
416331ca | 331 | ) -> (FxHashSet<MonoItem<'_>>, InliningMap<'_>) { |
e74abb32 XL |
332 | let _prof_timer = tcx.prof.generic_activity("monomorphization_collector"); |
333 | ||
dfeec247 XL |
334 | let roots = |
335 | tcx.sess.time("monomorphization_collector_root_collections", || collect_roots(tcx, mode)); | |
ea8adc8c | 336 | |
416331ca | 337 | debug!("building mono item graph, beginning at roots"); |
94b46f34 | 338 | |
0bf4aa26 | 339 | let mut visited = MTLock::new(FxHashSet::default()); |
94b46f34 | 340 | let mut inlining_map = MTLock::new(InliningMap::new()); |
136023e0 | 341 | let recursion_limit = tcx.recursion_limit(); |
94b46f34 XL |
342 | |
343 | { | |
353b0b11 FG |
344 | let visited: MTLockRef<'_, _> = &mut visited; |
345 | let inlining_map: MTLockRef<'_, _> = &mut inlining_map; | |
94b46f34 | 346 | |
dfeec247 | 347 | tcx.sess.time("monomorphization_collector_graph_walk", || { |
064997fb | 348 | par_for_each_in(roots, |root| { |
a1dfa0c6 | 349 | let mut recursion_depths = DefIdMap::default(); |
f035d41b XL |
350 | collect_items_rec( |
351 | tcx, | |
352 | dummy_spanned(root), | |
353 | visited, | |
354 | &mut recursion_depths, | |
136023e0 | 355 | recursion_limit, |
f035d41b XL |
356 | inlining_map, |
357 | ); | |
94b46f34 XL |
358 | }); |
359 | }); | |
ea8adc8c | 360 | } |
7453a54e | 361 | |
94b46f34 | 362 | (visited.into_inner(), inlining_map.into_inner()) |
7453a54e SL |
363 | } |
364 | ||
365 | // Find all non-generic items by walking the HIR. These items serve as roots to | |
366 | // start monomorphizing from. | |
923072b8 | 367 | #[instrument(skip(tcx, mode), level = "debug")] |
416331ca XL |
368 | fn collect_roots(tcx: TyCtxt<'_>, mode: MonoItemCollectionMode) -> Vec<MonoItem<'_>> { |
369 | debug!("collecting roots"); | |
923072b8 | 370 | let mut roots = MonoItems { compute_inlining: false, tcx, items: Vec::new() }; |
7453a54e SL |
371 | |
372 | { | |
17df50a5 | 373 | let entry_fn = tcx.entry_fn(()); |
abe05a73 | 374 | |
ff7c6d11 XL |
375 | debug!("collect_roots: entry_fn = {:?}", entry_fn); |
376 | ||
04454e1e FG |
377 | let mut collector = RootCollector { tcx, mode, entry_fn, output: &mut roots }; |
378 | ||
379 | let crate_items = tcx.hir_crate_items(()); | |
380 | ||
381 | for id in crate_items.items() { | |
382 | collector.process_item(id); | |
383 | } | |
7453a54e | 384 | |
04454e1e FG |
385 | for id in crate_items.impl_items() { |
386 | collector.process_impl_item(id); | |
387 | } | |
0531ce1d | 388 | |
04454e1e | 389 | collector.push_extra_entry_roots(); |
7453a54e SL |
390 | } |
391 | ||
94b46f34 | 392 | // We can only codegen items that are instantiable - items all of |
041b39d2 | 393 | // whose predicates hold. Luckily, items that aren't instantiable |
94b46f34 | 394 | // can't actually be used, so we can just skip codegenning them. |
7453a54e | 395 | roots |
923072b8 | 396 | .items |
f035d41b | 397 | .into_iter() |
923072b8 FG |
398 | .filter_map(|(Spanned { node: mono_item, .. }, _)| { |
399 | mono_item.is_instantiable(tcx).then_some(mono_item) | |
400 | }) | |
f035d41b | 401 | .collect() |
7453a54e SL |
402 | } |
403 | ||
17df50a5 XL |
404 | /// Collect all monomorphized items reachable from `starting_point`, and emit a note diagnostic if a |
405 | /// post-monorphization error is encountered during a collection step. | |
923072b8 | 406 | #[instrument(skip(tcx, visited, recursion_depths, recursion_limit, inlining_map), level = "debug")] |
dc9dc135 XL |
407 | fn collect_items_rec<'tcx>( |
408 | tcx: TyCtxt<'tcx>, | |
f035d41b | 409 | starting_point: Spanned<MonoItem<'tcx>>, |
353b0b11 | 410 | visited: MTLockRef<'_, FxHashSet<MonoItem<'tcx>>>, |
dc9dc135 | 411 | recursion_depths: &mut DefIdMap<usize>, |
136023e0 | 412 | recursion_limit: Limit, |
353b0b11 | 413 | inlining_map: MTLockRef<'_, InliningMap<'tcx>>, |
dc9dc135 | 414 | ) { |
f035d41b | 415 | if !visited.lock_mut().insert(starting_point.node) { |
7453a54e SL |
416 | // We've been here already, no need to search again. |
417 | return; | |
418 | } | |
7453a54e | 419 | |
923072b8 | 420 | let mut neighbors = MonoItems { compute_inlining: true, tcx, items: Vec::new() }; |
7453a54e SL |
421 | let recursion_depth_reset; |
422 | ||
17df50a5 XL |
423 | // |
424 | // Post-monomorphization errors MVP | |
425 | // | |
426 | // We can encounter errors while monomorphizing an item, but we don't have a good way of | |
427 | // showing a complete stack of spans ultimately leading to collecting the erroneous one yet. | |
428 | // (It's also currently unclear exactly which diagnostics and information would be interesting | |
429 | // to report in such cases) | |
430 | // | |
431 | // This leads to suboptimal error reporting: a post-monomorphization error (PME) will be | |
432 | // shown with just a spanned piece of code causing the error, without information on where | |
433 | // it was called from. This is especially obscure if the erroneous mono item is in a | |
434 | // dependency. See for example issue #85155, where, before minimization, a PME happened two | |
435 | // crates downstream from libcore's stdarch, without a way to know which dependency was the | |
436 | // cause. | |
437 | // | |
438 | // If such an error occurs in the current crate, its span will be enough to locate the | |
439 | // source. If the cause is in another crate, the goal here is to quickly locate which mono | |
440 | // item in the current crate is ultimately responsible for causing the error. | |
441 | // | |
442 | // To give at least _some_ context to the user: while collecting mono items, we check the | |
443 | // error count. If it has changed, a PME occurred, and we trigger some diagnostics about the | |
444 | // current step of mono items collection. | |
445 | // | |
04454e1e | 446 | // FIXME: don't rely on global state, instead bubble up errors. Note: this is very hard to do. |
17df50a5 XL |
447 | let error_count = tcx.sess.diagnostic().err_count(); |
448 | ||
f035d41b | 449 | match starting_point.node { |
0531ce1d | 450 | MonoItem::Static(def_id) => { |
ea8adc8c | 451 | let instance = Instance::mono(tcx, def_id); |
32a655c1 SL |
452 | |
453 | // Sanity check whether this ended up being collected accidentally | |
3dfed10e | 454 | debug_assert!(should_codegen_locally(tcx, &instance)); |
32a655c1 | 455 | |
3dfed10e | 456 | let ty = instance.ty(tcx, ty::ParamEnv::reveal_all()); |
f035d41b | 457 | visit_drop_use(tcx, ty, true, starting_point.span, &mut neighbors); |
a7813a04 | 458 | |
7453a54e | 459 | recursion_depth_reset = None; |
a7813a04 | 460 | |
1b1a35ee | 461 | if let Ok(alloc) = tcx.eval_static_initializer(def_id) { |
487cf647 | 462 | for &id in alloc.inner().provenance().ptrs().values() { |
1b1a35ee XL |
463 | collect_miri(tcx, id, &mut neighbors); |
464 | } | |
0531ce1d | 465 | } |
353b0b11 FG |
466 | |
467 | if tcx.needs_thread_local_shim(def_id) { | |
468 | neighbors.push(respan( | |
469 | starting_point.span, | |
470 | MonoItem::Fn(Instance { | |
471 | def: InstanceDef::ThreadLocalShim(def_id), | |
472 | substs: InternalSubsts::empty(), | |
473 | }), | |
474 | )); | |
475 | } | |
7453a54e | 476 | } |
ff7c6d11 | 477 | MonoItem::Fn(instance) => { |
32a655c1 | 478 | // Sanity check whether this ended up being collected accidentally |
3dfed10e | 479 | debug_assert!(should_codegen_locally(tcx, &instance)); |
32a655c1 | 480 | |
7453a54e | 481 | // Keep track of the monomorphization recursion depth |
136023e0 XL |
482 | recursion_depth_reset = Some(check_recursion_limit( |
483 | tcx, | |
484 | instance, | |
485 | starting_point.span, | |
486 | recursion_depths, | |
487 | recursion_limit, | |
488 | )); | |
ea8adc8c | 489 | check_type_length_limit(tcx, instance); |
7453a54e | 490 | |
f9f354fc XL |
491 | rustc_data_structures::stack::ensure_sufficient_stack(|| { |
492 | collect_neighbours(tcx, instance, &mut neighbors); | |
493 | }); | |
7453a54e | 494 | } |
17df50a5 | 495 | MonoItem::GlobalAsm(item_id) => { |
cc61c64b | 496 | recursion_depth_reset = None; |
17df50a5 XL |
497 | |
498 | let item = tcx.hir().item(item_id); | |
499 | if let hir::ItemKind::GlobalAsm(asm) = item.kind { | |
500 | for (op, op_sp) in asm.operands { | |
501 | match op { | |
502 | hir::InlineAsmOperand::Const { .. } => { | |
503 | // Only constants which resolve to a plain integer | |
504 | // are supported. Therefore the value should not | |
505 | // depend on any other items. | |
506 | } | |
04454e1e FG |
507 | hir::InlineAsmOperand::SymFn { anon_const } => { |
508 | let fn_ty = | |
509 | tcx.typeck_body(anon_const.body).node_type(anon_const.hir_id); | |
510 | visit_fn_use(tcx, fn_ty, false, *op_sp, &mut neighbors); | |
511 | } | |
512 | hir::InlineAsmOperand::SymStatic { path: _, def_id } => { | |
513 | let instance = Instance::mono(tcx, *def_id); | |
514 | if should_codegen_locally(tcx, &instance) { | |
515 | trace!("collecting static {:?}", def_id); | |
516 | neighbors.push(dummy_spanned(MonoItem::Static(*def_id))); | |
517 | } | |
518 | } | |
519 | hir::InlineAsmOperand::In { .. } | |
520 | | hir::InlineAsmOperand::Out { .. } | |
521 | | hir::InlineAsmOperand::InOut { .. } | |
522 | | hir::InlineAsmOperand::SplitInOut { .. } => { | |
523 | span_bug!(*op_sp, "invalid operand type for global_asm!") | |
524 | } | |
17df50a5 XL |
525 | } |
526 | } | |
527 | } else { | |
528 | span_bug!(item.span, "Mismatch between hir::Item type and MonoItem type") | |
529 | } | |
cc61c64b | 530 | } |
7453a54e SL |
531 | } |
532 | ||
17df50a5 | 533 | // Check for PMEs and emit a diagnostic if one happened. To try to show relevant edges of the |
04454e1e | 534 | // mono item graph. |
c295e0f8 | 535 | if tcx.sess.diagnostic().err_count() > error_count |
04454e1e | 536 | && starting_point.node.is_generic_fn() |
c295e0f8 | 537 | && starting_point.node.is_user_defined() |
17df50a5 | 538 | { |
5e7ed085 | 539 | let formatted_item = with_no_trimmed_paths!(starting_point.node.to_string()); |
9ffffee4 FG |
540 | tcx.sess.emit_note(EncounteredErrorWhileInstantiating { |
541 | span: starting_point.span, | |
542 | formatted_item, | |
543 | }); | |
17df50a5 | 544 | } |
923072b8 | 545 | inlining_map.lock_mut().record_accesses(starting_point.node, &neighbors.items); |
17df50a5 | 546 | |
923072b8 | 547 | for (neighbour, _) in neighbors.items { |
136023e0 | 548 | collect_items_rec(tcx, neighbour, visited, recursion_depths, recursion_limit, inlining_map); |
7453a54e SL |
549 | } |
550 | ||
551 | if let Some((def_id, depth)) = recursion_depth_reset { | |
552 | recursion_depths.insert(def_id, depth); | |
553 | } | |
7453a54e SL |
554 | } |
555 | ||
1b1a35ee | 556 | /// Format instance name that is already known to be too long for rustc. |
487cf647 | 557 | /// Show only the first 2 types if it is longer than 32 characters to avoid blasting |
1b1a35ee XL |
558 | /// the user's terminal with thousands of lines of type-name. |
559 | /// | |
560 | /// If the type name is longer than before+after, it will be written to a file. | |
a2a8927a | 561 | fn shrunk_instance_name<'tcx>( |
1b1a35ee XL |
562 | tcx: TyCtxt<'tcx>, |
563 | instance: &Instance<'tcx>, | |
1b1a35ee XL |
564 | ) -> (String, Option<PathBuf>) { |
565 | let s = instance.to_string(); | |
6c58768f XL |
566 | |
567 | // Only use the shrunk version if it's really shorter. | |
568 | // This also avoids the case where before and after slices overlap. | |
487cf647 FG |
569 | if s.chars().nth(33).is_some() { |
570 | let shrunk = format!("{}", ty::ShortInstance(instance, 4)); | |
571 | if shrunk == s { | |
572 | return (s, None); | |
573 | } | |
6c58768f | 574 | |
17df50a5 | 575 | let path = tcx.output_filenames(()).temp_path_ext("long-type.txt", None); |
1b1a35ee XL |
576 | let written_to_path = std::fs::write(&path, s).ok().map(|_| path); |
577 | ||
578 | (shrunk, written_to_path) | |
579 | } else { | |
580 | (s, None) | |
581 | } | |
6c58768f XL |
582 | } |
583 | ||
dc9dc135 XL |
584 | fn check_recursion_limit<'tcx>( |
585 | tcx: TyCtxt<'tcx>, | |
586 | instance: Instance<'tcx>, | |
f035d41b | 587 | span: Span, |
dc9dc135 | 588 | recursion_depths: &mut DefIdMap<usize>, |
136023e0 | 589 | recursion_limit: Limit, |
dc9dc135 | 590 | ) -> (DefId, usize) { |
cc61c64b XL |
591 | let def_id = instance.def_id(); |
592 | let recursion_depth = recursion_depths.get(&def_id).cloned().unwrap_or(0); | |
7453a54e SL |
593 | debug!(" => recursion depth={}", recursion_depth); |
594 | ||
dfeec247 | 595 | let adjusted_recursion_depth = if Some(def_id) == tcx.lang_items().drop_in_place_fn() { |
cc61c64b XL |
596 | // HACK: drop_in_place creates tight monomorphization loops. Give |
597 | // it more margin. | |
598 | recursion_depth / 4 | |
599 | } else { | |
600 | recursion_depth | |
601 | }; | |
602 | ||
7453a54e SL |
603 | // Code that needs to instantiate the same function recursively |
604 | // more than the recursion limit is assumed to be causing an | |
605 | // infinite expansion. | |
136023e0 | 606 | if !recursion_limit.value_within_limit(adjusted_recursion_depth) { |
f2b60f7d FG |
607 | let def_span = tcx.def_span(def_id); |
608 | let def_path_str = tcx.def_path_str(def_id); | |
487cf647 | 609 | let (shrunk, written_to_path) = shrunk_instance_name(tcx, &instance); |
f2b60f7d | 610 | let mut path = PathBuf::new(); |
9c376795 FG |
611 | let was_written = if let Some(written_to_path) = written_to_path { |
612 | path = written_to_path; | |
f2b60f7d FG |
613 | Some(()) |
614 | } else { | |
615 | None | |
616 | }; | |
617 | tcx.sess.emit_fatal(RecursionLimit { | |
618 | span, | |
619 | shrunk, | |
620 | def_span, | |
621 | def_path_str, | |
622 | was_written, | |
623 | path, | |
624 | }); | |
7453a54e SL |
625 | } |
626 | ||
cc61c64b | 627 | recursion_depths.insert(def_id, recursion_depth + 1); |
7453a54e | 628 | |
cc61c64b | 629 | (def_id, recursion_depth) |
7453a54e SL |
630 | } |
631 | ||
dc9dc135 | 632 | fn check_type_length_limit<'tcx>(tcx: TyCtxt<'tcx>, instance: Instance<'tcx>) { |
ba9703b0 XL |
633 | let type_length = instance |
634 | .substs | |
635 | .iter() | |
5099ac24 | 636 | .flat_map(|arg| arg.walk()) |
ba9703b0 XL |
637 | .filter(|arg| match arg.unpack() { |
638 | GenericArgKind::Type(_) | GenericArgKind::Const(_) => true, | |
639 | GenericArgKind::Lifetime(_) => false, | |
640 | }) | |
641 | .count(); | |
642 | debug!(" => type length={}", type_length); | |
476ff2be SL |
643 | |
644 | // Rust code can easily create exponentially-long types using only a | |
645 | // polynomial recursion depth. Even with the default recursion | |
646 | // depth, you can easily get cases that take >2^60 steps to run, | |
647 | // which means that rustc basically hangs. | |
648 | // | |
649 | // Bail out in these cases to avoid that bad user experience. | |
136023e0 | 650 | if !tcx.type_length_limit().value_within_limit(type_length) { |
487cf647 | 651 | let (shrunk, written_to_path) = shrunk_instance_name(tcx, &instance); |
f2b60f7d FG |
652 | let span = tcx.def_span(instance.def_id()); |
653 | let mut path = PathBuf::new(); | |
353b0b11 FG |
654 | let was_written = if let Some(path2) = written_to_path { |
655 | path = path2; | |
f2b60f7d FG |
656 | Some(()) |
657 | } else { | |
658 | None | |
659 | }; | |
660 | tcx.sess.emit_fatal(TypeLengthLimit { span, shrunk, was_written, path, type_length }); | |
476ff2be SL |
661 | } |
662 | } | |
663 | ||
dc9dc135 XL |
664 | struct MirNeighborCollector<'a, 'tcx> { |
665 | tcx: TyCtxt<'tcx>, | |
666 | body: &'a mir::Body<'tcx>, | |
923072b8 | 667 | output: &'a mut MonoItems<'tcx>, |
ba9703b0 XL |
668 | instance: Instance<'tcx>, |
669 | } | |
670 | ||
671 | impl<'a, 'tcx> MirNeighborCollector<'a, 'tcx> { | |
672 | pub fn monomorphize<T>(&self, value: T) -> T | |
673 | where | |
9ffffee4 | 674 | T: TypeFoldable<TyCtxt<'tcx>>, |
ba9703b0 XL |
675 | { |
676 | debug!("monomorphize: self.instance={:?}", self.instance); | |
29967ef6 XL |
677 | self.instance.subst_mir_and_normalize_erasing_regions( |
678 | self.tcx, | |
679 | ty::ParamEnv::reveal_all(), | |
fc512014 | 680 | value, |
29967ef6 | 681 | ) |
ba9703b0 | 682 | } |
7453a54e SL |
683 | } |
684 | ||
685 | impl<'a, 'tcx> MirVisitor<'tcx> for MirNeighborCollector<'a, 'tcx> { | |
9e0c209e | 686 | fn visit_rvalue(&mut self, rvalue: &mir::Rvalue<'tcx>, location: Location) { |
7453a54e SL |
687 | debug!("visiting rvalue {:?}", *rvalue); |
688 | ||
f035d41b XL |
689 | let span = self.body.source_info(location).span; |
690 | ||
7453a54e | 691 | match *rvalue { |
7453a54e SL |
692 | // When doing an cast from a regular pointer to a fat pointer, we |
693 | // have to instantiate all methods of the trait being cast to, so we | |
694 | // can build the appropriate vtable. | |
48663c56 | 695 | mir::Rvalue::Cast( |
dfeec247 XL |
696 | mir::CastKind::Pointer(PointerCast::Unsize), |
697 | ref operand, | |
698 | target_ty, | |
f2b60f7d FG |
699 | ) |
700 | | mir::Rvalue::Cast(mir::CastKind::DynStar, ref operand, target_ty) => { | |
ba9703b0 | 701 | let target_ty = self.monomorphize(target_ty); |
dc9dc135 | 702 | let source_ty = operand.ty(self.body, self.tcx); |
ba9703b0 | 703 | let source_ty = self.monomorphize(source_ty); |
dfeec247 | 704 | let (source_ty, target_ty) = |
487cf647 | 705 | find_vtable_types_for_unsizing(self.tcx.at(span), source_ty, target_ty); |
7453a54e SL |
706 | // This could also be a different Unsize instruction, like |
707 | // from a fixed sized array to a slice. But we are only | |
708 | // interested in things that produce a vtable. | |
f2b60f7d FG |
709 | if (target_ty.is_trait() && !source_ty.is_trait()) |
710 | || (target_ty.is_dyn_star() && !source_ty.is_dyn_star()) | |
711 | { | |
dfeec247 XL |
712 | create_mono_items_for_vtable_methods( |
713 | self.tcx, | |
714 | target_ty, | |
715 | source_ty, | |
f035d41b | 716 | span, |
dfeec247 XL |
717 | self.output, |
718 | ); | |
7453a54e SL |
719 | } |
720 | } | |
48663c56 | 721 | mir::Rvalue::Cast( |
dfeec247 XL |
722 | mir::CastKind::Pointer(PointerCast::ReifyFnPointer), |
723 | ref operand, | |
724 | _, | |
48663c56 | 725 | ) => { |
dc9dc135 | 726 | let fn_ty = operand.ty(self.body, self.tcx); |
ba9703b0 | 727 | let fn_ty = self.monomorphize(fn_ty); |
f035d41b | 728 | visit_fn_use(self.tcx, fn_ty, false, span, &mut self.output); |
cc61c64b | 729 | } |
48663c56 | 730 | mir::Rvalue::Cast( |
dfeec247 XL |
731 | mir::CastKind::Pointer(PointerCast::ClosureFnPointer(_)), |
732 | ref operand, | |
733 | _, | |
48663c56 | 734 | ) => { |
dc9dc135 | 735 | let source_ty = operand.ty(self.body, self.tcx); |
ba9703b0 | 736 | let source_ty = self.monomorphize(source_ty); |
1b1a35ee | 737 | match *source_ty.kind() { |
b7449926 | 738 | ty::Closure(def_id, substs) => { |
dc9dc135 | 739 | let instance = Instance::resolve_closure( |
dfeec247 XL |
740 | self.tcx, |
741 | def_id, | |
742 | substs, | |
743 | ty::ClosureKind::FnOnce, | |
064997fb FG |
744 | ) |
745 | .expect("failed to normalize and resolve closure during codegen"); | |
3dfed10e XL |
746 | if should_codegen_locally(self.tcx, &instance) { |
747 | self.output.push(create_fn_mono_item(self.tcx, instance, span)); | |
83c7162d | 748 | } |
8bb4bdeb XL |
749 | } |
750 | _ => bug!(), | |
751 | } | |
752 | } | |
f9f354fc XL |
753 | mir::Rvalue::ThreadLocalRef(def_id) => { |
754 | assert!(self.tcx.is_thread_local_static(def_id)); | |
755 | let instance = Instance::mono(self.tcx, def_id); | |
3dfed10e | 756 | if should_codegen_locally(self.tcx, &instance) { |
f9f354fc | 757 | trace!("collecting thread-local static {:?}", def_id); |
f035d41b | 758 | self.output.push(respan(span, MonoItem::Static(def_id))); |
f9f354fc XL |
759 | } |
760 | } | |
7453a54e SL |
761 | _ => { /* not interesting */ } |
762 | } | |
763 | ||
9e0c209e | 764 | self.super_rvalue(rvalue, location); |
7453a54e SL |
765 | } |
766 | ||
cdc7bbd5 XL |
767 | /// This does not walk the constant, as it has been handled entirely here and trying |
768 | /// to walk it would attempt to evaluate the `ty::Const` inside, which doesn't necessarily | |
769 | /// work, as some constants cannot be represented in the type system. | |
923072b8 | 770 | #[instrument(skip(self), level = "debug")] |
cdc7bbd5 XL |
771 | fn visit_constant(&mut self, constant: &mir::Constant<'tcx>, location: Location) { |
772 | let literal = self.monomorphize(constant.literal); | |
773 | let val = match literal { | |
774 | mir::ConstantKind::Val(val, _) => val, | |
923072b8 FG |
775 | mir::ConstantKind::Ty(ct) => match ct.kind() { |
776 | ty::ConstKind::Value(val) => self.tcx.valtree_to_const_val((ct.ty(), val)), | |
cdc7bbd5 | 777 | ty::ConstKind::Unevaluated(ct) => { |
923072b8 | 778 | debug!(?ct); |
cdc7bbd5 | 779 | let param_env = ty::ParamEnv::reveal_all(); |
f2b60f7d | 780 | match self.tcx.const_eval_resolve(param_env, ct.expand(), None) { |
cdc7bbd5 XL |
781 | // The `monomorphize` call should have evaluated that constant already. |
782 | Ok(val) => val, | |
487cf647 | 783 | Err(ErrorHandled::Reported(_)) => return, |
cdc7bbd5 XL |
784 | Err(ErrorHandled::TooGeneric) => span_bug!( |
785 | self.body.source_info(location).span, | |
786 | "collection encountered polymorphic constant: {:?}", | |
787 | literal | |
788 | ), | |
789 | } | |
790 | } | |
791 | _ => return, | |
792 | }, | |
f2b60f7d FG |
793 | mir::ConstantKind::Unevaluated(uv, _) => { |
794 | let param_env = ty::ParamEnv::reveal_all(); | |
795 | match self.tcx.const_eval_resolve(param_env, uv, None) { | |
cdc7bbd5 | 796 | // The `monomorphize` call should have evaluated that constant already. |
f2b60f7d | 797 | Ok(val) => val, |
487cf647 | 798 | Err(ErrorHandled::Reported(_)) => return, |
ba9703b0 | 799 | Err(ErrorHandled::TooGeneric) => span_bug!( |
3dfed10e | 800 | self.body.source_info(location).span, |
f2b60f7d FG |
801 | "collection encountered polymorphic constant: {:?}", |
802 | literal | |
ba9703b0 XL |
803 | ), |
804 | } | |
805 | } | |
f2b60f7d FG |
806 | }; |
807 | collect_const_value(self.tcx, val, self.output); | |
808 | MirVisitor::visit_ty(self, literal.ty(), TyContext::Location(location)); | |
a7813a04 XL |
809 | } |
810 | ||
f035d41b XL |
811 | fn visit_terminator(&mut self, terminator: &mir::Terminator<'tcx>, location: Location) { |
812 | debug!("visiting terminator {:?} @ {:?}", terminator, location); | |
813 | let source = self.body.source_info(location).span; | |
7cac9316 | 814 | |
ea8adc8c | 815 | let tcx = self.tcx; |
f035d41b | 816 | match terminator.kind { |
cc61c64b | 817 | mir::TerminatorKind::Call { ref func, .. } => { |
dc9dc135 | 818 | let callee_ty = func.ty(self.body, tcx); |
ba9703b0 | 819 | let callee_ty = self.monomorphize(callee_ty); |
f2b60f7d | 820 | visit_fn_use(self.tcx, callee_ty, true, source, &mut self.output) |
a7813a04 | 821 | } |
353b0b11 | 822 | mir::TerminatorKind::Drop { ref place, .. } => { |
f035d41b | 823 | let ty = place.ty(self.body, self.tcx).ty; |
ba9703b0 | 824 | let ty = self.monomorphize(ty); |
f035d41b | 825 | visit_drop_use(self.tcx, ty, true, source, self.output); |
cc61c64b | 826 | } |
f9f354fc XL |
827 | mir::TerminatorKind::InlineAsm { ref operands, .. } => { |
828 | for op in operands { | |
f035d41b XL |
829 | match *op { |
830 | mir::InlineAsmOperand::SymFn { ref value } => { | |
6a06907d | 831 | let fn_ty = self.monomorphize(value.literal.ty()); |
f035d41b XL |
832 | visit_fn_use(self.tcx, fn_ty, false, source, &mut self.output); |
833 | } | |
834 | mir::InlineAsmOperand::SymStatic { def_id } => { | |
835 | let instance = Instance::mono(self.tcx, def_id); | |
3dfed10e | 836 | if should_codegen_locally(self.tcx, &instance) { |
f035d41b XL |
837 | trace!("collecting asm sym static {:?}", def_id); |
838 | self.output.push(respan(source, MonoItem::Static(def_id))); | |
839 | } | |
840 | } | |
841 | _ => {} | |
f9f354fc XL |
842 | } |
843 | } | |
844 | } | |
3c0e092e XL |
845 | mir::TerminatorKind::Assert { ref msg, .. } => { |
846 | let lang_item = match msg { | |
847 | mir::AssertKind::BoundsCheck { .. } => LangItem::PanicBoundsCheck, | |
848 | _ => LangItem::Panic, | |
849 | }; | |
850 | let instance = Instance::mono(tcx, tcx.require_lang_item(lang_item, Some(source))); | |
851 | if should_codegen_locally(tcx, &instance) { | |
852 | self.output.push(create_fn_mono_item(tcx, instance, source)); | |
853 | } | |
854 | } | |
353b0b11 | 855 | mir::TerminatorKind::Terminate { .. } => { |
5099ac24 FG |
856 | let instance = Instance::mono( |
857 | tcx, | |
9c376795 | 858 | tcx.require_lang_item(LangItem::PanicCannotUnwind, Some(source)), |
5099ac24 FG |
859 | ); |
860 | if should_codegen_locally(tcx, &instance) { | |
861 | self.output.push(create_fn_mono_item(tcx, instance, source)); | |
862 | } | |
863 | } | |
dfeec247 XL |
864 | mir::TerminatorKind::Goto { .. } |
865 | | mir::TerminatorKind::SwitchInt { .. } | |
866 | | mir::TerminatorKind::Resume | |
dfeec247 | 867 | | mir::TerminatorKind::Return |
3c0e092e | 868 | | mir::TerminatorKind::Unreachable => {} |
dfeec247 XL |
869 | mir::TerminatorKind::GeneratorDrop |
870 | | mir::TerminatorKind::Yield { .. } | |
f035d41b | 871 | | mir::TerminatorKind::FalseEdge { .. } |
dfeec247 | 872 | | mir::TerminatorKind::FalseUnwind { .. } => bug!(), |
a7813a04 XL |
873 | } |
874 | ||
353b0b11 FG |
875 | if let Some(mir::UnwindAction::Terminate) = terminator.unwind() { |
876 | let instance = Instance::mono( | |
877 | tcx, | |
878 | tcx.require_lang_item(LangItem::PanicCannotUnwind, Some(source)), | |
879 | ); | |
880 | if should_codegen_locally(tcx, &instance) { | |
881 | self.output.push(create_fn_mono_item(tcx, instance, source)); | |
882 | } | |
883 | } | |
884 | ||
f035d41b | 885 | self.super_terminator(terminator, location); |
7453a54e | 886 | } |
3b2f2976 | 887 | |
cdc7bbd5 XL |
888 | fn visit_operand(&mut self, operand: &mir::Operand<'tcx>, location: Location) { |
889 | self.super_operand(operand, location); | |
136023e0 | 890 | let limit = self.tcx.move_size_limit().0; |
cdc7bbd5 XL |
891 | if limit == 0 { |
892 | return; | |
893 | } | |
894 | let limit = Size::from_bytes(limit); | |
895 | let ty = operand.ty(self.body, self.tcx); | |
896 | let ty = self.monomorphize(ty); | |
897 | let layout = self.tcx.layout_of(ty::ParamEnv::reveal_all().and(ty)); | |
898 | if let Ok(layout) = layout { | |
899 | if layout.size > limit { | |
900 | debug!(?layout); | |
901 | let source_info = self.body.source_info(location); | |
902 | debug!(?source_info); | |
903 | let lint_root = source_info.scope.lint_root(&self.body.source_scopes); | |
904 | debug!(?lint_root); | |
5e7ed085 | 905 | let Some(lint_root) = lint_root else { |
cdc7bbd5 XL |
906 | // This happens when the issue is in a function from a foreign crate that |
907 | // we monomorphized in the current crate. We can't get a `HirId` for things | |
908 | // in other crates. | |
909 | // FIXME: Find out where to report the lint on. Maybe simply crate-level lint root | |
910 | // but correct span? This would make the lint at least accept crate-level lint attributes. | |
5e7ed085 | 911 | return; |
cdc7bbd5 | 912 | }; |
f2b60f7d | 913 | self.tcx.emit_spanned_lint( |
cdc7bbd5 XL |
914 | LARGE_ASSIGNMENTS, |
915 | lint_root, | |
916 | source_info.span, | |
f2b60f7d FG |
917 | LargeAssignmentsLint { |
918 | span: source_info.span, | |
919 | size: layout.size.bytes(), | |
920 | limit: limit.bytes(), | |
cdc7bbd5 | 921 | }, |
f2b60f7d | 922 | ) |
cdc7bbd5 XL |
923 | } |
924 | } | |
925 | } | |
926 | ||
f9f354fc | 927 | fn visit_local( |
dfeec247 | 928 | &mut self, |
064997fb | 929 | _place_local: Local, |
dfeec247 XL |
930 | _context: mir::visit::PlaceContext, |
931 | _location: Location, | |
932 | ) { | |
3b2f2976 | 933 | } |
7453a54e SL |
934 | } |
935 | ||
dc9dc135 XL |
936 | fn visit_drop_use<'tcx>( |
937 | tcx: TyCtxt<'tcx>, | |
938 | ty: Ty<'tcx>, | |
939 | is_direct_call: bool, | |
f035d41b | 940 | source: Span, |
923072b8 | 941 | output: &mut MonoItems<'tcx>, |
dc9dc135 XL |
942 | ) { |
943 | let instance = Instance::resolve_drop_in_place(tcx, ty); | |
f035d41b | 944 | visit_instance_use(tcx, instance, is_direct_call, source, output); |
7453a54e SL |
945 | } |
946 | ||
dc9dc135 XL |
947 | fn visit_fn_use<'tcx>( |
948 | tcx: TyCtxt<'tcx>, | |
949 | ty: Ty<'tcx>, | |
950 | is_direct_call: bool, | |
f035d41b | 951 | source: Span, |
923072b8 | 952 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 953 | ) { |
1b1a35ee | 954 | if let ty::FnDef(def_id, substs) = *ty.kind() { |
f9f354fc | 955 | let instance = if is_direct_call { |
9c376795 | 956 | ty::Instance::expect_resolve(tcx, ty::ParamEnv::reveal_all(), def_id, substs) |
f9f354fc | 957 | } else { |
9c376795 FG |
958 | match ty::Instance::resolve_for_fn_ptr(tcx, ty::ParamEnv::reveal_all(), def_id, substs) |
959 | { | |
960 | Some(instance) => instance, | |
961 | _ => bug!("failed to resolve instance for {ty}"), | |
962 | } | |
f9f354fc | 963 | }; |
f035d41b | 964 | visit_instance_use(tcx, instance, is_direct_call, source, output); |
7453a54e | 965 | } |
cc61c64b | 966 | } |
7453a54e | 967 | |
dc9dc135 XL |
968 | fn visit_instance_use<'tcx>( |
969 | tcx: TyCtxt<'tcx>, | |
970 | instance: ty::Instance<'tcx>, | |
971 | is_direct_call: bool, | |
f035d41b | 972 | source: Span, |
923072b8 | 973 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 974 | ) { |
cc61c64b | 975 | debug!("visit_item_use({:?}, is_direct_call={:?})", instance, is_direct_call); |
3dfed10e | 976 | if !should_codegen_locally(tcx, &instance) { |
dfeec247 | 977 | return; |
7453a54e SL |
978 | } |
979 | ||
cc61c64b | 980 | match instance.def { |
dfeec247 | 981 | ty::InstanceDef::Virtual(..) | ty::InstanceDef::Intrinsic(_) => { |
cc61c64b | 982 | if !is_direct_call { |
60c5eb7d | 983 | bug!("{:?} being reified", instance); |
32a655c1 SL |
984 | } |
985 | } | |
353b0b11 FG |
986 | ty::InstanceDef::ThreadLocalShim(..) => { |
987 | bug!("{:?} being reified", instance); | |
988 | } | |
cc61c64b | 989 | ty::InstanceDef::DropGlue(_, None) => { |
60c5eb7d | 990 | // Don't need to emit noop drop glue if we are calling directly. |
cc61c64b | 991 | if !is_direct_call { |
3dfed10e | 992 | output.push(create_fn_mono_item(tcx, instance, source)); |
7453a54e SL |
993 | } |
994 | } | |
dfeec247 | 995 | ty::InstanceDef::DropGlue(_, Some(_)) |
064997fb | 996 | | ty::InstanceDef::VTableShim(..) |
dfeec247 XL |
997 | | ty::InstanceDef::ReifyShim(..) |
998 | | ty::InstanceDef::ClosureOnceShim { .. } | |
999 | | ty::InstanceDef::Item(..) | |
1000 | | ty::InstanceDef::FnPtrShim(..) | |
353b0b11 FG |
1001 | | ty::InstanceDef::CloneShim(..) |
1002 | | ty::InstanceDef::FnPtrAddrShim(..) => { | |
3dfed10e | 1003 | output.push(create_fn_mono_item(tcx, instance, source)); |
32a655c1 | 1004 | } |
7453a54e SL |
1005 | } |
1006 | } | |
1007 | ||
c295e0f8 XL |
1008 | /// Returns `true` if we should codegen an instance in the local crate, or returns `false` if we |
1009 | /// can just link to the upstream crate and therefore don't need a mono item. | |
3dfed10e | 1010 | fn should_codegen_locally<'tcx>(tcx: TyCtxt<'tcx>, instance: &Instance<'tcx>) -> bool { |
5099ac24 | 1011 | let Some(def_id) = instance.def.def_id_if_not_guaranteed_local_codegen() else { |
c295e0f8 | 1012 | return true; |
cc61c64b | 1013 | }; |
83c7162d | 1014 | |
a1dfa0c6 | 1015 | if tcx.is_foreign_item(def_id) { |
3dfed10e | 1016 | // Foreign items are always linked against, there's no way of instantiating them. |
a1dfa0c6 XL |
1017 | return false; |
1018 | } | |
1019 | ||
1020 | if def_id.is_local() { | |
3dfed10e | 1021 | // Local items cannot be referred to locally without monomorphizing them locally. |
a1dfa0c6 XL |
1022 | return true; |
1023 | } | |
1024 | ||
3dfed10e XL |
1025 | if tcx.is_reachable_non_generic(def_id) |
1026 | || instance.polymorphize(tcx).upstream_monomorphization(tcx).is_some() | |
1027 | { | |
1028 | // We can link to the item in question, no instance needed in this crate. | |
a1dfa0c6 XL |
1029 | return false; |
1030 | } | |
1031 | ||
f2b60f7d FG |
1032 | if let DefKind::Static(_) = tcx.def_kind(def_id) { |
1033 | // We cannot monomorphize statics from upstream crates. | |
1034 | return false; | |
1035 | } | |
1036 | ||
a1dfa0c6 | 1037 | if !tcx.is_mir_available(def_id) { |
5869c6ff | 1038 | bug!("no MIR available for {:?}", def_id); |
a1dfa0c6 | 1039 | } |
83c7162d | 1040 | |
ba9703b0 | 1041 | true |
7453a54e SL |
1042 | } |
1043 | ||
60c5eb7d | 1044 | /// For a given pair of source and target type that occur in an unsizing coercion, |
7453a54e SL |
1045 | /// this function finds the pair of types that determines the vtable linking |
1046 | /// them. | |
1047 | /// | |
5e7ed085 | 1048 | /// For example, the source type might be `&SomeStruct` and the target type |
f2b60f7d | 1049 | /// might be `&dyn SomeTrait` in a cast like: |
7453a54e | 1050 | /// |
f2b60f7d | 1051 | /// ```rust,ignore (not real code) |
7453a54e | 1052 | /// let src: &SomeStruct = ...; |
f2b60f7d FG |
1053 | /// let target = src as &dyn SomeTrait; |
1054 | /// ``` | |
7453a54e SL |
1055 | /// |
1056 | /// Then the output of this function would be (SomeStruct, SomeTrait) since for | |
1057 | /// constructing the `target` fat-pointer we need the vtable for that pair. | |
1058 | /// | |
1059 | /// Things can get more complicated though because there's also the case where | |
1060 | /// the unsized type occurs as a field: | |
1061 | /// | |
1062 | /// ```rust | |
1063 | /// struct ComplexStruct<T: ?Sized> { | |
1064 | /// a: u32, | |
1065 | /// b: f64, | |
1066 | /// c: T | |
1067 | /// } | |
1068 | /// ``` | |
1069 | /// | |
1070 | /// In this case, if `T` is sized, `&ComplexStruct<T>` is a thin pointer. If `T` | |
1071 | /// is unsized, `&SomeStruct` is a fat pointer, and the vtable it points to is | |
1072 | /// for the pair of `T` (which is a trait) and the concrete type that `T` was | |
1073 | /// originally coerced from: | |
1074 | /// | |
f2b60f7d | 1075 | /// ```rust,ignore (not real code) |
7453a54e | 1076 | /// let src: &ComplexStruct<SomeStruct> = ...; |
f2b60f7d FG |
1077 | /// let target = src as &ComplexStruct<dyn SomeTrait>; |
1078 | /// ``` | |
7453a54e SL |
1079 | /// |
1080 | /// Again, we want this `find_vtable_types_for_unsizing()` to provide the pair | |
1081 | /// `(SomeStruct, SomeTrait)`. | |
1082 | /// | |
0731742a | 1083 | /// Finally, there is also the case of custom unsizing coercions, e.g., for |
7453a54e | 1084 | /// smart pointers such as `Rc` and `Arc`. |
dc9dc135 | 1085 | fn find_vtable_types_for_unsizing<'tcx>( |
487cf647 | 1086 | tcx: TyCtxtAt<'tcx>, |
dc9dc135 XL |
1087 | source_ty: Ty<'tcx>, |
1088 | target_ty: Ty<'tcx>, | |
1089 | ) -> (Ty<'tcx>, Ty<'tcx>) { | |
ea8adc8c | 1090 | let ptr_vtable = |inner_source: Ty<'tcx>, inner_target: Ty<'tcx>| { |
416331ca | 1091 | let param_env = ty::ParamEnv::reveal_all(); |
ff7c6d11 | 1092 | let type_has_metadata = |ty: Ty<'tcx>| -> bool { |
487cf647 | 1093 | if ty.is_sized(tcx.tcx, param_env) { |
ff7c6d11 XL |
1094 | return false; |
1095 | } | |
416331ca | 1096 | let tail = tcx.struct_tail_erasing_lifetimes(ty, param_env); |
1b1a35ee | 1097 | match tail.kind() { |
b7449926 XL |
1098 | ty::Foreign(..) => false, |
1099 | ty::Str | ty::Slice(..) | ty::Dynamic(..) => true, | |
532ac7d7 | 1100 | _ => bug!("unexpected unsized tail: {:?}", tail), |
ff7c6d11 XL |
1101 | } |
1102 | }; | |
1103 | if type_has_metadata(inner_source) { | |
32a655c1 SL |
1104 | (inner_source, inner_target) |
1105 | } else { | |
416331ca | 1106 | tcx.struct_lockstep_tails_erasing_lifetimes(inner_source, inner_target, param_env) |
32a655c1 SL |
1107 | } |
1108 | }; | |
ff7c6d11 | 1109 | |
1b1a35ee | 1110 | match (&source_ty.kind(), &target_ty.kind()) { |
ba9703b0 | 1111 | (&ty::Ref(_, a, _), &ty::Ref(_, b, _) | &ty::RawPtr(ty::TypeAndMut { ty: b, .. })) |
dfeec247 | 1112 | | (&ty::RawPtr(ty::TypeAndMut { ty: a, .. }), &ty::RawPtr(ty::TypeAndMut { ty: b, .. })) => { |
5099ac24 | 1113 | ptr_vtable(*a, *b) |
32a655c1 | 1114 | } |
b7449926 | 1115 | (&ty::Adt(def_a, _), &ty::Adt(def_b, _)) if def_a.is_box() && def_b.is_box() => { |
32a655c1 | 1116 | ptr_vtable(source_ty.boxed_ty(), target_ty.boxed_ty()) |
7453a54e SL |
1117 | } |
1118 | ||
f2b60f7d FG |
1119 | // T as dyn* Trait |
1120 | (_, &ty::Dynamic(_, _, ty::DynStar)) => ptr_vtable(source_ty, target_ty), | |
1121 | ||
dfeec247 | 1122 | (&ty::Adt(source_adt_def, source_substs), &ty::Adt(target_adt_def, target_substs)) => { |
7453a54e SL |
1123 | assert_eq!(source_adt_def, target_adt_def); |
1124 | ||
74b04a01 | 1125 | let CustomCoerceUnsized::Struct(coerce_index) = |
c295e0f8 | 1126 | crate::custom_coerce_unsize_info(tcx, source_ty, target_ty); |
7453a54e | 1127 | |
2c00a5a8 XL |
1128 | let source_fields = &source_adt_def.non_enum_variant().fields; |
1129 | let target_fields = &target_adt_def.non_enum_variant().fields; | |
7453a54e | 1130 | |
dfeec247 | 1131 | assert!( |
353b0b11 FG |
1132 | coerce_index.index() < source_fields.len() |
1133 | && source_fields.len() == target_fields.len() | |
dfeec247 | 1134 | ); |
7453a54e | 1135 | |
dfeec247 XL |
1136 | find_vtable_types_for_unsizing( |
1137 | tcx, | |
487cf647 FG |
1138 | source_fields[coerce_index].ty(*tcx, source_substs), |
1139 | target_fields[coerce_index].ty(*tcx, target_substs), | |
60c5eb7d | 1140 | ) |
7453a54e | 1141 | } |
dfeec247 XL |
1142 | _ => bug!( |
1143 | "find_vtable_types_for_unsizing: invalid coercion {:?} -> {:?}", | |
1144 | source_ty, | |
1145 | target_ty | |
1146 | ), | |
7453a54e SL |
1147 | } |
1148 | } | |
1149 | ||
f2b60f7d | 1150 | #[instrument(skip(tcx), level = "debug", ret)] |
3dfed10e XL |
1151 | fn create_fn_mono_item<'tcx>( |
1152 | tcx: TyCtxt<'tcx>, | |
1153 | instance: Instance<'tcx>, | |
1154 | source: Span, | |
1155 | ) -> Spanned<MonoItem<'tcx>> { | |
136023e0 | 1156 | let def_id = instance.def_id(); |
064997fb | 1157 | if tcx.sess.opts.unstable_opts.profile_closures && def_id.is_local() && tcx.is_closure(def_id) { |
c295e0f8 | 1158 | crate::util::dump_closure_profile(tcx, instance); |
136023e0 XL |
1159 | } |
1160 | ||
f2b60f7d | 1161 | respan(source, MonoItem::Fn(instance.polymorphize(tcx))) |
7453a54e SL |
1162 | } |
1163 | ||
ff7c6d11 | 1164 | /// Creates a `MonoItem` for each method that is referenced by the vtable for |
7453a54e | 1165 | /// the given trait/impl pair. |
dc9dc135 XL |
1166 | fn create_mono_items_for_vtable_methods<'tcx>( |
1167 | tcx: TyCtxt<'tcx>, | |
1168 | trait_ty: Ty<'tcx>, | |
1169 | impl_ty: Ty<'tcx>, | |
f035d41b | 1170 | source: Span, |
923072b8 | 1171 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 1172 | ) { |
3dfed10e | 1173 | assert!(!trait_ty.has_escaping_bound_vars() && !impl_ty.has_escaping_bound_vars()); |
7453a54e | 1174 | |
1b1a35ee | 1175 | if let ty::Dynamic(ref trait_ty, ..) = trait_ty.kind() { |
0731742a XL |
1176 | if let Some(principal) = trait_ty.principal() { |
1177 | let poly_trait_ref = principal.with_self_ty(tcx, impl_ty); | |
1178 | assert!(!poly_trait_ref.has_escaping_bound_vars()); | |
1179 | ||
1180 | // Walk all methods of the trait, including those of its supertraits | |
136023e0 XL |
1181 | let entries = tcx.vtable_entries(poly_trait_ref); |
1182 | let methods = entries | |
dfeec247 | 1183 | .iter() |
136023e0 XL |
1184 | .filter_map(|entry| match entry { |
1185 | VtblEntry::MetadataDropInPlace | |
1186 | | VtblEntry::MetadataSize | |
1187 | | VtblEntry::MetadataAlign | |
1188 | | VtblEntry::Vacant => None, | |
94222f64 XL |
1189 | VtblEntry::TraitVPtr(_) => { |
1190 | // all super trait items already covered, so skip them. | |
1191 | None | |
1192 | } | |
1193 | VtblEntry::Method(instance) => { | |
1194 | Some(*instance).filter(|instance| should_codegen_locally(tcx, instance)) | |
1195 | } | |
dfeec247 | 1196 | }) |
3dfed10e | 1197 | .map(|item| create_fn_mono_item(tcx, item, source)); |
0731742a XL |
1198 | output.extend(methods); |
1199 | } | |
1200 | ||
60c5eb7d | 1201 | // Also add the destructor. |
f035d41b | 1202 | visit_drop_use(tcx, impl_ty, false, source, output); |
7453a54e SL |
1203 | } |
1204 | } | |
1205 | ||
1206 | //=----------------------------------------------------------------------------- | |
1207 | // Root Collection | |
1208 | //=----------------------------------------------------------------------------- | |
1209 | ||
dc9dc135 XL |
1210 | struct RootCollector<'a, 'tcx> { |
1211 | tcx: TyCtxt<'tcx>, | |
ff7c6d11 | 1212 | mode: MonoItemCollectionMode, |
923072b8 | 1213 | output: &'a mut MonoItems<'tcx>, |
cdc7bbd5 | 1214 | entry_fn: Option<(DefId, EntryFnType)>, |
7453a54e SL |
1215 | } |
1216 | ||
04454e1e FG |
1217 | impl<'v> RootCollector<'_, 'v> { |
1218 | fn process_item(&mut self, id: hir::ItemId) { | |
2b03887a | 1219 | match self.tcx.def_kind(id.owner_id) { |
04454e1e | 1220 | DefKind::Enum | DefKind::Struct | DefKind::Union => { |
9ffffee4 FG |
1221 | if self.mode == MonoItemCollectionMode::Eager |
1222 | && self.tcx.generics_of(id.owner_id).count() == 0 | |
1223 | { | |
1224 | debug!("RootCollector: ADT drop-glue for `{id:?}`",); | |
1225 | ||
1226 | let ty = self.tcx.type_of(id.owner_id.to_def_id()).no_bound_vars().unwrap(); | |
1227 | visit_drop_use(self.tcx, ty, true, DUMMY_SP, self.output); | |
7453a54e SL |
1228 | } |
1229 | } | |
04454e1e | 1230 | DefKind::GlobalAsm => { |
dfeec247 XL |
1231 | debug!( |
1232 | "RootCollector: ItemKind::GlobalAsm({})", | |
2b03887a | 1233 | self.tcx.def_path_str(id.owner_id.to_def_id()) |
dfeec247 | 1234 | ); |
04454e1e | 1235 | self.output.push(dummy_spanned(MonoItem::GlobalAsm(id))); |
cc61c64b | 1236 | } |
04454e1e | 1237 | DefKind::Static(..) => { |
353b0b11 FG |
1238 | let def_id = id.owner_id.to_def_id(); |
1239 | debug!("RootCollector: ItemKind::Static({})", self.tcx.def_path_str(def_id)); | |
1240 | self.output.push(dummy_spanned(MonoItem::Static(def_id))); | |
7453a54e | 1241 | } |
04454e1e | 1242 | DefKind::Const => { |
ff7c6d11 | 1243 | // const items only generate mono items if they are |
5bcae85e | 1244 | // actually used somewhere. Just declaring them is insufficient. |
a1dfa0c6 XL |
1245 | |
1246 | // but even just declaring them must collect the items they refer to | |
2b03887a | 1247 | if let Ok(val) = self.tcx.const_eval_poly(id.owner_id.to_def_id()) { |
74b04a01 | 1248 | collect_const_value(self.tcx, val, &mut self.output); |
a1dfa0c6 | 1249 | } |
5bcae85e | 1250 | } |
9ffffee4 | 1251 | DefKind::Impl { .. } => { |
04454e1e | 1252 | if self.mode == MonoItemCollectionMode::Eager { |
9ffffee4 | 1253 | create_mono_items_for_default_impls(self.tcx, id, self.output); |
04454e1e FG |
1254 | } |
1255 | } | |
1256 | DefKind::Fn => { | |
2b03887a | 1257 | self.push_if_root(id.owner_id.def_id); |
7453a54e | 1258 | } |
04454e1e | 1259 | _ => {} |
7453a54e | 1260 | } |
7453a54e SL |
1261 | } |
1262 | ||
04454e1e | 1263 | fn process_impl_item(&mut self, id: hir::ImplItemId) { |
2b03887a FG |
1264 | if matches!(self.tcx.def_kind(id.owner_id), DefKind::AssocFn) { |
1265 | self.push_if_root(id.owner_id.def_id); | |
7453a54e | 1266 | } |
7453a54e | 1267 | } |
fc512014 | 1268 | |
f9f354fc | 1269 | fn is_root(&self, def_id: LocalDefId) -> bool { |
dfeec247 XL |
1270 | !item_requires_monomorphization(self.tcx, def_id) |
1271 | && match self.mode { | |
1272 | MonoItemCollectionMode::Eager => true, | |
1273 | MonoItemCollectionMode::Lazy => { | |
cdc7bbd5 | 1274 | self.entry_fn.and_then(|(id, _)| id.as_local()) == Some(def_id) |
dfeec247 XL |
1275 | || self.tcx.is_reachable_non_generic(def_id) |
1276 | || self | |
1277 | .tcx | |
1278 | .codegen_fn_attrs(def_id) | |
1279 | .flags | |
1280 | .contains(CodegenFnAttrFlags::RUSTC_STD_INTERNAL_SYMBOL) | |
1281 | } | |
abe05a73 | 1282 | } |
abe05a73 | 1283 | } |
ff7c6d11 | 1284 | |
60c5eb7d | 1285 | /// If `def_id` represents a root, pushes it onto the list of |
ff7c6d11 | 1286 | /// outputs. (Note that all roots must be monomorphic.) |
923072b8 | 1287 | #[instrument(skip(self), level = "debug")] |
f9f354fc | 1288 | fn push_if_root(&mut self, def_id: LocalDefId) { |
ff7c6d11 | 1289 | if self.is_root(def_id) { |
f2b60f7d | 1290 | debug!("found root"); |
ff7c6d11 | 1291 | |
f9f354fc | 1292 | let instance = Instance::mono(self.tcx, def_id.to_def_id()); |
3dfed10e | 1293 | self.output.push(create_fn_mono_item(self.tcx, instance, DUMMY_SP)); |
ff7c6d11 XL |
1294 | } |
1295 | } | |
1296 | ||
1297 | /// As a special case, when/if we encounter the | |
1298 | /// `main()` function, we also have to generate a | |
1299 | /// monomorphized copy of the start lang item based on | |
1300 | /// the return type of `main`. This is not needed when | |
1301 | /// the user writes their own `start` manually. | |
0531ce1d | 1302 | fn push_extra_entry_roots(&mut self) { |
f2b60f7d | 1303 | let Some((main_def_id, EntryFnType::Main { .. })) = self.entry_fn else { |
5e7ed085 | 1304 | return; |
0531ce1d XL |
1305 | }; |
1306 | ||
487cf647 | 1307 | let start_def_id = self.tcx.require_lang_item(LangItem::Start, None); |
9ffffee4 | 1308 | let main_ret_ty = self.tcx.fn_sig(main_def_id).no_bound_vars().unwrap().output(); |
ff7c6d11 XL |
1309 | |
1310 | // Given that `main()` has no arguments, | |
1311 | // then its return type cannot have | |
1312 | // late-bound regions, since late-bound | |
1313 | // regions must appear in the argument | |
1314 | // listing. | |
5e7ed085 FG |
1315 | let main_ret_ty = self.tcx.normalize_erasing_regions( |
1316 | ty::ParamEnv::reveal_all(), | |
1317 | main_ret_ty.no_bound_vars().unwrap(), | |
1318 | ); | |
ff7c6d11 XL |
1319 | |
1320 | let start_instance = Instance::resolve( | |
1321 | self.tcx, | |
0531ce1d | 1322 | ty::ParamEnv::reveal_all(), |
ff7c6d11 | 1323 | start_def_id, |
9ffffee4 | 1324 | self.tcx.mk_substs(&[main_ret_ty.into()]), |
dfeec247 | 1325 | ) |
f9f354fc | 1326 | .unwrap() |
dfeec247 | 1327 | .unwrap(); |
ff7c6d11 | 1328 | |
3dfed10e | 1329 | self.output.push(create_fn_mono_item(self.tcx, start_instance, DUMMY_SP)); |
ff7c6d11 | 1330 | } |
abe05a73 XL |
1331 | } |
1332 | ||
f9f354fc | 1333 | fn item_requires_monomorphization(tcx: TyCtxt<'_>, def_id: LocalDefId) -> bool { |
3b2f2976 | 1334 | let generics = tcx.generics_of(def_id); |
94b46f34 | 1335 | generics.requires_monomorphization(tcx) |
3b2f2976 XL |
1336 | } |
1337 | ||
9ffffee4 | 1338 | #[instrument(level = "debug", skip(tcx, output))] |
dc9dc135 XL |
1339 | fn create_mono_items_for_default_impls<'tcx>( |
1340 | tcx: TyCtxt<'tcx>, | |
9ffffee4 | 1341 | item: hir::ItemId, |
923072b8 | 1342 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 1343 | ) { |
9ffffee4 FG |
1344 | let polarity = tcx.impl_polarity(item.owner_id); |
1345 | if matches!(polarity, ty::ImplPolarity::Negative) { | |
1346 | return; | |
1347 | } | |
487cf647 | 1348 | |
9ffffee4 FG |
1349 | if tcx.generics_of(item.owner_id).own_requires_monomorphization() { |
1350 | return; | |
1351 | } | |
7453a54e | 1352 | |
9ffffee4 FG |
1353 | let Some(trait_ref) = tcx.impl_trait_ref(item.owner_id) else { |
1354 | return; | |
1355 | }; | |
7453a54e | 1356 | |
353b0b11 FG |
1357 | // Lifetimes never affect trait selection, so we are allowed to eagerly |
1358 | // instantiate an instance of an impl method if the impl (and method, | |
1359 | // which we check below) is only parameterized over lifetime. In that case, | |
1360 | // we use the ReErased, which has no lifetime information associated with | |
1361 | // it, to validate whether or not the impl is legal to instantiate at all. | |
1362 | let only_region_params = |param: &ty::GenericParamDef, _: &_| match param.kind { | |
1363 | GenericParamDefKind::Lifetime => tcx.lifetimes.re_erased.into(), | |
1364 | GenericParamDefKind::Type { .. } | GenericParamDefKind::Const { .. } => { | |
1365 | unreachable!( | |
1366 | "`own_requires_monomorphization` check means that \ | |
1367 | we should have no type/const params" | |
1368 | ) | |
1369 | } | |
1370 | }; | |
1371 | let impl_substs = InternalSubsts::for_item(tcx, item.owner_id.to_def_id(), only_region_params); | |
1372 | let trait_ref = trait_ref.subst(tcx, impl_substs); | |
1373 | ||
1374 | // Unlike 'lazy' monomorphization that begins by collecting items transitively | |
1375 | // called by `main` or other global items, when eagerly monomorphizing impl | |
1376 | // items, we never actually check that the predicates of this impl are satisfied | |
1377 | // in a empty reveal-all param env (i.e. with no assumptions). | |
1378 | // | |
1379 | // Even though this impl has no type or const substitutions, because we don't | |
1380 | // consider higher-ranked predicates such as `for<'a> &'a mut [u8]: Copy` to | |
1381 | // be trivially false. We must now check that the impl has no impossible-to-satisfy | |
1382 | // predicates. | |
1383 | if tcx.subst_and_check_impossible_predicates((item.owner_id.to_def_id(), impl_substs)) { | |
1384 | return; | |
1385 | } | |
9c376795 | 1386 | |
9ffffee4 FG |
1387 | let param_env = ty::ParamEnv::reveal_all(); |
1388 | let trait_ref = tcx.normalize_erasing_regions(param_env, trait_ref); | |
1389 | let overridden_methods = tcx.impl_item_implementor_ids(item.owner_id); | |
1390 | for method in tcx.provided_trait_methods(trait_ref.def_id) { | |
1391 | if overridden_methods.contains_key(&method.def_id) { | |
1392 | continue; | |
1393 | } | |
7453a54e | 1394 | |
9ffffee4 FG |
1395 | if tcx.generics_of(method.def_id).own_requires_monomorphization() { |
1396 | continue; | |
1397 | } | |
7453a54e | 1398 | |
353b0b11 FG |
1399 | // As mentioned above, the method is legal to eagerly instantiate if it |
1400 | // only has lifetime substitutions. This is validated by | |
1401 | let substs = trait_ref.substs.extend_to(tcx, method.def_id, only_region_params); | |
9ffffee4 FG |
1402 | let instance = ty::Instance::expect_resolve(tcx, param_env, method.def_id, substs); |
1403 | ||
1404 | let mono_item = create_fn_mono_item(tcx, instance, DUMMY_SP); | |
1405 | if mono_item.node.is_instantiable(tcx) && should_codegen_locally(tcx, &instance) { | |
1406 | output.push(mono_item); | |
7453a54e | 1407 | } |
7453a54e SL |
1408 | } |
1409 | } | |
1410 | ||
60c5eb7d | 1411 | /// Scans the miri alloc in order to find function calls, closures, and drop-glue. |
923072b8 | 1412 | fn collect_miri<'tcx>(tcx: TyCtxt<'tcx>, alloc_id: AllocId, output: &mut MonoItems<'tcx>) { |
f9f354fc XL |
1413 | match tcx.global_alloc(alloc_id) { |
1414 | GlobalAlloc::Static(def_id) => { | |
1415 | assert!(!tcx.is_thread_local_static(def_id)); | |
dc9dc135 | 1416 | let instance = Instance::mono(tcx, def_id); |
3dfed10e | 1417 | if should_codegen_locally(tcx, &instance) { |
dc9dc135 | 1418 | trace!("collecting static {:?}", def_id); |
f035d41b | 1419 | output.push(dummy_spanned(MonoItem::Static(def_id))); |
94b46f34 | 1420 | } |
0531ce1d | 1421 | } |
f9f354fc | 1422 | GlobalAlloc::Memory(alloc) => { |
94b46f34 | 1423 | trace!("collecting {:?} with {:#?}", alloc_id, alloc); |
487cf647 | 1424 | for &inner in alloc.inner().provenance().ptrs().values() { |
f9f354fc XL |
1425 | rustc_data_structures::stack::ensure_sufficient_stack(|| { |
1426 | collect_miri(tcx, inner, output); | |
1427 | }); | |
94b46f34 | 1428 | } |
dfeec247 | 1429 | } |
f9f354fc | 1430 | GlobalAlloc::Function(fn_instance) => { |
3dfed10e | 1431 | if should_codegen_locally(tcx, &fn_instance) { |
94b46f34 | 1432 | trace!("collecting {:?} with {:#?}", alloc_id, fn_instance); |
3dfed10e | 1433 | output.push(create_fn_mono_item(tcx, fn_instance, DUMMY_SP)); |
94b46f34 | 1434 | } |
0531ce1d | 1435 | } |
064997fb FG |
1436 | GlobalAlloc::VTable(ty, trait_ref) => { |
1437 | let alloc_id = tcx.vtable_allocation((ty, trait_ref)); | |
1438 | collect_miri(tcx, alloc_id, output) | |
1439 | } | |
0531ce1d XL |
1440 | } |
1441 | } | |
1442 | ||
60c5eb7d | 1443 | /// Scans the MIR in order to find function calls, closures, and drop-glue. |
923072b8 | 1444 | #[instrument(skip(tcx, output), level = "debug")] |
dc9dc135 XL |
1445 | fn collect_neighbours<'tcx>( |
1446 | tcx: TyCtxt<'tcx>, | |
1447 | instance: Instance<'tcx>, | |
923072b8 | 1448 | output: &mut MonoItems<'tcx>, |
dc9dc135 XL |
1449 | ) { |
1450 | let body = tcx.instance_mir(instance.def); | |
ba9703b0 | 1451 | MirNeighborCollector { tcx, body: &body, output, instance }.visit_body(&body); |
7453a54e | 1452 | } |
476ff2be | 1453 | |
923072b8 | 1454 | #[instrument(skip(tcx, output), level = "debug")] |
74b04a01 XL |
1455 | fn collect_const_value<'tcx>( |
1456 | tcx: TyCtxt<'tcx>, | |
1457 | value: ConstValue<'tcx>, | |
923072b8 | 1458 | output: &mut MonoItems<'tcx>, |
74b04a01 XL |
1459 | ) { |
1460 | match value { | |
136023e0 | 1461 | ConstValue::Scalar(Scalar::Ptr(ptr, _size)) => collect_miri(tcx, ptr.provenance, output), |
74b04a01 | 1462 | ConstValue::Slice { data: alloc, start: _, end: _ } | ConstValue::ByRef { alloc, .. } => { |
487cf647 | 1463 | for &id in alloc.inner().provenance().ptrs().values() { |
74b04a01 XL |
1464 | collect_miri(tcx, id, output); |
1465 | } | |
1466 | } | |
1467 | _ => {} | |
1468 | } | |
1469 | } |