]>
Commit | Line | Data |
---|---|---|
ff7c6d11 | 1 | //! Mono Item Collection |
60c5eb7d | 2 | //! ==================== |
7453a54e | 3 | //! |
f035d41b | 4 | //! This module is responsible for discovering all items that will contribute |
7453a54e SL |
5 | //! to code generation of the crate. The important part here is that it not only |
6 | //! needs to find syntax-level items (functions, structs, etc) but also all | |
7 | //! their monomorphized instantiations. Every non-generic, non-const function | |
8 | //! maps to one LLVM artifact. Every generic function can produce | |
9 | //! from zero to N artifacts, depending on the sets of type arguments it | |
10 | //! is instantiated with. | |
11 | //! This also applies to generic items from other crates: A generic definition | |
12 | //! in crate X might produce monomorphizations that are compiled into crate Y. | |
13 | //! We also have to collect these here. | |
14 | //! | |
ff7c6d11 | 15 | //! The following kinds of "mono items" are handled here: |
7453a54e SL |
16 | //! |
17 | //! - Functions | |
18 | //! - Methods | |
19 | //! - Closures | |
20 | //! - Statics | |
21 | //! - Drop glue | |
22 | //! | |
23 | //! The following things also result in LLVM artifacts, but are not collected | |
24 | //! here, since we instantiate them locally on demand when needed in a given | |
25 | //! codegen unit: | |
26 | //! | |
27 | //! - Constants | |
064997fb | 28 | //! - VTables |
7453a54e SL |
29 | //! - Object Shims |
30 | //! | |
31 | //! | |
32 | //! General Algorithm | |
33 | //! ----------------- | |
34 | //! Let's define some terms first: | |
35 | //! | |
ff7c6d11 XL |
36 | //! - A "mono item" is something that results in a function or global in |
37 | //! the LLVM IR of a codegen unit. Mono items do not stand on their | |
fe692bf9 | 38 | //! own, they can use other mono items. For example, if function |
ff7c6d11 | 39 | //! `foo()` calls function `bar()` then the mono item for `foo()` |
fe692bf9 FG |
40 | //! uses the mono item for function `bar()`. In general, the |
41 | //! definition for mono item A using a mono item B is that | |
42 | //! the LLVM artifact produced for A uses the LLVM artifact produced | |
7453a54e SL |
43 | //! for B. |
44 | //! | |
fe692bf9 FG |
45 | //! - Mono items and the uses between them form a directed graph, |
46 | //! where the mono items are the nodes and uses form the edges. | |
ff7c6d11 | 47 | //! Let's call this graph the "mono item graph". |
7453a54e | 48 | //! |
ff7c6d11 | 49 | //! - The mono item graph for a program contains all mono items |
7453a54e SL |
50 | //! that are needed in order to produce the complete LLVM IR of the program. |
51 | //! | |
52 | //! The purpose of the algorithm implemented in this module is to build the | |
ff7c6d11 | 53 | //! mono item graph for the current crate. It runs in two phases: |
7453a54e SL |
54 | //! |
55 | //! 1. Discover the roots of the graph by traversing the HIR of the crate. | |
fe692bf9 | 56 | //! 2. Starting from the roots, find uses by inspecting the MIR |
7453a54e SL |
57 | //! representation of the item corresponding to a given node, until no more |
58 | //! new nodes are found. | |
59 | //! | |
60 | //! ### Discovering roots | |
cdc7bbd5 | 61 | //! The roots of the mono item graph correspond to the public non-generic |
7453a54e | 62 | //! syntactic items in the source code. We find them by walking the HIR of the |
cdc7bbd5 XL |
63 | //! crate, and whenever we hit upon a public function, method, or static item, |
64 | //! we create a mono item consisting of the items DefId and, since we only | |
65 | //! consider non-generic items, an empty type-substitution set. (In eager | |
66 | //! collection mode, during incremental compilation, all non-generic functions | |
67 | //! are considered as roots, as well as when the `-Clink-dead-code` option is | |
68 | //! specified. Functions marked `#[no_mangle]` and functions called by inlinable | |
69 | //! functions also always act as roots.) | |
7453a54e | 70 | //! |
fe692bf9 FG |
71 | //! ### Finding uses |
72 | //! Given a mono item node, we can discover uses by inspecting its MIR. We walk | |
73 | //! the MIR to find other mono items used by each mono item. Since the mono | |
74 | //! item we are currently at is always monomorphic, we also know the concrete | |
75 | //! type arguments of its used mono items. The specific forms a use can take in | |
76 | //! MIR are quite diverse. Here is an overview: | |
7453a54e SL |
77 | //! |
78 | //! #### Calling Functions/Methods | |
fe692bf9 | 79 | //! The most obvious way for one mono item to use another is a |
7453a54e | 80 | //! function or method call (represented by a CALL terminator in MIR). But |
fe692bf9 | 81 | //! calls are not the only thing that might introduce a use between two |
ff7c6d11 | 82 | //! function mono items, and as we will see below, they are just a |
f035d41b | 83 | //! specialization of the form described next, and consequently will not get any |
7453a54e SL |
84 | //! special treatment in the algorithm. |
85 | //! | |
86 | //! #### Taking a reference to a function or method | |
fe692bf9 | 87 | //! A function does not need to actually be called in order to be used by |
7453a54e SL |
88 | //! another function. It suffices to just take a reference in order to introduce |
89 | //! an edge. Consider the following example: | |
90 | //! | |
04454e1e FG |
91 | //! ``` |
92 | //! # use core::fmt::Display; | |
7453a54e SL |
93 | //! fn print_val<T: Display>(x: T) { |
94 | //! println!("{}", x); | |
95 | //! } | |
96 | //! | |
04454e1e | 97 | //! fn call_fn(f: &dyn Fn(i32), x: i32) { |
7453a54e SL |
98 | //! f(x); |
99 | //! } | |
100 | //! | |
101 | //! fn main() { | |
102 | //! let print_i32 = print_val::<i32>; | |
103 | //! call_fn(&print_i32, 0); | |
104 | //! } | |
105 | //! ``` | |
106 | //! The MIR of none of these functions will contain an explicit call to | |
ff7c6d11 | 107 | //! `print_val::<i32>`. Nonetheless, in order to mono this program, we need |
7453a54e | 108 | //! an instance of this function. Thus, whenever we encounter a function or |
fe692bf9 | 109 | //! method in operand position, we treat it as a use of the current |
ff7c6d11 | 110 | //! mono item. Calls are just a special case of that. |
7453a54e | 111 | //! |
7453a54e | 112 | //! #### Drop glue |
ff7c6d11 | 113 | //! Drop glue mono items are introduced by MIR drop-statements. The |
fe692bf9 | 114 | //! generated mono item will have additional drop-glue item uses if the |
7453a54e | 115 | //! type to be dropped contains nested values that also need to be dropped. It |
fe692bf9 | 116 | //! might also have a function item use for the explicit `Drop::drop` |
7453a54e SL |
117 | //! implementation of its type. |
118 | //! | |
119 | //! #### Unsizing Casts | |
fe692bf9 | 120 | //! A subtle way of introducing use edges is by casting to a trait object. |
7453a54e | 121 | //! Since the resulting fat-pointer contains a reference to a vtable, we need to |
f2b60f7d | 122 | //! instantiate all object-safe methods of the trait, as we need to store |
7453a54e SL |
123 | //! pointers to these functions even if they never get called anywhere. This can |
124 | //! be seen as a special case of taking a function reference. | |
125 | //! | |
7453a54e SL |
126 | //! |
127 | //! Interaction with Cross-Crate Inlining | |
128 | //! ------------------------------------- | |
129 | //! The binary of a crate will not only contain machine code for the items | |
130 | //! defined in the source code of that crate. It will also contain monomorphic | |
131 | //! instantiations of any extern generic functions and of functions marked with | |
ff7c6d11 XL |
132 | //! `#[inline]`. |
133 | //! The collection algorithm handles this more or less mono. If it is | |
134 | //! about to create a mono item for something with an external `DefId`, | |
7453a54e | 135 | //! it will take a look if the MIR for that item is available, and if so just |
9e0c209e | 136 | //! proceed normally. If the MIR is not available, it assumes that the item is |
7453a54e SL |
137 | //! just linked to and no node is created; which is exactly what we want, since |
138 | //! no machine code should be generated in the current crate for such an item. | |
139 | //! | |
140 | //! Eager and Lazy Collection Mode | |
141 | //! ------------------------------ | |
ff7c6d11 | 142 | //! Mono item collection can be performed in one of two modes: |
7453a54e SL |
143 | //! |
144 | //! - Lazy mode means that items will only be instantiated when actually | |
fe692bf9 | 145 | //! used. The goal is to produce the least amount of machine code |
7453a54e SL |
146 | //! possible. |
147 | //! | |
148 | //! - Eager mode is meant to be used in conjunction with incremental compilation | |
ff7c6d11 | 149 | //! where a stable set of mono items is more important than a minimal |
7453a54e | 150 | //! one. Thus, eager mode will instantiate drop-glue for every drop-able type |
f035d41b | 151 | //! in the crate, even if no drop call for that type exists (yet). It will |
7453a54e SL |
152 | //! also instantiate default implementations of trait methods, something that |
153 | //! otherwise is only done on demand. | |
154 | //! | |
155 | //! | |
156 | //! Open Issues | |
157 | //! ----------- | |
158 | //! Some things are not yet fully implemented in the current version of this | |
159 | //! module. | |
160 | //! | |
7453a54e | 161 | //! ### Const Fns |
ff7c6d11 | 162 | //! Ideally, no mono item should be generated for const fns unless there |
7453a54e | 163 | //! is a call to them that cannot be evaluated at compile time. At the moment |
ff7c6d11 | 164 | //! this is not implemented however: a mono item will be produced |
7453a54e SL |
165 | //! regardless of whether it is actually needed or not. |
166 | ||
dfeec247 | 167 | use rustc_data_structures::fx::{FxHashMap, FxHashSet}; |
353b0b11 | 168 | use rustc_data_structures::sync::{par_for_each_in, MTLock, MTLockRef}; |
dfeec247 | 169 | use rustc_hir as hir; |
04454e1e FG |
170 | use rustc_hir::def::DefKind; |
171 | use rustc_hir::def_id::{DefId, DefIdMap, LocalDefId}; | |
3dfed10e | 172 | use rustc_hir::lang_items::LangItem; |
ba9703b0 XL |
173 | use rustc_middle::mir::interpret::{AllocId, ConstValue}; |
174 | use rustc_middle::mir::interpret::{ErrorHandled, GlobalAlloc, Scalar}; | |
175 | use rustc_middle::mir::mono::{InstantiationMode, MonoItem}; | |
176 | use rustc_middle::mir::visit::Visitor as MirVisitor; | |
177 | use rustc_middle::mir::{self, Local, Location}; | |
49aad941 | 178 | use rustc_middle::query::TyCtxtAt; |
fe692bf9 | 179 | use rustc_middle::ty::adjustment::{CustomCoerceUnsized, PointerCoercion}; |
17df50a5 | 180 | use rustc_middle::ty::print::with_no_trimmed_paths; |
064997fb | 181 | use rustc_middle::ty::{ |
353b0b11 FG |
182 | self, GenericParamDefKind, Instance, InstanceDef, Ty, TyCtxt, TypeFoldable, TypeVisitableExt, |
183 | VtblEntry, | |
064997fb | 184 | }; |
add651ee | 185 | use rustc_middle::ty::{GenericArgKind, GenericArgs}; |
cdc7bbd5 | 186 | use rustc_middle::{middle::codegen_fn_attrs::CodegenFnAttrFlags, mir::visit::TyContext}; |
ba9703b0 | 187 | use rustc_session::config::EntryFnType; |
cdc7bbd5 | 188 | use rustc_session::lint::builtin::LARGE_ASSIGNMENTS; |
136023e0 | 189 | use rustc_session::Limit; |
f035d41b | 190 | use rustc_span::source_map::{dummy_spanned, respan, Span, Spanned, DUMMY_SP}; |
cdc7bbd5 | 191 | use rustc_target::abi::Size; |
1b1a35ee | 192 | use std::path::PathBuf; |
532ac7d7 | 193 | |
9ffffee4 FG |
194 | use crate::errors::{ |
195 | EncounteredErrorWhileInstantiating, LargeAssignmentsLint, RecursionLimit, TypeLengthLimit, | |
196 | }; | |
f2b60f7d | 197 | |
e74abb32 | 198 | #[derive(PartialEq)] |
ff7c6d11 | 199 | pub enum MonoItemCollectionMode { |
7453a54e | 200 | Eager, |
dfeec247 | 201 | Lazy, |
7453a54e SL |
202 | } |
203 | ||
fe692bf9 FG |
204 | pub struct UsageMap<'tcx> { |
205 | // Maps every mono item to the mono items used by it. | |
206 | used_map: FxHashMap<MonoItem<'tcx>, Vec<MonoItem<'tcx>>>, | |
923072b8 | 207 | |
fe692bf9 FG |
208 | // Maps every mono item to the mono items that use it. |
209 | user_map: FxHashMap<MonoItem<'tcx>, Vec<MonoItem<'tcx>>>, | |
923072b8 FG |
210 | } |
211 | ||
fe692bf9 | 212 | type MonoItems<'tcx> = Vec<Spanned<MonoItem<'tcx>>>; |
923072b8 | 213 | |
fe692bf9 FG |
214 | impl<'tcx> UsageMap<'tcx> { |
215 | fn new() -> UsageMap<'tcx> { | |
216 | UsageMap { used_map: FxHashMap::default(), user_map: FxHashMap::default() } | |
a7813a04 XL |
217 | } |
218 | ||
fe692bf9 | 219 | fn record_used<'a>( |
923072b8 | 220 | &mut self, |
fe692bf9 FG |
221 | user_item: MonoItem<'tcx>, |
222 | used_items: &'a [Spanned<MonoItem<'tcx>>], | |
923072b8 FG |
223 | ) where |
224 | 'tcx: 'a, | |
225 | { | |
fe692bf9 FG |
226 | let used_items: Vec<_> = used_items.iter().map(|item| item.node).collect(); |
227 | for &used_item in used_items.iter() { | |
228 | self.user_map.entry(used_item).or_default().push(user_item); | |
3b2f2976 XL |
229 | } |
230 | ||
fe692bf9 | 231 | assert!(self.used_map.insert(user_item, used_items).is_none()); |
a7813a04 XL |
232 | } |
233 | ||
fe692bf9 FG |
234 | pub fn get_user_items(&self, item: MonoItem<'tcx>) -> &[MonoItem<'tcx>] { |
235 | self.user_map.get(&item).map(|items| items.as_slice()).unwrap_or(&[]) | |
7453a54e | 236 | } |
3b2f2976 | 237 | |
fe692bf9 FG |
238 | /// Internally iterate over all inlined items used by `item`. |
239 | pub fn for_each_inlined_used_item<F>(&self, tcx: TyCtxt<'tcx>, item: MonoItem<'tcx>, mut f: F) | |
dfeec247 | 240 | where |
fe692bf9 | 241 | F: FnMut(MonoItem<'tcx>), |
3b2f2976 | 242 | { |
fe692bf9 FG |
243 | let used_items = self.used_map.get(&item).unwrap(); |
244 | for used_item in used_items.iter() { | |
245 | let is_inlined = used_item.instantiation_mode(tcx) == InstantiationMode::LocalCopy; | |
246 | if is_inlined { | |
247 | f(*used_item); | |
248 | } | |
3b2f2976 XL |
249 | } |
250 | } | |
7453a54e SL |
251 | } |
252 | ||
923072b8 | 253 | #[instrument(skip(tcx, mode), level = "debug")] |
416331ca XL |
254 | pub fn collect_crate_mono_items( |
255 | tcx: TyCtxt<'_>, | |
dc9dc135 | 256 | mode: MonoItemCollectionMode, |
fe692bf9 | 257 | ) -> (FxHashSet<MonoItem<'_>>, UsageMap<'_>) { |
e74abb32 XL |
258 | let _prof_timer = tcx.prof.generic_activity("monomorphization_collector"); |
259 | ||
dfeec247 XL |
260 | let roots = |
261 | tcx.sess.time("monomorphization_collector_root_collections", || collect_roots(tcx, mode)); | |
ea8adc8c | 262 | |
416331ca | 263 | debug!("building mono item graph, beginning at roots"); |
94b46f34 | 264 | |
0bf4aa26 | 265 | let mut visited = MTLock::new(FxHashSet::default()); |
fe692bf9 | 266 | let mut usage_map = MTLock::new(UsageMap::new()); |
136023e0 | 267 | let recursion_limit = tcx.recursion_limit(); |
94b46f34 XL |
268 | |
269 | { | |
353b0b11 | 270 | let visited: MTLockRef<'_, _> = &mut visited; |
fe692bf9 | 271 | let usage_map: MTLockRef<'_, _> = &mut usage_map; |
94b46f34 | 272 | |
dfeec247 | 273 | tcx.sess.time("monomorphization_collector_graph_walk", || { |
064997fb | 274 | par_for_each_in(roots, |root| { |
a1dfa0c6 | 275 | let mut recursion_depths = DefIdMap::default(); |
f035d41b XL |
276 | collect_items_rec( |
277 | tcx, | |
278 | dummy_spanned(root), | |
279 | visited, | |
280 | &mut recursion_depths, | |
136023e0 | 281 | recursion_limit, |
fe692bf9 | 282 | usage_map, |
f035d41b | 283 | ); |
94b46f34 XL |
284 | }); |
285 | }); | |
ea8adc8c | 286 | } |
7453a54e | 287 | |
fe692bf9 | 288 | (visited.into_inner(), usage_map.into_inner()) |
7453a54e SL |
289 | } |
290 | ||
291 | // Find all non-generic items by walking the HIR. These items serve as roots to | |
292 | // start monomorphizing from. | |
923072b8 | 293 | #[instrument(skip(tcx, mode), level = "debug")] |
416331ca XL |
294 | fn collect_roots(tcx: TyCtxt<'_>, mode: MonoItemCollectionMode) -> Vec<MonoItem<'_>> { |
295 | debug!("collecting roots"); | |
fe692bf9 | 296 | let mut roots = Vec::new(); |
7453a54e SL |
297 | |
298 | { | |
17df50a5 | 299 | let entry_fn = tcx.entry_fn(()); |
abe05a73 | 300 | |
ff7c6d11 XL |
301 | debug!("collect_roots: entry_fn = {:?}", entry_fn); |
302 | ||
04454e1e FG |
303 | let mut collector = RootCollector { tcx, mode, entry_fn, output: &mut roots }; |
304 | ||
305 | let crate_items = tcx.hir_crate_items(()); | |
306 | ||
307 | for id in crate_items.items() { | |
308 | collector.process_item(id); | |
309 | } | |
7453a54e | 310 | |
04454e1e FG |
311 | for id in crate_items.impl_items() { |
312 | collector.process_impl_item(id); | |
313 | } | |
0531ce1d | 314 | |
04454e1e | 315 | collector.push_extra_entry_roots(); |
7453a54e SL |
316 | } |
317 | ||
94b46f34 | 318 | // We can only codegen items that are instantiable - items all of |
041b39d2 | 319 | // whose predicates hold. Luckily, items that aren't instantiable |
94b46f34 | 320 | // can't actually be used, so we can just skip codegenning them. |
7453a54e | 321 | roots |
f035d41b | 322 | .into_iter() |
fe692bf9 | 323 | .filter_map(|Spanned { node: mono_item, .. }| { |
923072b8 FG |
324 | mono_item.is_instantiable(tcx).then_some(mono_item) |
325 | }) | |
f035d41b | 326 | .collect() |
7453a54e SL |
327 | } |
328 | ||
17df50a5 | 329 | /// Collect all monomorphized items reachable from `starting_point`, and emit a note diagnostic if a |
49aad941 | 330 | /// post-monomorphization error is encountered during a collection step. |
fe692bf9 | 331 | #[instrument(skip(tcx, visited, recursion_depths, recursion_limit, usage_map), level = "debug")] |
dc9dc135 XL |
332 | fn collect_items_rec<'tcx>( |
333 | tcx: TyCtxt<'tcx>, | |
fe692bf9 | 334 | starting_item: Spanned<MonoItem<'tcx>>, |
353b0b11 | 335 | visited: MTLockRef<'_, FxHashSet<MonoItem<'tcx>>>, |
dc9dc135 | 336 | recursion_depths: &mut DefIdMap<usize>, |
136023e0 | 337 | recursion_limit: Limit, |
fe692bf9 | 338 | usage_map: MTLockRef<'_, UsageMap<'tcx>>, |
dc9dc135 | 339 | ) { |
fe692bf9 | 340 | if !visited.lock_mut().insert(starting_item.node) { |
7453a54e SL |
341 | // We've been here already, no need to search again. |
342 | return; | |
343 | } | |
7453a54e | 344 | |
fe692bf9 | 345 | let mut used_items = Vec::new(); |
7453a54e SL |
346 | let recursion_depth_reset; |
347 | ||
17df50a5 XL |
348 | // Post-monomorphization errors MVP |
349 | // | |
350 | // We can encounter errors while monomorphizing an item, but we don't have a good way of | |
351 | // showing a complete stack of spans ultimately leading to collecting the erroneous one yet. | |
352 | // (It's also currently unclear exactly which diagnostics and information would be interesting | |
353 | // to report in such cases) | |
354 | // | |
355 | // This leads to suboptimal error reporting: a post-monomorphization error (PME) will be | |
356 | // shown with just a spanned piece of code causing the error, without information on where | |
357 | // it was called from. This is especially obscure if the erroneous mono item is in a | |
358 | // dependency. See for example issue #85155, where, before minimization, a PME happened two | |
359 | // crates downstream from libcore's stdarch, without a way to know which dependency was the | |
360 | // cause. | |
361 | // | |
362 | // If such an error occurs in the current crate, its span will be enough to locate the | |
363 | // source. If the cause is in another crate, the goal here is to quickly locate which mono | |
364 | // item in the current crate is ultimately responsible for causing the error. | |
365 | // | |
366 | // To give at least _some_ context to the user: while collecting mono items, we check the | |
367 | // error count. If it has changed, a PME occurred, and we trigger some diagnostics about the | |
368 | // current step of mono items collection. | |
369 | // | |
04454e1e | 370 | // FIXME: don't rely on global state, instead bubble up errors. Note: this is very hard to do. |
17df50a5 XL |
371 | let error_count = tcx.sess.diagnostic().err_count(); |
372 | ||
fe692bf9 | 373 | match starting_item.node { |
0531ce1d | 374 | MonoItem::Static(def_id) => { |
ea8adc8c | 375 | let instance = Instance::mono(tcx, def_id); |
32a655c1 SL |
376 | |
377 | // Sanity check whether this ended up being collected accidentally | |
3dfed10e | 378 | debug_assert!(should_codegen_locally(tcx, &instance)); |
32a655c1 | 379 | |
3dfed10e | 380 | let ty = instance.ty(tcx, ty::ParamEnv::reveal_all()); |
fe692bf9 | 381 | visit_drop_use(tcx, ty, true, starting_item.span, &mut used_items); |
a7813a04 | 382 | |
7453a54e | 383 | recursion_depth_reset = None; |
a7813a04 | 384 | |
1b1a35ee | 385 | if let Ok(alloc) = tcx.eval_static_initializer(def_id) { |
487cf647 | 386 | for &id in alloc.inner().provenance().ptrs().values() { |
add651ee | 387 | collect_alloc(tcx, id, &mut used_items); |
1b1a35ee | 388 | } |
0531ce1d | 389 | } |
353b0b11 FG |
390 | |
391 | if tcx.needs_thread_local_shim(def_id) { | |
fe692bf9 FG |
392 | used_items.push(respan( |
393 | starting_item.span, | |
353b0b11 FG |
394 | MonoItem::Fn(Instance { |
395 | def: InstanceDef::ThreadLocalShim(def_id), | |
add651ee | 396 | args: GenericArgs::empty(), |
353b0b11 FG |
397 | }), |
398 | )); | |
399 | } | |
7453a54e | 400 | } |
ff7c6d11 | 401 | MonoItem::Fn(instance) => { |
32a655c1 | 402 | // Sanity check whether this ended up being collected accidentally |
3dfed10e | 403 | debug_assert!(should_codegen_locally(tcx, &instance)); |
32a655c1 | 404 | |
7453a54e | 405 | // Keep track of the monomorphization recursion depth |
136023e0 XL |
406 | recursion_depth_reset = Some(check_recursion_limit( |
407 | tcx, | |
408 | instance, | |
fe692bf9 | 409 | starting_item.span, |
136023e0 XL |
410 | recursion_depths, |
411 | recursion_limit, | |
412 | )); | |
ea8adc8c | 413 | check_type_length_limit(tcx, instance); |
7453a54e | 414 | |
f9f354fc | 415 | rustc_data_structures::stack::ensure_sufficient_stack(|| { |
fe692bf9 | 416 | collect_used_items(tcx, instance, &mut used_items); |
f9f354fc | 417 | }); |
7453a54e | 418 | } |
17df50a5 | 419 | MonoItem::GlobalAsm(item_id) => { |
cc61c64b | 420 | recursion_depth_reset = None; |
17df50a5 XL |
421 | |
422 | let item = tcx.hir().item(item_id); | |
423 | if let hir::ItemKind::GlobalAsm(asm) = item.kind { | |
424 | for (op, op_sp) in asm.operands { | |
425 | match op { | |
426 | hir::InlineAsmOperand::Const { .. } => { | |
427 | // Only constants which resolve to a plain integer | |
428 | // are supported. Therefore the value should not | |
429 | // depend on any other items. | |
430 | } | |
04454e1e FG |
431 | hir::InlineAsmOperand::SymFn { anon_const } => { |
432 | let fn_ty = | |
433 | tcx.typeck_body(anon_const.body).node_type(anon_const.hir_id); | |
fe692bf9 | 434 | visit_fn_use(tcx, fn_ty, false, *op_sp, &mut used_items); |
04454e1e FG |
435 | } |
436 | hir::InlineAsmOperand::SymStatic { path: _, def_id } => { | |
437 | let instance = Instance::mono(tcx, *def_id); | |
438 | if should_codegen_locally(tcx, &instance) { | |
439 | trace!("collecting static {:?}", def_id); | |
fe692bf9 | 440 | used_items.push(dummy_spanned(MonoItem::Static(*def_id))); |
04454e1e FG |
441 | } |
442 | } | |
443 | hir::InlineAsmOperand::In { .. } | |
444 | | hir::InlineAsmOperand::Out { .. } | |
445 | | hir::InlineAsmOperand::InOut { .. } | |
446 | | hir::InlineAsmOperand::SplitInOut { .. } => { | |
447 | span_bug!(*op_sp, "invalid operand type for global_asm!") | |
448 | } | |
17df50a5 XL |
449 | } |
450 | } | |
451 | } else { | |
452 | span_bug!(item.span, "Mismatch between hir::Item type and MonoItem type") | |
453 | } | |
cc61c64b | 454 | } |
7453a54e SL |
455 | } |
456 | ||
17df50a5 | 457 | // Check for PMEs and emit a diagnostic if one happened. To try to show relevant edges of the |
04454e1e | 458 | // mono item graph. |
c295e0f8 | 459 | if tcx.sess.diagnostic().err_count() > error_count |
fe692bf9 FG |
460 | && starting_item.node.is_generic_fn() |
461 | && starting_item.node.is_user_defined() | |
17df50a5 | 462 | { |
fe692bf9 | 463 | let formatted_item = with_no_trimmed_paths!(starting_item.node.to_string()); |
9ffffee4 | 464 | tcx.sess.emit_note(EncounteredErrorWhileInstantiating { |
fe692bf9 | 465 | span: starting_item.span, |
9ffffee4 FG |
466 | formatted_item, |
467 | }); | |
17df50a5 | 468 | } |
fe692bf9 | 469 | usage_map.lock_mut().record_used(starting_item.node, &used_items); |
17df50a5 | 470 | |
fe692bf9 FG |
471 | for used_item in used_items { |
472 | collect_items_rec(tcx, used_item, visited, recursion_depths, recursion_limit, usage_map); | |
7453a54e SL |
473 | } |
474 | ||
475 | if let Some((def_id, depth)) = recursion_depth_reset { | |
476 | recursion_depths.insert(def_id, depth); | |
477 | } | |
7453a54e SL |
478 | } |
479 | ||
1b1a35ee | 480 | /// Format instance name that is already known to be too long for rustc. |
487cf647 | 481 | /// Show only the first 2 types if it is longer than 32 characters to avoid blasting |
1b1a35ee XL |
482 | /// the user's terminal with thousands of lines of type-name. |
483 | /// | |
484 | /// If the type name is longer than before+after, it will be written to a file. | |
a2a8927a | 485 | fn shrunk_instance_name<'tcx>( |
1b1a35ee XL |
486 | tcx: TyCtxt<'tcx>, |
487 | instance: &Instance<'tcx>, | |
1b1a35ee XL |
488 | ) -> (String, Option<PathBuf>) { |
489 | let s = instance.to_string(); | |
6c58768f XL |
490 | |
491 | // Only use the shrunk version if it's really shorter. | |
492 | // This also avoids the case where before and after slices overlap. | |
487cf647 FG |
493 | if s.chars().nth(33).is_some() { |
494 | let shrunk = format!("{}", ty::ShortInstance(instance, 4)); | |
495 | if shrunk == s { | |
496 | return (s, None); | |
497 | } | |
6c58768f | 498 | |
17df50a5 | 499 | let path = tcx.output_filenames(()).temp_path_ext("long-type.txt", None); |
1b1a35ee XL |
500 | let written_to_path = std::fs::write(&path, s).ok().map(|_| path); |
501 | ||
502 | (shrunk, written_to_path) | |
503 | } else { | |
504 | (s, None) | |
505 | } | |
6c58768f XL |
506 | } |
507 | ||
dc9dc135 XL |
508 | fn check_recursion_limit<'tcx>( |
509 | tcx: TyCtxt<'tcx>, | |
510 | instance: Instance<'tcx>, | |
f035d41b | 511 | span: Span, |
dc9dc135 | 512 | recursion_depths: &mut DefIdMap<usize>, |
136023e0 | 513 | recursion_limit: Limit, |
dc9dc135 | 514 | ) -> (DefId, usize) { |
cc61c64b XL |
515 | let def_id = instance.def_id(); |
516 | let recursion_depth = recursion_depths.get(&def_id).cloned().unwrap_or(0); | |
7453a54e SL |
517 | debug!(" => recursion depth={}", recursion_depth); |
518 | ||
dfeec247 | 519 | let adjusted_recursion_depth = if Some(def_id) == tcx.lang_items().drop_in_place_fn() { |
cc61c64b XL |
520 | // HACK: drop_in_place creates tight monomorphization loops. Give |
521 | // it more margin. | |
522 | recursion_depth / 4 | |
523 | } else { | |
524 | recursion_depth | |
525 | }; | |
526 | ||
7453a54e SL |
527 | // Code that needs to instantiate the same function recursively |
528 | // more than the recursion limit is assumed to be causing an | |
529 | // infinite expansion. | |
136023e0 | 530 | if !recursion_limit.value_within_limit(adjusted_recursion_depth) { |
f2b60f7d FG |
531 | let def_span = tcx.def_span(def_id); |
532 | let def_path_str = tcx.def_path_str(def_id); | |
487cf647 | 533 | let (shrunk, written_to_path) = shrunk_instance_name(tcx, &instance); |
f2b60f7d | 534 | let mut path = PathBuf::new(); |
9c376795 FG |
535 | let was_written = if let Some(written_to_path) = written_to_path { |
536 | path = written_to_path; | |
f2b60f7d FG |
537 | Some(()) |
538 | } else { | |
539 | None | |
540 | }; | |
541 | tcx.sess.emit_fatal(RecursionLimit { | |
542 | span, | |
543 | shrunk, | |
544 | def_span, | |
545 | def_path_str, | |
546 | was_written, | |
547 | path, | |
548 | }); | |
7453a54e SL |
549 | } |
550 | ||
cc61c64b | 551 | recursion_depths.insert(def_id, recursion_depth + 1); |
7453a54e | 552 | |
cc61c64b | 553 | (def_id, recursion_depth) |
7453a54e SL |
554 | } |
555 | ||
dc9dc135 | 556 | fn check_type_length_limit<'tcx>(tcx: TyCtxt<'tcx>, instance: Instance<'tcx>) { |
ba9703b0 | 557 | let type_length = instance |
add651ee | 558 | .args |
ba9703b0 | 559 | .iter() |
5099ac24 | 560 | .flat_map(|arg| arg.walk()) |
ba9703b0 XL |
561 | .filter(|arg| match arg.unpack() { |
562 | GenericArgKind::Type(_) | GenericArgKind::Const(_) => true, | |
563 | GenericArgKind::Lifetime(_) => false, | |
564 | }) | |
565 | .count(); | |
566 | debug!(" => type length={}", type_length); | |
476ff2be SL |
567 | |
568 | // Rust code can easily create exponentially-long types using only a | |
569 | // polynomial recursion depth. Even with the default recursion | |
570 | // depth, you can easily get cases that take >2^60 steps to run, | |
571 | // which means that rustc basically hangs. | |
572 | // | |
573 | // Bail out in these cases to avoid that bad user experience. | |
136023e0 | 574 | if !tcx.type_length_limit().value_within_limit(type_length) { |
487cf647 | 575 | let (shrunk, written_to_path) = shrunk_instance_name(tcx, &instance); |
f2b60f7d FG |
576 | let span = tcx.def_span(instance.def_id()); |
577 | let mut path = PathBuf::new(); | |
353b0b11 FG |
578 | let was_written = if let Some(path2) = written_to_path { |
579 | path = path2; | |
f2b60f7d FG |
580 | Some(()) |
581 | } else { | |
582 | None | |
583 | }; | |
584 | tcx.sess.emit_fatal(TypeLengthLimit { span, shrunk, was_written, path, type_length }); | |
476ff2be SL |
585 | } |
586 | } | |
587 | ||
fe692bf9 | 588 | struct MirUsedCollector<'a, 'tcx> { |
dc9dc135 XL |
589 | tcx: TyCtxt<'tcx>, |
590 | body: &'a mir::Body<'tcx>, | |
923072b8 | 591 | output: &'a mut MonoItems<'tcx>, |
ba9703b0 XL |
592 | instance: Instance<'tcx>, |
593 | } | |
594 | ||
fe692bf9 | 595 | impl<'a, 'tcx> MirUsedCollector<'a, 'tcx> { |
ba9703b0 XL |
596 | pub fn monomorphize<T>(&self, value: T) -> T |
597 | where | |
9ffffee4 | 598 | T: TypeFoldable<TyCtxt<'tcx>>, |
ba9703b0 XL |
599 | { |
600 | debug!("monomorphize: self.instance={:?}", self.instance); | |
29967ef6 XL |
601 | self.instance.subst_mir_and_normalize_erasing_regions( |
602 | self.tcx, | |
603 | ty::ParamEnv::reveal_all(), | |
fe692bf9 | 604 | ty::EarlyBinder::bind(value), |
29967ef6 | 605 | ) |
ba9703b0 | 606 | } |
7453a54e SL |
607 | } |
608 | ||
fe692bf9 | 609 | impl<'a, 'tcx> MirVisitor<'tcx> for MirUsedCollector<'a, 'tcx> { |
9e0c209e | 610 | fn visit_rvalue(&mut self, rvalue: &mir::Rvalue<'tcx>, location: Location) { |
7453a54e SL |
611 | debug!("visiting rvalue {:?}", *rvalue); |
612 | ||
f035d41b XL |
613 | let span = self.body.source_info(location).span; |
614 | ||
7453a54e | 615 | match *rvalue { |
7453a54e SL |
616 | // When doing an cast from a regular pointer to a fat pointer, we |
617 | // have to instantiate all methods of the trait being cast to, so we | |
618 | // can build the appropriate vtable. | |
48663c56 | 619 | mir::Rvalue::Cast( |
fe692bf9 | 620 | mir::CastKind::PointerCoercion(PointerCoercion::Unsize), |
dfeec247 XL |
621 | ref operand, |
622 | target_ty, | |
f2b60f7d FG |
623 | ) |
624 | | mir::Rvalue::Cast(mir::CastKind::DynStar, ref operand, target_ty) => { | |
ba9703b0 | 625 | let target_ty = self.monomorphize(target_ty); |
dc9dc135 | 626 | let source_ty = operand.ty(self.body, self.tcx); |
ba9703b0 | 627 | let source_ty = self.monomorphize(source_ty); |
dfeec247 | 628 | let (source_ty, target_ty) = |
487cf647 | 629 | find_vtable_types_for_unsizing(self.tcx.at(span), source_ty, target_ty); |
7453a54e SL |
630 | // This could also be a different Unsize instruction, like |
631 | // from a fixed sized array to a slice. But we are only | |
632 | // interested in things that produce a vtable. | |
f2b60f7d FG |
633 | if (target_ty.is_trait() && !source_ty.is_trait()) |
634 | || (target_ty.is_dyn_star() && !source_ty.is_dyn_star()) | |
635 | { | |
dfeec247 XL |
636 | create_mono_items_for_vtable_methods( |
637 | self.tcx, | |
638 | target_ty, | |
639 | source_ty, | |
f035d41b | 640 | span, |
dfeec247 XL |
641 | self.output, |
642 | ); | |
7453a54e SL |
643 | } |
644 | } | |
48663c56 | 645 | mir::Rvalue::Cast( |
fe692bf9 | 646 | mir::CastKind::PointerCoercion(PointerCoercion::ReifyFnPointer), |
dfeec247 XL |
647 | ref operand, |
648 | _, | |
48663c56 | 649 | ) => { |
dc9dc135 | 650 | let fn_ty = operand.ty(self.body, self.tcx); |
ba9703b0 | 651 | let fn_ty = self.monomorphize(fn_ty); |
f035d41b | 652 | visit_fn_use(self.tcx, fn_ty, false, span, &mut self.output); |
cc61c64b | 653 | } |
48663c56 | 654 | mir::Rvalue::Cast( |
fe692bf9 | 655 | mir::CastKind::PointerCoercion(PointerCoercion::ClosureFnPointer(_)), |
dfeec247 XL |
656 | ref operand, |
657 | _, | |
48663c56 | 658 | ) => { |
dc9dc135 | 659 | let source_ty = operand.ty(self.body, self.tcx); |
ba9703b0 | 660 | let source_ty = self.monomorphize(source_ty); |
1b1a35ee | 661 | match *source_ty.kind() { |
add651ee | 662 | ty::Closure(def_id, args) => { |
dc9dc135 | 663 | let instance = Instance::resolve_closure( |
dfeec247 XL |
664 | self.tcx, |
665 | def_id, | |
add651ee | 666 | args, |
dfeec247 | 667 | ty::ClosureKind::FnOnce, |
064997fb FG |
668 | ) |
669 | .expect("failed to normalize and resolve closure during codegen"); | |
3dfed10e XL |
670 | if should_codegen_locally(self.tcx, &instance) { |
671 | self.output.push(create_fn_mono_item(self.tcx, instance, span)); | |
83c7162d | 672 | } |
8bb4bdeb XL |
673 | } |
674 | _ => bug!(), | |
675 | } | |
676 | } | |
f9f354fc XL |
677 | mir::Rvalue::ThreadLocalRef(def_id) => { |
678 | assert!(self.tcx.is_thread_local_static(def_id)); | |
679 | let instance = Instance::mono(self.tcx, def_id); | |
3dfed10e | 680 | if should_codegen_locally(self.tcx, &instance) { |
f9f354fc | 681 | trace!("collecting thread-local static {:?}", def_id); |
f035d41b | 682 | self.output.push(respan(span, MonoItem::Static(def_id))); |
f9f354fc XL |
683 | } |
684 | } | |
7453a54e SL |
685 | _ => { /* not interesting */ } |
686 | } | |
687 | ||
9e0c209e | 688 | self.super_rvalue(rvalue, location); |
7453a54e SL |
689 | } |
690 | ||
cdc7bbd5 XL |
691 | /// This does not walk the constant, as it has been handled entirely here and trying |
692 | /// to walk it would attempt to evaluate the `ty::Const` inside, which doesn't necessarily | |
693 | /// work, as some constants cannot be represented in the type system. | |
923072b8 | 694 | #[instrument(skip(self), level = "debug")] |
cdc7bbd5 XL |
695 | fn visit_constant(&mut self, constant: &mir::Constant<'tcx>, location: Location) { |
696 | let literal = self.monomorphize(constant.literal); | |
697 | let val = match literal { | |
698 | mir::ConstantKind::Val(val, _) => val, | |
923072b8 FG |
699 | mir::ConstantKind::Ty(ct) => match ct.kind() { |
700 | ty::ConstKind::Value(val) => self.tcx.valtree_to_const_val((ct.ty(), val)), | |
cdc7bbd5 | 701 | ty::ConstKind::Unevaluated(ct) => { |
923072b8 | 702 | debug!(?ct); |
cdc7bbd5 | 703 | let param_env = ty::ParamEnv::reveal_all(); |
f2b60f7d | 704 | match self.tcx.const_eval_resolve(param_env, ct.expand(), None) { |
cdc7bbd5 XL |
705 | // The `monomorphize` call should have evaluated that constant already. |
706 | Ok(val) => val, | |
487cf647 | 707 | Err(ErrorHandled::Reported(_)) => return, |
cdc7bbd5 XL |
708 | Err(ErrorHandled::TooGeneric) => span_bug!( |
709 | self.body.source_info(location).span, | |
710 | "collection encountered polymorphic constant: {:?}", | |
711 | literal | |
712 | ), | |
713 | } | |
714 | } | |
715 | _ => return, | |
716 | }, | |
f2b60f7d FG |
717 | mir::ConstantKind::Unevaluated(uv, _) => { |
718 | let param_env = ty::ParamEnv::reveal_all(); | |
719 | match self.tcx.const_eval_resolve(param_env, uv, None) { | |
cdc7bbd5 | 720 | // The `monomorphize` call should have evaluated that constant already. |
f2b60f7d | 721 | Ok(val) => val, |
487cf647 | 722 | Err(ErrorHandled::Reported(_)) => return, |
ba9703b0 | 723 | Err(ErrorHandled::TooGeneric) => span_bug!( |
3dfed10e | 724 | self.body.source_info(location).span, |
f2b60f7d FG |
725 | "collection encountered polymorphic constant: {:?}", |
726 | literal | |
ba9703b0 XL |
727 | ), |
728 | } | |
729 | } | |
f2b60f7d FG |
730 | }; |
731 | collect_const_value(self.tcx, val, self.output); | |
732 | MirVisitor::visit_ty(self, literal.ty(), TyContext::Location(location)); | |
a7813a04 XL |
733 | } |
734 | ||
f035d41b XL |
735 | fn visit_terminator(&mut self, terminator: &mir::Terminator<'tcx>, location: Location) { |
736 | debug!("visiting terminator {:?} @ {:?}", terminator, location); | |
737 | let source = self.body.source_info(location).span; | |
7cac9316 | 738 | |
ea8adc8c | 739 | let tcx = self.tcx; |
f035d41b | 740 | match terminator.kind { |
cc61c64b | 741 | mir::TerminatorKind::Call { ref func, .. } => { |
dc9dc135 | 742 | let callee_ty = func.ty(self.body, tcx); |
ba9703b0 | 743 | let callee_ty = self.monomorphize(callee_ty); |
f2b60f7d | 744 | visit_fn_use(self.tcx, callee_ty, true, source, &mut self.output) |
a7813a04 | 745 | } |
353b0b11 | 746 | mir::TerminatorKind::Drop { ref place, .. } => { |
f035d41b | 747 | let ty = place.ty(self.body, self.tcx).ty; |
ba9703b0 | 748 | let ty = self.monomorphize(ty); |
f035d41b | 749 | visit_drop_use(self.tcx, ty, true, source, self.output); |
cc61c64b | 750 | } |
f9f354fc XL |
751 | mir::TerminatorKind::InlineAsm { ref operands, .. } => { |
752 | for op in operands { | |
f035d41b XL |
753 | match *op { |
754 | mir::InlineAsmOperand::SymFn { ref value } => { | |
6a06907d | 755 | let fn_ty = self.monomorphize(value.literal.ty()); |
f035d41b XL |
756 | visit_fn_use(self.tcx, fn_ty, false, source, &mut self.output); |
757 | } | |
758 | mir::InlineAsmOperand::SymStatic { def_id } => { | |
759 | let instance = Instance::mono(self.tcx, def_id); | |
3dfed10e | 760 | if should_codegen_locally(self.tcx, &instance) { |
f035d41b XL |
761 | trace!("collecting asm sym static {:?}", def_id); |
762 | self.output.push(respan(source, MonoItem::Static(def_id))); | |
763 | } | |
764 | } | |
765 | _ => {} | |
f9f354fc XL |
766 | } |
767 | } | |
768 | } | |
3c0e092e | 769 | mir::TerminatorKind::Assert { ref msg, .. } => { |
49aad941 | 770 | let lang_item = match &**msg { |
3c0e092e XL |
771 | mir::AssertKind::BoundsCheck { .. } => LangItem::PanicBoundsCheck, |
772 | _ => LangItem::Panic, | |
773 | }; | |
774 | let instance = Instance::mono(tcx, tcx.require_lang_item(lang_item, Some(source))); | |
775 | if should_codegen_locally(tcx, &instance) { | |
776 | self.output.push(create_fn_mono_item(tcx, instance, source)); | |
777 | } | |
778 | } | |
353b0b11 | 779 | mir::TerminatorKind::Terminate { .. } => { |
5099ac24 FG |
780 | let instance = Instance::mono( |
781 | tcx, | |
9c376795 | 782 | tcx.require_lang_item(LangItem::PanicCannotUnwind, Some(source)), |
5099ac24 FG |
783 | ); |
784 | if should_codegen_locally(tcx, &instance) { | |
785 | self.output.push(create_fn_mono_item(tcx, instance, source)); | |
786 | } | |
787 | } | |
dfeec247 XL |
788 | mir::TerminatorKind::Goto { .. } |
789 | | mir::TerminatorKind::SwitchInt { .. } | |
790 | | mir::TerminatorKind::Resume | |
dfeec247 | 791 | | mir::TerminatorKind::Return |
3c0e092e | 792 | | mir::TerminatorKind::Unreachable => {} |
dfeec247 XL |
793 | mir::TerminatorKind::GeneratorDrop |
794 | | mir::TerminatorKind::Yield { .. } | |
f035d41b | 795 | | mir::TerminatorKind::FalseEdge { .. } |
dfeec247 | 796 | | mir::TerminatorKind::FalseUnwind { .. } => bug!(), |
a7813a04 XL |
797 | } |
798 | ||
353b0b11 FG |
799 | if let Some(mir::UnwindAction::Terminate) = terminator.unwind() { |
800 | let instance = Instance::mono( | |
801 | tcx, | |
802 | tcx.require_lang_item(LangItem::PanicCannotUnwind, Some(source)), | |
803 | ); | |
804 | if should_codegen_locally(tcx, &instance) { | |
805 | self.output.push(create_fn_mono_item(tcx, instance, source)); | |
806 | } | |
807 | } | |
808 | ||
f035d41b | 809 | self.super_terminator(terminator, location); |
7453a54e | 810 | } |
3b2f2976 | 811 | |
cdc7bbd5 XL |
812 | fn visit_operand(&mut self, operand: &mir::Operand<'tcx>, location: Location) { |
813 | self.super_operand(operand, location); | |
136023e0 | 814 | let limit = self.tcx.move_size_limit().0; |
cdc7bbd5 XL |
815 | if limit == 0 { |
816 | return; | |
817 | } | |
818 | let limit = Size::from_bytes(limit); | |
819 | let ty = operand.ty(self.body, self.tcx); | |
820 | let ty = self.monomorphize(ty); | |
821 | let layout = self.tcx.layout_of(ty::ParamEnv::reveal_all().and(ty)); | |
822 | if let Ok(layout) = layout { | |
823 | if layout.size > limit { | |
824 | debug!(?layout); | |
825 | let source_info = self.body.source_info(location); | |
826 | debug!(?source_info); | |
827 | let lint_root = source_info.scope.lint_root(&self.body.source_scopes); | |
828 | debug!(?lint_root); | |
5e7ed085 | 829 | let Some(lint_root) = lint_root else { |
cdc7bbd5 XL |
830 | // This happens when the issue is in a function from a foreign crate that |
831 | // we monomorphized in the current crate. We can't get a `HirId` for things | |
832 | // in other crates. | |
833 | // FIXME: Find out where to report the lint on. Maybe simply crate-level lint root | |
834 | // but correct span? This would make the lint at least accept crate-level lint attributes. | |
5e7ed085 | 835 | return; |
cdc7bbd5 | 836 | }; |
f2b60f7d | 837 | self.tcx.emit_spanned_lint( |
cdc7bbd5 XL |
838 | LARGE_ASSIGNMENTS, |
839 | lint_root, | |
840 | source_info.span, | |
f2b60f7d FG |
841 | LargeAssignmentsLint { |
842 | span: source_info.span, | |
843 | size: layout.size.bytes(), | |
844 | limit: limit.bytes(), | |
cdc7bbd5 | 845 | }, |
f2b60f7d | 846 | ) |
cdc7bbd5 XL |
847 | } |
848 | } | |
849 | } | |
850 | ||
f9f354fc | 851 | fn visit_local( |
dfeec247 | 852 | &mut self, |
064997fb | 853 | _place_local: Local, |
dfeec247 XL |
854 | _context: mir::visit::PlaceContext, |
855 | _location: Location, | |
856 | ) { | |
3b2f2976 | 857 | } |
7453a54e SL |
858 | } |
859 | ||
dc9dc135 XL |
860 | fn visit_drop_use<'tcx>( |
861 | tcx: TyCtxt<'tcx>, | |
862 | ty: Ty<'tcx>, | |
863 | is_direct_call: bool, | |
f035d41b | 864 | source: Span, |
923072b8 | 865 | output: &mut MonoItems<'tcx>, |
dc9dc135 XL |
866 | ) { |
867 | let instance = Instance::resolve_drop_in_place(tcx, ty); | |
f035d41b | 868 | visit_instance_use(tcx, instance, is_direct_call, source, output); |
7453a54e SL |
869 | } |
870 | ||
dc9dc135 XL |
871 | fn visit_fn_use<'tcx>( |
872 | tcx: TyCtxt<'tcx>, | |
873 | ty: Ty<'tcx>, | |
874 | is_direct_call: bool, | |
f035d41b | 875 | source: Span, |
923072b8 | 876 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 877 | ) { |
add651ee | 878 | if let ty::FnDef(def_id, args) = *ty.kind() { |
f9f354fc | 879 | let instance = if is_direct_call { |
add651ee | 880 | ty::Instance::expect_resolve(tcx, ty::ParamEnv::reveal_all(), def_id, args) |
f9f354fc | 881 | } else { |
add651ee | 882 | match ty::Instance::resolve_for_fn_ptr(tcx, ty::ParamEnv::reveal_all(), def_id, args) { |
9c376795 FG |
883 | Some(instance) => instance, |
884 | _ => bug!("failed to resolve instance for {ty}"), | |
885 | } | |
f9f354fc | 886 | }; |
f035d41b | 887 | visit_instance_use(tcx, instance, is_direct_call, source, output); |
7453a54e | 888 | } |
cc61c64b | 889 | } |
7453a54e | 890 | |
dc9dc135 XL |
891 | fn visit_instance_use<'tcx>( |
892 | tcx: TyCtxt<'tcx>, | |
893 | instance: ty::Instance<'tcx>, | |
894 | is_direct_call: bool, | |
f035d41b | 895 | source: Span, |
923072b8 | 896 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 897 | ) { |
cc61c64b | 898 | debug!("visit_item_use({:?}, is_direct_call={:?})", instance, is_direct_call); |
3dfed10e | 899 | if !should_codegen_locally(tcx, &instance) { |
dfeec247 | 900 | return; |
7453a54e SL |
901 | } |
902 | ||
cc61c64b | 903 | match instance.def { |
dfeec247 | 904 | ty::InstanceDef::Virtual(..) | ty::InstanceDef::Intrinsic(_) => { |
cc61c64b | 905 | if !is_direct_call { |
60c5eb7d | 906 | bug!("{:?} being reified", instance); |
32a655c1 SL |
907 | } |
908 | } | |
353b0b11 FG |
909 | ty::InstanceDef::ThreadLocalShim(..) => { |
910 | bug!("{:?} being reified", instance); | |
911 | } | |
cc61c64b | 912 | ty::InstanceDef::DropGlue(_, None) => { |
60c5eb7d | 913 | // Don't need to emit noop drop glue if we are calling directly. |
cc61c64b | 914 | if !is_direct_call { |
3dfed10e | 915 | output.push(create_fn_mono_item(tcx, instance, source)); |
7453a54e SL |
916 | } |
917 | } | |
dfeec247 | 918 | ty::InstanceDef::DropGlue(_, Some(_)) |
064997fb | 919 | | ty::InstanceDef::VTableShim(..) |
dfeec247 XL |
920 | | ty::InstanceDef::ReifyShim(..) |
921 | | ty::InstanceDef::ClosureOnceShim { .. } | |
922 | | ty::InstanceDef::Item(..) | |
923 | | ty::InstanceDef::FnPtrShim(..) | |
353b0b11 FG |
924 | | ty::InstanceDef::CloneShim(..) |
925 | | ty::InstanceDef::FnPtrAddrShim(..) => { | |
3dfed10e | 926 | output.push(create_fn_mono_item(tcx, instance, source)); |
32a655c1 | 927 | } |
7453a54e SL |
928 | } |
929 | } | |
930 | ||
c295e0f8 XL |
931 | /// Returns `true` if we should codegen an instance in the local crate, or returns `false` if we |
932 | /// can just link to the upstream crate and therefore don't need a mono item. | |
3dfed10e | 933 | fn should_codegen_locally<'tcx>(tcx: TyCtxt<'tcx>, instance: &Instance<'tcx>) -> bool { |
5099ac24 | 934 | let Some(def_id) = instance.def.def_id_if_not_guaranteed_local_codegen() else { |
c295e0f8 | 935 | return true; |
cc61c64b | 936 | }; |
83c7162d | 937 | |
a1dfa0c6 | 938 | if tcx.is_foreign_item(def_id) { |
3dfed10e | 939 | // Foreign items are always linked against, there's no way of instantiating them. |
a1dfa0c6 XL |
940 | return false; |
941 | } | |
942 | ||
943 | if def_id.is_local() { | |
3dfed10e | 944 | // Local items cannot be referred to locally without monomorphizing them locally. |
a1dfa0c6 XL |
945 | return true; |
946 | } | |
947 | ||
3dfed10e XL |
948 | if tcx.is_reachable_non_generic(def_id) |
949 | || instance.polymorphize(tcx).upstream_monomorphization(tcx).is_some() | |
950 | { | |
951 | // We can link to the item in question, no instance needed in this crate. | |
a1dfa0c6 XL |
952 | return false; |
953 | } | |
954 | ||
f2b60f7d FG |
955 | if let DefKind::Static(_) = tcx.def_kind(def_id) { |
956 | // We cannot monomorphize statics from upstream crates. | |
957 | return false; | |
958 | } | |
959 | ||
a1dfa0c6 | 960 | if !tcx.is_mir_available(def_id) { |
5869c6ff | 961 | bug!("no MIR available for {:?}", def_id); |
a1dfa0c6 | 962 | } |
83c7162d | 963 | |
ba9703b0 | 964 | true |
7453a54e SL |
965 | } |
966 | ||
60c5eb7d | 967 | /// For a given pair of source and target type that occur in an unsizing coercion, |
7453a54e SL |
968 | /// this function finds the pair of types that determines the vtable linking |
969 | /// them. | |
970 | /// | |
5e7ed085 | 971 | /// For example, the source type might be `&SomeStruct` and the target type |
f2b60f7d | 972 | /// might be `&dyn SomeTrait` in a cast like: |
7453a54e | 973 | /// |
f2b60f7d | 974 | /// ```rust,ignore (not real code) |
7453a54e | 975 | /// let src: &SomeStruct = ...; |
f2b60f7d FG |
976 | /// let target = src as &dyn SomeTrait; |
977 | /// ``` | |
7453a54e SL |
978 | /// |
979 | /// Then the output of this function would be (SomeStruct, SomeTrait) since for | |
980 | /// constructing the `target` fat-pointer we need the vtable for that pair. | |
981 | /// | |
982 | /// Things can get more complicated though because there's also the case where | |
983 | /// the unsized type occurs as a field: | |
984 | /// | |
985 | /// ```rust | |
986 | /// struct ComplexStruct<T: ?Sized> { | |
987 | /// a: u32, | |
988 | /// b: f64, | |
989 | /// c: T | |
990 | /// } | |
991 | /// ``` | |
992 | /// | |
993 | /// In this case, if `T` is sized, `&ComplexStruct<T>` is a thin pointer. If `T` | |
994 | /// is unsized, `&SomeStruct` is a fat pointer, and the vtable it points to is | |
995 | /// for the pair of `T` (which is a trait) and the concrete type that `T` was | |
996 | /// originally coerced from: | |
997 | /// | |
f2b60f7d | 998 | /// ```rust,ignore (not real code) |
7453a54e | 999 | /// let src: &ComplexStruct<SomeStruct> = ...; |
f2b60f7d FG |
1000 | /// let target = src as &ComplexStruct<dyn SomeTrait>; |
1001 | /// ``` | |
7453a54e SL |
1002 | /// |
1003 | /// Again, we want this `find_vtable_types_for_unsizing()` to provide the pair | |
1004 | /// `(SomeStruct, SomeTrait)`. | |
1005 | /// | |
0731742a | 1006 | /// Finally, there is also the case of custom unsizing coercions, e.g., for |
7453a54e | 1007 | /// smart pointers such as `Rc` and `Arc`. |
dc9dc135 | 1008 | fn find_vtable_types_for_unsizing<'tcx>( |
487cf647 | 1009 | tcx: TyCtxtAt<'tcx>, |
dc9dc135 XL |
1010 | source_ty: Ty<'tcx>, |
1011 | target_ty: Ty<'tcx>, | |
1012 | ) -> (Ty<'tcx>, Ty<'tcx>) { | |
ea8adc8c | 1013 | let ptr_vtable = |inner_source: Ty<'tcx>, inner_target: Ty<'tcx>| { |
416331ca | 1014 | let param_env = ty::ParamEnv::reveal_all(); |
ff7c6d11 | 1015 | let type_has_metadata = |ty: Ty<'tcx>| -> bool { |
487cf647 | 1016 | if ty.is_sized(tcx.tcx, param_env) { |
ff7c6d11 XL |
1017 | return false; |
1018 | } | |
416331ca | 1019 | let tail = tcx.struct_tail_erasing_lifetimes(ty, param_env); |
1b1a35ee | 1020 | match tail.kind() { |
b7449926 XL |
1021 | ty::Foreign(..) => false, |
1022 | ty::Str | ty::Slice(..) | ty::Dynamic(..) => true, | |
532ac7d7 | 1023 | _ => bug!("unexpected unsized tail: {:?}", tail), |
ff7c6d11 XL |
1024 | } |
1025 | }; | |
1026 | if type_has_metadata(inner_source) { | |
32a655c1 SL |
1027 | (inner_source, inner_target) |
1028 | } else { | |
416331ca | 1029 | tcx.struct_lockstep_tails_erasing_lifetimes(inner_source, inner_target, param_env) |
32a655c1 SL |
1030 | } |
1031 | }; | |
ff7c6d11 | 1032 | |
1b1a35ee | 1033 | match (&source_ty.kind(), &target_ty.kind()) { |
ba9703b0 | 1034 | (&ty::Ref(_, a, _), &ty::Ref(_, b, _) | &ty::RawPtr(ty::TypeAndMut { ty: b, .. })) |
dfeec247 | 1035 | | (&ty::RawPtr(ty::TypeAndMut { ty: a, .. }), &ty::RawPtr(ty::TypeAndMut { ty: b, .. })) => { |
5099ac24 | 1036 | ptr_vtable(*a, *b) |
32a655c1 | 1037 | } |
b7449926 | 1038 | (&ty::Adt(def_a, _), &ty::Adt(def_b, _)) if def_a.is_box() && def_b.is_box() => { |
32a655c1 | 1039 | ptr_vtable(source_ty.boxed_ty(), target_ty.boxed_ty()) |
7453a54e SL |
1040 | } |
1041 | ||
f2b60f7d FG |
1042 | // T as dyn* Trait |
1043 | (_, &ty::Dynamic(_, _, ty::DynStar)) => ptr_vtable(source_ty, target_ty), | |
1044 | ||
add651ee | 1045 | (&ty::Adt(source_adt_def, source_args), &ty::Adt(target_adt_def, target_args)) => { |
7453a54e SL |
1046 | assert_eq!(source_adt_def, target_adt_def); |
1047 | ||
74b04a01 | 1048 | let CustomCoerceUnsized::Struct(coerce_index) = |
c295e0f8 | 1049 | crate::custom_coerce_unsize_info(tcx, source_ty, target_ty); |
7453a54e | 1050 | |
2c00a5a8 XL |
1051 | let source_fields = &source_adt_def.non_enum_variant().fields; |
1052 | let target_fields = &target_adt_def.non_enum_variant().fields; | |
7453a54e | 1053 | |
dfeec247 | 1054 | assert!( |
353b0b11 FG |
1055 | coerce_index.index() < source_fields.len() |
1056 | && source_fields.len() == target_fields.len() | |
dfeec247 | 1057 | ); |
7453a54e | 1058 | |
dfeec247 XL |
1059 | find_vtable_types_for_unsizing( |
1060 | tcx, | |
add651ee FG |
1061 | source_fields[coerce_index].ty(*tcx, source_args), |
1062 | target_fields[coerce_index].ty(*tcx, target_args), | |
60c5eb7d | 1063 | ) |
7453a54e | 1064 | } |
dfeec247 XL |
1065 | _ => bug!( |
1066 | "find_vtable_types_for_unsizing: invalid coercion {:?} -> {:?}", | |
1067 | source_ty, | |
1068 | target_ty | |
1069 | ), | |
7453a54e SL |
1070 | } |
1071 | } | |
1072 | ||
f2b60f7d | 1073 | #[instrument(skip(tcx), level = "debug", ret)] |
3dfed10e XL |
1074 | fn create_fn_mono_item<'tcx>( |
1075 | tcx: TyCtxt<'tcx>, | |
1076 | instance: Instance<'tcx>, | |
1077 | source: Span, | |
1078 | ) -> Spanned<MonoItem<'tcx>> { | |
136023e0 | 1079 | let def_id = instance.def_id(); |
064997fb | 1080 | if tcx.sess.opts.unstable_opts.profile_closures && def_id.is_local() && tcx.is_closure(def_id) { |
c295e0f8 | 1081 | crate::util::dump_closure_profile(tcx, instance); |
136023e0 XL |
1082 | } |
1083 | ||
f2b60f7d | 1084 | respan(source, MonoItem::Fn(instance.polymorphize(tcx))) |
7453a54e SL |
1085 | } |
1086 | ||
ff7c6d11 | 1087 | /// Creates a `MonoItem` for each method that is referenced by the vtable for |
7453a54e | 1088 | /// the given trait/impl pair. |
dc9dc135 XL |
1089 | fn create_mono_items_for_vtable_methods<'tcx>( |
1090 | tcx: TyCtxt<'tcx>, | |
1091 | trait_ty: Ty<'tcx>, | |
1092 | impl_ty: Ty<'tcx>, | |
f035d41b | 1093 | source: Span, |
923072b8 | 1094 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 1095 | ) { |
3dfed10e | 1096 | assert!(!trait_ty.has_escaping_bound_vars() && !impl_ty.has_escaping_bound_vars()); |
7453a54e | 1097 | |
1b1a35ee | 1098 | if let ty::Dynamic(ref trait_ty, ..) = trait_ty.kind() { |
0731742a XL |
1099 | if let Some(principal) = trait_ty.principal() { |
1100 | let poly_trait_ref = principal.with_self_ty(tcx, impl_ty); | |
1101 | assert!(!poly_trait_ref.has_escaping_bound_vars()); | |
1102 | ||
1103 | // Walk all methods of the trait, including those of its supertraits | |
136023e0 XL |
1104 | let entries = tcx.vtable_entries(poly_trait_ref); |
1105 | let methods = entries | |
dfeec247 | 1106 | .iter() |
136023e0 XL |
1107 | .filter_map(|entry| match entry { |
1108 | VtblEntry::MetadataDropInPlace | |
1109 | | VtblEntry::MetadataSize | |
1110 | | VtblEntry::MetadataAlign | |
1111 | | VtblEntry::Vacant => None, | |
94222f64 XL |
1112 | VtblEntry::TraitVPtr(_) => { |
1113 | // all super trait items already covered, so skip them. | |
1114 | None | |
1115 | } | |
1116 | VtblEntry::Method(instance) => { | |
1117 | Some(*instance).filter(|instance| should_codegen_locally(tcx, instance)) | |
1118 | } | |
dfeec247 | 1119 | }) |
3dfed10e | 1120 | .map(|item| create_fn_mono_item(tcx, item, source)); |
0731742a XL |
1121 | output.extend(methods); |
1122 | } | |
1123 | ||
60c5eb7d | 1124 | // Also add the destructor. |
f035d41b | 1125 | visit_drop_use(tcx, impl_ty, false, source, output); |
7453a54e SL |
1126 | } |
1127 | } | |
1128 | ||
1129 | //=----------------------------------------------------------------------------- | |
1130 | // Root Collection | |
1131 | //=----------------------------------------------------------------------------- | |
1132 | ||
dc9dc135 XL |
1133 | struct RootCollector<'a, 'tcx> { |
1134 | tcx: TyCtxt<'tcx>, | |
ff7c6d11 | 1135 | mode: MonoItemCollectionMode, |
923072b8 | 1136 | output: &'a mut MonoItems<'tcx>, |
cdc7bbd5 | 1137 | entry_fn: Option<(DefId, EntryFnType)>, |
7453a54e SL |
1138 | } |
1139 | ||
04454e1e FG |
1140 | impl<'v> RootCollector<'_, 'v> { |
1141 | fn process_item(&mut self, id: hir::ItemId) { | |
2b03887a | 1142 | match self.tcx.def_kind(id.owner_id) { |
04454e1e | 1143 | DefKind::Enum | DefKind::Struct | DefKind::Union => { |
9ffffee4 FG |
1144 | if self.mode == MonoItemCollectionMode::Eager |
1145 | && self.tcx.generics_of(id.owner_id).count() == 0 | |
1146 | { | |
1147 | debug!("RootCollector: ADT drop-glue for `{id:?}`",); | |
1148 | ||
1149 | let ty = self.tcx.type_of(id.owner_id.to_def_id()).no_bound_vars().unwrap(); | |
1150 | visit_drop_use(self.tcx, ty, true, DUMMY_SP, self.output); | |
7453a54e SL |
1151 | } |
1152 | } | |
04454e1e | 1153 | DefKind::GlobalAsm => { |
dfeec247 XL |
1154 | debug!( |
1155 | "RootCollector: ItemKind::GlobalAsm({})", | |
49aad941 | 1156 | self.tcx.def_path_str(id.owner_id) |
dfeec247 | 1157 | ); |
04454e1e | 1158 | self.output.push(dummy_spanned(MonoItem::GlobalAsm(id))); |
cc61c64b | 1159 | } |
04454e1e | 1160 | DefKind::Static(..) => { |
353b0b11 FG |
1161 | let def_id = id.owner_id.to_def_id(); |
1162 | debug!("RootCollector: ItemKind::Static({})", self.tcx.def_path_str(def_id)); | |
1163 | self.output.push(dummy_spanned(MonoItem::Static(def_id))); | |
7453a54e | 1164 | } |
04454e1e | 1165 | DefKind::Const => { |
ff7c6d11 | 1166 | // const items only generate mono items if they are |
5bcae85e | 1167 | // actually used somewhere. Just declaring them is insufficient. |
a1dfa0c6 XL |
1168 | |
1169 | // but even just declaring them must collect the items they refer to | |
2b03887a | 1170 | if let Ok(val) = self.tcx.const_eval_poly(id.owner_id.to_def_id()) { |
74b04a01 | 1171 | collect_const_value(self.tcx, val, &mut self.output); |
a1dfa0c6 | 1172 | } |
5bcae85e | 1173 | } |
9ffffee4 | 1174 | DefKind::Impl { .. } => { |
04454e1e | 1175 | if self.mode == MonoItemCollectionMode::Eager { |
9ffffee4 | 1176 | create_mono_items_for_default_impls(self.tcx, id, self.output); |
04454e1e FG |
1177 | } |
1178 | } | |
1179 | DefKind::Fn => { | |
2b03887a | 1180 | self.push_if_root(id.owner_id.def_id); |
7453a54e | 1181 | } |
04454e1e | 1182 | _ => {} |
7453a54e | 1183 | } |
7453a54e SL |
1184 | } |
1185 | ||
04454e1e | 1186 | fn process_impl_item(&mut self, id: hir::ImplItemId) { |
2b03887a FG |
1187 | if matches!(self.tcx.def_kind(id.owner_id), DefKind::AssocFn) { |
1188 | self.push_if_root(id.owner_id.def_id); | |
7453a54e | 1189 | } |
7453a54e | 1190 | } |
fc512014 | 1191 | |
f9f354fc | 1192 | fn is_root(&self, def_id: LocalDefId) -> bool { |
dfeec247 XL |
1193 | !item_requires_monomorphization(self.tcx, def_id) |
1194 | && match self.mode { | |
1195 | MonoItemCollectionMode::Eager => true, | |
1196 | MonoItemCollectionMode::Lazy => { | |
cdc7bbd5 | 1197 | self.entry_fn.and_then(|(id, _)| id.as_local()) == Some(def_id) |
dfeec247 XL |
1198 | || self.tcx.is_reachable_non_generic(def_id) |
1199 | || self | |
1200 | .tcx | |
1201 | .codegen_fn_attrs(def_id) | |
1202 | .flags | |
1203 | .contains(CodegenFnAttrFlags::RUSTC_STD_INTERNAL_SYMBOL) | |
1204 | } | |
abe05a73 | 1205 | } |
abe05a73 | 1206 | } |
ff7c6d11 | 1207 | |
60c5eb7d | 1208 | /// If `def_id` represents a root, pushes it onto the list of |
ff7c6d11 | 1209 | /// outputs. (Note that all roots must be monomorphic.) |
923072b8 | 1210 | #[instrument(skip(self), level = "debug")] |
f9f354fc | 1211 | fn push_if_root(&mut self, def_id: LocalDefId) { |
ff7c6d11 | 1212 | if self.is_root(def_id) { |
f2b60f7d | 1213 | debug!("found root"); |
ff7c6d11 | 1214 | |
f9f354fc | 1215 | let instance = Instance::mono(self.tcx, def_id.to_def_id()); |
3dfed10e | 1216 | self.output.push(create_fn_mono_item(self.tcx, instance, DUMMY_SP)); |
ff7c6d11 XL |
1217 | } |
1218 | } | |
1219 | ||
1220 | /// As a special case, when/if we encounter the | |
1221 | /// `main()` function, we also have to generate a | |
1222 | /// monomorphized copy of the start lang item based on | |
1223 | /// the return type of `main`. This is not needed when | |
1224 | /// the user writes their own `start` manually. | |
0531ce1d | 1225 | fn push_extra_entry_roots(&mut self) { |
f2b60f7d | 1226 | let Some((main_def_id, EntryFnType::Main { .. })) = self.entry_fn else { |
5e7ed085 | 1227 | return; |
0531ce1d XL |
1228 | }; |
1229 | ||
487cf647 | 1230 | let start_def_id = self.tcx.require_lang_item(LangItem::Start, None); |
9ffffee4 | 1231 | let main_ret_ty = self.tcx.fn_sig(main_def_id).no_bound_vars().unwrap().output(); |
ff7c6d11 XL |
1232 | |
1233 | // Given that `main()` has no arguments, | |
1234 | // then its return type cannot have | |
1235 | // late-bound regions, since late-bound | |
1236 | // regions must appear in the argument | |
1237 | // listing. | |
5e7ed085 FG |
1238 | let main_ret_ty = self.tcx.normalize_erasing_regions( |
1239 | ty::ParamEnv::reveal_all(), | |
1240 | main_ret_ty.no_bound_vars().unwrap(), | |
1241 | ); | |
ff7c6d11 XL |
1242 | |
1243 | let start_instance = Instance::resolve( | |
1244 | self.tcx, | |
0531ce1d | 1245 | ty::ParamEnv::reveal_all(), |
ff7c6d11 | 1246 | start_def_id, |
add651ee | 1247 | self.tcx.mk_args(&[main_ret_ty.into()]), |
dfeec247 | 1248 | ) |
f9f354fc | 1249 | .unwrap() |
dfeec247 | 1250 | .unwrap(); |
ff7c6d11 | 1251 | |
3dfed10e | 1252 | self.output.push(create_fn_mono_item(self.tcx, start_instance, DUMMY_SP)); |
ff7c6d11 | 1253 | } |
abe05a73 XL |
1254 | } |
1255 | ||
f9f354fc | 1256 | fn item_requires_monomorphization(tcx: TyCtxt<'_>, def_id: LocalDefId) -> bool { |
3b2f2976 | 1257 | let generics = tcx.generics_of(def_id); |
94b46f34 | 1258 | generics.requires_monomorphization(tcx) |
3b2f2976 XL |
1259 | } |
1260 | ||
9ffffee4 | 1261 | #[instrument(level = "debug", skip(tcx, output))] |
dc9dc135 XL |
1262 | fn create_mono_items_for_default_impls<'tcx>( |
1263 | tcx: TyCtxt<'tcx>, | |
9ffffee4 | 1264 | item: hir::ItemId, |
923072b8 | 1265 | output: &mut MonoItems<'tcx>, |
dc9dc135 | 1266 | ) { |
9ffffee4 FG |
1267 | let polarity = tcx.impl_polarity(item.owner_id); |
1268 | if matches!(polarity, ty::ImplPolarity::Negative) { | |
1269 | return; | |
1270 | } | |
487cf647 | 1271 | |
9ffffee4 FG |
1272 | if tcx.generics_of(item.owner_id).own_requires_monomorphization() { |
1273 | return; | |
1274 | } | |
7453a54e | 1275 | |
9ffffee4 FG |
1276 | let Some(trait_ref) = tcx.impl_trait_ref(item.owner_id) else { |
1277 | return; | |
1278 | }; | |
7453a54e | 1279 | |
353b0b11 FG |
1280 | // Lifetimes never affect trait selection, so we are allowed to eagerly |
1281 | // instantiate an instance of an impl method if the impl (and method, | |
1282 | // which we check below) is only parameterized over lifetime. In that case, | |
1283 | // we use the ReErased, which has no lifetime information associated with | |
1284 | // it, to validate whether or not the impl is legal to instantiate at all. | |
1285 | let only_region_params = |param: &ty::GenericParamDef, _: &_| match param.kind { | |
1286 | GenericParamDefKind::Lifetime => tcx.lifetimes.re_erased.into(), | |
1287 | GenericParamDefKind::Type { .. } | GenericParamDefKind::Const { .. } => { | |
1288 | unreachable!( | |
1289 | "`own_requires_monomorphization` check means that \ | |
1290 | we should have no type/const params" | |
1291 | ) | |
1292 | } | |
1293 | }; | |
add651ee FG |
1294 | let impl_args = GenericArgs::for_item(tcx, item.owner_id.to_def_id(), only_region_params); |
1295 | let trait_ref = trait_ref.instantiate(tcx, impl_args); | |
353b0b11 FG |
1296 | |
1297 | // Unlike 'lazy' monomorphization that begins by collecting items transitively | |
1298 | // called by `main` or other global items, when eagerly monomorphizing impl | |
1299 | // items, we never actually check that the predicates of this impl are satisfied | |
1300 | // in a empty reveal-all param env (i.e. with no assumptions). | |
1301 | // | |
1302 | // Even though this impl has no type or const substitutions, because we don't | |
1303 | // consider higher-ranked predicates such as `for<'a> &'a mut [u8]: Copy` to | |
1304 | // be trivially false. We must now check that the impl has no impossible-to-satisfy | |
1305 | // predicates. | |
add651ee | 1306 | if tcx.subst_and_check_impossible_predicates((item.owner_id.to_def_id(), impl_args)) { |
353b0b11 FG |
1307 | return; |
1308 | } | |
9c376795 | 1309 | |
9ffffee4 FG |
1310 | let param_env = ty::ParamEnv::reveal_all(); |
1311 | let trait_ref = tcx.normalize_erasing_regions(param_env, trait_ref); | |
1312 | let overridden_methods = tcx.impl_item_implementor_ids(item.owner_id); | |
1313 | for method in tcx.provided_trait_methods(trait_ref.def_id) { | |
1314 | if overridden_methods.contains_key(&method.def_id) { | |
1315 | continue; | |
1316 | } | |
7453a54e | 1317 | |
9ffffee4 FG |
1318 | if tcx.generics_of(method.def_id).own_requires_monomorphization() { |
1319 | continue; | |
1320 | } | |
7453a54e | 1321 | |
353b0b11 FG |
1322 | // As mentioned above, the method is legal to eagerly instantiate if it |
1323 | // only has lifetime substitutions. This is validated by | |
add651ee FG |
1324 | let args = trait_ref.args.extend_to(tcx, method.def_id, only_region_params); |
1325 | let instance = ty::Instance::expect_resolve(tcx, param_env, method.def_id, args); | |
9ffffee4 FG |
1326 | |
1327 | let mono_item = create_fn_mono_item(tcx, instance, DUMMY_SP); | |
1328 | if mono_item.node.is_instantiable(tcx) && should_codegen_locally(tcx, &instance) { | |
1329 | output.push(mono_item); | |
7453a54e | 1330 | } |
7453a54e SL |
1331 | } |
1332 | } | |
1333 | ||
add651ee FG |
1334 | /// Scans the CTFE alloc in order to find function calls, closures, and drop-glue. |
1335 | fn collect_alloc<'tcx>(tcx: TyCtxt<'tcx>, alloc_id: AllocId, output: &mut MonoItems<'tcx>) { | |
f9f354fc XL |
1336 | match tcx.global_alloc(alloc_id) { |
1337 | GlobalAlloc::Static(def_id) => { | |
1338 | assert!(!tcx.is_thread_local_static(def_id)); | |
dc9dc135 | 1339 | let instance = Instance::mono(tcx, def_id); |
3dfed10e | 1340 | if should_codegen_locally(tcx, &instance) { |
dc9dc135 | 1341 | trace!("collecting static {:?}", def_id); |
f035d41b | 1342 | output.push(dummy_spanned(MonoItem::Static(def_id))); |
94b46f34 | 1343 | } |
0531ce1d | 1344 | } |
f9f354fc | 1345 | GlobalAlloc::Memory(alloc) => { |
94b46f34 | 1346 | trace!("collecting {:?} with {:#?}", alloc_id, alloc); |
487cf647 | 1347 | for &inner in alloc.inner().provenance().ptrs().values() { |
f9f354fc | 1348 | rustc_data_structures::stack::ensure_sufficient_stack(|| { |
add651ee | 1349 | collect_alloc(tcx, inner, output); |
f9f354fc | 1350 | }); |
94b46f34 | 1351 | } |
dfeec247 | 1352 | } |
f9f354fc | 1353 | GlobalAlloc::Function(fn_instance) => { |
3dfed10e | 1354 | if should_codegen_locally(tcx, &fn_instance) { |
94b46f34 | 1355 | trace!("collecting {:?} with {:#?}", alloc_id, fn_instance); |
3dfed10e | 1356 | output.push(create_fn_mono_item(tcx, fn_instance, DUMMY_SP)); |
94b46f34 | 1357 | } |
0531ce1d | 1358 | } |
064997fb FG |
1359 | GlobalAlloc::VTable(ty, trait_ref) => { |
1360 | let alloc_id = tcx.vtable_allocation((ty, trait_ref)); | |
add651ee | 1361 | collect_alloc(tcx, alloc_id, output) |
064997fb | 1362 | } |
0531ce1d XL |
1363 | } |
1364 | } | |
1365 | ||
60c5eb7d | 1366 | /// Scans the MIR in order to find function calls, closures, and drop-glue. |
923072b8 | 1367 | #[instrument(skip(tcx, output), level = "debug")] |
fe692bf9 | 1368 | fn collect_used_items<'tcx>( |
dc9dc135 XL |
1369 | tcx: TyCtxt<'tcx>, |
1370 | instance: Instance<'tcx>, | |
923072b8 | 1371 | output: &mut MonoItems<'tcx>, |
dc9dc135 XL |
1372 | ) { |
1373 | let body = tcx.instance_mir(instance.def); | |
fe692bf9 | 1374 | MirUsedCollector { tcx, body: &body, output, instance }.visit_body(&body); |
7453a54e | 1375 | } |
476ff2be | 1376 | |
923072b8 | 1377 | #[instrument(skip(tcx, output), level = "debug")] |
74b04a01 XL |
1378 | fn collect_const_value<'tcx>( |
1379 | tcx: TyCtxt<'tcx>, | |
1380 | value: ConstValue<'tcx>, | |
923072b8 | 1381 | output: &mut MonoItems<'tcx>, |
74b04a01 XL |
1382 | ) { |
1383 | match value { | |
add651ee | 1384 | ConstValue::Scalar(Scalar::Ptr(ptr, _size)) => collect_alloc(tcx, ptr.provenance, output), |
74b04a01 | 1385 | ConstValue::Slice { data: alloc, start: _, end: _ } | ConstValue::ByRef { alloc, .. } => { |
487cf647 | 1386 | for &id in alloc.inner().provenance().ptrs().values() { |
add651ee | 1387 | collect_alloc(tcx, id, output); |
74b04a01 XL |
1388 | } |
1389 | } | |
1390 | _ => {} | |
1391 | } | |
1392 | } |