]>
Commit | Line | Data |
---|---|---|
a7813a04 XL |
1 | //! Partitioning Codegen Units for Incremental Compilation |
2 | //! ====================================================== | |
3 | //! | |
94b46f34 | 4 | //! The task of this module is to take the complete set of monomorphizations of |
a7813a04 | 5 | //! a crate and produce a set of codegen units from it, where a codegen unit |
94b46f34 XL |
6 | //! is a named set of (mono-item, linkage) pairs. That is, this module |
7 | //! decides which monomorphization appears in which codegen units with which | |
a7813a04 XL |
8 | //! linkage. The following paragraphs describe some of the background on the |
9 | //! partitioning scheme. | |
10 | //! | |
11 | //! The most important opportunity for saving on compilation time with | |
94b46f34 XL |
12 | //! incremental compilation is to avoid re-codegenning and re-optimizing code. |
13 | //! Since the unit of codegen and optimization for LLVM is "modules" or, how | |
a7813a04 XL |
14 | //! we call them "codegen units", the particulars of how much time can be saved |
15 | //! by incremental compilation are tightly linked to how the output program is | |
16 | //! partitioned into these codegen units prior to passing it to LLVM -- | |
17 | //! especially because we have to treat codegen units as opaque entities once | |
18 | //! they are created: There is no way for us to incrementally update an existing | |
19 | //! LLVM module and so we have to build any such module from scratch if it was | |
20 | //! affected by some change in the source code. | |
21 | //! | |
22 | //! From that point of view it would make sense to maximize the number of | |
23 | //! codegen units by, for example, putting each function into its own module. | |
24 | //! That way only those modules would have to be re-compiled that were actually | |
25 | //! affected by some change, minimizing the number of functions that could have | |
26 | //! been re-used but just happened to be located in a module that is | |
27 | //! re-compiled. | |
28 | //! | |
29 | //! However, since LLVM optimization does not work across module boundaries, | |
30 | //! using such a highly granular partitioning would lead to very slow runtime | |
31 | //! code since it would effectively prohibit inlining and other inter-procedure | |
32 | //! optimizations. We want to avoid that as much as possible. | |
33 | //! | |
34 | //! Thus we end up with a trade-off: The bigger the codegen units, the better | |
35 | //! LLVM's optimizer can do its work, but also the smaller the compilation time | |
36 | //! reduction we get from incremental compilation. | |
37 | //! | |
38 | //! Ideally, we would create a partitioning such that there are few big codegen | |
39 | //! units with few interdependencies between them. For now though, we use the | |
40 | //! following heuristic to determine the partitioning: | |
41 | //! | |
42 | //! - There are two codegen units for every source-level module: | |
43 | //! - One for "stable", that is non-generic, code | |
0731742a | 44 | //! - One for more "volatile" code, i.e., monomorphized instances of functions |
a7813a04 | 45 | //! defined in that module |
a7813a04 XL |
46 | //! |
47 | //! In order to see why this heuristic makes sense, let's take a look at when a | |
48 | //! codegen unit can get invalidated: | |
49 | //! | |
50 | //! 1. The most straightforward case is when the BODY of a function or global | |
51 | //! changes. Then any codegen unit containing the code for that item has to be | |
52 | //! re-compiled. Note that this includes all codegen units where the function | |
53 | //! has been inlined. | |
54 | //! | |
55 | //! 2. The next case is when the SIGNATURE of a function or global changes. In | |
56 | //! this case, all codegen units containing a REFERENCE to that item have to be | |
57 | //! re-compiled. This is a superset of case 1. | |
58 | //! | |
59 | //! 3. The final and most subtle case is when a REFERENCE to a generic function | |
60 | //! is added or removed somewhere. Even though the definition of the function | |
61 | //! might be unchanged, a new REFERENCE might introduce a new monomorphized | |
62 | //! instance of this function which has to be placed and compiled somewhere. | |
63 | //! Conversely, when removing a REFERENCE, it might have been the last one with | |
64 | //! that particular set of generic arguments and thus we have to remove it. | |
65 | //! | |
66 | //! From the above we see that just using one codegen unit per source-level | |
67 | //! module is not such a good idea, since just adding a REFERENCE to some | |
68 | //! generic item somewhere else would invalidate everything within the module | |
69 | //! containing the generic item. The heuristic above reduces this detrimental | |
70 | //! side-effect of references a little by at least not touching the non-generic | |
71 | //! code of the module. | |
72 | //! | |
a7813a04 XL |
73 | //! A Note on Inlining |
74 | //! ------------------ | |
75 | //! As briefly mentioned above, in order for LLVM to be able to inline a | |
76 | //! function call, the body of the function has to be available in the LLVM | |
77 | //! module where the call is made. This has a few consequences for partitioning: | |
78 | //! | |
79 | //! - The partitioning algorithm has to take care of placing functions into all | |
80 | //! codegen units where they should be available for inlining. It also has to | |
81 | //! decide on the correct linkage for these functions. | |
82 | //! | |
83 | //! - The partitioning algorithm has to know which functions are likely to get | |
84 | //! inlined, so it can distribute function instantiations accordingly. Since | |
85 | //! there is no way of knowing for sure which functions LLVM will decide to | |
86 | //! inline in the end, we apply a heuristic here: Only functions marked with | |
ff7c6d11 | 87 | //! `#[inline]` are considered for inlining by the partitioner. The current |
32a655c1 SL |
88 | //! implementation will not try to determine if a function is likely to be |
89 | //! inlined by looking at the functions definition. | |
a7813a04 XL |
90 | //! |
91 | //! Note though that as a side-effect of creating a codegen units per | |
92 | //! source-level module, functions from the same module will be available for | |
dc9dc135 | 93 | //! inlining, even when they are not marked `#[inline]`. |
a7813a04 | 94 | |
a1dfa0c6 | 95 | use std::cmp; |
dfeec247 | 96 | use std::collections::hash_map::Entry; |
a1dfa0c6 XL |
97 | use std::sync::Arc; |
98 | ||
dfeec247 | 99 | use rustc::middle::codegen_fn_attrs::CodegenFnAttrFlags; |
2c00a5a8 | 100 | use rustc::middle::exported_symbols::SymbolExportLevel; |
dfeec247 XL |
101 | use rustc::mir::mono::{CodegenUnit, CodegenUnitNameBuilder, Linkage, Visibility}; |
102 | use rustc::mir::mono::{InstantiationMode, MonoItem}; | |
532ac7d7 | 103 | use rustc::ty::print::characteristic_def_id_of_type; |
a1dfa0c6 | 104 | use rustc::ty::query::Providers; |
dfeec247 XL |
105 | use rustc::ty::{self, DefIdTree, InstanceDef, TyCtxt}; |
106 | use rustc_data_structures::fx::{FxHashMap, FxHashSet}; | |
107 | use rustc_data_structures::sync; | |
108 | use rustc_hir::def::DefKind; | |
109 | use rustc_hir::def_id::{CrateNum, DefId, DefIdSet, CRATE_DEF_INDEX, LOCAL_CRATE}; | |
110 | use rustc_span::symbol::Symbol; | |
a1dfa0c6 | 111 | |
9fa01778 XL |
112 | use crate::monomorphize::collector::InliningMap; |
113 | use crate::monomorphize::collector::{self, MonoItemCollectionMode}; | |
a7813a04 XL |
114 | |
115 | pub enum PartitioningStrategy { | |
9fa01778 | 116 | /// Generates one codegen unit per source-level module. |
a7813a04 XL |
117 | PerModule, |
118 | ||
119 | /// Partition the whole crate into a fixed number of codegen units. | |
dfeec247 | 120 | FixedUnitCount(usize), |
a7813a04 XL |
121 | } |
122 | ||
123 | // Anything we can't find a proper codegen unit for goes into this. | |
e74abb32 | 124 | fn fallback_cgu_name(name_builder: &mut CodegenUnitNameBuilder<'_>) -> Symbol { |
b7449926 | 125 | name_builder.build_cgu_name(LOCAL_CRATE, &["fallback"], Some("cgu")) |
ff7c6d11 XL |
126 | } |
127 | ||
dc9dc135 XL |
128 | pub fn partition<'tcx, I>( |
129 | tcx: TyCtxt<'tcx>, | |
130 | mono_items: I, | |
131 | strategy: PartitioningStrategy, | |
132 | inlining_map: &InliningMap<'tcx>, | |
133 | ) -> Vec<CodegenUnit<'tcx>> | |
134 | where | |
135 | I: Iterator<Item = MonoItem<'tcx>>, | |
a7813a04 | 136 | { |
e74abb32 XL |
137 | let _prof_timer = tcx.prof.generic_activity("cgu_partitioning"); |
138 | ||
94b46f34 XL |
139 | // In the first step, we place all regular monomorphizations into their |
140 | // respective 'home' codegen unit. Regular monomorphizations are all | |
a7813a04 | 141 | // functions and statics defined in the local crate. |
e74abb32 XL |
142 | let mut initial_partitioning = { |
143 | let _prof_timer = tcx.prof.generic_activity("cgu_partitioning_place_roots"); | |
144 | place_root_mono_items(tcx, mono_items) | |
145 | }; | |
5bcae85e | 146 | |
48663c56 | 147 | initial_partitioning.codegen_units.iter_mut().for_each(|cgu| cgu.estimate_size(tcx)); |
2c00a5a8 | 148 | |
3b2f2976 | 149 | debug_dump(tcx, "INITIAL PARTITIONING:", initial_partitioning.codegen_units.iter()); |
a7813a04 XL |
150 | |
151 | // If the partitioning should produce a fixed count of codegen units, merge | |
152 | // until that count is reached. | |
153 | if let PartitioningStrategy::FixedUnitCount(count) = strategy { | |
e74abb32 | 154 | let _prof_timer = tcx.prof.generic_activity("cgu_partitioning_merge_cgus"); |
b7449926 | 155 | merge_codegen_units(tcx, &mut initial_partitioning, count); |
cc61c64b | 156 | debug_dump(tcx, "POST MERGING:", initial_partitioning.codegen_units.iter()); |
a7813a04 XL |
157 | } |
158 | ||
3b2f2976 | 159 | // In the next step, we use the inlining map to determine which additional |
94b46f34 XL |
160 | // monomorphizations have to go into each codegen unit. These additional |
161 | // monomorphizations can be drop-glue, functions from external crates, and | |
dc9dc135 | 162 | // local functions the definition of which is marked with `#[inline]`. |
e74abb32 | 163 | let mut post_inlining = { |
dfeec247 | 164 | let _prof_timer = tcx.prof.generic_activity("cgu_partitioning_place_inline_items"); |
e74abb32 XL |
165 | place_inlined_mono_items(initial_partitioning, inlining_map) |
166 | }; | |
5bcae85e | 167 | |
48663c56 | 168 | post_inlining.codegen_units.iter_mut().for_each(|cgu| cgu.estimate_size(tcx)); |
2c00a5a8 | 169 | |
3b2f2976 XL |
170 | debug_dump(tcx, "POST INLINING:", post_inlining.codegen_units.iter()); |
171 | ||
172 | // Next we try to make as many symbols "internal" as possible, so LLVM has | |
173 | // more freedom to optimize. | |
ff7c6d11 | 174 | if !tcx.sess.opts.cg.link_dead_code { |
dfeec247 | 175 | let _prof_timer = tcx.prof.generic_activity("cgu_partitioning_internalize_symbols"); |
ff7c6d11 XL |
176 | internalize_symbols(tcx, &mut post_inlining, inlining_map); |
177 | } | |
5bcae85e | 178 | |
dc9dc135 | 179 | // Finally, sort by codegen unit name, so that we get deterministic results. |
3b2f2976 XL |
180 | let PostInliningPartitioning { |
181 | codegen_units: mut result, | |
94b46f34 | 182 | mono_item_placements: _, |
3b2f2976 XL |
183 | internalization_candidates: _, |
184 | } = post_inlining; | |
185 | ||
e74abb32 | 186 | result.sort_by_cached_key(|cgu| cgu.name().as_str()); |
5bcae85e SL |
187 | |
188 | result | |
a7813a04 XL |
189 | } |
190 | ||
191 | struct PreInliningPartitioning<'tcx> { | |
192 | codegen_units: Vec<CodegenUnit<'tcx>>, | |
ff7c6d11 XL |
193 | roots: FxHashSet<MonoItem<'tcx>>, |
194 | internalization_candidates: FxHashSet<MonoItem<'tcx>>, | |
a7813a04 XL |
195 | } |
196 | ||
94b46f34 | 197 | /// For symbol internalization, we need to know whether a symbol/mono-item is |
3b2f2976 XL |
198 | /// accessed from outside the codegen unit it is defined in. This type is used |
199 | /// to keep track of that. | |
200 | #[derive(Clone, PartialEq, Eq, Debug)] | |
94b46f34 | 201 | enum MonoItemPlacement { |
e74abb32 | 202 | SingleCgu { cgu_name: Symbol }, |
3b2f2976 XL |
203 | MultipleCgus, |
204 | } | |
205 | ||
206 | struct PostInliningPartitioning<'tcx> { | |
207 | codegen_units: Vec<CodegenUnit<'tcx>>, | |
94b46f34 | 208 | mono_item_placements: FxHashMap<MonoItem<'tcx>, MonoItemPlacement>, |
ff7c6d11 | 209 | internalization_candidates: FxHashSet<MonoItem<'tcx>>, |
3b2f2976 | 210 | } |
a7813a04 | 211 | |
dc9dc135 XL |
212 | fn place_root_mono_items<'tcx, I>(tcx: TyCtxt<'tcx>, mono_items: I) -> PreInliningPartitioning<'tcx> |
213 | where | |
214 | I: Iterator<Item = MonoItem<'tcx>>, | |
a7813a04 | 215 | { |
0bf4aa26 XL |
216 | let mut roots = FxHashSet::default(); |
217 | let mut codegen_units = FxHashMap::default(); | |
32a655c1 | 218 | let is_incremental_build = tcx.sess.opts.incremental.is_some(); |
0bf4aa26 | 219 | let mut internalization_candidates = FxHashSet::default(); |
a7813a04 | 220 | |
83c7162d XL |
221 | // Determine if monomorphizations instantiated in this crate will be made |
222 | // available to downstream crates. This depends on whether we are in | |
223 | // share-generics mode and whether the current crate can even have | |
224 | // downstream crates. | |
dfeec247 | 225 | let export_generics = tcx.sess.opts.share_generics() && tcx.local_crate_exports_generics(); |
83c7162d | 226 | |
b7449926 | 227 | let cgu_name_builder = &mut CodegenUnitNameBuilder::new(tcx); |
0bf4aa26 | 228 | let cgu_name_cache = &mut FxHashMap::default(); |
b7449926 | 229 | |
94b46f34 XL |
230 | for mono_item in mono_items { |
231 | match mono_item.instantiation_mode(tcx) { | |
ea8adc8c XL |
232 | InstantiationMode::GloballyShared { .. } => {} |
233 | InstantiationMode::LocalCopy => continue, | |
234 | } | |
a7813a04 | 235 | |
94b46f34 | 236 | let characteristic_def_id = characteristic_def_id_of_mono_item(tcx, mono_item); |
dfeec247 | 237 | let is_volatile = is_incremental_build && mono_item.is_generic_fn(); |
a7813a04 | 238 | |
ea8adc8c | 239 | let codegen_unit_name = match characteristic_def_id { |
dfeec247 XL |
240 | Some(def_id) => compute_codegen_unit_name( |
241 | tcx, | |
242 | cgu_name_builder, | |
243 | def_id, | |
244 | is_volatile, | |
245 | cgu_name_cache, | |
246 | ), | |
b7449926 | 247 | None => fallback_cgu_name(cgu_name_builder), |
ea8adc8c | 248 | }; |
a7813a04 | 249 | |
dfeec247 XL |
250 | let codegen_unit = codegen_units |
251 | .entry(codegen_unit_name) | |
e74abb32 | 252 | .or_insert_with(|| CodegenUnit::new(codegen_unit_name)); |
ea8adc8c | 253 | |
ff7c6d11 | 254 | let mut can_be_internalized = true; |
b7449926 XL |
255 | let (linkage, visibility) = mono_item_linkage_and_visibility( |
256 | tcx, | |
257 | &mono_item, | |
258 | &mut can_be_internalized, | |
259 | export_generics, | |
260 | ); | |
ff7c6d11 | 261 | if visibility == Visibility::Hidden && can_be_internalized { |
94b46f34 | 262 | internalization_candidates.insert(mono_item); |
a7813a04 | 263 | } |
ea8adc8c | 264 | |
94b46f34 XL |
265 | codegen_unit.items_mut().insert(mono_item, (linkage, visibility)); |
266 | roots.insert(mono_item); | |
a7813a04 XL |
267 | } |
268 | ||
dc9dc135 XL |
269 | // Always ensure we have at least one CGU; otherwise, if we have a |
270 | // crate with just types (for example), we could wind up with no CGU. | |
3157f602 | 271 | if codegen_units.is_empty() { |
b7449926 | 272 | let codegen_unit_name = fallback_cgu_name(cgu_name_builder); |
e74abb32 | 273 | codegen_units.insert(codegen_unit_name, CodegenUnit::new(codegen_unit_name)); |
3157f602 XL |
274 | } |
275 | ||
a7813a04 | 276 | PreInliningPartitioning { |
dfeec247 | 277 | codegen_units: codegen_units.into_iter().map(|(_, codegen_unit)| codegen_unit).collect(), |
3b2f2976 XL |
278 | roots, |
279 | internalization_candidates, | |
a7813a04 XL |
280 | } |
281 | } | |
282 | ||
b7449926 | 283 | fn mono_item_linkage_and_visibility( |
dc9dc135 | 284 | tcx: TyCtxt<'tcx>, |
b7449926 XL |
285 | mono_item: &MonoItem<'tcx>, |
286 | can_be_internalized: &mut bool, | |
287 | export_generics: bool, | |
288 | ) -> (Linkage, Visibility) { | |
289 | if let Some(explicit_linkage) = mono_item.explicit_linkage(tcx) { | |
dfeec247 | 290 | return (explicit_linkage, Visibility::Default); |
b7449926 | 291 | } |
dfeec247 | 292 | let vis = mono_item_visibility(tcx, mono_item, can_be_internalized, export_generics); |
b7449926 XL |
293 | (Linkage::External, vis) |
294 | } | |
295 | ||
296 | fn mono_item_visibility( | |
dc9dc135 | 297 | tcx: TyCtxt<'tcx>, |
b7449926 XL |
298 | mono_item: &MonoItem<'tcx>, |
299 | can_be_internalized: &mut bool, | |
300 | export_generics: bool, | |
301 | ) -> Visibility { | |
302 | let instance = match mono_item { | |
dc9dc135 | 303 | // This is pretty complicated; see below. |
b7449926 XL |
304 | MonoItem::Fn(instance) => instance, |
305 | ||
dc9dc135 | 306 | // Misc handling for generics and such, but otherwise: |
b7449926 XL |
307 | MonoItem::Static(def_id) => { |
308 | return if tcx.is_reachable_non_generic(*def_id) { | |
309 | *can_be_internalized = false; | |
310 | default_visibility(tcx, *def_id, false) | |
311 | } else { | |
312 | Visibility::Hidden | |
313 | }; | |
314 | } | |
532ac7d7 | 315 | MonoItem::GlobalAsm(hir_id) => { |
416331ca | 316 | let def_id = tcx.hir().local_def_id(*hir_id); |
b7449926 XL |
317 | return if tcx.is_reachable_non_generic(def_id) { |
318 | *can_be_internalized = false; | |
319 | default_visibility(tcx, def_id, false) | |
320 | } else { | |
321 | Visibility::Hidden | |
322 | }; | |
323 | } | |
324 | }; | |
325 | ||
326 | let def_id = match instance.def { | |
dfeec247 | 327 | InstanceDef::Item(def_id) | InstanceDef::DropGlue(def_id, Some(_)) => def_id, |
b7449926 XL |
328 | |
329 | // These are all compiler glue and such, never exported, always hidden. | |
dfeec247 XL |
330 | InstanceDef::VtableShim(..) |
331 | | InstanceDef::ReifyShim(..) | |
332 | | InstanceDef::FnPtrShim(..) | |
333 | | InstanceDef::Virtual(..) | |
334 | | InstanceDef::Intrinsic(..) | |
335 | | InstanceDef::ClosureOnceShim { .. } | |
336 | | InstanceDef::DropGlue(..) | |
337 | | InstanceDef::CloneShim(..) => return Visibility::Hidden, | |
b7449926 XL |
338 | }; |
339 | ||
340 | // The `start_fn` lang item is actually a monomorphized instance of a | |
341 | // function in the standard library, used for the `main` function. We don't | |
342 | // want to export it so we tag it with `Hidden` visibility but this symbol | |
343 | // is only referenced from the actual `main` symbol which we unfortunately | |
344 | // don't know anything about during partitioning/collection. As a result we | |
345 | // forcibly keep this symbol out of the `internalization_candidates` set. | |
346 | // | |
347 | // FIXME: eventually we don't want to always force this symbol to have | |
348 | // hidden visibility, it should indeed be a candidate for | |
349 | // internalization, but we have to understand that it's referenced | |
350 | // from the `main` symbol we'll generate later. | |
351 | // | |
352 | // This may be fixable with a new `InstanceDef` perhaps? Unsure! | |
353 | if tcx.lang_items().start_fn() == Some(def_id) { | |
354 | *can_be_internalized = false; | |
dfeec247 | 355 | return Visibility::Hidden; |
b7449926 XL |
356 | } |
357 | ||
532ac7d7 | 358 | let is_generic = instance.substs.non_erasable_generics().next().is_some(); |
b7449926 | 359 | |
dc9dc135 | 360 | // Upstream `DefId` instances get different handling than local ones. |
b7449926 XL |
361 | if !def_id.is_local() { |
362 | return if export_generics && is_generic { | |
dc9dc135 | 363 | // If it is a upstream monomorphization and we export generics, we must make |
b7449926 XL |
364 | // it available to downstream crates. |
365 | *can_be_internalized = false; | |
366 | default_visibility(tcx, def_id, true) | |
367 | } else { | |
368 | Visibility::Hidden | |
dfeec247 | 369 | }; |
b7449926 XL |
370 | } |
371 | ||
372 | if is_generic { | |
373 | if export_generics { | |
374 | if tcx.is_unreachable_local_definition(def_id) { | |
dc9dc135 | 375 | // This instance cannot be used from another crate. |
b7449926 XL |
376 | Visibility::Hidden |
377 | } else { | |
dc9dc135 | 378 | // This instance might be useful in a downstream crate. |
b7449926 XL |
379 | *can_be_internalized = false; |
380 | default_visibility(tcx, def_id, true) | |
381 | } | |
382 | } else { | |
dc9dc135 XL |
383 | // We are not exporting generics or the definition is not reachable |
384 | // for downstream crates, we can internalize its instantiations. | |
b7449926 XL |
385 | Visibility::Hidden |
386 | } | |
387 | } else { | |
b7449926 XL |
388 | // If this isn't a generic function then we mark this a `Default` if |
389 | // this is a reachable item, meaning that it's a symbol other crates may | |
390 | // access when they link to us. | |
391 | if tcx.is_reachable_non_generic(def_id) { | |
392 | *can_be_internalized = false; | |
393 | debug_assert!(!is_generic); | |
dfeec247 | 394 | return default_visibility(tcx, def_id, false); |
b7449926 XL |
395 | } |
396 | ||
397 | // If this isn't reachable then we're gonna tag this with `Hidden` | |
398 | // visibility. In some situations though we'll want to prevent this | |
399 | // symbol from being internalized. | |
400 | // | |
401 | // There's two categories of items here: | |
402 | // | |
403 | // * First is weak lang items. These are basically mechanisms for | |
404 | // libcore to forward-reference symbols defined later in crates like | |
a1dfa0c6 | 405 | // the standard library or `#[panic_handler]` definitions. The |
b7449926 XL |
406 | // definition of these weak lang items needs to be referenceable by |
407 | // libcore, so we're no longer a candidate for internalization. | |
408 | // Removal of these functions can't be done by LLVM but rather must be | |
409 | // done by the linker as it's a non-local decision. | |
410 | // | |
411 | // * Second is "std internal symbols". Currently this is primarily used | |
412 | // for allocator symbols. Allocators are a little weird in their | |
413 | // implementation, but the idea is that the compiler, at the last | |
414 | // minute, defines an allocator with an injected object file. The | |
415 | // `alloc` crate references these symbols (`__rust_alloc`) and the | |
416 | // definition doesn't get hooked up until a linked crate artifact is | |
417 | // generated. | |
418 | // | |
419 | // The symbols synthesized by the compiler (`__rust_alloc`) are thin | |
420 | // veneers around the actual implementation, some other symbol which | |
421 | // implements the same ABI. These symbols (things like `__rg_alloc`, | |
422 | // `__rdl_alloc`, `__rde_alloc`, etc), are all tagged with "std | |
423 | // internal symbols". | |
424 | // | |
425 | // The std-internal symbols here **should not show up in a dll as an | |
426 | // exported interface**, so they return `false` from | |
427 | // `is_reachable_non_generic` above and we'll give them `Hidden` | |
428 | // visibility below. Like the weak lang items, though, we can't let | |
429 | // LLVM internalize them as this decision is left up to the linker to | |
430 | // omit them, so prevent them from being internalized. | |
431 | let attrs = tcx.codegen_fn_attrs(def_id); | |
432 | if attrs.flags.contains(CodegenFnAttrFlags::RUSTC_STD_INTERNAL_SYMBOL) { | |
433 | *can_be_internalized = false; | |
434 | } | |
435 | ||
436 | Visibility::Hidden | |
437 | } | |
438 | } | |
439 | ||
dc9dc135 | 440 | fn default_visibility(tcx: TyCtxt<'_>, id: DefId, is_generic: bool) -> Visibility { |
b7449926 | 441 | if !tcx.sess.target.target.options.default_hidden_visibility { |
dfeec247 | 442 | return Visibility::Default; |
b7449926 XL |
443 | } |
444 | ||
dc9dc135 | 445 | // Generic functions never have export-level C. |
b7449926 | 446 | if is_generic { |
dfeec247 | 447 | return Visibility::Hidden; |
b7449926 XL |
448 | } |
449 | ||
450 | // Things with export level C don't get instantiated in | |
dc9dc135 | 451 | // downstream crates. |
b7449926 | 452 | if !id.is_local() { |
dfeec247 | 453 | return Visibility::Hidden; |
b7449926 XL |
454 | } |
455 | ||
456 | // C-export level items remain at `Default`, all other internal | |
dc9dc135 | 457 | // items become `Hidden`. |
b7449926 XL |
458 | match tcx.reachable_non_generics(id.krate).get(&id) { |
459 | Some(SymbolExportLevel::C) => Visibility::Default, | |
460 | _ => Visibility::Hidden, | |
461 | } | |
462 | } | |
463 | ||
dc9dc135 XL |
464 | fn merge_codegen_units<'tcx>( |
465 | tcx: TyCtxt<'tcx>, | |
466 | initial_partitioning: &mut PreInliningPartitioning<'tcx>, | |
467 | target_cgu_count: usize, | |
468 | ) { | |
a7813a04 XL |
469 | assert!(target_cgu_count >= 1); |
470 | let codegen_units = &mut initial_partitioning.codegen_units; | |
471 | ||
ff7c6d11 XL |
472 | // Note that at this point in time the `codegen_units` here may not be in a |
473 | // deterministic order (but we know they're deterministically the same set). | |
474 | // We want this merging to produce a deterministic ordering of codegen units | |
475 | // from the input. | |
476 | // | |
477 | // Due to basically how we've implemented the merging below (merge the two | |
478 | // smallest into each other) we're sure to start off with a deterministic | |
479 | // order (sorted by name). This'll mean that if two cgus have the same size | |
480 | // the stable sort below will keep everything nice and deterministic. | |
e74abb32 | 481 | codegen_units.sort_by_cached_key(|cgu| cgu.name().as_str()); |
ff7c6d11 | 482 | |
a7813a04 | 483 | // Merge the two smallest codegen units until the target size is reached. |
a7813a04 XL |
484 | while codegen_units.len() > target_cgu_count { |
485 | // Sort small cgus to the back | |
83c7162d | 486 | codegen_units.sort_by_cached_key(|cgu| cmp::Reverse(cgu.size_estimate())); |
ea8adc8c | 487 | let mut smallest = codegen_units.pop().unwrap(); |
a7813a04 XL |
488 | let second_smallest = codegen_units.last_mut().unwrap(); |
489 | ||
2c00a5a8 | 490 | second_smallest.modify_size_estimate(smallest.size_estimate()); |
ea8adc8c XL |
491 | for (k, v) in smallest.items_mut().drain() { |
492 | second_smallest.items_mut().insert(k, v); | |
a7813a04 | 493 | } |
dfeec247 XL |
494 | debug!( |
495 | "CodegenUnit {} merged in to CodegenUnit {}", | |
496 | smallest.name(), | |
497 | second_smallest.name() | |
498 | ); | |
a7813a04 XL |
499 | } |
500 | ||
b7449926 | 501 | let cgu_name_builder = &mut CodegenUnitNameBuilder::new(tcx); |
a7813a04 | 502 | for (index, cgu) in codegen_units.iter_mut().enumerate() { |
b7449926 | 503 | cgu.set_name(numbered_codegen_unit_name(cgu_name_builder, index)); |
a7813a04 XL |
504 | } |
505 | } | |
506 | ||
dfeec247 XL |
507 | fn place_inlined_mono_items<'tcx>( |
508 | initial_partitioning: PreInliningPartitioning<'tcx>, | |
509 | inlining_map: &InliningMap<'tcx>, | |
510 | ) -> PostInliningPartitioning<'tcx> { | |
a7813a04 | 511 | let mut new_partitioning = Vec::new(); |
0bf4aa26 | 512 | let mut mono_item_placements = FxHashMap::default(); |
3b2f2976 | 513 | |
dfeec247 XL |
514 | let PreInliningPartitioning { codegen_units: initial_cgus, roots, internalization_candidates } = |
515 | initial_partitioning; | |
a7813a04 | 516 | |
3b2f2976 XL |
517 | let single_codegen_unit = initial_cgus.len() == 1; |
518 | ||
519 | for old_codegen_unit in initial_cgus { | |
dc9dc135 | 520 | // Collect all items that need to be available in this codegen unit. |
0bf4aa26 | 521 | let mut reachable = FxHashSet::default(); |
ea8adc8c | 522 | for root in old_codegen_unit.items().keys() { |
a7813a04 XL |
523 | follow_inlining(*root, inlining_map, &mut reachable); |
524 | } | |
525 | ||
e74abb32 | 526 | let mut new_codegen_unit = CodegenUnit::new(old_codegen_unit.name()); |
a7813a04 | 527 | |
dc9dc135 | 528 | // Add all monomorphizations that are not already there. |
94b46f34 XL |
529 | for mono_item in reachable { |
530 | if let Some(linkage) = old_codegen_unit.items().get(&mono_item) { | |
dc9dc135 | 531 | // This is a root, just copy it over. |
94b46f34 | 532 | new_codegen_unit.items_mut().insert(mono_item, *linkage); |
a7813a04 | 533 | } else { |
94b46f34 | 534 | if roots.contains(&mono_item) { |
dfeec247 XL |
535 | bug!( |
536 | "GloballyShared mono-item inlined into other CGU: \ | |
537 | {:?}", | |
538 | mono_item | |
539 | ); | |
32a655c1 SL |
540 | } |
541 | ||
dc9dc135 | 542 | // This is a CGU-private copy. |
dfeec247 XL |
543 | new_codegen_unit |
544 | .items_mut() | |
545 | .insert(mono_item, (Linkage::Internal, Visibility::Default)); | |
3b2f2976 XL |
546 | } |
547 | ||
548 | if !single_codegen_unit { | |
549 | // If there is more than one codegen unit, we need to keep track | |
dc9dc135 | 550 | // in which codegen units each monomorphization is placed. |
94b46f34 | 551 | match mono_item_placements.entry(mono_item) { |
3b2f2976 XL |
552 | Entry::Occupied(e) => { |
553 | let placement = e.into_mut(); | |
554 | debug_assert!(match *placement { | |
e74abb32 XL |
555 | MonoItemPlacement::SingleCgu { cgu_name } => { |
556 | cgu_name != new_codegen_unit.name() | |
3b2f2976 | 557 | } |
94b46f34 | 558 | MonoItemPlacement::MultipleCgus => true, |
3b2f2976 | 559 | }); |
94b46f34 | 560 | *placement = MonoItemPlacement::MultipleCgus; |
3b2f2976 XL |
561 | } |
562 | Entry::Vacant(e) => { | |
94b46f34 | 563 | e.insert(MonoItemPlacement::SingleCgu { |
dfeec247 | 564 | cgu_name: new_codegen_unit.name(), |
3b2f2976 XL |
565 | }); |
566 | } | |
567 | } | |
a7813a04 XL |
568 | } |
569 | } | |
570 | ||
571 | new_partitioning.push(new_codegen_unit); | |
572 | } | |
573 | ||
3b2f2976 XL |
574 | return PostInliningPartitioning { |
575 | codegen_units: new_partitioning, | |
94b46f34 | 576 | mono_item_placements, |
3b2f2976 XL |
577 | internalization_candidates, |
578 | }; | |
a7813a04 | 579 | |
dfeec247 XL |
580 | fn follow_inlining<'tcx>( |
581 | mono_item: MonoItem<'tcx>, | |
582 | inlining_map: &InliningMap<'tcx>, | |
583 | visited: &mut FxHashSet<MonoItem<'tcx>>, | |
584 | ) { | |
94b46f34 | 585 | if !visited.insert(mono_item) { |
a7813a04 XL |
586 | return; |
587 | } | |
588 | ||
94b46f34 | 589 | inlining_map.with_inlining_candidates(mono_item, |target| { |
a7813a04 XL |
590 | follow_inlining(target, inlining_map, visited); |
591 | }); | |
592 | } | |
593 | } | |
594 | ||
dc9dc135 XL |
595 | fn internalize_symbols<'tcx>( |
596 | _tcx: TyCtxt<'tcx>, | |
597 | partitioning: &mut PostInliningPartitioning<'tcx>, | |
598 | inlining_map: &InliningMap<'tcx>, | |
599 | ) { | |
3b2f2976 XL |
600 | if partitioning.codegen_units.len() == 1 { |
601 | // Fast path for when there is only one codegen unit. In this case we | |
602 | // can internalize all candidates, since there is nowhere else they | |
603 | // could be accessed from. | |
604 | for cgu in &mut partitioning.codegen_units { | |
605 | for candidate in &partitioning.internalization_candidates { | |
dfeec247 | 606 | cgu.items_mut().insert(*candidate, (Linkage::Internal, Visibility::Default)); |
3b2f2976 XL |
607 | } |
608 | } | |
609 | ||
610 | return; | |
611 | } | |
612 | ||
94b46f34 | 613 | // Build a map from every monomorphization to all the monomorphizations that |
3b2f2976 | 614 | // reference it. |
0bf4aa26 | 615 | let mut accessor_map: FxHashMap<MonoItem<'tcx>, Vec<MonoItem<'tcx>>> = Default::default(); |
3b2f2976 XL |
616 | inlining_map.iter_accesses(|accessor, accessees| { |
617 | for accessee in accessees { | |
dfeec247 | 618 | accessor_map.entry(*accessee).or_default().push(accessor); |
3b2f2976 XL |
619 | } |
620 | }); | |
621 | ||
94b46f34 | 622 | let mono_item_placements = &partitioning.mono_item_placements; |
3b2f2976 XL |
623 | |
624 | // For each internalization candidates in each codegen unit, check if it is | |
625 | // accessed from outside its defining codegen unit. | |
626 | for cgu in &mut partitioning.codegen_units { | |
dfeec247 | 627 | let home_cgu = MonoItemPlacement::SingleCgu { cgu_name: cgu.name() }; |
3b2f2976 | 628 | |
ea8adc8c | 629 | for (accessee, linkage_and_visibility) in cgu.items_mut() { |
3b2f2976 XL |
630 | if !partitioning.internalization_candidates.contains(accessee) { |
631 | // This item is no candidate for internalizing, so skip it. | |
dfeec247 | 632 | continue; |
3b2f2976 | 633 | } |
94b46f34 | 634 | debug_assert_eq!(mono_item_placements[accessee], home_cgu); |
3b2f2976 XL |
635 | |
636 | if let Some(accessors) = accessor_map.get(accessee) { | |
dfeec247 XL |
637 | if accessors |
638 | .iter() | |
639 | .filter_map(|accessor| { | |
640 | // Some accessors might not have been | |
641 | // instantiated. We can safely ignore those. | |
642 | mono_item_placements.get(accessor) | |
643 | }) | |
644 | .any(|placement| *placement != home_cgu) | |
645 | { | |
3b2f2976 XL |
646 | // Found an accessor from another CGU, so skip to the next |
647 | // item without marking this one as internal. | |
dfeec247 | 648 | continue; |
3b2f2976 XL |
649 | } |
650 | } | |
651 | ||
652 | // If we got here, we did not find any accesses from other CGUs, | |
94b46f34 | 653 | // so it's fine to make this monomorphization internal. |
ea8adc8c | 654 | *linkage_and_visibility = (Linkage::Internal, Visibility::Default); |
3b2f2976 XL |
655 | } |
656 | } | |
657 | } | |
658 | ||
dc9dc135 XL |
659 | fn characteristic_def_id_of_mono_item<'tcx>( |
660 | tcx: TyCtxt<'tcx>, | |
661 | mono_item: MonoItem<'tcx>, | |
662 | ) -> Option<DefId> { | |
94b46f34 | 663 | match mono_item { |
ff7c6d11 | 664 | MonoItem::Fn(instance) => { |
cc61c64b XL |
665 | let def_id = match instance.def { |
666 | ty::InstanceDef::Item(def_id) => def_id, | |
dfeec247 XL |
667 | ty::InstanceDef::VtableShim(..) |
668 | | ty::InstanceDef::ReifyShim(..) | |
669 | | ty::InstanceDef::FnPtrShim(..) | |
670 | | ty::InstanceDef::ClosureOnceShim { .. } | |
671 | | ty::InstanceDef::Intrinsic(..) | |
672 | | ty::InstanceDef::DropGlue(..) | |
673 | | ty::InstanceDef::Virtual(..) | |
674 | | ty::InstanceDef::CloneShim(..) => return None, | |
cc61c64b XL |
675 | }; |
676 | ||
a7813a04 XL |
677 | // If this is a method, we want to put it into the same module as |
678 | // its self-type. If the self-type does not provide a characteristic | |
679 | // DefId, we use the location of the impl after all. | |
680 | ||
cc61c64b | 681 | if tcx.trait_of_item(def_id).is_some() { |
9e0c209e | 682 | let self_ty = instance.substs.type_at(0); |
74b04a01 | 683 | // This is a default implementation of a trait method. |
cc61c64b | 684 | return characteristic_def_id_of_type(self_ty).or(Some(def_id)); |
a7813a04 XL |
685 | } |
686 | ||
cc61c64b | 687 | if let Some(impl_def_id) = tcx.impl_of_method(def_id) { |
74b04a01 XL |
688 | if tcx.sess.opts.incremental.is_some() |
689 | && tcx.trait_id_of_impl(impl_def_id) == tcx.lang_items().drop_trait() | |
690 | { | |
691 | // Put `Drop::drop` into the same cgu as `drop_in_place` | |
692 | // since `drop_in_place` is the only thing that can | |
693 | // call it. | |
694 | return None; | |
695 | } | |
696 | // This is a method within an impl, find out what the self-type is: | |
0531ce1d XL |
697 | let impl_self_ty = tcx.subst_and_normalize_erasing_regions( |
698 | instance.substs, | |
699 | ty::ParamEnv::reveal_all(), | |
700 | &tcx.type_of(impl_def_id), | |
701 | ); | |
a7813a04 XL |
702 | if let Some(def_id) = characteristic_def_id_of_type(impl_self_ty) { |
703 | return Some(def_id); | |
704 | } | |
705 | } | |
706 | ||
cc61c64b | 707 | Some(def_id) |
a7813a04 | 708 | } |
0531ce1d | 709 | MonoItem::Static(def_id) => Some(def_id), |
416331ca | 710 | MonoItem::GlobalAsm(hir_id) => Some(tcx.hir().local_def_id(hir_id)), |
a7813a04 XL |
711 | } |
712 | } | |
713 | ||
e74abb32 | 714 | type CguNameCache = FxHashMap<(DefId, bool), Symbol>; |
b7449926 | 715 | |
dc9dc135 XL |
716 | fn compute_codegen_unit_name( |
717 | tcx: TyCtxt<'_>, | |
718 | name_builder: &mut CodegenUnitNameBuilder<'_>, | |
719 | def_id: DefId, | |
720 | volatile: bool, | |
721 | cache: &mut CguNameCache, | |
e74abb32 | 722 | ) -> Symbol { |
dc9dc135 | 723 | // Find the innermost module that is not nested within a function. |
b7449926 XL |
724 | let mut current_def_id = def_id; |
725 | let mut cgu_def_id = None; | |
dc9dc135 | 726 | // Walk backwards from the item we want to find the module for. |
b7449926 | 727 | loop { |
48663c56 XL |
728 | if current_def_id.index == CRATE_DEF_INDEX { |
729 | if cgu_def_id.is_none() { | |
730 | // If we have not found a module yet, take the crate root. | |
dfeec247 | 731 | cgu_def_id = Some(DefId { krate: def_id.krate, index: CRATE_DEF_INDEX }); |
b7449926 | 732 | } |
dfeec247 | 733 | break; |
48663c56 XL |
734 | } else if tcx.def_kind(current_def_id) == Some(DefKind::Mod) { |
735 | if cgu_def_id.is_none() { | |
736 | cgu_def_id = Some(current_def_id); | |
b7449926 | 737 | } |
48663c56 XL |
738 | } else { |
739 | // If we encounter something that is not a module, throw away | |
740 | // any module that we've found so far because we now know that | |
741 | // it is nested within something else. | |
742 | cgu_def_id = None; | |
b7449926 | 743 | } |
a7813a04 | 744 | |
48663c56 | 745 | current_def_id = tcx.parent(current_def_id).unwrap(); |
a7813a04 XL |
746 | } |
747 | ||
b7449926 XL |
748 | let cgu_def_id = cgu_def_id.unwrap(); |
749 | ||
dfeec247 | 750 | *cache.entry((cgu_def_id, volatile)).or_insert_with(|| { |
b7449926 XL |
751 | let def_path = tcx.def_path(cgu_def_id); |
752 | ||
dfeec247 | 753 | let components = def_path.data.iter().map(|part| part.data.as_symbol()); |
b7449926 | 754 | |
60c5eb7d | 755 | let volatile_suffix = volatile.then_some("volatile"); |
ff7c6d11 | 756 | |
b7449926 | 757 | name_builder.build_cgu_name(def_path.krate, components, volatile_suffix) |
dfeec247 | 758 | }) |
a7813a04 | 759 | } |
5bcae85e | 760 | |
dc9dc135 XL |
761 | fn numbered_codegen_unit_name( |
762 | name_builder: &mut CodegenUnitNameBuilder<'_>, | |
763 | index: usize, | |
e74abb32 | 764 | ) -> Symbol { |
b7449926 | 765 | name_builder.build_cgu_name_no_mangle(LOCAL_CRATE, &["cgu"], Some(index)) |
5bcae85e SL |
766 | } |
767 | ||
dc9dc135 XL |
768 | fn debug_dump<'a, 'tcx, I>(tcx: TyCtxt<'tcx>, label: &str, cgus: I) |
769 | where | |
770 | I: Iterator<Item = &'a CodegenUnit<'tcx>>, | |
771 | 'tcx: 'a, | |
5bcae85e SL |
772 | { |
773 | if cfg!(debug_assertions) { | |
774 | debug!("{}", label); | |
775 | for cgu in cgus { | |
e74abb32 | 776 | debug!("CodegenUnit {} estimated size {} :", cgu.name(), cgu.size_estimate()); |
5bcae85e | 777 | |
94b46f34 | 778 | for (mono_item, linkage) in cgu.items() { |
e1599b0c | 779 | let symbol_name = mono_item.symbol_name(tcx).name.as_str(); |
9e0c209e | 780 | let symbol_hash_start = symbol_name.rfind('h'); |
dfeec247 XL |
781 | let symbol_hash = |
782 | symbol_hash_start.map(|i| &symbol_name[i..]).unwrap_or("<no hash>"); | |
783 | ||
784 | debug!( | |
785 | " - {} [{:?}] [{}] estimated size {}", | |
786 | mono_item.to_string(tcx, true), | |
787 | linkage, | |
788 | symbol_hash, | |
789 | mono_item.size_estimate(tcx) | |
790 | ); | |
5bcae85e SL |
791 | } |
792 | ||
793 | debug!(""); | |
794 | } | |
795 | } | |
796 | } | |
a1dfa0c6 | 797 | |
dc9dc135 XL |
798 | #[inline(never)] // give this a place in the profiler |
799 | fn assert_symbols_are_distinct<'a, 'tcx, I>(tcx: TyCtxt<'tcx>, mono_items: I) | |
800 | where | |
801 | I: Iterator<Item = &'a MonoItem<'tcx>>, | |
802 | 'tcx: 'a, | |
a1dfa0c6 | 803 | { |
dfeec247 XL |
804 | let _prof_timer = tcx.prof.generic_activity("assert_symbols_are_distinct"); |
805 | ||
806 | let mut symbols: Vec<_> = | |
807 | mono_items.map(|mono_item| (mono_item, mono_item.symbol_name(tcx))).collect(); | |
dc9dc135 XL |
808 | |
809 | symbols.sort_by_key(|sym| sym.1); | |
810 | ||
811 | for pair in symbols.windows(2) { | |
812 | let sym1 = &pair[0].1; | |
813 | let sym2 = &pair[1].1; | |
814 | ||
815 | if sym1 == sym2 { | |
816 | let mono_item1 = pair[0].0; | |
817 | let mono_item2 = pair[1].0; | |
818 | ||
819 | let span1 = mono_item1.local_span(tcx); | |
820 | let span2 = mono_item2.local_span(tcx); | |
821 | ||
822 | // Deterministically select one of the spans for error reporting | |
823 | let span = match (span1, span2) { | |
824 | (Some(span1), Some(span2)) => { | |
dfeec247 | 825 | Some(if span1.lo().0 > span2.lo().0 { span1 } else { span2 }) |
dc9dc135 XL |
826 | } |
827 | (span1, span2) => span1.or(span2), | |
828 | }; | |
829 | ||
830 | let error_message = format!("symbol `{}` is already defined", sym1); | |
831 | ||
832 | if let Some(span) = span { | |
833 | tcx.sess.span_fatal(span, &error_message) | |
834 | } else { | |
835 | tcx.sess.fatal(&error_message) | |
836 | } | |
837 | } | |
838 | } | |
839 | } | |
840 | ||
416331ca XL |
841 | fn collect_and_partition_mono_items( |
842 | tcx: TyCtxt<'_>, | |
dc9dc135 | 843 | cnum: CrateNum, |
416331ca | 844 | ) -> (Arc<DefIdSet>, Arc<Vec<Arc<CodegenUnit<'_>>>>) { |
a1dfa0c6 XL |
845 | assert_eq!(cnum, LOCAL_CRATE); |
846 | ||
847 | let collection_mode = match tcx.sess.opts.debugging_opts.print_mono_items { | |
848 | Some(ref s) => { | |
849 | let mode_string = s.to_lowercase(); | |
850 | let mode_string = mode_string.trim(); | |
851 | if mode_string == "eager" { | |
852 | MonoItemCollectionMode::Eager | |
853 | } else { | |
854 | if mode_string != "lazy" { | |
dfeec247 XL |
855 | let message = format!( |
856 | "Unknown codegen-item collection mode '{}'. \ | |
a1dfa0c6 | 857 | Falling back to 'lazy' mode.", |
dfeec247 XL |
858 | mode_string |
859 | ); | |
a1dfa0c6 XL |
860 | tcx.sess.warn(&message); |
861 | } | |
862 | ||
863 | MonoItemCollectionMode::Lazy | |
864 | } | |
865 | } | |
866 | None => { | |
867 | if tcx.sess.opts.cg.link_dead_code { | |
868 | MonoItemCollectionMode::Eager | |
869 | } else { | |
870 | MonoItemCollectionMode::Lazy | |
871 | } | |
872 | } | |
873 | }; | |
874 | ||
dfeec247 | 875 | let (items, inlining_map) = collector::collect_crate_mono_items(tcx, collection_mode); |
a1dfa0c6 XL |
876 | |
877 | tcx.sess.abort_if_errors(); | |
878 | ||
dfeec247 XL |
879 | let (codegen_units, _) = tcx.sess.time("partition_and_assert_distinct_symbols", || { |
880 | sync::join( | |
881 | || { | |
882 | let strategy = if tcx.sess.opts.incremental.is_some() { | |
883 | PartitioningStrategy::PerModule | |
884 | } else { | |
885 | PartitioningStrategy::FixedUnitCount(tcx.sess.codegen_units()) | |
886 | }; | |
887 | ||
888 | partition(tcx, items.iter().cloned(), strategy, &inlining_map) | |
889 | .into_iter() | |
890 | .map(Arc::new) | |
891 | .collect::<Vec<_>>() | |
892 | }, | |
893 | || assert_symbols_are_distinct(tcx, items.iter()), | |
a1dfa0c6 | 894 | ) |
a1dfa0c6 XL |
895 | }); |
896 | ||
dfeec247 XL |
897 | let mono_items: DefIdSet = items |
898 | .iter() | |
899 | .filter_map(|mono_item| match *mono_item { | |
a1dfa0c6 XL |
900 | MonoItem::Fn(ref instance) => Some(instance.def_id()), |
901 | MonoItem::Static(def_id) => Some(def_id), | |
902 | _ => None, | |
dfeec247 XL |
903 | }) |
904 | .collect(); | |
a1dfa0c6 XL |
905 | |
906 | if tcx.sess.opts.debugging_opts.print_mono_items.is_some() { | |
907 | let mut item_to_cgus: FxHashMap<_, Vec<_>> = Default::default(); | |
908 | ||
909 | for cgu in &codegen_units { | |
910 | for (&mono_item, &linkage) in cgu.items() { | |
dfeec247 | 911 | item_to_cgus.entry(mono_item).or_default().push((cgu.name(), linkage)); |
a1dfa0c6 XL |
912 | } |
913 | } | |
914 | ||
915 | let mut item_keys: Vec<_> = items | |
916 | .iter() | |
917 | .map(|i| { | |
9fa01778 | 918 | let mut output = i.to_string(tcx, false); |
a1dfa0c6 XL |
919 | output.push_str(" @@"); |
920 | let mut empty = Vec::new(); | |
921 | let cgus = item_to_cgus.get_mut(i).unwrap_or(&mut empty); | |
0731742a | 922 | cgus.sort_by_key(|(name, _)| *name); |
a1dfa0c6 XL |
923 | cgus.dedup(); |
924 | for &(ref cgu_name, (linkage, _)) in cgus.iter() { | |
925 | output.push_str(" "); | |
926 | output.push_str(&cgu_name.as_str()); | |
927 | ||
928 | let linkage_abbrev = match linkage { | |
929 | Linkage::External => "External", | |
930 | Linkage::AvailableExternally => "Available", | |
931 | Linkage::LinkOnceAny => "OnceAny", | |
932 | Linkage::LinkOnceODR => "OnceODR", | |
933 | Linkage::WeakAny => "WeakAny", | |
934 | Linkage::WeakODR => "WeakODR", | |
935 | Linkage::Appending => "Appending", | |
936 | Linkage::Internal => "Internal", | |
937 | Linkage::Private => "Private", | |
938 | Linkage::ExternalWeak => "ExternalWeak", | |
939 | Linkage::Common => "Common", | |
940 | }; | |
941 | ||
942 | output.push_str("["); | |
943 | output.push_str(linkage_abbrev); | |
944 | output.push_str("]"); | |
945 | } | |
946 | output | |
947 | }) | |
948 | .collect(); | |
949 | ||
950 | item_keys.sort(); | |
951 | ||
952 | for item in item_keys { | |
953 | println!("MONO_ITEM {}", item); | |
954 | } | |
955 | } | |
956 | ||
957 | (Arc::new(mono_items), Arc::new(codegen_units)) | |
958 | } | |
959 | ||
9fa01778 | 960 | pub fn provide(providers: &mut Providers<'_>) { |
dfeec247 | 961 | providers.collect_and_partition_mono_items = collect_and_partition_mono_items; |
a1dfa0c6 XL |
962 | |
963 | providers.is_codegened_item = |tcx, def_id| { | |
dfeec247 | 964 | let (all_mono_items, _) = tcx.collect_and_partition_mono_items(LOCAL_CRATE); |
a1dfa0c6 XL |
965 | all_mono_items.contains(&def_id) |
966 | }; | |
967 | ||
968 | providers.codegen_unit = |tcx, name| { | |
969 | let (_, all) = tcx.collect_and_partition_mono_items(LOCAL_CRATE); | |
970 | all.iter() | |
e74abb32 | 971 | .find(|cgu| cgu.name() == name) |
a1dfa0c6 XL |
972 | .cloned() |
973 | .unwrap_or_else(|| panic!("failed to find cgu with name {:?}", name)) | |
974 | }; | |
975 | } |