rustc: Enable LTO and multiple codegen units
This commit is a refactoring of the LTO backend in Rust to support compilations
with multiple codegen units. The immediate result of this PR is to remove the
artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer
term this is intended to lay the groundwork for LTO with incremental compilation
and ultimately be the underpinning of ThinLTO support.
The problem here that needed solving is that when rustc is producing multiple
codegen units in one compilation LTO needs to merge them all together.
Previously only upstream dependencies were merged and it was inherently relied
on that there was only one local codegen unit. Supporting this involved
refactoring the optimization backend architecture for rustc, namely splitting
the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM
module has been optimized it may be blocked and queued up for LTO, and only
after LTO are modules code generated.
Non-LTO compilations should look the same as they do today backend-wise, we'll
spin up a thread for each codegen unit and optimize/codegen in that thread. LTO
compilations will, however, send the LLVM module back to the coordinator thread
once optimizations have finished. When all LLVM modules have finished optimizing
the coordinator will invoke the LTO backend, producing a further list of LLVM
modules. Currently this is always a list of one LLVM module. The coordinator
then spawns further work to run LTO and code generation passes over each module.
In the course of this refactoring a number of other pieces were refactored:
* Management of the bytecode encoding in rlibs was centralized into one module
instead of being scattered across LTO and linking.
* Some internal refactorings on the link stage of the compiler was done to work
directly from `CompiledModule` structures instead of lists of paths.
* The trans time-graph output was tweaked a little to include a name on each
bar and inflate the size of the bars a little
This commit is a refactoring of the LTO backend in Rust to support compilations
with multiple codegen units. The immediate result of this PR is to remove the
artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer
term this is intended to lay the groundwork for LTO with incremental compilation
and ultimately be the underpinning of ThinLTO support.
The problem here that needed solving is that when rustc is producing multiple
codegen units in one compilation LTO needs to merge them all together.
Previously only upstream dependencies were merged and it was inherently relied
on that there was only one local codegen unit. Supporting this involved
refactoring the optimization backend architecture for rustc, namely splitting
the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM
module has been optimized it may be blocked and queued up for LTO, and only
after LTO are modules code generated.
Non-LTO compilations should look the same as they do today backend-wise, we'll
spin up a thread for each codegen unit and optimize/codegen in that thread. LTO
compilations will, however, send the LLVM module back to the coordinator thread
once optimizations have finished. When all LLVM modules have finished optimizing
the coordinator will invoke the LTO backend, producing a further list of LLVM
modules. Currently this is always a list of one LLVM module. The coordinator
then spawns further work to run LTO and code generation passes over each module.
In the course of this refactoring a number of other pieces were refactored:
* Management of the bytecode encoding in rlibs was centralized into one module
instead of being scattered across LTO and linking.
* Some internal refactorings on the link stage of the compiler was done to work
directly from `CompiledModule` structures instead of lists of paths.
* The trans time-graph output was tweaked a little to include a name on each
bar and inflate the size of the bars a little
Remove new and index methods already implement for Idx
These are the rest of the repeated implementations for new and index methods. Follow up of https://github.com/rust-lang/rust/pull/44889
adding E0623 for return types - both parameters are anonymous
This is a fix for #44018
```
error[E0621]: explicit lifetime required in the type of `self`
--> $DIR/ex3-both-anon-regions-return-type-is-anon.rs:17:5
|
16 | fn foo<'a>(&self, x: &i32) -> &i32 {
| ---- ----
| |
| this parameter and the return type are
declared with different lifetimes...
17 | x
| ^ ...but data from `x` is returned here
error: aborting due to previous error
```
It also works for the below case where we have self as anonymous
```
error[E0623]: lifetime mismatch
--> src/test/ui/lifetime-errors/ex3-both-anon-regions-self-is-anon.rs:17:19
|
16 | fn foo<'a>(&self, x: &Foo) -> &Foo {
| ---- ----
| |
| this parameter and the return type are
declared with different lifetimes...
17 | if true { x } else { self }
| ^ ...but data from `x` is returned here
error: aborting due to previous error
```
r? @nikomatsakis
Currently, I have enabled E0621 where return type and self are anonymous, hence WIP.
First step toward implementing impl Trait in argument position
First step implementing #44721.
Add a flag to hir and ty TypeParameterDef and raise an error when using
explicit type parameters when calling a function using impl Trait in
argument position.
I don't know if there is a procedure to add an error code so I just took an available code. Is that ok ?
r? @nikomatsakis
rustc: Default 32 codegen units at O0
This commit changes the default of rustc to use 32 codegen units when compiling
in debug mode, typically an opt-level=0 compilation. Since their inception
codegen units have matured quite a bit, gaining features such as:
* Parallel translation and codegen enabling codegen units to get worked on even
more quickly.
* Deterministic and reliable partitioning through the same infrastructure as
incremental compilation.
* Global rate limiting through the `jobserver` crate to avoid overloading the
system.
The largest benefit of codegen units has forever been faster compilation through
parallel processing of modules on the LLVM side of things, using all the cores
available on build machines that typically have many available. Some downsides
have been fixed through the features above, but the major downside remaining is
that using codegen units reduces opportunities for inlining and optimization.
This, however, doesn't matter much during debug builds!
In this commit the default number of codegen units for debug builds has been
raised from 1 to 32. This should enable most `cargo build` compiles that are
bottlenecked on translation and/or code generation to immediately see speedups
through parallelization on available cores.
Work is being done to *always* enable multiple codegen units (and therefore
parallel codegen) but it requires #44841 at least to be landed and stabilized,
but stay tuned if you're interested in that aspect!
Point at signature on unused lint
```
warning: struct is never used: `Struct`
--> $DIR/unused-warning-point-at-signature.rs:22:1
|
22 | struct Struct {
| ^^^^^^^^^^^^^
```
Fix#33961.
Point at parameter type on E0301
On "the parameter type `T` may not live long enough" error, point to the
parameter type suggesting lifetime bindings:
```
error[E0310]: the parameter type `T` may not live long enough
--> $DIR/lifetime-doesnt-live-long-enough.rs:28:5
|
27 | struct Foo<T> {
| - help: consider adding an explicit lifetime bound `T: 'static`...
28 | foo: &'static T
| ^^^^^^^^^^^^^^^
|
note: ...so that the reference type `&'static T` does not outlive the data it points at
--> $DIR/lifetime-doesnt-live-long-enough.rs:28:5
|
28 | foo: &'static T
| ^^^^^^^^^^^^^^^
```
Fix#36700.
Initial support for `..=` syntax
#28237
This PR adds `..=` as a synonym for `...` in patterns and expressions.
Since `...` in expressions was never stable, we now issue a warning.
cc @durka
r? @aturon
Allow unused extern crate again
This is a partial revert of #42588. There is a usability concern reported in #44294 that was not considered in the discussion of the PR, so I would like to back this out of 1.21. As is, I think users would have a worse and more confusing experience with this lint enabled by default. We can re-enabled once there are better diagnostics or the case in #44294 does not trigger the lint.
use param_env on the trait_cache key
We bailed from making trans_fulfill_obligation return `Option` or `Result`, just made it less prone to crashing outside trans
r? @nikomatsakis
Add a flag to hir and ty TypeParameterDef and raise an error when using
explicit type parameters when calling a function using impl Trait in
argument position.
This commit changes the default of rustc to use 32 codegen units when compiling
in debug mode, typically an opt-level=0 compilation. Since their inception
codegen units have matured quite a bit, gaining features such as:
* Parallel translation and codegen enabling codegen units to get worked on even
more quickly.
* Deterministic and reliable partitioning through the same infrastructure as
incremental compilation.
* Global rate limiting through the `jobserver` crate to avoid overloading the
system.
The largest benefit of codegen units has forever been faster compilation through
parallel processing of modules on the LLVM side of things, using all the cores
available on build machines that typically have many available. Some downsides
have been fixed through the features above, but the major downside remaining is
that using codegen units reduces opportunities for inlining and optimization.
This, however, doesn't matter much during debug builds!
In this commit the default number of codegen units for debug builds has been
raised from 1 to 32. This should enable most `cargo build` compiles that are
bottlenecked on translation and/or code generation to immediately see speedups
through parallelization on available cores.
Work is being done to *always* enable multiple codegen units (and therefore
parallel codegen) but it requires #44841 at least to be landed and stabilized,
but stay tuned if you're interested in that aspect!
Require rlibs for dependent crates when linking static executables
This handles the case for `CrateTypeExecutable` and `+crt_static`. I reworked the match block to avoid duplicating the `attempt_static` and error checking code again (this case would have been a copy of the `CrateTypeCdylib`/`CrateTypeStaticlib` case).
On `linux-musl` targets where `std` was built with `crt_static = false` in `config.toml`, this change brings the test suite from entirely failing to mostly passing.
This change should not affect behavior for other crate types, or for targets which do not respect `+crt_static`.
encode region::Scope using fewer bytes
Now that region::Scope is no longer interned, its size is more important. This PR encodes region::Scope in 8 bytes instead of 12, which should speed up region inference somewhat (perf testing needed) and should improve the margins on #36799 by 64MB (that's not a lot, I did this PR mostly to speed up region inference).
This is a perf-sensitive PR. Please don't roll me up.
r? @eddyb
This is based on #44743 so I could get more accurate measurements on #36799.
This is a partial revert of #42588. There is a usability concern
reported in #44294 that was not considered in the discussion of the PR,
so I would like to back this out of 1.21. As is, I think users would
have a worse and more confusing experience with this lint enabled by
default. We can re-enabled once there are better diagnostics or the case
in #44294 does not trigger the lint.
Move effect-checking to MIR
This allows emitting lints from MIR and moves the effect-checking pass to work on it.
I'll make `repr(packed)` misuse unsafe in a separate PR.
r? @eddyb
put empty generic lists behind a pointer
This reduces the size of hir::Expr from 128 to 88 bytes (!) and shaves
200MB out of #36799.
This is a performance-sensitive PR so please don't roll it up.
r? @eddyb
