rustc: Stabilize options for pipelined compilation
This commit stabilizes options in the compiler necessary for Cargo to
enable "pipelined compilation" by default. The concept of pipelined
compilation, how it's implemented, and what it means for rustc are
documented in #60988. This PR is coupled with a PR against Cargo
(rust-lang/cargo#7143) which updates Cargo's support for pipelined
compliation to rustc, and also enables support by default in Cargo.
(note that the Cargo PR cannot land until this one against rustc lands).
The technical changes performed here were to stabilize the functionality
proposed in #60419 and #60987, the underlying pieces to enable pipelined
compilation support in Cargo. The issues have had some discussion during
stabilization, but the newly stabilized surface area here is:
* A new `--json` flag was added to the compiler.
* The `--json` flag can be passed multiple times.
* The value of the `--json` flag is a comma-separated list of
directives.
* The `--json` flag cannot be combined with `--color`
* The `--json` flag must be combined with `--error-format=json`
* The acceptable list of directives to `--json` are:
* `diagnostic-short` - the `rendered` field of diagnostics will have a
"short" rendering matching `--error-format=short`
* `diagnostic-rendered-ansi` - the `rendered` field of diagnostics
will be colorized with ansi color codes embedded in the string field
* `artifacts` - JSON blobs will be emitted for artifacts being emitted
by the compiler
The unstable `-Z emit-artifact-notifications` and `--json-rendered`
flags have also been removed during this commit as well.
Closes#60419Closes#60987Closes#60988
rustc: Update wasm32 support for LLVM 9
This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:
* The compiler now no longer manually inserts the `producers` section,
instead relying on LLVM to do so. LLVM uses the `llvm.ident` metadata
for the `processed-by` directive (which is now emitted on the wasm
target in this PR) and it uses debuginfo to figure out what `language`
to put in the `producers` section.
* Threaded WebAssembly code now requires different flags to be passed
with LLD. In LLD we now pass:
* `--shared-memory` - required since objects are compiled with
atomics. This also means that the generated memory will be marked as
`shared`.
* `--max-memory=1GB` - required with the `--shared-memory` argument
since shared memories in WebAssembly must have a maximum size. The
1GB number is intended to be a conservative estimate for rustc, but
it should be overridable with `-C link-arg` if necessary.
* `--passive-segments` - this has become the default for multithreaded
memory, but when compiling a threaded module all data segments need
to be marked as passive to ensure they don't re-initialize memory
for each thread. This will also cause LLD to emit a synthetic
function to initialize memory which users will have to arrange to
call.
* The `__heap_base` and `__data_end` globals are explicitly exported
since they're now hidden by default due to the `--export` flags we
pass to LLD.
This commit stabilizes options in the compiler necessary for Cargo to
enable "pipelined compilation" by default. The concept of pipelined
compilation, how it's implemented, and what it means for rustc are
documented in #60988. This PR is coupled with a PR against Cargo
(rust-lang/cargo#7143) which updates Cargo's support for pipelined
compliation to rustc, and also enables support by default in Cargo.
(note that the Cargo PR cannot land until this one against rustc lands).
The technical changes performed here were to stabilize the functionality
proposed in #60419 and #60987, the underlying pieces to enable pipelined
compilation support in Cargo. The issues have had some discussion during
stabilization, but the newly stabilized surface area here is:
* A new `--json` flag was added to the compiler.
* The `--json` flag can be passed multiple times.
* The value of the `--json` flag is a comma-separated list of
directives.
* The `--json` flag cannot be combined with `--color`
* The `--json` flag must be combined with `--error-format=json`
* The acceptable list of directives to `--json` are:
* `diagnostic-short` - the `rendered` field of diagnostics will have a
"short" rendering matching `--error-format=short`
* `diagnostic-rendered-ansi` - the `rendered` field of diagnostics
will be colorized with ansi color codes embedded in the string field
* `artifacts` - JSON blobs will be emitted for artifacts being emitted
by the compiler
The unstable `-Z emit-artifact-notifications` and `--json-rendered`
flags have also been removed during this commit as well.
Closes#60419Closes#60987Closes#60988
Add support for UWP targets
Hi,
This pull request aims at adding support for UWP (Universal Windows Apps) platform.
A few notes:
- This requires a very recent mingw-w64 version (containing this commit and the previous related ones: e8c433c871 (diff-eefdfbfe9cec5f4ebab88c9a64d423a9))
- This was tested using LLVM/clang rather than gcc, and so far it assumes that LLVM/clang will be the native compiler. This is mostly due to the fact that the support for exceptions/stack unwinding for UWP got much more attention in libunwind
- The "uwp" part of the target needs support for it in the `cc-rs` & `backtrace-rs` crates. I'll create the MR there right after I submit this one and will link everything together, but I'm not sure what's the correct way of dealing with external dependencies in the context of rust
- Enabling import libraries and copying them across stages requires a change in cargo, for which I'll open a MR right after I submit this one as well
- The i686 stack unwinding is unsupported for now, because LLVM assumes SjLj, while rust seems to assume SEH will be used. I'm unsure how to fix this
Also, this is my first encounter with rust, so please bear with my code, it might not feel so idiomatic or even correct :)
I'm pretty sure there's a way of doing things in a cleaner way when it comes to win/c.rs, maybe having a UWP & desktop specific modules, and import those conditionally? It doesn't feel right to sprinkle `#[cfg(...)]` all over the place
Off course, I'll gladly update anything you see fit (to the extent of my abilities/knowledge :) )!
Thanks,
So far it is assumed that using a DLL as a -l parameter argument is ok,
but the assumption doesn't hold when compiling the native code with
llvm.
In which case, an import library is required, so let's build one
This also requires the cargo counterpart to add the import library in
the stamp files, at least when compiling libstd. Otherwise, the files
don't get uplifted
This commit moves `thread_local!` on WebAssembly targets to using the
`#[thread_local]` attribute in LLVM. This was recently implemented
upstream and is [in the process of being documented][dox]. This change
only takes affect if modules are compiled with `+atomics` which is
currently unstable and a pretty esoteric method of compiling wasm
artifacts.
This "new power" of the wasm toolchain means that the old
`wasm-bindgen-threads` feature of the standard library can be removed
since it should now be possible to create a fully functioning threaded
wasm module without intrusively dealing with libstd symbols or
intrinsics. Yay!
[dox]: https://github.com/WebAssembly/tool-conventions/pull/116
This commit brings in a number of minor updates for rustc's support for
the wasm target which has changed in the LLVM 9 update. Notable updates
include:
* The compiler now no longer manually inserts the `producers` section,
instead relying on LLVM to do so. LLVM uses the `llvm.ident` metadata
for the `processed-by` directive (which is now emitted on the wasm
target in this PR) and it uses debuginfo to figure out what `language`
to put in the `producers` section.
* Threaded WebAssembly code now requires different flags to be passed
with LLD. In LLD we now pass:
* `--shared-memory` - required since objects are compiled with
atomics. This also means that the generated memory will be marked as
`shared`.
* `--max-memory=1GB` - required with the `--shared-memory` argument
since shared memories in WebAssembly must have a maximum size. The
1GB number is intended to be a conservative estimate for rustc, but
it should be overridable with `-C link-arg` if necessary.
* `--passive-segments` - this has become the default for multithreaded
memory, but when compiling a threaded module all data segments need
to be marked as passive to ensure they don't re-initialize memory
for each thread. This will also cause LLD to emit a synthetic
function to initialize memory which users will have to arrange to
call.
* The `__heap_base` and `__data_end` globals are explicitly exported
since they're now hidden by default due to the `--export` flags we
pass to LLD.
rustc_codegen_ssa: fix range check in codegen_get_discr.
Fixes#61696, see https://github.com/rust-lang/rust/issues/61696#issuecomment-505473018 for more details.
In short, I had wanted to use `x - a <= b - a` to check whether `x` is in `a..=b` (as it's 1 comparison instead of 2 *and* `b - a` is guaranteed to fit in the same data type, while `b` itself might not), but I ended up with `x - a + c <= b - a + c` instead, because `x - a + c` was the final value needed.
That latter comparison is equivalent to checking that `x` is in `(a - c)..=b`, i.e. it also includes `(a - c)..a`, not just `a..=b`, so if `c` is not `0`, it will cause false positives.
This presented itself as the non-niche ("dataful") variant sometimes being treated like a niche variant, in the presence of uninhabited variants (which made `c`, aka the index of the first niche variant, arbitrarily large).
r? @nagisa, @rkruppe or @oli-obk
The old struct tail functions did not deal with `<T as Trait>::A` and `impl
Trait`, at least not explicitly. (We didn't notice this bug before because it
is only exposed when the tail (post deep normalization) is not `Sized`, so it
was a rare case to deal with.)
For post type-checking (i.e. during codegen), there is now
`struct_tail_erasing_lifetimes` and `struct_lockstep_tails_erasing_lifetimes`,
which each take an additional `ParamEnv` argument to drive normalization.
For pre type-checking cases where normalization is not needed, there is
`struct_tail_without_normalization`. (Currently, the only instance of this is
`Expectation::rvalue_hint`.)
All of these new entrypoints work by calling out to common helper routines.
The helpers are parameterized over a closure that handles the normalization.
Prepare for LLVM 9 update
Main changes:
* In preparation for opaque pointer types, the `byval` attribute now takes a type. As such, the argument type needs to be threaded through to the function/callsite attribute application logic.
* On ARM the `+fp-only-sp` and `+d16` features have become `-fp64` and `-d32`. I've switched the target definitions to use the new names, but also added bidirectional emulation so either can be used on any LLVM version for backwards compatibility.
* The datalayout can now specify function pointer alignment. In particular on ARM `Fi8` is specified, which means that function pointer alignment is independent of function alignment. I've added this to our datalayouts to match LLVM (which is something we check) and strip the fnptr alignment for older LLVM versions.
* The fmul/fadd reductions now always respect the accumulator (including for unordered reductions), so we should pass the identity instead of undef.
Open issues:
* https://reviews.llvm.org/D62106 causes linker errors with ld.bdf due to https://sourceware.org/bugzilla/show_bug.cgi?id=24784. To avoid this I've enabled `RelaxELFRelocations`, which results in a GOTPCRELX relocation for `__tls_get_addr` and avoids the issue. However, this is likely not acceptable because relax relocations are not supported by older linker versions. We may need an LLVM option to keep using PLT for `__tls_get_addr` despite `RtLibUseGOT`.
The corresponding llvm-project PR is https://github.com/rust-lang/llvm-project/pull/19.
r? @ghost
Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!
The (almost) culmination of HirIdification
It's finally over.
This PR removes old `FIXME`s and renames some functions so that the `HirId` variant has the shorter name.
All that remains (and rightfully so) is stuff in `resolve`, `save_analysis` and (as far as I can tell) in a few places where we can't replace `NodeId` with `HirId`.
Implement another internal lints
cc #49509
This adds ~~two~~ one internal lint~~s~~:
1. LINT_PASS_IMPL_WITHOUT_MACRO: Make sure, that the `{declare,impl}_lint_pass` macro is used to implement lint passes. cc #59669
2. ~~USAGE_OF_TYCTXT_AND_SPAN_ARGS: item 2 on the list in #49509~~
~~With 2. I wasn't sure, if this lint should be applied everywhere. That means a careful review of 0955835 would be great. Also 73fb9b4 allows this lint on some functions. Should I also apply this lint there?~~
TODO (not directly relevant for review):
- [ ] https://github.com/rust-lang/rust/pull/59316#discussion_r280186517 (not sure yet, if this works or how to query for `rustc_private`, since it's not in [`Features`](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/feature_gate/struct.Features.html) 🤔 cc @eddyb)
- [x] https://github.com/rust-lang/rust/pull/61735#discussion_r292389870
- [x] Check explicitly for the `{declare,impl}_lint_pass!` macros
r? @oli-obk
This is useful for getting the total compilation time at the end.
To do this, the patch changes `print_time_passes_entry` to not increment
the depth, which means that `print_time_passes_entry_internal` is no
longer needed.
Stabilize support for Profile-guided Optimization
This PR makes profile-guided optimization available via the `-C profile-generate` / `-C profile-use` pair of commandline flags and adds end-user documentation for the feature to the [rustc book](https://doc.rust-lang.org/rustc/). The PR thus ticks the last two remaining checkboxes of the [stabilization tracking issue](https://github.com/rust-lang/rust/issues/59913).
From the tracking issue:
> Profile-guided optimization (PGO) is a common optimization technique for ahead-of-time compilers. It works by collecting data about a program's typical execution (e.g. probability of branches taken, typical runtime values of variables, etc) and then uses this information during program optimization for things like inlining decisions, machine code layout, or indirect call promotion.
If you are curious about how this can be used, there is a rendered version of the documentation this PR adds available [here](
https://github.com/michaelwoerister/rust/blob/stabilize-pgo/src/doc/rustc/src/profile-guided-optimization.md).
r? @alexcrichton
cc @rust-lang/compiler
