Extract `rustc_ast_passes`, move gating, & refactor linting
Based on https://github.com/rust-lang/rust/pull/67770.
This PR extracts a crate `rustc_ast_passes`:
- `ast_validation.rs`, which is contributed by `rustc_passes` (now only has HIR based passes) -- the goal here is to improve recompilation of the parser,
- `feature_gate.rs`, which is contributed by `syntax` and performs post-expansion-gating & final erroring for pre-expansion gating,
- `show_span`, which is contributed by `syntax`.
To facilitate this, we first have to also:
- Move `{leveled_}feature_err{_err}` from `syntax::feature_gate::check` into `rustc_session::parse`.
- Move `get_features` into `rustc_parse::config`, the only place it is used.
- Move some some lint datatypes and traits, e.g. `LintBuffer`, `BufferedEarlyLint`, `BuiltinLintDiagnostics`, `LintPass`, and `LintArray` into `rustc_session::lint`.
- Move all the hard-wired lint `static`s into `rustc_session::lint::builtin`.
build-std compatible sanitizer support
### Motivation
When using `-Z sanitizer=*` feature it is essential that both user code and
standard library is instrumented. Otherwise the utility of sanitizer will be
limited, or its use will be impractical like in the case of memory sanitizer.
The recently introduced cargo feature build-std makes it possible to rebuild
standard library with arbitrary rustc flags. Unfortunately, those changes alone
do not make it easy to rebuild standard library with sanitizers, since runtimes
are dependencies of std that have to be build in specific environment,
generally not available outside rustbuild process. Additionally rebuilding them
requires presence of llvm-config and compiler-rt sources.
The goal of changes proposed here is to make it possible to avoid rebuilding
sanitizer runtimes when rebuilding the std, thus making it possible to
instrument standard library for use with sanitizer with simple, although
verbose command:
```
env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu
```
### Implementation
* Sanitizer runtimes are no long packed into crates. Instead, libraries build
from compiler-rt are used as is, after renaming them into `librusc_rt.*`.
* rustc obtains runtimes from target libdir for default sysroot, so that
they are not required in custom build sysroots created with build-std.
* The runtimes are only linked-in into executables to address issue #64629.
(in previous design it was hard to avoid linking runtimes into static
libraries produced by rustc as demonstrated by sanitizer-staticlib-link
test, which still passes despite changes made in #64780).
cc @kennytm, @japaric, @firstyear, @choller
Extract `rustc_hir` out of `rustc`
The new crate contains:
```rust
pub mod def;
pub mod def_id;
mod hir;
pub mod hir_id;
pub mod itemlikevisit;
pub mod pat_util;
pub mod print;
mod stable_hash_impls;
pub use hir::*;
pub use hir_id::*;
pub use stable_hash_impls::HashStableContext;
```
Remains to be done in follow-up PRs:
- Move `rustc::hir::map` into `rustc_hir_map` -- this has to be a separate crate due to the `dep_graph` (blocked on https://github.com/rust-lang/rust/pull/67761).
- Move references to `rustc::hir` to `rustc_hir` where possible.
cc https://github.com/rust-lang/rust/issues/65031
r? @Zoxc
Support `-Z ui-testing=yes/no`
`ui-testing` is now a boolean option (`-Z ui-testing=yes/no`) and can be specified multiple times with later values overriding earlier values (`-Z ui-testing=yes -Z ui-testing=no` == `-Z ui-testing=no`), so it can be set in a hierarchical way, e.g. UI testing infra may enable it by default with specific tests being able to opt-out.
This way we can remove the special opt-out support from `compiletest`.
Inspired by https://github.com/rust-lang/rust/pull/67709.
config.rs has reached the 3000 line tidy limit, this commit moves
command line option definitions into a new file - options.rs, and
leaves the rest of configuration infrastructure in config.rs.
This reverts commit 3ed3b8bb7b, reversing
changes made to 99b89533d4.
We will reland a similar patch at a future date but for now we should get a nightly
released in a few hours with the parallel patch, so this should be
reverted to make sure that the next nightly is not parallel-enabled.
This avoids the problems of high thread counts (i.e., contention in the
kernel on the jobserver pipe due to thundering herd of readers) while
stil giving rustc some parallelism to work with.
Fix `-Z print-type-sizes`'s handling of zero-sized fields.
Currently, the type `struct S { x: u32, y: u32, tag: () }` is
incorrectly described like this:
```
print-type-size type: `S`: 8 bytes, alignment: 4 bytes
print-type-size field `.x`: 4 bytes
print-type-size field `.tag`: 0 bytes, offset: 0 bytes, alignment: 1 bytes
print-type-size padding: 4 bytes
print-type-size field `.y`: 4 bytes, alignment: 4 bytes
```
Specifically:
- The `padding` line is wrong. (There is no padding.)
- The `offset` and `alignment` on the `.tag` line shouldn't be printed.
The problem is that multiple fields can end up with the same offset, and
the printing code doesn't handle this correctly.
This commit fixes it by adjusting the field sorting so that zero-sized fields
are dealt with before non-zero-sized fields. With that in place, the
printing code works correctly.
The commit also corrects the "something is very wrong" comment.
The new output looks like this:
```
print-type-size type: `S`: 8 bytes, alignment: 4 bytes
print-type-size field `.tag`: 0 bytes
print-type-size field `.x`: 4 bytes
print-type-size field `.y`: 4 bytes
```
r? @pnkfelix
Add options to --extern flag.
This changes the `--extern` flag so that it can take a series of options that changes its behavior. The general syntax is `[opts ':'] name ['=' path]` where `opts` is a comma separated list of options. Two options are supported, `priv` which replaces `--extern-private` and `noprelude` which avoids adding the crate to the extern prelude.
```text
--extern priv:mylib=/path/to/libmylib.rlib
--extern noprelude:alloc=/path/to/liballoc.rlib
```
`noprelude` is to be used by Cargo's build-std feature in order to use `--extern` to reference standard library crates.
This also includes a second commit which adds the `aux-crate` directive to compiletest. I can split this off into a separate PR if desired, but it helps with defining these kinds of tests. It is based on #54020, and can be used in the future to replace and simplify some of the Makefile tests.
Currently, the type `struct S { x: u32, y: u32, tag: () }` is
incorrectly described like this:
```
print-type-size type: `S`: 8 bytes, alignment: 4 bytes
print-type-size field `.x`: 4 bytes
print-type-size field `.tag`: 0 bytes, offset: 0 bytes, alignment: 1 bytes
print-type-size padding: 4 bytes
print-type-size field `.y`: 4 bytes, alignment: 4 bytes
```
Specifically:
- The `padding` line is wrong. (There is no padding.)
- The `offset` and `alignment` on the `.tag` line shouldn't be printed.
The problem is that multiple fields can end up with the same offset, and
the printing code doesn't handle this correctly.
This commit fixes it by adjusting the field sorting so that zero-sized fields
are dealt with before non-zero-sized fields. With that in place, the
printing code works correctly.
The commit also corrects the "something is very wrong" comment.
The new output looks like this:
```
print-type-size type: `S`: 8 bytes, alignment: 4 bytes
print-type-size field `.tag`: 0 bytes
print-type-size field `.x`: 4 bytes
print-type-size field `.y`: 4 bytes
```
This commit breaks early-lint registration, which will be fixed in the
next commit. This movement will allow essentially all crates in the compiler
tree to declare lints (though not lint passes).
