Validate lint docs separately.
This addresses some concerns raised in https://github.com/rust-lang/rust/pull/76549#issuecomment-727638552 about errors with the lint docs being confusing and cumbersome. Errors from validating the lint documentation were being generated during `x.py doc` (and `x.py dist`), since extraction and validation are being done in a single step. This changes it so that extraction and validation are separated, so that `x.py doc` will not error if there is a validation problem, and tests are moved to `x.py test src/tools/lint-docs`.
This includes the following changes:
* Separate validation to `x.py test`.
* Added some more documentation on how to more easily modify and test the docs.
* Added more help to the error messages to hopefully provide more information on how to fix things.
The first commit just moves the code around, so you may consider looking at the other commits for a smaller diff.
x.py: allow a custom string appended to the version
This adds `rust.description` to the config as a descriptive string to be
appended to `rustc --version` output, which is also used in places like
debuginfo `DW_AT_producer`. This may be useful for supplementary build
information, like distro-specific package versions.
For example, in Fedora 33, `gcc --version` outputs:
gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6)
With this change, we can add similar vendor info to `rustc --version`.
Make compiletest testing use the local sysroot
We already set `compiletest` to use the local sysroot in #68019, but
that missed the configuration for testing `compiletest` itself.
fix handling the default config for profiler and sanitizers
#78354 don't handle the case that user don't add any target-specific config in `[target.*]` of `config.toml`:
```toml
changelog-seen = 2
[llvm]
link-shared = true
[build]
sanitizers = true
profiler = true
[install]
[rust]
[dist]
```
The previes code handle the default config in `Config::prase()`:
```rust
target.sanitizers = cfg.sanitizers.unwrap_or(build.sanitizers.unwrap_or_default());
target.profiler = cfg.profiler.unwrap_or(build.profiler.unwrap_or_default());
config.target_config.insert(TargetSelection::from_user(&triple), target);
```
In this case, `toml.target` don't contain any target, so the above code won't execute. Instead, a default `Target` is insert in c919f490bb/src/bootstrap/sanity.rs (L162-L166)
The default value for `bool` is false, hence the issue in #79124
This fix change the type of `sanitizers` and `profiler` to `Option<bool>`, so the default value is `None`, and fallback config is handled in `Config::sanitizers_enabled` and `Config::profiler_enabled`
fix#79124
cc `@Mark-Simulacrum` `@richkadel`
Historically the stable tarballs were named after the version number of
the specific tool, instead of the version number of Rust. For example,
both of the following tarballs were part of the same release:
rustc-1.48.0-x86_64-unknown-linux-gnu.tar.xz
cargo-0.49.0-x86_64-unknown-linux-gnu.tar.xz
PR #77336 changed the dist code to instead use Rust's version number for
all the tarballs, regardless of the tool they contain:
rustc-1.48.0-x86_64-unknown-linux-gnu.tar.xz
cargo-1.48.0-x86_64-unknown-linux-gnu.tar.xz
Because of that there is no need anymore to have a separate `cargo`
field in src/stage0.txt, as the Cargo version will always be the same as
the rustc version. This PR removes the field, simplifying the code and
the maintenance work required while producing releases.
This adds `rust.description` to the config as a descriptive string to be
appended to `rustc --version` output, which is also used in places like
debuginfo `DW_AT_producer`. This may be useful for supplementary build
information, like distro-specific package versions.
For example, in Fedora 33, `gcc --version` outputs:
gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6)
With this change, we can add similar vendor info to `rustc --version`.
Install CI llvm into the library directory
In other words, my concern in https://github.com/rust-lang/rust/issues/78932#issuecomment-725781767 was perfectly justified by something we were already doing. For now just special case CI LLVM, but in the future we may want a more general fix.
Fixes#79071.
r? `@alexcrichton`
Avoid installing external LLVM dylibs
If the LLVM was externally provided, then we don't currently copy artifacts into
the sysroot. This is not necessarily the right choice (in particular, it will
require the LLVM dylib to be in the linker's load path at runtime), but the
common use case for external LLVMs is distribution provided LLVMs, and in that
case they're usually in the standard search path (e.g., /usr/lib) and copying
them here is going to cause problems as we may end up with the wrong files and
isn't what distributions want.
This behavior may be revisited in the future though.
Fixes#78932.
Include llvm-as in llvm-tools-preview component
Including `llvm-as` adds the ability to include assembly language fragments that can be inlined using LTO while making sure the correct version of LLVM is always used.
If the LLVM was externally provided, then we don't currently copy artifacts into
the sysroot. This is not necessarily the right choice (in particular, it will
require the LLVM dylib to be in the linker's load path at runtime), but the
common use case for external LLVMs is distribution provided LLVMs, and in that
case they're usually in the standard search path (e.g., /usr/lib) and copying
them here is going to cause problems as we may end up with the wrong files and
isn't what distributions want.
This behavior may be revisited in the future though.
Support enable/disable sanitizers/profiler per target
This PR add options under `[target.*]` of `config.toml` which can enable or disable sanitizers/profiler runtime for corresponding target.
If these options are empty, the global options under `[build]` will take effect.
Fix#78329
Infer the default host target from the host toolchain if possible
- `beta-x86_64-unknown-linux-gnu` has beta stripped
- `rustc2` is ignored
This fixes ongoing issues where x.py will detect the wrong host triple
between MSVC and GNU.
I don't think this will break anyone's workflow - I'd be very surprised if you a) had no `[build]` section in `config.toml`, b) had rustc installed, and c) expected the default target to be something other than the default target used by `rustc`. But I could be wrong - I'm happy to hear user stories :)
Fixes https://github.com/rust-lang/rust/issues/78150.
r? ``@Mark-Simulacrum``
cc ``@Lokathor``
Compile rustc crates with the initial-exec TLS model
This should produce more efficient code, with fewer calls to
__tls_get_addr. The tradeoff is that libraries using it won't work with
dlopen, but that shouldn't be a problem for rustc's internal libraries.
Honor the rustfmt setting in config.toml
Prior to this, setting the rustfmt configuration was ignored:
```
% mkdir example
% cd example
% ../configure --set build.rustfmt=/usr/bin/true
% ../x.py fmt
./x.py fmt is not supported on this channel
failed to run: /Users/shep/Projects/rust/example/build/bootstrap/debug/bootstrap fmt
Build completed unsuccessfully in 0:00:01
```
And after:
```
% ../x.py fmt
Build completed successfully in 0:00:11
```
r? `@Mark-Simulacrum`
Fix run-make tests running when LLVM is disabled
The `--cc`, `--cxx`, `--cflags` and `--ar` flags were only passed to compiletest when `builder.config.llvm_enabled()` returned true. This is preventing me from running the tests on cg_clif.
Prior to this, setting the rustfmt configuration was ignored:
```
% mkdir example
% cd example
% ../configure --set build.rustfmt=/usr/bin/true
% ../x.py fmt
./x.py fmt is not supported on this channel
failed to run: /Users/shep/Projects/rust/example/build/bootstrap/debug/bootstrap fmt
Build completed unsuccessfully in 0:00:01
```
And after:
```
% ../x.py fmt
Build completed successfully in 0:00:11
```
This should produce more efficient code, with fewer calls to
__tls_get_addr. The tradeoff is that libraries using it won't work with
dlopen, but that shouldn't be a problem for tools or for our own
internal libraries.
Co-authored-by: Mark Rousskov <mark.simulacrum@gmail.com>
Print a summary of which test suite failed
Especially on CI, where cross-compiling is common and single builder may end up
with multiple hosts and multiple targets, it can be annoying to scroll back to
the nearest start of test marker. This prints out a summary of the test suite
being run directly in compiletest.
For example, on a mir-opt failure, this would show something like this:
```
failures:
[mir-opt] mir-opt/while-storage.rs
test result: FAILED. 140 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out
Some tests failed in compiletest suite=mir-opt mode=mir-opt host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
```
Fixes#78517
Fix `x.py clippy`
I don't think this ever worked.
Fixes https://github.com/rust-lang/rust/issues/77309. `--fix` support is a work in progress, but works for a very small subset of `libtest`.
This works by using the host `cargo-clippy` driver; it does not use `stage0.txt` at all. To mitigate confusion from this, it gives an error if you don't have `rustc +nightly` as the default rustc in `$PATH`. Additionally, it means that bootstrap can't set `RUSTC`; this makes it no longer possible for clippy to detect the sysroot itself. Instead, bootstrap passes the sysroot to cargo.
r? `@ghost`
Here's the error for rustdoc:
```
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
error: no library targets found in package `rustdoc-tool`
```
Fix panic in bootstrap for non-workspace path dependencies.
If you add a `path` dependency to a `Cargo.toml` that is located outside of the workspace, then the `in_tree_crates` function can panic because it finds a path dependency that is not defined (since it uses `cargo metadata --no-deps`). This fixes it by skipping over those entries, which are usually not things you select on the command-line.
Fixes#78617
