We don't distribute a miri build for beta/stable so it needs to be kept
optional. In the future it likely makes sense to switch the miri
*artifacts* to always be built, but the rustup component to not be
included -- this will avoid some of this pain.
Set `download-ci-llvm = "if-available"` by default when `channel = dev`
See https://github.com/rust-lang/compiler-team/issues/566. The motivation for changing the default is to avoid downloading and building LLVM when someone runs `x build` before running `x setup`. The motivation for only doing it on `channel = "dev"` is to avoid breaking distros or users installing from source. It works because `dev` is also the default channel.
The diff looks larger than it is; most of it is moving the `llvm` branch below the `rust` so `config.channel` is set.
r? `@Mark-Simulacrum` cc `@oli-obk` `@bjorn3` `@cuviper`
Add prototype to generate `COPYRIGHT` from REUSE metadata
This PR adds a prototype to generate the `COPYRIGHT` file from the metadata gathered with REUSE. There are two new tools:
* `src/tools/collect-license-metadata` invokes REUSE, parses its output and stores a concise JSON representation of the metadata in `src/etc/license-metadata.json`.
* `src/tools/generate-copyright` parses the metadata generated above, (in the future will) gather crate dependencies metadata, and renders the `COPYRIGHT.md` file.
Note that since the contents of those files are currently incorrect, rather than outputting in the paths above, the files will be stored in `build/` and not committed. This will be changed once we're confident about the metadata.
Eventually, `src/etc/license-metadata.json` will be committed into the repository and verified to be up to date by CI (similar to our GitHub Actions configuration), to avoid having people install REUSE on their local machine in most cases.
You can see the (incorrect) generated files in https://gist.github.com/pietroalbini/3f3f22b6f9cc8533abf7494b6a50cf97.
r? `@pnkfelix`
Ensure required submodules at the same time as updating existing submodules
In practice, this would always happen at the same time, but putting them next to each other makes that more obvious and ensures it doesn't change in the future. It also avoids the difference affecting `cargo metadata` somehow.
This is based on https://github.com/rust-lang/rust/pull/104952 for convenience to avoid merge conflicts, but doesn't depend on that PR.
Streamline the user experience for `x.py setup`
## Don't update submodules for x setup
Before, the submodule handling was very jank and would update *between two interactive prompts*:
```
; x setup
Building rustbuild
Finished dev [unoptimized] target(s) in 0.05s
Welcome to the Rust project! What do you want to do with x.py?
a) library: Contribute to the standard library
Please choose one (a/b/c/d/e): a
Updating submodule library/backtrace
Submodule 'library/backtrace' (https://github.com/rust-lang/backtrace-rs.git) registered for path 'library/backtrace'
error: you asked `x.py` to setup a new config file, but one already exists at `config.toml`
Build completed unsuccessfully in 0:00:02
```
That's not a great user experience because you need to wait a long time between prompts.
It would be possible to move the submodule handling either before or after the prompt, but it seems
better to just not require submodules to be checked out at all, to minimize the time spend waiting
just to create a new configuration.
## Revamp the order setup executes
- Create `config.toml` last. It's the most likely to error, and used to stop later steps from executing
- Don't print an error message + exit if the git hook already exists; that's expected
In practice, this would always happen at the same time, but putting them next to each other makes that more obvious and ensures it doesn't change in the future.
It also avoids the difference avoiding `cargo metadata` somehow.
- Create `config.toml` last. It's the most likely to error, and used to stop later steps from executing
- Don't print an error message + exit if the git hook already exists; that's expected
Before, the submodule handling was very jank and would update *between two interactive prompts*:
```
; x setup
Building rustbuild
Finished dev [unoptimized] target(s) in 0.05s
Welcome to the Rust project! What do you want to do with x.py?
a) library: Contribute to the standard library
Please choose one (a/b/c/d/e): a
Updating submodule library/backtrace
Submodule 'library/backtrace' (https://github.com/rust-lang/backtrace-rs.git) registered for path 'library/backtrace'
error: you asked `x.py` to setup a new config file, but one already exists at `config.toml`
Build completed unsuccessfully in 0:00:02
```
That's not a great user experience because you need to wait a long time between prompts.
It would be possible to move the submodule handling either before or after the prompt, but it seems
better to just not require submodules to be checked out at all, to minimize the time spend waiting
just to create a new configuration.
rustbuild: Don't build doc::SharedAssets when building JSON docs.
Previously, running `./x doc library/core/ --json` on a plain build would panic bootstrap.
```
$ ./x doc library/core/ --json
Building rustbuild
Blocking waiting for file lock on package cache
Compiling bootstrap v0.0.0 (/home/nixon/dev/rust/rust/src/bootstrap)
Finished dev [unoptimized] target(s) in 4.47s
thread 'main' panicked at 'fs::write(&version_info, &info) failed with No such file or directory (os error 2) ("/home/nixon/dev/rust/rust/build/x86_64-unknown-linux-gnu/doc/version_info.html")', doc.rs:410:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:04
```
Becuase the `SharedAssets` step assumes that the HTML out dir has been created. This isn't true for JSON.
The fix is to not build shared assets when doing a JSON doc build, as it doesn't need them.
r? ``@jyn514``
``@rustbot`` modify labels: +A-rustdoc-json
Don't set `is_preview` for clippy and rustfmt
These have been shipped on stable for many years now and it would be very disruptive to ever remove them.
Remove the `-preview` suffix from their dist components.
Based on https://github.com/rust-lang/rust/pull/102565.
These have been shipped on stable for many years now and it would be very disruptive to ever remove them.
Remove the `-preview` suffix from their dist components.
copy doc output files by format
This pr provides copying doc outputs by checking output format without removing output directory on each trigger.
Resolves#103785
Followup to https://github.com/rust-lang/rust/pull/103778
Sorry for the quick succession but this fixes one more building issue for Tier 3 `windows-gnullvm` that I have previously missed, and it would be nice to have it in the release.
Use clang for the UEFI targets
This fixes an issue where the C and asm sources built by compiler_builtins were being compiled as ELF objects instead of PE objects. This wasn't noticed before because it doesn't cause compiler_builtins or rustc to fail to build. You only see a failure when a program is built that references one of the symbols in an ELF object.
Compiling with clang fixes this because the cc crate converts the UEFI targets into Windows targets that clang understands, causing it to produce PE objects.
Also update compiler_builtins to 0.1.84 to pull in some necessary fixes for compiling the UEFI targets with clang.
Fixes https://github.com/rust-lang/rust/issues/104326
Revert "Update CI to use Android NDK r25b"
This reverts commit bf7f1ca316 (pull request #102332).
The relevant discussion can be found in #103673, where it was agreed that more time is needed to warn the community of the upcoming breakage.
This PR is for the `master` branch, where a conflict was recently introduced due to 6d8160261f. The conflict is in `cc_detect.rs`, where the code that corrects the target triple was moved to a new function called `ndk_compiler()`. This puts the old logic in the `ndk_compiler` function, and assumes that it works properly in the other location where that code is being called. I would appreciate review from ``@pietroalbini`` to understand how we can test that the reverted logic is also suitable for the additional use case (seems to be related to setting `cc` and `cxx`). I've confirmed already that with these changes I can compile for `armv7-linux-androideabi`, `aarch64-linux-android`, `i686-linux-android`, and `x86_64-linux-android` using `x.py`.
A separate revert for the `beta` branch will be required, since the original change has already made it to beta. The beta revert is available at 3fa0d94674, but I'm not sure of the process for staging that PR.
If clang isn't the C compiler used for the UEFI targets, or if the wrong
`--target` is passed to clang, we will get ELF objects in some
rlibs. This will cause problems at link time when trying to compile a
UEFI program that uses any of those objects. Add a check to the dist
step for UEFI targets that reads each rlib with the `object` crate and
fails with an error if any non-COFF objects are found.
See https://github.com/rust-lang/compiler-team/issues/566.
The motivation for changing the default is to avoid downloading and building LLVM when someone runs `x build` before running `x setup`.
The motivation for only doing it on `channel = "dev"` is to avoid breaking distros or users installing from source. It works because `dev` is also the default channel.
The diff looks larger than it is; most of it is moving the `llvm` branch below the `rust` so `config.channel` is set.
fix sysroot issue which appears for ci downloaded rustc
Currently when compiler is downloaded rather than compiled, sysroot is being `ci-rustc-sysroot` because of 7eef946fc0/src/bootstrap/compile.rs (L1125-L1131) this.
And rustdoc is overriding the downloaded one at the end of the process.
With the condition I add, we simply check if the current compiler stage is target build stage, if so use the proper sysroot instead of `ci-rustc-sysroot`.
Resolves#103206
Partial support for running UI tests with `download-rustc`
Right now trying to run UI tests with `download-rustc` results in a bunch of test failures, requiring someone who wants to only work on tests to also build the full compiler. This PR **partially** addresses the problem by solving a lot of the errors (but not all).
* This installs the `rust-src` component on CI toolchains, since the test output depends on whether the standard library source code is installed; We can't just symlink the current source because the `rustc-dev` component already includes the compiler sources, but not the library sources, and mixing things is worse IMO.
* This ensures the `$SRC_DIR` normalization done by compiletest correctly handles the source code added above.
* This unconditionally sets `remap-prefix` to the prefix used in the downloaded toolchain, to ensure compiletest normalizes it.
