Run the tools builder on all PRs
Previously, it would only run on changes to subtrees, submodules, or select directories. That made it so that changes to the compiler that broke tools would only be detected on a full bors merge. This makes it so the tools builder runs by default, making it easier to catch breaking changes to clippy (which was the most affected).
r? ``@Mark-Simulacrum`` cc ``@pietroalbini`` ``@flip1995`` ``@m-ou-se``
Add nicer output to PGO build timer
This PR modifies the timer used in the PGO build script to contain nicer, hierarchical output of the individual build steps. It's not trivial to test locally, so I'll fire up a dist build right away.
r? ``@Mark-Simulacrum``
Previously, it would only run on changes to subtrees, submodules, or select directories.
That made it so that changes to the compiler that broke tools would only be detected on a full bors merge.
This makes it so the tools builder runs by default, making it easier to catch breaking changes to clippy (which was the most effected).
Rollup of 6 pull requests
Successful merges:
- #106575 (Suggest `move` in nested closure when appropriate)
- #106805 (Suggest `{var:?}` when finding `{?:var}` in inline format strings)
- #107500 (Add tests to assert current behavior of large future sizes)
- #107598 (Fix benchmarks in library/core with black_box)
- #107602 (Parse and recover from type ascription in patterns)
- #107608 (Use triple rather than arch for fuchsia test-runner)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Enable Cargo's sparse protocol in CI
This enables the sparse protocol in CI in order to exercise and dogfood it. This is intended test the production server in a real-world situation.
Closes#107342
Port pgo.sh to Python
This PR ports the `pgo.sh` multi stage build file from bash to Python, to make it easier to add new functionality and gather statistics. Main changes:
1) `pgo.sh` rewritten from Bash to Python. Jump from ~200 Bash LOC to ~650 Python LOC. Bash is, unsurprisingly, more concise for running scripts and binaries.
2) Better logging. Each separate stage is now clearly separated in logs, and the logs can be quickly grepped to find out which stage has completed or failed, and how long it took.
3) Better statistics. At the end of the run, there is now a table that shows the duration of the individual stages, along with a percentual ratio of the total workflow run:
```
2023-01-15T18:13:49.9896916Z stage-build INFO: Timer results
2023-01-15T18:13:49.9902185Z ---------------------------------------------------------
2023-01-15T18:13:49.9902605Z Build rustc (LLVM PGO): 1815.67s (21.47%)
2023-01-15T18:13:49.9902949Z Gather profiles (LLVM PGO): 418.73s ( 4.95%)
2023-01-15T18:13:49.9903269Z Build rustc (rustc PGO): 584.46s ( 6.91%)
2023-01-15T18:13:49.9903835Z Gather profiles (rustc PGO): 806.32s ( 9.53%)
2023-01-15T18:13:49.9904154Z Build rustc (LLVM BOLT): 1662.92s (19.66%)
2023-01-15T18:13:49.9904464Z Gather profiles (LLVM BOLT): 715.18s ( 8.46%)
2023-01-15T18:13:49.9914463Z Final build: 2454.00s (29.02%)
2023-01-15T18:13:49.9914798Z Total duration: 8457.27s
2023-01-15T18:13:49.9915305Z ---------------------------------------------------------
```
A sample run can be seen [here](https://github.com/rust-lang/rust/actions/runs/3923980164/jobs/6707932029).
I tried to keep the code compatible with Python 3.6 and don't use dependencies, which required me to reimplement some small pieces of functionality (like formatting bytes). I suppose that it shouldn't be so hard to upgrade to a newer Python or install dependencies in the CI container, but I'd like to avoid it if it won't be needed.
The code is in a single file `stage-build.py`, so it's a bit cluttered. I can also separate it into multiple files, although having it in a single file has some benefits. The code could definitely be nicer, but I'm a bit wary of introducing a lot of abstraction and similar stuff, as long as the code is stuffed into a single file.
Currently, the Python pipeline should faithfully mirror the bash pipeline one by one. After this PR, I'd like to try to optimize it, e.g. by caching the LLVM builds on S3.
r? `@Mark-Simulacrum`
Rollup of 11 pull requests
Successful merges:
- #96763 (Fix maintainer validation message)
- #106540 (Insert whitespace to avoid ident concatenation in suggestion)
- #106763 (print why a test was ignored if its the only test specified)
- #106769 (libtest: Print why a test was ignored if it's the only test specified.)
- #106798 (Implement `signum` with `Ord`)
- #107006 (Output tree representation on thir-tree)
- #107078 (Update wording of invalid_doc_attributes docs.)
- #107169 (Pass `--locked` to the x test tidy call)
- #107431 (docs: remove colon from time header)
- #107432 (rustdoc: remove unused class `has-srclink`)
- #107448 (When stamp doesn't exist, should say Error, and print path to stamp file)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Fix maintainer validation message
State clearly why maintainer verification is not activating, ruling out the user thinking they misconfigured something.
Enable sanitizers for s390x-linux
Include sanitizers supported by LLVM on s390x (asan, lsan, msan, tsan) in the target definition, as well as in the compiletest supported list.
Build sanitizer runtime for the target. Enable sanitizers in the CI.
Include sanitizers supported by LLVM on s390x (asan, lsan, msan, tsan)
in the target definition, as well as in the compiletest supported list.
Build sanitizer runtime for the target. Enable sanitizers in the CI.
Update the wasi-libc used for the wasm32-wasi target
This commit updates the wasi-libc revision used to build with the wasm32-wasi target. This notably pulls in WebAssembly/wasi-libc#377 which is needed to fix a use case I've been working on recently. This should be a relatively small update hopefully and is not expected to have any user impact.
This commit updates the wasi-libc revision used to build with the
wasm32-wasi target. This notably pulls in WebAssembly/wasi-libc#377
which is needed to fix a use case I've been working on recently. This
should be a relatively small update hopefully and is not expected to
have any user impact.
Use CI LLVM in `test-various` builder
It was disabled because it needs `lld`, but since #104748 was merged it is no longer needed.
This will speed this test, since it no longer needs to build LLVM.
Historically, Rust's Fuchsia targets have been labeled x86_64-fuchsia
and aarch64-fuchsia. However, they should technically contain vendor
information. This CL changes Fuchsia's target triples to include the
"unknown" vendor since Clang now does normalization and handles all
triple spellings.
This was previously attempted in #90510, which was closed due to
inactivity.
This duplicates mingw-check into two jobs where one job
runs `tidy` only while the other job does not. The tidy
job will not cancel other jobs on failure.
Re-enable ThinLTO for rustc on `x86_64-apple-darwin`
ThinLTO was disabled on x64 mac in https://github.com/rust-lang/rust/pull/105646 because of the https://github.com/rust-lang/rust/issues/105637 regression.
It was later discovered that the issue was present on other targets as well, as the mac revert was already landing. The linux/win reverts, however, did not land before the root cause was identified.
https://github.com/rust-lang/rust/pull/105800 fixed the underlying issue in `-Zdylib-lto` handling, and the x64 msvc and linux targets are now fixed, ICEs are using the correct `rustc_driver` panic hook.
This PR re-enables ThinLTO on mac for improved perf now that the issue should be fixed everywhere.
Run `x test tidy` sooner in mingw-check
It takes less time to run than the other tests and is more likely to fail. `expand-yaml-anchors` is still run first to make sure the CI files are internally consistent.
Note that changing to `--stage 0` doesn't actually do anything since bootstrap tools are always built with the bootstrap compiler, this just makes it less confusing.
cc https://github.com/rust-lang/rust/pull/105058/commits/83bab41b5b2d4752d187dd91b05c88ac74cf3783
This commit updates the CI definitions to use the most recent Android
LTS NDK release: r25b. Changes since the last NDK used by Rust negate
the need to generate "standalone toolchains" and newer NDKs can be used
in-place.
See https://developer.android.com/ndk/guides/other_build_systems#overview
It takes less time to run than the other tests and is more likely to fail.
`expand-yaml-anchors` is still run first to make sure the CI files are internally consistent.
