Update the minimum external LLVM to 14
With this change, we'll have stable support for LLVM 14 through 16 (pending release).
For reference, the previous increase to LLVM 13 was #100460.
Port pgo.sh to Python
This PR ports the `pgo.sh` multi stage build file from bash to Python, to make it easier to add new functionality and gather statistics. Main changes:
1) `pgo.sh` rewritten from Bash to Python. Jump from ~200 Bash LOC to ~650 Python LOC. Bash is, unsurprisingly, more concise for running scripts and binaries.
2) Better logging. Each separate stage is now clearly separated in logs, and the logs can be quickly grepped to find out which stage has completed or failed, and how long it took.
3) Better statistics. At the end of the run, there is now a table that shows the duration of the individual stages, along with a percentual ratio of the total workflow run:
```
2023-01-15T18:13:49.9896916Z stage-build INFO: Timer results
2023-01-15T18:13:49.9902185Z ---------------------------------------------------------
2023-01-15T18:13:49.9902605Z Build rustc (LLVM PGO): 1815.67s (21.47%)
2023-01-15T18:13:49.9902949Z Gather profiles (LLVM PGO): 418.73s ( 4.95%)
2023-01-15T18:13:49.9903269Z Build rustc (rustc PGO): 584.46s ( 6.91%)
2023-01-15T18:13:49.9903835Z Gather profiles (rustc PGO): 806.32s ( 9.53%)
2023-01-15T18:13:49.9904154Z Build rustc (LLVM BOLT): 1662.92s (19.66%)
2023-01-15T18:13:49.9904464Z Gather profiles (LLVM BOLT): 715.18s ( 8.46%)
2023-01-15T18:13:49.9914463Z Final build: 2454.00s (29.02%)
2023-01-15T18:13:49.9914798Z Total duration: 8457.27s
2023-01-15T18:13:49.9915305Z ---------------------------------------------------------
```
A sample run can be seen [here](https://github.com/rust-lang/rust/actions/runs/3923980164/jobs/6707932029).
I tried to keep the code compatible with Python 3.6 and don't use dependencies, which required me to reimplement some small pieces of functionality (like formatting bytes). I suppose that it shouldn't be so hard to upgrade to a newer Python or install dependencies in the CI container, but I'd like to avoid it if it won't be needed.
The code is in a single file `stage-build.py`, so it's a bit cluttered. I can also separate it into multiple files, although having it in a single file has some benefits. The code could definitely be nicer, but I'm a bit wary of introducing a lot of abstraction and similar stuff, as long as the code is stuffed into a single file.
Currently, the Python pipeline should faithfully mirror the bash pipeline one by one. After this PR, I'd like to try to optimize it, e.g. by caching the LLVM builds on S3.
r? `@Mark-Simulacrum`
Fix maintainer validation message
State clearly why maintainer verification is not activating, ruling out the user thinking they misconfigured something.
Enable sanitizers for s390x-linux
Include sanitizers supported by LLVM on s390x (asan, lsan, msan, tsan) in the target definition, as well as in the compiletest supported list.
Build sanitizer runtime for the target. Enable sanitizers in the CI.
Include sanitizers supported by LLVM on s390x (asan, lsan, msan, tsan)
in the target definition, as well as in the compiletest supported list.
Build sanitizer runtime for the target. Enable sanitizers in the CI.
Update the wasi-libc used for the wasm32-wasi target
This commit updates the wasi-libc revision used to build with the wasm32-wasi target. This notably pulls in WebAssembly/wasi-libc#377 which is needed to fix a use case I've been working on recently. This should be a relatively small update hopefully and is not expected to have any user impact.
This commit updates the wasi-libc revision used to build with the
wasm32-wasi target. This notably pulls in WebAssembly/wasi-libc#377
which is needed to fix a use case I've been working on recently. This
should be a relatively small update hopefully and is not expected to
have any user impact.
Use CI LLVM in `test-various` builder
It was disabled because it needs `lld`, but since #104748 was merged it is no longer needed.
This will speed this test, since it no longer needs to build LLVM.
Historically, Rust's Fuchsia targets have been labeled x86_64-fuchsia
and aarch64-fuchsia. However, they should technically contain vendor
information. This CL changes Fuchsia's target triples to include the
"unknown" vendor since Clang now does normalization and handles all
triple spellings.
This was previously attempted in #90510, which was closed due to
inactivity.
This duplicates mingw-check into two jobs where one job
runs `tidy` only while the other job does not. The tidy
job will not cancel other jobs on failure.
Run `x test tidy` sooner in mingw-check
It takes less time to run than the other tests and is more likely to fail. `expand-yaml-anchors` is still run first to make sure the CI files are internally consistent.
Note that changing to `--stage 0` doesn't actually do anything since bootstrap tools are always built with the bootstrap compiler, this just makes it less confusing.
cc https://github.com/rust-lang/rust/pull/105058/commits/83bab41b5b2d4752d187dd91b05c88ac74cf3783
This commit updates the CI definitions to use the most recent Android
LTS NDK release: r25b. Changes since the last NDK used by Rust negate
the need to generate "standalone toolchains" and newer NDKs can be used
in-place.
See https://developer.android.com/ndk/guides/other_build_systems#overview
It takes less time to run than the other tests and is more likely to fail.
`expand-yaml-anchors` is still run first to make sure the CI files are internally consistent.
Enable profiler in dist-powerpc64le-linux
Build the profiler runtime to allow using -C profile-generate and -C instrument-coverage on POWER little endian systems.
I have verified locally that the runtime builds and the profiler is working fine on the platform.
Similar pull request for a different system: https://github.com/rust-lang/rust/pull/104304
