Allow older LLVM versions to have missing components
This check was introduced by #77280 to ensure that all tests that are filtered by LLVM component are actually tested in CI. However this causes issues for new targets (e.g. #101069) where support is only available on the latest LLVM version.
This PR restricts the tests to only CI jobs that use the latest LLVM version.
Like #107044, this will let us track compatibility with LLVM 16 going
forward, especially after we eventually upgrade our own to the next.
This also drops `tidy` here and in `x86_64-gnu-llvm-15`, syncing with
that change in #106085.
This check was introduced by #77280 to ensure that all tests that are
filtered by LLVM component are actually tested in CI. However this
causes issues for new targets (e.g. #101069) where support is only
available on the latest LLVM version.
This PR restricts the tests to only CI jobs that use the latest LLVM
version.
These builders aren't particularly high on overall average CPU usage and finish in typically around
30 minutes. Cutting their core counts will hopefully not significantly increase wall-time while
cutting costs, allowing us to shift some of the wins into our slower builders.
This is a companion PR to rust-lang/promote-release#58, which moves the
relevant optimal code to rust-lang/promote-release. As mentioned in the
comments of that PR, this is expected to cut CI costs (and time, though
predominantly felt on fast builders) and reduce wasted resources due to
in-practice single-threaded compression not using the full 8+ vCPU
builders we have available.
Use `getentropy()` instead of `/dev/urandom` on Emscripten
`/dev/urandom` is usually available on Emscripten, except when using
the special `NODERAWFS` filesystem backend, which replaces all normal
filesystem access with direct Node.js operations.
Since this filesystem backend directly access the filesystem on the
OS, it is not recommended to depend on `/dev/urandom`, especially
when trying to run the Wasm binary on OSes that are not Unix-based.
This can be considered a non-functional change, since Emscripten
implements `/dev/urandom` in the same way as `getentropy()` when not
linking with `-sNODERAWFS`.
Rename 'src/bootstrap/native.rs' to llvm.rs
Fixed#108240
Renamed 'native.rs' to 'llvm.rs', also moved `TestHelpers` to `test.rs`. Replaced all the `native.rs` occurrences at `src/bootstrap` files to `llvm.rs`
Upgrade to LLVM 16, again
Relative to the previous attempt in https://github.com/rust-lang/rust/pull/107224:
* Update to GCC 8.5 on dist-x86_64-linux, to avoid std::optional ABI-incompatibility between libstdc++ 7 and 8.
* Cherry-pick 96df79af02.
* Cherry-pick 6fc670e5e3.
r? `@cuviper`
Fix cross-compiling with dlltool for raw-dylib
Fix for #103939
Issue Details:
When attempting to cross-compile using the `raw-dylib` feature and the GNU toolchain, rustc would attempt to find a cross-compiling version of dlltool (e.g., `i686-w64-mingw32-dlltool`). The has two issues 1) on Windows dlltool is always `dlltool` (no cross-compiling named versions exist) and 2) it only supported compiling to i686 and x86_64 resulting in ARM 32 and 64 compiling as x86_64.
Fix Details:
* On Windows always use the normal `dlltool` binary.
* Add the ARM64 cross-compiling dlltool name (support for this is coming: https://sourceware.org/bugzilla/show_bug.cgi?id=29964)
* Provide the `-m` argument to dlltool to indicate the target machine type.
(This is the first of two PRs to fix the remaining issues for the `raw-dylib` feature (#58713) that is blocking stabilization (#104218))
Add RANLIB_x86_64_unknown_illumos env for dist-x86_64-illumos dockerfile
close https://github.com/rust-lang/cc-rs/issues/798
We already set `AR_x86_64_unknown_illumos` in the dockerfile. So it is reasonable to set the `RANLIB_x86_64_unknown_illumos`.
Limit the number of parallel link jobs during LLVM build for mingw.
This PR is an attempt to unblock https://github.com/rust-lang/rust/pull/108355, which keeps failing while trying to link various LLVM artifacts on mingw runners. It looks like doing too many linking jobs might put too much load on the system? (Although I don't understand why the jobs are only failing for #108355 while they seem to pass for others)
r? infra-ci
While we don't use GCC for the LLVM build, we do use its libstdc++,
and there has been an std::optional ABI break in this version.
This makes the libLLVM.so for LLVM 16 ABI-incompatible with newer
libstdc++ versions, which we use on all other builders, and which
download-ci-llvm users are likely to use.
Add `dist.compression-profile` option to control compression speed
PR #108534 reduced the size of compressed archives, but (as expected) it also resulted in way longer compression times and memory usage during compression.
It's desirable to keep status quo (smaller archives but more CI usage), but it should also be configurable so that downstream users don't have to waste that much time on CI. As a data point, this resulted in doubling the time of Ferrocene's dist jobs, and required us to increase the RAM allocation for one of such jobs.
This PR adds a new `config.toml` setting, `dist.compression-profile`. The values can be:
* `fast`: equivalent to the gzip and xz preset of "1"
* `balanced`: equivalent to the gzip and xz preset of "6" (the CLI defaults as far as I'm aware)
* `best`: equivalent to the gzip present of "9", and our custom xz profile
The default has also been moved back to `balanced`, to try and avoid the compression time regression for downstream users. I don't feel too strongly on the default, and I'm open to changing it.
Also, for the `best` profile the XZ settings do not match the "9" preset used by the CLI, and it might be confusing. Should we create a `custom-rustc-ci`/`ultra` profile for that?
r? ``@Mark-Simulacrum``
Add tests for configure.py
I highly recommend reviewing this with whitespace disabled.
Notably, verifying that we generate valid toml relies on python 3.11 so
we can use `tomllib`, so this also switches`x86_64-gnu-llvm-14` (one of the PR builders) to use 3.11.
While fixing that, I noticed that we stopped testing python2.7 support on PR CI in https://github.com/rust-lang/rust/pull/106085. `@fee1-dead` `@pietroalbini` please be more careful in the future, there is no CI for CI itself that verifies we are testing everything we should be.
- Separate out functions so that each unit test doesn't create a file on disk
- Add a few unit tests
Move us to the new large runners pool
For now this keeps all the configuration identical (AFAICT) but we'll likely want to play with the specifics to move some of the slower builders to larger machines and the faster builders to smaller machines, likely reducing overall usage and improving CI times. I think we should leave that to later though, not worry about it just yet.
r? `@pietroalbini`
For now this keeps all the configuration identical (AFAICT) but we'll
likely want to play with the specifics to move some of the slower
builders to larger machines and the faster builders to smaller machines,
likely reducing overall usage and improving CI times.
Update host compiler to LLVM 16
Update the host compiler for dist-x86_64-linux to LLVM 16. In particular, this pulls in 1de305da42, which is needed to update Rust's own LLVM (https://github.com/rust-lang/rust/pull/107224).
This removes a dependency on ntapi 0.3.x which failed to compiled with
latest beta.
rust-lang/rustc-perf@93dc60d995 removed
that dependency in the upstream rustc-perf.
