Add QEMU test for x86_64-unknown-uefi
The UEFI targets don't have std support yet, so the normal tests don't work. However, we can compile a simple no-std program and run it under QEMU to at least check that the target compiles, links, and runs.
Tested locally with: `src/ci/docker/run.sh x86_64-uefi`
ci: Bring back ninja for dist builders
The primary reason for this is that make can result in a substantial under utilization of parallelism (noticed while testing on a workstation), mostly due to the submake structure preventing good dependency tracking and scheduling.
In f758c7b2a7 (Debian 6 doesn't have ninja, so use make for the dist builds) llvm.ninja was disabled due to lack of distro package. This is no longer the case with the CentOS 7 base, so bring ninja back for a performance boost.
This reverts commit 3acb505ee5
(PR #101833).
The changes in this commit caused several bugs or at least
incompatibilies. For now we're reverting this commit and will re-land it
alongside fixes for those bugs.
update Miri
I had to use a hacked version of josh to create this, so let's be careful with merging this and maybe wait a bit to see if the josh issue becomes more clear. But the history looks good to me, we are not adding duplicates of rustc commits that were previously mirrored to Miri.
Also I want to add some cross-testing of Miri in x.py.
Upgrade dist-mips*-linux to ubuntu:22.04 + crosstool-ng
These have no change in compatibility, still Linux 4.4 and glibc 2.23.
The main motivation for upgrading is that LLVM 16 will require at least GCC 7.1. Using crosstool-ng lets us choose our own toolchain versions, and then the Ubuntu version doesn't matter so much, just for the host compilation while we cross-compile.
The primary reason for this is that make can result in a substantial
under utilization of parallelism, mostly due to the submake structure
preventing good dependency tracking and scheduling.
In f758c7b2a7 (Debian 6 doesn't have ninja, so use make for the dist builds)
llvm.ninja was disabled due to lack of distro package. This is no longer the
case with the CentOS 7 base, so bring ninja back for a performance boost.
Improve rustdoc-gui search-color test
Thanks to the add of "functions" in `browser-ui-test`, we can start to reduce the size of the scripts. It'll be very useful for all color checks.
r? `@notriddle`
Use BOLT in CI to optimize LLVM
This PR adds an optimization step in the Linux `dist` CI pipeline that uses [BOLT](https://github.com/llvm/llvm-project/tree/main/bolt) to optimize the `libLLVM.so` library built by boostrap.
Steps:
- [x] Use LLVM 15 as a bootstrap compiler and use it to build BOLT
- [x] Compile LLVM with support for relocations (`-DCMAKE_SHARED_LINKER_FLAGS="-Wl,-q"`)
- [x] Gather profile data using instrumented LLVM
- [x] Apply profile to LLVM that has already been PGOfied
- [x] Run with BOLT profiling on more benchmarks
- [x] Decide on the order of optimization (PGO -> BOLT?)
- [x] Decide how we should get `bolt` (currently we use the host `bolt`)
- [x] Clean up
The latest perf results can be found [here](https://github.com/rust-lang/rust/pull/94381#issuecomment-1258269440). The current CI build time with BOLT applied is around 1h 55 minutes.
Update CI to use Android NDK r25b
This commit updates the CI definitions to use the most recent Android LTS NDK release: r25b. Changes since the last NDK used by Rust negate the need to generate "standalone toolchains" and newer NDKs can be used in-place.
See https://developer.android.com/ndk/guides/other_build_systems#overview
Update browser UI test version
It added the possibility to concatenate strings and numbers and updated the `goto` command so it doesn't stand on its own anymore.
r? ````@notriddle````
Compressed debug is enabled by default for gas (assembly) on Linux/x86
targets, and we started building our own in #102530, but that made our
`compiler_builtins` incompatible with binutils < 2.32. Add an explicit
option to disable that in our crosstool-ng config. Fixes#102703.
Miri sync
This is a Miri sync created with my experimental fork of josh. We should probably not merge this yet, but we can use this to check if the sync looks the way it should.
r? `@oli-obk`
This commit updates the CI definitions to use the most recent Android
LTS NDK release: r25b. Changes since the last NDK used by Rust negate
the need to generate "standalone toolchains" and newer NDKs can be used
in-place.
See https://developer.android.com/ndk/guides/other_build_systems#overview
The system GCC 5 in ubuntu:16.04 will be too old to compile LLVM 16, so
we need an upgrade. To avoid raising the minimum glibc requirements for
`i586-unknown-linux-gnu`, this target is converted to a crosstool-ng
toolchain, *relaxing* it to the same Linux 3.2 / glibc 2.17 minimum we
use elsewhere. The musl targets still use Ubuntu's system toolchain, but
this doesn't have the same compatibility concerns.
Make the `c` feature for `compiler-builtins` an explicit opt-in
Its build script doesn't support cross-compilation. I tried fixing it, but the cc crate itself doesn't appear to support cross-compiling to windows either unless you use the -gnu toolchain:
```
error occurred: Failed to find tool. Is `lib.exe` installed?
```
Fixes https://github.com/rust-lang/rust/issues/101172.
The build script for `compiler_builtins` doesn't support cross-compilation. I tried fixing it, but the cc crate itself
doesn't appear to support cross-compiling to windows either unless you use the -gnu toolchain:
```
error occurred: Failed to find tool. Is `lib.exe` installed?
```
Rather than trying to fix it or special-case the platforms without bugs,
make it opt-in instead of automatic.
Distribute bootstrap in CI
This pre-compiles bootstrap from source and adds it to the existing `rust-dev` component. There are two main goals here:
1. Make it faster to build rust from source, both the first time and incrementally
2. Make it easier to add non-python entrypoints, since they can call out to bootstrap directly rather than having to figure out the right flags to pre-compile it. This second part is still in a bit of flux, see the tracking issue below for more information.
There are also several changes to make bootstrap able to run on a machine other than the one it was built (particularly around `config.src` and `config.out` detection). I (`@jyn514)` am slightly concerned these will regress unless tested - maybe we should add an automated test that runs bootstrap in a chroot or something? Unclear whether the effort is worth the test coverage.
Helps with https://github.com/rust-lang/rust/issues/94829.
Improve rustdoc GUI tests
I finally finished the update so we can now store values in variables and use them. It improves things nicely.
r? ``@notriddle``
The UEFI targets don't have std support yet, so the normal tests don't
work. However, we can compile a simple no-std program and run it under
QEMU to at least check that the target compiles, links, and runs.
Tested locally with: src/ci/docker/run.sh test-various
- Add a new `bootstrap` component
Originally, we planned to combine this with the `rust-dev` component.
However, I realized that would force LLVM to be redownloaded whenever bootstrap is modified.
LLVM is a much larger download, so split this to get better caching.
- Build bootstrap for all tier 1 and 2 targets
