ci: Upgrade non-dist Linux testers from ubuntu:16.04 to 22.04
The main goal of updating to 22.04 is to get away from `llvm.allow-old-toolchain`.
A side benefit is that they can also use the system `cmake` instead of building one.
Rootless podman creates a separate user namespace, where an inner
`LOCAL_USER_ID` will map to a different subuid range on the host.
The "keep-id" mode maps the current UID directly into the container.
This makes `src/ci/docker/run.sh` work better for testing container
images on systems running podman, where "docker" is just a shim.
Download, rather than sccache-cache, LLVM in CI
My hope/expectation is that we can do better than sccache in CI for cached builds -- currently it looks like on macOS those still take upwards of 10-11 minutes, which is a significant amount of time that we could potentially cut.
This enables this mode for all non-dist builders; this should avoid any problems with the artifacts we distribute, while also providing for faster test builders (since they'll make use of PGO'd LLVM on the platforms we do that on, which is hopefully a nice win). It slightly increases the chance of test builders starting to fail only after a PR is merged (if PGO changes runtime behavior), but that should hopefully never happen, so I think this is worthwhile.
Measurements on the PR for apple-1 don't show any noticeable improvement in CI times, but those can be pretty noisy -- I'm inclined to land this since it *should* pretty much always be better and we can reconsider if that ever turns out not to be the case.
Initial implementation of REUSE
This PR implements the first two steps of #99414 by:
* Adding some scaffolding for REUSE. The `.reuse/dep5` file now marks every file as the custom "TODO" license, which I'll remove in a future PR once Debian imports their metadata. The TODO license is needed so that `reuse lint` works.
* Runs `reuse lint` in CI, in the `mingw-check` builder. REUSE currently has a bug when parsing some files in the LLVM source code. This means REUSE will fail when running it in source tarballs of rustc, and that bug prevents us from passing the `--include-submodules` flag in CI. I opened https://github.com/fsfe/reuse-tool/pull/560 upstream with a fix, and as soon as it's merged/released I planned to bump the pinned version to include the fix we need.
r? `@Mark-Simulacrum`
See comment added for details on the test builder restriction. This is primarily
intended for macOS CI, but is likely to be a slight win on other builders too.
Update to LLVM 15
For preliminary testing. Some LLVM 15 compatibility fixes were applied separately in #99512.
Release timeline:
* LLVM 15 branched on Jul 26.
* The final LLVM 15.0.0 release is scheduled for Sep 6.
* Current nightly (1.65.0) is scheduled for Nov 3.
Changes in this PR (apart from the LLVM update):
* Pass `--set llvm.allow-old-toolchain` for many Docker images. LLVM 16 will require GCC >= 7.1, while LLVM 15 still allows older compilers with an option. Specify the option for builders still using GCC 5.4. #95026 updated some of the used toolchains, but not all.
* Use the `+atomics-32` target feature for thumbv6m.
* Explicitly link libatomic when cross-compiling LLVM to 32-bit target.
* Explicitly disable zstd support, to avoid libzstd.so dependency.
New LLVM patches ([commits](https://github.com/rust-lang/llvm-project/commits/rustc/15.0-2022-08-09)):
* [rust-only] Fix ICE with GCC 5.4 (15be58d7f0)
* [rust-only] Fix build with GCC 5.4 (774edc10fa)
* ~~[rust-only] Fix build with GCC 5.2 (1a6069a7bb)~~
* ~~[rust-only] Fix ICE with GCC 5.2 (493081f290)~~
* ~~[rust-only] Fix build with GCC 5.2 (0fc5979d73)~~
* [backported] Addition of `+atomics` target feature (57bdd9892d).
* [backported] Revert compiler-rt change that broke powerpc (9c68b43915)
* [awaiting backport] Fix RelLookupTableConverter on gnux32 (639388a05f / https://github.com/llvm/llvm-project/issues/57021)
Tested images: dist-x86_64-linux, armhf-gnu, arm-android, dist-s390x-linux, dist-x86_64-illumos, dist-x86_64-freebsd, wasm32, dist-x86_64-musl, dist-various-1, dist-riscv64-linux, dist-mips-linux, dist-mipsel-linux, dist-powerpc-linux, dist-aarch64-linux, dist-x86_64-apple, x86_64-msvc-1, x86_64-msvc-2, dist-various-2, dist-arm-linux
Tested up to the usual ipv6 error: test-various, i686-gnu, x86_64-gnu-nopt
r? `@ghost`
Add `x.sh` and `x.ps1` shell scripts
This is a more ambitious version of https://github.com/rust-lang/rust/pull/98716.
It still changes the x.py shebang back to python3, for compatibility with non-Unix systems,
but also adds alternative entrypoints for systems without `python3` installed.
These scripts will be necessary for the rust entrypoint (#94829), so I see
little downside in adding them early.
I'll update the dev-guide to suggest using these instead of x.py once this is merged.
Fixes https://github.com/rust-lang/rust/issues/98650
r? `@Mark-Simulacrum` cc `@dtolnay` `@CAD97` `@yoshuawuyts`
This is a more ambitious version of https://github.com/rust-lang/rust/pull/98716.
It still changes the shebang back to python3, for compatibility with non-Unix systems,
but also adds alternative entrypoints for systems without `python3` installed.
These scripts will be necessary for the rust entrypoint (#94829), so I see
little downside in adding them early.
Fuchsia CI zircon lib improvement
Removing Zircon build process, instead pulling `sysroot` and related libs directly from Fuchsia SDK
cc. `@tmandry` `@djkoloski`
With binutils 2.32, we were getting errors like this:
relocation truncated to fit: R_PPC_PLTREL24 against symbol
`__cxa_atexit@@GLIBC_2.1.3' defined in .plt section in
/x-tools/powerpc-unknown-linux-gnu/powerpc-unknown-linux-gnu/sysroot/usr/lib/crt1.o
but it builds okay with binutils 2.30.
This is implementing the MCP from rust-lang/compiler-team#493. It is
increasing the minimum requirements of a couple Tier 1 targets, and
others at lower tiers, so this should go through FCP sign-offs for both
`T-compiler` and `T-release`.
The new `linux-gnu` baseline is kernel 3.2 and glibc 2.17. We will also
take that kernel as the minimum floor for _all_ `*-linux-*` targets, so
it may be broadly assumed in the implementation of the standard library.
That does not preclude specific targets from having greater requirements
where it makes sense, like a new arch needing something newer, or a
platform like `linux-android` choosing a newer baseline.
Remove unsupported options in configure.py
I've seen people using `optimize = false` and `full-bootstrap = true` in the past, without knowing
that they're not recommended. Remove `optimize` and a few other options that are always a bad idea,
and document that full-bootstrap is only for testing reproducible builds.
I've seen people using `optimize = false` and `full-bootstrap = true` in the past, without knowing
that they're not recommended. Remove `optimize` and a few other options that are always a bad idea,
and document that full-bootstrap is only for testing reproducible builds.
Clean up submodule checkout scripts
This is just some small cleanup:
* Removed unused CACHE_DIR stuff
* Removed duplicate fetch_github_commit_archive function which is no longer used
* Combined init_repo.sh and checkout-submodules.sh, as checkout-submodules.sh was doing nothing but calling init_repo.sh
