ci: Update centos:7 to use vault repos
CentOS 7 is going EOL on June 30, after which its package repos will no
longer exist on the regular mirrors. We'll still be able to access
packages from the vault server though, and can start doing so now. This
affects `dist-i686-linux` and `dist-x86_64-linux`.
I also removed `epel-release` because we were only using that for its
`cmake3`, but we've been building our own version for a while.
try-job: dist-i686-linux
try-job: dist-x86_64-linux
Don't build a broken/untested profiler runtime on mingw targets
Context: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Why.20build.20a.20broken.2Funtested.20profiler.20runtime.20on.20mingw.3F#75872 added `--enable-profiler` to the `x86_64-mingw` job (to cause some additional tests to run), but had to also add `//@ ignore-windows-gnu` to all of the tests that rely on the profiler runtime actually *working*, because it's broken on that target.
We can achieve a similar outcome by going through all the `//@ needs-profiler-support` tests that don't actually need to produce/run a binary, and making them use `-Zno-profiler-runtime` instead, so that they can run even in configurations that don't have the profiler runtime available. Then we can remove `--enable-profiler` from `x86_64-mingw`, and still get the same amount of testing.
This PR also removes `--enable-profiler` from the mingw dist builds, since it is broken/untested on that target. Those builds have had that flag for a very long time.
Update fuchsia commit, and SDK to 21.20240610.2.1
This includes a fix to the race when publishing multiple packages at the same time.
try-job: x86_64-gnu-integration
CentOS 7 is going EOL on June 30, after which its package repos will no
longer exist on the regular mirrors. We'll still be able to access
packages from the vault server though, and can start doing so now. This
affects `dist-i686-linux` and `dist-x86_64-linux`.
I also removed `epel-release` because we were only using that for its
`cmake3`, but we've been building our own version for a while.
CI: Update riscv64gc-linux job to Ubuntu 22.04, rename to riscv64gc-gnu
Together with joshua.zivkovic@codethink.co.uk, we've been starting to explore improving the state of the `riscv64gc-unknown-linux-gnu` target. Additionally, I'm looking to add support for this platform in [Ferrocene](https://github.com/ferrocene/ferrocene) ([Related PR](https://github.com/ferrocene/ferrocene/pull/618)).
There currently exists a `src/ci/docker/host-x86_64/disabled/riscv64gc-linux` job for the CI, however it is currently experiencing errors.
<details>
<summary>Errors</summary>
```bash
$ DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-linux
# ...
[RUSTC-TIMING] addr2line test:false 0.371
[RUSTC-TIMING] gimli test:false 3.159
[RUSTC-TIMING] object test:false 4.249
error: linking with `riscv64-linux-gnu-gcc` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "riscv64-linux-gnu-gcc" "-Wl,--version-script=/tmp/rustcQaIpWi/list" "-Wl,--no-undefined-version" "/tmp/rustcQaIpWi/symbols.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.std.1b5d59225ff40bd2-cgu.0.rcgu.o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.dalhl7sfna1ffn4nhy6pyfa7f.rcgu.rmeta" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/std-ff89a9732cd5d858.ef0znsdf1ihn2bjkmclodhclp.rcgu.o" "-Wl,--as-needed" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/release/deps" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/build/compiler_builtins-9e9a40064e2f2bd3/out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libpanic_unwind-d968371aba64a26c.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libobject-da5b6473912e89d6.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libmemchr-9cfa08d2baa3643e.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libaddr2line-06e0d2153cecb6ce.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libgimli-6fdf5551cec83840.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_demangle-8ada6466f763fa2e.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libstd_detect-edc0d12d029c4c86.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libhashbrown-9c782935934c8c14.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_std_workspace_alloc-b6984e43b381efa4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libminiz_oxide-37ee29bf49ccaa96.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libadler-591133f6804fa0f4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libunwind-94d98075f42175f3.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcfg_if-e267a7b9dd7af3a7.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/liblibc-503571a038f8d9fd.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/liballoc-e36c72a5cf0ee45f.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/librustc_std_workspace_core-076c2b8501e25f03.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcore-c446fff80486d0bb.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libstd-ff89a9732cd5d858.so" "-shared" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-Wl,-z,origin" "-Wl,-rpath,$ORIGIN/../lib"
# ...
/usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: failed to merge target specific data of file /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib(compiler_builtins-26dc6b5e31e1fdb9.compiler_builtins.74504a151a6bdbbf-cgu.124.rcgu.o)
/usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: -march=rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0: unsupported ISA subset `z'
/usr/lib/gcc-cross/riscv64-linux-gnu/9/../../../../riscv64-linux-gnu/bin/ld: failed to merge target specific data of file /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/riscv64gc-unknown-linux-gnu/release/deps/libcompiler_builtins-26dc6b5e31e1fdb9.rlib(compiler_builtins-26dc6b5e31e1fdb9.compiler_builtins.74504a151a6bdbbf-cgu.004.rcgu.o)
collect2: error: ld returned 1 exit status
[RUSTC-TIMING] std test:false 15.138
error: could not compile `std` (lib) due to 1 previous error
Building bootstrap
Build completed unsuccessfully in 0:04:41
local time: Tue May 28 16:25:09 UTC 2024
network time: Tue, 28 May 2024 16:25:17 GMT
```
</details>
This PR fixes the breakage enough to get the tests running. It does so through bringing the `riscv64gc-unknown-linux-gnu` related test job in line with other related jobs, adopting many of the recent changes present in `src/ci/docker/host-x86_64/armhf-gnu` such as:
* Using Ubuntu 22.04
* Installing a more narrowly scoped package set
* Using `curl` instead of `debootstrap` to set up the root
* No longer patching `busybox`
* Removing the `cmake.sh` script related steps
## Justifying Renaming `riscv64gc-linux` to `riscv64gc-gnu`
The `src/ci/docker/host-x86_64/disabled/riscv64gc-linux` job runs the tests for `risv64gc-unknown-linux-gnu`, it is based off `src/ci/docker/host-x86_64/armhf-gnu`.
There are other jobs that follow a `$arch-gnu` naming scheme:
* `src/ci/docker/host-x86_64/armhf-gnu`
* `src/ci/docker/host-x86_64/x86_64-gnu`
* `src/ci/docker/host-aarch64/aarch64-gnu`
It follows that the name `src/ci/docker/host-x86_64/disabled/riscv64gc-linux` should be `src/ci/docker/host-x86_64/disabled/riscv64gc-gnu`, like the others.
## Testing
> [!NOTE]
> `riscv64gc-unknown-linux-gnu` is a [**Tier 2 with Host Tools** platform](https://doc.rust-lang.org/beta/rustc/platform-support.html), all tests may not necessarily pass! There is work in https://github.com/rust-lang/rust/pull/125220 which helps fix several related tests.
You can test out the renamed job:
```sh
DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu
```
`DEPLOY=1` helps reproduce the CI's environment and also avoids the chance of a `llvm-c/BitReader.h` error (detailed in https://github.com/rust-lang/rust/issues/85424 and https://github.com/rust-lang/rust/issues/56650).
<details>
<summary>Sample of output (expected test failure)</summary>
```bash
$ DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu
# ...
test [ui] tests/ui/where-clauses/where-clause-method-substituion-rpass.rs ... ok
failures:
---- [ui] tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs stdout ----
error: test compilation failed although it shouldn't!
status: exit status: 1
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2" "--target=riscv64gc-unknown-linux-gnu" "--check-cfg" "cfg(FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo" "-A" "unused" "-A" "internal_features" "-Crpath" "-Lnative=/checkout/obj/build/riscv64gc-unknown-linux-gnu/native/rust-test-helpers" "-Clinker=riscv64-linux-gnu-gcc" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo/auxiliary" "-g" "--emit=llvm-ir" "-Csplit-debuginfo=unpacked"
stdout: none
--- stderr -------------------------------
error: `-Csplit-debuginfo=unpacked` is unstable on this platform
error: aborting due to 1 previous error
------------------------------------------
failures:
[ui] tests/ui/debuginfo/debuginfo-emit-llvm-ir-and-split-debuginfo.rs
test result: FAILED. 5 passed; 1 failed; 16897 ignored; 0 measured; 0 filtered out; finished in 410.99ms
Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=riscv64gc-unknown-linux-gnu
local time: Tue May 28 16:28:22 UTC 2024
network time: Tue, 28 May 2024 16:28:30 GMT
```
</details>
try-job: riscv64gc-gnu
CI: use a native `arm64` runner
Opening to test if we can use a native GitHub ARM runner instead of the self-hosted one.
The CI has finished [successfully](https://github.com/rust-lang/rust/actions/runs/9414573751/job/25933776975?pr=126113). It took 1h 29m, which is longer than the previous duration (~1h 10m). This is expected, since we currently use 8 cores on GitHub, rather than 14 cores which are configured for the self-hosted runner.
try-job: aarch64-gnu
Weekly `cargo update`
Replaces #125562
`r-efi` needs to be bumped in two places.
The `icu4x` dependencies also added a SPDX license identifer, so remove the license checking exception and add `Unicode-3.0` to the list.
ci: use GCC 13 as cross compiler in `dist-aarch64-linux`
I'm proposing this GCC upgrade since it addresses bug https://github.com/rust-lang/rust/issues/125619. The
regression in question affects stable release consumers who tend to have
no exposure to Rust build tools, so if at all possible, I would like to
have it resolved in the next stable release. I have tried to fix the bug
in `compiler-builtins`, which led to submitting a PR for `compiler-rt`
in upstream LLVM, but it may take a long time before these upstreams
address this regression.
A summary of why upgrading GCC solves the regression follows.
`__multc3()` is a builtin function `compiler-builtins` exposes for
specifically aarch64, non-Windows targets [1]. The object file for it is
included in `staticlib` archives through `libstd`. The implementation
for `__multc3()` is from `multc3.c`, part of LLVM's `compiler-rt`.
Upstream `compiler-rt` normally builds the C file using the Clang
from the same LLVM version. On the other hand, `compiler-builtins`
builds the C file using GCC, outside of the usual LLVM build system.
The upstream implementation doesn't have feature detection which
works for GCC version older than 10, and ends up producing an
unlinkable object.
Upstream LLVM might be slow to respond to this issue as they might deem
`compiler-builtin` as doing something out of the ordinary from their
perspective. They might reasonably assume everyone builds `compiler-rt`
through LLVM's build system.
I have done the following to test this change:
- verified that a local build without this patch exhibits the
regression.
- verified that with this patch, the object for `__multc3()` has no
reference to undefined functions in the symbol table.
- verified that with this patch, `rustc` is usable to build Ruby with
YJIT, and that the reported regression is resolved.
Since `loongarch64-linux-gnu` already uses GCC 13.2.0, I hope we can upgrade without issues.
try-job: dist-aarch64-linux
[1]: c04eb9e1af/build.rs (L524-L539)
It looks like this job was intending to run all of the `needs-matching-clang`
tests (since they don't run without `RUSTBUILD_FORCE_CLANG_BASED_TESTS`), but
over time developed two problems:
- The tests it cares about were moved from run-make-fulldeps to run-make.
- Some of the relevant tests don't actually have "clang" in their name.
Switching to run-make solves the first problem, but we still don't run the
tests without "clang" in their name, because some of them are currently broken.
Add debugging utils and comments to Fuchsia scripts
This should help when debugging a failure in the Fuchsia build in CI.
I plan to follow up with a PR to the testing section of the dev guide with more details, along with more improvements happening in the Fuchsia repo itself.
try-job: x86_64-gnu-integration
Make html rendered by rustdoc allow searching non-English identifier / alias
Fix alias search result showing `undefined` description.
Inspired by https://github.com/rust-lang/mdBook/issues/2393 .
Not sure if it's worth it adding full-text search functionality to rustdoc rendered html.
CI: fix publishing of toolstate history
Hopefully fixes the upload of toolstate history that I broke in https://github.com/rust-lang/rust/pull/125145. The problem is that the toolstate environment variables need to be available not just when updating `latest.json` through the Bash/Python script, but also in the main CI workflow, where `history` is updated in bootstrap (the toolstate logic is distributed in two places :/ ).
The only tricky thing is how to pass the token secret to the CI job, as I need to enable it only for privileged (`auto`/`try`) builds. I assume that the secret is filled only on `rust-lang-ci`, not on `rust-lang`, so I chose the easiest way of just configuring the environment variable globally. We'll see if that works on PR CI.
r? `@ehuss`
Remove unused(?) `~/rustsrc` folder from docker script
Every time I run a docker image, the script creates an empty `~/rustsrc` folder. It doesn't seem to be referenced anywhere else, so I'd like to remove it.
Incidentally, this also documents DIST_TRY_BUILD as I fail to find it from a simple search any time I look for it to eg enable tests on try builds.
I'm proposing this GCC upgrade since it addresses bug #125619. The
regression in question affects stable release consumers who tend to have
no exposure to Rust build tools, so if at all possible, I would like to
have it resolved in the next stable release. I have tried to fix the bug
in `compiler-builtins`, which led to submitting a PR for `compiler-rt`
in upstream LLVM, but it may take a long time before these upstreams
address this regression.
A summary of why upgrading GCC solves the regression follows.
`__multc3()` is a builtin function `compiler-builtins` exposes for
specifically aarch64, non-Windows targets [1]. The object file for it is
included in `staticlib` archives through `libstd`. The implementation
for `__multc3()` is from `multc3.c`, part of LLVM's `compiler-rt`.
Upstream `compiler-rt` normally builds the C file using the Clang
from the same LLVM version. On the other hand, `compiler-builtins`
builds the C file using GCC, outside of the usual LLVM build system.
The upstream implementation doesn't have feature detection which
works for GCC version older than 10, and ends up producing an
unlinkable object.
Upstream LLVM might be slow to respond to this issue as they might deem
`compiler-builtin` as doing something out of the ordinary from their
perspective. They might reasonably assume everyone builds `compiler-rt`
through LLVM's build system.
I have done the following to test this change:
- verified that a local build without this patch exhibits the
regression.
- verified that with this patch, the object for `__multc3()` has no
reference to undefined functions in the symbol table.
- verified that with this patch, `rustc` is usable to build Ruby with
YJIT, and that the reported regression is resolved.
[1]: c04eb9e1af/build.rs (L524-L539)
We want to only demand that we check for all components we expect
if we actually built the components we expect, which means
we built the LLVM. Otherwise, it isn't worth checking.
Add opt-for-size core lib feature flag
Adds a feature flag to the core library that enables the possibility to have smaller implementations for certain algorithms.
So far, the core lib has traded performance for binary size. This is likely what most people want since they have big simd-capable machines. However, people on small machines, like embedded devices, don't enjoy the potential speedup of the bigger algorithms, but do have to pay for them. These microcontrollers often only have 16-1024kB of flash memory.
This PR is the result of some talks with project members like `@Amanieu` at RustNL.
There are some open questions of how this is eventually stabilized, but it's a similar question as with the existing `panic_immediate_abort` feature.
Speaking as someone from the embedded side, we'd rather have this unstable for a while as opposed to not having it at all. In the meantime we can try to use it and also add additional PRs to the core lib that uses the feature flag in areas where we find benefit.
Open questions from my side:
- Is this a good feature name?
- `panic_immediate_abort` is fairly verbose, so I went with something equally verbose
- It's easy to refactor later
- I've added the feature to `std` and `alloc` as well as they might benefit too. Do we agree?
- I expect these to get less usage out of the flag since most size-constraint projects don't use these libraries often.
