Upgrade pre-built Clang used in MSVC and MacOS builds, move MSVC builds to Server 2022
Fixes#92948
Example working MacOS and Windows builds: <https://github.com/rust-lang/rust/actions/runs/8989360201>
There is a [bug in Clang 18](https://github.com/llvm/llvm-project/pull/81849) that causes issues when building for Arm64 in later parts of the build (specifically `libgit2`). As a workaround, we will still use the pre-built Clang to build LLVM but will use MSVC for the rest of the Arm64 build.
CI: enable arbitrary try builds, take two
Fixed version of https://github.com/rust-lang/rust/pull/124631, which hopefully won't completely break our CI this time 🤦♂️ Sorry once again. Only the last commit is new.
r? `@pietroalbini`
This commit fixes several issues in the fuchsia-test-runner.py script:
1. Migrate from `pm` to `ffx` for package management, as `pm` is now
deprecated. Furthermore, the `pm` calls used in this script no longer
work at Fuchsia's HEAD. This is the largest change in this commit, and
impacts all steps around repository management (creation and
registration of the repo, as well as package publishing).
2. Allow for `libtest` to be either statically or dynamically linked.
The script assumed it was dynamically linked, but the current Rust
behavior at HEAD is to statically link it.
3. Minor cleanup to use `ffx --machine json` rather than string parsing.
4. Minor cleanup to the docs around the script.
Enable reusing CI Docker cache when running CI images locally
When running a CI image locally, e.g. using `DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux`, it can take a long time until the Docker image is built, which is annoying.
Since we now use proper Docker caching on CI, it should be possible to just `docker pull` the prebuilt image to reuse the cache. We didn't want to do this on CI, since our caching key isn't perfect and it's possible that we can miss some changes, but I think that for local usage it is fine (we could introduce some env. var. to force disable the image download, if needed).
r? `@Mark-Simulacrum`
Use Server 2022 in CI for mingw jobs
From my [previous testing](https://github.com/rust-lang/rust/issues/92948#issuecomment-2077771347), there is nothing blocking using Server 2022 for the Windows MinGW jobs.
This change:
* Adds new `os` entries for Server 2022.
* Renamed the Server 2019 `os` entries (as we'll be removing these over time).
* Moves the mingw jobs to Server 2022 and leaves the msvc jobs on Server 2019.
Partially fixes#92948
We are now using Docker caching directly rather than downloading archives from S3, so it should not be needed.
This should unblock reusing the cache locally.
CI: implement job skipping in Python matrix calculation
This removes the `step` YAML anchor and the corresponding bash script.
Best reviewed commit-by-commit.
r? ```@pietroalbini```
Bump Fuchsia versions
This updates the Fuchsia commit used in `auto - x86_64-gnu-integration` CI bot to use the Rust commit 703dc9ce64. This should help improve the coverage of this builder.
It also updates the SDK version to F20.20240412.3.1, and the Fuchsia Clang version to c777c011a709dffd4fa5e79cad7947b7c3405d02.
r? ``@tmandry``
CI: dynamic jobs
This PR modifies our CI workflows to be dynamic. This means that when a GitHub event is generated, we will run a Python script (`calculate-job-matrix.py`), which decides which CI jobs should be generated. These jobs are defined in `src/ci/github-actions/jobs.yml`).
This should provide a few benefits:
- Once the migration to dynamic jobs is complete, we shouldn't need `expand-yaml-anchors` anymore.
- The job table on PRs (and also the left job column on auto/try builds) should be much cleaner and contain only the jobs that are actually relevant/executed.
- It should be much easier to support dynamic try builds, i.e. to run an arbitrary CI job on a try build.
See [this Zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/job.20matrix.20re-ordered.20PR.20list) for more context.
r? `@ghost`
CI: add script for installing NodeJS and update it to v20
I centralized the installation on a single place to make it simple to update the NodeJS version across the board.
Fixes: https://github.com/rust-lang/rust/issues/123965
r? `@Mark-Simulacrum`
