Errors started showing up, and I read somewhere that this might be
because of old ninja versions. This ninja version is indeed *ancient*.
```
multiple outputs aren't (yet?) supported by depslog; bring this up on the mailing list if it affects you
```
Improvements to building and CI for mingw/msys
I was getting error messages when trying to follow the build instructions the mingw build for Rust, and managed to track the issue down to an incomparability of Rust's bootstrap program with MSYS2's version of git. Essentially, the problem is that MSYS2's git works in emulated unix-y paths, but bootstrap expects a Windows path. I found a workaround for this by using relative paths instead of absolute paths.
Along with that fix, this PR also updates the build instructions for MinGW to be compatible with modern versions of MSYS2, and some changes to CI to make sure that MSYS2's version of git is tested. In particular, I'm suggesting using the [MSYS2 github action](https://github.com/marketplace/actions/setup-msys2) specially made for this purpose, which is much less hacky than the old approach and gives us more control of what packages are installed. I also cleaned up as many alternate versions of key tools as I could find from PATH, to avoid accidental usage, and cleaned up some abuses of the `CUSTOM_MINGW` environment variable.
This fixes https://github.com/rust-lang/rust/issues/105696 and fixes https://github.com/rust-lang/rust/issues/117567
Allowing the Xcode version to "float" based on whatever default GitHub
selects creates an unreliable environment. When GitHub changes the
default, we can have multiple jobs in the same run using different
versions as it rolls out across machines. It can also cause oscillation
between runs as different machines are used. It also causes
unpredictable timing when the updates happen.
This change helps ensure that the version that is used is pinned. The
downside is that it requires manually bumping the version, and the risk
that if we take too long, older Xcodes will be removed and that will
break the build.
Previously, it would only run on changes to subtrees, submodules, or select directories.
That made it so that changes to the compiler that broke tools would only be detected on a full bors merge.
This makes it so the tools builder runs by default, making it easier to catch breaking changes to clippy (which was the most effected).
See comment added for details on the test builder restriction. This is primarily
intended for macOS CI, but is likely to be a slight win on other builders too.
Re-enable submodule archive downloads.
This is effectively a revert of #98423 (though it keeps the `--depth 1` flag since that is still helpful).
GitHub has indicated that they have been working on the original issue, and my testing shows that the llvm-project archive download now succeeds 100% of the time.
This should save about a minute on every job.
The script wasn't referenced anywhere, and it's not useful anymore:
/opt/ghc is not present in new images, while /usr/share/dotnet is only
2.3 GB rather than 16 GB.
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
We currently upload approximately 166 GB/day into this bucket (estimate based on
duration of storage and total current size). My estimate is that this change
should decrease our costs (which are currently in credits) and is in the worst
case (if all objects are brought into hot storage due to unanticipated frequent
access) only going to add an additional ~$4 to the monthly bill. If access is
rare (as expected) to most objects then we expect to save approximately
~$350/month (after this change takes full effect in ~168 days).
