[CI] Use the latest Python available on Windows
This PR changes our Windows CI to always use the latest Python interpreter available in the GHA tool cache instead of hardcoding Python 3.7.6. This is needed because occasionally GitHub bumps the installed version, deleting the previous one.
This fixes the current GHA outage we're having. I fully expect the outage to propagate to Azure Pipelines in the coming days if we don't merge this, as both GHA and Azure use the same underlying image. Once the PR is merged we can re-enabled the double-gating.
r? @Mark-Simulacrum
This commit changes our Windows CI to always use the latest Python
interpreter available in the GHA tool cache instead of hardcoding Python
3.7.6. This is needed because occasionally GitHub bumps the installed
version, deleting the previous one.
Ensure that `./x.py doc --stage 0 src/libstd` works via CI
This was split off from #71645, which recommends that users first try building `libstd` docs with the bootstrap `rustdoc`. This should work in most cases, but will fail if we start using a very recent `rustdoc` feature outside a `#[cfg(not(bootstrap))]`.
It would be very nice to guarantee that `./x.py doc --stage 0 src/libstd` works, since it allows documentation changes to be rendered locally without needing to build the compiler. However, it may put too big a burden on `rustdoc` developers who presumably want to dogfood new features.
Enable "full tools" option on ARM dist builders
This commit switches the `--enable-extended` option on the arm-related
dist builders to `--enable-full-tools`. This alias in `config.py`
corresponds to enabling a few more options:
* `rust.lld = true` - this is the main purpose of this PR, to enable LLD
on ARM-related platforms. This means it will effectively unlock
compilation of wasm programs from an arm host.
* `rust.llvm-tools = true` - it turns out that this option is largely
ignored in rustbuild today. This is only read in one location to set
some flags for the `llvm-tools` package, but the `llvm-tools` package
is already produced on all of these builders. It's predicted that this
will have no effect on build times.
* `rust.codegen-backends = ['llvm']` - historically this also enabled
the emscripten backend, but that has long since been removed.
This brings the ARM dist builders in line with the x86_64 dist builders
using this flag. The hope is that the extra time spent on CI building
LLD will acceptable because it's cached by `sccache`, LLD is a
relatively small C++ project, and the dist builders are all clocking
well under 3 hours (the slowest of all builders) around 2 hours.
There's likely some possible cleanup that can happen with these
configure options since it doesn't look like they've aged too too well,
but I'm hopeful that possible refactorings, if necessary, could be
deferred to future PRs.
This commit switches the `--enable-extended` option on the arm-related
dist builders to `--enable-full-tools`. This alias in `config.py`
corresponds to enabling a few more options:
* `rust.lld = true` - this is the main purpose of this PR, to enable LLD
on ARM-related platforms. This means it will effectively unlock
compilation of wasm programs from an arm host.
* `rust.llvm-tools = true` - it turns out that this option is largely
ignored in rustbuild today. This is only read in one location to set
some flags for the `llvm-tools` package, but the `llvm-tools` package
is already produced on all of these builders. It's predicted that this
will have no effect on build times.
* `rust.codegen-backends = ['llvm']` - historically this also enabled
the emscripten backend, but that has long since been removed.
This brings the ARM dist builders in line with the x86_64 dist builders
using this flag. The hope is that the extra time spent on CI building
LLD will acceptable because it's cached by `sccache`, LLD is a
relatively small C++ project, and the dist builders are all clocking
well under 3 hours (the slowest of all builders) around 2 hours.
There's likely some possible cleanup that can happen with these
configure options since it doesn't look like they've aged too too well,
but I'm hopeful that possible refactorings, if necessary, could be
deferred to future PRs.
ci: run mir-opt tests on PR CI also as 32-bit (for `EMIT_MIR_FOR_EACH_BIT_WIDTH`).
Background: #69916 and [`src/test/mir-opt/README.md`](https://github.com/rust-lang/rust/blob/master/src/test/mir-opt/README.md):
> By default 32 bit and 64 bit targets use the same dump files, which can be problematic in the
presence of pointers in constants or other bit width dependent things. In that case you can add
>
> ```
> // EMIT_MIR_FOR_EACH_BIT_WIDTH
> ```
>
> to your test, causing separate files to be generated for 32bit and 64bit systems.
However, if you change the output of such a test (intentionally or not), or if you add a test and it varies between 32-bit and 64-bit platforms, you have to run this command (for a x64 linux host):
`./x.py test --stage 1 --target x86_64-unknown-linux-gnu --target i686-unknown-linux-gnu --bless src/test/mir-opt`
Otherwise, bors trying to merge the PR will fail, since we test 32-bit targets there.
But we don't on PR CI, which means there's no way the PR author would know (unless they were burnt by this already and know what to look for).
This PR resolves that by running `mir-opt` tests for ~~`i686-unknown-linux-gnu`~~, on PR CI.
**EDIT**: switched to `armv5te-unknown-linux-gnueabi` to work around LLVM 7 crashes (see https://github.com/rust-lang/compiler-builtins/pull/311#issuecomment-612270089), found during testing.
cc @rust-lang/wg-mir-opt @rust-lang/infra
Stop explicitly depending on python 2
This PR revises our previous policy of officially only supporting and testing with python 2 in the CI environment to instead test with python 3. It also changes the defaults to python 3 in our various scripts (usually, by way of `python` rather than `python3` to preserve compatibility with systems that do not have a python 3 available).
The effect of this is that we expect all new patches to support python 3 (and will test as such). We explicitly also expect that patches support python 2.7 as well -- and test as such, though only on one builder. This is intended as a temporary, though likely long-lived, measure to preserve compatibility while looking towards the future which is likely to be a python 3 only world. We do not at this point set a timeline for when we'll drop support for python 2.7; it's plausible that this is months or years into the future, depending on how quickly the ecosystem drops support and how painful it is for us to maintain that support over time.
Closes#65063 (as far as I can tell; please file explicit and separate issues or PRs if not).
Enable rust-lld on dist-x86_64-musl
Add rust-lld to rustup llvm-tools-preview on nightly for musl
I am using a musl distro on my workstation, with `RUSTFLAGS="-C target-feature=-crt-static"` this works fine. I know that `x86_64-unknown-linux-musl` was originally only meant as a target and not as a host. But most problems have been fixed, and I have fewer problems with `unknown` (rustup) than when I am using `x86_64-alpine-linux-musl` (rust installed by the distro). The only thing I am missing is rust-lld in llvm-tools-preview on nightly.
I needed rust-lld for a wasm tutorial. I built rust-lld and tested it with that tutorial, and it worked well. I asked [here](https://users.rust-lang.org/t/enable-rust-lld-on-x86-64-unknown-linux-musl/39851) where to request to enable lld and ended up doing this PR.
I compared llvm-tools-preview `nightly-x86_64-unknown-linux-musl` and `nightly-x86_64-unknown-linux-gnu`: only rust-lld is missing in musl.
I tested the change using:
```bash
./src/ci/docker/run.sh dist-x86_64-musl
```
And I checked that the resulting rust-lld binary runs.
There are some builders that are running out of disk space while
building the Docker images, such as arm-android. This moves and symlinks
/var/lib/docker to the /mnt partition on Linux GHA.
GHA: enable running multiple try builds at the same time
While for auto, try and PR builds we only want the latest commit to be tested, that's not true for try builds: each commit pushed to the branch is a different PR being tested, and we want multiple PRs to be tested in parallel if there is enough demand.
Fixes#70569
While for auto, try and PR builds we only want the latest commit to be
tested, that's not true for try builds: each commit pushed to the branch
is a different PR being tested, and we want multiple PRs to be tested in
parallel if there is enough demand.
Fixes#70569
The /mnt mount point has 53GB of free disk space at the time of writing
this commit, so this moves the build there to avoid running out of disk
space during builds.
On Azure Pipeliones, the C: filesystem is huge with a lot of free space,
while D: is small. By default builds happened in D:, so we added a
script to symlink the big directories to C:, granting us more space.
Filesystem Size Used Avail Use%
C: 256G 143G 114G 56%
D: 14G 2.0G 13G 15%
On GitHub Actions instead C: is almost full, and we have a lot of free
space on D:, where the build happens.
Filesystem Size Used Avail Use%
C: 128G 114G 15G 89%
D: 56G 4.8G 52G 9%
This commit stops creating the symlink on GitHub Actions, fixing the out
of disk space errors we were seeing on some Windows builders.
Apparently the old path we were using for Python 2 on Windows was not
documented, and eventually got removed. This switches our CI to use the
correct path.
Toolstate: remove redundant beta-week check.
I made a bit of a mistake in #69624. The "beta regression" doesn't need to be checked twice.
I also rolled up #69693 to avoid merge conflicts.
