Remove extra unshallow from cherry-pick checker
This is already done by 13db8440bb/src/ci/init_repo.sh (L32-L36) on the beta channel, and git throws an error if you attempt to unshallow an already non-shallow repository.
r? ```@pietroalbini```
This skips bumping Windows sccache because we run into compilation failures when
doing so (-m32 not supported by clang-cl?). Not clear on cause, but seems
easiest to just hold back.
This should avoid PGO-related failures encountered on Linux, and more broadly
seems like a good idea on other platforms as well (though it is likely not
necessary right this moment).
The only change here is a fix for `sys.platform` on Linux. Python 3.3
changed the API to return "linux" instead of "linux2"/"linux3", so this
commit uses `.startswith("python")` to make the code work on Python 3
without breaking Python 2.
Convert rustfmt from a submodule to a subtree
r? `@calebcartwright` cc `@Manishearth` `@Mark-Simulacrum`
The motivation is that submodule updates cause rustfmt to not be available on nightly a lot; most recently it was unavailable for over 10 days, causing the beta release to be delayed. Additionally this is much less work on the part of the rustfmt maintainers to keep the rustfmt compiling, since now people making breaking changes will be responsible for fixing them.
I kept the rustfmt git history so it looks like there are thousands of commits. The important commits are https://github.com/rust-lang/rust/compare/851dee3af9404bf399c3c4ffefe5105edb3debad~..pull/82208/head. This adds about 10 MB of git history, which is not terribly much compared to the 702 MB that already exist.
- Add `src/tools/rustfmt` to `x.py check`
- Fix CRLF issues with rustfmt tests (see commit for details)
- Use `rustc_private` instead of crates.io dependencies
This was already switched upstream and would have landed in the next submodule bump anyway. This just updates Cargo.lock for rust-lang/rust.
- Add `yansi-term` to the list of allowed dependencies.
This is a false positive - rustc doesn't actually use it, only rustfmt, but because it's activated by the cargo feature of a dependency, tidy gets confused. It's fairly innocuous in any case, it's used for color printing.
This would have happened in the next submodule bump.
- Remove rustfmt from the list of toolstate tools.
- Give a hard error if testing or building rustfmt fails.
- Update log to 0.4.14
This avoids a warning about semicolons in macros; see the commit for details.
- Don't add tools to the sysroot when they finish building.
This is the only change that could be considered a regression - this avoids a "colliding StableCrateId" error due to a bug in resolve (https://github.com/rust-lang/rust/issues/56935). The regression is that this rebuilds dependencies more often than strictly necessary. See the commit for details.
Fixes https://github.com/rust-lang/rust/issues/85226 (permanently). Closes https://github.com/rust-lang/rust/issues/82385. Helps with https://github.com/rust-lang/rust/issues/70651. Helps with https://github.com/rust-lang/rust/issues/80639.
Error out if a PR is sent to the wrong channel
It happened multiple times that a PR meant to go on beta ends up being opened (and occasionally merged) to master. This PR does two things:
* Moves the definition of the channel in `src/ci/channel` so it's easier for tools to read it. I was not sure whether to move it to `src/channel` (like `src/version`): ended up with `src/ci` as it's currently only used for CI, but I'm open to moving it to `src`. We'll need to update the release process after this.
* Adds a check on **non-bors** builds that errors out if the base branch is not the expected one for the currently defined channel. This will not cause problems for promotion PRs, as those PRs are meant to also update the channel name.
r? `@Mark-Simulacrum`
Retry clang+llvm download
We've been seeing a pretty high rate of spurious network failures (e.g., openssl
connection reset by peer). Not clear why, but let's add a retry.
r? `@pietroalbini`
Currently, we have LLVM tarballs for win64, generated by someone running
the installer via wine and tarring up the result.
7z knows how to extract NSIS installers directly, and the result is
identical to our tarball, except that it doesn't include `Uninstall.exe`
(which we don't care about) and it includes the NSIS plugin directory
(which we also don't care about).
This simplifies the process of upgrading CI, and allows us to just
mirror the upstream release .exe directly. This also improves our
supply chain.
We're copying the binary to make sure we can call python3.exe around,
but it seems like the base image of GitHub Actions changed, copying the
file before we do so.
This commit changes the CI script to only copy the file if it doesn't
already exist.
Promote aarch64-pc-windows-msvc to Tier 2 Development Platform
Adds a GitHub Actions CI build for `aarch64-pc-windows-msvc` via cross-compilation on an x86_64 host.
This promotes `aarch64-pc-windows-msvc` from a Tier 2 Compilation Target (std) to a Tier 2 Development Platform (std+rustc+cargo+tools).
Fixes#72881
r? `@pietroalbini`
While waiting for a PR job to start testing my code, I noticed the
symlink-build-dir step took 10 minutes to complete, so I investigated
what caused that.
It seems like something changed in the build environment between version
20200901.1 (where the step took 45 seconds) and version 20200908.1
(where the step took 10 minutes). At the time of writing this commit,
the rust-lang organization is on vertsion 20200908.1, while the
rust-lang-ci organization is at version 20200901.1 (and is not affected
by this yet).
There is no need for this step anymore on GHA, as our XL builders got an
increase in the root paritition size, so this commit removes the code
that moved stuff around on GHA (while keeping it on Azure).
For the record, at the time of writing this, the disk situation is:
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 667G 60G 607G 9% /
/dev/sdb1 110G 4.1G 101G 4% /mnt
This will render the src/ci/exec-with-shell.py script more or less
useless, but we're going to replace that by just using the system bash
instead.
Signed-off-by: Kristofer Rye <kristofer.rye@gmail.com>
Remove legacy InnoSetup GUI installer
On Windows the InnoSetup `.exe` installer was superseded by the MSI installer long ago. It's no longer needed.
The `.exe` installer hasn't been linked from the [other installation methods](https://forge.rust-lang.org/infra/other-installation-methods.html#standalone) page in many years. As far as I can tell the intent was always to remove this installer once the MSI proved itself. Though admittedly both installers feel very "legacy" at this point.
Removing this would mean we only maintain one Windows GUI installer and would speed up the distribution phase.
As a result of removing InnoSetup, this closes#24397
This commit changes our Windows CI to always use the latest Python
interpreter available in the GHA tool cache instead of hardcoding Python
3.7.6. This is needed because occasionally GitHub bumps the installed
version, deleting the previous one.
Stop explicitly depending on python 2
This PR revises our previous policy of officially only supporting and testing with python 2 in the CI environment to instead test with python 3. It also changes the defaults to python 3 in our various scripts (usually, by way of `python` rather than `python3` to preserve compatibility with systems that do not have a python 3 available).
The effect of this is that we expect all new patches to support python 3 (and will test as such). We explicitly also expect that patches support python 2.7 as well -- and test as such, though only on one builder. This is intended as a temporary, though likely long-lived, measure to preserve compatibility while looking towards the future which is likely to be a python 3 only world. We do not at this point set a timeline for when we'll drop support for python 2.7; it's plausible that this is months or years into the future, depending on how quickly the ecosystem drops support and how painful it is for us to maintain that support over time.
Closes#65063 (as far as I can tell; please file explicit and separate issues or PRs if not).
There are some builders that are running out of disk space while
building the Docker images, such as arm-android. This moves and symlinks
/var/lib/docker to the /mnt partition on Linux GHA.
The /mnt mount point has 53GB of free disk space at the time of writing
this commit, so this moves the build there to avoid running out of disk
space during builds.
On Azure Pipeliones, the C: filesystem is huge with a lot of free space,
while D: is small. By default builds happened in D:, so we added a
script to symlink the big directories to C:, granting us more space.
Filesystem Size Used Avail Use%
C: 256G 143G 114G 56%
D: 14G 2.0G 13G 15%
On GitHub Actions instead C: is almost full, and we have a lot of free
space on D:, where the build happens.
Filesystem Size Used Avail Use%
C: 128G 114G 15G 89%
D: 56G 4.8G 52G 9%
This commit stops creating the symlink on GitHub Actions, fixing the out
of disk space errors we were seeing on some Windows builders.
