Improvements to building and CI for mingw/msys
I was getting error messages when trying to follow the build instructions the mingw build for Rust, and managed to track the issue down to an incomparability of Rust's bootstrap program with MSYS2's version of git. Essentially, the problem is that MSYS2's git works in emulated unix-y paths, but bootstrap expects a Windows path. I found a workaround for this by using relative paths instead of absolute paths.
Along with that fix, this PR also updates the build instructions for MinGW to be compatible with modern versions of MSYS2, and some changes to CI to make sure that MSYS2's version of git is tested. In particular, I'm suggesting using the [MSYS2 github action](https://github.com/marketplace/actions/setup-msys2) specially made for this purpose, which is much less hacky than the old approach and gives us more control of what packages are installed. I also cleaned up as many alternate versions of key tools as I could find from PATH, to avoid accidental usage, and cleaned up some abuses of the `CUSTOM_MINGW` environment variable.
This fixes https://github.com/rust-lang/rust/issues/105696 and fixes https://github.com/rust-lang/rust/issues/117567
Make most bootstrap step types !Copy
This makes all bootstrap types except for `Compiler` and `TargetSelection` `!Copy`. This makes it easier to modify them by adding !Copy types in the future, something that `@saethlin` has complained about before, and comes at no cost of code clarity, the impls were completely unused.
Making `Compiler` and `TargetSelection` `!Copy` (which would allow getting rid of interning) is highly nontrivial as they are used and copied **all over the place**. This should hopefully get most of the benefits.
speed up `x install` by skipping archiving and compression
Performing archiving and compression on `x install` is nothing more than a waste of time and resources. Additionally, for systems like gentoo(which uses `x install`) this should be highly beneficial.
[benchmark report](https://github.com/rust-lang/rust/pull/118724#issuecomment-1848964908)
Resolves#109308
r? Mark-Simulacrum (I think you want to review this, feel free to change it if otherwise.)
This makes all bootstrap types except for `Compiler` and
`TargetSelection` `!Copy`. This makes it easier to modify them by adding
!Copy types in the future and comes at no cost of code clarity, the
impls were completely unused.
remove `llvm.assertions=true` in compiler profile
Having this set to true disrupts compiler development workflows for people who use `llvm.download-ci-llvm = true` because we don't provide ci-llvm on the `rustc-alt-builds` server. Therefore, it is kept off by default.
cc `@Nilstrieb` `@compiler-errors`
For more context, see https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/CI.20LLVM.20for.20aarch64
bootstrap: don't resolve symlinks for initial_cargo
I have put the following in my `config.toml`:
```toml
# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
change-id = 121203
[build]
cargo = "/usr/bin/cargo"
rustc = "/usr/bin/rustc"
rustfmt = "/usr/bin/rustfmt"
```
I have rustup installed from Arch's repos, which has all of the above paths be symlinks to `/usr/bin/rustup`. This works just fine with the `argv[0]` trick that rustup uses.
However, `bootstrap` resolves symlinks to check whether `cargo` exists and then uses the resolved path, so it ends up calling `rustup` directly expecting it to behave like `cargo`. Which it doesn't.
This PR removes the canonicalization step, in turn fixing the issue, but sacrificing a pretty error message. However, this exact thing is checked by `x.py` in advance, so I hope it is not a big deal?
Having this set to true disrupts compiler development workflows for people who use `llvm.download-ci-llvm = true`
because we don't provide ci-llvm on the `rustc-alt-builds` server. Therefore, it is kept off by default.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
+ Do not resolve symlinks (as this may break rustup)
+ Check version not just for rustc, but also for cargo.
+ Check that the program's self-reported name
matches the expected name (such as "rustc" or "cargo").
This profile originally made sense when download-ci-llvm = if-unchanged
didn't exist and we had the bad tradeoff of "never modify or always
compile".
Thankfully, these grim times are over and we have discovered clean
water, so the only differentiator between the two profiles is the
codegen profile having LLVM assertions. Adding them doesn't cause that
much of a slowdown, <10% on UI tests from an unscientific benchmark.
It also had LLVM warnings when compiling, which makes sense for every
compiler contributor brave enough to compile LLVM.
The way I removed is by just issueing a nice error message. Given that
everyone with this profile should be a contributor and not someone like
a distro who is more upset when things break, this should be fine.
If it isn't, we can always fall back to just letting codegen mean
compiler.
distribute tool documentations and avoid file conflicts on `x install`
I suggest reading commits one-by-one with the descriptions for more context about the changes.
Fixes#115213
create stamp file for clippy
Due to missing stamp file, we were downloading (and applying nix patches if enabled) continuously every time `Config::download_clippy` was called. This change fixes that by creating stamp file at the end of the function.
Fixes#119442
Due to missing stamp file, we were downloading (and applying nix patches if enabled)
continuously every time `Config::download_clippy` was called. This change fixes that by
creating stamp file at the end of the function.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Support configuring the set of codegen backends to build per host triple
This allows building the compiler itself with one backend while using another backend at runtime. For example this allows compiling rustc to wasm using LLVM, while using Cranelift at runtime to produce actual code. Cranelift can't compile to wasm, but is perfectly capable of running on wasm. LLVM can compile to wasm, but can't run on wasm. [^1]
[^1]: The prototype of this still requires a couple of other patches.
src/bootstrap runs git to find the root of the repository, but this can
go awry when building in MSYS for the mingw target. This is because
MSYS git returns a unix-y path, but bootstrap requires a Windows-y path.
Previously, we were trying to install all doc files under "share/doc/rust"
which caused `rust-installer` tool to create backup files (*.old) due to filename
conflicts. With this change, doc files is now installed under "share/doc/{package}",
where {package} could be rustc, cargo, clippy, etc.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
always run `configure_linker` except for mir-opt tests
`configure_linker` now runs consistently unless it's for mir-opt tests. Previously `!= "check"` condition was causing dirt in the cargo cache between runs of `x anything-but-not-check` and `x check`.
Fixes#120768
cc `@saethlin`
Introducing a new config for this purpose as NetBSD 9 or 8 will be still around
for a good while. For now, we re finally enabling sys::unix::rand::getrandom.
fix `llvm_out` to use the correct LLVM root
When `download-ci-llvm` is enabled, `llvm_out` ends up with the
error below due to an incorrect path on cross-compilations. This change fixes that.
```sh
failed to execute command: "/rust/build/x86_64-unknown-linux-gnu/llvm/build/bin/llvm-config" "--version"
ERROR: No such file or directory (os error 2)
```
When launching tests with --keep-stage option, startup objects
such as rsbegin.o an rsend.o may disappear from the corresponding
stageN compiler.
Fix issue #120784
`configure_linker` now runs consistently unless it's for mir-opt tests.
Previously `!= "check"` condition was causing dirt in the cargo cache between
runs of `x anything-but-not-check` and `x check`
Signed-off-by: onur-ozkan <work@onurozkan.dev>
When `download-ci-llvm` is enabled, `llvm_out` ends up with the
error below due to an incorrect path on cross-compilations. This change fixes that.
```
failed to execute command: "/rust/build/x86_64-unknown-linux-gnu/llvm/build/bin/llvm-config" "--version"
ERROR: No such file or directory (os error 2)
```
Signed-off-by: onur-ozkan <work@onurozkan.dev>
