Compile run-make-support and run-make tests with the bootstrap compiler
It does not seem necessary to have to recompile run-make-support on changes to the local compiler/stdlib. This PR simplifies the implementation of a few tools, then switches rms to stage0 and also makes the handling of environment variables in run-make tests simpler.
Best reviewed commit-by-commit. I can split it into multiple PRs if you want.
Also tested that `COMPILETEST_FORCE_STAGE0=1 ./x test tests/run-make --stage 0` still works. Incredibly, it looks like it even passes more tests than on `master` 😆
r? ``@jieyouxu``
do not build additional stage on compiler paths
When calling `x build compiler (or rustc) --stage N` bootstrap builds stage N+1 compiler, which is clearly not what we requested. This doesn't happen when running `x build --stage N` without explicitly targeting the compiler.
The changes applied fix this issue.
r? ghost
When calling `x build compiler (or rustc) --stage N` bootstrap builds stage N+1 compiler,
which is clearly not what we requested. This doesn't happen when running `x build --stage N`
without explicitly targeting the compiler.
The changes applied fix this issue.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
linker: Fix escaping style for response files on Windows
If we use a С/С++ compiler as linker, then Posix-style escaping should be used.
Also temporarily fixup rustbuild to not fail at least in common scenarios, until the bootstrap compiler is updated.
Fixes https://github.com/rust-lang/rust/issues/137498
[`compiletest`-related cleanups 4/7] Make the distinction between root build directory vs test suite specific build directory in compiletest less confusing
Reference for overall changes: https://github.com/rust-lang/rust/pull/136437
Part **4** of **7** of the *`compiletest`-related cleanups* PR series.
### Summary
- Remove `--build-base` compiletest flag, and introduce `--build-{root,test-suite-root}` flags instead. `--build-base` previously actually is test suite specific build directory, not the root `build/` directory.
- Feed the root build directory directly from bootstrap to compiletest via `--build-root` instead of doing multiple layers of parent unwrapping[^parent] based on the test suite specific build directory.
- Remove a redundant `to_path_buf()`.
[^parent]: Please do not unwrap the parents.
r? bootstrap
Include version number of libs being built in cargo lib metadata (esp. `librustc_driver*.so`)
Previously, on a non-stable channel, it's possible for two builds from different versioned sources (e.g. 1.84.0 vs 1.84.1) to produce a `librustc_driver*.so` with the same filename hashes. This causes problems with side-by-side installs wrt. linker search paths because 1.84.1 rustc bin and 1.84.0 rustc bin may try to link to the "same" `librustc_driver*.so` (same filename hash) but fail because the contents of the so is actually different.
We try to mitigate this by including the version number of artifacts being built via `__CARGO_DEFAULT_LIB_METADATA` (kind of an ugly hack, but I don't think cargo has a way for us to tell cargo to use a package version override).
Fixes#136701 (mitigates, really).
### Testing
Tested manually[^host] by:
```bash
$ cat src/version
1.86.0
$ ./x build library # w/ compiler profile, (non-stable) dev channel
$ lddtree build/host/stage1/bin/rustc
rustc => build/host/stage1/bin/rustc (interpreter => /lib64/ld-linux-x86-64.so.2)
librustc_driver-ea1b1b2291881cc4.so => build/host/stage1/bin/../lib/librustc_driver-ea1b1b2291881cc4.so
[...]
```
and observing that changing `src/version` to bump a point release causes `librustc_driver*.so` to have a different hash while sources are unmodified otherwise.
```bash
$ cat src/version
1.86.1
$ ./x build library # w/ compiler profile, (non-stable) dev channel
$ lddtree build/host/stage1/bin/rustc
rustc => build/host/stage1/bin/rustc (interpreter => /lib64/ld-linux-x86-64.so.2)
librustc_driver-746badadbcb74721.so => build/host/stage1/bin/../lib/librustc_driver-746badadbcb74721.so
[...]
```
cc `@clan` `@demize` could you check that if you backport this change against 1.84.{0,1} as reported in #136701, that the produced `rustc` binary works, under the context of the Gentoo build system setup?
[^host]: on a `x86_64-unknown-linux-gnu` host, no cross
Build GCC on CI
Previously, we have downloaded a specific commit of GCC and prebuilt it inside Docker using the `build-gccjit.sh` script. This PR removes that scripts and uses the bootstrap GCC step. This allows us to use the `src/gcc` submodule for determining which GCC should be built, and it also moves the logic closer to LLVM, which is also built by bootstrap.
A few things to note:
- The `sccache` option is currently in the `llvm` block, so the GCC build uses `llvm.ccache`, which is a bit weird :) We could either add `gcc.ccache`, or (what I think would be better) to just move `ccache` to the `build` section, as I don't think that it will be necessary to use ccache for LLVM, but not for GCC.
- When the GCC codegen backend is built, it needs to depend on a step that first builds GCC. This is currently done in a hacky way. The proper solution is to create a separate step for the GCC codegen backend, but that is a larger change. Let me know what you think.
r? `@onur-ozkan`
try-job: i686-msvc-1
try-job: x86_64-mingw-1
configure.py: don't instruct user to run nonexistent program
```shell-session
$ ./configure
configure: processing command line
configure:
configure: build.configure-args := []
configure: profile := dist
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python /mnt/filling/store/nabijaczleweli/code/rust/x.py --help`
```
This is naturally not valid since I don't have a "python" executable (and this will hopefully become more and more true as Python 2 dies out).
./configure knows this since it does `try python3 "$``@"`,`` then `python2.7` &c.
After, this now says
```
configure: run `python3 /mnt/filling/store/nabijaczleweli/code/rust/x.py --help`
```
which is possible, and corresponds to the interpreter actually running.
avoid `compiler_for` for dist tools and force the current compiler
Using `compiler_for` in dist steps was causing to install stage1 tools into the dist tarballs, which doesn't match with the stage2 compiler.
Fixes https://github.com/rust-lang/rust/issues/137469
$ ./configure
configure: processing command line
configure:
configure: build.configure-args := []
configure: profile := dist
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python /mnt/filling/store/nabijaczleweli/code/rust/x.py --help`
This is naturally not valid since I don't have a "python" executable
(and this will hopefully become more and more true as Python 2 dies out).
./configure knows this since it does try python3 "$@", then python2.7 &c.
After, this now says
configure: run `python3 /mnt/filling/store/nabijaczleweli/code/rust/x.py --help`
which is possible, and corresponds to the interpreter actually running.
Rollup of 9 pull requests
Successful merges:
- #135354 ([Debuginfo] Add MSVC Synthetic and Summary providers to LLDB)
- #136826 (Replace mem::zeroed with mem::MaybeUninit::uninit for large struct in Unix)
- #137194 (More const {} init in thread_local)
- #137334 (Greatly simplify lifetime captures in edition 2024)
- #137382 (bootstrap: add doc for vendor build step)
- #137423 (Improve a bit HIR pretty printer)
- #137435 (Fix "missing match arm body" suggestion involving `!`)
- #137448 (Fix bugs due to unhandled `ControlFlow` in compiler)
- #137458 (Fix missing self subst when rendering `impl Fn*<T>` with no output type)
r? `@ghost`
`@rustbot` modify labels: rollup
stabilize stage management for rustc tools
https://github.com/rust-lang/rust/pull/135990 got out of control due to excessive complexity. This PR aims to achieve the same goal with a simpler approach, likely through multiple smaller PRs. I will keep the other one read-only and open as a reference for future work.
This work stabilizes the staging logic for `ToolRustc` programs, so you no longer need to handle build and target compilers separately in steps. Previously, most tools didn't do this correctly, which was causing the compiler to be built twice (e.g., `x test cargo --stage 1` would compile the stage 2 compiler before, but now it only compiles the stage 1 compiler).
I also tried to document how we should write `ToolRustc` steps as they are quite different and require more attention than other tools.
Next goal is to stabilize how stages are handled for the rustc itself. Currently, `x build --stage 1` builds the stage 1 compiler which is fine, but `x build compiler --stage 1` builds stage 2 compiler.
~~for now, r? ghost~~
test building enzyme in CI
1) This PR fixes a significant compile-time regression, by only running the expensive autodiff pipeline, if the users pass the newly introduced Enable value to the `-Zautodiff=` flag. It updates the test(s) accordingly. It gives a nice error if users forget that.
2) It fixes macos support by explicitly linking against the Enzyme build folder. This doesn't cover CI macos yet.
3) It fixes the issue that setting ENZYME_RUNPASS was ignored by enzyme and in fact did not schedule enzyme's opt pass.
4) It also re-enables support for various other values for the autodiff flag, which were ignored since the refactor.
5) I merged some improvements to Enzyme core, which means we do not longer depend on LLVM being build with the Plugin Interface enabled.
6) Unrelated to other fixes, this changes `rustc_autodiff` to `EncodeCrossCrate::Yes`. It is not enough on it's own to enable usage of Enzyme in libraries, but it is for sure a piece of the fixes needed to get this to work.
try-job: x86_64-gnu
r? `@oli-obk`
Tracking:
- https://github.com/rust-lang/rust/issues/124509
Add a notice about missing GCC sources into source tarballs
I didn't use `.gitkeep`, because that would be a hidden file that might not be noticed, whereas we want to let people inspecting the archive know why the sources are missing.
Inspired by https://github.com/rust-lang/rust/issues/137332.
r? `@onur-ozkan`
