- Use stage 2 for makefile
- Move assert to builder
- Don't add an assert for --help
- Allow --stage 0 if passed explicitly
- Don't assert defaults during tests
Otherwise it's impossible to test the defaults!
From [a conversation in discord](https://discordapp.com/channels/442252698964721669/443151243398086667/719200989269327882):
> Linking seems to consume all available RAM, leading to the OS to swap memory to disk and slowing down everything in the process
Compiling itself doesn't seem to take up as much RAM, and I'm only looking to check whether a minimal testcase can be compiled by rustc, where the runtime performance isn't much of an issue
> do you have debug = true or debuginfo-level = 2 in config.toml?
> if so I think that results in over 2GB of debuginfo nowadays and is likely the culprit
> which might mean we're giving out bad advice :(
Anecdotally, this sped up my stage 1 build from 15 to 10 minutes.
This still adds line numbers, it only removes variable and type information.
- Improve wording for debuginfo description
Co-authored-by: Teymour Aldridge <42674621+teymour-aldridge@users.noreply.github.com>
- Set rustc to build only when explicitly asked for
This allows building the stage2 rustc artifacts, which nothing depends
on.
Previously the behavior was as follows (where stageN <-> stage(N-1) artifacts, except for stage0 libstd):
- `x.py build --stage 0`:
- stage0 libstd
- stage1 rustc (but without putting rustc in stage0/)
This leaves you without any rustc at all except for the beta compiler
(https://github.com/rust-lang/rust/issues/73519). This is never what you want.
- `x.py build --stage 1`:
- stage0 libstd
- stage1 rustc
- stage1 libstd
- stage1 rustdoc
- stage2 rustc
This leaves you with a broken stage2 rustc which doesn't even have
libcore and is effectively useless. Additionally, it compiles rustc
twice, which is not normally what you want.
- `x.py build --stage 2`:
- stage0 libstd
- stage1 rustc
- stage1 libstd
- stage2 rustc
- stage2 rustdoc and tools
This builds all tools in release mode. This is the correct usage for CI,
but takes far to long for development.
Now the behavior is as follows:
- `x.py build --stage 0`:
- stage0 libstd
This is suitable for contributors only working on the standard library,
as it means rustc never has to be compiled.
- `x.py build --stage 1`:
- stage0 libstd
- stage1 rustc
- stage1 libstd
- stage1 rustdoc
This is suitable for contributors working on the compiler. It ensures
that you have a working rustc and libstd without having to pass
`src/libstd` in addition.
- `x.py build --stage 2`:
- stage0 libstd
- stage1 rustc
- stage1 libstd
- stage2 rustc
- stage2 libstd
- stage2 rustdoc
This is suitable for debugging errors which only appear with the stage2
compiler.
- `x.py build --stage 2 src/libstd src/rustc`
- stage0 libstd
- stage1 rustc
- stage1 libstd
- stage2 rustc
- stage2 libstd
- stage2 rustdoc, tools, etc.
- stage2 rustc artifacts ('stage3')
This is suitable for CI, which wants all tools in release mode.
However, most of the use cases for this should use `x.py dist` instead,
which builds all the tools without each having to be named individually.
### x.py build/test: stage 1
I've seen very few people who actually use full stage 2 builds on purpose. These compile rustc and libstd twice and don't give you much more information than a stage 1 build (except in rare cases like https://github.com/rust-lang/rust/pull/68692#discussion_r376392145). For new contributors, this makes the build process even more daunting than it already is. As long as CI is changed to use `--stage 2` I see no downside here.
### x.py bench/dist/install: stage 2
These commands have to do with a finished, optimized version of rustc. It seems very rare to want to use these with a stage 1 build.
### x.py doc: stage 0
Normally when you document things you're just fixing a typo. In this case there is no need to build the whole rust compiler, since the documentation will usually be the same when generated with the beta compiler or with stage 1.
Note that for this release cycle only there will be a significant different between stage0 and stage1 docs: https://github.com/rust-lang/rust/pull/73101. However most of the time this will not be the case.
Revert libbacktrace -> gimli
This reverts 4cbd265c11028f8d7b8513db3cc1e8d7a36d8964 (and technically 79673d3009 but it's made empty by previous reverts).
The current plan is to land this PR as a temporary change, so that we can get a better handle on the regressions introduced by it. Trying to fix/examine them in master is difficult, and we want to be better able to evaluate them without impact to other PRs being landed in the mean time.
That said, it is currently *my* belief that gimli, in one form or another, will need to land sometime soon. I think it's quite likely that it may slip a week or two, but I would personally push for re-landing it then "regardless" of the regressions. We should try to focus efforts on understanding and removing as much of the performance impact as possible, as everyone pretty much agrees that it should be quite minimal (and entirely in the linker, basically).
r? @nnethercote
Stabilize control-flow-guard codegen option
This is the stabilization PR discussed in #68793. It converts the `-Z control-flow-guard` debugging option into a codegen option (`-C control-flow-guard`), and changes the associated tests.
Teach bootstrap install and dist commands about TargetSelection
With this, we can now use a target JSON file to build a
cross-compiler:
```
x.py install --host ../aarch64-apple-darwin.json --target aarch64-apple-darwin
```
r? @Mark-Simulacrum
With this, we can now use a target JSON file to build a
cross-compiler:
```
x.py install --host ../aarch64-apple-darwin.json --target aarch64-apple-darwin
```
Generating the coverage map
@tmandry @wesleywiser
rustc now generates the coverage map and can support (limited)
coverage report generation, at the function level.
Example commands to generate a coverage report:
```shell
$ BUILD=$HOME/rust/build/x86_64-unknown-linux-gnu
$ $BUILD/stage1/bin/rustc -Zinstrument-coverage \
$HOME/rust/src/test/run-make-fulldeps/instrument-coverage/main.rs
$ LLVM_PROFILE_FILE="main.profraw" ./main
called
$ $BUILD/llvm/bin/llvm-profdata merge -sparse main.profraw -o main.profdata
$ $BUILD/llvm/bin/llvm-cov show --instr-profile=main.profdata main
```
r? @wesleywiser
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage instrumentation
bootstrap.py: patch RPATH on NixOS to handle the new zlib dependency.
This is a stop-gap until #74420 is resolved (assuming we'll patch beta to statically link zlib).
However, I've been meaning to rewrite the NixOS support we have in `bootstrap.py` for a while now, and had to in order to cleanly add zlib as a dependency (the second commit is a relatively small delta in functionality, compared to the first).
Previously, we would extract the `ld-linux.so` path from the output of `ldd /run/current-system/sw/bin/sh`, which assumes a lot. On top of that we didn't use any symlinks, which meant if the user ran GC (`nix-collect-garbage`), e.g. after updating their system, their `stage0` binaries would suddenly be broken (i.e. referring to files that no longer exist).
We were also using `patchelf` directly, assuming it can be found in `$PATH` (which is not necessarily true).
My new approach relies on using `nix-build` to get the following "derivations" (packages, more or less):
* `stdenv.cc.bintools`, which has a `nix-support/dynamic-linker` file containing the path to `ld-linux.so`
* reading this file is [the canonical way to run `patchelf --set-interpreter`](https://github.com/NixOS/nixpkgs/search?l=Nix&q=%22--set-interpreter+%24%28cat+%24NIX_CC%2Fnix-support%2Fdynamic-linker%29%22)
* `patchelf` (so that the user doesn't need to have it installed)
* `zlib`, for the `libz.so` dependency of `libLLVM-*.so` (until #74420 is resolved, presumably)
This is closer to how software is built on Nix, but I've tried to keep it as simple as possible (and not add e.g. a `stage0.nix` file).
Symlinks to each of those dependencies are kept in `stage0/.nix-deps`, which prevents GC from invalidating `stage0` binaries.
r? @nagisa cc @Mark-Simulacrum @oli-obk @davidtwco
Teach bootstrap about target files vs target triples
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
Enforce even more the code blocks attributes check through rustdoc
`rustdoc` now has a lint which allows it to warn if a code block attribute is malformated (which can end up in bad situations, even more in case of testing examples!). Now it'll fail if such a situation is encountered when testing markdown code blocks examples.
r? @Mark-Simulacrum
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
Use local links in the alloc docs.
Links to other crates (like core) from the alloc crate were incorrectly using the `https://doc.rust-lang.org/nightly/` absolute (remote) links, instead of relative (local) links. For example, the link to `Result` at https://doc.rust-lang.org/1.44.1/alloc/vec/struct.Vec.html#method.try_reserve goes to /nightly/.
This is because alloc was being documented before core, and rustdoc relies on the existence of the local directory to know if it should use a local or remote link.
There was code that tried to compensate for this (`create_dir_all`), but in #54543 it was broken because instead of running `cargo doc` once for all the crates, it was changed to run `cargo rustdoc` for each crate individually. This means that `create_dir_all` was no longer doing what it was supposed to be doing (creating all the directories before starting).
The solution here is to just build in the correct order (from the dependency leaves towards the root). An alternate solution would be to switch back to running `cargo doc` once (and use RUSTDOCFLAGS for passing in flags). Another alternate solution would be to iterate over the list twice, creating the directories during the first pass.
I also did a little cleanup to remove the "crate-docs" directory. This was added in the past because different crates were built in different directories. Over time, things have been unified (and rustc docs no longer include std), so it is no longer necessary.
RISC-V GNU/Linux as host platform
This PR add a new builder named `dist-riscv64-linux` that builds the compiler toolchain for RISC-V 64-bit GNU/Linux.
r? @alexcrichton
Don't allow `DESTDIR` to influence LLVM builds
When running a command like `DESTDIR=foo x.py install` in a completely
clean build directory, this will cause LLVM to be installed into
`DESTDIR`, which then causes the build to fail later when it attempts
to *use* those LLVM files.
Fix caching issue when building tools.
This fixes a problem with tool builds not being cached properly.
#73297 changed it so that Clippy will participate in the "deny warnings" setting. Unfortunately this causes a problem because Clippy shares the build directory with other tools which do not participate in "deny warnings". Because Cargo does not independently cache artifacts based on different RUSTFLAGS settings, it causes all the shared dependencies to get rebuilt if Clippy ever gets built.
The solution here is to stop using RUSTFLAGS, and just sneak the settings in through the rustc wrapper. Cargo won't know about the different settings, so it will not bust the cache. This should be safe since lint settings on dependencies are ignored. This is how things used to work in the past before #64316.
Alternate solutions:
* Treat Clippy as a "submodule" and don't enforce warnings on it. This was the behavior before #73297. The consequence is that if a warning sneaks into clippy, that the clippy maintainers will need to fix it when they sync clippy back to the clippy repo.
* Just deny warnings on all tools (removing the in-tree/submodule distinction). This is tempting, but with some issues (cc #52336):
* Adding or changing warnings in rustc can be difficult to land because tools have to be updated if they trip the warning. In practice, this isn't too bad. Cargo (and rustfmt) already runs with `deny(warnings)`, so this has been the de-facto standard already (although they do not use the extra lints like `unused_lifetimes`).
* Teach Cargo to add flags to the workspace members, but not dependencies.
* Teach Cargo to add flags without fingerprinting them?
* Teach Cargo to independently cache different RUSTFLAGS artifacts (this was [reverted](https://github.com/rust-lang/cargo/pull/7417) due to complications). This would also unnecessarily rebuild dependencies, but would avoid cache thrashing.
* Teach Cargo about lint settings.
Closes#74016
