Fix cross compiling dist/build invocations
I am uncertain why the first commit is not affecting CI. I suspect it's because we pass --disable-docs on most of our cross-compilation builders. The second commit doesn't affect CI because CI runs x.py dist, not x.py build.
Both commits are standalone; together they should resolve#76733. The first commit doesn't really fix that issue but rather just fixes cross-compiled x.py dist, resolving a bug introduced in #76549.
Add sample defaults for config.toml
- Allow including defaults in `src/bootstrap/defaults` using `profile = "..."`.
- Add default config files, with a README noting they're experimental and asking you to open an issue if you run into trouble. The config files have comments explaining why the defaults are set.
- Combine config files using the `merge` dependency.
This introduces a new dependency on `merge` that hasn't yet been vetted.
I want to improve the output when `include = "x"` isn't found:
```
thread 'main' panicked at 'fs::read_to_string(&file) failed with No such file or directory (os error 2) ("configuration file did not exist")', src/bootstrap/config.rs:522:28
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failed to run: /home/joshua/rustc/build/bootstrap/debug/bootstrap test tidy
Build completed unsuccessfully in 0:00:00
```
However that seems like it could be fixed in a follow-up.
Closes#76619
- Allow including defaults in `src/bootstrap/defaults` using `profile = "..."`
- Add default config files
- Combine config files using the merge dependency.
- Add comments to default config files
- Add a README asking to open an issue if the defaults are bad
- Give a loud error if trying to merge `.target`, since it's not
currently supported
- Use an exhaustive match
- Use `<none>` in config.toml.example to avoid confusion
- Fix bugs in `Merge` derives
Previously, it would completely ignore the profile defaults if there
were any settings in `config.toml`. I sent an email to the `merge` maintainer
asking them to make the behavior in this commit the default.
This introduces a new dependency on `merge` that hasn't yet been vetted.
I want to improve the output when `include = "x"` isn't found:
```
thread 'main' panicked at 'fs::read_to_string(&file) failed with No such file or directory (os error 2) ("configuration file did not exist")', src/bootstrap/config.rs:522:28
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failed to run: /home/joshua/rustc/build/bootstrap/debug/bootstrap test tidy
Build completed unsuccessfully in 0:00:00
```
However that seems like it could be fixed in a follow-up.
This requires that bootstrap is run from the same worktree as the sources it'll
build, but this is basically required for the build to work anyway. You can
still run it from a different directory, just that the files it builds must be
beside it.
This moves build triple discovery for rustbuild from bootstrap.py into a build
script, meaning it will "just work" if building rustbuild via Cargo rather than
Python.
Move the version number to a plaintext file
The Rust version number is currently embedded in bootstrap's source code, which makes it hard to update it automatically or access it outside of ./x.py (as you'd have to parse the source code).
This PR moves the version number to a standalone plaintext file, which makes accessing or updating it trivial.
r? @Mark-Simulacrum
Don't generate bootstrap usage unless it's needed
Previously, `x.py` would unconditionally run `x.py build` to get the
help message. After https://github.com/rust-lang/rust/issues/76165,
when checking the CI stage was moved into `Config`, that would cause an
assertion failure (but only only in CI!):
```
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `1`,
right: `2`', src/bootstrap/config.rs:619:49
```
This changes bootstrap to only generate a help message when it needs
to (when someone passes `--help`).
r? @Mark-Simulacrum
This should fix the CI failures in https://github.com/rust-lang/rust/pull/76797 and https://github.com/rust-lang/rust/pull/75991.
The Rust version number is currently embedded in bootstrap's source
code, which makes it hard to update it automatically or access it
outside of ./x.py (as you'd have to parse the source code).
This commit moves the version number to a standalone plaintext file,
which makes accessing or updating it trivial.
This isn't an issue for most folks who use x.py dist, which will directly depend
on this. But for x.py build, if we don't properly set target here rustdoc will
not be built.
Currently, there is not a default-on step for generating a rustc for a given
target either, so we will fail to build a rustc as well.
Previously, `x.py` would unconditionally run `x.py build` to get the
help message. After https://github.com/rust-lang/rust/issues/76165,
when checking the CI stage was moved into `Config`, that would cause an
assertion failure (but only only in CI!):
```
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `1`,
right: `2`', src/bootstrap/config.rs:619:49
```
This changes bootstrap to only generate a help message when it needs
to (when someone passes `--help`).
The performance difference is negligible, but it makes me feel better.
Note that this does not remove some clones in `config`, because it would
require changing the logic around (and performance doesn't matter
for bootstrap).
Fix generating rustc docs with non-default lib directory.
If `libdir` is set in `config.toml`, then the tool to generate the rustc docs was unable to run `rustc` because it could not find the shared libraries. The solution is to set the dylib search path to include the libdir.
I changed the API of `add_rustc_lib_path` to take `Command` instead of `Cargo` to try to share the code in several places. This is how it worked before https://github.com/rust-lang/rust/pull/64316, and I think this still retains the spirit of that change.
Fixes#76702
Rollup of 10 pull requests
Successful merges:
- #76056 (Add more info for Vec Drain doc)
- #76062 (Vec slice example fix style and show type elision)
- #76262 (Use inline(never) instead of cold)
- #76335 (Make all methods of `Duration` unstably const)
- #76366 (Add Arith Tests in Library)
- #76369 (Move Various str tests in library)
- #76534 (Add doc comments for From impls)
- #76622 (Update bootstrap readme)
- #76641 (Some cleanup changes and commenting)
- #76662 (Fix liballoc test suite for Miri)
Failed merges:
r? `@ghost`
Update bootstrap readme
- Reflect changes in x.py defaults
- Remove recommendation to use nightly for incremental; it works fine on
beta
- Remove note that incremental chooses stage 1 by default; stage 1 is
already the default
- Update Discord -> Zulip
r? @Mark-Simulacrum
Make the default stage for x.py configurable
This also allows configuring each sub-command individually.
Possibly #76617 should land before this? I don't feel strongly either way, I don't mind waiting.
Closes https://github.com/rust-lang/rust/issues/76165.
r? `@Mark-Simulacrum`
This allows configuring the default stage for each sub-command individually.
- Normalize the stage as early as possible, so there's no confusion
about which to use.
- Don't add an explicit `stage` option in config.toml
This offers no more flexibility than `*_stage` and makes it confusing
which takes precedence.
- Always give `--stage N` precedence over config.toml
- Fix bootstrap tests
This changes the tests to go through `Config::parse` so that they test
the actual defaults, not the dummy ones provided by `default_opts`. To
make this workable (and independent of the environment), it does not
read `config.toml` for tests.
Windows doesn't quite support dynamic linking to LLVM yet, but on other
platforms we do. In #76708, it was discovered that we dynamically link to LLVM
from the LLVM tools (e.g., rust-lld), so we need the shared LLVM library to link
against. That means that if we do not have a shared link to LLVM, and want LLVM
tools to work, we'd be shipping two copies of LLVM on all of these platforms:
one in librustc_driver and one in libLLVM.
Also introduce an error into rustbuild if we do end up configured for shared
linking on Windows.
Auto-generate lint documentation.
This adds a tool which will generate the lint documentation in the rustc book automatically. This is motivated by keeping the documentation up-to-date, and consistently formatted. It also ensures the examples are correct and that they actually generate the expected lint. The lint groups table is also auto-generated. See https://github.com/rust-lang/compiler-team/issues/349 for the original proposal.
An outline of how this works:
- The `declare_lint!` macro now accepts a doc comment where the documentation is written. This is inspired by how clippy works.
- A new tool `src/tools/lint-docs` scrapes the documentation and adds it to the rustc book during the build.
- It runs each example and verifies its output and embeds the output in the book.
- It does a few formatting checks.
- It verifies that every lint is documented.
- Groups are collected from `rustc -W help`.
I updated the documentation for all the missing lints. I have also added an "Explanation" section to each lint providing a reason for the lint and suggestions on how to resolve it.
This can lead towards a future enhancement of possibly showing these docs via the `--explain` flag to make them easily accessible and discoverable.
Add a dedicated debug-logging option to config.toml
`@Mark-Simulacrum` and I were talking in zulip and we found that turning on debug/trace logging in rustc is fairly confusing, as it effectively depends on debug-assertions and is not documented as such. `@Mark-Simulacrum` mentioned that we should probably have a separate option for logging anyways.
this diff adds that, having the option follow debug-assertions (so everyone's existing config.toml should be fine) and if the option is false
to test I ran ./x.py test <something> twice, once with `debug-logging = false` and once with `debug-logging = true` and made sure i only saw trace's when it was true
Download LLVM from CI to bootstrap (linux-only to start)
This follows #76332, adding support for using CI-built LLVM rather than building it locally. This should essentially "just work," but is left off by default in this PR.
While we can support downloading LLVM for multiple host triples, this currently only downloads it for the build triple. That said, it should be possible to expand this relatively easily should multiple host triples be desired. Most people shouldn't be adjusting host/target triples though, so this should cover most use cases.
Currently this downloads LLVM for the last bors-authored commit in the `git log`. This is a bit suboptimal -- we want the last bors-authored commit that touched the llvm-project submodule in basically all cases. But for now this just adds an extra ~20 MB download when rebasing atop latest master. Once we have a submodule bump landing after #76332, we can fix this behavior to reduce downloads further.
This avoids missing a shared build when uplifting LLVM artifacts into the
sysroot. We were already producing a shared link anyway, though, so this is not
a visible change from the end user's perspective.
Add host triples to CI builders
This is a follow-up to #76415, which changed how x.py interprets cross-compilation target/host flags. This should fix the known cases, but I'm still working through CI logs before/after that PR to identify if anything else is missing.
It is really painful to inspect differences in what was built in CI if things
are appearing and disappearing randomly as they hover around the 100ms mark. No
matter what we choose there's always going to be quite a bit of variability on
CI in timing, so we're going to see things appear and vanish.
- Reflect changes in x.py defaults
- Remove recommendation to use nightly for incremental; it works fine on
beta
- Remove note that incremental chooses stage 1 by default; stage 1 is
already the default
- Update Discord -> Zulip
rustbuild: avoid trying to inversely cross-compile for build triple from host triples
This changes rustbuild's cross compilation logic to better match what users expect,
particularly, avoiding trying to inverse cross-compile for the build triple from host triples.
That is, if build=A, host=B, target=B, we do not want to try and compile for A from B.
Indeed, the only "known to run" triple when cross-compiling is the build triple A.
When testing for a particular target we need to be able to run binaries compiled for
that target though.
The last commit also modifies the default set of host/target triples to avoid producing
needless artifacts for the build triple:
The new behavior is to respect --host and --target when passed as the *only*
configured triples (no triples are implicitly added). The default for --host is
the build triple, and the default for --target is the host triple(s), either
configured or the default build triple.
Fixes#76333
r? `@alexcrichton` if possible, otherwise we'll need to hunt down a reviewer
Previously, the CLI --target/--host definitions and configured options differed
in their effect: when setting these on the CLI, only the passed triples would be
compiled for, while in config.toml we would also compile for the build triple
and any host triples. This is needlessly confusing; users expect --target and
--host to be identical to editing the configuration file.
The new behavior is to respect --host and --target when passed as the *only*
configured triples (no triples are implicitly added). The default for --host is
the build triple, and the default for --target is the host triple(s), either
configured or the default build triple.
rustc is a natively cross-compiling compiler, and generally none of our steps
should care whether they are using a compiler built of triple A or B, just the
--target directive being passed to the running compiler. e.g., when building for
some target C, you don't generally want to build two stds: one with a host A
compiler and the other with a host B compiler. Just one std is sufficient.
Only copy LLVM into rust-dev with internal LLVM
This avoids needing to figure out where to locate each of the components with an
external LLVM. This component isn't manifested for rustup consumption and
generally shouldn't matter for anyone except Rust's CI, so it is fine for it to not be
complete elsewhere.
Fixes#76572.
r? `@alexcrichton`
rustbuild: Do not use `rust-mingw` component when bootstrapping windows-gnu targets
Addresses https://github.com/rust-lang/rust/pull/76326#issuecomment-687273473 (ancient `x86_64-w64-mingw32-gcc` is selected as a linker wrapper, which is not usable in `use_lld=true` mode).
Perhaps the comment about incompatible mingw was true in the past, but many things changed since then.
With this change I was able to build everything successfully locally using a newer mingw toolchain, if it passes through the older toolchain on CI, then it should be good, I think.
