This implements support for applying PGO to the rustc compilation step (not
standard library or any tooling, including rustdoc). Expanding PGO to more tools
is not terribly difficult but will involve more work and greater CI time
commitment.
For the same reason of avoiding greater time commitment, this currently avoids
implementing for platforms outside of x86_64-unknown-linux-gnu, though in
practice it should be quite simple to extend over time to more platforms. The
initial implementation is intentionally minimal here to avoid too much work
investment before we start seeing wins for a subset of Rust users.
The choice of workloads to profile here is somewhat arbitrary, but the general
rationale was to aim for a small set that largely avoided time regressions on
perf.rust-lang.org's full suite of crates. The set chosen is libcore, cargo (and
its dependencies), and a few ad-hoc stress tests from perf.rlo. The stress tests
are arguably the most controversial, but they benefit those cases (avoiding
regressions) and do not really remove wins from other benchmarks.
The primary next step after this PR lands is to implement support for PGO in
LLVM. It is unclear whether we can afford a full LLVM rebuild in CI, though, so
the approach taken there may need to be more staggered. rustc-only PGO seems
well affordable on linux at least, giving us up to 20% wall time wins on some
crates for 15 minutes of extra CI time (1 hour up from 45 minutes).
The PGO data is uploaded to allow others to reuse it if attempting to reproduce
the CI build or potentially, in the future, on other platforms where an
off-by-one strategy is used for dist builds at minimal performance cost.
Put all()'s otuput in the order we want to print things in, and add a
comment about why they are in this order. Provide purpose() and
all_for_help(). Use these things everywhere.
Move all the abbrev character ("a", "b", etc.) processing into
interactive_path.
No functional change.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Add c as a shorthand check alternative for new options #77603
There is a missing "c" that is a shorthand for "check" in newly added match arm for handling check-specific options.
This matches Cargo behavior and avoids the (somewhat expensive) double checking,
as well as the unfortunate duplicate error messages (#76822,
rust-lang/cargo#5128).
Add `x.py setup`
Closes#76503.
- Suggest `x.py setup` if config.toml doesn't exist yet
- Prompt for a profile if not given on the command line
- Print the configuration that will be used
- Print helpful starting commands after setup
- Link to the dev-guide after finishing
- Suggest `x.py setup` if config.toml doesn't exist yet (twice, once
before and once after the build)
- Prompt for a profile if not given on the command line
- Print the configuration file that will be used
- Print helpful starting commands after setup
- Link to the dev-guide after finishing
- Note that distro maintainers will see the changelog warning
This keeps only the `std` artifacts compiled by the given stage, not the
compiler. This is useful when working on the latter stages of the
compiler in tandem with the standard library, since you don't have to
rebuild the *entire* compiler when the standard library changes.
Previously, `x.py` would unconditionally run `x.py build` to get the
help message. After https://github.com/rust-lang/rust/issues/76165,
when checking the CI stage was moved into `Config`, that would cause an
assertion failure (but only only in CI!):
```
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `1`,
right: `2`', src/bootstrap/config.rs:619:49
```
This changes bootstrap to only generate a help message when it needs
to (when someone passes `--help`).
Previously, the CLI --target/--host definitions and configured options differed
in their effect: when setting these on the CLI, only the passed triples would be
compiled for, while in config.toml we would also compile for the build triple
and any host triples. This is needlessly confusing; users expect --target and
--host to be identical to editing the configuration file.
The new behavior is to respect --host and --target when passed as the *only*
configured triples (no triples are implicitly added). The default for --host is
the build triple, and the default for --target is the host triple(s), either
configured or the default build triple.
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
Speed up bootstrap a little.
The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4.
An overview of the changes:
- Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config).
- Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior.
- Remove check for `_shim`, I believe all the `_shim` crates are now gone.
- Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree.
- Use relative paths in `./x.py test -h -v` output.
- Some code cleanup (remove unnecessary `find_compiler_crates`, etc.).
- Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output.
- Some doc comments.
bootstrap: read config from $RUST_BOOTSTRAP_CONFIG
This PR modifies bootstrap so that `config.toml` is read first from `RUST_BOOTSTRAP_CONFIG`, then `--config` and finally `config.toml` in the current directory.
This is a subjective change, intended to improve the ergnomics when using "development shells" for rustc development (for example, using tools such as Nix) which set environment variables to ensure a reproducible environment (these development shells can then be version controlled, e.g. [my rustc shell](6b74a5c170/nix/shells/rustc.nix)). By optionally reading `config.toml` from an environment variable, a `config.toml` can be defined in the development shell and a path to it exposed in the `RUST_BOOTSTRAP_CONFIG` environment variable - avoiding the need to manually symlink the contents of this file to `config.toml` in the working directory.
This commit modifies bootstrap so that `config.toml` is read first from
`RUST_BOOTSTRAP_CONFIG`, then `--config` and finally `config.toml` in the
current directory.
This is a subjective change, intended to improve the ergnomics when
using "development shells" for rustc development (for example, using tools
such as Nix) which set environment variables to ensure a reproducible
environment (these development shells can then be version controlled). By
optionally reading `config.toml` from an environment variable, a `config.toml`
can be defined in the development shell and a path to it exposed in the
`RUST_BOOTSTRAP_CONFIG` environment variable - avoiding the need to manually
symlink the contents of this file to `config.toml` in the working
directory.
Signed-off-by: David Wood <david@davidtw.co>
tidy: replace `make check` with `./x.py test` in documentation
This PR includes a minor documentation update for tidy. It replaces the `make check` approach with `./x.py test` and describes how to execute the tidy checks (only) with ~~`./x.py test src/tools/tidy`~~ `./x.py test tidy`.
Clarify where the clippy used in `./x.py clippy` is coming from.
It uses whatever clippy binary was installed via rustup, cargo-install
or otherwise and does NOT use the binary generated by `./x.py build src/tools/clippy`.
