rust/src/tools/opt-dist
Weihang Lo e24be071e3
feat: vendor crates required by opt-dist to collect profiles
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.

The size of `rustc-src-nightly.tar.xz` before and after this change:

* Before: 298M
* After: 323M (+8%)

Size change might or might not be a concern.
See the previous discussion: https://github.com/rust-lang/rust/pull/125166#issuecomment-2113626468

Previous efforts on making:

* https://github.com/rust-lang/rust/pull/125125
* https://github.com/rust-lang/rust/pull/125166

---

Note that extra works still need to be done to make it fully vendored.

* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
  directory when collecting profiles from some of these packages.
  This "tmp" working directory usage make it impossible for Cargo to pick
  up the correct vendor sources setting in `.cargo/config.toml` bundled
  in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
  suite. However it rolls out its own `config.toml` without setting
  `vendor = true`, and that results in `./vendor/` directory removed.
  [^2]

[^1]: 4f313add60/collector/src/compile/benchmark/mod.rs (L164-L173)
[^2]: 606afbb617/src/tools/opt-dist/src/tests.rs (L62-L77)
2024-06-09 12:33:11 -04:00
..
src feat: vendor crates required by opt-dist to collect profiles 2024-06-09 12:33:11 -04:00
Cargo.toml refactor(opt-dist): use rustc-perf from rustc checkout 2024-05-20 15:01:10 +00:00
README.md Port PGO/LTO/BOLT optimized build pipeline to Rust 2023-07-09 08:39:50 +02:00

Optimized build pipeline

This binary implements a heavily optimized build pipeline for rustc and LLVM artifacts that are used for both for benchmarking using the perf. bot and for final distribution to users.

It uses LTO, PGO and BOLT to optimize the compiler and LLVM as much as possible. This logic is not part of bootstrap, because it needs to invoke bootstrap multiple times, force-rebuild various artifacts repeatedly and sometimes go around bootstrap's cache mechanism.