opt-dist: add a flag for running tests
when using `opt-dist local` user probably won't need to run tests (for various reasons). currently the only way to disable them is to set `TRY_DIST_BUILD=1`, which is not obvious and can be bad for non-CI envronments (as I guess)
possibly the `run_tests` name can be confusing too...
r? Kobzol
try-job: dist-x86_64-linux
try-job: dist-x86_64-msvc
when using `opt-dist local` user probably won't need to run tests (for
various reasons). currently the only way to disable them is to set
`TRY_DIST_BUILD=1`, which is not obvious and can be bad for non-CI
envronments (as I guess)
Add `dist::Gcc` build step
This PR adds a `dist:Gcc` bootstrap step to distribute a prebuilt `libgccjit.so` from CI on x64 Linux.
With primed sccache, the build takes ~4 minutes on CI, and produces a 50 MiB archive.
I want to land this before adding something akin to `[gcc] download-ci-gcc = true`, to already have the artifacts available on CI, to make it easier to setup the download merge-base logic.
r? ``@ghost``
Skip const OOM tests on AArch64 Linux through explicit annotations
instead of inside opt-dist.
Intended to avoid confusion in cases like #135952.
Prerequisite for https://github.com/rust-lang/rust/pull/135960.
As of the stabilization of `std::backtrace` in Rust 1.65, this package flag has
no effect other than to enable an unused dependency on the `backtrace` crate.
opt-dist overrides the stage 0 compiler with previously compiled compilers,
which can cause confusion in bootstrap's target sanity checks. It is best to
skip that check.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
These are the default package set required by opt-dist to correctly work,
hence for people wanting to build a production grade of rustc in a
sandboxed / air-gapped environment, these need to be vendored.
The size of `rustc-src-nightly.tar.xz` before and after this change:
* Before: 298M
* After: 323M (+8%)
These crates are the default set of packages required by opt-dist
to correctly work, hence for people wanting to build a production grade
of rustc in an sandboxed / air-gapped environment, these need to be vendored.
The size of `rustc-src-nightly.tar.xz` before and after this change:
* Before: 298M
* After: 323M (+8%)
Size change might or might not be a concern.
See the previous discussion: https://github.com/rust-lang/rust/pull/125166#issuecomment-2113626468
Previous efforts on making:
* https://github.com/rust-lang/rust/pull/125125
* https://github.com/rust-lang/rust/pull/125166
---
Note that extra works still need to be done to make it fully vendored.
* The current pinned rustc-perf uses `tempfile::Tempdir` as the working
directory when collecting profiles from some of these packages.
This "tmp" working directory usage make it impossible for Cargo to pick
up the correct vendor sources setting in `.cargo/config.toml` bundled
in the rustc-src tarball. [^1]
* opt-dist verifies the final built rustc against a subset of rustc test
suite. However it rolls out its own `config.toml` without setting
`vendor = true`, and that results in `./vendor/` directory removed.
[^2]
[^1]: 4f313add60/collector/src/compile/benchmark/mod.rs (L164-L173)
[^2]: 606afbb617/src/tools/opt-dist/src/tests.rs (L62-L77)
The flag propagates cargo configs to `rustc-perf --cargo-config`,
which is particularly useful when the environment is air-gapped,
and you want to use the default set of training crates vendored
in the rustc-src tarball.
This is another step toward making opt-dist work in sandboxed environments
opt-dist verifies the final built rustc against a subset of rustc test
suite. However it overwrote the pre-existing `config.toml` [^1],
and that results in ./vendor/ directory removed [^2].
Instead of overwriting, this patch use `--set <config-value>` to
override paths to rustc / cargo / llvm-config.
[^1]: 606afbb617/src/tools/opt-dist/src/tests.rs (L62-L77)
[^2]: 8679004993/src/bootstrap/bootstrap.py (L1057)
This replaces the hardcoded rustc-perf commit and ad-hoc downloading and
unpacking of its zipped source with defaulting to use the new rustc-perf
submodule.
While it would be nice to make `opt-dist` able to initialize the
submodule automatically when pointing to a Rust checkout _other_ than
the one opt-dist was built in, that would require a bigger refactor that
moved `update_submodule`, from bootstrap, into build_helper.
Regardless, I imagine it must be quite rare to use `opt-dist` with a
checkout that is neither from a rust-src tarball (which will contain the
submodule), nor the checkout opt-dist itself was built (bootstrap will
update the submodule when opt-dist is built).
[BOLT] Use CDSort and CDSplit
CDSort and CDSplit are the most recent versions of function ordering and function splitting algorithms with some improvements over the previous baseline (ext-tsp and two-way splitting).
LLVM now includes the minor version in the soname, and also changed
the names of shared object files. libLLVM-18.so is now a symlink to
libLLVM.so.18.1. We need to make some changes to support this:
First, we need to run the installed llvm-config binary, rather
than the one from the build directory. This is because the symlink
does not exist in the build directory, but llvm-config requires it.
This looks like an LLVM bug to me, but it's probably a good idea to
use the installed version anyway.
Second, when installing LLVM into the libdir, we need to install
the target of the symlink, ans this is what will get loaded at
runtime.
However, the rust-dev component in particular also needs to
distribute the symlink itself, as download-ci-llvm will end up
invoking llvm-config, which requires the symlink to exist. The
symlink is not shipped in other components.
