This resolves issues where the cross-build of LLVM fails because it tries to
link to the host's system libraries instead of the target's system libraries.
Detect configuration for LLVM during setup
This is a first draft to address #77579, setting `download-ci-llvm` to true on Linux, but I could also implement the `if-available` setting mentioned in the issue.
On other platforms I was thinking about using [the which crate](https://crates.io/crates/which), if adding a dependency on it is considered okay of course, to detect the presence of `llvm-config` in the path, and use it if found. Still a work in progress of course.
We need a fresh input buffer each time, or we reuse the previous
data (since `read_line` appends).
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Put all()'s otuput in the order we want to print things in, and add a
comment about why they are in this order. Provide purpose() and
all_for_help(). Use these things everywhere.
Move all the abbrev character ("a", "b", etc.) processing into
interactive_path.
No functional change.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
bootstrap: only use compiler-builtins-c if they exist
The assignment of `features` above was added in rust-lang#60981, but
never used. Presumably the intent was to replace the string literal here
with it.
While I'm in the area, `compiler_builtins_c_feature` doesn't need to be
a `String`.
I'm not entirely sure of a great way to locally test this -- `./x.py test`
passed on my machine, but 🤷♂️.
r? @alexcrichton
bootstrap: set correct path for the build-manifest binary
This changes the path of the binary inside the tarball to be:
```
build-manifest-{release}-{target}/build-manifest/bin/build-manifest
```
...instead of:
```
build-manifest-{release}-{target}/build-manifest/bin/build-manifest/build-manifest
```
r? @Mark-Simulacrum
Include `llvm-dis`, `llc` and `opt` in `llvm-tools-preview` component
Fixes#55890
It's useful to have `llc` and `opt` available when debugging an LLVM
miscompilation,.
Mention rustdoc in `x.py setup`
This lets new contributors know which option they should pick; previously it wasn't clear 'compiler' also included rustdoc.
Unresolved questions: should this say 'compiler and tools' instead? I don't know of any tools that are modified in-tree other than rustdoc, though.
r? @Mark-Simulacrum
Give an error when running `x.py test --stage 0 src/test/ui`
The error can be overridden with `COMPILETEST_FORCE_STAGE0=1`. In practice I don't know why anyone would do this.
r? @Mark-Simulacrum
Closes https://github.com/rust-lang/rust/issues/77711
Dist build manifest
This PR makes two changes that should remove a significant chunk of the time spent in our release process: cloning the `rust-lang/rust` monorepo, all its submodules, and building `bootstrap` to then invoke `build-manifest`:
* `build-manifest` doesn't rely on a clone of the monorepo being present anymore. The only remaining bit of information it fetched from it (the Rust version) is instead bundled in the binary.
* A new "component" is added, `build-manifest`. That component includes a prebuilt version of the tool, and it's *not* included in the Rustup manifest. This will allow `promote-release` to directly invoke the tool without interacting with our build system.
* The Linux x86_64 CI is changed to also build the component mentioned above. It's the only CI builder tasked to do so, and to cleanly support this a new `--include-default-paths` flag was added to `./x.py`.
* The `BUILD_MANIFEST_NUM_THREADS` environment variable is added to configure the number of threads at runtime.
This PR is best reviewed commit-by-commit.
r? `@Mark-Simulacrum`
This commit changes the way build-manifest is invoked, to let it accept
the Rust version directly instead of requiring the path of the Rust
monorepo and letting build-manifest figure out the path on its own.
This allows to run build-manifest without a clone of the monorepo.
More print statementsstatements lol
Solved the basic case of eliminating check_version ifk_version if subcommand = setup
Finished v1
checking out old bootstrap.py
checked out old irrelevant files
fixed tidy
Moved VERSION from bin/main.rs to lib.rs
Fixed semicolon return issue
x.py fmt
The assignment of `features` above was added in rust-lang#60981, but
never used. Presumably the intent was to replace the string literal here
with it.
While I'm in the area, `compiler_builtins_c_feature` doesn't need to be
a `String`.
The format of the tarballs produced by CI is roughly the following:
{component}-{release}-{target}.{ext}
While on the beta and nightly channels `{release}` is just the channel
name, on the stable channel is either the Rust version or the version of
the component we're shipping:
cargo-0.47.0-x86_64-unknown-linux-gnu.tar.xz
clippy-0.0.212-x86_64-unknown-linux-gnu.tar.xz
llvm-tools-1.46.0-x86_64-unknown-linux-gnu.tar.xz
miri-0.1.0-x86_64-unknown-linux-gnu.tar.xz
rls-1.41.0-x86_64-unknown-linux-gnu.tar.xz
rust-1.46.0-x86_64-unknown-linux-gnu.tar.xz
...
This makes it really hard to get the package URL without having access
to the manifest (and there is no manifest on ci-artifacts.rlo), as there
is no consistent version number to use.
This commit addresses the problem by always using the Rust version
number as `{release}` for the stable channel, regardless of the version
number of the component we're shipping. I chose that instead of "stable"
to avoid breaking the URL scheme *that* much.
Rustup should not be affected by this change, as it fetches the URLs
from the manifest. Unfortunately we don't have a way to test other
clients before making a stable release, as this change only affects the
stable channel.
Add c as a shorthand check alternative for new options #77603
There is a missing "c" that is a shorthand for "check" in newly added match arm for handling check-specific options.
Fix suggestions for x.py setup
#76631 introduced a new `setup` command to x.py
By default the command prompts for a profile to use:
```
Welcome to the Rust project! What do you want to do with x.py?
a) Contribute to the standard library
b) Contribute to the compiler
c) Contribute to the compiler, and also modify LLVM or codegen
d) Install Rust from source
```
and then displays command suggestions, depending on which profile was chosen. However [the mapping between chosen profile](9cba260df0/src/bootstrap/setup.rs (L75-L85)) and [suggestion](9cba260df0/src/bootstrap/setup.rs (L42-L47)) isn't exact, leading to suggestions not being shown if the user presses `c` or `d`. (because "c" is translated to "llvm" and "d" to "maintainer", but suggestions trigger for "codegen" and "user" respectively)
A more thorough refactor would stop using "strings-as-type" to make sure this kind of error doesn't happen, but it may be overkill for that kind of "script" program?
Tagging the setup command author: @jyn514
Add a command to install a git hook to automatically run `x.py test tidy --bless`
Some folks (such as myself) would probably find a lot of convenience in a pre-commit hook that automatically runs tidy before committing, to avoid burning CI time learning that your commit wasn't tidy.
I'm absolutely positive I have missed some stuff. I basically just got this to where you can run `./x.py run install-git-hook` and then clicked the commit button. Please let me know what else you'd like me to add before this can be merged!
[rustc-dev-guide companion PR](https://github.com/rust-lang/rustc-dev-guide/pull/848)
Rename bootstrap/defaults/{config.toml.PROFILE => config.PROFILE.toml}
This allows these files to have okay syntax highlighting in editors, and helps avoid nagging from editors which want to suggest that I install a plugin for `*.library` files to view the `config.toml.library` or whatever.
It's a very minor change.
r?@jyn514
Improve build-manifest to work with the improved promote-release
This PR makes some changes to build-manifest to have it work better with the other improvements I'm making to [promote-release](https://github.com/rust-lang/promote-release).
A new way to invoke the tool was added: `./x.py run src/tools/build-manifest`. The new invocation disables the generation of `.sha256` files and the generation of GPG signatures, as those steps are not tied to the Rust version we're building the manifest of: handling them in `promote-release` will improve the maintenability of our release process. Invocations through the old command (`./x.py dist hash-and-sign`) are referred inside the source code as "legacy". The new invocation also enables internal parallelism, disabled on legacy to avoid overloading our old server.
Improvements were also made on how the checksums included in the manifest are generated:
* The manifest is first generated with placeholder checksums, and then a function walks through the manifes and calculates only the needed hashes. Before this PR, all the hashes were calculated beforehand, including the hashes of unused files.
* Calculating the hashes is now done in parallel with rayon, to better utilize all the available disk bandwidth.
* The `sha2` crate is now used instead of the `sha256sum` CLI tool: this avoids the overhead of calling another process, but more importantly enables hardware acceleration whenever available (the `sha256sum` CLI tool doesn't support it at all).
r? @Mark-Simulacrum
This PR is best reviewed commit-by-commit.
This matches Cargo behavior and avoids the (somewhat expensive) double checking,
as well as the unfortunate duplicate error messages (#76822,
rust-lang/cargo#5128).
