miri-test-libstd: add missing BOOTSTRAP_ARGS
Note sure if BOOTSTRAP_ARGS will make any difference here, but all the other x.py invocations have it and I did not *deliberately* leave it away when I added these, so... probably best to add them?
Also don't unnecessarily set BOOTSTRAP_SKIP_TARGET_SANITY while we are at it.
use `download-ci-llvm=true` in the default compiler config
1ca2708e77 made it so that the `src/llvm-project` submodule has to be checkout for `download-ci-llvm = "if-unchanged"` to know if the submodule has been changed, but that is not required, if the submodule hasn't been checkout it cannot have been modified.
~~This PR restore the previous behavior by only updating the submodule if it has already been checkout.~~
This PR makes `download-ci-llvm = true` check if CI llvm is available and make it the default for the compiler profile, as to prevent unnecessarily checking out `src/llvm-project` with `"if-unchanged"`.
r? `````@onur-ozkan`````
Rollup of 9 pull requests
Successful merges:
- #129929 (`rustc_mir_transform` cleanups, round 2)
- #130022 (Dataflow/borrowck lifetime cleanups)
- #130064 (fix ICE in CMSE type validation)
- #130067 (Remove redundant check in `symlink_hard_link` test)
- #130131 (Print a helpful message if any tests were skipped for being up-to-date)
- #130137 (Fix ICE caused by missing span in a region error)
- #130153 (use verbose flag as a default value for `rust.verbose-tests`)
- #130154 (Stabilize `char::MIN`)
- #130158 (Update books)
r? `@ghost`
`@rustbot` modify labels: rollup
Print a helpful message if any tests were skipped for being up-to-date
When running tests without the `--force-rerun` flag, compiletest will automatically skip any tests that (in its judgement) don't need to be run again since the last time they were run.
This is normally very useful, but can occasionally be confusing, especially in edge-cases where up-to-date checking is not completely accurate (or the test is flaky).
This PR makes bootstrap count the number of tests that were ignored for being up-to-date (via a hard-coded check on the ignore reason), and prints a helpful message when that number is nonzero.
---
Sample output:
```text
test result: ok. 4 passed; 0 failed; 17578 ignored; 0 measured; 0 filtered out; finished in 463.79ms
help: ignored 17295 up-to-date tests; use `--force-rerun` to prevent this
Build completed successfully in 0:00:07
```
Distribute rustc_codegen_cranelift for Windows
With support for raw-dylib recently added to cg_clif, and inline assembly support working on Windows for quite a while now, all blockers for distributing cg_clif on Windows that I mentioned in https://github.com/rust-lang/rust/pull/81746#issuecomment-1774099637 are fixed now.
Previously, the logic here was simply checking whether the option was set in `config.toml`.
This approach was not manageable in our CI runners as we set so many options in config.toml.
In reality, those values are not incompatible since they are usually the same value used to generate
the CI llvm. Now, the new logic compares the configuration values with the values used to generate
the CI llvm, so we get more precise results and make the process more manageable.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
explain the options bootstrap passes to curl
also fixes a discrepancy where the rust side doesn't use -L
docs are only on the rust side, since duplicated prose has a tendancy to get out-of-sync, and also because there are talks of removing the python script all together eventually.
bypass linker configuration and cross target check for specific commands
Avoids configuring the linker and checking cross-target-specific tools unless necessary.
Resolves#128180
cc `@ChrisDenton`
It was [pointed out recently][comment] that enabling `wasm-component-ld`
as a host tool is different from other host tools. This commit refactors
the logic to match by deduplicating selection of when to build other
tools and then using the same logic for `wasm-component-ld`.
[comment]: https://github.com/rust-lang/rust/pull/127866#issuecomment-2333434720
Autodiff Upstreaming - enzyme backend
Tracking issue: https://github.com/rust-lang/rust/issues/124509
Part of https://github.com/rust-lang/rust/pull/129175
This PR should allow building Enzyme from source on Tier 1 targets (when also building LLVM), except MSVC.
It's only a small fraction (~200 lines) of the whole upstream PR, but due to bootstrapping and the number of configurations in which rustc can be build I assume that this will be the hardest to merge, so I'm starting with it.
Happy to hear what changes are required to be able to upstream this code.
**Content:**
It contains a new configure flag `--enable-llvm-enzyme`, and will build the new Enzyme submodule when it is set.
**Discussion:**
Apparently Rust CI isn't able to clone repositories outside the rust-lang org? At least I'm seeing this error in CI:
```
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
```
Does that mean we would need to mirror github.com/EnzymeAD/Enzyme in rust-lang, until LLVM upgrades Enzyme from an Incubator project to something that ships as part of the monorepo?
Tracking:
- https://github.com/rust-lang/rust/issues/124509
Make `./x.py <cmd> compiler/<crate>` aware of the crate's features
Does not fix https://github.com/rust-lang/rust/issues/129727 on its own as the way the parallel-compiler cfg and feature flags are setup being generally incompatible with `resolver = 2` but it progresses on the issue. But this should in theory allow compiler crates to work that do not depend on the parallel compiler stuff (so some leaf crates).
forward linker option to lint-docs
This fixes an error found when building the doc for a cross-built toolchain.
```
warning: the code example in lint `unstable_syntax_pre_expansion` in /buildroots/chenx97/rustc-1.80.1-src/compiler/rustc_lint_defs/src/builtin.rs failed to generate the expected output: did not find lint `unstable_syntax_p
re_expansion` in output of example, got:
error: linking with `cc` failed: exit status: 1
...
```
Closes: #129956
copy rustc rustlib artifacts from ci-rustc
We recently (since https://github.com/rust-lang/rust/pull/129311) had an issue because some rustlib files were missing (like: "error[E0463]: can't find crate for rustc_ast") when building tools that rely on rustc. This patch fixes that by copying those files as required.
r? Kobzol
Blocker for https://github.com/rust-lang/rust/pull/122709
bootstrap: Try to track down why `initial_libdir` sometimes fails
When I try to run `x` commands from the command-line, I occasionally see a mysterious failure that looks something like this:
```text
thread 'main' panicked at src/lib.rs:341:14:
called `Result::unwrap()` on an `Err` value: StripPrefixError(())
```
It happens often enough to be annoying, but rarely enough that I can't reproduce it at will. The error message points to a particular `unwrap` call, but doesn't include enough context to determine *why* the failure occurs.
Re-running the command almost always works, so I suspect some kind of filesystem race condition (possibly involving VSCode invoking bootstrap at the same time), but there's not much I can do with the information I currently have.
So this PR includes some relevant information in the panic message when the failure occurs, in the hope that doing so will make the cause easier to track down when the failure occurs again.
Add target support for RTEMS Arm
# `armv7-rtems-eabihf`
This PR adds a new target for the RTEMS RTOS. To get things started it focuses on Xilinx/AMD Zynq-based targets, but in theory it should also support other armv7-based board support packages in the future.
Given that RTEMS has support for many POSIX functions it is mostly enabling corresponding unix features for the new target.
I also previously started a PR in libc (https://github.com/rust-lang/libc/pull/3561) to add the needed OS specific C-bindings and was told that a PR in this repo is needed first. I will update the PR to the newest version after approval here.
I will probably also need to change one line in the backtrace repo.
Current status is that I could compile rustc for the new target locally (with the updated libc and backtrace) and could compile binaries, link, and execute a simple "Hello World" RTEMS application for the target hardware.
> A proposed target or target-specific patch that substantially changes code shared with other targets (not just target-specific code) must be reviewed and approved by the appropriate team for that shared code before acceptance.
There should be no breaking changes for existing targets. Main changes are adding corresponding `cfg` switches for the RTEMS OS and adding the C binding in libc.
# Tier 3 target policy
> - A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)
I will do the maintenance (for now) further members of the RTEMS community will most likely join once the first steps have been done.
> - Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.
> - Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.
> - If possible, use only letters, numbers, dashes and underscores for the name. Periods (`.`) are known to cause issues in Cargo.
The proposed triple is `armv7-rtems-eabihf`
> - Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.
> - The target must not introduce license incompatibilities.
> - Anything added to the Rust repository must be under the standard Rust license (`MIT OR Apache-2.0`).
> - The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the `tidy` tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
> - Compiling, linking, and emitting functional binaries, libraries, or other code for the target (whether hosted on the target itself or cross-compiling from another target) must not depend on proprietary (non-FOSS) libraries. Host tools built for the target itself may depend on the ordinary runtime libraries supplied by the platform and commonly used by other applications built for the target, but those libraries must not be required for code generation for the target; cross-compilation to the target must not require such libraries at all. For instance, `rustc` built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
> - "onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are _not_ limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.
The tools consists of the cross-compiler toolchain (gcc-based). The RTEMS kernel (BSD license) and parts of the driver stack of FreeBSD (BSD license). All tools are FOSS and publicly available here: https://gitlab.rtems.org/rtems
There are also no new features or dependencies introduced to the Rust code.
> - Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
N/A to me. I am not a reviewer nor Rust team member.
> - Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (`core` for most targets, `alloc` for targets that can support dynamic memory allocation, `std` for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.
`core` and `std` compile. Some advanced features of the `std` lib might not work yet. However, the goal of this tier 3 target it to make it easier for other people to build and run test applications to better identify the unsupported features and work towards enabling them.
> - The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running binaries, or running tests (even if they do not pass), the documentation must explain how to run such binaries or tests for the target, using emulation if possible or dedicated hardware if necessary.
Building is described in platform support doc. Running simple unit tests works. Running the test suite of the stdlib is currently not that easy. Trying to work towards that after the this target has been added to the nightly.
> - Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via ````@`)``` to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
Understood.
> - Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.
Ok
> - Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
> - In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.
I think, I didn't add any breaking changes for any existing targets (see the comment regarding features above).
> - Tier 3 targets must be able to produce assembly using at least one of rustc's supported backends from any host target.
Can produce assembly code via the llvm backend (tested on Linux).
>
> If a tier 3 target stops meeting these requirements, or the target maintainers no longer have interest or time, or the target shows no signs of activity and has not built for some time, or removing the target would improve the quality of the Rust codebase, we may post a PR to remove it; any such PR will be CCed to the target maintainers (and potentially other people who have previously worked on the target), to check potential interest in improving the situation.GIAt this tier, the Rust project provides no official support for a target, so we place minimal requirements on the introduction of targets.
Understood.
r? compiler-team
enable -Zrandomize-layout in debug CI builds
This builds rustc/libs/tools with `-Zrandomize-layout` on *-debug CI runners.
Only a handful of tests and asserts break with that enabled, which is promising. One test was fixable, the rest is dealt with by disabling them through new cargo features or compiletest directives.
The config.toml flag `rust.randomize-layout` defaults to false, so it has to be explicitly enabled for now.
We recently had an issue because some rustlib files were missing (like: "error[E0463]: can't find crate for rustc_ast")
when building tools that rely on rustc. This patch fixes that by copying those files as required.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
custom/external clippy support for bootstrapping
Similar to cargo, rustc, and rustfmt, this adds the support of using custom clippy on bootstrap. It’s designed for those who want to test their own clippy builds or avoid downloading the stage0 clippy.
Closes#121518
