Refactor git change detection in bootstrap
While working on https://github.com/rust-lang/rust/pull/138395, I finally found the courage to delve into the insides of git path change detection in bootstrap, which is used (amongst other things) to detect if we should rebuilt od download `[llvm|rustc|gcc]`. I found it a bit hard to understand, and given that this code was historically quite fragile, I thought that it would be better to rebuild it from scratch.
The previous approach had a bunch of limitations:
- It separated the computation of "are there local changes?" and "what upstream SHA should we use?" even though these two things are intertwined.
- It used hacks to work around what happens on CI.
- It had special cases for CI scattered throughout the codebase, rather than centralized in one place.
- It wasn't documented enough and didn't have tests for the git behavior.
The current approach should hopefully resolve all of that. I implemented a single entrypoint called `check_path_modifications` (naming bikeshed pending, half of the time I spend on this PR was thinking about names, as it's quite tricky here..) that explicitly receives a mode of operation (in CI or outside CI), and accordingly figures out that upstream SHA that we should use for downloading artifacts and it also figures out if there are any local changes. Users of this function can then use this unified output to implement `download-ci-X` and other functionality. Notably, this change detection no longer uses `git merge-base`, which makes it easier to use and doesn't require setting up remotes.
I also added a bunch of integration tests that literally spawn a git repository on disk and then check that the function can deal with various situations (PR CI, auto/try CI, local builds).
After I built this inner layer, I used it for downloading GCC, LLVM and rustc. The latter two (and especially rustc) were using the `last_modified_commit` function before, but in all cases but one this function was actually only used to check if there are any local changes, which was IMO confusing. The LLVM handling would deserve a bit of refactoring, but that's a larger change that can be done as a follow-up.
I hope that the implementation is now clear and easy to understand, so that in combination with the tests we can have more confidence that it does what we want. I tried to include a lot of documentation in the code, so I won't be repeating the actual implementation details here, if there are any questions, I'll add the answers to the documentation too :)
The new approach explicitly supports three scenarios:
- Running on PR CI, where we have one upstream bors parent commit and one PR merge commit made by GitHub.
- Running on try/auto CI, where we have one upstream bors parent commit and one PR merge commit made by bors.
- Running locally, where we assume that we have at least one upstream bors parent commit in our git history.
I removed the handling of upstreams on CI, as I think that it shouldn't be needed and I considered it to be a hack. However, it's possible that there are other use-cases that I haven't considered, so I want to ask around if people have other situations than the three use-cases described above. If there are other such use-cases, I would like to include them in the new centralized implementation and add them to the git test suite, rather than going back to the old ways :)
In particular, the code before relied on `git merge-base`, but I don't see why we can't just lookup the most recent bors commit and assume that is a merge commit that is also upstream? I might be running into Chesterton's Fence here :)
CC `@pietroalbini` To make sure that this won't break downstream users of Rust's CI.
Best reviewed commit by commit.
Companion PRs:
- For testing beta: https://github.com/rust-lang/rust/pull/138597
r? `@onur-ozkan`
Fixes: https://github.com/rust-lang/rust/issues/101907
try-job: x86_64-gnu-aux
try-job: aarch64-gnu
try-job: dist-x86_64-apple
Rollup of 8 pull requests
Successful merges:
- #139617 (Use posix_spawn on cygwin)
- #139921 (improve diagnostic for raw pointer field access with ->)
- #140031 (compiletest: Fix deadline bugs in new executor)
- #140072 (handle function alignment in miri)
- #140104 (Fix auto diff failing on inherent impl blocks)
- #140124 (Update books)
- #140144 (Handle another negated literal in `eat_token_lit`.)
- #140149 (test_nan: ensure the NAN contant is quiet)
r? `@ghost`
`@rustbot` modify labels: rollup
Handle another negated literal in `eat_token_lit`.
Extends the change from #139653, which was on expressions, to literals.
Fixes#140098.
r? ``@petrochenkov``
Update books
## rust-lang/nomicon
1 commits in 0c10c30cc54736c5c194ce98c50e2de84eeb6e79..c76a20f0d987145dcedf05c5c073ce8d91f2e82a
2025-04-15 20:54:57 UTC to 2025-04-15 20:54:57 UTC
- Say that dereferencing a pointer to a ZST is no longer undefined (rust-lang/nomicon#467)
## rust-lang/reference
7 commits in 3340922df189bddcbaad17dc3927d51a76bcd5ed..3bf3402aea982b876eb56c87da17b0685c6461d5
2025-04-18 13:44:45 UTC to 2025-04-17 17:27:01 UTC
- Use `cfg(false)` instead of `cfg(FALSE)` (rust-lang/reference#1763)
- Add `cfg(true)` and `cfg(false)` to conditional compilation (RFC 3695) (rust-lang/reference#1762)
- Refactor rendering with `RenderCtx` (rust-lang/reference#1796)
- attributes/codegen: update aarch64 features (rust-lang/reference#1791)
- Simplify GenericParams grammar (rust-lang/reference#1795)
- Add rule identifiers for the ABI chapter (rust-lang/reference#1793)
- Remove broken footnote links from grammar summary (rust-lang/reference#1794)
compiletest: Fix deadline bugs in new executor
The experimental new executor for compiletest (#139660) was found to have two major bugs in deadline handling for detecting slow tests:
- The comparison between `now` and test deadlines was reversed, causing no timeouts to ever be recognised.
- After fixing that bug, it was found that the existing code would issue timeouts for any test that had started more than 60 seconds ago, even if the test had finished long before its deadline was reached.
This PR fixes those bugs.
(The new executor is not yet enabled by default, so this PR has no immediate effect on contributors.)
---
I noted in https://github.com/rust-lang/rust/pull/139998#issuecomment-2815127046 that I hoped to have some unit tests to accompany these fixes. Unfortunately that turned out to be infeasible, because `DeadlineQueue` is tightly coupled to concrete `mpsc::Receiver` APIs (in addition to `Instant::now`), and trying to mock all of those would make the code much more complicated.
I did, however, add a few assertions that would have caught the failure to remove tests from the queue after their deadline.
r? jieyouxu
improve diagnostic for raw pointer field access with ->
This PR enhances the error messages emitted by the Rust compiler when users attempt to use the `->` operator for field access on raw pointers or when dereferencing is needed. The changes aim to provide clearer guidance, by suggesting the correct use of the `.` operator and explicit dereferencing.
**Before:**
```
help: `xs` is a raw pointer; try dereferencing it
|
LL | (*xs)->count += 1;
| ++ +
```
**Now:**
```
help: use `.` on a dereferenced raw pointer instead
|
LL - xs->count += 1;
LL + (*xs).count += 1;
|
```
I added extra clarification in the message. Since this error occurs in the parser, we can't be certain that the type is a raw pointer. That's why the message includes only a small note in brackets. (In contrast, the message above is emitted in HIR, where we *can* check whether it's a raw pointer.)
**Before:**
```
--> main.rs:11:11
|
11 | xs->count += 1;
| ^^
|
= help: the . operator will dereference the value if needed
```
**After:**
```
--> main.rs:11:11
|
11 | xs->count += 1;
| ^^
|
= help: the `.` operator will automatically dereference the value, except if the value is a raw pointer
```
Add tests for two untested cases of placeholder relations
During work on #130227, I discovered several situations not covered by any previously existing UI test. This commit introudces tests to cover that.
r? lcnr
Use correct annotation for CSS pseudo elements
The list of CSS pseudo elements is pretty short so it was easy to go through. Even though the `:` is accepted, it's incorrect.
For a description of CSS pseudo elements: https://developer.mozilla.org/en-US/docs/Web/CSS/Pseudo-elements
r? ``@notriddle``
Improve diagnostics for pointer arithmetic += and -= (fixes#137391)
**Description**:
This PR improves the diagnostic message for cases where a binary assignment operation like `ptr += offset` or `ptr -= offset` is attempted on `*mut T`. These operations are not allowed, and the compiler previously suggested calling `.add()` or `.wrapping_add()`, which is misleading if not assigned.
This PR updates the diagnostics to suggest assigning the result of `.wrapping_add()` or `.wrapping_sub()` back to the pointer, e.g.:
**Examples**
For this code
```rust
let mut arr = [0u8; 10];
let mut ptr = arr.as_mut_ptr();
ptr += 2;
```
it will say:
```rust
10 | ptr += 2;
| ---^^^^^
| |
| cannot use `+=` on type `*mut u8`
|
help: consider replacing `ptr += offset` with `ptr = ptr.wrapping_add(offset)` or `ptr.add(offset)`
|
10 - ptr += 2;
10 + ptr = ptr.wrapping_add(2);
```
**Related issue**: #137391
cc `@nabijaczleweli` for context (issue author)
Construct OutputType using macro and print [=FILENAME] help info
Closes#139805
Use define_output_types to define variants of OutputType, as well as refactor all of its methods for clarity. This way no variant is missed when pattern matching or output help messages.
On top of that, I optimized for `emit` help messages.
r? ```@jieyouxu```
Remove `token::{Open,Close}Delim`
By replacing them with `{Open,Close}{Param,Brace,Bracket,Invisible}`.
PR #137902 made `ast::TokenKind` more like `lexer::TokenKind` by
replacing the compound `BinOp{,Eq}(BinOpToken)` variants with fieldless
variants `Plus`, `Minus`, `Star`, etc. This commit does a similar thing
with delimiters. It also makes `ast::TokenKind` more similar to
`parser::TokenType`.
This requires a few new methods:
- `TokenKind::is_{,open_,close_}delim()` replace various kinds of
pattern matches.
- `Delimiter::as_{open,close}_token_kind` are used to convert
`Delimiter` values to `TokenKind`.
Despite these additions, it's a net reduction in lines of code. This is
because e.g. `token::OpenParen` is so much shorter than
`token::OpenDelim(Delimiter::Parenthesis)` that many multi-line forms
reduce to single line forms. And many places where the number of lines
doesn't change are still easier to read, just because the names are
shorter, e.g.:
```
- } else if self.token != token::CloseDelim(Delimiter::Brace) {
+ } else if self.token != token::CloseBrace {
```
r? `@petrochenkov`
jsondocck: Require command is at start of line
In one place we use `///``@``` instead of `//``@`.`` The test-runner allowed it, but it probably shouldn't. Ran into by ``@lolbinarycat`` in https://github.com/rust-lang/rust/pull/132748#issuecomment-2816469322:
```
error: unknown disambiguator `?(`
##[error] --> /checkout/tests/rustdoc-json/fns/return_type_alias.rs:3:25
|
3 | ///@ set foo = "$.index[?(``@.name=='Foo')].id"``
| ^^
|
```
Maybe it's also worth erroring on this like we added in #137103
r? ``@GuillaumeGomez``
rustdoc-json: Improve test for auto-trait impls
The TODO is fixable now due-to #138763. While I was here I realized there's probably a a few more things we should also test.
r? ```@GuillaumeGomez```
Fix error when an intra doc link is trying to resolve an empty associated item
Fixes https://github.com/rust-lang/rust/issues/140026.
Assigning ```@nnethercote``` since they're the one who wrote the initial change.
I updated rustdoc code instead of compiler's because I think it makes more sense that the caller ensures on their side that the name they're looking for isn't empty.
r? ```@nnethercote```
Advent of `tests/ui` (misc cleanups and improvements) [4/N]
Some `tests/ui/` housekeeping, to trim down number of tests directly under `tests/ui/`. Part of #133895.
### Review advice
- Best reviewed commit-by-commit.
- I can squash commits before merge, commits are separate to make it easier to review.
Clarify why SGX code specifies linkage/symbol names for certain statics
Specifying linkage/symbol name is solely to ensure a single instance between the `std` crate and its unit tests.
Also update the symbol names as items have moved around a bit. The actual name isn't that important, it just needs to be unique. But for debugging it can be useful for it to point to the right place.
Hermit: Unify `std::env::args` with Unix
The only differences between these implementations of `std::env::args` are that Unix uses relaxed ordering, but Hermit uses acquire/release, and Unix truncates `argv` at the first null pointer, but Hermit doesn't. Since Hermit aims for Unix compatibility, unify it with Unix.
The atomic orderings were established in https://github.com/rust-lang/rust/pull/74006 (cc `@euclio)` for Unix and https://github.com/rust-lang/rust/pull/100579 (cc `@joboet)` for Hermit and, before those, they used mutexes and non-atomic statics. I think the difference in orderings is simply from them being changed at different times. The commented explanation for using acquire/release for Hermit is “to broadcast writes by the OS”. I'm not experienced enough with atomics to accurately judge, but I think acquire/release is stronger than needed. Either way, they should match.
Truncating at the first null pointer seems desirable, though I don't know whether it is necessary in practice on Hermit.
cc `@mkroening` `@stlankes` for Hermit
Stabilize `naked_functions`
tracking issue: https://github.com/rust-lang/rust/issues/90957
request for stabilization on tracking issue: https://github.com/rust-lang/rust/issues/90957#issuecomment-2539270352
reference PR: https://github.com/rust-lang/reference/pull/1689
# Request for Stabilization
Two years later, we're ready to try this again. Even though this issue is already marked as having passed FCP, given the amount of time that has passed and the changes in implementation strategy, we should follow the process again.
## Summary
The `naked_functions` feature has two main parts: the `#[naked]` function attribute, and the `naked_asm!` macro.
An example of a naked function:
```rust
const THREE: usize = 3;
#[naked]
pub extern "sysv64" fn add_n(number: usize) -> usize {
// SAFETY: the validity of the used registers
// is guaranteed according to the "sysv64" ABI
unsafe {
core::arch::naked_asm!(
"add rdi, {}",
"mov rax, rdi",
"ret",
const THREE,
)
}
}
```
When the `#[naked]` attribute is applied to a function, the compiler won't emit a [function prologue](https://en.wikipedia.org/wiki/Function_prologue_and_epilogue) or epilogue when generating code for this function. This attribute is analogous to [`__attribute__((naked))`](https://developer.arm.com/documentation/100067/0608/Compiler-specific-Function--Variable--and-Type-Attributes/--attribute----naked---function-attribute) in C. The use of this feature allows the programmer to have precise control over the assembly that is generated for a given function.
The body of a naked function must consist of a single `naked_asm!` invocation, a heavily restricted variant of the `asm!` macro: the only legal operands are `const` and `sym`, and the only legal options are `raw` and `att_syntax`. In lieu of specifying operands, the `naked_asm!` within a naked function relies on the function's calling convention to determine the validity of registers.
## Documentation
The Rust Reference: https://github.com/rust-lang/reference/pull/1689
(Previous PR: https://github.com/rust-lang/reference/pull/1153)
## Tests
* [tests/run-make/naked-symbol-visiblity](https://github.com/rust-lang/rust/tree/master/tests/codegen/naked-fn) verifies that `pub`, `#[no_mangle]` and `#[linkage = "..."]` work correctly for naked functions
* [tests/codegen/naked-fn](https://github.com/rust-lang/rust/tree/master/tests/codegen/naked-fn) has tests for function alignment, use of generics, and validates the exact assembly output on linux, macos, windows and thumb
* [tests/ui/asm/naked-*](https://github.com/rust-lang/rust/tree/master/tests/ui/asm) tests for incompatible attributes, generating errors around incorrect use of `naked_asm!`, etc
## Interaction with other (unstable) features
### [fn_align](https://github.com/rust-lang/rust/issues/82232)
Combining `#[naked]` with `#[repr(align(N))]` works well, and is tested e.g. here
- https://github.com/rust-lang/rust/blob/master/tests/codegen/naked-fn/aligned.rs
- https://github.com/rust-lang/rust/blob/master/tests/codegen/naked-fn/min-function-alignment.rs
It's tested extensively because we do need to explicitly support the `repr(align)` attribute (and make sure we e.g. don't mistake powers of two for number of bytes).
## History
This feature was originally proposed in [RFC 1201](https://github.com/rust-lang/rfcs/pull/1201), filed on 2015-07-10 and accepted on 2016-03-21. Support for this feature was added in [#32410](https://github.com/rust-lang/rust/pull/32410), landing on 2016-03-23. Development languished for several years as it was realized that the semantics given in RFC 1201 were insufficiently specific. To address this, a minimal subset of naked functions was specified by [RFC 2972](https://github.com/rust-lang/rfcs/pull/2972), filed on 2020-08-07 and accepted on 2021-11-16. Prior to the acceptance of RFC 2972, all of the stricter behavior specified by RFC 2972 was implemented as a series of warn-by-default lints that would trigger on existing uses of the `naked` attribute; these lints became hard errors in [#93153](https://github.com/rust-lang/rust/pull/93153) on 2022-01-22. As a result, today RFC 2972 has completely superseded RFC 1201 in describing the semantics of the `naked` attribute.
More recently, the `naked_asm!` macro was added to replace the earlier use of a heavily restricted `asm!` invocation. The `naked_asm!` name is clearer in error messages, and provides a place for documenting the specific requirements of inline assembly in naked functions.
The implementation strategy was changed to emitting a global assembly block. In effect, an extern function
```rust
extern "C" fn foo() {
core::arch::naked_asm!("ret")
}
```
is emitted as something similar to
```rust
core::arch::global_asm!(
"foo:",
"ret"
);
extern "C" {
fn foo();
}
```
The codegen approach was chosen over the llvm naked function attribute because:
- the rust compiler can guarantee the behavior (no sneaky additional instructions, no inlining, etc.)
- behavior is the same on all backends (llvm, cranelift, gcc, etc)
Finally, there is now an allow list of compatible attributes on naked functions, so that e.g. `#[inline]` is rejected with an error. The `#[target_feature]` attribute on naked functions was later made separately unstable, because implementing it is complex and we did not want to block naked functions themselves on how target features work on them. See also https://github.com/rust-lang/rust/issues/138568.
relevant PRs for these recent changes
- https://github.com/rust-lang/rust/pull/127853
- https://github.com/rust-lang/rust/pull/128651
- https://github.com/rust-lang/rust/pull/128004
- https://github.com/rust-lang/rust/pull/138570
-
### Various historical notes
#### `noreturn`
[RFC 2972](https://github.com/rust-lang/rfcs/blob/master/text/2972-constrained-naked.md) mentions that naked functions
> must have a body which contains only a single asm!() statement which:
> iii. must contain the noreturn option.
Instead of `asm!`, the current implementation mandates that the body contain a single `naked_asm!` statement. The `naked_asm!` macro is a heavily restricted version of the `asm!` macro, making it easier to talk about and document the rules of assembly in naked functions and give dedicated error messages.
For `naked_asm!`, the behavior of the `asm!`'s `noreturn` option is implicit. The `noreturn` option means that it is UB for control flow to fall through the end of the assembly block. With `asm!`, this option is usually used for blocks that diverge (and thus have no return and can be typed as `!`). With `naked_asm!`, the intent is different: usually naked funtions do return, but they must do so from within the assembly block. The `noreturn` option was used so that the compiler would not itself also insert a `ret` instruction at the very end.
#### padding / `ud2`
A `naked_asm!` block that violates the safety assumption that control flow must not fall through the end of the assembly block is UB. Because no return instruction is emitted, whatever bytes follow the naked function will be executed, resulting in truly undefined behavior. There has been discussion whether rustc should emit an invalid instruction (e.g. `ud2` on x86) after the `naked_asm!` block to at least fail early in the case of an invalid `naked_asm!`. It was however decided that it is more useful to guarantee that `#[naked]` functions NEVER contain any instructions besides those in the `naked_asm!` block.
# unresolved questions
None
r? ``@Amanieu``
I've validated the tests on x86_64 and aarch64
Use `output_base_dir` for `mir_dump_dir`
It just occurred to me that the problem might be due to multiple revisions using the same dump directory (and therefore deleting the other revision's dir). This fixes that by simply using the normal per-test output directory, which is revision safe.
Relocate tests in `tests/ui`
Part of #133895
Moved tests from a top-level directory into a more appropriate subdirectory.
If there is anything else that could be improved, please let me know!
r? jieyouxu
Don't ICE on pending obligations from deep normalization in a loop
See the comment I left inline in `compiler/rustc_trait_selection/src/traits/normalize.rs`.
Fixes https://github.com/rust-lang/rust/issues/133868
r? lcnr
