Make Tree Borrows Provenance GC no longer produce stack overflows
Most functions operating on Tree Borrows' trees are carefully written to not cause stack overflows due to too much recursion. The one exception is [`Tree::keep_only_needed`](94f5588faf/src/borrow_tracker/tree_borrows/tree.rs (L724)), which just uses regular recursion.
This function is part of the provenance GC, so it is called regularly for every allocation in the program.
Tests show that this is a problem in practice. For example, the test `fill::horizontal_line` in crate `tiny-skia` (version 0.11.4) is such a test.
This PR changes this, this test no now longer crashes. Instead, it succeeds (after a _long_ time).
Update cargo
12 commits in ba8b39413c74d08494f94a7542fe79aa636e1661..8f40fc59fb0c8df91c97405785197f3c630304ea
2024-08-16 22:48:57 +0000 to 2024-08-21 22:37:06 +0000
- Tests rely on absence of RUST_BACKTRACE (rust-lang/cargo#14441)
- fix: -Cmetadata includes whether extra rustflags is same as host (rust-lang/cargo#14432)
- [mdman] Normalize newlines when rendering options (rust-lang/cargo#14428)
- fix: doctest respects Cargo's color options (rust-lang/cargo#14425)
- Be more permissive while packaging unpublishable crates. (rust-lang/cargo#14408)
- fix: Limiting pre-release match semantics to use only on `OptVersionReq::Req` (rust-lang/cargo#14412)
- test: add a regression test for Issue 14409 (rust-lang/cargo#14430)
- chore: update label trigger for Command-info (rust-lang/cargo#14422)
- doc: add lockfile-path unstable doc section (rust-lang/cargo#14423)
- doc: update lockfile-path tracking issue (rust-lang/cargo#14424)
- fix: remove list owners feature of info subcommand (rust-lang/cargo#14418)
- Lockfile path tests (follow-up) (rust-lang/cargo#14417)
Force `LC_ALL=C` for all run-make tests
This PR adds `LC_ALL=C` for all run-make tests so that they become locale independent.
Fixes#129362
r? `@jieyouxu`
compiletest: use `std::fs::remove_dir_all` now that it is available
It turns out `aggressive_rm_rf` is not sufficiently aggressive (RAGEY) on Windows and obviously handles Windows symlinks incorrectly. Instead of rolling our own version, let's use `std::fs::remove_dir_all` now that it's available (well, it's been available for a good while, but probably wasn't available when this helper was written).
cc #129187 since basically this is failing due to similar problems.
Blocker for #128562.
Fixes#129155.
Fixes#126334.
improve submodule updates
During config parsing, some bootstrap logic (e.g., `download-ci-llvm`) checks certain sources (for `download-ci-llvm`, it's `src/llvm-project`) and acts based on their state. This means that if path is a git submodule, bootstrap needs to update it before checking its state. Otherwise it may make incorrect assumptions by relying on outdated sources. To enable submodule updates during config parsing, we need to move the `update_submodule` function from the `Build` to `Config`, so we can access to it during the parsing process.
Closes#122787
bootstrap: fix clean's remove_dir_all implementation
It turns out bootstrap's `clean.rs`'s hand-rolled `rm_rf` (which probably comes before `std::fs::remove_dir_all` was stable) is very broken on native Windows around both read-only files/directories and especially symbolic links. So instead of rolling our own, just use `std::fs::remove_dir_all`.
This is a blocker for compiletest's own `rm_rf` implementation #129155 which happens to be also buggy, which in turn is a blocker for the rmake.rs test port #128562 that heavily exercises symlinks (I was reviewing #128562 and testing it on native Windows which is how I found out).
I also left a FIXME for `detect_src_and_out` due to a failing assertion on native Windows (opened #129188):
```
---- core::config::tests::detect_src_and_out stdout ----
thread 'core::config::tests::detect_src_and_out' panicked at src\core\config\tests.rs:72:13:
assertion `left == right` failed
left: "E:\\tmp"
right: "C:\\tmp"
```
Fixes#112544 (because now we handle Windows symlinks properly).
try-job: x86_64-msvc
try-job: i686-mingw
try-job: test-various
try-job: armhf-gnu
try-job: aarch64-apple
try-job: aarch64-gnu
Rollup of 9 pull requests
Successful merges:
- #128662 (Lint on tail expr drop order change in Edition 2024)
- #128932 (skip updating when external binding is existed)
- #129270 (Don't consider locals to shadow inner items' generics)
- #129277 (Update annotate-snippets to 0.11)
- #129294 (Stabilize `iter::repeat_n`)
- #129308 (fix: simple typo in compiler directory)
- #129309 (ctfe: make CompileTimeInterpCx type alias public)
- #129314 (fix a broken link in `mir/mod.rs`)
- #129318 (Remove unneeded conversion to `DefId` for `ExtraInfo`)
r? `@ghost`
`@rustbot` modify labels: rollup
modularize rustdoc's write_shared
Refactor src/librustdoc/html/render/write_shared.rs to reduce code duplication, adding unit tests
* Extract + unit test code for sorting and rendering JSON, which is duplicated 9 times in the current impl
* Extract + unit test code for encoding JSON as single quoted strings, which is duplicated twice in the current impl
* Unit tests for cross-crate information file formats
* Generic interface to add new kinds of cross-crate information files in the future
* Intended to match current behavior exactly, except for a merge info comment it adds to the bottom of cci files
* This PR is intended to reduce the review burden from my [mergeable rustdoc rfc](https://github.com/rust-lang/rfcs/pull/3662) implementation PR, which is a [small commit based on this branch](https://github.com/EtomicBomb/rust/tree/rfc). This code is agnostic to the RFC and does not include any of the flags discussed there, but cleanly enables the addition of these flags in the future because it is more modular
Remove unneeded conversion to `DefId` for `ExtraInfo`
I'm working on adding support for "unit test doctests" and this first cleanup came up so just sending it ahead of the rest.
r? ``@notriddle``
skip updating when external binding is existed
Fixes#128813
For following code:
```rs
extern crate core;
fn f() {
use ::core;
}
macro_rules! m {
() => {
extern crate std as core;
};
}
m!();
fn main() {}
```
- In the first loop, we define `extern crate core` and `use ::core` will be referred to `core` (yes, it does not consider if there are some macros that are not expanded. Ideally, this should be delayed if there are some unexpanded macros in the root, but I didn't change it like that because it seems like a huge change).
- Then `m` is expanded, which makes `extern_prelude('core')` return `std` rather than `core`, causing the inconsistency.
r? `@petrochenkov`
Implement SHA256 SIMD intrinsics on x86
Disclaimer: this is my first contribution to `miri`'s code. It's quite possible I'm missing something. This code works but may not be the cleanest/best possible.
It'd be useful to be able to verify code implementing SHA256 using SIMD since such code is a bit more complicated and at some points requires use of pointers. Until now `miri` didn't support x86 SHA256 intrinsics. This commit implements them.
It'd be useful to be able to verify code implementing SHA256 using SIMD
since such code is a bit more complicated and at some points requires
use of pointers. Until now `miri` didn't support x86 SHA256 intrinsics.
This commit implements them.
This approach is, roughly, based on how Discourse does it.
It came up while discussing some other possible sidebar changes,
as a design that made rapid scanning easier while avoiding the
inherent trade-offs in summarizing.
Delete debuginfo test suite infra for gdb without Rust support and lldb with Rust support
Implements https://github.com/rust-lang/rust/issues/128953
I also deleted all the `min-lldb-version: 310` comments, because the oldest compatible distro I can find is Ubuntu 16.04 which ships lldb 3.8, though of course the package that the Ubuntu maintainers put together for that is broken.
Rocky Linux 8 amusingly ships lldb 17, even though it has a similar glibc and kernel version.
This PR is multiple highly mechanical changes. Some of the commits were created by just running `sed`. You may find it easier to review each commit separately.
Switch to using the v2 resolver in most workspaces
Pinning the resolver to v1 was done in 5abff3753a ("Explicit set workspace.resolver ...") in order to suppress warnings. Since there is no specific reason not to use the new resolver and since it fixes issues, change to `resolver = "2"` everywhere except library.
Include a copy of `compiler-rt` source in the `download-ci-llvm` tarball
This will make it possible to experiment with allowing `download-ci-llvm` builds to build `library/profiler_builtins`, without needing to check out the `src/llvm-project` submodule.
By itself, this PR just adds the files to the tarball, but doesn't actually do anything with them. The idea is that once this is merged, it will then be much easier to proceed with work on the necessary bootstrap changes (using the real downloaded tarball), without having to rig up weird hacks to simulate downloading a modified tarball.
---
Adding these files to the compressed tarballs appears to increase its size by a negligible amount (<1 MB out of 400/800+ MB).
The uncompressed size is about 14 MB (out of 2+ GB for the whole tarball).
(The excluded test files would have been another 35 MB.)
Update `crosstool-ng` for loongarch64
The current cross-compilation toolchain for the LoongArch64 target consists of GCC 13.2.0, Binutils 2.40, and Glibc 2.36. However, Binutils 2.40 has known issues that in broken binaries without any error reports:
- https://github.com/rust-lang/rust/issues/121289
- https://github.com/cross-rs/cross/issues/1538
This patch upgrades the cross-compilation toolchain for the LoongArch64 target to resolve these issues.
- GCC: 13.2.0 -> 14.2.0
- Binutils: 2.40 -> 2.42
The new binaries remain compatible with the existing GCC 13.2.0/Glibc 2.36 distribution, and no issues have been identified.
try-job: dist-loongarch64-linux
Stabilize `raw_ref_op` (RFC 2582)
This stabilizes the syntax `&raw const $expr` and `&raw mut $expr`. It has existed unstably for ~4 years now, and has been exposed on stable via the `addr_of` and `addr_of_mut` macros since Rust 1.51 (released more than 3 years ago). I think it has become clear that these operations are here to stay. So it is about time we give them proper primitive syntax. This has two advantages over the macro:
- Being macros, `addr_of`/`addr_of_mut` could in theory do arbitrary magic with the expression on which they work. The only "magic" they actually do is using the argument as a place expression rather than as a value expression. Place expressions are already a subtle topic and poorly understood by many programmers; having this hidden behind a macro using unstable language features makes this even worse. Conversely, people do have an idea of what happens below `&`/`&mut`, so we can make the subtle topic a lot more approachable by connecting to existing intuition.
- The name `addr_of` is quite unfortunate from today's perspective, given that we have accepted provenance as a reality, which means that a pointer is *not* just an address. Strict provenance has a method, `addr`, which extracts the address of a pointer; using the term `addr` in two different ways is quite unfortunate. That's why this PR soft-deprecates `addr_of` -- we will wait a long time before actually showing any warning here, but we should start telling people that the "addr" part of this name is somewhat misleading, and `&raw` avoids that potential confusion.
In summary, this syntax improves developers' ability to conceptualize the operational semantics of Rust, while making a fundamental operation frequently used in unsafe code feel properly built in.
Possible questions to consider, based on the RFC and [this](https://github.com/rust-lang/rust/issues/64490#issuecomment-1163802912) great summary by `@CAD97:`
- Some questions are entirely about the semantics. The semantics are the same as with the macros so I don't think this should have any impact on this syntax PR. Still, for completeness' sake:
- Should `&raw const *mut_ref` give a read-only pointer?
- Tracked at: https://github.com/rust-lang/unsafe-code-guidelines/issues/257
- I think ideally the answer is "no". Stacked Borrows says that pointer is read-only, but Tree Borrows says it is mutable.
- What exactly does `&raw const (*ptr).field` require? Answered in [the reference](https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html): the arithmetic to compute the field offset follows the rules of `ptr::offset`, making it UB if it goes out-of-bounds. Making this a safe operation (using `wrapping_offset` rules) is considered too much of a loss for alias analysis.
- Choose a different syntax? I don't want to re-litigate the RFC. The only credible alternative that has been proposed is `&raw $place` instead of `&raw const $place`, which (IIUC) could be achieved by making `raw` a contextual keyword in a new edition. The type is named `*const T`, so the explicit `const` is consistent in that regard. `&raw expr` lacks the explicit indication of immutability. However, `&raw const expr` is quite a but longer than `addr_of!(expr)`.
- Shouldn't we have a completely new, better raw pointer type instead? Yes we all want to see that happen -- but I don't think we should block stabilization on that, given that such a nicer type is not on the horizon currently and given the issues with `addr_of!` mentioned above. (If we keep the `&raw $place` syntax free for this, we could use it in the future for that new type.)
- What about the lint the RFC talked about? It hasn't been implemented yet. Given that the problematic code is UB with or without this stabilization, I don't think the lack of the lint should block stabilization.
- I created an issue to track adding it: https://github.com/rust-lang/rust/issues/127724
- Other points from the "future possibilites of the RFC
- "Syntactic sugar" extension: this has not been implemented. I'd argue this is too confusing, we should stick to what the RFC suggested and if we want to do anything about such expressions, add the lint.
- Encouraging / requiring `&raw` in situations where references are often/definitely incorrect: this has been / is being implemented. On packed fields this already is a hard error, and for `static mut` a lint suggesting raw pointers is being rolled out.
- Lowering of casts: this has been implemented. (It's also an invisible implementation detail.)
- `offsetof` woes: we now have native `offset_of` so this is not relevant any more.
To be done before landing:
- [x] Suppress `unused_parens` lint around `&raw {const|mut}` expressions
- See bottom of https://github.com/rust-lang/rust/pull/127679#issuecomment-2264073752 for rationale
- Implementation: https://github.com/rust-lang/rust/pull/128782
- [ ] Update the Reference.
- https://github.com/rust-lang/reference/pull/1567
Fixes https://github.com/rust-lang/rust/issues/64490
cc `@rust-lang/lang` `@rust-lang/opsem`
try-job: x86_64-msvc
try-job: test-various
try-job: dist-various-1
try-job: armhf-gnu
try-job: aarch64-apple
Move ZST ABI handling to `rustc_target`
Currently, target specific handling of ZST function call ABI (specifically passing them indirectly instead of ignoring them) is handled in `rustc_ty_utils`, whereas all other target specific function call ABI handling is located in `rustc_target`. This PR moves the ZST handling to `rustc_target` so that all the target-specific function call ABI handling is in one place. In the process of doing so, this PR fixes#125850 by ensuring that ZST arguments are always correctly ignored in the x86-64 `"sysv64"` ABI; any code which would be affected by this fix would have ICEd before this PR. Tests are also added using `#[rustc_abi(debug)]` to ensure this behaviour does not regress.
Fixes#125850
Recent versions of wasm-tools are now Apache-2.0 or MIT or Apache-2.0
with the LLVM exception, rather than strictly Apache-2.0 with the LLVM
exception. The only component with the exception has moved to a new
dependency `wasi-preview1-component-adapter-provider`.
With the new resolver, a few dependencies get brought in twice with
different licenses. For example, all dependencies from `wasm-tools`
gained Apache-2.0 and MIT options, and with the v2 resolver we were
using one version from before and one version from after this change.
This made tidy's license check difficult.
Update some minimum versions to remove duplicate dependencies and smooth
out license checking.
Pinning the resolver to v1 was done in 5abff3753a ("Explicit set
workspace.resolver ...") in order to suppress warnings. Since there is
no specific reason not to use the new resolver and since it fixes
issues, change to `resolver = "2"` everywhere except library and
submodules.
Promote Mac Catalyst targets to Tier 2, and ship with rustup
Promote the Mac Catalyst targets `x86_64-apple-ios-macabi` and `aarch64-apple-ios-macabi` to Tier 2, as per [the MCP](https://github.com/rust-lang/compiler-team/issues/761) (see that for motivation and details).
These targets are now also distributed with rustup, although without the sanitizer runtime, as that currently has trouble building, see https://github.com/rust-lang/rust/issues/129069.
