Migrate `run-make/issue-53964` to `rmake`
Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).
This is extremely similar to #125146. Could it be interesting to merge the two in some way? This one seems to do the same thing as the #125146, but with an added check that a useless lint is not shown.
Rewrite native thread-local storage
(part of #110897)
The current native thread-local storage implementation has become quite messy, uses indescriptive names and unnecessarily adds code to the macro expansion. This PR tries to fix that by using a new implementation that also allows more layout optimizations and potentially increases performance by eliminating unnecessary TLS accesses.
This does not change the recursive initialization behaviour I described in [this comment](https://github.com/rust-lang/rust/issues/110897#issuecomment-1525705682), so it should be a library-only change. Changing that behaviour should be quite easy now, however.
r? `@m-ou-se`
`@rustbot` label +T-libs
Rewrite `core-no-oom-handling`, `issue-24445` and `issue-38237` `run-make` tests to new `rmake.rs` format
Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).
The test which is now called `non-pie-thread-local` has an unexplained "only-linux" flag. Could it be worth trying to remove it and changing the CI to test non-Linux platforms on it?
refactor: add rustc-perf submodule to src/tools
Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:
`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:
```dockerfile
ENV PERF_COMMIT 4f313add609f43e928e98132358e8426ed3969ae
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
unzip perf.zip && \
mv rustc-perf-$PERF_COMMIT rustc-perf && \
rm perf.zip
```
`src/tools/opt-dist/src/main.rs`
```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";
let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
.timeout(Duration::from_secs(60 * 2))
.connect_timeout(Duration::from_secs(60 * 2))
.build()?;
let response = retry_action(
|| Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
"Download rustc-perf archive",
5,
)?;
```
This causes a few issues:
1. Maintainers need to be careful to bump PERF_COMMIT in both places
every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
`rustc-perf` (https://github.com/rust-lang/rust/pull/125125), but to
figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
dependencies are not included in the `vendor/` dir created during
`dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
`rustc-perf` in use, leading to staleness
Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.
This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313add609f43e928e98132358e8426ed3969ae. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
This replaces the hardcoded rustc-perf commit and ad-hoc downloading and
unpacking of its zipped source with defaulting to use the new rustc-perf
submodule.
While it would be nice to make `opt-dist` able to initialize the
submodule automatically when pointing to a Rust checkout _other_ than
the one opt-dist was built in, that would require a bigger refactor that
moved `update_submodule`, from bootstrap, into build_helper.
Regardless, I imagine it must be quite rare to use `opt-dist` with a
checkout that is neither from a rust-src tarball (which will contain the
submodule), nor the checkout opt-dist itself was built (bootstrap will
update the submodule when opt-dist is built).
Currently, it's very challenging to perform a sandboxed `opt-dist`
bootstrap because the tool requires `rustc-perf` to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:
`src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile`:
```dockerfile
ENV PERF_COMMIT 4f313add609f43e928e98132358e8426ed3969ae
RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip && \
unzip perf.zip && \
mv rustc-perf-$PERF_COMMIT rustc-perf && \
rm perf.zip
```
`src/tools/opt-dist/src/main.rs`
```rust
// FIXME: add some mechanism for synchronization of this commit SHA with
// Linux (which builds rustc-perf in a Dockerfile)
// rustc-perf version from 2023-10-22
const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";
let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip");
let client = reqwest::blocking::Client::builder()
.timeout(Duration::from_secs(60 * 2))
.connect_timeout(Duration::from_secs(60 * 2))
.build()?;
let response = retry_action(
|| Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()),
"Download rustc-perf archive",
5,
)?;
```
This causes a few issues:
1. Maintainers need to be careful to bump PERF_COMMIT in both places
every time
2. In order to run `opt-dist` in a sandbox, you need to provide your own
`rustc-perf` (https://github.com/rust-lang/rust/pull/125125), but to
figure out which commit to provide you need to grep the Dockerfile
3. Even if you manage to provide the correct `rustc-perf`, its
dependencies are not included in the `vendor/` dir created during
`dist`, so it will fail to build from the published source tarballs
4. It is hard to provide any level of automation around updating the
`rustc-perf` in use, leading to staleness
Fundamentally, this means `rustc-src` tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
`opt-dist` need to go out of their way to keep track of this "hidden"
dependency on `rustc-perf`.
This change adds rustc-perf as a git submodule, pinned to the current
`PERF_COMMIT` 4f313add609f43e928e98132358e8426ed3969ae. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in `opt-dist`.
Add tests for `-Zunpretty=expanded` ported from stringify's tests
This PR adds a new set of tests for the AST pretty-printer.
Previously, pretty-printer edge cases were tested by way of `stringify!` in [tests/ui/macros/stringify.rs](https://github.com/rust-lang/rust/blob/1.78.0/tests/ui/macros/stringify.rs), such as the tests added by 419b26931b and 527e2eac17.
Those tests will no longer provide effective coverage of the AST pretty-printer after #124141. `Nonterminal` and `TokenKind::Interpolated` are being removed, and a consequence is that `stringify!` will perform token stream pretty printing, instead of AST pretty printing, in all of the `stringify!` cases including $:expr and all other interpolations.
This PR adds 2 new ui tests with `compile-flags: -Zunpretty=expanded`:
- **tests/ui/unpretty/expanded-exhaustive.rs** — this test aims for exhaustive coverage of all the variants of `ExprKind`, `ItemKind`, `PatKind`, `StmtKind`, `TyKind`, and `VisibilityKind`. Some parts could use being fleshed out further, but the current state is roughly on par with what exists in the old stringify-based tests.
- **tests/ui/unpretty/expanded-interpolation.rs** — this test covers tricky macro metavariable edge cases that require the AST pretty printer to synthesize parentheses in order for the printed code to be valid Rust syntax.
r? `@nnethercote`
Migrate `run-make/issue64319` to `rmake` and rename
Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).
~~I noticed that the Makefile was not listed in `allowed-run-makefiles` in Tidy. Does this mean the test was being ignored?~~ EDIT: No, it was there, just not in its expected alphabetical order.
EDIT2: Perhaps it could be interesting to clean this test visually by looping over the `rustc` calls, like in #125227.
Migrate `run-make/no-cdylib-as-rdylib` to `rmake`
Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).
> "the test will fail if the cdylib is picked, because it doesn't export any rust symbols"
Is that true? Is there a way to verify?
I suggest maybe extending the test with: (after cleaning the directory)
```rust
rustc()
.input("bar.rs")
.crate_type("cdylib")
.run();
rustc()
.input("foo.rs")
.prefer_dynamic()
.run();
fail();
```
to make sure we're actually testing something here.
