compiletest: Encapsulate all of the code that touches libtest
Compiletest currently relies on unstable libtest APIs in order to actually execute tests. That's unfortunate, but removing the dependency isn't trivial.
However, we can make a small step towards removing the libtest dependency by encapsulating the libtest interactions into a single dedicated module. That makes it easier to see what parts of libtest are actually used.
---
As a side-effect of moving the `test_opts` function into that dedicated module, this PR also ends up allowing `--fail-fast` to be passed on the command line, instead of requiring an environment variable.
---
There is still (at least) one other aspect of the libtest dependency that this PR does not address, namely the fact that we rely on libtest's output capture (via unstable std APIs) to capture the output that we print during individual tests. I hope to do something about that at some point.
r? jieyouxu
compiletest: Support matching diagnostics on lines below
Using `//~vvv ERROR`.
This is not needed often, but it's easy to support, and it allows to eliminate a class of `error-pattern`s that cannot be eliminated in any other way.
See the diff for the examples of such patterns coming from parser.
Some of them can be matched by `//~ ERROR` or `//~^ ERROR` as well (when the final newline is allowed), but it changes the shape of reported spans, so I chose to keep the spans by using `//~v ERROR`.
Reject `{true,false}` as revision names
Because they would imply `--cfg={true,false}` otherwise, and the test writer has to use `cfg(r#true)` and `cfg(r#false)` in the test.
Closes#138663.
Prevent `rmake.rs` from using unstable features, and fix 3 run-make tests that currently do
Addresses (mostly) #137532.
Follow-up to #137373.
### Summary
- Fix 3 run-make tests that currently use unstable features:
1. `tests/run-make/issue-107495-archive-permissions/rmake.rs` uses `#![feature(rustc_private)]` for `libc` on `unix`, but `run_make_support` already exports `libc`, so just use that.
2. `tests/run-make/cross-lang-lto/rmake.rs` uses `#![feature(path_file_prefix)]` for convenience, replaced with similar filename prefix logic.
3. `tests/run-make/broken-pipe-no-ice/rmake.rs` uses `#![feature(anonymous_pipe)]` for anonymous pipes. This is more complicated[^race-condition], and I decided to temporarily introduce a dependency on [`os_pipe`] before std's `anonymous_pipe` library feature is stabilized[^pipe-stab]. I left a FIXME tracked by #137532 to make the switch once `anonymous_pipe` stabilizes and reaches beta.
- Use `RUSTC_BOOTSTRAP=-1` when building `rmake.rs` to have the stage 0 rustc reject any unstable features used in `rmake.rs`.
- The requirement that `rmake.rs` may not use any unstable features is now documented in rustc-dev-guide.
- This PR does not impose `RUSTC_BOOTSTRAP=-1` when building `run-make-support`, but I suppose we could.
r? `@Kobzol`
try-job: x86_64-msvc-1
try-job: x86_64-mingw-1
[`os_pipe`]: https://github.com/oconnor663/os_pipe.rs
[^race-condition]: We can't just try to spawn `rustc` and immediate close the stderr handle because of race condition, as there's no guarantee `rustc` will not try to print to stderr before the handle gets closed.
[^pipe-stab]: In-progress stabilization PR over at https://github.com/rust-lang/rust/pull/135822.
bootstrap and compiletest: Use `size_of_val` from the prelude instead of imported
Use `std::mem::size_of_val` from the prelude instead of importing or qualifying it.
This function was added to all preludes in Rust 1.80.
r? ``@jieyouxu``
Perform deeper compiletest path normalization for `$TEST_BUILD_DIR` to account for compare-mode/debugger cases, and normalize long type file filename hashes
Fixes#136510.
### Summary
- Whereas previously `$TEST_BUILD_DIR` is a normalization of `/path/to/build/test/<test_suite_name>/`, we now more deeply normalize. `$TEST_BUILD_DIR` now becomes a normalization of `/path/to/build/test/<test_suite_name>/<subdirs>/$name.$revision.$compare_mode.$debugger/` to normalize away path name differences when `--compare-mode` and/or `--debugger` are specified.
- We also centralize the normalization of long type name hashes
cf. https://github.com/rust-lang/rust/pull/136328#discussion_r1936760908.
### Review advice
- Best reviewed commit-by-commit.
- Split into 3 commits:
- **Commit 1**: compiletest changes to have `$TEST_BUILD_DIR` more deeply normalize.
- **Commit 2**: remove per-test hacks for long type path hash normalizations, and rebless tests *specifically* affected by that.
- **Commit 3**: rebless other tests that were changed as a side-effect of deeper `$TEST_BUILD_DIR` normalizations.
**Commit 2** is created via first finding tests that try to perform long type file hash normalizations on an ad hoc, per-test basis:
```
rg --no-ignore -l --no-ignore -F -e "long-type" tests/ui/**/*.rs
```
<details>
<summary>Tests with ad hoc long-type hash normalizations</summary>
```
tests/ui/type_length_limit.rs
tests/ui/traits/on_unimplemented_long_types.rs
tests/ui/regions/issue-102374.rs
tests/ui/recursion/recursion.rs
tests/ui/recursion/issue-83150.rs
tests/ui/recursion/issue-23122-2.rs
tests/ui/methods/inherent-bound-in-probe.rs
tests/ui/issues/issue-67552.rs
tests/ui/issues/issue-37311-type-length-limit/issue-37311.rs
tests/ui/issues/issue-20413.rs
tests/ui/issues/issue-8727.rs
tests/ui/infinite/infinite-instantiation.rs
tests/ui/infinite/infinite-instantiation-struct-tail-ice-114484.rs
tests/ui/higher-ranked/trait-bounds/hrtb-doesnt-borrow-self-1.rs
tests/ui/higher-ranked/trait-bounds/hrtb-doesnt-borrow-self-2.rs
tests/ui/higher-ranked/trait-bounds/hang-on-deeply-nested-dyn.rs
tests/ui/error-codes/E0275.rs
tests/ui/diagnostic-width/secondary-label-with-long-type.rs
tests/ui/diagnostic-width/long-e0277.rs
tests/ui/diagnostic-width/non-copy-type-moved.rs
tests/ui/diagnostic-width/long-E0308.rs
tests/ui/diagnostic-width/E0271.rs
tests/ui/diagnostic-width/binop.rs
```
</details>
These ad hoc normalizations were removed, and they are reblessed.
r? `````@lqd`````
Retire the legacy `Makefile`-based `run-make` test infra
The final piece of [porting run-make tests to use Rust #121876](https://github.com/rust-lang/rust/issues/121876).
Closes#121876.
Closes#40713.
Closes#81791 (no longer using `wc`).
Closes#56475 (no longer a problem in current form of that test; we don't ignore the test on `aarch64-unknown-linux-gnu`).
### Summary
This PR removes the legacy `Makefile`-based `run-make` test infra which has served us well over the years. The legacy infra is no longer needed since we ported all of `Makefile`-based `run-make` tests to the new `rmake.rs` infra.
Additionally, this PR:
- Removes `tests/run-make/tools.mk` since no more `Makefile`-based tests remain.
- Updates `tests/run-make/README.md` and rustc-dev-guide docs to remove mention about `Makefile`-based `run-make` tests
- Update test suite requirements in rustc-dev-guide on Windows to no longer need MSYS2 (they should also now run successfully on native Windows MSVC).
- Update `triagebot.toml` to stop backlinking to #121876.
**Thanks to everyone who helped in this effort to modernize the `run-make` test infra and test suite!**
r? bootstrap
Compile run-make-support and run-make tests with the bootstrap compiler
It does not seem necessary to have to recompile run-make-support on changes to the local compiler/stdlib. This PR simplifies the implementation of a few tools, then switches rms to stage0 and also makes the handling of environment variables in run-make tests simpler.
Best reviewed commit-by-commit. I can split it into multiple PRs if you want.
Also tested that `COMPILETEST_FORCE_STAGE0=1 ./x test tests/run-make --stage 0` still works. Incredibly, it looks like it even passes more tests than on `master` 😆
r? ``@jieyouxu``
Support raw-dylib link kind on ELF
raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.
This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.
I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694
r? bjorn3
try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various
This is to make test stderr insensitive to compare-mode / debugger that
changes the test build dir output name.
Previously, this normalized paths up to test-suite-specific build root,
e.g. `/path/to/build/test/ui/`. Now, this normalizes up to test-specific
build root, e.g.
`/path/to/build/test/ui/subdir/$name.$revision.$mode.$debugger/`.
[`compiletest`-related cleanups 4/7] Make the distinction between root build directory vs test suite specific build directory in compiletest less confusing
Reference for overall changes: https://github.com/rust-lang/rust/pull/136437
Part **4** of **7** of the *`compiletest`-related cleanups* PR series.
### Summary
- Remove `--build-base` compiletest flag, and introduce `--build-{root,test-suite-root}` flags instead. `--build-base` previously actually is test suite specific build directory, not the root `build/` directory.
- Feed the root build directory directly from bootstrap to compiletest via `--build-root` instead of doing multiple layers of parent unwrapping[^parent] based on the test suite specific build directory.
- Remove a redundant `to_path_buf()`.
[^parent]: Please do not unwrap the parents.
r? bootstrap
raw-dylib is a link kind that allows rustc to link against a library
without having any library files present.
This currently only exists on Windows. rustc will take all the symbols
from raw-dylib link blocks and put them in an import library, where they
can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient
to have this same functionality. Not having the libraries present at
build-time can be convenient for several reasons, especially
cross-compilation. With raw-dylib, code linking against a library can be
cross-compiled without needing to have these libraries available on the
build machine. If the libc crate makes use of this, it would allow
cross-compilation without having any libc available on the build
machine. This is not yet possible with this implementation, at least
against libc's like glibc that use symbol versioning.
The raw-dylib kind could be extended with support for symbol versioning
in the future.
This implementation is very experimental and I have not tested it very
well. I have tested it for a toy example and the lz4-sys crate, where it
was able to successfully link a binary despite not having a
corresponding library at build-time.
Instead of only having `--src-base` and `src_base` which *actually*
refers to the directory containing the test suite and not the sources
root. More importantly, kill off `find_rust_src_root` when we can simply
pass that info from bootstrap.
This commit removes the `avr-unknown-gnu-atmega328` target and replaces
it with a more generic `avr-none` variant that must be specialized with
the `-C target-cpu` flag (e.g. `-C target-cpu=atmega328p`).
[`compiletest`-related cleanups 2/7] Feed stage number to compiletest directly
Reference for overall changes: https://github.com/rust-lang/rust/pull/136437
Part **2** of **7** of the *`compiletest`-related cleanups* PR series.
### Summary
- Pass stage number via new `--stage` compiletest flag directly from bootstrap, instead of deriving that info in compiletest by doing gymnastics on `--stage-id`.
- Just a cleanup, should have no functional changes.
r? bootstrap
[`compiletest`-related cleanups 1/7] Cleanup `is_rustdoc` logic and remove a useless path join in rustdoc-json runtest logic
Reference for overall changes: https://github.com/rust-lang/rust/pull/136437
Part **1** of **7** of the *`compiletest`-related cleanups* PR series.
### Summary
- Don't match on path when we already have test suite names.
- Remove a useless path join.
r? bootstrap (or compiler)
- Don't show environment variables. Seeing PATH is almost never useful, and it can be extremely long.
- For .rlibs in the sysroot, replace crate hashes with a `"-*"` string. This will expand to the full crate name when pasted into the shell.
- Move `.rlib` to outside the glob.
- Abbreviate the sysroot path to `<sysroot>` wherever it appears in the arguments.
This also adds an example of the linker output as a run-make test. Currently it only runs on x86_64-unknown-linux-gnu, because each platform has its own linker arguments. So that it's stable across machines, pass BUILD_ROOT as an argument through compiletest through to run-make tests.
- Only use linker-flavor=gnu-cc if we're actually going to compare the output. It doesn't exist on MacOS.
Add test for checking used glibc symbols
This test checks that we do not use too new glibc symbols in the compiler on x64 GNU Linux, in order not to break our [glibc promises](https://blog.rust-lang.org/2022/08/01/Increasing-glibc-kernel-requirements.html).
One thing that isn't solved in the PR yet is to make sure that this test will only run on `dist` CI, more specifically on the `dist-x86_64-linux` runner, in the opt-dist post-optimization tests (it can fail elsewhere, that doesn't matter). Any suggestions on how to do that are welcome.
Fixes: https://github.com/rust-lang/rust/issues/134037
r? `@jieyouxu`
Point at invalid utf-8 span on user's source code
```
error: couldn't read `$DIR/not-utf8-bin-file.rs`: stream did not contain valid UTF-8
--> $DIR/not-utf8-2.rs:6:5
|
LL | include!("not-utf8-bin-file.rs");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
note: byte `193` is not valid utf-8
--> $DIR/not-utf8-bin-file.rs:2:14
|
LL | let _ = "�|�␂!5�cc␕␂��";
| ^
= note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)
```
When we attempt to load a Rust source code file, if there is a OS file failure we try reading the file as bytes. If that succeeds we try to turn it into UTF-8. If *that* fails, we provide additional context about *where* the file has the first invalid UTF-8 character.
Fix#76869.
