This was implemented incorrectly during the porting process, where we
relied on std consts. However, `run-make-support` is a host-only
library, which meant that these artifact names were for the *host* and
not the *target*.
The old Makefile-based infrastructure included support for executing
binaries with remote-test-client if configured, but that didn't get
ported to run_make_support as part of the rmake migration.
This PR re-introduces back that support, with the same implementation
(and limitations) of the original Makefile-based support.
Retire the legacy `Makefile`-based `run-make` test infra
The final piece of [porting run-make tests to use Rust #121876](https://github.com/rust-lang/rust/issues/121876).
Closes#121876.
Closes#40713.
Closes#81791 (no longer using `wc`).
Closes#56475 (no longer a problem in current form of that test; we don't ignore the test on `aarch64-unknown-linux-gnu`).
### Summary
This PR removes the legacy `Makefile`-based `run-make` test infra which has served us well over the years. The legacy infra is no longer needed since we ported all of `Makefile`-based `run-make` tests to the new `rmake.rs` infra.
Additionally, this PR:
- Removes `tests/run-make/tools.mk` since no more `Makefile`-based tests remain.
- Updates `tests/run-make/README.md` and rustc-dev-guide docs to remove mention about `Makefile`-based `run-make` tests
- Update test suite requirements in rustc-dev-guide on Windows to no longer need MSYS2 (they should also now run successfully on native Windows MSVC).
- Update `triagebot.toml` to stop backlinking to #121876.
**Thanks to everyone who helped in this effort to modernize the `run-make` test infra and test suite!**
r? bootstrap
Compile run-make-support and run-make tests with the bootstrap compiler
It does not seem necessary to have to recompile run-make-support on changes to the local compiler/stdlib. This PR simplifies the implementation of a few tools, then switches rms to stage0 and also makes the handling of environment variables in run-make tests simpler.
Best reviewed commit-by-commit. I can split it into multiple PRs if you want.
Also tested that `COMPILETEST_FORCE_STAGE0=1 ./x test tests/run-make --stage 0` still works. Incredibly, it looks like it even passes more tests than on `master` 😆
r? ``@jieyouxu``
This commit removes the `avr-unknown-gnu-atmega328` target and replaces
it with a more generic `avr-none` variant that must be specialized with
the `-C target-cpu` flag (e.g. `-C target-cpu=atmega328p`).
- Don't show environment variables. Seeing PATH is almost never useful, and it can be extremely long.
- For .rlibs in the sysroot, replace crate hashes with a `"-*"` string. This will expand to the full crate name when pasted into the shell.
- Move `.rlib` to outside the glob.
- Abbreviate the sysroot path to `<sysroot>` wherever it appears in the arguments.
This also adds an example of the linker output as a run-make test. Currently it only runs on x86_64-unknown-linux-gnu, because each platform has its own linker arguments. So that it's stable across machines, pass BUILD_ROOT as an argument through compiletest through to run-make tests.
- Only use linker-flavor=gnu-cc if we're actually going to compare the output. It doesn't exist on MacOS.
This facade is like other `run_make_support::fs` APIs that
panic-on-failure but includes the path that the operation was called on
in the panic message.
Fix `--nocapture` for run-make tests
This was confusing because there are three layers of output hiding.
1. libtest shoves all output into a buffer and does not print it unless the test fails or `--nocapture` is passed.
2. compiletest chooses whether to print the output from any given process.
3. run-make-support chooses what output to print.
This modifies 2 and 3.
- compiletest: Don't require both `--verbose` and `--nocapture` to show the output of run-make tests.
- compiletest: Print the output from `rmake` processes if they succeed. Previously this was only printed on failure.
- compiletest: Distinguish rustc and rmake stderr by printing the command name (e.g. "--stderr--" to "--rustc stderr--").
- run-make-support: Unconditionally print the needle/haystack being searched. Previously this was only printed on failure.
Before:
```
$ x t tests/run-make/linker-warning --force-rerun -- --nocapture
running 1 tests
.
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 377 filtered out; finished in 281.64ms
$ x t tests/run-make/linker-warning --force-rerun -v -- --nocapture 2>&1 | wc -l
1004
$ x t tests/run-make/linker-warning --force-rerun -v -- --nocapture | tail -n40
running 1 tests
------stdout------------------------------
------stderr------------------------------
warning: unused import: `std::path::Path`
--> /home/jyn/src/rust2/tests/run-make/linker-warning/rmake.rs:1:5
|
1 | use std::path::Path;
| ^^^^^^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
warning: unused import: `run_make_support::rfs::remove_file`
--> /home/jyn/src/rust2/tests/run-make/linker-warning/rmake.rs:3:5
|
3 | use run_make_support::rfs::remove_file;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
warning: 2 warnings emitted
------------------------------------------
test [run-make] tests/run-make/linker-warning ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 377 filtered out; finished in 285.89ms
```
After:
```
Testing stage1 compiletest suite=run-make mode=run-make (x86_64-unknown-linux-gnu)
running 1 tests
------rmake stdout------------------------------
------rmake stderr------------------------------
assert_contains_regex:
=== HAYSTACK ===
error: linking with `./fake-linker` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:...:/bin" VSLANG="1033" "./fake-linker" "-m64" "/tmp/rustcYqdAZT/symbols.o" "main.main.d17f5fbe6225cf88-cgu.0.rcgu.o" "main.2uoctswmurc6ir5rvoay0p9ke.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-B/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/test/run-make/linker-warning/rmake_out" "-L" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "main" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" "run_make_error"
= note: error: baz
error: aborting due to 1 previous error
=== NEEDLE ===
fake-linker.*run_make_error
assert_not_contains_regex:
=== HAYSTACK ===
=== NEEDLE ===
fake-linker.*run_make_error
------------------------------------------
.
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 377 filtered out; finished in 314.81ms
```
r? `@jieyouxu`
This was confusing because there are three layers of output hiding.
1. libtest shoves all output into a buffer and does not print it unless the test fails or `--nocapture` is passed.
2. compiletest chooses whether to print the output from any given process.
3. run-make-support chooses what output to print.
This modifies 2 and 3.
- compiletest: Don't require both `--verbose` and `--nocapture` to show the output of run-make tests.
- compiletest: Distinguish rustc and rmake stderr by printing the command name (e.g. "--stderr--" to "--rustc stderr--").
- run-make-support: Unconditionally print the needle/haystack being searched. Previously this was only printed on failure.
Before:
```
$ x t tests/run-make/linker-warning --force-rerun -- --nocapture
running 1 tests
.
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 377 filtered out; finished in 281.64ms
$ x t tests/run-make/linker-warning --force-rerun -v -- --nocapture 2>&1 | wc -l
1004
$ x t tests/run-make/linker-warning --force-rerun -v -- --nocapture | tail -n40
running 1 tests
------stdout------------------------------
------stderr------------------------------
warning: unused import: `std::path::Path`
--> /home/jyn/src/rust2/tests/run-make/linker-warning/rmake.rs:1:5
|
1 | use std::path::Path;
| ^^^^^^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
warning: unused import: `run_make_support::rfs::remove_file`
--> /home/jyn/src/rust2/tests/run-make/linker-warning/rmake.rs:3:5
|
3 | use run_make_support::rfs::remove_file;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
warning: 2 warnings emitted
------------------------------------------
test [run-make] tests/run-make/linker-warning ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 377 filtered out; finished in 285.89ms
```
After:
```
Testing stage1 compiletest suite=run-make mode=run-make (x86_64-unknown-linux-gnu)
running 1 tests
------rmake stdout------------------------------
------rmake stderr------------------------------
assert_contains_regex:
=== HAYSTACK ===
error: linking with `./fake-linker` failed: exit status: 1
|
= note: LC_ALL="C" PATH="/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin:...:/bin" VSLANG="1033" "./fake-linker" "-m64" "/tmp/rustcYqdAZT/symbols.o" "main.main.d17f5fbe6225cf88-cgu.0.rcgu.o" "main.2uoctswmurc6ir5rvoay0p9ke.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-B/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/bin/gcc-ld" "-fuse-ld=lld" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/test/run-make/linker-warning/rmake_out" "-L" "/home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "main" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs" "run_make_error"
= note: error: baz
error: aborting due to 1 previous error
=== NEEDLE ===
fake-linker.*run_make_error
assert_not_contains_regex:
=== HAYSTACK ===
=== NEEDLE ===
fake-linker.*run_make_error
------------------------------------------
.
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 377 filtered out; finished in 314.81ms
```
the linker arguments can be *very* long, especially for crates with many dependencies. some parts of them are not very useful. unless specifically requested:
- omit object files specific to the current invocation
- fold rlib files into a single braced argument (in shell expansion format)
this shortens the output significantly without removing too much information.
Liberate `aarch64-gnu-debug` from the shackles of `--test-args=clang`
### Changes
- Drop `--test-args=clang` from `aarch64-gnu-debug` so run-make tests that are `//@ needs-force-clang-based-tests` no longer only run if their test name contains `clang` (which is a very cool footgun).
- Reorganize run-make-suport library slightly to accommodate a raw gcc invocation.
- Fix `tests/run-make/mte-ffi/rmake.rs` to use `gcc` instead of *a* c compiler.
try-job: aarch64-gnu
try-job: aarch64-gnu-debug
