compiletest: Handle non-utf8 paths (fix FIXME)
Removes the last FIXME in the code for #9639🎉 (which was closed 8 years ago)
Part of #44366 which is E-help-wanted.
(The other two PRs that does this are #114377 and #114427)
It lints against features that are inteded to be internal to the
compiler and standard library. Implements MCP #596.
We allow `internal_features` in the standard library and compiler as those
use many features and this _is_ the standard library from the "internal to the compiler and
standard library" after all.
Marking some features as internal wasn't exactly the most scientific approach, I just marked some
mostly obvious features. While there is a categorization in the macro,
it's not very well upheld (should probably be fixed in another PR).
We always pass `-Ainternal_features` in the testsuite
About 400 UI tests and several other tests use internal features.
Instead of throwing the attribute on each one, just always allow them.
There's nothing wrong with testing internal features^^
new unstable option: -Zwrite-long-types-to-disk
This option guards the logic of writing long type names in files and instead using short forms in error messages in rustc_middle/ty/error behind a flag. The main motivation for this change is to disable this behaviour when running ui tests.
This logic can be triggered by running tests in a directory that has a long enough path, e.g. /my/very-long-path/where/rust-codebase/exists/
This means ui tests can fail depending on how long the path to their file is.
Some ui tests actually rely on this behaviour for their assertions, so for those we enable the flag manually.
This option guards the logic of writing long type names in files and
instead using short forms in error messages in rustc_middle/ty/error
behind a flag. The main motivation for this change is to disable this
behaviour when running ui tests.
This logic can be triggered by running tests in a directory that has a
long enough path, e.g. /my/very-long-path/where/rust-codebase/exists/
This means ui tests can fail depending on how long the path to their
file is.
Some ui tests actually rely on this behaviour for their assertions,
so for those we enable the flag manually.
Implement rust-lang/compiler-team#578.
When an ICE is encountered on nightly releases, the new rustc panic
handler will also write the contents of the backtrace to disk. If any
`delay_span_bug`s are encountered, their backtrace is also added to the
file. The platform and rustc version will also be collected.
Update debuginfo test runner to provide more useful output
This change makes debuginfo tests more user friendly. Changes:
- Print all lines that fail to match the patterns instead of just the first
- Provide better error messages that also say what did match
- Strip leading whitespace from directives so they are not skipped if indented
- Improve documentation and improve nesting on some related items
As an example, given the following intentional fail (and a few not shown):
```rust
// from tests/debuginfo/rc_arc.rs
// cdb-command:dx rc,d
// cdb-check:rc,d : 111 [Type: alloc::rc::Rc<i32>]
// cdb-check: [Reference count] : 11 [Type: core::cell FAIL::Cell<usize>]
// cdb-check: [Weak reference count] : 2 [Type: core::cell FAIL::Cell<usize>]
```
The current output (tested in #113313) will show:
```
2023-07-04T08:10:00.1939267Z ---- [debuginfo-cdb] tests\debuginfo\rc_arc.rs stdout ----
2023-07-04T08:10:00.1942182Z
2023-07-04T08:10:00.1957463Z error: line not found in debugger output: [Reference count] : 11 [Type: core:: cell FAIL::Cell<usize>]
2023-07-04T08:10:00.1958272Z status: exit code: 0
```
With this chane, you are able to see all failures in that check group, as well as what parts were successful. The output is now:
```
2023-07-04T09:45:57.2514224Z error: check directive(s) from `C:\a\rust\rust\tests\debuginfo\rc_arc.rs` not found in debugger output. errors:
2023-07-04T09:45:57.2514631Z (rc_arc.rs:31) ` [Reference count] : 11 [Type: core::cell FAIL::Cell<usize>]`
2023-07-04T09:45:57.2514908Z (rc_arc.rs:32) ` [Weak reference count] : 2 [Type: core::cell FAIL::Cell<usize>]`
2023-07-04T09:45:57.2515181Z (rc_arc.rs:41) ` [Reference count] : 21 [Type: core::sync::atomic FAIL::AtomicUsize]`
2023-07-04T09:45:57.2515452Z (rc_arc.rs:50) `dyn_rc,d [Type: alloc::rc::Rc<dyn$<core::fmt FAIL::Debug> >]`
2023-07-04T09:45:57.2515695Z the following subset of check directive(s) was found successfully::
2023-07-04T09:45:57.2516080Z (rc_arc.rs:30) `rc,d : 111 [Type: alloc::rc::Rc<i32>]`
2023-07-04T09:45:57.2516312Z (rc_arc.rs:35) `weak_rc,d : 111 [Type: alloc::rc::Weak<i32>]`
2023-07-04T09:45:57.2516555Z (rc_arc.rs:36) ` [Reference count] : 11 [Type: core::cell::Cell<usize>]`
2023-07-04T09:45:57.2516881Z (rc_arc.rs:37) ` [Weak reference count] : 2 [Type: core::cell::Cell<usize>]`
...
```
Which makes it easier to see what did and didn't succeed without manual comparison against the source test file.
fix compiletest crash
### Motivation
When running compiler-tests locally for the `wasm32` platform, one test repeatedly crashed. It does not crash on the CI, only locally. Investigation shows that the `compiletest` itself crashes
> panicked-at-attempt-to-subtract-with-overflow
```rust
let mut head = replace(bytes, Vec::new());
let mut middle = head.split_off(HEAD_LEN);
// The following line will panic
let tail = middle.split_off(middle.len() - TAIL_LEN).into_boxed_slice();
let skipped = new_len - HEAD_LEN - TAIL_LEN;
```
### Background
The code in question collects the output of a process. Small output is kept completely, but larger output is kept only partially: the first 160 kB and the last 256 kB.
The code that performs this split crashes if the data size is less than 416 kB. There is an early out based on the "filtered" length, but it is possible that the filtered length is greater than the real length. It seems that this code was written with the assumption that the filtered length is larger than the real length, which is not true in general.
When running CI tests locally using `src/ci/docker/run.sh`, the filtered folder is `/checkout`, which is shorter than the placeholder length of 32 bytes.
### Note
This PR should not change any behaviour. It only adds an early our for a case which will definitely crash (at least if compiletest is build with integer checks).
Note that an early out makes sense here: If the real data is too small, it does not sense to split it.
Fix the tests-listing-format-json test on Windows
tests/ui/test-attrs/tests-listing-json-format.rs was failing on Windows because each path in the json-formatted output contained "\\\\" instead of "\\". `runtest::TestCx::normalize_output` already checks the compile flags for json-related arguments to handle this case, so I added an equivalent check for the new run flag.
This change makes debuginfo tests more user friendly. Changes:
- Print all lines that fail to match the patterns instead of just
the first
- Provide better error messages that also say what did match
- Strip leading whitespace from directives so they are not skipped if
indented
- Improve documentation and improve nesting on some related items
As an example, given the following debuginfo test with intentional
fails:
```rust
// from tests/debuginfo/rc_arc.rs
// cdb-command:dx rc,d
// cdb-check:rc,d : 111 [Type: alloc::rc::Rc<i32>]
// cdb-check: [Reference count] : 11 [Type: core::cell FAIL::Cell<usize>]
// cdb-check: [Weak reference count] : 2 [Type: core::cell FAIL::Cell<usize>]
// ...
```
The current output (tested in #113313) only shows the first mismatch:
```
2023-07-04T08:10:00.1939267Z ---- [debuginfo-cdb] tests\debuginfo\rc_arc.rs stdout ----
2023-07-04T08:10:00.1942182Z
2023-07-04T08:10:00.1957463Z error: line not found in debugger output: [Reference count] : 11 [Type: core::cell FAIL::Cell<usize>]
2023-07-04T08:10:00.1958272Z status: exit code: 0
```
With this change, you are able to see all failures in that check
group, as well as what parts were successful. The output is now:
```
2023-07-04T09:45:57.2514224Z error: check directive(s) from `C:\a\rust\rust\tests\debuginfo\rc_arc.rs` not found in debugger output. errors:
2023-07-04T09:45:57.2514631Z (rc_arc.rs:31) ` [Reference count] : 11 [Type: core::cell FAIL::Cell<usize>]`
2023-07-04T09:45:57.2514908Z (rc_arc.rs:32) ` [Weak reference count] : 2 [Type: core::cell FAIL::Cell<usize>]`
2023-07-04T09:45:57.2515181Z (rc_arc.rs:41) ` [Reference count] : 21 [Type: core::sync::atomic FAIL::AtomicUsize]`
2023-07-04T09:45:57.2515452Z (rc_arc.rs:50) `dyn_rc,d [Type: alloc::rc::Rc<dyn$<core::fmt FAIL::Debug> >]`
2023-07-04T09:45:57.2515695Z the following subset of check directive(s) was found successfully::
2023-07-04T09:45:57.2516080Z (rc_arc.rs:30) `rc,d : 111 [Type: alloc::rc::Rc<i32>]`
2023-07-04T09:45:57.2516312Z (rc_arc.rs:35) `weak_rc,d : 111 [Type: alloc::rc::Weak<i32>]`
2023-07-04T09:45:57.2516555Z (rc_arc.rs:36) ` [Reference count] : 11 [Type: core::cell::Cell<usize>]`
2023-07-04T09:45:57.2516881Z (rc_arc.rs:37) ` [Weak reference count] : 2 [Type: core::cell::Cell<usize>]`
...
```
Which makes it easier to see what did and didn't succeed without
manual comparison against the source test file.
compiletest: Only trim the end of process output
As of #94196, compiletest automatically trims process stderr/stdout output before printing it, to make failure info more compact.
This causes the first line of `run-coverage` output to be displayed incorrectly, because it uses leading whitespace to align line numbers.
Trimming only the end of the output string should still have the intended effect (e.g. removing trailing newlines), without causing problems for output that deliberately uses leading whitespace on the first line.
## Before
```
--- stdout -------------------------------
1| 1|fn main() { //
2| 1| let num = 9;
3| 1| while num >= 10 {
4| 0| }
5| 1|}
------------------------------------------
stderr: none
```
## After
```
--- stdout -------------------------------
1| 1|fn main() { //
2| 1| let num = 9;
3| 1| while num >= 10 {
4| 0| }
5| 1|}
------------------------------------------
stderr: none
```
To make it easier to verify that the output snapshots have been migrated
faithfully, this change adds some temporary helper code that lets us avoid
having to completely re-bless the existing snapshots.
A later change in this PR will then re-bless the tests and remove the temporary
helper code.
Currently a test without a `failure-status` directive is treated as having an
expected failure-status of 1, but `run-coverage` tests will want to treat those
tests as expecting success instead.
This will allow the `run-coverage` mode to easily set environment variable
`LLVM_PROFILE_FILE`, and to prevent the executable from being deleted after a
successful run.
Stop normalizing so many different prefixes
Previously, we would normalize *all* of
- the absolute path to the repository checkout
- the /rustc/$sha for stage1 (if `remap-debuginfo` was enabled)
- the /rustc/$sha for download-rustc
- the sysroot for download-rustc
Now, we consistently only normalize /rustc/FAKE_PREFIX. Not only is this much simpler, but it also avoids ongoing maintenance for download-rustc and makes it much less likely that tests break by accident.
- Change `tests/ui/track-diagnostics/track6.rs` to use a relative path instead of an absolute one. I am not actually sure why `track_caller` works here, but it does seem to work 🤷
- Pass `-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX` to all suites, not just UI. In particular, mir-opt tests emit /rustc/ paths in their output.
r? ```@cjgillot``` since you reviewed https://github.com/rust-lang/rust/pull/110699 - this is the test that it doesn't regress :)
MIR: opt-in normalization of `BasicBlock` and `Local` numbering
This doesn't matter at all for actual codegen, but after spending some time reading pre-codegen MIR, I was wishing I didn't have to jump around so much in reading post-inlining code.
So this add two passes that are off by default for every mir level, but can be enabled (`-Zmir-enable-passes=+ReorderBasicBlocks,+ReorderLocals`) for humans.
