Auto merge of #43083 - kennytm:fix-42434-custom-stdxxx-normalization, r=nikomatsakis

compilertest (UI test): Support custom normalization.

Closes #42434.

Adds this header for UI tests:

```rust
// normalize-stderr-32bit: "fn() (32 bits)" -> "fn() ($PTR bits)"
```

It will normalize the `stderr` output on 32-bit platforms, by replacing all instances of `fn() (32 bits)` by `fn() ($PTR bits)`.

Extends the UI tests in #42304 and #41968 to 32-bit targets.

r? @nikomatsakis
This commit is contained in:
bors 2017-07-11 22:02:20 +00:00
commit b360b44ecf
13 changed files with 234 additions and 163 deletions

View file

@ -37,7 +37,7 @@ The error levels that you can have are:
Header commands specify something about the entire test file as a
whole, instead of just a few lines inside the test.
* `ignore-X` where `X` is an architecture, OS or stage will ignore the test accordingly
* `ignore-X` where `X` is a target detail or stage will ignore the test accordingly (see below)
* `ignore-pretty` will not compile the pretty-printed test (this is done to test the pretty-printer, but might not always work)
* `ignore-test` always ignores the test
* `ignore-lldb` and `ignore-gdb` will skip the debuginfo tests
@ -50,6 +50,14 @@ whole, instead of just a few lines inside the test.
feature is attempted without the proper `#![feature(X)]` tag.
Each unstable lang feature is required to have a gate test.
Some examples of `X` in `ignore-X`:
* Architecture: `aarch64`, `arm`, `asmjs`, `mips`, `wasm32`, `x86_64`, `x86`, ...
* OS: `android`, `emscripten`, `freebsd`, `ios`, `linux`, `macos`, `windows`, ...
* Environment (fourth word of the target triple): `gnu`, `msvc`, `musl`.
* Pointer width: `32bit`, `64bit`.
* Stage: `stage0`, `stage1`, `stage2`.
## Revisions
Certain classes of tests support "revisions" (as of the time of this
@ -86,3 +94,66 @@ For example, the `ignore-test` header (and all "ignore" headers)
currently only apply to the test as a whole, not to particular
revisions. The only headers that are intended to really work when
customized to a revision are error patterns and compiler flags.
## Guide to the UI Tests
The UI tests are intended to capture the compiler's complete output,
so that we can test all aspects of the presentation. They work by
compiling a file (e.g., `ui/hello_world/main.rs`), capturing the output,
and then applying some normalization (see below). This normalized
result is then compared against reference files named
`ui/hello_world/main.stderr` and `ui/hello_world/main.stdout`. If either of
those files doesn't exist, the output must be empty. If the test run
fails, we will print out the current output, but it is also saved in
`build/<target-triple>/test/ui/hello_world/main.stdout` (this path is
printed as part of the test failure mesage), so you can run `diff` and
so forth.
### Editing and updating the reference files
If you have changed the compiler's output intentionally, or you are
making a new test, you can use the script `ui/update-references.sh` to
update the references. When you run the test framework, it will report
various errors: in those errors is a command you can use to run the
`ui/update-references.sh` script, which will then copy over the files
from the build directory and use them as the new reference. You can
also just run `ui/update-all-references.sh`. In both cases, you can run
the script with `--help` to get a help message.
### Normalization
The normalization applied is aimed at eliminating output difference
between platforms, mainly about filenames:
- the test directory is replaced with `$DIR`
- all backslashes (`\`) are converted to forward slashes (`/`) (for Windows)
- all CR LF newlines are converted to LF
Sometimes these built-in normalizations are not enough. In such cases, you
may provide custom normalization rules using the header commands, e.g.
```
// normalize-stderr-32bit: "fn() (32 bits)" -> "fn() ($PTR bits)"
// normalize-stderr-64bit: "fn() (64 bits)" -> "fn() ($PTR bits)"
```
This tells the test, on 32-bit platforms, whenever the compiler writes
`fn() (32 bits)` to stderr, it should be normalized to read `fn() ($PTR bits)`
instead. Similar for 64-bit.
The corresponding reference file will use the normalized output to test both
32-bit and 64-bit platforms:
```
...
|
= note: source type: fn() ($PTR bits)
= note: target type: u16 (16 bits)
...
```
Please see `ui/transmute/main.rs` and `.stderr` for a concrete usage example.
Besides `normalize-stderr-32bit` and `-64bit`, one may use any target
information or stage supported by `ignore-X` here as well (e.g.
`normalize-stderr-windows`).

View file

@ -13,12 +13,7 @@
// should look like.
// ignore-windows
// Ignore 32 bit targets:
// ignore-x86
// ignore-arm
// ignore-emscripten
// ignore-32bit
#![feature(i128_type)]

View file

@ -1,31 +0,0 @@
# Guide to the UI Tests
The UI tests are intended to capture the compiler's complete output,
so that we can test all aspects of the presentation. They work by
compiling a file (e.g., `hello_world/main.rs`), capturing the output,
and then applying some normalization (see below). This normalized
result is then compared against reference files named
`hello_world/main.stderr` and `hello_world/main.stdout`. If either of
those files doesn't exist, the output must be empty. If the test run
fails, we will print out the current output, but it is also saved in
`build/<target-triple>/test/ui/hello_world/main.stdout` (this path is
printed as part of the test failure mesage), so you can run `diff` and
so forth.
# Editing and updating the reference files
If you have changed the compiler's output intentionally, or you are
making a new test, you can use the script `update-references.sh` to
update the references. When you run the test framework, it will report
various errors: in those errors is a command you can use to run the
`update-references.sh` script, which will then copy over the files
from the build directory and use them as the new reference. You can
also just run `update-all-references.sh`. In both cases, you can run
the script with `--help` to get a help message.
# Normalization
The normalization applied is aimed at filenames:
- the test directory is replaced with `$DIR`
- all backslashes (\) are converted to forward slashes (/) (for windows)

View file

@ -9,10 +9,6 @@
// except according to those terms.
// run-pass
// ignore-x86
// ignore-arm
// ignore-emscripten
// ^ ignore 32-bit targets, as the error message is target-dependent. see PR #41968.
#![warn(variant_size_differences)]
#![allow(dead_code)]
@ -24,26 +20,26 @@ enum Enum1 { }
enum Enum2 { A, B, C }
enum Enum3 { D(isize), E, F }
enum Enum3 { D(i64), E, F }
enum Enum4 { H(isize), I(isize), J }
enum Enum4 { H(i64), I(i64), J }
enum Enum5 {
L(isize, isize, isize, isize), //~ WARNING three times larger
M(isize),
L(i64, i64, i64, i64), //~ WARNING three times larger
M(i64),
N
}
enum Enum6<T, U> {
O(T),
P(U),
Q(isize)
Q(i64)
}
#[allow(variant_size_differences)]
enum Enum7 {
R(isize, isize, isize, isize),
S(isize),
R(i64, i64, i64, i64),
S(i64),
T
}
pub fn main() { }

View file

@ -1,12 +1,12 @@
warning: enum variant is more than three times larger (32 bytes) than the next largest
--> $DIR/enum-size-variance.rs:32:5
--> $DIR/enum-size-variance.rs:28:5
|
32 | L(isize, isize, isize, isize), //~ WARNING three times larger
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
28 | L(i64, i64, i64, i64), //~ WARNING three times larger
| ^^^^^^^^^^^^^^^^^^^^^
|
note: lint level defined here
--> $DIR/enum-size-variance.rs:17:9
--> $DIR/enum-size-variance.rs:13:9
|
17 | #![warn(variant_size_differences)]
13 | #![warn(variant_size_differences)]
| ^^^^^^^^^^^^^^^^^^^^^^^^

View file

@ -8,10 +8,10 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// ignore-x86
// ignore-arm
// ignore-emscripten
// ignore 32-bit platforms (test output is different)
// normalize-stderr-32bit: "&str (64 bits)" -> "&str ($STR bits)"
// normalize-stderr-64bit: "&str (128 bits)" -> "&str ($STR bits)"
#![feature(untagged_unions)]
use std::mem::transmute;

View file

@ -22,7 +22,7 @@ error[E0512]: transmute called with types of different sizes
34 | let x: u8 = transmute("test"); //~ ERROR transmute called with types of different sizes
| ^^^^^^^^^
|
= note: source type: &str (128 bits)
= note: source type: &str ($STR bits)
= note: target type: u8 (8 bits)
error[E0512]: transmute called with types of different sizes

View file

@ -8,14 +8,9 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// ignore-x86
// ignore-arm
// ignore-emscripten
// ignore 32-bit platforms (test output is different)
use std::mem;
unsafe fn foo() -> (i32, *const (), Option<fn()>) {
unsafe fn foo() -> (i8, *const (), Option<fn()>) {
let i = mem::transmute(bar);
//~^ ERROR is zero-sized and can't be transmuted
//~^^ NOTE cast with `as` to a pointer instead
@ -46,7 +41,7 @@ unsafe fn bar() {
//~^^ NOTE cast with `as` to a pointer instead
// No error if a coercion would otherwise occur.
mem::transmute::<fn(), u32>(main);
mem::transmute::<fn(), usize>(main);
}
unsafe fn baz() {
@ -63,7 +58,7 @@ unsafe fn baz() {
//~^^ NOTE cast with `as` to a pointer instead
// No error if a coercion would otherwise occur.
mem::transmute::<Option<fn()>, u32>(Some(main));
mem::transmute::<Option<fn()>, usize>(Some(main));
}
fn main() {

View file

@ -1,26 +1,26 @@
error[E0512]: transmute called with types of different sizes
--> $DIR/transmute-from-fn-item-types-error.rs:19:13
--> $DIR/transmute-from-fn-item-types-error.rs:14:13
|
19 | let i = mem::transmute(bar);
14 | let i = mem::transmute(bar);
| ^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() {bar} (0 bits)
= note: target type: i32 (32 bits)
= note: target type: i8 (8 bits)
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:23:13
--> $DIR/transmute-from-fn-item-types-error.rs:18:13
|
23 | let p = mem::transmute(foo);
18 | let p = mem::transmute(foo);
| ^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() -> (i32, *const (), std::option::Option<fn()>) {foo}
= note: source type: unsafe fn() -> (i8, *const (), std::option::Option<fn()>) {foo}
= note: target type: *const ()
= help: cast with `as` to a pointer instead
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:27:14
--> $DIR/transmute-from-fn-item-types-error.rs:22:14
|
27 | let of = mem::transmute(main);
22 | let of = mem::transmute(main);
| ^^^^^^^^^^^^^^
|
= note: source type: fn() {main}
@ -28,57 +28,28 @@ error[E0591]: can't transmute zero-sized type
= help: cast with `as` to a pointer instead
error[E0512]: transmute called with types of different sizes
--> $DIR/transmute-from-fn-item-types-error.rs:36:5
--> $DIR/transmute-from-fn-item-types-error.rs:31:5
|
36 | mem::transmute::<_, u8>(main);
31 | mem::transmute::<_, u8>(main);
| ^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: fn() {main} (0 bits)
= note: target type: u8 (8 bits)
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:40:5
--> $DIR/transmute-from-fn-item-types-error.rs:35:5
|
40 | mem::transmute::<_, *mut ()>(foo);
35 | mem::transmute::<_, *mut ()>(foo);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() -> (i32, *const (), std::option::Option<fn()>) {foo}
= note: source type: unsafe fn() -> (i8, *const (), std::option::Option<fn()>) {foo}
= note: target type: *mut ()
= help: cast with `as` to a pointer instead
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:44:5
--> $DIR/transmute-from-fn-item-types-error.rs:39:5
|
44 | mem::transmute::<_, fn()>(bar);
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() {bar}
= note: target type: fn()
= help: cast with `as` to a pointer instead
error[E0512]: transmute called with types of different sizes
--> $DIR/transmute-from-fn-item-types-error.rs:49:5
|
49 | mem::transmute::<fn(), u32>(main);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: fn() (64 bits)
= note: target type: u32 (32 bits)
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:53:5
|
53 | mem::transmute::<_, *mut ()>(Some(foo));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() -> (i32, *const (), std::option::Option<fn()>) {foo}
= note: target type: *mut ()
= help: cast with `as` to a pointer instead
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:57:5
|
57 | mem::transmute::<_, fn()>(Some(bar));
39 | mem::transmute::<_, fn()>(bar);
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() {bar}
@ -86,23 +57,34 @@ error[E0591]: can't transmute zero-sized type
= help: cast with `as` to a pointer instead
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:61:5
--> $DIR/transmute-from-fn-item-types-error.rs:48:5
|
61 | mem::transmute::<_, Option<fn()>>(Some(baz));
48 | mem::transmute::<_, *mut ()>(Some(foo));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() -> (i8, *const (), std::option::Option<fn()>) {foo}
= note: target type: *mut ()
= help: cast with `as` to a pointer instead
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:52:5
|
52 | mem::transmute::<_, fn()>(Some(bar));
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() {bar}
= note: target type: fn()
= help: cast with `as` to a pointer instead
error[E0591]: can't transmute zero-sized type
--> $DIR/transmute-from-fn-item-types-error.rs:56:5
|
56 | mem::transmute::<_, Option<fn()>>(Some(baz));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: unsafe fn() {baz}
= note: target type: std::option::Option<fn()>
= help: cast with `as` to a pointer instead
error[E0512]: transmute called with types of different sizes
--> $DIR/transmute-from-fn-item-types-error.rs:66:5
|
66 | mem::transmute::<Option<fn()>, u32>(Some(main));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: source type: std::option::Option<fn()> (64 bits)
= note: target type: u32 (32 bits)
error: aborting due to 11 previous errors
error: aborting due to 9 previous errors

View file

@ -8,10 +8,10 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.
// ignore-x86
// ignore-arm
// ignore-emscripten
// ignore 32-bit platforms (test output is different)
// Tests that `transmute` cannot be called on type parameters.

View file

@ -40,15 +40,8 @@ impl EarlyProps {
None,
&mut |ln| {
props.ignore =
props.ignore || config.parse_name_directive(ln, "ignore-test") ||
config.parse_name_directive(ln, &ignore_target(config)) ||
config.parse_name_directive(ln, &ignore_architecture(config)) ||
config.parse_name_directive(ln, &ignore_stage(config)) ||
config.parse_name_directive(ln, &ignore_env(config)) ||
(config.mode == common::Pretty &&
config.parse_name_directive(ln, "ignore-pretty")) ||
(config.target != config.host &&
config.parse_name_directive(ln, "ignore-cross-compile")) ||
props.ignore ||
config.parse_cfg_name_directive(ln, "ignore") ||
ignore_gdb(config, ln) ||
ignore_lldb(config, ln) ||
ignore_llvm(config, ln);
@ -62,28 +55,11 @@ impl EarlyProps {
return props;
fn ignore_target(config: &Config) -> String {
format!("ignore-{}", util::get_os(&config.target))
}
fn ignore_architecture(config: &Config) -> String {
format!("ignore-{}", util::get_arch(&config.target))
}
fn ignore_stage(config: &Config) -> String {
format!("ignore-{}", config.stage_id.split('-').next().unwrap())
}
fn ignore_env(config: &Config) -> String {
format!("ignore-{}",
util::get_env(&config.target).unwrap_or("<unknown>"))
}
fn ignore_gdb(config: &Config, line: &str) -> bool {
if config.mode != common::DebugInfoGdb {
return false;
}
if config.parse_name_directive(line, "ignore-gdb") {
return true;
}
if let Some(actual_version) = config.gdb_version {
if line.starts_with("min-gdb-version") {
let (start_ver, end_ver) = extract_gdb_version_range(line);
@ -144,10 +120,6 @@ impl EarlyProps {
return false;
}
if config.parse_name_directive(line, "ignore-lldb") {
return true;
}
if let Some(ref actual_version) = config.lldb_version {
if line.starts_with("min-lldb-version") {
let min_version = line.trim_right()
@ -239,6 +211,9 @@ pub struct TestProps {
// The test must be compiled and run successfully. Only used in UI tests for
// now.
pub run_pass: bool,
// customized normalization rules
pub normalize_stdout: Vec<(String, String)>,
pub normalize_stderr: Vec<(String, String)>,
}
impl TestProps {
@ -265,6 +240,8 @@ impl TestProps {
must_compile_successfully: false,
check_test_line_numbers_match: false,
run_pass: false,
normalize_stdout: vec![],
normalize_stderr: vec![],
}
}
@ -379,6 +356,13 @@ impl TestProps {
if !self.run_pass {
self.run_pass = config.parse_run_pass(ln);
}
if let Some(rule) = config.parse_custom_normalization(ln, "normalize-stdout") {
self.normalize_stdout.push(rule);
}
if let Some(rule) = config.parse_custom_normalization(ln, "normalize-stderr") {
self.normalize_stderr.push(rule);
}
});
for key in &["RUST_TEST_NOCAPTURE", "RUST_TEST_THREADS"] {
@ -427,7 +411,6 @@ fn iter_header(testfile: &Path, cfg: Option<&str>, it: &mut FnMut(&str)) {
}
impl Config {
fn parse_error_pattern(&self, line: &str) -> Option<String> {
self.parse_name_value_directive(line, "error-pattern")
}
@ -525,6 +508,46 @@ impl Config {
}
}
fn parse_custom_normalization(&self, mut line: &str, prefix: &str) -> Option<(String, String)> {
if self.parse_cfg_name_directive(line, prefix) {
let from = match parse_normalization_string(&mut line) {
Some(s) => s,
None => return None,
};
let to = match parse_normalization_string(&mut line) {
Some(s) => s,
None => return None,
};
Some((from, to))
} else {
None
}
}
/// Parses a name-value directive which contains config-specific information, e.g. `ignore-x86`
/// or `normalize-stderr-32bit`. Returns `true` if the line matches it.
fn parse_cfg_name_directive(&self, line: &str, prefix: &str) -> bool {
if line.starts_with(prefix) && line.as_bytes().get(prefix.len()) == Some(&b'-') {
let name = line[prefix.len()+1 ..].split(&[':', ' '][..]).next().unwrap();
name == "test" ||
name == util::get_os(&self.target) || // target
name == util::get_arch(&self.target) || // architecture
name == util::get_pointer_width(&self.target) || // pointer width
name == self.stage_id.split('-').next().unwrap() || // stage
Some(name) == util::get_env(&self.target) || // env
match self.mode {
common::DebugInfoGdb => name == "gdb",
common::DebugInfoLldb => name == "lldb",
common::Pretty => name == "pretty",
_ => false,
} ||
(self.target != self.host && name == "cross-compile")
} else {
false
}
}
fn parse_name_directive(&self, line: &str, directive: &str) -> bool {
// Ensure the directive is a whole word. Do not match "ignore-x86" when
// the line says "ignore-x86_64".
@ -572,3 +595,29 @@ fn expand_variables(mut value: String, config: &Config) -> String {
value
}
/// Finds the next quoted string `"..."` in `line`, and extract the content from it. Move the `line`
/// variable after the end of the quoted string.
///
/// # Examples
///
/// ```
/// let mut s = "normalize-stderr-32bit: \"something (32 bits)\" -> \"something ($WORD bits)\".";
/// let first = parse_normalization_string(&mut s);
/// assert_eq!(first, Some("something (32 bits)".to_owned()));
/// assert_eq!(s, " -> \"something ($WORD bits)\".");
/// ```
fn parse_normalization_string(line: &mut &str) -> Option<String> {
// FIXME support escapes in strings.
let begin = match line.find('"') {
Some(i) => i + 1,
None => return None,
};
let end = match line[begin..].find('"') {
Some(i) => i + begin,
None => return None,
};
let result = line[begin..end].to_owned();
*line = &line[end+1..];
Some(result)
}

View file

@ -2228,8 +2228,10 @@ actual:\n\
let expected_stdout_path = self.expected_output_path("stdout");
let expected_stdout = self.load_expected_output(&expected_stdout_path);
let normalized_stdout = self.normalize_output(&proc_res.stdout);
let normalized_stderr = self.normalize_output(&proc_res.stderr);
let normalized_stdout =
self.normalize_output(&proc_res.stdout, &self.props.normalize_stdout);
let normalized_stderr =
self.normalize_output(&proc_res.stderr, &self.props.normalize_stderr);
let mut errors = 0;
errors += self.compare_output("stdout", &normalized_stdout, &expected_stdout);
@ -2375,13 +2377,17 @@ actual:\n\
mir_dump_dir
}
fn normalize_output(&self, output: &str) -> String {
fn normalize_output(&self, output: &str, custom_rules: &[(String, String)]) -> String {
let parent_dir = self.testpaths.file.parent().unwrap();
let parent_dir_str = parent_dir.display().to_string();
output.replace(&parent_dir_str, "$DIR")
let mut normalized = output.replace(&parent_dir_str, "$DIR")
.replace("\\", "/") // normalize for paths on windows
.replace("\r\n", "\n") // normalize for linebreaks on windows
.replace("\t", "\\t") // makes tabs visible
.replace("\t", "\\t"); // makes tabs visible
for rule in custom_rules {
normalized = normalized.replace(&rule.0, &rule.1);
}
normalized
}
fn expected_output_path(&self, kind: &str) -> PathBuf {

View file

@ -72,6 +72,14 @@ pub fn get_env(triple: &str) -> Option<&str> {
triple.split('-').nth(3)
}
pub fn get_pointer_width(triple: &str) -> &'static str {
if triple.contains("64") || triple.starts_with("s390x") {
"64bit"
} else {
"32bit"
}
}
pub fn make_new_path(path: &str) -> String {
assert!(cfg!(windows));
// Windows just uses PATH as the library search path, so we have to