rust/src/test/ui
Ariel Ben-Yehuda fb7ab9e43d report the total number of errors on compilation failure
Prior to this PR, when we aborted because a "critical pass" failed, we
displayed the number of errors from that critical pass. While that's the
number of errors that caused compilation to abort in *that place*,
that's not what people really want to know. Instead, always report the
total number of errors, and don't bother to track the number of errors
from the last pass that failed.

This changes the compiler driver API to handle errors more smoothly,
and therefore is a compiler-api-[breaking-change].

Fixes #42793.
2017-07-02 16:16:44 +03:00
..
block-result report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
borrowck Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
check_match Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
closure_context report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
codemap_tests Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
compare-method Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
cross-crate-macro-backtrace Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
did_you_mean report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
dropck Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
fmt Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
hello_world add UI testing framework 2016-05-13 15:22:45 -07:00
impl-trait report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
interior-mutability report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
invalid-module-declaration Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
issue-37311-type-length-limit Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
issue-38875 report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
issue-40402-ref-hints Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
issue-41652 Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
lifetime-errors report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
lifetimes Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
lint report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
macros Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
mismatched_types report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
missing-items report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
print-fuel UI test for -Z print-fuel=foo 2017-04-11 14:36:08 +03:00
print_type_sizes Stabilize unions with Copy fields and no destructor 2017-05-27 00:52:20 +03:00
pub Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
reachable Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
resolve report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
span report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
suggestions Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
token Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
transmute report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
type-check report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
coercion-missing-tail-expected-type.rs Add test for #24889 2017-06-18 14:30:33 +02:00
coercion-missing-tail-expected-type.stderr report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
deprecated-macro_escape-inner.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
deprecated-macro_escape-inner.stderr Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
deprecated-macro_escape.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
deprecated-macro_escape.stderr Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
deriving-meta-empty-trait-list.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
deriving-meta-empty-trait-list.stderr Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
enum-size-variance.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
enum-size-variance.stderr Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
explain.rs Modify --explain to handle hidden code (# ...) and indented code blocks. 2017-06-23 15:31:53 +08:00
explain.stdout Modify --explain to handle hidden code (# ...) and indented code blocks. 2017-06-23 15:31:53 +08:00
issue-19100.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
issue-19100.stderr Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
issue-22644.rs Suggest non-ambiguous comparison after cast 2017-06-11 23:47:26 -07:00
issue-22644.stderr report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
issue-26548.rs remove layout_cache and fix -Z print-type-sizes 2017-06-01 13:31:11 -04:00
issue-26548.stderr report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
issue-33525.rs report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
issue-33525.stderr report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
loop-break-value-no-repeat.rs Stabilize the loop_break_value feature 2017-05-17 21:34:37 +02:00
loop-break-value-no-repeat.stderr Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
path-lookahead.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
path-lookahead.stderr make lint on-by-default/implied-by messages appear only once 2017-06-26 16:10:06 -07:00
README.md add UI testing framework 2016-05-13 15:22:45 -07:00
regions-fn-subtyping-return-static.rs make lifetimes that only appear in return type early-bound 2017-02-05 16:30:03 -05:00
regions-fn-subtyping-return-static.stderr Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
shadowed-type-parameter.rs Move type parameter shadowing test to ui 2017-06-28 15:11:05 -07:00
shadowed-type-parameter.stderr report the total number of errors on compilation failure 2017-07-02 16:16:44 +03:00
static-lifetime.rs Improve E0477 error message 2017-05-09 20:51:18 +02:00
static-lifetime.stderr Revert "Change error count messages" 2017-07-02 13:49:30 +03:00
test-should-panic-attr.rs Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
test-should-panic-attr.stderr Introduce 'run-pass' header to 'ui' tests in compiletest. Fix issue #36516. 2017-06-02 23:28:22 +08:00
update-all-references.sh pacify the mercilous tidy 2016-05-16 16:48:48 -04:00
update-references.sh cleanup error reporting and add ui tests 2016-11-01 14:04:14 -04:00

Guide to the UI Tests

The UI tests are intended to capture the compiler's complete output, so that we can test all aspects of the presentation. They work by compiling a file (e.g., hello_world/main.rs), capturing the output, and then applying some normalization (see below). This normalized result is then compared against reference files named hello_world/main.stderr and hello_world/main.stdout. If either of those files doesn't exist, the output must be empty. If the test run fails, we will print out the current output, but it is also saved in build/<target-triple>/test/ui/hello_world/main.stdout (this path is printed as part of the test failure mesage), so you can run diff and so forth.

Editing and updating the reference files

If you have changed the compiler's output intentionally, or you are making a new test, you can use the script update-references.sh to update the references. When you run the test framework, it will report various errors: in those errors is a command you can use to run the update-references.sh script, which will then copy over the files from the build directory and use them as the new reference. You can also just run update-all-references.sh. In both cases, you can run the script with --help to get a help message.

Normalization

The normalization applied is aimed at filenames:

  • the test directory is replaced with $DIR
  • all backslashes () are converted to forward slashes (/) (for windows)