user0/rust
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

sembr src/tests/directives.md

Browse source
This commit is contained in:
Tshepang Mbambo 2026-01-12 20:52:54 +02:00
parent ef5bae8393
commit 419655be81
1 changed files with 77 additions and 90 deletions
Download patch file Download diff file Expand all files Collapse all files

167
src/doc/rustc-dev-guide/src/tests/directives.md
View file

@ -7,8 +7,8 @@ FIXME(jieyouxu) completely revise this chapter.
Directives are special comments that tell compiletest how to build and interpret a test.
They may also appear in `rmake.rs` [run-make tests](compiletest.md#run-make-tests).
They are normally put after the short comment that explains the point of this
test. Compiletest test suites use `//@` to signal that a comment is a directive.
They are normally put after the short comment that explains the point of this test.
Compiletest test suites use `//@` to signal that a comment is a directive.
For example, this test uses the `//@ compile-flags` command to specify a custom
flag to give to rustc when the test is compiled:
@ -27,15 +27,16 @@ Directives can be standalone (like `//@ run-pass`) or take a value (like `//@
compile-flags: -C overflow-checks=off`).
Directives are written one directive per line: you cannot write multiple
directives on the same line. For example, if you write `//@ only-x86
only-windows` then `only-windows` is interpreted as a comment, not a separate
directive.
directives on the same line.
For example, if you write `//@ only-x86
only-windows` then `only-windows` is interpreted as a comment, not a separate directive.
## Listing of compiletest directives
The following is a list of compiletest directives. Directives are linked to
sections that describe the command in more detail if available. This list may
not be exhaustive. Directives can generally be found by browsing the
The following is a list of compiletest directives.
Directives are linked to sections that describe the command in more detail if available.
This list may not be exhaustive.
Directives can generally be found by browsing the
`TestProps` structure found in [`directives.rs`] from the compiletest source.
[`directives.rs`]: https://github.com/rust-lang/rust/tree/HEAD/src/tools/compiletest/src/directives.rs
@ -65,8 +66,7 @@ See [Building auxiliary crates](compiletest.html#building-auxiliary-crates)
### Controlling outcome expectations
See [Controlling pass/fail
expectations](ui.md#controlling-passfail-expectations).
See [Controlling pass/fail expectations](ui.md#controlling-passfail-expectations).
| Directive | Explanation | Supported test suites | Possible values |
|-----------------------------|---------------------------------------------|-------------------------------------------|-----------------|
@ -87,8 +87,7 @@ expectations](ui.md#controlling-passfail-expectations).
### Controlling output snapshots and normalizations
See [Normalization](ui.md#normalization), [Output
comparison](ui.md#output-comparison) and [Rustfix tests](ui.md#rustfix-tests)
for more details.
comparison](ui.md#output-comparison) and [Rustfix tests](ui.md#rustfix-tests) for more details.
| Directive | Explanation | Supported test suites | Possible values |
|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------|-----------------------------------------------------------------------------------------|
@ -115,8 +114,8 @@ for more details.
[^check_stdout]: presently <!-- date-check: Oct 2024 --> this has a weird quirk
where the test binary's stdout and stderr gets concatenated and then
`error-pattern`s are matched on this combined output, which is ??? slightly
questionable to say the least.
`error-pattern`s are matched on this combined output, which is ???
slightly questionable to say the least.
### Controlling when tests are run
@ -124,14 +123,13 @@ These directives are used to ignore the test in some situations, which
means the test won't be compiled or run.
* `ignore-X` where `X` is a target detail or other criteria on which to ignore the test (see below)
* `only-X` is like `ignore-X`, but will *only* run the test on that target or
stage
* `only-X` is like `ignore-X`, but will *only* run the test on that target or stage
* `ignore-auxiliary` is intended for files that *participate* in one or more other
main test files but that `compiletest` should not try to build the file itself.
Please backlink to which main test is actually using the auxiliary file.
* `ignore-test` always ignores the test. This can be used to temporarily disable
a test if it is currently not working, but you want to keep it in-tree to
re-enable it later.
* `ignore-test` always ignores the test.
This can be used to temporarily disable
a test if it is currently not working, but you want to keep it in-tree to re-enable it later.
Some examples of `X` in `ignore-X` or `only-X`:
@ -158,16 +156,15 @@ Some examples of `X` in `ignore-X` or `only-X`:
- This needs to be enabled with `COMPILETEST_ENABLE_DIST_TESTS=1`
- The `rustc_abi` of the target: e.g. `rustc_abi-x86_64-sse2`
The following directives will check rustc build settings and target
settings:
The following directives will check rustc build settings and target settings:
- `needs-asm-support` — ignores if the **host** architecture doesn't have
stable support for `asm!`. For tests that cross-compile to explicit targets
stable support for `asm!`.
For tests that cross-compile to explicit targets
via `--target`, use `needs-llvm-components` instead to ensure the appropriate
backend is available.
- `needs-profiler-runtime` — ignores the test if the profiler runtime was not
enabled for the target
(`build.profiler = true` in rustc's `bootstrap.toml`)
enabled for the target (`build.profiler = true` in rustc's `bootstrap.toml`)
- `needs-sanitizer-support` — ignores if the sanitizer support was not enabled
for the target (`sanitizers = true` in rustc's `bootstrap.toml`)
- `needs-sanitizer-{address,hwaddress,leak,memory,thread}` — ignores if the
@ -175,41 +172,36 @@ settings:
hardware-assisted AddressSanitizer, LeakSanitizer, MemorySanitizer or
ThreadSanitizer respectively)
- `needs-run-enabled` — ignores if it is a test that gets executed, and running
has been disabled. Running tests can be disabled with the `x test --run=never`
flag, or running on fuchsia.
has been disabled.
Running tests can be disabled with the `x test --run=never` flag, or running on fuchsia.
- `needs-unwind` — ignores if the target does not support unwinding
- `needs-rust-lld` — ignores if the rust lld support is not enabled (`rust.lld =
true` in `bootstrap.toml`)
- `needs-threads` — ignores if the target does not have threading support
- `needs-subprocess` — ignores if the target does not have subprocess support
- `needs-symlink` — ignores if the target does not support symlinks. This can be
the case on Windows if the developer did not enable privileged symlink
- `needs-symlink` — ignores if the target does not support symlinks.
This can be the case on Windows if the developer did not enable privileged symlink
permissions.
- `ignore-std-debug-assertions` — ignores if std was built with debug
assertions.
- `needs-std-debug-assertions` — ignores if std was not built with debug
assertions.
- `ignore-std-remap-debuginfo` — ignores if std was built with remapping of
it's sources.
- `needs-std-remap-debugino` — ignores if std was not built with remapping of
it's sources.
- `ignore-rustc-debug-assertions` — ignores if rustc was built with debug
assertions.
- `needs-rustc-debug-assertions` — ignores if rustc was not built with debug
assertions.
- `ignore-std-debug-assertions` — ignores if std was built with debug assertions.
- `needs-std-debug-assertions` — ignores if std was not built with debug assertions.
- `ignore-std-remap-debuginfo` — ignores if std was built with remapping of it's sources.
- `needs-std-remap-debugino` — ignores if std was not built with remapping of it's sources.
- `ignore-rustc-debug-assertions` — ignores if rustc was built with debug assertions.
- `needs-rustc-debug-assertions` — ignores if rustc was not built with debug assertions.
- `needs-target-has-atomic` — ignores if target does not have support for all
specified atomic widths, e.g. the test with `//@ needs-target-has-atomic: 8,
16, ptr` will only run if it supports the comma-separated list of atomic
widths.
16, ptr` will only run if it supports the comma-separated list of atomic widths.
- `needs-dynamic-linking` — ignores if target does not support dynamic linking
(which is orthogonal to it being unable to create `dylib` and `cdylib` crate types)
- `needs-crate-type` — ignores if target platform does not support one or more
of the comma-delimited list of specified crate types. For example,
of the comma-delimited list of specified crate types.
For example,
`//@ needs-crate-type: cdylib, proc-macro` will cause the test to be ignored
on `wasm32-unknown-unknown` target because the target does not support the
`proc-macro` crate type.
- `needs-target-std` — ignores if target platform does not have std support.
- `ignore-backends` — ignores the listed backends, separated by whitespace characters. Please note
- `ignore-backends` — ignores the listed backends, separated by whitespace characters.
Please note
that this directive can be overriden with the `--bypass-ignore-backends=[BACKEND]` command line
flag.
- `needs-backends` — only runs the test if current codegen backend is listed.
@ -220,29 +212,23 @@ The following directives will check LLVM support:
- `exact-llvm-major-version: 19` — ignores if the llvm major version does not
match the specified llvm major version.
- `min-llvm-version: 13.0` — ignored if the LLVM version is less than the given
value
- `min-llvm-version: 13.0` — ignored if the LLVM version is less than the given value
- `min-system-llvm-version: 12.0` — ignored if using a system LLVM and its
version is less than the given value
- `max-llvm-major-version: 19` — ignored if the LLVM major version is higher
than the given major version
- `ignore-llvm-version: 9.0` — ignores a specific LLVM version
- `ignore-llvm-version: 7.0 - 9.9.9` — ignores LLVM versions in a range
(inclusive)
- `needs-llvm-components: powerpc` — ignores if the specific LLVM component was
not built. Note: The test will fail on CI (when
`COMPILETEST_REQUIRE_ALL_LLVM_COMPONENTS` is set) if the component does not
exist.
- `ignore-llvm-version: 7.0 - 9.9.9` — ignores LLVM versions in a range (inclusive)
- `needs-llvm-components: powerpc` — ignores if the specific LLVM component was not built.
Note: The test will fail on CI (when
`COMPILETEST_REQUIRE_ALL_LLVM_COMPONENTS` is set) if the component does not exist.
- `needs-forced-clang-based-tests` — test is ignored unless the environment
variable `RUSTBUILD_FORCE_CLANG_BASED_TESTS` is set, which enables building
clang alongside LLVM
variable `RUSTBUILD_FORCE_CLANG_BASED_TESTS` is set, which enables building clang alongside LLVM
- This is only set in two CI jobs ([`x86_64-gnu-debug`] and
[`aarch64-gnu-debug`]), which only runs a
subset of `run-make` tests. Other tests with this directive will not
run at all, which is usually not what you want.
[`aarch64-gnu-debug`]), which only runs a subset of `run-make` tests.
Other tests with this directive will not run at all, which is usually not what you want.
See also [Debuginfo tests](compiletest.md#debuginfo-tests) for directives for
ignoring debuggers.
See also [Debuginfo tests](compiletest.md#debuginfo-tests) for directives for ignoring debuggers.
[remote testing]: running.md#running-tests-on-a-remote-machine
[compare modes]: ui.md#compare-modes
@ -311,7 +297,8 @@ Asked in
The test suites [`rustdoc-html`][rustdoc-html-tests], [`rustdoc-js`/`rustdoc-js-std`][rustdoc-js-tests]
and [`rustdoc-json`][rustdoc-json-tests] each feature an additional set of directives whose basic
syntax resembles the one of compiletest directives but which are ultimately read and checked by
separate tools. For more information, please read their respective chapters as linked above.
separate tools.
For more information, please read their respective chapters as linked above.
[rustdoc-html-tests]: ../rustdoc-internals/rustdoc-html-test-suite.md
[rustdoc-js-tests]: ../rustdoc-internals/search.html#testing-the-search-engine
@ -327,8 +314,7 @@ See [Pretty-printer](compiletest.md#pretty-printer-tests).
- [`revisions`](compiletest.md#revisions) — compile multiple times
-[`forbid-output`](compiletest.md#incremental-tests) — incremental cfail rejects
output pattern
- [`should-ice`](compiletest.md#incremental-tests) — incremental cfail should
ICE
- [`should-ice`](compiletest.md#incremental-tests) — incremental cfail should ICE
- [`reference`] — an annotation linking to a rule in the reference
- `disable-gdb-pretty-printers` — disable gdb pretty printers for debuginfo tests
@ -348,40 +334,37 @@ test suites that use those tools:
### Tidy specific directives
The following directives control how the [tidy script](../conventions.md#formatting)
verifies tests.
The following directives control how the [tidy script](../conventions.md#formatting) verifies tests.
- `ignore-tidy-target-specific-tests` disables checking that the appropriate
LLVM component is required (via a `needs-llvm-components` directive) when a
test is compiled for a specific target (via the `--target` flag in a
`compile-flag` directive).
test is compiled for a specific target (via the `--target` flag in a `compile-flag` directive).
- [`unused-revision-names`](compiletest.md#ignoring-unused-revision-names) -
suppress tidy checks for mentioning unknown revision names.
## Substitutions
Directive values support substituting a few variables which will be replaced
with their corresponding value. For example, if you need to pass a compiler flag
with their corresponding value.
For example, if you need to pass a compiler flag
with a path to a specific file, something like the following could work:
```rust,ignore
//@ compile-flags: --remap-path-prefix={{src-base}}=/the/src
```
Where the sentinel `{{src-base}}` will be replaced with the appropriate path
described below:
Where the sentinel `{{src-base}}` will be replaced with the appropriate path described below:
- `{{cwd}}`: The directory where compiletest is run from. This may not be the
root of the checkout, so you should avoid using it where possible.
- `{{cwd}}`: The directory where compiletest is run from.
This may not be the root of the checkout, so you should avoid using it where possible.
- Examples: `/path/to/rust`, `/path/to/build/root`
- `{{src-base}}`: The directory where the test is defined. This is equivalent to
`$DIR` for [output normalization].
- `{{src-base}}`: The directory where the test is defined.
This is equivalent to `$DIR` for [output normalization].
- Example: `/path/to/rust/tests/ui/error-codes`
- `{{build-base}}`: The base directory where the test's output goes. This is
equivalent to `$TEST_BUILD_DIR` for [output normalization].
- `{{build-base}}`: The base directory where the test's output goes.
This is equivalent to `$TEST_BUILD_DIR` for [output normalization].
- Example: `/path/to/rust/build/x86_64-unknown-linux-gnu/test/ui`
- `{{rust-src-base}}`: The sysroot directory where libstd/libcore/... are
located
- `{{rust-src-base}}`: The sysroot directory where libstd/libcore/... are located
- `{{sysroot-base}}`: Path of the sysroot directory used to build the test.
- Mainly intended for `ui-fulldeps` tests that run the compiler via API.
- `{{target-linker}}`: Linker that would be passed to `-Clinker` for this test,
@ -400,7 +383,8 @@ for an example of a test that uses this substitution.
## Adding a directive
One would add a new directive if there is a need to define some test property or
behavior on an individual, test-by-test basis. A directive property serves as
behavior on an individual, test-by-test basis.
A directive property serves as
the directive's backing store (holds the command's current value) at runtime.
To add a new directive property:
@ -420,19 +404,21 @@ declaration block is found in [`src/tools/compiletest/src/common.rs`]).
`TestProps`'s `load_from()` method will try passing the current line of text to
each parser, which, in turn typically checks to see if the line begins with a
particular commented (`//@`) directive such as `//@ must-compile-successfully`
or `//@ failure-status`. Whitespace after the comment marker is optional.
or `//@ failure-status`.
Whitespace after the comment marker is optional.
Parsers will override a given directive property's default value merely by being
specified in the test file as a directive or by having a parameter value
specified in the test file, depending on the directive.
Parsers defined in `impl Config` are typically named `parse_<directive-name>`
(note kebab-case `<directive-command>` transformed to snake-case
`<directive_command>`). `impl Config` also defines several 'low-level' parsers
(note kebab-case `<directive-command>` transformed to snake-case `<directive_command>`).
`impl Config` also defines several 'low-level' parsers
which make it simple to parse common patterns like simple presence or not
(`parse_name_directive()`), `directive:parameter(s)`
(`parse_name_value_directive()`), optional parsing only if a particular `cfg`
attribute is defined (`has_cfg_prefix()`) and many more. The low-level parsers
attribute is defined (`has_cfg_prefix()`) and many more.
The low-level parsers
are found near the end of the `impl Config` block; be sure to look through them
and their associated parsers immediately above to see how they are used to avoid
writing additional parsing code unnecessarily.
@ -483,15 +469,16 @@ As a concrete example, here is the implementation for the
### Implementing the behavior change
When a test invokes a particular directive, it is expected that some behavior
will change as a result. What behavior, obviously, will depend on the purpose of
the directive. In the case of `failure-status`, the behavior that changes is
will change as a result.
What behavior, obviously, will depend on the purpose of the directive.
In the case of `failure-status`, the behavior that changes is
that `compiletest` expects the failure code defined by the directive invoked in
the test, rather than the default value.
Although specific to `failure-status` (as every directive will have a different
implementation in order to invoke behavior change) perhaps it is helpful to see
the behavior change implementation of one case, simply as an example. To
implement `failure-status`, the `check_correct_failure_status()` function found
the behavior change implementation of one case, simply as an example.
To implement `failure-status`, the `check_correct_failure_status()` function found
in the `TestCx` implementation block, located in
[`src/tools/compiletest/src/runtest.rs`], was modified as per below:
@ -532,10 +519,10 @@ in the `TestCx` implementation block, located in
}
```
Note the use of `self.props.failure_status` to access the directive property. In
tests which do not specify the failure status directive,
`self.props.failure_status` will evaluate to the default value of 101 at the
time of this writing. But for a test which specifies a directive of, for
Note the use of `self.props.failure_status` to access the directive property.
In tests which do not specify the failure status directive,
`self.props.failure_status` will evaluate to the default value of 101 at the time of this writing.
But for a test which specifies a directive of, for
example, `//@ failure-status: 1`, `self.props.failure_status` will evaluate to
1, as `parse_failure_status()` will have overridden the `TestProps` default
value, for that test specifically.