Use dynamic dispatch for queries
This replaces most concrete query values `V` with `MaybeUninit<[u8; { size_of::<V>() }]>` reducing the code instantiated by queries. The compile time of `rustc_query_impl` is reduced by 27%. It is an alternative to https://github.com/rust-lang/rust/pull/107937 which uses unstable const generics while this uses a `EraseType` trait which maps query values to their erased variant.
This is achieved by introducing an `Erased` type which does sanity check with `cfg(debug_assertions)`. The query caches gets instantiated with these erased types leaving the code in `rustc_query_system` unaware of them. `rustc_query_system` is changed to use instances of `QueryConfig` so that `rustc_query_impl` can pass in `DynamicConfig` which holds a pointer to a virtual table.
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">1.7055s</td><td align="right">1.6949s</td><td align="right"> -0.62%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.2547s</td><td align="right">0.2528s</td><td align="right"> -0.73%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.9590s</td><td align="right">0.9553s</td><td align="right"> -0.39%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">1.5457s</td><td align="right">1.5440s</td><td align="right"> -0.11%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check</td><td align="right">5.9092s</td><td align="right">5.9009s</td><td align="right"> -0.14%</td></tr><tr><td>Total</td><td align="right">10.3741s</td><td align="right">10.3479s</td><td align="right"> -0.25%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9960s</td><td align="right"> -0.40%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:initial</td><td align="right">2.0605s</td><td align="right">2.0575s</td><td align="right"> -0.15%</td></tr><tr><td>🟣 <b>hyper</b>:check:initial</td><td align="right">0.3218s</td><td align="right">0.3216s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>regex</b>:check:initial</td><td align="right">1.1848s</td><td align="right">1.1839s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>syn</b>:check:initial</td><td align="right">1.9409s</td><td align="right">1.9376s</td><td align="right"> -0.17%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:initial</td><td align="right">7.3105s</td><td align="right">7.2928s</td><td align="right"> -0.24%</td></tr><tr><td>Total</td><td align="right">12.8185s</td><td align="right">12.7935s</td><td align="right"> -0.20%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9986s</td><td align="right"> -0.14%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4606s</td><td align="right">0.4617s</td><td align="right"> 0.24%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1335s</td><td align="right">0.1336s</td><td align="right"> 0.08%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3324s</td><td align="right">0.3346s</td><td align="right"> 0.65%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.6268s</td><td align="right">0.6307s</td><td align="right"> 0.64%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.8248s</td><td align="right">1.8508s</td><td align="right">💔 1.43%</td></tr><tr><td>Total</td><td align="right">3.3779s</td><td align="right">3.4113s</td><td align="right"> 0.99%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">1.0061s</td><td align="right"> 0.61%</td></tr></table>
It's based on https://github.com/rust-lang/rust/pull/108167.
r? `@cjgillot`
Generate shell completions for bootstrap with Clap
Now that #110693 has been merged, we can look at generating shell completions for x.py with `clap_complete`. Leaving this as draft for now as I'm not sure of the best way to integration the completion generator. Additionally, the generated completions for zsh are completely broken (will need to be resolved upstream, it doesn't seem to handle subcommands + global arguments well).
I don't have Fish installed and would be interested to know how well completions work there.
Alternative to #107827
Add a tidy check to find unexpected files in UI tests, and clean up the results
While looking at UI tests, I noticed several weird files that were not being tested, some from even pre-1.0. I added a tidy check that fails if any files not known to compiletest or not used in tests (via manual list) are present in the ui tests.
Unfortunately the root entry limit had to be raised by 1 to accommodate the stderr file for one of the tests.
r? `@fee1-dead`
bump windows crate 0.46 -> 0.48
This drops duped version of crate(0.46), reduces `rustc_driver.dll` ~800kb and reduces exported functions number from 26k to 22k.
Also while here, added `tidy-alphabetical` sorting to lists in tidy allowed lists.
tidy: remove ENTRY_LIMIT maximum checking and set it to 900
Removes checking of `ENTRY_LIMIT` towards an actually reached maximum, and sets it to 900.
The number 900 is safely below github's limit of 1000 entries for a directory.
PRs to move tests can still decrease the sizes of various directories,
but adjusting the limit won't be neccessary any more. In fact, such reduction PRs are a great idea so that no unrelated PR is hitting the limit: ideally there would always be a (manually maintained) safety margin between the actually reached maximum and `ENTRY_LIMIT`, for all directories.
In general, the limit is a bad tool to direct people to put tests into
fitting directories because when those are available, usually the limit
is not hit, while the limit is hit in directories that have a weak
substructure themselves. I got into this situation myself when writing #110694: tests/ui/parser is hitting the limit, but has few directories of its own.
Suggested by ```@petrochenkov``` in https://github.com/rust-lang/rust/pull/110694#discussion_r1177694339.
r? ```@petrochenkov```
The number 900 is safely below github's limit of 1000 entries for a directory.
PRs to move tests can still decrease the sizes of various directories,
but adjusting the limit won't be neccessary any more.
In general, the limit is a bad tool to direct people to put tests into
fitting directories because when those are available, usually the limit
is not hit, while the limit is hit in directories that have a weak
substructure themselves.
Support `x test --stage 1 ui-fulldeps`
`@Nilstrieb` had an excellent idea the other day: the same way that rustdoc is able to load `rustc_driver` from the sysroot, ui-fulldeps tests should also be able to load it from the sysroot. That allows us to run fulldeps tests with stage1, without having to fully rebuild the compiler twice. It does unfortunately have the downside that we're building the tests with the *bootstrap* compiler, not the in-tree sources, but since most of the fulldeps tests are for the *API* of the compiler, that seems ok.
I think it's possible to extend this to `run-make-fulldeps`, but I've run out of energy for tonight.
- Move `plugin` tests into a subdirectory.
Plugins are loaded at runtime with `dlopen` and so require the ABI of the running compile to match the ABI of the compiler linked with `rustc_driver`. As a result they can't be supported in stage 1 and have to use `// ignore-stage1`.
- Remove `ignore-stage1` from most non-plugin tests
- Ignore diagnostic tests in stage 1. Even though this requires a stage 2 build to load rustc_driver, it's primarily testing the error message that the *running* compiler emits when the diagnostic struct is malformed.
- Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That allows running `hash-stable-is-unstable` in stage1, since it now suggests adding `rustc_private` to enable loading the crates.
- Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests that act as custom drivers to load it at runtime.
- Pass `--sysroot stage0-sysroot` in compiletest so that we use the correct version of std.
- Move a few lint tests from ui-fulldeps to ui
These had an `aux-build:lint-group-plugin-test.rs` that they never actually loaded with `feature(plugin)` nor tested. I removed the unused aux-build and they pass fine with stage 1.
Fixes https://github.com/rust-lang/rust/issues/75905.
Nils had an excellent idea the other day: the same way that rustdoc is
able to load `rustc_driver` from the sysroot, ui-fulldeps tests should
also be able to load it from the sysroot. That allows us to run fulldeps
tests with stage1, without having to fully rebuild the compiler twice.
It does unfortunately have the downside that we're running the tests on
the *bootstrap* compiler, not the in-tree sources, but since most of the
fulldeps tests are for the *API* of the compiler, that seems ok.
I think it's possible to extend this to `run-make-fulldeps`, but I've
run out of energy for tonight.
- Move `plugin` tests into a subdirectory.
Plugins are loaded at runtime with `dlopen` and so require the ABI of
the running compile to match the ABI of the compiler linked with
`rustc_driver`. As a result they can't be supported in stage 1 and have
to use `// ignore-stage1`.
- Remove `ignore-stage1` from most non-plugin tests
- Ignore diagnostic tests in stage 1. Even though this requires a stage
2 build to load rustc_driver, it's primarily testing the error message
that the *running* compiler emits when the diagnostic struct is malformed.
- Pass `-Zforce-unstable-if-unmarked` in stage1, not just stage2. That
allows running `hash-stable-is-unstable` in stage1, since it now
suggests adding `rustc_private` to enable loading the crates.
- Add libLLVM.so to the stage0 target sysroot, to allow fulldeps tests
that act as custom drivers to load it at runtime.
- Pass `--sysroot stage0-sysroot` in compiletest so that we use the
correct version of std.
This also
* bumps cargo to the latest in rust-lang/cargo.
* adds 0BSD to allowed list of licenses
Co-authored-by: Scott Schafer <schaferjscott@gmail.com>
Co-authored-by: Eric Huss <eric@huss.org>
Do not attempt to commute comparison and cast to codegen discriminants
The general algorithm to compute a discriminant is:
```
relative_tag = tag - niche_start
is_niche = relative_tag <= (ule) relative_max
discr = if is_niche {
cast(relative_tag) + niche_variants.start()
} else {
untagged_variant
}
```
We have an optimization branch which attempts to merge the addition and the subtraction by commuting them with the cast. We currently get this optimization wrong.
This PR takes the easiest and safest way: remove the optimization, and let LLVM handle it. (Perf may not agree with that course of action 😅)
There may be a less invasive solution, but I don't have the necessary knowledge of LLVM semantics to find it. Cranelift has the same optimization, which should be handled similarly.
cc `@nikic` and `@bjorn3` if you have a better solution.
Fixes https://github.com/rust-lang/rust/issues/110128
When running `x.py test` on a downloaded source distribution (e.g.
https://static.rust-lang.org/dist/rustc-<version>-src.tar.gz), the
crates in the vendor directory contain a number of executable files that
cause the tidy test to fail with the following message:
tidy error: binary checked into source: <path>
I see 26 such errors with the 1.68.0 source distribution. A few of these
are .rs source files with incorrect executable permission, but most are
scripts that are correctly marked executable.
Ignore files in .gitignore in mir opt check
This caused `./x test tidy` to fail for me when Finder (macOS) added `.DS_Store` files. They are ignored by git, so tidy should ignore them, too.
Previously, it would walk each directory twice: once in the main `Walk`
iterator, and once to count the number of entries in the directory. Now
it only walks each directory once.
- Skip files in `skip` wherever possible to avoid reading their contents
- Don't look for `tidy-alphabetic-start` in tests. It's never currently used and slows the check down a lot.
- Add new `filter_not_rust` helper function
Some comments may be formed like:
// This function takes a tuple `(Vec<String>,
// Box<[u8]>)` and transforms it into `Vec<u8>`.
where the "back-ticked" section wraps around.
Therefore, we can't make a single-line based
lint.
We also cannot make the lint paragraph based,
as it would otherwise complain about inline
code blocks:
/// ```
/// use super::Foo;
///
/// fn main() { Foo::new(); }
/// ```
For the future, one could introduce some checks
to treat code blocks specially, but such code
would make the check even more complicated.
