This adds a bit more data than "pure sharding" by
including information about which items have no description
at all. This way, it can sort the results, then truncate,
then finally download the description.
With the "e" bitmap: 2380KiB
Without the "e" bitmap: 2364KiB
The descriptions are, on almost all crates[^1], the majority
of the size of the search index, even though they aren't really
used for searching. This makes it relatively easy to separate
them into their own files.
This commit also bumps us to ES8. Out of the browsers we support,
all of them support async functions according to caniuse.
https://caniuse.com/async-functions
[^1]:
<https://microsoft.github.io/windows-docs-rs/>, a crate with
44MiB of pure names and no descriptions for them, is an outlier
and should not be counted.
Visually mark 👻hidden👻 items with document-hidden-items
Fixes#122485
This adds a 👻 in the item list (much like the 🔒 used for private items), and also shows `#[doc(hidden)]` in the code view, where `pub(crate)` etc gets shown for private items.
This does not do anything for enum variants, if people have ideas. I think we can just show the attribute.
hir: Remove `opt_local_def_id_to_hir_id` and `opt_hir_node_by_def_id`
Also replace a few `hir_node()` calls with `hir_node_by_def_id()`.
Follow up to https://github.com/rust-lang/rust/pull/120943.
rustdoc: add `--test-builder-wrapper` arg to support wrappers such as RUSTC_WRAPPER when building doctests
Currently, `rustdoc` builds test crates with `rustc` directly instead of using [`RUSTC_WRAPPER`](https://doc.rust-lang.org/cargo/reference/config.html#buildrustc-wrapper) (if any is set).
This causes build issues in build systems that use `cargo` but tweak linking flags by setting the `RUSTC_WRAPPER` environment variable.
This change is not meant to be final--it's only a minimal proof of concept.
Please advise on the best way to proceed.
Open questions:
- [x] Does supporting the `rustc` wrappers make sense?
- yes, `cargo-miri` for example needs a "hack" to workaround the issue
- [X] What environment variable(s) should be read for the rustc wrapper? Should `rustdoc` [use the same names as `cargo`](https://doc.rust-lang.org/cargo/reference/config.html#buildrustc-wrapper)?
- None, since `rustdoc` takes arguments
- [X] What name should be used for a `rustdoc` CLI option?
- `--test-builder-wrapper`
- [X] Should a separate workspace wrapper (like `RUSTC_WORKSPACE_WRAPPER`) be supported?
- `--test-builder-wrapper` can be passed multiple times to get multiple wrappers passed
- [X] How/where should this be documented? It's not obvious to all users that `cargo doc` actually causes `rustdoc` to compile tests with rust
- Added doc to `src/doc/rustdoc/src/command-line-arguments.md` per `@GuillaumeGomez`
Instead of executing the test builder directly, the test builder wrapper
will be called with test builder as the first argument and subsequent
arguments. This is similar to cargo's RUSTC_WRAPPER argument.
The `--test-builder-wrapper` argument can be passed multiple times to
allow "nesting" of wrappers.
Rollup of 10 pull requests
Successful merges:
- #117118 ([AIX] Remove AixLinker's debuginfo() implementation)
- #121650 (change std::process to drop supplementary groups based on CAP_SETGID)
- #121764 (Make incremental sessions identity no longer depend on the crate names provided by source code)
- #122212 (Copy byval argument to alloca if alignment is insufficient)
- #122322 (coverage: Initial support for branch coverage instrumentation)
- #122373 (Fix the conflict problem between the diagnostics fixes of lint `unnecessary_qualification` and `unused_imports`)
- #122479 (Implement `Duration::as_millis_{f64,f32}`)
- #122487 (Rename `StmtKind::Local` variant into `StmtKind::Let`)
- #122498 (Update version of cc crate)
- #122503 (Make `SubdiagMessageOp` well-formed)
r? `@ghost`
`@rustbot` modify labels: rollup
coverage: Initial support for branch coverage instrumentation
(This is a review-ready version of the changes that were drafted in #118305.)
This PR adds support for branch coverage instrumentation, gated behind the unstable flag value `-Zcoverage-options=branch`. (Coverage instrumentation must also be enabled with `-Cinstrument-coverage`.)
During THIR-to-MIR lowering (MIR building), if branch coverage is enabled, we collect additional information about branch conditions and their corresponding then/else blocks. We inject special marker statements into those blocks, so that the `InstrumentCoverage` MIR pass can reliably identify them even after the initially-built MIR has been simplified and renumbered.
The rest of the changes are mostly just plumbing needed to gather up the information that was collected during MIR building, and include it in the coverage metadata that we embed in the final binary.
Note that `llvm-cov show` doesn't print branch coverage information in its source views by default; that needs to be explicitly enabled with `--show-branches=count` or similar.
---
The current implementation doesn't have any support for instrumenting `if let` or let-chains. I think it's still useful without that, and adding it would be non-trivial, so I'm happy to leave that for future work.
more eagerly instantiate binders
The old solver sometimes incorrectly used `sub`, change it to explicitly instantiate binders and use `eq` instead. While doing so I also moved the instantiation before the normalize calls. This caused some observable changes, will explain these inline. This PR therefore requires a crater run and an FCP.
r? types
Various cleanups around the const eval query providers
r? `@RalfJung`
after this, working on running validation before interning starts with swapping the order of two lines of code
rustdoc-search: depth limit `T<U>` -> `U` unboxing
Profiler output:
https://notriddle.com/rustdoc-html-demo-9/search-unbox-limit/ (the only significant change is that one of the `rust` tests went from 378416ms to 16ms).
This is a performance enhancement aimed at a problem I found while using type-driven search on the Rust compiler. It is caused by [`Interner`], a trait with 41 associated types, many of which recurse back to `Self` again.
This caused search.js to struggle. It eventually terminates, after about 10 minutes of turning my PC into a space header, but it's doing `41!` unifications and that's too slow.
[`Interner`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/trait.Interner.html
fix unsoundness in Step::forward_unchecked for signed integers
Fixes#122420
```rust
pub fn foo(a: i8, b: u8) -> i8 {
unsafe { a.checked_add_unsigned(b).unwrap_unchecked() }
}
```
still compiles down to a single arithmetic instruction ([godbolt](https://rust.godbolt.org/z/qsd3xYWfE)).
But we may be losing some loop optimizations if llvm can no longer easily derive that it's a finite counted loop from the no-wrapping flags.
rustdoc-search: search types by higher-order functions
This feature extends rustdoc with syntax and search index information for searching function pointers and closures (Higher-Order Functions, or HOF). Part of https://github.com/rust-lang/rust/issues/60485
This PR adds two syntaxes: a high-level one for finding any kind of HOF, and a direct implementation of the parenthesized path syntax that Rust itself uses.
## Preview pages
| Query | Results |
|-------|---------|
| [`option<T>, (fnonce (T) -> bool) -> option<T>`][optionfilter] | `Option::filter` |
| [`option<T>, (T -> bool) -> option<T>`][optionfilter2] | `Option::filter` |
Updated chapter of the book: https://notriddle.com/rustdoc-html-demo-9/search-hof/rustdoc/read-documentation/search.html
[optionfilter]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(fnonce+(T)+->+bool)+->+option<T>&filter-crate=std
[optionfilter2]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=option<T>%2C+(T+->+bool)+->+option<T>&filter-crate=std
## Motivation
When type-based search was first landed, it was directly [described as incomplete][a comment].
[a comment]: https://github.com/rust-lang/rust/pull/23289#issuecomment-79437386
Filling out the missing functionality is going to mean adding support for more of Rust's [type expression] syntax, such as references, raw pointers, function pointers, and closures. This PR adds function pointers and closures.
[type expression]: https://doc.rust-lang.org/reference/types.html#type-expressions
There's been demand for something "like Hoogle, but for Rust" expressed a few times [1](https://www.reddit.com/r/rust/comments/y8sbid/is_there_a_website_like_haskells_hoogle_for_rust/) [2](https://users.rust-lang.org/t/rust-equivalent-of-haskells-hoogle/102280) [3](https://internals.rust-lang.org/t/std-library-inclusion-policy/6852/2) [4](https://discord.com/channels/442252698964721669/448238009733742612/1109502307495858216). Some of them just don't realize what functionality already exists ([`Duration -> u64`](https://doc.rust-lang.org/nightly/std/?search=duration%20-%3E%20u64) already works), but a lot of them specifically want to search for higher-order functions like option combinators.
## Guide-level explanation (from the Rustdoc book)
To search for a function that accepts a function as a parameter, like `Iterator::all`, wrap the nested signature in parenthesis, as in [`Iterator<T>, (T -> bool) -> bool`][iterator-all]. You can also search for a specific closure trait, such as `Iterator<T>, (FnMut(T) -> bool) -> bool`, but you need to know which one you want.
[iterator-all]: https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=Iterator<T>%2C+(T+->+bool)+->+bool&filter-crate=std
## Reference-level description (also from the Rustdoc book)
### Primitives with Special Syntax
<table>
<thead>
<tr>
<th>Shorthand</th>
<th>Explicit names</th>
</tr>
</thead>
<tbody>
<tr><td colspan="2">Before this PR</td></tr>
<tr>
<td><code>[]</code></td>
<td><code>primitive:slice</code> and/or <code>primitive:array</code></td>
</tr>
<tr>
<td><code>[T]</code></td>
<td><code>primitive:slice<T></code> and/or <code>primitive:array<T></code></td>
</tr>
<tr>
<td><code>!</code></td>
<td><code>primitive:never</code></td>
</tr>
<tr>
<td><code>()</code></td>
<td><code>primitive:unit</code> and/or <code>primitive:tuple</code></td>
</tr>
<tr>
<td><code>(T)</code></td>
<td><code>T</code></td>
</tr>
<tr>
<td><code>(T,)</code></td>
<td><code>primitive:tuple<T></code></td>
</tr>
<tr><td colspan="2">After this PR</td></tr>
<tr>
<td><code>(T, U -> V, W)</code></td>
<td><code>fn(T, U) -> (V, W)</code>, Fn, FnMut, and FnOnce</td>
</tr>
</tbody>
</table>
The `->` operator has lower precedence than comma. If it's not wrapped in brackets, it delimits the return value for the function being searched for. To search for functions that take functions as parameters, use parenthesis.
### Search query grammar
```ebnf
ident = *(ALPHA / DIGIT / "_")
path = ident *(DOUBLE-COLON ident) [BANG]
slice-like = OPEN-SQUARE-BRACKET [ nonempty-arg-list ] CLOSE-SQUARE-BRACKET
tuple-like = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN
arg = [type-filter *WS COLON *WS] (path [generics] / slice-like / tuple-like)
type-sep = COMMA/WS *(COMMA/WS)
nonempty-arg-list = *(type-sep) arg *(type-sep arg) *(type-sep) [ return-args ]
generic-arg-list = *(type-sep) arg [ EQUAL arg ] *(type-sep arg [ EQUAL arg ]) *(type-sep)
normal-generics = OPEN-ANGLE-BRACKET [ generic-arg-list ] *(type-sep)
CLOSE-ANGLE-BRACKET
fn-like-generics = OPEN-PAREN [ nonempty-arg-list ] CLOSE-PAREN [ RETURN-ARROW arg ]
generics = normal-generics / fn-like-generics
return-args = RETURN-ARROW *(type-sep) nonempty-arg-list
exact-search = [type-filter *WS COLON] [ RETURN-ARROW ] *WS QUOTE ident QUOTE [ generics ]
type-search = [ nonempty-arg-list ]
query = *WS (exact-search / type-search) *WS
; unchanged parts of the grammar, like the full list of type filters, are omitted
```
## Future direction
### The remaining type expression grammar
As described in https://github.com/rust-lang/rust/pull/118194, this is another step in the type expression grammar: BareFunction, and the function-like mode of TypePath, are now supported.
* RawPointerType and ReferenceType actually are a priority.
* ImplTraitType and TraitObjectType (and ImplTraitTypeOneBound and TraitObjectTypeOneBound) aren't as much of a priority, since they desugar pretty easily.
### Search subtyping and traits
This is the other major factor that makes it less useful than it should be.
* `iterator<result<t>> -> result<t>` doesn't find `Result::from_iter`. You have to search [`intoiterator<result<t>> -> result<t>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=intoiterator%3Cresult%3Ct%3E%3E%20-%3E%20result%3Ct%3E&filter-crate=std). Nobody's going to search for IntoIterator unless they basically already know about it and don't need the search engine anyway.
* Iterator combinators are usually structs that happen to implement Iterator, like `std::iter::Map`.
To solve these cases, it needs to look at trait implementations, knowing that Iterator is a "subtype of" IntoIterator, and Map is a "subtype of" Iterator, so `iterator -> result` is a subtype of `intoiterator -> result` and `iterator<t>, (t -> u) -> iterator<u>` is a subtype of [`iterator<t>, (t -> u) -> map<t -> u>`](https://notriddle.com/rustdoc-html-demo-9/search-hof/std/vec/struct.Vec.html?search=iterator%3Ct%3E%2C%20(t%20-%3E%20u)%20-%3E%20map%3Ct%20-%3E%20u%3E&filter-crate=std).
interpret: ensure that Place is never used for a different frame
We store the address where the stack frame stores its `locals`. The idea is that even if we pop and push, or switch to a different thread with a larger number of frames, then the `locals` address will most likely change so we'll notice that problem. This is made possible by some recent changes by `@WaffleLapkin,` where we no longer use `Place` across things that change the number of stack frames.
I made these debug assertions for now, just to make sure this can't cost us any perf.
The first commit is unrelated but it's a one-line comment change so it didn't warrant a separate PR...
r? `@oli-obk`
