rustdoc-search: add impl disambiguator to duplicate assoc items
Preview (to see the difference, click the link and pay attention to the specific function that comes up):
| Before | After |
|--|--|
| [`simd<i64>, simd<i64> -> simd<i64>`](https://doc.rust-lang.org/nightly/std/?search=simd%3Ci64%3E%2C%20simd%3Ci64%3E%20-%3E%20simd%3Ci64%3E) | [`simd<i64>, simd<i64> -> simd<i64>`](https://notriddle.com/rustdoc-demo-html-3/impl-disambiguate-search/std/index.html?search=simd%3Ci64%3E%2C%20simd%3Ci64%3E%20-%3E%20simd%3Ci64%3E) |
| [`cow, vec -> bool`](https://doc.rust-lang.org/nightly/std/?search=cow%2C%20vec%20-%3E%20bool) | [`cow, vec -> bool`](https://notriddle.com/rustdoc-demo-html-3/impl-disambiguate-search/std/index.html?search=cow%2C%20vec%20-%3E%20bool)
Helps with #90929
This changes the search results, specifically, when there's more than one impl with an associated item with the same name. For example, the search queries `simd<i8> -> simd<i8>` and `simd<i64> -> simd<i64>` don't link to the same function, but most of the functions have the same names.
This change should probably be FCP-ed, especially since it adds a new anchor link format for `main.js` to handle, so that URLs like `struct.Vec.html#impl-AsMut<[T]>-for-Vec<T,+A>/method.as_mut` redirect to `struct.Vec.html#method.as_mut-2`. It's a strange design, but there are a few reasons for it:
* I'd like to avoid making the HTML bigger. Obviously, fixing this bug is going to add at least a little more data to the search index, but adding more HTML penalises viewers for the benefit of searchers.
* Breaking `struct.Vec.html#method.len` would also be a disappointment.
On the other hand:
* The path-style anchors might be less prone to link rot than the numbered anchors. It's definitely less likely to have URLs that appear to "work", but silently point at the wrong thing.
* This commit arranges the path-style anchor to redirect to the numbered anchor. Nothing stops rustdoc from doing the opposite, making path-style anchors the default and redirecting the "legacy" numbered ones.
### The bug
On the "Before" links, this example search calls for `i64`:
But if I click any of the results, I get `f64` instead.
The PR fixes this problem by adding enough information to the search result `href` to disambiguate methods with different types but the same name.
More detailed description of the problem at:
https://github.com/rust-lang/rust/pull/109422#issuecomment-1491089293
> When a struct/enum/union has multiple impls with different type parameters, it can have multiple methods that have the same name, but which are on different impls. Besides Simd, [Any](https://doc.rust-lang.org/nightly/std/any/trait.Any.html?search=any%3A%3Adowncast) also demonstrates this pattern. It has three methods named `downcast`, on three different impls.
>
> When that happens, it presents a challenge in linking to the method. Normally we link like `#method.foo`. When there are multiple `foo`, we number them like `#method.foo`, `#method.foo-1`, `#method.foo-2`, etc.
>
> It also presents a challenge for our search code. Currently we store all the variants in the index, but don’t have any way to generate unambiguous URLs in the results page, or to distinguish them in the SERP.
>
> To fix this, we need three things:
>
> 1. A fragment format that fully specifies the impl type parameters when needed to disambiguate (`#impl-SimdOrd-for-Simd<i64,+LANES>/method.simd_max`)
> 2. A search index that stores methods with enough information to disambiguate the impl they were on.
> 3. A search results interface that can display multiple methods on the same type with the same name, when appropriate OR a disambiguation landing section on item pages?
>
> For reviewers: it can be hard to see the new fragment format in action since it immediately gets rewritten to the numbered form.
[rustdoc] Show enum discrimant if it is a C-like variant
Fixes https://github.com/rust-lang/rust/issues/101337.
We currently display values for associated constant items in traits:
And we also display constant values like [here](file:///home/imperio/rust/rust/build/x86_64-unknown-linux-gnu/doc/std/f32/consts/constant.E.html).
I think that for coherency, we should display values of C-like enum variants.
With this change, it looks like this:
As for the display of the constant value itself, I used what we already have to keep coherency.
We display the C-like variants value in the following scenario:
1. It is a C-like variant with a value set => all the time
2. It is a C-like variant without a value set: All other variants are C-like variants and at least one them has its value set.
Here is the result in code:
```rust
// Ax and Bx value will be displayed.
enum A {
Ax = 12,
Bx,
}
// Ax and Bx value will not be displayed
enum B {
Ax,
Bx,
}
// Bx value will not be displayed
enum C {
Ax(u32),
Bx,
}
// Bx value will not be displayed, Cx value will be displayed.
#[repr(u32)]
enum D {
Ax(u32),
Bx,
Cx = 12,
}
```
r? `@notriddle`
This commit makes three changes for consistency and readability:
- It shows the sibling navigation on module pages. It's weird
that it didn't work before, and is inconsistent with everything
else (even Crates have sibling navigation with other Crates).
- It hides the "In [parent]" header if it's the same as the
current crate, and if there's no other header between them.
We need to keep it on modules and types, since they have
their own header and data between them, and we don't want
to show siblings under a header implying that they're children.
- It adds a margin to deal with the headers butting directly into
the branding lockup.
This should result in a layout for the actual standard library,
when built on CI, that looks like this:
_____
/ \ std
| R | 1.74.0-nightly
\_____/
(203c57dbe 2023-09-17)
Having the whole version as one string caused it to flex wrap,
because the sidebar isn't wide enough to fit the whole thing.
This commit changes the layout to something a bit less "look at my logo!!!111"
gigantic, and makes it clearer where clicking the logo will actually take you.
It also means the crate name is persistently at the top of the sidebar, even
when in a sub-item page, and clicking that name takes you back to the root.
| | Short crate name | Long crate name |
|---------|------------------|-----------------|
| Root | ![short-root] | ![long-root]
| Subpage | ![short-subpage] | ![long-subpage]
[short-root]: https://github.com/rust-lang/rust/assets/1593513/fe2ce102-d4b8-44e6-9f7b-68636a907f56
[short-subpage]: https://github.com/rust-lang/rust/assets/1593513/29501663-56c0-4151-b7de-d2637e167125
[long-root]: https://github.com/rust-lang/rust/assets/1593513/f6a385c0-b4c5-4a9c-954b-21b38de4192f
[long-subpage]: https://github.com/rust-lang/rust/assets/1593513/97ec47b4-61bf-4ebe-b461-0d2187b8c6cahttps://notriddle.com/rustdoc-html-demo-4/logo-lockup/image/index.htmlhttps://notriddle.com/rustdoc-html-demo-4/logo-lockup/crossbeam_channel/index.htmlhttps://notriddle.com/rustdoc-html-demo-4/logo-lockup/adler/struct.Adler32.htmlhttps://notriddle.com/rustdoc-html-demo-4/logo-lockup/crossbeam_channel/struct.Sender.html
This improves visual information density (the construct with the logo and
crate name is *shorter* than the logo on its own, because it's not
square) and navigation clarity (we can now see what clicking the Rust logo
does, specifically).
Compare this with the layout at [Phoenix's Hexdocs] (which is what this
proposal is closely based on), the old proposal on [Internals Discourse]
(which always says "Rust standard library" in the sidebar, but doesn't do the
side-by-side layout).
[Phoenix's Hexdocs]: https://hexdocs.pm/phoenix/1.7.7/overview.html
[Internals Discourse]: https://internals.rust-lang.org/t/poc-of-a-new-design-for-the-generated-rustdoc/11018
In newer versions of rustdoc, the crate name and version are always shown in
the sidebar, even in subpages. Clicking the crate name does the same thing
clicking the logo always did: return you to the crate root.
While this actually takes up less screen real estate than the old layout on
desktop, it takes up more HTML. It's also a bit more visually complex.
I could do what the Internals POC did and keep the vertically stacked layout
all the time, instead of doing a horizontal stack where possible. It would
take up more screen real estate, though.
This design is lifted almost verbatim from Hexdocs. It seems to work for them.
[`opentelemetry_process_propagator`], for example, has a long application name.
[`opentelemetry_process_propagator`]: https://hexdocs.pm/opentelemetry_process_propagator/OpentelemetryProcessPropagator.html
Has anyone written the rationale on why the Rust logo shows up on projects that
aren't the standard library? If we turned it off on non-standard crates by
default, it would line wrap crate names a lot less often.
Or maybe we should encourage crate authors to include their own logo more
often? It certainly helps give people a better sense of "place."
I'm not sure of anything that directly follows up this one. Plenty of other
changes could be made to improve the layout, like
* coming up with a less cluttered way to do disclosure (there's a lot of `[-]`
on the page)
* doing a better job of separating lateral navigation (vec::Vec links to
vec::IntoIter) and the table of contents (vec::Vec links to vec::Vec::new)
* giving readers more control of how much rustdoc hows them, and giving doc
authors more control of how much it generates
* better search that reduces the need to browse
But those are mostly orthogonal, not future possibilities unlocked by this change.
rustdoc: fix & clean up handling of cross-crate higher-ranked parameters
Preparatory work for the refactoring planned in #113015 (for correctness & maintainability).
---
1. Render the higher-ranked parameters of cross-crate function pointer types **(*)**.
2. Replace occurrences of `collect_referenced_late_bound_regions()` (CRLBR) with `bound_vars()`.
The former is quite problematic and the use of the latter allows us to yank a lot of hacky code **(†)**
as you can tell from the diff! :)
3. Add support for cross-crate higher-ranked types (`#![feature(non_lifetime_binders)]`).
We were previously ICE'ing on them (see `inline_cross/non_lifetime_binders.rs`).
---
**(*)**: Extracted from test `inline_cross/fn-type.rs`:
```diff
- fn(_: &'z fn(_: &'b str), _: &'a ()) -> &'a ()
+ for<'z, 'a, '_unused> fn(_: &'z for<'b> fn(_: &'b str), _: &'a ()) -> &'a ()
```
**(†)**: It returns an `FxHashSet` which isn't *predictable* or *stable* wrt. source code (`.rmeta`) changes. To elaborate, the ordering of late-bound regions doesn't necessarily reflect the ordering found in the source code. It does seem to be stable across compilations but modifying the source code of the to-be-documented crates (like adding or renaming items) may result in a different order:
<details><summary>Example</summary>
Let's assume that we're documenting the cross-crate re-export of `produce` from the code below. On `master`, rustdoc would render the list of binders as `for<'x, 'y, 'z>`. However, once you add back the functions `a`–`l`, it would be rendered as `for<'z, 'y, 'x>` (reverse order)! Results may vary. `bound_vars()` fixes this as it returns them in source order.
```rs
// pub fn a() {}
// pub fn b() {}
// pub fn c() {}
// pub fn d() {}
// pub fn e() {}
// pub fn f() {}
// pub fn g() {}
// pub fn h() {}
// pub fn i() {}
// pub fn j() {}
// pub fn k() {}
// pub fn l() {}
pub fn produce() -> impl for<'x, 'y, 'z> Trait<'z, 'y, 'x> {}
pub trait Trait<'a, 'b, 'c> {}
impl Trait<'_, '_, '_> for () {}
```
</details>
Further, as the name suggests, CRLBR only collects *referenced* regions and thus we drop unused binders. `bound_vars()` contains unused binders on the other hand. Let's stay closer to the source where possible and keep unused binders.
Lastly, using `bound_vars()` allows us to get rid of
* the deduplication and alphabetical sorting hack in `simplify.rs`
* the weird field `bound_params` on `EqPredicate`
both of which were introduced by me in #102707 back when I didn't know better.
To illustrate, let's look at the cross-crate bound `T: for<'a, 'b> Trait<A<'a> = (), B<'b> = ()>`.
* With CRLBR + `EqPredicate.bound_params`, *before* bounds simplification we would have the bounds `T: Trait`, `for<'a> <T as Trait>::A<'a> == ()` and `for<'b> <T as Trait>::B<'b> == ()` which required us to merge `for<>`, `for<'a>` and `for<'b>` into `for<'a, 'b>` in a deterministic manner and without introducing duplicate binders.
* With `bound_vars()`, we now have the bounds `for<'a, b> T: Trait`, `<T as Trait>::A<'a> == ()` and `<T as Trait>::B<'b> == ()` before bound simplification similar to rustc itself. This obviously no longer requires any funny merging of `for<>`s. On top of that `for<'a, 'b>` is guaranteed to be in source order.
Helps with #90929
This changes the search results, specifically, when there's more than
one impl with an associated item with the same name. For example,
the search queries `simd<i8> -> simd<i8>` and `simd<i64> -> simd<i64>`
don't link to the same function, but most of the functions have the
same names.
This change should probably be FCP-ed, especially since it adds a new
anchor link format for `main.js` to handle, so that URLs like
`struct.Vec.html#impl-AsMut<[T]>-for-Vec<T,+A>/method.as_mut` redirect
to `struct.Vec.html#method.as_mut-2`. It's a strange design, but there
are a few reasons for it:
* I'd like to avoid making the HTML bigger. Obviously, fixing this bug
is going to add at least a little more data to the search index, but
adding more HTML penalises viewers for the benefit of searchers.
* Breaking `struct.Vec.html#method.len` would also be a disappointment.
On the other hand:
* The path-style anchors might be less prone to link rot than the numbered
anchors. It's definitely less likely to have URLs that appear to "work",
but silently point at the wrong thing.
* This commit arranges the path-style anchor to redirect to the numbered
anchor. Nothing stops rustdoc from doing the opposite, making path-style
anchors the default and redirecting the "legacy" numbered ones.
Custom code classes in docs warning
Fixes https://github.com/rust-lang/rust/issues/115938.
This PR does two things:
1. Unless the `custom_code_classes_in_docs` feature is enabled, it will use the old codeblock tag parser.
2. If there is a codeblock tag that starts with a `.`, it will emit a behaviour change warning.
Hopefully this is the last missing part for this feature until stabilization.
Follow-up of https://github.com/rust-lang/rust/pull/110800.
r? `@notriddle`
rustdoc-search: add support for type parameters
r? `@GuillaumeGomez`
## Preview
* https://notriddle.com/rustdoc-html-demo-4/advanced-search/rustdoc/read-documentation/search.html
* https://notriddle.com/rustdoc-html-demo-4/advanced-search/std/index.html?search=option%3Coption%3CT%3E%3E%20-%3E%20option%3CT%3E
* https://notriddle.com/rustdoc-html-demo-4/advanced-search/std/index.html?search=option%3CT%3E,%20E%20-%3E%20result%3CT,%20E%3E
* https://notriddle.com/rustdoc-html-demo-4/advanced-search/std/index.html?search=-%3E%20option%3CT%3E
## Description
When writing a type-driven search query in rustdoc, specifically one with more than one query element, non-existent types become generic parameters instead of auto-correcting (which is currently only done for single-element queries) or giving no result. You can also force a generic type parameter by writing `generic:T` (and can force it to not use a generic type parameter with something like `struct:T` or whatever, though if this happens it means the thing you're looking for doesn't exist and will give you no results).
There is no syntax provided for specifying type constraints for generic type parameters.
When you have a generic type parameter in a search query, it will only match up with generic type parameters in the actual function, not concrete types that match, not concrete types that implement a trait. It also strictly matches based on when they're the same or different, so `option<T>, option<U> -> option<U>` matches `Option::and`, but not `Option::or`. Similarly, `option<T>, option<T> -> option<T>` matches `Option::or`, but not `Option::and`.
## Motivation
This feature is motivated by the many "combinitor"-type functions found in generic libraries, such as Option, Future, Iterator, and Entry. These highly-generic functions have names that are almost completely arbitrary, and a type signature that tells you what it actually does.
This PR is a major step towards[^closure] being able to easily search for generic functions by their type signature instead of by name. Some examples of combinators that can be found using this PR (try them out in the preview):
* `option<option<T>> -> option<T>` returns Option::flatten
* `option<T> -> result<T>` returns Option::ok_or
* `option<result<T>> -> result<option<T>>` returns Option::transpose
* `entry<K, V>, FnOnce -> V` returns `Entry::or_insert_with` (and `or_insert_with_key`, since there's no way to specify the generics on FnOnce)
[^closure]:
For this feature to be as useful as it ought to be, you should be able to search for *trait-associated types* and *closures*. This PR does not implement either of these: they are **Future possibilities**.
Trait-associated types would allow queries like `option<T> -> iterator<item=T>` to return `Option::iter`. We should also allow `option<T> -> iterator<T>` to match the associated type version.
Closures would make a good way to query for things like `Option::map`. Closure support needs associated types to be represented in the search index, since `FnOnce() -> i32` desugars to `FnOnce<Output=i32, ()>`, so associated trait types should be implemented first. Also, we'd want to expose an easy way to query closures without specifying which of the three traits you want.
Accept additional user-defined syntax classes in fenced code blocks
Part of #79483.
This is a re-opening of https://github.com/rust-lang/rust/pull/79454 after a big update/cleanup. I also converted the syntax to pandoc as suggested by `@notriddle:` the idea is to be as compatible as possible with the existing instead of having our own syntax.
## Motivation
From the original issue: https://github.com/rust-lang/rust/issues/78917
> The technique used by `inline-c-rs` can be ported to other languages. It's just super fun to see C code inside Rust documentation that is also tested by `cargo doc`. I'm sure this technique can be used by other languages in the future.
Having custom CSS classes for syntax highlighting will allow tools like `highlight.js` to be used in order to provide highlighting for languages other than Rust while not increasing technical burden on rustdoc.
## What is the feature about?
In short, this PR changes two things, both related to codeblocks in doc comments in Rust documentation:
* Allow to disable generation of `language-*` CSS classes with the `custom` attribute.
* Add your own CSS classes to a code block so that you can use other tools to highlight them.
#### The `custom` attribute
Let's start with the new `custom` attribute: it will disable the generation of the `language-*` CSS class on the generated HTML code block. For example:
```rust
/// ```custom,c
/// int main(void) {
/// return 0;
/// }
/// ```
```
The generated HTML code block will not have `class="language-c"` because the `custom` attribute has been set. The `custom` attribute becomes especially useful with the other thing added by this feature: adding your own CSS classes.
#### Adding your own CSS classes
The second part of this feature is to allow users to add CSS classes themselves so that they can then add a JS library which will do it (like `highlight.js` or `prism.js`), allowing to support highlighting for other languages than Rust without increasing burden on rustdoc. To disable the automatic `language-*` CSS class generation, you need to use the `custom` attribute as well.
This allow users to write the following:
```rust
/// Some code block with `{class=language-c}` as the language string.
///
/// ```custom,{class=language-c}
/// int main(void) {
/// return 0;
/// }
/// ```
fn main() {}
```
This will notably produce the following HTML:
```html
<pre class="language-c">
int main(void) {
return 0;
}</pre>
```
Instead of:
```html
<pre class="rust rust-example-rendered">
<span class="ident">int</span> <span class="ident">main</span>(<span class="ident">void</span>) {
<span class="kw">return</span> <span class="number">0</span>;
}
</pre>
```
To be noted, we could have written `{.language-c}` to achieve the same result. `.` and `class=` have the same effect.
One last syntax point: content between parens (`(like this)`) is now considered as comment and is not taken into account at all.
In addition to this, I added an `unknown` field into `LangString` (the parsed code block "attribute") because of cases like this:
```rust
/// ```custom,class:language-c
/// main;
/// ```
pub fn foo() {}
```
Without this `unknown` field, it would generate in the DOM: `<pre class="language-class:language-c language-c">`, which is quite bad. So instead, it now stores all unknown tags into the `unknown` field and use the first one as "language". So in this case, since there is no unknown tag, it'll simply generate `<pre class="language-c">`. I added tests to cover this.
Finally, I added a parser for the codeblock attributes to make it much easier to maintain. It'll be pretty easy to extend.
As to why this syntax for adding attributes was picked: it's [Pandoc's syntax](https://pandoc.org/MANUAL.html#extension-fenced_code_attributes). Even if it seems clunkier in some cases, it's extensible, and most third-party Markdown renderers are smart enough to ignore Pandoc's brace-delimited attributes (from [this comment](https://github.com/rust-lang/rust/pull/110800#issuecomment-1522044456)).
## Raised concerns
#### It's not obvious when the `language-*` attribute generation will be added or not.
It is added by default. If you want to disable it, you will need to use the `custom` attribute.
#### Why not using HTML in markdown directly then?
Code examples in most languages are likely to contain `<`, `>`, `&` and `"` characters. These characters [require escaping](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/pre) when written inside the `<pre>` element. Using the \`\`\` code blocks allows rustdoc to take care of escaping, which means doc authors can paste code samples directly without manually converting them to HTML.
cc `@poliorcetics`
r? `@notriddle`
Merge settings.css into rustdoc.css
There aren't that many CSS rules in `settings.css`, however quite some code is needed around it, making it more costly than useful. I think it's better to merge into `rustdoc.css` to simplify this.
r? `@notriddle`
