user0/rust
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Refinement of Providers into Providers and ExternProviders

Browse source
This commit is contained in:
Boxy 2025-12-12 17:45:22 +00:00 • committed by GitHub
commit f936b9f5c4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 110 additions and 32 deletions
Download patch file Download diff file Expand all files Collapse all files

142
src/doc/rustc-dev-guide/src/query.md
View file

@ -69,22 +69,24 @@ are cheaply cloneable; insert an `Rc` if necessary).
If, however, the query is *not* in the cache, then the compiler will
call the corresponding **provider** function. A provider is a function
implemented in a specific module and **manually registered** into the
[`Providers`][providers_struct] struct during compiler initialization.
The macro system generates the [`Providers`][providers_struct] struct,
which acts as a function table for all query implementations, where each
implemented in a specific module and **manually registered** into either
the [`Providers`][providers_struct] struct (for local crate queries) or
the [`ExternProviders`][extern_providers_struct] struct (for external crate queries)
during compiler initialization. The macro system generates both structs,
which act as function tables for all query implementations, where each
field is a function pointer to the actual provider.
**Note:** The `Providers` struct is generated by macros and acts as a function table for all query implementations.
It is **not** a Rust trait, but a plain struct with function pointer fields.
**Note:** Both the `Providers` and `ExternProviders` structs are generated by macros and act as function tables for all query implementations.
They are **not** Rust traits, but plain structs with function pointer fields.
**Providers are defined per-crate.** The compiler maintains,
internally, a table of providers for every crate, at least
conceptually. Right now, there are really two sets: the providers for
queries about the **local crate** (that is, the one being compiled)
and providers for queries about **external crates** (that is,
dependencies of the local crate). Note that what determines the crate
that a query is targeting is not the *kind* of query, but the *key*.
conceptually. There are two sets of providers:
- The `Providers` struct for queries about the **local crate** (that is, the one being compiled)
- The `ExternProviders` struct for queries about **external crates** (that is,
dependencies of the local crate)
Note that what determines the crate that a query is targeting is not the *kind* of query, but the *key*.
For example, when you invoke `tcx.type_of(def_id)`, that could be a
local query or an external query, depending on what crate the `def_id`
is referring to (see the [`self::keys::Key`][Key] trait for more
@ -117,31 +119,31 @@ they define both a `provide` and a `provide_extern` function, through
### How providers are set up
When the tcx is created, it is given the providers by its creator using
the [`Providers`][providers_struct] struct. This struct is generated by
the macros here, but it is basically a big list of function pointers:
[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html
When the tcx is created, it is given both the local and external providers by its creator using
the `Providers` struct from `rustc_middle::util`. This struct contains both the local and external providers:
```rust,ignore
struct Providers {
type_of: for<'tcx> fn(TyCtxt<'tcx>, DefId) -> Ty<'tcx>,
// ... one field for each query
pub struct Providers {
pub queries: crate::query::Providers, // Local crate providers
pub extern_queries: crate::query::ExternProviders, // External crate providers
pub hooks: crate::hooks::Providers,
}
```
Each of these provider structs is generated by the macros and contains function pointers for their respective queries.
#### How are providers registered?
The `Providers` struct is filled in during compiler initialization, mainly by the `rustc_driver` crate.
But the actual provider functions are implemented in various `rustc_*` crates (like `rustc_middle`, `rustc_hir_analysis`, etc).
The `util::Providers` struct is filled in during compiler initialization, by the `rustc_interface` crate from the `DEFAULT_QUERY_PROVIDERS` static.
The actual provider functions are defined across various `rustc_*` crates (like `rustc_middle`, `rustc_hir_analysis`, etc).
To register providers, each crate exposes a [`provide`][provide_fn] function that looks like this:
[provide_fn]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/hir/fn.provide.html
```rust,ignore
pub fn provide(providers: &mut Providers) {
*providers = Providers {
pub fn provide(providers: &mut query::Providers) {
*providers = query::Providers {
type_of,
// ... add more providers here
..*providers
@ -149,27 +151,104 @@ pub fn provide(providers: &mut Providers) {
}
```
- This function takes a mutable reference to the `Providers` struct and sets the fields to point to the correct provider functions.
- You can also assign fields individually, e.g. `providers.type_of = type_of;`.
Note that this function accepts `query::Providers` not `util::Providers`. It is exceedingly rare to need a `provide` function that doesn't just accept `query::Providers`. If more than the `queries` field of `util::Providers` is being updated then `util::Providers` can be accepted instead:
```rust,ignore
pub fn provide(providers: &mut rustc_middle::util::Providers) {
providers.queries.type_of = type_of;
// ... add more local providers here
providers.extern_queries.type_of = extern_type_of;
// ... add more external providers here
providers.hooks.some_hook = some_hook;
// ... add more hooks here
}
```
Note that `util::Providers` implements `DerefMut` to `query::Providers` so callers of the `provide` functions can pass in a `util::Providers` and it will just work for provider functions that accept `query::Providers` too
- This function takes a mutable reference to the `query::Providers` struct and sets the fields to point to the correct provider functions.
- You can also assign queries individually, e.g. `providers.type_of = type_of;`.
- You can assign fields individually for each provider type (local, external, and hooks).
#### Adding a new provider
Suppose you want to add a new query called `fubar`. You would:
Suppose you want to add a new query called `fubar`. This section focuses on wiring up the providers; for how to declare the query itself in the big `rustc_queries!` macro, see [Adding a new query](#adding-a-new-query) below.
1. Implement the provider function:
In practice you usually:
1. Decide which crate "owns" the query (for example `rustc_hir_analysis`, `rustc_mir_build`, or another `rustc_*` crate).
2. In that crate, look for an existing `provide` function:
```rust,ignore
fn fubar<'tcx>(tcx: TyCtxt<'tcx>, key: DefId) -> Fubar<'tcx> { ... }
pub fn provide(providers: &mut query::Providers) {
// existing assignments
}
```
2. Register it in the `provide` function:
If it exists, you will extend it to set the field for your new query. If the crate does not yet have a `provide` function, add one and make sure it is included in `DEFAULT_QUERY_PROVIDERS` in the `rustc_interface` crate so that it actually gets called during initialization (see the discussion above).
3. Implement the provider function itself:
```rust,ignore
pub fn provide(providers: &mut Providers) {
*providers = Providers {
fn fubar<'tcx>(tcx: TyCtxt<'tcx>, key: LocalDefId) -> Fubar<'tcx> { ... }
```
4. Register it in the crate's `provide` function:
```rust,ignore
pub fn provide(providers: &mut query::Providers) {
*providers = query::Providers {
fubar,
..*providers
};
}
```
### How queries interact with external crate metadata
When a query is made for an external crate (i.e., a dependency), the query system needs to load the information from that crate's metadata.
This is handled by the [`rustc_metadata` crate][rustc_metadata], which is responsible for decoding and providing the information stored in the `.rmeta` files.
The process works like this:
1. When a query is made, the query system first checks if the `DefId` refers to a local or external crate by checking if `def_id.krate == LOCAL_CRATE`.
This determines whether to use the local provider from [`Providers`][providers_struct] or the external provider from [`ExternProviders`][extern_providers_struct].
2. For external crates, the query system will look for a provider in the [`ExternProviders`][extern_providers_struct] struct.
The `rustc_metadata` crate registers these external providers through the `provide_extern` function in `rustc_metadata/src/rmeta/decoder/cstore_impl.rs`. Just like:
```rust
pub fn provide_extern(providers: &mut ExternProviders) {
providers.foo = |tcx, def_id| {
// Load and decode metadata for external crate
let cdata = CStore::from_tcx(tcx).get_crate_data(def_id.krate);
cdata.foo(def_id.index)
};
// Register other external providers...
}
```
3. The metadata is stored in a binary format in `.rmeta` files that contains pre-computed information about the external crate, such as types, function signatures, trait implementations, and other information needed by the compiler. When an external query is made, the `rustc_metadata` crate:
- Loads the `.rmeta` file for the external crate
- Decodes the metadata using the `Decodable` trait
- Returns the decoded information to the query system
This approach avoids recompiling external crates, allows for faster compilation of dependent crates, and enables incremental compilation to work across crate boundaries.
Here is a simplified example, when you call `tcx.type_of(def_id)` for a type defined in an external crate, the query system will:
1. Detect that the `def_id` refers to an external crate by checking `def_id.krate != LOCAL_CRATE`
2. Call the appropriate provider from `ExternProviders` which was registered by `rustc_metadata`
3. The provider will load and decode the type information from the external crate's metadata
4. Return the decoded type to the caller
This is why most `rustc_*` crates only need to provide local providers - the external providers are handled by the metadata system.
The only exception is when a crate needs to provide special handling for external queries, in which case it would implement both local and external providers.
When we define a new query that should work across crates, it does not automatically become cross-crate just because it is listed in `rustc_queries!`. You will typically need to:
- Add the query to `rustc_queries!` with appropriate modifiers (for example whether it is cached on disk).
- Implement a local provider in the owning crate and register it via that crate's `provide` function.
- Add an external provider in `rustc_metadata` via `provide_extern`, and ensure the query's result is encoded and decoded in the crate metadata.
An example of introducing such a cross-crate query can be found in commit [`996a185`](https://github.com/rust-lang/rust/commit/996a185) in the `rust-lang/rust` repository.
[rustc_metadata]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/index.html
[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html
[extern_providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.ExternProviders.html
---
## Adding a new query
@ -263,4 +342,3 @@ More discussion and issues:
[GitHub issue #42633]: https://github.com/rust-lang/rust/issues/42633
[Incremental Compilation Beta]: https://internals.rust-lang.org/t/incremental-compilation-beta/4721
[Incremental Compilation Announcement]: https://blog.rust-lang.org/2016/09/08/incremental.html