Most of the files within the `dep_graph` module can be moved wholesale
into `rustc_middle`. But two of them (`mod.rs` and `dep_node.rs`) have
the same name as existing files in `rustc_middle`, so for those I just
copied the contents into the existing files.
The commit also moves `QueryContext` and `incremental_verify_ich*`
because they are tightly intertwined with the dep graph code. And a
couple of error structs moved as well.
This includes the types `QueryInfo`, `QueryJob`, `QueryJobId`,
`QueryWaiter`, `QueryLatch`, and `QueryLatchInfo`.
`CycleError` and `QueryStack*` had to come along too, due to type
interdependencies. The `QueryStack*` types are put into a new submodule
`rustc_middle::query::stack`.
Move more query system code
Towards the goal of eliminating `rustc_query_system`, this commit moves some code from `rustc_query_system` to `rustc_middle` and `rustc_query_impl`, and from `rustc_middle` to `rustc_query_impl`.
r? @Zalathar
Handle race when coloring nodes concurrently as both green and red
This fixes a race where a duplicate dep node gets written to the dep graph if a node was marked as green and promoted during execution, then marked as red after execution.
This can occur when a `no_hash` query A depends on a query B which cannot be forced so it was not colored when starting execution of query A. During the execution of query A it will execute query B and color it green. Before A finishes another thread tries to mark A green, this time succeeding as B is now green, and A gets promoted and written to metadata. Execution of A then finishes and because it's `no_hash` we assume the result changed and thus we color the node again, now as red and write it to metadata again. This doesn't happen with non-`no_hash` queries as they will be green if all their dependencies are green.
This changes the code coloring nodes red to also use `compare_exchange` to deal with this race ensuring that the coloring of nodes only happens once.
Fixesrust-lang/rust#150018Fixesrust-lang/rust#142778Fixesrust-lang/rust#141540
The latter is a new module.
As well as the code motion, some other changes were required.
- `QueryJobId` methods became free functions so they could move while
`QueryJobId` itself stayed put. This was so `QueryMap` and
`QueryJobInfo` could be moved.
- Some visibilities in `rustc_query_system` required changing.
- `collect_active_jobs_from_all_queries` is no longer required in `trait
QueryContext`.
Remove `QueryDispatcher`
`QueryDispatcher` is a trait that existed purely because `rustc_query_system` had code that didn't have access to `TyCtxt`. That is no longer the case, so the trait can be removed.
r? @Zalathar
It existed only to bridge the divide between `rustc_query_system` and
`rustc_query_impl`. But enough pieces have been moved from the former to
the latter that the trait is no longer needed. Less indirection and
abstraction makes the code easier to understand.
Streamline `rustc_span::HashStableContext`.
Currently this trait has five methods. But it only really needs three.
For example, currently stable hashing of spans is implemented in `rustc_span`, except a couple of sub-operations are delegated to `rustc_query_system`: `def_span` and `span_data_to_lines_and_cols`. These two delegated sub-operations can be reduced to a single delegated operation that does the full hash computation.
Likewise, `assert_default_hashing_controls` depends on two delegated sub-operations, `hashing_controls` and
`unstable_opts_incremental_ignore_spans`, and can be simplified.
I find the resulting code simpler and clearer -- when necessary, we do a whole operation in `rustc_query_system` instead of doing it partly in `rustc_span` and partly in `rustc_query_system`.
r? @cjgillot
Query keys must be stably hashable. Currently this requirement is
expressed as a where-clause on `impl QueryDispatcher for
SemiDynamicQueryDispatcher` and a where-clause on
`create_deferred_query_stack_frame`.
This commit removes those where-clause bounds and adds a single bound to
`QueryCache::Key`, which already has some other bounds. I.e. it
consolidates the bounds. It also gives them a name (`QueryCacheKey`) to
avoid repeating them. There is also a related `Key` trait in
`rustc_middle`; it should probably be merged with `QueryCacheKey` in the
future, but not today.
This cleanup helps with the next two commits, which do bigger
rearrangements, and where the where-clauses caused me some difficulties.
Start cutting down `rustc_query_system`
The query system is implemented in `rustc_query_system`, `rustc_middle`, and `rustc_query_impl`. `rustc_query_system` is hamstrung by not having access to `TyCtxt`, and there seems to be consensus to eliminate it. It's contents can be moved into the other two crates. Moving as much stuff as possible to `rustc_query_impl` is preferred, because `rustc_middle` is already so big.
This PR starts this process. It moves one small function to `rustc_middle` and a good chunk of code to `rustc_query_impl`.
Once `rustc_query_system` is gone (or at least shrunk down a lot more) some of the traits like `DepContext`, `QueryContext`, and `QueryDispatcher` will be removable.
r? @Zalathar
Currently this trait has five methods. But it only really needs three.
For example, currently stable hashing of spans is implemented in
`rustc_span`, except a couple of sub-operations are delegated to
`rustc_query_system`: `def_span` and `span_data_to_lines_and_cols`.
These two delegated sub-operations can be reduced to a single delegated
operation that does the full hash computation.
Likewise, `assert_default_hashing_controls` depends on two delegated
sub-operations, `hashing_controls` and
`unstable_opts_incremental_ignore_spans`, and can be simplified.
I find the resulting code simpler and clearer -- when necessary, we do a
whole operation in `rustc_query_system` instead of doing it partly in
`rustc_span` and partly in `rustc_query_system`.
The previous commit moved some code from `rustc_query_system`, which
doesn't have access to `TyCtxt` and `QueryCtxt`, to `rustc_query_impl`,
which does. We can now remove quite a bit of indirection.
- Three methods in `trait QueryContext` are no longer needed
(`next_job_id`, `current_query_job`, `start_query`). As a result,
`QueryCtxt`'s trait impls of these methods are changed to inherent
methods.
- `qcx: Q::Qcx` parameters are simplified to `qcx: QueryCtxt<'tcx>`.
- `*qcx.dep_context()` occurrences are simplified to `qcx.tcx`, and
things like `qcx.dep_context().profiler()` become `qcx.tcx.prof`.
- `DepGraphData<<Q::Qcx as HasDepContext>::Deps>` becomes
`DepGraphData<DepsType>`.
In short, various layers of indirection and abstraction are cut away.
The resulting code is simpler, more concrete, and easier to understand.
It's a good demonstration of the benefits of eliminating
`rustc_query_system`, and there will be more to come.
[Note: this commit conceptually moves 75% of a file to another location,
and leaves 25% of it behind. It's impossible to preserve all the git
history. To preserve git history of the moved 75%, in this commit we
rename the file and remove the 25% from it, leaving the code in an
incomplete (uncompilable) state. In the next commit we add back the
25% in the old location.]
We are in the process of eliminating `rustc_query_system`. Chunks of it
are unused by `rustc_middle`, and so can be moved into
`rustc_query_impl`. This commit does some of that.
Mostly it's just moving code from one file to a new file. There are a
couple of non-trivial changes.
- `QueryState` and `ActiveKeyStatus` must remain in `rustc_query_system`
because they are used by `rustc_middle`. But their inherent methods
are not used by `rustc_middle`. So these methods are moved and
converted to free functions.
- The visibility of some things must increase. This includes `DepGraphData`
and some of its methods, which are now used in `rustc_query_impl`.
This is a bit annoying but seems hard to avoid.
What little is left behind in
`compiler/rustc_query_system/src/query/plumbing.rs` will be able to
moved into `rustc_query_impl` or `rustc_middle` in the future.
Weaken `assert_dep_node_not_yet_allocated_in_current_session` for multiple threads
This changes `assert_dep_node_not_yet_allocated_in_current_session` to not panic if the node is already marked green. Another thread may manage to mark it green if there was uncolored node when we initially tried to mark it green.
Move `rustc_query_system::cache`.
It only defines two types, `Cache` and `WithDepNode`. Neither has anything much to do with queries -- they use `DepNodeIndex`, that's all.
This commit moves the module into `rustc_middle`, to where they are used. It also renames the extremely non-descriptive `Cache` as `WithDepNodeCache`.
r? @Zalathar
`HashStableContext` impls should be in `hcx.rs`, and `HashStable` impls
should be in `impls_syntax.rs`. This commit moves a few that are in the
wrong file.
It has a single impl, which does nothing, as you'd expect when hashing a
type called `HashIgnoredAttrId`!
So this commit just removes it, and `HashIgnoredAttrId::hash_stable`
ends up doing nothing (with a comment) instead of calling the trait
method to do nothing.
It has a single use. It doesn't need to be public. It doesn't use `self`
and so doesn't need to be in the trait. And `IGNORED_ATTRIBUTES` can be
moved within it.
Port depgraph testing attributes to parser
Tracking issue: rust-lang/rust#131229
Ports `#[rustc_clean]`, `#[rustc_if_this_changed]` and `#[rustc_then_this_would_need]` attributes.
Removes references to `rustc_dirty` as that attribute was folded into `rustc_clean` some time ago and rename some code accordingly.
r? JonathanBrouwer
Some `rustc_query_system` cleanups
Small improvements I found while looking closely at `rustc_query_system`. Best reviewed one commit at a time.
r? @cjgillot
Rename trait `DepNodeParams` to `DepNodeKey`
In query system plumbing, we usually refer to a query's explicit argument value as a “key”.
The first few commits do some preliminary cleanup that would conflict with the rename; the rename itself is in the final commit.
r? nnethercote (or compiler)
They are defined in `rustc_query_system` but used in `rustc_query_impl`.
This is very much *not* how things are supposed to be done; I suspect
someone got lazy and took a shortcut at some point.
This commit moves the errors into `rustc_query_impl`. This requires more
lines of code to give `rustc_query_impl` an errors module, but it's
worthwhile to do things in the normal way instead of a weird exceptional
way.
It's a tiny module with one trait and a default impl. It's not used in
`rustc_query_system`; all uses and non-default impls are in
`rustc_middle` and `rustc_query_impl`.
This commit moves it into `rustc_middle`, which makes things simpler
overall.
Use the query vtable in `query_feed` plumbing
The `query_feed` function needs to be able to do two important things with (erased) query values: hash them, and debug-print them.
Both of those are things that the query's vtable already knows how to do. So by passing in a vtable to `query_feed`, we can give it a nicer signature, avoid having to unerase values in the function itself, and clean up some caller-side code as well.
