rust/compiler/rustc_data_structures/src
bors b94bd12401 Auto merge of #142978 - Kobzol:query-hit, r=oli-obk
Add new self-profiling event to cheaply aggregate query cache hit counts

Self-profile can record various types of things, some of them are not enabled, like query cache hits. Rustc currently records cache hits as "instant" measureme events, which records the thread ID, current timestamp, and constructs an individual event for each such cache hit. This is incredibly expensive, in a small hello world benchmark that just depends on serde, it makes compilation with nightly go from ~3s (with `-Zself-profile`) to ~15s (with `-Zself-profile -Zself-profile-events=default,query-cache-hit`).

We'd like to add query cache hits to rustc-perf (https://github.com/rust-lang/rustc-perf/pull/2168), but there we only need the actualy cache hit counts, not the timestamp/thread ID metadata associated with it.

This PR adds a new `query-cache-hit-count` event. Instead of generating individual instant events, it simply aggregates cache hit counts per *query invocation* (so a combination of a query and its arguments, if I understand it correctly) using an atomic counter. At the end of the compilation session, these counts are then dumped to the self-profile log using integer events (in a similar fashion as how we record artifact sizes). I suppose that we could dedup the query invocations in rustc directly, but I don't think it's really required. In local experiments with the hello world + serde case, the query invocation records generated ~30 KiB more data in the self-profile, which was ~10% increase in this case.

With this PR, the overhead of `-Zself-profile` seems to be the same as before, at least on my machine, so I also enabled query cache hit counts by default when self profiling is enabled.

We should also modify `analyzeme`, specifically [this](https://github.com/rust-lang/measureme/blob/master/analyzeme/src/analysis.rs#L139), and make it load the integer events with query cache hit counts. I can do that as a follow-up, it's not required to be done in sync with this PR, and it doesn't require changes in rustc.

CC `@cjgillot`

r? `@oli-obk`
2025-07-02 11:41:14 +00:00
..
base_n Stabilize the size of incr comp object file names 2024-04-22 10:50:07 -04:00
binary_search_util Remove invariant comments 2023-11-05 17:35:37 -06:00
fingerprint remove redundant imports 2023-12-10 10:56:22 +08:00
flock bumpt compiler and tools to windows 0.59 2025-01-21 16:48:44 +03:00
graph Rename graph::implementation::Graph to LinkedGraph 2025-05-06 14:35:06 +10:00
intern remove redundant imports 2023-12-10 10:56:22 +08:00
obligation_forest update cfgs 2025-04-09 12:29:59 +01:00
owned_slice Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
profiling Don't use serde_json to serialize a simple JSON object 2023-04-16 15:00:06 +02:00
small_c_str Fix SmallCStr conversion from CStr 2024-02-14 18:40:53 -08:00
snapshot_map Reformat use declarations. 2024-07-29 08:26:52 +10:00
sorted_map Rollup merge of #136610 - Jarcho:range_idx, r=Noratrieb 2025-02-24 02:11:32 -05:00
sso Greatly simplify lifetime captures in edition 2024 2025-02-22 22:24:52 +00:00
stable_hasher rename BitSet to DenseBitSet 2025-01-11 11:34:01 +00:00
sync Rollup merge of #142868 - klensy:dc, r=oli-obk 2025-06-22 17:35:36 +02:00
tagged_ptr Add unreachable_pub to RUSTC_LINT_FLAGS for compiler/ crates. 2025-03-11 13:14:21 +11:00
thousands Overhaul the thousands module. 2025-06-12 15:26:06 +10:00
transitive_relation Add unit tests for minimal_scc_representative 2025-04-14 10:40:44 +00:00
vec_cache Simplify vec_cache::tests::slot_index_exhaustive by pulling out 0 case 2025-06-05 12:12:28 -07:00
aligned.rs Update cfg(bootstrap) 2025-07-01 10:55:49 -07:00
atomic_ref.rs mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
base_n.rs chore: Fix typos in 'compiler' (batch 1) 2024-09-02 07:42:38 +02:00
fingerprint.rs Move hashes from rustc_data_structure to rustc_hashes so they can be shared with rust-analyzer 2025-02-16 16:18:30 -05:00
flat_map_in_place.rs Fix UB in ThinVec::flat_map_in_place 2025-02-26 15:49:19 +00:00
flock.rs Update cfg(bootstrap) 2025-07-01 10:55:49 -07:00
frozen.rs Remove double spaces after dots in comments 2023-01-17 08:09:33 +00:00
fx.rs Invalidate all dereferences for non-local assignments 2025-04-02 19:58:35 +08:00
intern.rs Enforce T: Hash for Interned<...> 2025-02-17 21:35:52 -05:00
jobserver.rs Add a jobserver proxy to ensure at least one token is always held 2025-04-29 07:20:13 +02:00
lib.rs Update cfg(bootstrap) 2025-07-01 10:55:49 -07:00
marker.rs Update cfg(bootstrap) 2025-07-01 10:55:49 -07:00
memmap.rs Couple of changes to run rustc in miri 2025-02-05 13:52:22 +00:00
owned_slice.rs compiler/rustc_data_structures/src/sync.rs: delete Sync and Send 2025-02-11 09:15:54 +03:00
packed.rs MatchBranchSimplification: Consider empty-unreachable otherwise branch 2024-12-27 10:57:46 +00:00
profiling.rs Auto merge of #142978 - Kobzol:query-hit, r=oli-obk 2025-07-02 11:41:14 +00:00
sharded.rs Optimize hash map operations in the query system 2025-03-21 07:51:20 +01:00
small_c_str.rs Fix SmallCStr conversion from CStr 2024-02-14 18:40:53 -08:00
sorted_map.rs Use {Decodable,Encodable}_NoContext in type_ir 2025-03-15 06:34:36 +00:00
stable_hasher.rs Add TyCtx::env_var_os 2025-03-26 15:46:05 +01:00
stack.rs Stacker now handles miri using a noop impl itself 2025-02-21 13:54:45 +00:00
steal.rs update bootstrap configs 2024-10-15 20:30:23 -07:00
svh.rs Use {Decodable,Encodable}_NoContext in type_ir 2025-03-15 06:34:36 +00:00
sync.rs Another round of tidy / warning fixes 2025-06-11 16:56:01 -07:00
tagged_ptr.rs De-abstract tagged pointer abstraction 2025-01-12 14:56:10 +00:00
temp_dir.rs tempfile dep bump fallout 2025-05-15 19:55:43 +02:00
thinvec.rs Add ExtractIf for ThinVec 2024-10-12 15:17:03 +01:00
transitive_relation.rs Handle regions equivalent to 'static in non_local_bounds 2025-04-14 10:00:28 +00:00
unhash.rs Handle clippy cases of rustc::potential_query_instability lint 2024-10-05 07:34:14 +03:00
unord.rs Avoid a reverse map that is only used in diagnostics paths 2025-04-11 09:33:38 +00:00
vec_cache.rs SlotIndex::from_index: Factor out a constant for the first bucket size 2025-06-05 16:57:59 -07:00
work_queue.rs rename BitSet to DenseBitSet 2025-01-11 11:34:01 +00:00