rust/src/test/debuginfo
bors 1be5c8f909 Auto merge of #93432 - Kobzol:stable-hash-isize-hash-compression, r=the8472
Compress amount of hashed bytes for `isize` values in StableHasher

This is another attempt to land https://github.com/rust-lang/rust/pull/92103, this time hopefully with a correct implementation w.r.t. stable hashing guarantees. The previous PR was [reverted](https://github.com/rust-lang/rust/pull/93014) because it could produce the [same hash](https://github.com/rust-lang/rust/pull/92103#issuecomment-1014625442) for different values even in quite simple situations. I have since added a basic [test](https://github.com/rust-lang/rust/pull/93193) that should guard against that situation, I also added a new test in this PR, specialised for this optimization.

## Why this optimization helps
Since the original PR, I have tried to analyze why this optimization even helps (and why it especially helps for `clap`). I found that the vast majority of stable-hashing `i64` actually comes from hashing `isize` (which is converted to `i64` in the stable hasher). I only found a single place where is this datatype used directly in the compiler, and this place has also been showing up in traces that I used to find out when is `isize` being hashed. This place is `rustc_span::FileName::DocTest`, however, I suppose that isizes also come from other places, but they might not be so easy to find (there were some other entries in the trace). `clap` hashes about 8.5 million `isize`s, and all of them fit into a single byte, which is why this optimization has helped it [quite a lot](https://github.com/rust-lang/rust/pull/92103#issuecomment-1005711861).

Now, I'm not sure if special casing `isize` is the correct solution here, maybe something could be done with that `isize` inside `DocTest` or in other places, but that's for another discussion I suppose. In this PR, instead of hardcoding a special case inside `SipHasher128`, I instead put it into `StableHasher`, and only used it for `isize` (I tested that for `i64` it doesn't help, or at least not for `clap` and other few benchmarks that I was testing).

## New approach
Since the most common case is a single byte, I added a fast path for hashing `isize` values which positive value fits within a single byte, and a cold path for the rest of the values.

To avoid the previous correctness problem, we need to make sure that each unique `isize` value will produce a unique hash stream to the hasher. By hash stream I mean a sequence of bytes that will be hashed (a different sequence should produce a different hash, but that is of course not guaranteed).

We have to distinguish different values that produce the same bit pattern when we combine them. For example, if we just simply skipped the leading zero bytes for values that fit within a single byte, `(0xFF, 0xFFFFFFFFFFFFFFFF)` and `(0xFFFFFFFFFFFFFFFF, 0xFF)` would send the same hash stream to the hasher, which must not happen.

To avoid this situation, values `[0, 0xFE]` are hashed as a single byte. When we hash a larger (treating `isize` as `u64`) value, we first hash an additional byte `0xFF`. Since `0xFF` cannot occur when we apply the single byte optimization, we guarantee that the hash streams will be unique when hashing two values `(a, b)` and `(b, a)` if `a != b`:
1) When both `a` and `b` are within `[0, 0xFE]`, their hash streams will be different.
2) When neither `a` and `b` are within `[0, 0xFE]`, their hash streams will be different.
3) When `a` is within `[0, 0xFE]` and `b` isn't, when we hash `(a, b)`, the hash stream will definitely not begin with `0xFF`. When we hash `(b, a)`, the hash stream will definitely begin with `0xFF`. Therefore the hash streams will be different.

r? `@the8472`
2022-02-03 01:08:45 +00:00
..
auxiliary Moved issue tests to subdirs and normalised names. 2019-03-14 01:00:49 +00:00
associated-types.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
basic-types-globals-metadata.rs Remove licenses 2018-12-25 21:08:33 -07:00
basic-types-globals.rs Remove licenses 2018-12-25 21:08:33 -07:00
basic-types-metadata.rs Remove licenses 2018-12-25 21:08:33 -07:00
basic-types-mut-globals.rs Remove licenses 2018-12-25 21:08:33 -07:00
basic-types.rs Fix debuginfo tests for the latest version of the Windows SDK. 2021-09-10 21:46:40 -04:00
borrowed-basic.rs Remove licenses 2018-12-25 21:08:33 -07:00
borrowed-c-style-enum.rs Remove licenses 2018-12-25 21:08:33 -07:00
borrowed-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
borrowed-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
borrowed-tuple.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
borrowed-unique-basic.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
box.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
boxed-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
by-value-non-immediate-argument.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
by-value-self-argument-in-trait-impl.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
c-style-enum-in-composite.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
c-style-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
captured-fields-1.rs debuginfo: Make sure that type names for closure and generator environments are unique in debuginfo. 2022-02-01 10:39:40 +01:00
captured-fields-2.rs Store names of captured variables in optimized_mir 2021-07-09 23:09:48 +08:00
closure-in-generic-function.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
constant-debug-locs.rs Remove licenses 2018-12-25 21:08:33 -07:00
constant-in-match-pattern.rs Remove licenses 2018-12-25 21:08:33 -07:00
cross-crate-spans.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
cross-crate-type-uniquing.rs Remove licenses 2018-12-25 21:08:33 -07:00
destructured-fn-argument.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
destructured-for-loop-variable.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
destructured-local.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
drop-locations.rs Remove licenses 2018-12-25 21:08:33 -07:00
duration-type.rs Respond to review feedback 2021-07-09 18:29:08 -04:00
empty-string.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
enum-thinlto.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
evec-in-struct.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
extern-c-fn.rs Fix failed tests related to pointer printing when using GDB 10 2021-04-27 23:07:36 +08:00
fixed-sized-array.rs Add debug info tests for range, fix-sized array, and cell types. 2021-06-25 14:07:06 -07:00
function-arg-initialization.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
function-arguments.rs Remove licenses 2018-12-25 21:08:33 -07:00
function-call.rs tests: support -Zsymbol-mangling-version=v0 being the default. 2021-08-24 19:07:50 +03:00
function-names.rs Auto merge of #93432 - Kobzol:stable-hash-isize-hash-compression, r=the8472 2022-02-03 01:08:45 +00:00
function-prologue-stepping-regular.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
gdb-pretty-struct-and-enums.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
generator-locals.rs Add a resume type parameter to Generator 2020-02-02 13:20:57 +01:00
generator-objects.rs debuginfo: Make sure that type names for closure and generator environments are unique in debuginfo. 2022-02-01 10:39:40 +01:00
generic-enum-with-different-disr-sizes.rs Update the minimum external LLVM to 8 2020-04-14 12:44:41 -07:00
generic-function.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
generic-functions-nested.rs Remove licenses 2018-12-25 21:08:33 -07:00
generic-method-on-generic-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
generic-static-method-on-struct-and-enum.rs Remove licenses 2018-12-25 21:08:33 -07:00
generic-struct-style-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
generic-struct.rs [debuginfo] Adapt CDB tests after changes to whitespace usage in debuginfo type names. 2021-07-19 16:00:04 +02:00
generic-tuple-style-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
include_string.rs Remove licenses 2018-12-25 21:08:33 -07:00
issue-7712.rs Remove licenses 2018-12-25 21:08:33 -07:00
issue-12886.rs rustc_codegen_ssa: only "spill" SSA-like values to the stack for debuginfo. 2020-02-09 16:39:23 +02:00
issue-13213.rs Add basic CDB support to debuginfo compiletest s, to help catch *.natvis regressions, like those fixed in #60687. 2019-05-19 17:10:48 -07:00
issue-14411.rs Remove licenses 2018-12-25 21:08:33 -07:00
issue-22656.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
issue-57822.rs debuginfo: Make sure that type names for closure and generator environments are unique in debuginfo. 2022-02-01 10:39:40 +01:00
lexical-scope-in-for-loop.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-if.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-match.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-parameterless-closure.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-stack-closure.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-unconditional-loop.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-unique-closure.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-in-while.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scope-with-macro.rs Remove licenses 2018-12-25 21:08:33 -07:00
lexical-scopes-in-block-expression.rs Remove licenses 2018-12-25 21:08:33 -07:00
limited-debuginfo.rs Remove licenses 2018-12-25 21:08:33 -07:00
macro-stepping.inc Remove licenses 2018-12-25 21:08:33 -07:00
macro-stepping.rs Remove licenses 2018-12-25 21:08:33 -07:00
marker-types.rs Fix tests for i686 2021-07-14 16:50:11 -04:00
method-on-enum.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
method-on-generic-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
method-on-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
method-on-trait.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
method-on-tuple-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
msvc-pretty-enums.rs Fix debuginfo tests for the latest version of the Windows SDK. 2021-09-10 21:46:40 -04:00
msvc-scalarpair-params.rs Fix debuginfo for ScalarPair abi parameters 2021-09-13 10:51:47 -04:00
multi-byte-chars.rs Stablize non_ascii_idents feature. 2021-04-08 02:52:00 +08:00
multi-cgu.rs Remove licenses 2018-12-25 21:08:33 -07:00
multiple-functions-equal-var-names.rs Remove licenses 2018-12-25 21:08:33 -07:00
multiple-functions.rs Remove licenses 2018-12-25 21:08:33 -07:00
mutable-locs.rs Add natvis for cell types 2021-07-08 12:55:49 -04:00
mutex.rs Re-enable the src/test/debuginfo/mutex.rs test on Windows 2021-09-20 15:26:56 -04:00
name-shadowing-and-scope-nesting.rs Remove licenses 2018-12-25 21:08:33 -07:00
numeric-types.rs Add natvis for Atomic types 2021-07-08 12:55:49 -04:00
option-like-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
packed-struct-with-destructor.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
packed-struct.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
pretty-huge-vec.rs Support pretty printing slices using GDB 2021-07-03 23:42:07 +08:00
pretty-slices.rs Ignore Windows debugger pretty-printing tests 2021-07-08 01:04:59 +08:00
pretty-std-collections-hash.rs [debuginfo] Adapt CDB tests after changes to whitespace usage in debuginfo type names. 2021-07-19 16:00:04 +02:00
pretty-std-collections.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
pretty-std.rs Fix debuginfo tests for the latest version of the Windows SDK. 2021-09-10 21:46:40 -04:00
pretty-uninitialized-vec.rs Add missing : after min-gdb-version 2020-07-19 09:29:11 +00:00
range-types.rs Add natvis for Range types 2021-07-08 12:55:49 -04:00
rc_arc.rs tests: Ignore test/debuginfo/rc_arc.rs on windows-gnu 2021-11-28 23:09:33 +08:00
recursive-enum.rs Remove licenses 2018-12-25 21:08:33 -07:00
recursive-struct.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
result-types.rs [debuginfo] Adapt CDB tests after changes to whitespace usage in debuginfo type names. 2021-07-19 16:00:04 +02:00
rwlock-read.rs Add natvis for cell types 2021-07-08 12:55:49 -04:00
rwlock-write.rs Add debug info tests for range, fix-sized array, and cell types. 2021-06-25 14:07:06 -07:00
self-in-default-method.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
self-in-generic-default-method.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
shadowed-argument.rs Remove licenses 2018-12-25 21:08:33 -07:00
shadowed-variable.rs Remove licenses 2018-12-25 21:08:33 -07:00
should-fail.rs Add needs-run-enabled directive for should-fail tests 2021-04-30 04:12:37 +00:00
simd.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
simple-lexical-scope.rs Remove licenses 2018-12-25 21:08:33 -07:00
simple-struct.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
simple-tuple.rs Update cdb tests for expected output 2021-07-01 14:26:20 -04:00
static-method-on-struct-and-enum.rs Remove licenses 2018-12-25 21:08:33 -07:00
step-into-match.rs Add test for stepping though match expressions 2021-08-26 09:34:31 -04:00
struct-in-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
struct-in-struct.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
struct-namespace.rs Remove licenses 2018-12-25 21:08:33 -07:00
struct-style-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
struct-with-destructor.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
text-to-include-1.txt Re-bork whitespace for text file (fixup #23385) 2015-03-17 16:00:39 +05:30
text-to-include-2.txt Re-bork whitespace for text file (fixup #23385) 2015-03-17 16:00:39 +05:30
text-to-include-3.txt Re-bork whitespace for text file (fixup #23385) 2015-03-17 16:00:39 +05:30
thread.rs Improve debug symbol names to avoid ambiguity and work better with MSVC's debugger 2021-06-30 11:10:29 -07:00
trait-pointers.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
tuple-in-struct.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
tuple-in-tuple.rs Update cdb tests for expected output 2021-07-01 14:26:20 -04:00
tuple-struct.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
tuple-style-enum.rs Remove redundant ignore-tidy-linelength annotations 2021-04-03 22:30:20 +02:00
type-names.cdb.js Improve debug symbol names to avoid ambiguity and work better with MSVC's debugger 2021-06-30 11:10:29 -07:00
type-names.rs debuginfo: Make sure that type names for closure and generator environments are unique in debuginfo. 2022-02-01 10:39:40 +01:00
union-smoke.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00
unique-enum.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
unreachable-locals.rs Remove double trailing newlines 2019-04-22 16:57:01 +01:00
unsized.rs [debuginfo] Fix and unify handling of fat pointers in debuginfo: Fix some debuginfo tests for old GDB versions and 32-bit targets. 2022-01-27 11:45:45 +01:00
var-captured-in-nested-closure.rs debuginfo: Make sure that type names for closure and generator environments are unique in debuginfo. 2022-02-01 10:39:40 +01:00
var-captured-in-sendable-closure.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
var-captured-in-stack-closure.rs Remove box syntax from most places in src/test outside of the issues dir 2021-09-26 04:07:44 +02:00
vec-slices.rs [debuginfo] Fix and unify handling of fat pointers in debuginfo: Fix some debuginfo tests for old GDB versions and 32-bit targets. 2022-01-27 11:45:45 +01:00
vec.rs Implement new gdb/lldb pretty-printers 2020-06-09 16:13:11 +03:00