Fix the start/end byte positions in the compiler JSON output
Track the changes made during normalization in the `SourceFile` and use this information to correct the `start_byte` and `end_byte` fields in the JSON output.
This should ensure the start/end byte fields can be used to index the original file, even if Rust normalized the source code for parsing purposes. Both CRLF to LF and BOM removal are handled with this one.
The rough plan was discussed with @matklad in rust-lang-nursery/rustfix#176 - although I ended up going with `u32` offset tracking so I wouldn't need to deal with `u32 + i32` arithmetics when applying the offset to the span byte positions.
Fixes#65029
Remove `InternedString`
This PR removes `InternedString` by converting all occurrences to `Symbol`. There are a handful of places that need to use the symbol chars instead of the symbol index, e.g. for stable sorting; local conversions `LocalInternedString` is used in those places.
r? @eddyb
Currently, `Symbol::Debug` and `Symbol::Display` produce the same
output; neither wraps the symbol in double quotes.
This commit changes `Symbol::Debug` so it wraps the symbol in quotes.
This change brings `Symbol`'s behaviour in line with `String` and
`InternedString`. The change requires a couple of trivial test output
adjustments.
Remove last uses of gensyms
Underscore bindings now use unique `SyntaxContext`s to avoid collisions. This was the last use of gensyms in the compiler, so this PR also removes them.
closes#49300
cc #60869
r? @petrochenkov
Remove custom `PartialEq` impls for `LocalInternedString`.
This is on-trend with the recent changes simplifying
`LocalInternedString` and reducing its use.
r? @petrochenkov
Split non-CAS atomic support off into target_has_atomic_load_store
This PR implements my proposed changes in https://github.com/rust-lang/rust/issues/32976#issuecomment-518542029 by removing `target_has_atomic = "cas"` and splitting `target_has_atomic` into two separate `cfg`s:
* `target_has_atomic = 8/16/32/64/128`: This indicates the largest width that the target can atomically CAS (which implies support for all atomic operations).
* ` target_has_atomic_load_store = 8/16/32/64/128`: This indicates the largest width that the target can support loading or storing atomically (but may not support CAS).
cc #32976
r? @alexcrichton
- The attribute is behind a feature gate.
- Error if both #[naked] and #[track_caller] are applied to the same function.
- Error if #[track_caller] is applied to a non-function item.
- Error if ABI is not "rust"
- Error if #[track_caller] is applied to a trait function.
Error codes and descriptions are pending.
Add support for `const unsafe? extern fn`
This works just as you might expect - an `const extern fn` is a `const fn` that is callable from foreign code.
Currently, panicking is not allowed in `const`s. When https://github.com/rust-lang/rfcs/pull/2345 (https://github.com/rust-lang/rust/issues/51999) is stabilized, then panicking in an `const extern fn` will produce a compile-time error when invoked at compile time, and an abort when invoked at runtime.
Since this is extending the language (we're allowing the `const` keyword in a new context), I believe that this will need an FCP. However, it's a very minor change, so I didn't think that filing an RFC was necessary.
This will allow libc (and other FFI crates) to make many functions `const`, without having to give up on making them `extern` as well.
Tracking issue: https://github.com/rust-lang/rust/issues/64926.
This works just as you might expect - an 'extern const fn' is a 'const
fn' that is callable from foreign code.
Currently, panicking is not allowed in consts. When RFC 2345 is
stabilized, then panicking in an 'extern const fn' will produce a
compile-time error when invoked at compile time, and an abort when
invoked at runtime.
Since this is extending the language (we're allowing the `const` keyword
in a new context), I believe that this will need an FCP. However, it's a
very minor change, so I didn't think that filing an RFC was necessary.
This will allow libc (and other FFI crates) to make many functions
`const`, without having to give up on making them `extern` as well.
A small amount of tidying-up factored out from PR #64648
As requested by @Mark-Simulacrum, I put this in a separate commit to make it easier to review. (As far as I can tell, no violations of the policy here, and they are simply in a separate PR because they're not directly related to the import of that PR.)
r? @Mark-Simulacrum
No StableHasherResult everywhere
This removes the generic parameter on `StableHasher`, instead moving it to the call to `finish`. This has the side-effect of making all `HashStable` impls nicer, since we no longer need the verbose `<W: StableHasherResult>` that previously existed -- often forcing line wrapping.
This is done for two reasons:
* we should avoid false "generic" dependency on the result of StableHasher
* we don't need to codegen two/three copies of all the HashStable impls when they're transitively used to produce a fingerprint, u64, or u128. I haven't measured, but this might actually make our artifacts somewhat smaller too.
* Easier to understand/read/write code -- the result of the stable hasher is irrelevant when writing a hash impl.
Use hygiene for AST passes
AST passes are now able to have resolve consider their expansions as if they were opaque macros defined either in some module in the current crate, or a fake empty module with `#[no_implicit_prelude]`.
* Add an ExpnKind for AST passes.
* Remove gensyms in AST passes.
* Remove gensyms in`#[test]`, `#[bench]` and `#[test_case]`.
* Allow opaque macros to define tests.
* Move tests for unit tests to their own directory.
* Remove `Ident::{gensym, is_gensymed}` - `Ident::gensym_if_underscore` still exists.
cc #60869, #61019
r? @petrochenkov
And remove the `unsafe` blocks, they're not necessary.
Also rewrite `InternedString::{with,with2}` to use the new functions.
Finally, add some comments about the speed of the
`as_str()`/`as_interned_str()` functions.
This allows lints and other diagnostics to refer to items
by a unique ID instead of relying on whacky path
resolution schemes that may break when items are
relocated.
