Display `\t` in diagnostics code as four spaces
Follow up to #44386 using the unicode variable width machinery from #45711 to replace tabs in the source code when displaying a diagnostic error with four spaces (instead of only one), while properly accounting for this when calculating underlines.
Partly addresses #44618.
The previous method ran into problems because ICH would treat Spans
as (file,line,col) but the cache contained byte offsets and its
possible for the latter to change while the former stayed stable.
Implement CompilerDesugaringKind enum
This is the first step outlined in #35946. I think that the variants of `CompilerDesugaringKind` should be changed, I didn't know what the official names for `...` and `<-` are.
I'm not to sure how tests for the compiler work, but I would imagine that tests should be added such that
`Symbol::intern(s) == CompilerDesugaringKind::from(s).as_symbol()` for valid `s`.
We already had a cache for file contents, but we read the source-file
before testing the cache, causing obvious slowness, so this just avoids
loading the source-file when the cache already has the contents.
* The lazy loading mechanism has been moved to a more appropriate place.
* Return values from the functions invoked there are properly used.
* Documentation has gotten some minor improvements.
* Possibly some larger restructuring will need to take place still.
We now fetch source lines from the `external_src` member as a secondary
fallback if no regular source is present, that is, if the file map
belongs to an external crate and the source has been fetched from disk.
They are now handled in their own member to prevent mutating access to
the `src` member. This way, we can safely load external sources, while
keeping the mutation of local source strings off-limits.
The rationale is that BOM stripping is needed for lazy source loading
for external crates, and duplication can be avoided by moving the
corresponding functionality to libsyntax_pos.
We can use these to perform lazy loading of source files belonging to
external crates. That way we will be able to show the source code of
external spans that have been translated.
* Point at where the token was expected instead of the last token
successfuly parsed.
* Only show `unexpected token` if the next char and the unexpected token
don't have the same span.
* Change some cfail and pfail tests to ui test.
* Don't show all possible tokens in span label if they are more than 6.
Historically this was done to accommodate bugs in lints, but there hasn't been a
bug in a lint since this feature was added which the warnings affected. Let's
completely purge warnings from all our stages by denying warnings in all stages.
This will also assist in tracking down `stage0` code to be removed whenever
we're updating the bootstrap compiler.
