Rollup merge of #68845 - dwrensha:fix-68783, r=estebank

Browse source

stop using BytePos for computing spans in librustc_parse/parser/mod.rs

Computing spans using logic such as `self.token.span.lo() + BytePos(1)` can cause internal compiler errors like #68730 when non-ascii characters are given as input.

#68735 partially addressed this problem, but only for one case. Moreover, its usage of `next_point()` does not actually align with what `bump_with()` expects. For example, given the token `>>=`, we should pass the span consisting of the final two characters `>=`, but `next_point()` advances the span beyond the end of the `=`.

This pull request instead computes the start of the new span by doing `start_point(self.token.span).hi()`. This matches `self.token.span.lo() + BytePos(1)` in the common case where the characters are ascii, and it gracefully handles multibyte characters.

Fixes #68783.

...

This commit is contained in:

Dylan DPC 2020-02-06 15:37:47 +01:00 • committed by GitHub

parent 16e4e8f2de 9ac68e128b

commit ec24833325

No known key found for this signature in database

GPG key ID: 4AEE18F83AFDEB23

2 changed files with 22 additions and 17 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

BIN

src/test/ui/parser/issue-68730.stderr

View file

Binary file not shown.