syntax: record multibyte chars' positions absolutely, not relative to

file.

Previously multibyte UTF-8 chars were being recorded as byte offsets
from the start of the file, and then later compared against global byte
positions, resulting in the compiler possibly thinking it had a byte
position pointing inside a multibyte character, if there were multibyte
characters in any non-crate files. (Although, sometimes the byte offsets
line up just right to not ICE, but that was a coincidence.)

Fixes #11136.
Fixes #11178.
This commit is contained in:
Huon Wilson 2014-02-23 16:40:04 +11:00 committed by Alex Crichton
parent dad52cfcb5
commit ff79a4471c
3 changed files with 61 additions and 2 deletions

View file

@ -264,8 +264,7 @@ pub fn bump(rdr: &StringReader) {
}
if byte_offset_diff > 1 {
rdr.filemap.record_multibyte_char(
Pos::from_uint(current_byte_offset), byte_offset_diff);
rdr.filemap.record_multibyte_char(rdr.last_pos.get(), byte_offset_diff);
}
} else {
rdr.curr.set(None);