This commit shrinks the size of the aforementioned table from
2,102 bytes to 1,197 bytes. This is achieved by an observation that
most u16 entries are common in its upper byte. Specifically:
- SINGLETONS now uses two tables, one for (upper byte, lower count)
and another for a series of lower bytes. For each upper byte given
number of lower bytes are read and compared.
- NORMAL now uses a variable length format for the count of "true"
codepoints and "false" codepoints (one byte with MSB unset, or
two big-endian bytes with the first MSB set).
The code size and relative performance roughly remains same as this
commit tries to optimize for both. The new table and algorithm has
been verified for the equivalence to older ones.
The problem occured due to lines like
```
3400;<CJK Ideograph Extension A, First>;Lo;0;L;;;;;N;;;;;
4DB5;<CJK Ideograph Extension A, Last>;Lo;0;L;;;;;N;;;;;
```
in `UnicodeData.txt`, which the script previously interpreted as two
characters, although it represents the whole range.
Fixes#34318.
Use the same procedure as Python to determine whether a character is
printable, described in [PEP 3138]. In particular, this means that the
following character classes are escaped:
- Cc (Other, Control)
- Cf (Other, Format)
- Cs (Other, Surrogate), even though they can't appear in Rust strings
- Co (Other, Private Use)
- Cn (Other, Not Assigned)
- Zl (Separator, Line)
- Zp (Separator, Paragraph)
- Zs (Separator, Space), except for the ASCII space `' '` (`0x20`)
This allows for user-friendly inspection of strings that are not
English (e.g. compare `"\u{e9}\u{e8}\u{ea}"` to `"éèê"`).
Fixes#34318.
[PEP 3138]: https://www.python.org/dev/peps/pep-3138/
