fix: align styled-text style positions with grapheme clusters#1115
fix: align styled-text style positions with grapheme clusters#1115greymoth-jp wants to merge 2 commits into
Conversation
compileStyledText keyed charStyleMap by renderText.length (UTF-16 code units) while formatText walks the rendered text with runes() and looks up styles by grapheme index. For any emoji, ZWJ sequence or astral character (e.g. a CJK Extension B ideograph), the two indexings diverge, so every style after such a character was applied to the wrong grapheme or dropped. Walk grapheme clusters via runes() and normalize to NFC so the style keys line up with how formatText consumes them.
dragoncoder047
left a comment
There was a problem hiding this comment.
If you're going to use runes() like this, you should change it to have a parameter to just get the first rune and not all of them, otherwise the runtime will be
Also, using const { 0: grapheme } = ... instead of const [grapheme] = ... is faster and uses less memory since it doesn't create an iterator.
|
Good catch. I added a firstRune() helper next to runes() that pulls just the first grapheme from the Intl.Segmenter iterator and returns on the first segment, so it no longer splits the whole remaining string on every character and the walk is back to O(n). compileStyledText now calls firstRune(text) directly, so there is no array or destructuring at the call site either. It falls back to runesLegacy()[0] where Intl.Segmenter is unavailable. |
|
I just thought of another optimization: what is effectively being done is that compileStyledText is chopping runes off incrementally, and then formatText is calling runes() again, so it's duplicating the work. Maybe the |
compileStyledTextbuildscharStyleMapkeyed byrenderText.length, i.e. UTF-16 code units, while emitting the text one code unit at a time.formatTextthen splits the rendered text withrunes()and applies styles withcharStyleMap[cursor], wherecursoris a grapheme index.For plain ASCII those two indexings match, but they drift apart as soon as the text contains a character that is more than one code unit: an emoji, a ZWJ sequence, a skin-tone or flag sequence, or an astral-plane CJK ideograph (CJK Extension B, e.g. names written with 𠮷). After such a character every following style is shifted, so it lands on the wrong grapheme or is dropped.
A couple of concrete cases:
😀[c]x[/c]keys the style at2, butrunes("😀x")putsxat grapheme index1, so the style is lost.[c]😀b[/c]produces keys{0,1,2}for a stringrunes()sees as two graphemes.The fix makes
compileStyledTextwalk grapheme clusters with the samerunes()helperformatTextalready uses, keyingcharStyleMapby grapheme index. The input is normalized to NFC up front (whichrunes()does internally too) so the slice lengths stay consistent. Escapes (\x) now also consume a whole grapheme rather than a single code unit.Added
tests/auto/styledText.spec.tscovering ASCII (unchanged), an emoji, an astral CJK character, and a ZWJ family sequence.