No clue how to fix the failures intypst-ide andtypst-docs.

I guess for the IDE,CompletionKind::Symbol can becomeSymbol(EcoString), but I think that field is only used in proprietary code, so someone from the Team would have to say whether that's a worthwhile change.

As for the docs, this is probably a way more substantial change, given how the current docs store the codepoint as au32, so I also don't think I can fix this by myself.

I'm REALLY unsure aboutlayout_symbol... I'm not really familiar with the layout code, and this change just made it work, but I think a single symbol should maybe be a single fragment? Either way, this probably interacts withcodex#20 and any typst-side integration of that (idk if there's already a PR for that), so it's probably best to wait on that before merging this.

We can now just callGlyphFragment::new with the whole string (#6336) (this function will error if after shaping the string we don't get a single glyph, i.e. if the string isn't a single grapheme cluster), and instead do something likeelem.text().chars().flat_map(|c| to_style(c, ...)) to style it.

Not sure whether we made a decision on whether we want to only accept a single grapheme cluster, but worth bringing up again.

Good point, let's bring that discussion to this thread.

I'm personally against it (as in, I think arbitrary strings should be allowed) and I don't really remember the arguments for why this would be necessary (So if anyone else has any, please introduce or repeat them here). The only one I can think of islayout_symbol requiring a single glyph, but requiring a single grapheme cluster doesn't fix that either since e.g. emoji sequences can fall back to multiple glyphs if the font doesn't support them. For example, the "people holding hands" emoji (🧑‍🤝‍🧑) I added incodex#104 is a single grapheme cluster that can fall back to "🧑🤝🧑".

Another technical detail: I don't know ifSymbolElem::func would be better by first checking if the string is a single codepoint and then matching on that. Maybe this is an irrelevant concern tho and then the decision could be made on principle (either by deciding that matching on strings makes it easier to add multi-char callables in the future, or by deciding that we'll never have multi-char callables anyway, so it'd be better to convert tochar first and then match on that).

I changed the symbol repr since I think the previous behavior was probably unintended (such assymbol("\"") repr'ing assymbol(""")), so it now usesrepr again to format the strings (both for the modifier sets and for the values).

Other than that, I made some minor changes to no longer use the variable namec for things that are now strings and to changeoption::IntoIter into the more idiomaticiter::Once.

It would indeed be nice if those cases wouldn't allocate.

After thinking about this a bit and trying some stuff, I found that this isn't really possible sinceSymbol::get returns&str, butchar is not UTF-8 encoded. We could usehttps://docs.rs/utf8char/0.2.0/utf8char/struct.Utf8Char.html or hand-roll something similar.¹ Do you think that'd be worth it?

With this, the tests are now passing, butlayout_symbol remains unable to handle multi-char values. As mentioned in#6489 (comment), I think it needs to be able to handle multiple glyphs regardless of whether we require symbols to be single grapheme clusters or not, but I don't know how to implement this.

And secondly, I should mention that the code as it is now permits empty strings as symbol values, which might cause some other issues. We probably just want to forbid those, right? (In which case I think onlySymbol::construct would need to be adjusted to reject them).

With this, the tests are now passing, butlayout_symbol remains unable to handle multi-char values. As mentioned in#6489 (comment), I think it needs to be able to handle multiple glyphs regardless of whether we require symbols to be single grapheme clusters or not, but I don't know how to implement this.

diff --git a/crates/typst-layout/src/math/fragment.rs b/crates/typst-layout/src/math/fragment.rsindex 758dd401..53aaaa4f 100644--- a/crates/typst-layout/src/math/fragment.rs+++ b/crates/typst-layout/src/math/fragment.rs@@ -15,6 +15,7 @@ use typst_library::text::{features, language, Font, Glyph, TextElem, TextItem}; use typst_syntax::Span; use typst_utils::{default_math_class, Get}; use unicode_math_class::MathClass;+use unicode_segmentation::UnicodeSegmentation;  use super::MathContext; use crate::inline::create_shape_plan;@@ -278,6 +279,8 @@ impl GlyphFragment {         text: &str,         span: Span,     ) -> SourceResult<GlyphFragment> {+        assert!(text.graphemes(true).count() == 1);+         let mut buffer = UnicodeBuffer::new();         buffer.push_str(text);         buffer.set_language(language(styles));@@ -300,6 +303,7 @@ impl GlyphFragment {         );          let buffer = rustybuzz::shape_with_plan(font.rusty(), &plan, buffer);+        // TODO: deal with multiple glyphs.         if buffer.len() != 1 {             bail!(span, "did not get a single glyph after shaping {}", text);         }diff --git a/crates/typst-layout/src/math/text.rs b/crates/typst-layout/src/math/text.rsindex e76fb2e7..4e819144 100644--- a/crates/typst-layout/src/math/text.rs+++ b/crates/typst-layout/src/math/text.rs@@ -129,44 +129,48 @@ pub fn layout_symbol(     ctx: &mut MathContext,     styles: StyleChain, ) -> SourceResult<()> {-    assert!(-        elem.text.len() <= 4 && elem.text.chars().count() == 1,-        "TODO: layout multi-char symbol"-    );-    let elem_char = elem-        .text-        .chars()-        .next()-        .expect("TODO: should an empty symbol value forbidden?");--    // Switch dotless char to normal when we have the dtls OpenType feature.-    // This should happen before the main styling pass.-    let dtls = style_dtls();-    let (unstyled_c, symbol_styles) = match try_dotless(elem_char) {-        Some(c) if has_dtls_feat(ctx.font) => (c, styles.chain(&dtls)),-        _ => (elem_char, styles),-    };-     let variant = styles.get(EquationElem::variant);     let bold = styles.get(EquationElem::bold);     let italic = styles.get(EquationElem::italic);+    let dtls = style_dtls();+    let has_dtls_feat = has_dtls_feat(ctx.font);+    for text in elem.text.graphemes(true) {+        // Switch dotless char to normal when we have the dtls OpenType feature.+        // This should happen before the main styling pass.+        let (text, symbol_styles) =+            if has_dtls_feat && text.chars().any(|c| try_dotless(c).is_some()) {+                (+                    text.chars()+                        .map(|c| try_dotless(c).unwrap_or(c))+                        .collect::<EcoString>(),+                    styles.chain(&dtls),+                )+            } else {+                (text.into(), styles)+            };++        let text: EcoString = text+            .chars()+            .flat_map(|c| {+                let style = MathStyle::select(c, variant, bold, italic);+                to_style(c, style)+            })+            .collect();-    let style = MathStyle::select(unstyled_c, variant, bold, italic);-    let text: EcoString = to_style(unstyled_c, style).collect();--    let fragment: MathFragment =-        match GlyphFragment::new(ctx.font, symbol_styles, &text, elem.span()) {-            Ok(mut glyph) => {-                adjust_glyph_layout(&mut glyph, ctx, styles);-                glyph.into()-            }-            Err(_) => {-                // Not in the math font, fallback to normal inline text layout.-                // TODO: Should replace this with proper fallback in [`GlyphFragment::new`].-                layout_inline_text(&text, elem.span(), ctx, styles)?.into()-            }-        };-    ctx.push(fragment);+        let fragment: MathFragment =+            match GlyphFragment::new(ctx.font, symbol_styles, &text, elem.span()) {+                Ok(mut glyph) => {+                    adjust_glyph_layout(&mut glyph, ctx, styles);+                    glyph.into()+                }+                Err(_) => {+                    // Not in the math font, fallback to normal inline text layout.+                    // TODO: Should replace this with proper fallback in [`GlyphFragment::new`].+                    layout_inline_text(&text, elem.span(), ctx, styles)?.into()+                }+            };+        ctx.push(fragment);+    }     Ok(()) }

(The code is a bit ugly, sorry about that)

the "people holding hands" emoji (🧑‍🤝‍🧑) I added incodex#104 is a single grapheme cluster that can fall back to "🧑🤝🧑".

Regarding this if the fallback happens during shaping in math (which it basically always will), the above code just lets it error out for the time being with the "didn't get a single glyph after shaping error", after which it'll fallback anyways to the inline text layout.

Another thing: I thought that the text0x0030 0xFE0E or0x0030 0xFE0F (can replace 0x0030 with any of the digits 0-9) could lead to problems as if it were styled, say bold, the codepoint would change from0x0030 and this would now be a non-standardised variation sequence. But this is a non-issue as the shaper will just ignore it.