Status#

• Discussion

Branches and Pull requests#

Issue #253 demonstrates a bug where using the bounding box rather thanthe advance width of text results in misaligned text. This is a minorpoint in the grand scheme of things, but it should be addressed aspart of this MEP.

Abstract#

By reorganizing how text is handled, this MEP aims to:

• improve support for Unicode and non-ltr languages

• improve text layout (especially multi-line text)

• allow support for more fonts, especially non-Apple-format TrueTypefonts and OpenType fonts.

• make the font configuration easier and more transparent

Detailed description#

Text layout

At present, matplotlib has two different ways to render text:"built-in" (based on FreeType and our own Python code), and "usetex"(based on calling out to a TeX installation). Adjunct to the"built-in" renderer there is also the Python-based "mathtext" systemfor rendering mathematical equations using a subset of the TeXlanguage without having a TeX installation available. Support forthese two engines in strewn about many source files, including everybackend, where one finds clauses like

ifrcParams['text.usetex']:# do one thing else: # do another

Adding a third text rendering approach (more on that later) wouldrequire editing all of these places as well, and therefore doesn'tscale.

Instead, this MEP proposes adding a concept of "text engines", wherethe user could select one of many different approaches for renderingtext. The implementations of each of these would be localized totheir own set of modules, and not have little pieces around the wholesource tree.

Why add more text rendering engines? The "built-in" text renderinghas a number of shortcomings.

• It only handles right-to-left languages, and doesn't handle manyspecial features of Unicode, such as combining diacriticals.

• The multiline support is imperfect and only supports manualline-breaking -- it cannot break up a paragraph into lines of acertain length.

• It also does not handle inline formatting changes in order tosupport something like Markdown, reStructuredText or HTML. (Thoughrich-text formatting is contemplated in this MEP, since we want tomake sure this design allows it, the specifics of a rich-textformatting implementation is outside of the scope of this MEP.)

Supporting these things is difficult, and is the "full-time job" of anumber of other projects:

• pango/harfbuzz

• QtTextLayout

• Microsoft DirectWrite

• Apple Core Text

Of the above options, it should be noted thatharfbuzz is designedfrom the start as a cross platform option with minimal dependencies,so therefore is a good candidate for a single option to support.

Additionally, for supporting rich text, we could consider usingWebKit, and possibly whether thanrepresents a good single cross-platform option. Again, however, richtext formatting is outside of the scope of this project.

Rather than trying to reinvent the wheel and add these features tomatplotlib's "built-in" text renderer, we should provide a way toleverage these projects to get more powerful text layout. The"built-in" renderer will still need to exist for reasons of ease ofinstallation, but its feature set will be more limited compared to theothers. [TODO: This MEP should clearly decide what those limitedfeatures are, and fix any bugs to bring the implementation into astate of working correctly in all cases that we want it to work. Iknow @leejjoon has some thoughts on this.]

Font selection

Going from an abstract description of a font to a file on disk is thetask of the font selection algorithm -- it turns out to be much morecomplicated than it seems at first.

The "built-in" and "usetex" renderers have very different ways ofhandling font selection, given their different technologies. TeXrequires the installation of TeX-specific font packages, for example,and cannot use TrueType fonts directly. Unfortunately, despite thedifferent semantics for font selection, the same set of fontproperties are used for each. This is true of both theFontProperties class and the font-relatedrcParams (whichbasically share the same code underneath). Instead, we should definea core set of font selection parameters that will work across all textengines, and have engine-specific configuration to allow the user todo engine-specific things when required. For example, it is possibleto directly select a font by name in the "built-in" usingrcParams["font.family"] (default:['sans-serif']), but the same is not possible with "usetex". It may bepossible to make it easier to use TrueType fonts by using XeTeX, butusers will still want to use the traditional metafonts through TeXfont packages. So the issue still stands that different text engineswill need engine-specific configuration, and it should be more obviousto the user which configuration will work across text engines andwhich are engine-specific.

Note that even excluding "usetex", there are different ways to findfonts. The default is to use the font list cache infont_managerwhich matches fonts using our own algorithm based on theCSS fontmatching algorithm.It doesn't always do the same thing as the native font selectionalgorithms on Linux (fontconfig), Mac andWindows, and it doesn't always find all of the fonts on the systemthat the OS would normally pick up. However, it is cross-platform,and always finds the fonts that ship with matplotlib. The Cairo andMacOSX backends (and presumably a future HTML5-based backend)currently bypass this mechanism and use the OS-native ones. The sameis true when not embedding fonts in SVG, PS or PDF files and openingthem in a third-party viewer. A downside there is that (at least withCairo, need to confirm with MacOSX) they don't always find the fontswe ship with matplotlib. (It may be possible to add the fonts totheir search path, though, or we may need to find a way to install ourfonts to a location the OS expects to find them).

There are also special modes in the PS and PDF to only use the corefonts that are always available to those formats. There, the fontlookup mechanism must only match against those fonts. It is unclearwhether the OS-native font lookup systems can handle this case.

There is also experimental support for usingfontconfig for fontselection in matplotlib, turned off by default. fontconfig is thenative font selection algorithm on Linux, but is also cross platformand works well on the other platforms (though obviously is anadditional dependency there).

Many of the text layout libraries proposed above (pango, QtTextLayout,DirectWrite and CoreText etc.) insist on using the font selectionlibrary from their own ecosystem.

All of the above seems to suggest that we should move away from ourself-written font selection algorithm and use the native APIs wherepossible. That's what Cairo and MacOSX backends already want to use,and it will be a requirement of any complex text layout library. OnLinux, we already have the bones of afontconfig implementation(which could also be accessed through pango). On Windows and Mac wemay need to write custom wrappers. The nice thing is that the API forfont lookup is relatively small, and essentially consist of "given adictionary of font properties, give me a matching font file".

Font subsetting

Font subsetting is currently handled using ttconv. ttconv was astandalone commandline utility for converting TrueType fonts tosubsetted Type 3 fonts (among other features) written in 1995, whichmatplotlib (well, I) forked in order to make it work as a library. Itonly handles Apple-style TrueType fonts, not ones with the Microsoft(or other vendor) encodings. It doesn't handle OpenType fonts at all.This means that even though the STIX fonts come as .otf files, we haveto convert them to .ttf files to ship them with matplotlib. The Linuxpackagers hate this -- they'd rather just depend on the upstream STIXfonts. ttconv has also been shown to have a few bugs that have beendifficult to fix over time.

Instead, we should be able to use FreeType to get the font outlinesand write our own code (probably in Python) to output subsetted fonts(Type 3 on PS and PDF and paths on SVG). Freetype, as a popular andwell-maintained project, handles a wide variety of fonts in the wild.This would remove a lot of custom C code, and remove some codeduplication between backends.

Note that subsetting fonts this way, while the easiest route, doeslose the hinting in the font, so we will need to continue, as we donow, provide a way to embed the entire font in the file wherepossible.

Alternative font subsetting options include using the subsettingbuilt-in to Cairo (not clear if it can be used without the rest ofCairo), or usingfontforge (which is a heavy and not terriblycross-platform dependency).

Freetype wrappers

Our FreeType wrapper could really use a reworking. It defines its ownimage buffer class (when a Numpy array would be easier). WhileFreeType can handle a huge diversity of font files, there arelimitations to our wrapper that make it much harder to supportnon-Apple-vendor TrueType files, and certain features of OpenTypefiles. (See #2088 for a terrible result of this, just to support thefonts that ship with Windows 7 and 8). I think a fresh rewrite ofthis wrapper would go a long way.

Text anchoring and alignment and rotation

The handling of baselines was changed in 1.3.0 such that the backendsare now given the location of the baseline of the text, not the bottomof the text. This is probably the correct behavior, and the MEPrefactoring should also follow this convention.

In order to support alignment on multi-line text, it should be theresponsibility of the (proposed) text engine to handle text alignment.For a given chunk of text, each engine calculates a bounding box forthat text and the offset of the anchor point within that box.Therefore, if the va of a block was "top", the anchor point would beat the top of the box.

Rotating of text should always be around the anchor point. I'm notsure that lines up with current behavior in matplotlib, but it seemslike the sanest/least surprising choice. [This could be revisitedonce we have something working]. Rotation of text should not behandled by the text engine -- that should be handled by a layerbetween the text engine and the rendering backend so it can be handledin a uniform way. [I don't see any advantage to rotation beinghandled by the text engines individually...]

There are other problems with text alignment and anchoring that shouldbe resolved as part of this work. [TODO: enumerate these].

Other minor problems to fix

The mathtext code has backend-specific code -- it should insteadprovide its output as just another text engine. However, it's stilldesirable to have mathtext layout inserted as part of a larger layoutperformed by another text engine, so it should be possible to do this.It's an open question whether embedding the text layout of anarbitrary text engine in another should be possible.

The text mode is currently set by a global rcParam ("text.usetex") soit's either all on or all off. We should continue to have a globalrcParam to choose the text engine ("text.layout_engine"), but itshould under the hood be an overridable property on theText object,so the same figure can combine the results of multiple text layoutengines if necessary.

Implementation#

A concept of a "text engine" will be introduced. Each text enginewill implement a number of abstract classes. TheTextFont interfacewill represent text for a given set of font properties. It isn'tnecessarily limited to a single font file -- if the layout enginesupports rich text, it may handle a number of font files in a family.Given aTextFont instance, the user can get aTextLayout instance,which represents the layout for a given string of text in a givenfont. From aTextLayout, an iterator overTextSpans is returnedso the engine can output raw editable text using as few spans aspossible. If the engine would rather get individual characters, theycan be obtained from theTextSpan instance:

classTextFont(TextFontBase):def__init__(self,font_properties):"""        Create a new object for rendering text using the given font properties.        """passdefget_layout(self,s,ha,va):"""        Get the TextLayout for the given string in the given font and        the horizontal (left, center, right) and verticalalignment (top,        center, baseline, bottom)        """passclassTextLayout(TextLayoutBase):defget_metrics(self):"""        Return the bounding box of the layout, anchored at (0, 0).        """passdefget_spans(self):"""        Returns an iterator over the spans of different in the layout.        This is useful for backends that want to editable raw text as        individual lines.  For rich text where the font may change,        each span of different font type will have its own span.        """passdefget_image(self):"""        Returns a rasterized image of the text.  Useful for raster backends,        like Agg.        In all likelihood, this will be overridden in the backend, as it can        be created from get_layout(), but certain backends may want to        override it if their library provides it (as freetype does).        """passdefget_rectangles(self):"""        Returns an iterator over the filled black rectangles in the layout.        Used by TeX and mathtext for drawing, for example, fraction lines.        """passdefget_path(self):"""        Returns a single Path object of the entire laid out text.        [Not strictly necessary, but might be useful for textpath        functionality]        """passclassTextSpan(TextSpanBase):x,y# Position of the span -- relative to the text layout as a whole# where (0, 0) is the anchor.  y is the baseline of the span.fontfile# The font file to use for the spantext# The text content of the spandefget_path(self):pass# See TextLayout.get_pathdefget_chars(self):"""        Returns an iterator over the characters in the span.        """passclassTextChar(TextCharBase):x,y# Position of the character -- relative to the text layout as# a whole, where (0, 0) is the anchor.  y is in the baseline# of the character.codepoint# The unicode code point of the character -- only for informational# purposes, since the mapping of codepoint to glyph_id may have been# handled in a complex way by the layout engine.  This is an int# to avoid problems on narrow Unicode builds.glyph_id# The index of the glyph within the fontfontfile# The font file to use for the chardefget_path(self):"""        Get the path for the character.        """pass

Graphic backends that want to output subset of fonts would likelybuild up a file-global dictionary of characters where the keys are(fontname, glyph_id) and the values are the paths so that only onecopy of the path for each character will be stored in the file.

Special casing: The "usetex" functionality currently is able to getPostscript directly from TeX to insert directly in a Postscript file,but for other backends, parses a DVI file and generates something moreabstract. For a case like this,TextLayout would implementget_spans for most backends, but addget_ps for the Postscriptbackend, which would look for the presence of this method and use itif available, or fall back toget_spans. This kind of specialcasing may also be necessary, for example, when the graphics backendand text engine belong to the same ecosystem, e.g. Cairo and Pango, orMacOSX and CoreText.

There are three main pieces to the implementation:

1. Rewriting the freetype wrapper, and removing ttconv.

   1. Once (1) is done, as a proof of concept, we can move to theupstream STIX .otf fonts

   2. Add support for web fonts loaded from a remote URL. (Enabled by using freetype for font subsetting).

2. Refactoring the existing "builtin" and "usetex" code into separate text engines and to follow the API outlined above.

3. Implementing support for advanced text layout libraries.

(1) and (2) are fairly independent, though having (1) done first willallow (2) to be simpler. (3) is dependent on (1) and (2), but even ifit doesn't get done (or is postponed), completing (1) and (2) willmake it easier to move forward with improving the "builtin" textengine.

Backward compatibility#

The layout of text with respect to its anchor and rotation will changein hopefully small, but improved, ways. The layout of multiline textwill be much better, as it will respect horizontal alignment. Thelayout of bidirectional text or other advanced Unicode features willnow work inherently, which may break some things if users arecurrently using their own workarounds.

Fonts will be selected differently. Hacks that used to sort of workbetween the "builtin" and "usetex" text rendering engines may nolonger work. Fonts found by the OS that weren't previously found bymatplotlib may be selected.

Alternatives#

TBD