WOFF 2.0 Evaluation Report

W3C Working Group Note, 15 March 2016

This version:

http://www.w3.org/TR/2016/NOTE-WOFF20ER-20160315/

Latest version:

http://www.w3.org/TR/WOFF20ER/

Previous version:

http://www.w3.org/TR/2015/WD-WOFF20ER-20150414/

Latest editor's draft:

http://www.w3.org/Fonts/WG/WOFF2ER/

Editor:

Chris Lilley (W3C)

Copyright © 2016W3C^® (MIT,ERCIM,Keio,Beihang). W3Cliability,trademark anddocument use rules apply.

Requirements

A successor to WOFF 1.0 must be deployable in parallel with WOFF 1.0, without relying on style sheet switching, content negotiation, user-agent version sniffing or suchlike fragile methods.

An improved compression method used in WOFF must:

• Be freely implementable in a Royalty-free manner consistent with the W3C Patent Policy
• Have a freely available specification which is sufficient by itself to implement the compression method (it must not depend on opaque 'reference code')
• Produce significantly better compression than WOFF 1.0 in the best case
• Produce the same (or, ideally, better) compression than WOFF 1.0 in the worst case
• Produce notably better median compression than WOFF 1.0
• Not take significantly longer than WOFF 1.0 to decompress
• Not take significantly more RAM than WOFF 1.0 to decompress

It should be noted that pathological fonts can be found which compress badly. Equally, some poorly constructed fonts trivially compress well, due to needless duplication. The main goal of this work is to optimally compress fonts that are already well optimized.

Deployment

In WOFF 1.0 [WOFF 1.0], fonts are deployed using the@font-face feature of CSS3 [CSS3 Fonts]. The format part of the src descriptor identifies the font as WOFF 1.0. Multiple formats are allowed, and form a prioritized list; the font with the first supported format in the list will be loaded. This mechanism thus allows fonts in WOFF 2.0 to be deployed in parallel with those in WOFF 1.0, using the same style sheet and without any user-agent sniffing or server configuration.

/* load WOFF 2.0 font if possible, otherwise use WOFF 1.0 font */@font-face {  font-family: something;  src: url(example.woff2) format("woff2"),       url(example.woff) format("woff");  }

This feature of CSS3 Fonts fulfills theparallel deployment requirement, independent of the compression method ultimately chosen.

Preprocessing

WOFF 2.0 adds a preprocessing step, before entropy coding.The reason for this is that many fonts contain redundant or duplicate information, or information that can be deduced from other items of data also present. Data may also be padded to a convenient multiple of 8 bits, because aligned data is faster to access. They are thus optimized for convenience and speed of access at the expense of size. Preprocessing detects and eliminates redundant data, more efficiently encodes derived data, and stores data in the minimal number of bits. Following decompression, a reconstruction step is needed to re-align data and to compute derived values.

This step is based on the MicroType Express-style [MTX] submission, but uses a subset of the methods in that specification, chosen for maximal gain balanced against minimal processing cost.

The preprocessing steps used for initial explorations are summarized inAppendix D.

The placement of knots in a glyph path follows a set of rules, which can in some cases allow a point to be predicted based on nearby points and the overall position on the curve (for example, at local curve minima or maxima). A novel preprocessing stage was considered which removes predictable points before entropy compression, and restores them in the reconstruction step. Experiments showed that a modest reduction in filesize could be obtained, but that the prediction capabilities of the entropy coder were already accounting for these points and thus, the size of the compressed result was not significantly reduced. These experiments are summarized inAppendix C.

Lossless compression

The compression in WOFF 1.0 is (subject to certain preconditions, such as prior removal of extraneous spacebetweentables)bitwise lossless; the exact same bitstream is produced on decompression as was input to the compression process.

The use of a  preprocessing step means that the overall compression in WOFF 2.0 will not be bitwise lossless. The preprocessing converts some data to the most space-efficient form, and removes redundant information or information which could be regenerated on decompression. This is not a disadvantage; the resulting font functions the same as the original one and in some cases will be more consistent between rendering implementations. The secondary, entropy coding step is bitwise lossless.

Thus, WOFF 2.0 is said to befunctionally lossless.

Continuation streams

In WOFF 1.0, each OFF table is separately compressed. This was originally done to allow individual tables to be decompressed as needed, although that facility is rarely used in practice. For WOFF 1.0 with Flate compression, resetting the compression stream like this also turns out to give a slightly smaller filesize compared to compressing all the tables in a single stream.This is because Flate compression uses a small buffer and cannot cache previous states.

For WOFF 2.0, the impact of per-table compression versus entire-font compression with continuation streams was examined. For the better entropy coding schemes, there was an overall benefit to entire-font streams. As an example, WOFF2 with Brotli compression and continuation streams gave an overall reduction of -29.21% compared to WOFF 1.0, while resetting the compressor on a per-table basis gave a less good filesize reduction of -28.26%.

Candidate A: MicroType Express-style plus LZMA

Specification

LZMA is defined primarily by reference code, portions of which are (in the words of the originator) “hard to explain”. A specification would need to reverse engineer this code. Work was started on this, but it was abandoned when other difficulties indicated that the LZMA approach was no longer worth pursuing.

Intellectual Property

The WebFonts Working Group was unable, despite reaching out to LZMA developers, to secure a commitment to the development of a re-implementable, Royalty-Free specification.

Compression compared to WOFF 1.0

Compression compared to WOFF 1.0 was impressive. (Part of this was due to the MicroType Express-style preprocessing step). Several complete font collections were examined, each font in the collection compressed and the compression gains tabulated. The results are presented in detail inAppendix A.

Decompression time and memory requirements

The decompression time required ( for example, a decompression rate of 36.5 MB/s on the Google Fonts corpus) was considerably worse than for WOFF 1.0 (194.3 MB/s on the same hardware with the same corpus).This was primarily due to the LZMA step; MicroType Express-style step is an optimization and removal of redundancy, so the impact of reconstruction on decompression time was minimal (in fact, omitting the preprocessing stepdecreaseddecompression throughput slightly, to 36.0 MB/s, as the entropy coder was working on a larger data stream).

The memory required for LZMA  decompression was around twice that needed for gzip decompression.

Analysis

Compression gains over WOFF 1.0 are largely consistent (within a few percent) across all of the font collections studied. This implies that they would be applicable to other collections too. In general 20-26% median compression improvements for TrueType were observed, and 12-13% for CFF. This satisfies the requirement to producenotably better median compression than WOFF 1.0.

Improvements up to 94.51% were observed in some cases (in the Monotype collection, for fonts with exceptionally large kern tables.) The smallest improvements seen were of the order of 1% (again, in the Monotype collection); most other collections had smallest improvements of 4-6%. It is notable that in no case was a font found that compressed to the same size as WOFF 1.0, and no cases were observed where a font became larger.

Decompression time precluded use of this candidate if multiple fonts were decompressed in parallel, or if decompressed on resource-constrained mobile devices. The requirement fordecompression time was not satisfied.

After extensive discussion with the original LZMA developer, and examination of the sample code which is the primary documentation, the Working group concluded that theRoyalty-Free andreimplementable algorithm requirements could not be met.

Accordingly, the WebFonts Working Group hasdiscontinued efforts on this candidate.

Candidate B: MicroType Express-style plus Brotli

Specification

The specification [Brotli] is being developed by the Google Compression team in Zürich. Being a derivative of the widely deployed Flate [Flate] algorithm, it proceeds from a well-understood basis. The improvements are well documented and open source, sample code is freely available.

Intellectual Property

The proposers have made the specification freely available as IETF Drafts, and are pursueing it's standardization through the IETF.

Compression compared to WOFF 1.0

Compression compared to WOFF 1.0 was good. It was initially less impressive than candidate A but improved significantly over the course of study as the algorithm was tuned. (Part of this was due to the MicroType Express-style preprocessing step). To date, three font collections have been  examined. Each font in each collection was compressed and the compression gains tabulated. The initial results are presented in detail inAppendix B.

In summary, for fonts using TTF outlines, median gains of23.94% over WOFF1 were observed on the Google font corpus and26.79% over WOFF1 for the Adobe font corpus. The minimum gain observed for any font containing TTF outlines was 6.61% and the maximum, 70.82%. For fonts using CFF outlines, a smaller but still significant median gain of13.51% over WOFF1 was observed on the Adobe font corpus. The minimum gain observed for any font containing CFF outlines was 5.32% and the maximum, 57.96%. The lower compression is expected, because CFF fonts do not benefit from the preprocessing step. The Working Group plans to continue with analysis of other font corpi.

Decompression time and memory requirements

The decompression time required (for example, a decompression rate of 61.1 MB/s on the Google Fonts corpus) was worse than for WOFF 1.0 (194.3 MB/s on the same hardware with the same corpus) but twice as good as for WOFF 2.0 with LZMA (36.5 MB/s). Omitting the preprocessor step made decompression throughput worse (56.4 Mb/s).

The memory required for Brotli decompression of WOFF2 fonts was on average 1.05x that required for WOFF1 with gzip decompression, in a study on the Google Fonts corpus (1654files. At the medianthere was no memory increase. The worst result was 1.7x more memory forWOFF2 decompression.

Analysis

Results are consistently good over the three large corpi studied. The Working Group will continue to examine other data sets.

Conclusions

Initial results indicated that the final compression gains achieved by Brotli (median 23.94% on the Google Fonts corpus) have significantly outperformed our initial goal of half WOFF 2.0 Candidate A (LZMA) gains (median 24.96% on the same corpus). Additional results from the Adobe corpi confirm that similar compression gains should be expected for fonts using TTF outlines, with lower gains for fonts using CFF outlines. This has been achieved while significantly increasing decompression speed and lowering decompression memory requirements.

References

[Brotli]

Brotli Compressed Data Format, draft-alakuijala-brotli-08 J. Alakuijala and Z. Szabadka, December 2015.

[CSS3Fonts]

John Daggett, editor.CSS Fonts Module Level 3. W3C Candidate Recommendation 3 October 2013. Available athttp://www.w3.org/TR/2013/CR-css-fonts-3-20131003/.

[LZMA]

7-Zip, LZMA SDK (Software Development Kit). Available athttp://www.7-zip.org/sdk.html

[Flate]

Deutsch, P.RFC 1951: DEFLATE Compressed Data Format Specification version 1.3. May 1996. Available athttp://tools.ietf.org/html/rfc1951

[MTX]

Sarah Martin,et. al, editors.MicroType® Express (MTX) Font Format. W3C Member Submission 5 March 2008. Available athttp://www.w3.org/Submission/2008/SUBM-MTX-20080305/

[WOFF 1.0]

Jonathan Kew, Tal Leming, Erik van Blokland, editors.WOFF File Format 1.0. W3C Recommendation 13 December 2012. Available athttp://www.w3.org/TR/2012/REC-WOFF-20121213/.

[WOFF 2.0]

Vladimir Levantovsky, Raph Levien, editors.WOFF File Format 2.0. W3C Working Draft, 15 March 2016. Available athttp://www.w3.org/TR/WOFF2/.

Changes

Since first publication

• Compression results for Adobe CFF and TTF corpi added to Appendix B
• Discussion of compression results updated
• Decompression memory performance added
• Brotli IPR section updated to reflect ongoing standardization in IETF
• References updated to latest versions

Since previous publication

• Added info on memory useage for decoding
• Added appendix on CFF desubroutinisation
• Added appendix on hmtx optimization
• References updated to latest versions

Appendix A: MicroType Express-style plus LZMA compression data

Compression results, Google Fonts corpus, with LZMA [June 2013]

Results for 1,194 fonts using TTF outlines, from the Google font corpus.