Introduction
This is the implementation report for theIMSC Hypothetical Render Model Recommendation. It demonstratesadequate implementation experience and that the CR exit criteria have been met by subjecting avalidating implementation to a test suite and 4 independentcontent producing implementations.
Validating implementations
Thevalidating implementations are:
- imscHRM, which is an open-source implementation of the Recommendation in pure Python.
- bbc/ebu-tt-live-toolkit#66, which is an open-source implementation of the Recommendation in Python, built as part of the BBC fork of the EBU-TT Live Interoperability Toolkit
Test Suite
Thetest suite consists of hand-crafted IMSC documents intended exercise the boundaries of the HRM algorithm. They are organized into two sets, each in its own directory:
- each IMSC document in the
pass directory conforms to the Hypothetical Render Model - each IMSC document in the
fail directory does not conform to the Hypothetical Render Model
Conformance is defined athttps://www.w3.org/TR/imsc-hrm/#conformance.
Each IMSC document contains a short description of the IMSC HRM feature being tested.
All IMSC documents conform to theIMSC Text Profile. They also conform to theEBU-TT-D specification, unless the feature being tested cannot be exercised by a valid EBU-TT-D document, in which case the non-conformance is also identified within the test content itself.
The EBU-TT Live Toolkit implementation cannot process documents that do not conform to EBU-TT-D and therefore the tests that exercise features prohibited in EBU-TT-D are not applicable, and are marked as "n/a".
Note that those "n/a" tests are allowed for within the specification: they refer to style attributes that are included within the definition of aglyph tuple, which, in the case that they are not permitted by a profile, are a) the same across all glyphs and b) if it is necessary to assign a value, have whatever computed value TTML2 specifies.
The following records the result of subjecting thevalidating implementations to the test suite:
Content producing implementations
- The results of subjecting the imscHRMvalidating implementation to IMSC documents generating using two independentcontent producing implementations was reported athttps://lists.w3.org/Archives/Public/public-tt/2023Oct/0007.html. One of thecontent producing implementations was expected to produce IMSC documents that conform to the HRM whereas the other was expected to produce IMSC documents that did not conform to the HRM. The imscHRMvalidating implementation returned the expected result in both cases.
- The results of subjecting the imscHRMvalidating implementation to ~3,500 IMSC documents generated using an independentcontent producing implementation were reported by Cyril Concolato (Netflix) via email to the TTWG chair. The set of documents covered 40+ different languages and different kinds of timed text (SDH, SUBS, or forced narratives). While all documents were initially expected to conform to the HRM, the imscHRMvalidating implementation reported that ~10 of them did not conform to the HRM. It was ultimately determined that these ~10 documents were constructed to specifically address a limitation of the ecosystem targeted by thecontent producing implementation and were not generally valid IMSC documents. 2 of the documents also identified errors in the imscHRMvalidating implementation', which were subsequently fixed.
- The results of subjecting the imscHRMvalidating implementation to 25,000 IMSC documents generated using an independentcontent producing implementation were reported privately to the TTWG chair. While all documents were initially expected to conform to the HRM, the imscHRMvalidating implementation reported that 97 of them did not conform to the HRM. Out of these 97 documents, 60 turned out to contain syntax errors that made them non-conformant with the IMSC Recommendation. The remaining 37 documents were determined to contain authorial mistakes like duplicated or partially overlapping subtitles, which were caught by the imscHRMvalidating implementation.
In all cases, all valid IMSC documents were reported as conforming to the Hypothetical Render Model (HRM).
