In this document it is difficult to distinguish between thosestring elements that are intended for translation and those that are not to be translated. Explicit metadata is needed to resolve the issue.

<resources><sectionid="Homepage"><arguments><string>page</string><string>childlist</string></arguments><variables><string>POLICY</string><string>Corporate Policy</string></variables><keyvalue_pairs><string>Page</string><string>ABC Corporation - Policy Repository</string><string>Footer_Last</string><string>Pages</string><string>bgColor</string><string>NavajoWhite</string><string>title</string><string>List of Available Policies</string></keyvalue_pairs></section></resources>

[Source file:examples/xml/EX-motivation-its-1.xml]

ITS provides two mechanisms to explicitly associate metadata with one or more pieces of content (e.g. XML nodes): aglobal, rule-based approach as well as alocal, attribute-based approached. Here, for instance, atranslateRule first specifies that only every second element insidekeyvalue_pairs is intended for translation; later, an ITStranslate attribute specifies that one of these elements is not to be translated.

<resourcesxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><its:rulesversion="2.0"><its:translateRuleselector="//arguments"translate="no"/><its:translateRuleselector="//keyvalue_pairs/string[(position() mod 2)=1]"translate="no"/></its:rules><sectionid="Homepage"><arguments><string>page</string><string>childlist</string></arguments><variables><stringits:translate="no">POLICY</string><string>Corporate Policy</string></variables><keyvalue_pairs><string>Page</string><string>ABC Corporation - Policy Repository</string><string>Footer_Last</string><string>Pages</string><string>bgColor</string><stringits:translate='no'>NavajoWhite</string><string>title</string><string>List of Available Policies</string></keyvalue_pairs></section></resources>

[Source file:examples/xml/EX-motivation-its-2.xml]

2.2.1 Local Approach

The document inExample 3 shows how a content author can use the ITStranslate attribute to indicate that all content inside theauthor element is not intended for translation (i.e. has to be left untranslated). Translation tools that are aware of the meaning of the attribute can protect the relevant content from being translated (possibly still allowing translators to see the protected content as context information).

<articlexmlns="http://docbook.org/ns/docbook"xmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"version="5.0"xml:lang="en"><info><title>An example article</title><authorits:translate="no"><personname><firstname>John</firstname><surname>Doe</surname></personname><affiliation><address><email>foo@example.com</email></address></affiliation></author></info><para>This is a short article.</para></article>

For the local approach (andExample 3) to work for a whole markup vocabulary, a schema developer would need to add thetranslate attribute to the schema as a common attribute or on all the relevant element definitions. The example indicates thatinheritance plays a part in identifying which content does have to be translated and which does not: Although only theauthor element is marked as “do not translate”, its descendants (personname,firstname,surname) are considered to be implicitly marked as well. Tools that process this content for translation need to implement the expected inheritance.

For XML content, the local approach cannot be applied to a particular attribute. If ITS needs to be applied to a particular attribute, the global approach has to be used. The local approach applies to content of the current element and all its inherited nodes as described inSection 8.1: Position, Defaults, Inheritance, and Overriding of Data Categories. For theTranslate data category used in[HTML5], this is different, see the explanation of theHTML5 definition of Translate.

2.2.2 Global Approach

The document inExample 4 shows a different approach to identifying non-translatable content, similar to that used with astyle element in[XHTML 1.0], but using an ITS-defined element calledrules. It works as follows: A document can contain arules element (placed where it does not impact the structure of the document, e.g., in a “head” section, or even outside of the document itself). Therules element contains one or more ITS children/rule elements (for exampletranslateRule). Each of these children elements contains aselector attribute. As its name suggests, this attribute selects the node or nodes to which the corresponding ITS information pertains. The values of ITSselector attributes are XPath absolute location paths (or CSS Selectors if queryLanguage is set to "css"). Via theparam element variables can be provided and used in selectors.

Information for the handling of namespaces in XPath expressions is taken from namespace declarations[XML Names] in the current rule element.

<myTopicxmlns="http://mynsuri.example.com"id="topic01"xml:lang="en-us"><prolog><title>Using ITS</title><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:translateRuleselector="//n:term"translate="no"xmlns:n="http://mynsuri.example.com"/></its:rules></prolog><body><p>ITS defines<term>data category</term> as an abstract concept for a particular type of      information for internationalization and localization of XML schemas and documents.</p></body></myTopic>

For the global approach (andExample 4) to work, a schema developer may need to add arules element and associated markup to the schema. In some cases, global rules may be sufficient and other ITS markup (such as antranslate attribute on the elements and attributes) may not be needed in the schema. However, it is likely that authors may need the local approach from time to time to override the general rule.

For specification of theTranslate data category information, the contents of thetranslateRule element would normally be designed by an information architect familiar with the document format and familiar with, or working with someone familiar with, the needs of localization/translation.

The global, rule-based approach has the following benefits:

• Content authors do not have to concern themselves with creating additional markup or verifying that the markup was applied correctly. ITS data categories are associated with sets of nodes (for example allp elements in an XML instance)

• Changes can be made in a single location, rather than by searching and modifying local markup throughout a document (or documents, if therules element is stored as an external entity)

• ITS data categories can designate attribute values (as well as elements)

• It is possible to associate ITS markup with existing markup (for example theterm element in DITA)

The commonality in both examples above is the markuptranslate='no'. This piece of ITS markup can be interpreted as follows:

• it pertains to theTranslate data category

• the attributetranslate holds a value of "no"

<textxmlns:its="http://www.w3.org/2005/11/its"><head><revision>Sep-10-2006 v5</revision><author>Ealasaidh McIan</author><contact>ealasaidh@hogw.ac.uk</contact><titleits:translate="yes">The Origins of Modern Novel</title><its:rulesversion="2.0"><its:translateRuletranslate="no"selector="/text/head"/></its:rules></head><body><divxml:id="intro"><head>Introduction</head><p>It would certainly be quite a<spanits:translate="no">faux pas</span> to start a        dissertation on the origin of modern novel without mentioning the<tl>Epic of        Gilgamesh</tl>...</p></div></body></text>

[Source file:examples/xml/EX-basic-concepts-3.xml]

2.5.1 Global approach in HTML5

To account for the so-calledglobal approach in HTML, this specification (seeSection 6.2: Global rules) defines:

• A link type for referring to external files with global rules from alink element.

• An approach to have inline global rules in the HTMLscript element.

It is preferable to use external global rules linked via thelink element rather than to have inline global rules in the HTML document. The advantage is in being able to reuse the same rules file for many documents and also inline rules require secondary parsing of thescript element.

<!DOCTYPE html><html><head><metacharset=utf-8><title>Translate flag global rules example</title><linkhref=EX-translateRule-html5-1.xmlrel=its-rules></head><body><p>This sentence should be translated, but code names like      the<code>span</code> element should not be translated.      Of course there are always exceptions: certain       code values should be translated, e.g. to a value       in your language like<codetranslate=yes>warning</code>.</p></body></html>

<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"xmlns:h="http://www.w3.org/1999/xhtml"><its:translateRuletranslate="no"selector="//h:code"/></its:rules>

<!DOCTYPE html><html><head><metacharset=utf-8><title>Translate flag global rules example</title><scripttype=application/its+xmlid=ru1><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"xmlns:h="http://www.w3.org/1999/xhtml"><its:translateRuletranslate="no"selector="//h:code"/></its:rules></script></head><body><p>This sentence should be translated, but code names like      the<code>span</code> element should not be translated.      Of course there are always exceptions: certain      code values should be translated, e.g. to a value in      your language like<codetranslate=yes>warning</code>.</p></body></html>

2.5.2 Local approach

In HTML, an ITS 2.0 local data category is realized with the prefixits-. The general mapping of the XML based ITS 2.0 attributes to their HTML counterparts is defined inSection 6.1: Mapping of Local Data Categories to HTML. An informative table inAppendix I: List of ITS 2.0 Global Elements and Local Attributes provides an overview of the mapping for all data categories.

2.5.3 HTML markup with ITS 2.0 counterparts

There are four ITS 2.0 data categories, which have counterparts in HTML markup. In these cases, native HTML markup provides some information in terms of ITS 2.0 data categories. For these data categories, ITS 2.0 defines the following:

• TheLanguage Information data category has the HTMLlang attribute as a counterpart. In XHTML the counterpart is thexml:lang attribute. These HTML attributes act as local markup for theLanguage Information data category in HTML and takeprecedence over language information conveyed via a globallangRule.

• TheId Value data category has the HTML or XHTMLid attribute as counterpart. This HTML attribute acts as local markup for theId Value data category in HTML and takesprecedence over identifier information conveyed via a globalidValueRule.

• TheElements within Text data category has a set of HTML elements (the so-calledphrasing content) as counterpart. In the absence of anElements within Text local attribute or global rules selecting the element in question, most of the phrasing content elements are interpreted aswithinText="yes" by default. The phrasing content elementsiframe,noscript,script andtextarea are interpreted aswithinText="nested".

• TheTranslate data category has a direct counterpart in[HTML5], namely the[HTML5]translate attribute. ITS 2.0 does not define its own behavior for[HTML5]translate, but just refers tothe HTML5 definition. That definition also applies to nodes selected via global rules. That is, atranslateRule like<its:translateRule selector="//h:img" translate="yes"/> will set theimg element and its translatable attributes likealt to "yes".

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>HTML native markup expressing three ITS 2.0 data categories</title></head><body><pid="p1"translate="no">This is a<em>motherboard</em> and image:<imgsrc="http://example.com/myimg.png"alt="My image"/>.</p></body></html>

There are also some HTML markup elements that have or can have similar, but not necessarily identical, roles and behaviors as certain ITS 2.0 data categories. For example, the HTMLdfn element could be used to identify a term in the sense of theTerminology data category. However, this is not always the case and it depends on the intentions of the HTML content author. To accommodate this situation, users of ITS 2.0 are encouraged to specify the semantics of existing HTML markup in an ITS 2.0 context with a dedicated global rules file. For example, a rule can be used to define that the HTMLdfn has the semantics of ITSterm="yes". For additional examples, see theXML I18N Best Practices document.

2.5.4 Standoff markup in HTML5

TheProvenance and theLocalization Quality Issue data categories allow for using so-called standoff markup, see the XMLExample 58. In HTML such standoff markup is placed into ascript element. If this is done, the constraints forProvenance standoff markup in HTML andLocalization quality issue markup in HTML need to be taken into account. Examples of standoff markup in HTML for the two data categories areExample 61 andExample 76.

2.5.5 Version of HTML

ITS 2.0 does not define how to use ITS in HTML versions prior to version 5. Users are thus encouraged to migrate their content to[HTML5] or XHTML. While it is possible to useits-* attributes introduced for[HTML5] in older versions of HTML (such as 3.2 or 4.01) and pages using these attributes will work without any problems,its-* attributes will be marked as invalid by validators.

2.7.1 ITS and RDF/NIF

ITS 2.0 provides a non-normative algorithm to convert XML or HTML documents (or their DOM representations) that contain ITS metadata to the RDF format based on[NIF]. NIF is an RDF/OWL-based format that aims at interoperability between Natural Language Processing (NLP) tools, language resources and annotations.

The conversion fromITS 2.0 to NIF results in RDF triples. These triples represent the textual content of the original document as RDF typed information. The ITS annotation is represented as properties of content-related triples and relies on anITS RDF vocabulary.

The back conversion fromNIF to ITS 2.0 is defined informatively as well. One motivation for the back conversion is a roundtrip workflow like: 1) conversion to NIF 2) in NIF representation detection of named entities using NLP tools 3) back conversion to HTML and generation ofText Analysis markup. The outcome are HTML documents with linked information, seeExample 52.

2.7.2 ITS and XLIFF

The XML Localization Interchange File Format[XLIFF 1.2] is an OASIS standard that enables translatable source text and its translation to be passed between different tools within localization and translation workflows.[XLIFF 2.0] is the successor of[XLIFF 1.2] and under development. XLIFF has been widely implemented in various translation management systems, computer aided translation tools and in utilities for extracting translatable content from source documents and merging back the content in the target language.

The mapping between ITS and XLIFF therefore underpins several important ITS 2.0 usage scenarios[MLW US IMPL]. These usage scenarios involve:

• the extraction of ITS metadata from a source language file into XLIFF

• the addition of ITS metadata into an XLIFF file by translation tools

• the mapping of ITS metadata in an XLIFF file into ITS metadata in the resulting target language files.

ITS 2.0 has no normative dependency on XLIFF, however anon-normative definition of how to represent ITS 2.0 data categories in XLIFF 1.2 or XLIFF 2.0 is being defined within theInternationalization Tag Set Interest Group.

TheTranslate data category conveys information as to whether a piece of content is intended for translation or not.

The simplest formalization of this prose description on a schema language-independent level is atranslate attribute with two possible values: "yes" and "no". An implementation on a schema language-specific level would be the declaration of thetranslate attribute in, for example, an XML Schema document or a RELAX NG document. A different implementation would be atranslateRule element that allows for specifyingglobal rules about theTranslate data category.

Note:

The selection of the ITS data categories applies to textual values contained within element or attribute nodes. In some cases these nodes form pointers to other resources; a well-known example is thesrc attribute on theimg element in HTML. The ITSTranslate data category applies to the text of the pointer itself, not the object to which it points. Thus in the following example, the translation information specified via thetranslateRule element applies to the filename "instructions.jpg", and is not an instruction to open the graphic and change the words therein.

<text><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:translateRuletranslate="yes"selector="//p/img/@src"/></its:rules>  ...<pxmlns:its="http://www.w3.org/2005/11/its">As you can see in<imgsrc="instructions.jpg"/>, the truth is not always out there.</p></text>

Note:

Application-specific processing (that is processing that goes beyond the computation of ITS information for a node), such as automated filtering of translatable content based on theTranslate data category, is not covered by the conformance clauses below.

Note:

The above conformance clauses are directly reflected in theITS 2.0 test suite. All tests specify which data category is processed (clause2-1); they are relevant for (clause2-1-1) global or local selection, or both; they require the processing of defaults and precedence of selections (clauses2-1-2 and2-1-3); for each data category there are tests with linked rules (2-2); and all types of tests are given for XML (clause2-3). Implementers are encouraged to organize their documentation in a similar way, so that users of ITS 2.0 easily can understand the processing capabilities available.

Note:

Application-specific processing (that is processing that goes beyond the computation of ITS information for a node) such as automated filtering of translatable content based on theTranslate data category is not covered by the conformance clauses below.

5.2.1 Global, Rule-based Selection

Global, rule-based selection is implemented using therules element. Therules element contains zero or morerule elements. Eachrule element has a mandatoryselector attribute. This attribute and all other possible attributes onrule elements are in the empty namespace and used without a prefix.

If there is more than onerules element in an XML document, the rules from each section are to be processed at the same precedence level. Therules sections are to be read in document order, and the ITS rules with them processed sequentially. The versions of theserules elementsMUST NOT be different.

Depending on the data category and its usage, there are additional attributes for adding information to the selected nodes, or for pointing to existing information in the document. For example, theLocalization Note data category can be used for adding notes to selected nodes, or for pointing to existing notes in the document. For the former purpose, alocNote element can be used. For the latter purpose, alocNotePointer attribute can be used.

Thedata category overview table, inSection 8.1: Position, Defaults, Inheritance, and Overriding of Data Categories, provides an overview of what data categories allow to point to existing information or to add information.

The functionalities of adding information and pointing to existing information aremutually exclusive. That is: markup for pointing and adding the same informationMUST NOT appear in the same rule element.

Global rules can appear in the XML document they will be applied to, or in a separate XML document. The precedence of their processing depends on these variations. See alsoSection 5.5: Precedence between Selections.

5.2.2 Local Selection in an XML Document

Local selection in XML documents is realized withITS local attributes or thespan element.span serves just as a carrier for the local ITS attributes.

The data category determines what is being selected. The necessary data category specific defaults are described inSection 8.1: Position, Defaults, Inheritance, and Overriding of Data Categories.

<textxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"xml:lang="en"><headits:translate="no"><author>Sven Corneliusson</author><date>2006-09-26T17:34:04Z</date><titleits:translate="yes"role="header">Bidirectional Text</title></head><body><par>In Arabic, the title<quotexml:lang="ar"its:dir="rtl">نشاط التدويل، W3C</quote> means<quote>Internationalization Activity, W3C</quote>.</par></body></text>

5.3.1 Choosing Query Language

Rule elements have attributes that contain absolute and relative selectors. Interpretation of these selectors depends on the actual query language. The query language is set byqueryLanguage attribute onrules element. IfqueryLanguge is not specified XPath 1.0 is used as a default query language.

5.3.2 XPath 1.0

XPath 1.0 is identified byxpath value inqueryLanguage attribute.

<!-- Definitions for TEI --><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:termRuleselector="//tei:term"term="yes"xmlns:tei="http://www.tei-c.org/ns/1.0"/></its:rules>

<!-- Definitions for DocBook --><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:termRuleselector="//term"term="yes"/></its:rules>

5.3.3 CSS Selectors

CSS Selectors are identified by the valuecss in thequeryLanguage attribute.

5.3.4 Additional query languages

ITS processorsMAY support additional query languages. For each additional query language the processorMUST define:

• the identifier of the query language used inqueryLanguage;

• rules for evaluating an absolute selector to a collection of nodes;

• rules for evaluating a relative selector to a collection of nodes.

Because future versions of this specification are likely to define additional query languages, the following query language identifiers are reserved:xpath,css,xpath2,xpath3,xquery,xquery3,xslt2,xslt3.

5.3.5 Variables in selectors

Aparam element (or several ones) can be placed as the first child element(s) of therules element to define the default values of variables used in the various selectors used in the rules.

An implementationMUST support theparam element for all query languages it supports and at the same time define how variables are bound for evaluation of the selector expression. ImplementationsSHOULD also provide means for changing the default values of theparam elements. Such means are implementation-specific.

Theparam element has a requiredname attribute. The value of thename attribute is aQName, see[XML Names]. The content of the element is a string used as default value for the corresponding variable.

<docits:version="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:rulesversion="2.0"><its:paramname="LCID">0x0409</its:param><its:translateRuleselector="/doc"translate="no"/><its:translateRuleselector="//msg[@lcid=$LCID]"translate="yes"/></its:rules><msglcid="0x0409"num="1">Create a folder</msg><msglcid="0x0411"num="1">フォルダーを作成する</msg><msglcid="0x0407"num="1">Erstellen Sie einen Ordner</msg><msglcid="0x040c"num="1">Créer un dossier</msg></doc>

The example demonstrates how metadata can be added to ITS rules.

<myFormatInfo><desc>ITS rules used by the Open University</desc><hostVoc>http://www.tei-c.org/ns/1.0</hostVoc><rulesId>98ECED99DF63D511B1250008C784EFB1</rulesId><rulesVersion>v 1.81 2006/03/28 07:43:21</rulesVersion>  ...<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:translateRuleselector="//header"translate="no"/><its:translateRuleselector="//term"translate="no"/><its:termRuleselector="//term"term="yes"/><its:withinTextRulewithinText="yes"selector="//term | //b"/></its:rules></myFormatInfo>

[Source file:examples/xml/EX-link-external-rules-1.xml]

<myDoc><header><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"xmlns:xlink="http://www.w3.org/1999/xlink"xlink:href="EX-link-external-rules-1.xml"><its:translateRuleselector="//term"translate="yes"/></its:rules><author>Theo Brumble</author><lastUpdate>Apr-01-2006</lastUpdate></header><body><p>A<term>Palouse horse</term> has a spotted coat.</p></body></myDoc>

[Source file:examples/xml/EX-link-external-rules-2.xml]

<myDoc><header><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:translateRuleselector="//header"translate="no"/><its:translateRuleselector="//term"translate="no"/><its:termRuleselector="//term"term="yes"/><its:withinTextRulewithinText="yes"selector="//term | //b"/><its:translateRuleselector="//term"translate="yes"/></its:rules><author>Theo Brumble</author><lastUpdate>Apr-01-2006</lastUpdate></header><body><p>A<term>Palouse horse</term> has a spotted coat.</p></body></myDoc>

[Source file:examples/xml/EX-link-external-rules-3.xml]

As withExample 16, these rules can be applied toExample 17. The only difference is that inExample 19, therules element is the root element of the external file.

<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:translateRuleselector="//header"translate="no"/><its:translateRuleselector="//term"translate="no"/><its:termRuleselector="//term"term="yes"/><its:withinTextRulewithinText="yes"selector="//term | //b"/></its:rules>

[Source file:examples/xml/EX-link-external-rules-4.xml]

Note:

The precedence order fulfills the same purpose as the built-in template rules of[XSLT 1.0]. Override semantics are always complete, that is all information provided via lower precedence is overridden by the higher precedence. E.g. defaults are overridden by inherited values and these are overridden by nodes selected via global rules, which are in turn overridden by local markup.

The two elementstitle andauthor of this document are intended as separate content when inside aprolog element, but in other contexts as part of the content of their parent element. In order to make this distinction twowithinTextRule elements are used:

The first rule specifies thattitle andauthor in general are to be treated as an element within text. This overrides the default.

The second rule indicates that whentitle orauthor are found in aprolog element their content is to be treated separately. This is normally the default, but the rule is needed to override the first rule.

<text><prolog><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:withinTextRulewithinText="yes"selector="//title|//author"/><its:withinTextRulewithinText="no"selector="//prolog/title|//prolog/author"/></its:rules><title>Designing User Interfaces</title><author>Janice Prakash</author><keywords>user interface, ui, software interface</keywords></prolog><body><p>The book<title>Of Mice and Screens</title> by<author>Aldus Brandywine</author> is one of      the best introductions to the vast topic of designing user interfaces.</p></body></text>

[Source file:examples/xml/EX-selection-precedence-1.xml]

In this example, there is an existingtranslate attribute in DITA, and it is associated with the ITS semantics using the its:rules section. Similarly, the DITAdt andterm elements are associated with the ITSTerminology data category.

<topicid="myTopic"><title>The ITS Topic</title><prolog><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:translateRuleselector="//*[@translate='no']"translate="no"/><its:translateRuleselector="//*[@translate='yes']"translate="yes"/><its:termRuleselector="//term | //dt"term="yes"/></its:rules></prolog><body><dl><dlentryid="tDataCat"><dt>Data category</dt><dd>ITS defines<term>data category</term> as an abstract concept for a particular type of          information related to internationalization and localization of XML schemas and          documents.</dd></dlentry></dl><p>For the implementation of ITS, apply the rules in the order:</p><ul><li>Defaults</li><li>Rules in external files</li><li>Rules in the document</li><li>Local attributes</li></ul><p><phtranslate="no"xml:lang="fr">Et voilà !</ph>.</p></body></topic>

[Source file:examples/xml/EX-associating-its-with-existing-markup-1.xml]

Note:

• Three cases of providing tool information can be expected:

  1. information about tools used for creating or modifying the textual content;

  2. information about tools that do 1), but also create ITS annotations, seeAppendix I: List of ITS 2.0 Global Elements and Local Attributes;

  3. information about tools that don’t modify or create content, but just create ITS annotations.

  annotatorsRef is only meant to be used when actual ITS annotation is involved, that is for 2) and 3). To express tool information related only to the creation or modification of textual content and independent of ITS data categories, that is case 1), the tool ortoolRef attribute provided by theProvenance data category is to be used.

• An example of case 2) is an MT engine that modifies content and creates ITSMT Confidence annotations. Here the situation may occur that several tools are involved in creating MT Confidence annotations: the MT engine and the tool inserting the markup. The annotatorsRef attribute is to identify the tool most useful in further processes, in this case the MT engine.

In this example, the text shows the computed tools reference information for the given node. Note that the references are ordered alphabetically and that the IRI values are always the ones of the inner-most declaration.

<docits:version="2.0"xmlns:its="http://www.w3.org/2005/11/its"its:annotatorsRef="mt-confidence|MT1"  >doc node: "mt-confidence|MT1"<groupits:annotatorsRef="terminology|ABC"    >group node: "mt-confidence|MT1 terminology|ABC"<pits:annotatorsRef="text-analysis|Tool3"      >This p node: "text-analysis|Tool3 mt-confidence|MT1 terminology|ABC"</p><pits:annotatorsRef="mt-confidence|MT123"      >This p node: "mt-confidence|MT123 terminology|ABC"</p></group><!-- To make this example usable in real life, we would have     annotations of the three data categories - text-analysis, mt-confidence and terminology     - in the document --><pits:annotatorsRef="text-analysis|XYZ"   >This p node: "text-analysis|XYZ mt-confidence|MT1"</p></doc>

[Source file:examples/xml/EX-its-tool-annotation-1.xml]

TheannotatorsRef attribute is used in this XML document to indicate that information about the processor that generated themtConfidence values for the first twop elements are found in element withid="T1" in the external document tools.xml, while that information for the thirdp element is found in the element withid="T2" in the same document. In addition,annotatorsRef is used to identify a Web resource with information about the QA tool used to generate theLocalization Quality Issue annotation in the document.

<docits:version="2.0"its:annotatorsRef="mt-confidence|file:///tools.xml#T1 localization-quality-issue    |http://www.qalsp-ex.com/qatools/transcheckv1.3"xmlns:its="http://www.w3.org/2005/11/its"><pits:mtConfidence="0.78">Text translated with tool T1</p><pits:mtConfidence="0.55"its:locQualityIssueType="typographical"its:locQualityIssueComment="Sentence without capitalization"its:locQualityIssueSeverity="50">text also translated with tool T1</p><pits:mtConfidence="0.34"its:annotatorsRef="mt-confidence|file:///tools.xml#T2"> Text translated        with tool T2</p></doc>

[Source file:examples/xml/EX-its-tool-annotation-2.xml]

Theits-annotators-ref attributes are used in this HTML document to indicate that theMT Confidence annotation on the first twospan elements come from one MT (French to English) engine, while the annotation on the third comes from another (Italian to English) engine. Bothits-annotators-ref attributes refer to a Web resource for information about the engine generating theMT Confidence annotation.

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Sentences about capital cities                 machine translated into English with mtConfidence defined                 locally.</title></head><bodyits-annotators-ref="mt-confidence|http://www.exmt-prov.com/2012/11/9/fr-t-en"><p><spanits-mt-confidence=0.8982>Dublin is the capital of Ireland.</span><spanits-mt-confidence=0.8536>The capital of the Czech Republic is Prague.</span><spanits-mt-confidence=0.7009its-annotators-ref="mt-confidence|http://www.exmt-prov.com/2012/11/9/it-t-en">  The capital Italia is Roma.</span></p></body></html>

[Source file:examples/html5/EX-its-tool-annotation-html5-1.html]

Note:

The above-mentioned data categories are excluded because HTML has native markup for them.

Note:

Case of attribute names is also irrelevant given the nature of HTML syntax. So in HTML theterminology data category can be stored asits-term,ITS-TERM,its-Term etc. All of those attributes are treated as equivalent and will be normalized upon DOM construction.

Note:

By default XPath 1.0 will be used for selection in global rules. If users prefer an easier selection mechanism, they can switch query language to CSS selectors by using thequeryLanguage attribute, seeSection 5.3.1: Choosing Query Language.

Note:

The HTML5 parsing algorithm automatically puts all HTML elements into the XHTML namespace (http://www.w3.org/1999/xhtml). Selectors used in global rules need to take this into account.

Note:

Using XPath in global rules linked from HTML documents does not create an additional burden to implementers. Parsing HTML content produces a DOM tree that can be directly queried using XPath, functionality supported by all major browsers.

Note:

It is preferred to use external global rules linked using thelink element than to have global rules embedded in the document.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><htmlxmlns="http://www.w3.org/1999/xhtml"xml:lang="en"><head><title>XHTML and ITS2.0</title></head><body><h1>XHTML and ITS2.0</h1><p>Don't use<spanits-loc-note="Internationalization Tag Set">ITS</span> prefixed            attributes inside the content, like its:locNote.</p></body></html>

Note:

An ITS application is free to decide what pieces of content it uses. For example:

• Terminology information is added to aterm element. The information pertains only to the content of the element, since there is no inheritance forTerminology. Nevertheless an ITS application can make use of the complete element, e.g., including attribute nodes etc.

• UsingID Value, a unique identifier is provided for ap element. An application can make use of the completep element, including child nodes and attributes nodes. The application is also free to make use just of the string value ofp. Nevertheless the id provided viaID Value pertains only to thep element. It cannot be used to identify nested elements or attributes.

• Usingtarget pointer, selectedsource elements have the ITS information that their translation is available in atarget element; seeExample 65. This information does not inherit to child elements oftarget pointer. E.g., the translation of aspan element nested insource is not available in a specifictarget element. Nevertheless, an application is free to use the complete content ofsource, includingspan, and, e.g., present it to a translator.

In this example, the content of all thedata elements is translatable and none of the attributes are translatable, because the default for theTranslate data category in elements is "yes" and in attributes is "no", and neither of their values are overridden at all. The firsttranslateRule is overridden by the localits:translate="no" attribute. The content ofrevision,profile,reviser andlocNote elements are not translatable. This is because the default is overridden by the sameits:translate="no" that these elements inherit from the local ITS markup in theprolog element. The exception is thefield element where the secondtranslateRule takes precedence over the inherited value. The lasttranslateRule indicates that the content oftype is not translatable because the global rule takes precedence over the default value.

The localization note for the two firstdata elements is the text defined globally with thelocNoteRule element. This note is overridden for the lastdata element by the locallocNote attribute.

<Resxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><prologits:translate="no"><revision>Sep-07-2006</revision><profile><reviser>John Doe</reviser><field>Computing Engineering</field></profile><its:rulesversion="2.0"><its:translateRuleselector="//prolog"translate="yes"/><its:translateRuleselector="/Res/prolog/profile/field"translate="yes"/><its:translateRuleselector="//msg/type"translate="no"/><its:locNoteRulelocNoteType="description"selector="//msg/data"><its:locNote>The variable {0} is the name of the host.</its:locNote></its:locNoteRule></its:rules></prolog><body><msgid="HostNotFound"><type>Error</type><data>Host {0} cannot be found.</data></msg><msgid="HostDisconnected"><type>Error</type><data>The connection with {0} has been lost.</data></msg><msgid="FileNotFound"><type>Error</type><dataits:locNote="{0} is a filename">{0} not found.</data></msg></body></Res>

[Source file:examples/xml/EX-datacat-behavior-1.xml]

Note:

The data categories differ with respect to defaults. This difference is due to existing standards and practices. It is common practice for example that information about translation refers only to textual content of an element. Thus, the default selection for theTranslate data category is the textual content.

8.2.1 Definition

TheTranslate data category expresses information about whether the content of an element or attribute is intended for translation or not. The values of this data category are "yes" (translatable) or "no" (not translatable).

8.2.2 Implementation

TheTranslate data category can be expressed with global rules, or locally on an individual element. Handling of inheritance and interaction between elements and attributes is different for XML content versus[HTML5] content.

For XML: for elements, the data category informationinherits to the textual content of the element,including child elements, butexcluding attributes. The default is that elements are translatable and attributes are not.

For HTML: The interpretation of thetranslate attribute is given inHTML5. Nodes in an HTML document selected via aglobal rule are also interpreted followingHTML5.

GLOBAL: ThetranslateRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredtranslate attribute with the value "yes" or "no".

<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:translateRuletranslate="no"selector="//code"/></its:rules>

LOCAL: The following local markup is available for theTranslate data category:

• Atranslate attribute with the value "yes" or "no".

In[HTML5] the native[HTML5]translate attributeMUST be used to express theTranslate data category.

<messagesits:version="2.0"xmlns:its="http://www.w3.org/2005/11/its"><msgnum="123">Click Resume Button on Status Display or<panelmsgits:translate="no"      >CONTINUE</panelmsg> Button on printer panel</msg></messages>

<!DOCTYPE html><html><head><metacharset=utf-8><title>Translate flag test: Default</title></head><body><p>The<spantranslate=no>World Wide Web Consortium</span> is      making the World Wide Web worldwide!</p></body></html>

8.3.1 Definition

TheLocalization Note data category is used to communicate notes to localizers about a particular item of content.

This data category can be used for several purposes, including, but not limited to:

• Tell the translator how to translate parts of the content

• Expand on the meaning or contextual usage of a specific element, such as what a variable refers to or how a string will be used in the user interface

• Clarify ambiguity and show relationships between items sufficiently to allow correct translation (e.g., in many languages it is impossible to translate the word"enabled" in isolation without knowing the gender, number, and case of the thing it refers to.)

• Indicate why a piece of text is emphasized (important, sarcastic, etc.)

Two types of informative notes are needed:

• An alert contains information that the translator has to read before translating a piece of text. Example: an instruction to the translator to leave parts of the text in the source language.

• A description provides useful background information that the translator will refer to only if they wish. Example: a clarification of ambiguity in the source text.

Editing tools may offer an easy way to create this type of information. Translation tools can be made to recognize the difference between these two types of localization notes, and present the information to translators in different ways.

8.3.2 Implementation

TheLocalization Note data category can be expressed with global rules, or locally on an individual element. For elements, the data category informationinherits to the textual content of the element,including child elements, butexcluding attributes.

GLOBAL: ThelocNoteRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredlocNoteType attribute with the value "description" or "alert".

• Exactly one of the following:

  • AlocNote element that contains the note itself and allows forlocal ITS markup.

  • AlocNotePointer attribute that contains arelative selector pointing to a node that holds the localization note.

  • AlocNoteRef attribute that contains an IRI referring to the location of the localization note.

  • AlocNoteRefPointer attribute that contains arelative selector pointing to a node that holds the IRI referring to the location of the localization note.

<myRes><head><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"its:translate="no"><its:locNoteRulelocNoteType="alert"selector="//msg[@id='DisableInfo']"><its:locNote>The variable {0} has three possible values: 'printer', 'stacker' and 'stapler          options'.</its:locNote></its:locNoteRule></its:rules></head><body><msgid="DisableInfo">The {0} has been disabled.</msg></body></myRes>

<Res><prolog><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:translateRuleselector="//msg/notes"translate="no"/><its:locNoteRulelocNoteType="description"selector="//msg/data"locNotePointer="../notes"/></its:rules></prolog><body><msgid="FileNotFound"><notes>Indicates that the resource file {0} could not be loaded.</notes><data>Cannot find the file {0}.</data></msg><msgid="DivByZero"><notes>A division by 0 was going to be computed.</notes><data>Invalid parameter.</data></msg></body></Res>

<myRes><head><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:locNoteRulelocNoteType="description"selector="//msg[@id='NotFound']"locNoteRef="ErrorsInfo.html#NotFound"/></its:rules></head><body><msgid="NotFound">Cannot find {0} on {1}.</msg></body></myRes>

<dataFile><prolog><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:locNoteRulelocNoteType="description"selector="//data"locNoteRefPointer="../@noteFile"/></its:rules></prolog><body><stringid="FileNotFound"noteFile="Comments.html#FileNotFound"><data>Cannot find the file {0}.</data></string><stringid="DivByZero"noteFile="Comments.html#DivByZero"><data>Invalid parameter.</data></string></body></dataFile>

LOCAL: The following local markup is available for theLocalization Note data category:

• Exactly one of the following:

  • AlocNote attribute that contains the note itself.

  • AlocNoteRef attribute that contains an IRI referring to the location of the localization note.

• An optionallocNoteType attribute with the value "description" or "alert". If thelocNoteType attribute is not present, the type of localization note will be assumed to be "description".

<msgListxmlns:its="http://www.w3.org/2005/11/its"xml:space="preserve"its:version="2.0"><dataname="LISTFILTERS_VARIANT"its:locNote="Keep the leading space!"its:locNoteType="alert"><value> Variant {0} = {1} ({2})</value></data><dataits:locNote="%1\$s is the original text's date in the format YYYY-MM-DD HH:MM always in GMT"><value>Translated from English content dated<spanid="version-info">%1\$s</span> GMT.</value></data></msgList>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>LocNote test: Default</title></head><body><p>This is a<spanits-loc-note="Check with terminology engineer"its-loc-note-type=alert>     motherboard</span>.</p></body></html>

8.4.1 Definition

TheTerminology data category is used to mark terms and optionally associate them with information, such as definitions. This helps to increase consistency across different parts of the documentation. It is also helpful for translation.

8.4.2 Implementation

TheTerminology data category can be expressed with global rules, or locally on an individual element. There is no inheritance. The default is that neither elements nor attributes are terms.

GLOBAL: ThetermRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredterm attribute with the value "yes" or "no".

• Zero or one of the following:

  • AtermInfoPointer attribute that contains arelative selector pointing to a node that holds the terminology information.

  • AtermInfoRef attribute that contains an IRI referring to the resource providing information about the term.

  • AtermInfoRefPointer attribute that contains arelative selector pointing to a node that holds the IRI referring to the location of the terminology information.

<text><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:termRuleselector="//term"term="yes"termInfoPointer="id(@def)"/></its:rules><p>We may define<termdef="TDPV">discoursal point of view</term> as<glossxml:id="TDPV">the relationship, expressed through discourse structure, between the    implied author or some other addresser, and the fiction.</gloss></p></text>

<text><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:termRuleselector="//term[1]"term="yes"termInfoRef="#TDPV"/></its:rules><p>We may define<term>discoursal point of view</term>as<glossxml:id="TDPV">the relationship, expressed through discoursestructure, between the implied author or some other addresser,and the fiction.</gloss></p></text>

<text><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:termRuleselector="//term"term="yes"termInfoRefPointer="@target"/></its:rules><p>We may define<termtarget="#TDPV">discoursal point of view</term>as<glossxml:id="TDPV">the relationship, expressed through discoursestructure, between the implied author or some other addresser,and the fiction.</gloss></p></text>

LOCAL: The following local markup is available for theTerminology data category:

• Aterm attribute with the value "yes" or "no".

• An optionaltermInfoRef attribute that contains an IRI referring to the resource providing information about the term.

• An optionaltermConfidence attribute with the value of a rational number in the interval 0 to 1 (inclusive). The value follows theXML Schema double data type with the constraining facetsminInclusive set to 0 andmaxInclusive set to 1.termConfidence represents the confidence of the agents producing the annotation that the annotated unit is a term or not. 1 represents the highest level of confidence.termConfidence does not provide confidence information related totermInfoRef.

Any node selected by the terminology data category with thetermConfidence attribute specifiedMUST be contained in an element with theannotatorsRef (or in HTMLits-annotators-ref) attribute specified for theTerminology data category. SeeSection 5.7: ITS Tools Annotation for more information.

<bookits:version="2.0"xmlns:its="http://www.w3.org/2005/11/its"its:annotatorsRef="terminology|http://example.com/term-tool"><head>...</head><body>    ...<p>And he said: you need a new<quoteits:term="yes"its:termInfoRef="http://www.directron.com/motherboards1.html"its:termConfidence="0.5">motherboard</quote></p>    ...</body></book>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Terminology test: default</title></head><body><p>We need a new<spanits-term=yes>motherboard</span></p></body></html>

Note:

At the time of writing, enhancements are being discussed in the context of HTML5 that are expected to change the approach to marking upDirectionality, in particular to support content where directionality needs to be isolated from that of surrounding content. However, these enhancements are not finalized yet. This section therefore reflects directionality markup in[HTML 4.01]; enhancements in HTML5 will be reflected in a future revision.

8.5.1 Definition

TheDirectionality data category allows the user to specify the base writing direction of blocks, embeddings, and overrides for the Unicode bidirectional algorithm. It has four values: "ltr", "rtl", "lro" and "rlo".

8.5.2 Implementation

TheDirectionality data category can be expressed with global rules, or locally on an individual element. For elements, the data category informationinherits to the textual content of the element,including child elements and attributes. The default is that both elements and attributes have the directionality of left-to-right.

GLOBAL: ThedirRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requireddir attribute with the value "ltr", "rtl", "lro" or "rlo".

<textxml:lang="en"><body><par>In Hebrew, the title<quotexml:lang="he"direction="rtlText">פעילות      הבינאום, W3C</quote> means "Internationalization Activity, W3C",      and the order of characters is<bdodirection='rtlText'>פעילות           הבינאום, W3C</bdo>.</par></body></text>

<its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:dirRuledir="rtl"selector="//*[@direction='rtlText']"/><its:dirRuledir="rlo"selector="//bdo[@direction='rtlText']"/></its:rules>

LOCAL: The following local markup is available for theDirectionality data category:

• Adir attribute with the value "ltr", "rtl", "lro" or "rlo".

<textxml:lang="en"xmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><body><par>In Arabic, the title<quotexml:lang="ar"its:dir="rtl">نشاط التدويل،        W3C</quote> means "Internationalization Activity, W3C".</par></body></text>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Dir test: Default</title></head><body><p>In Arabic, the title<qdir=rtllang=ar>نشاط التدويل، W3C</q>      means "Internationalization Activity, W3C".</p></body></html>

8.6.1 Definition

The elementlangRule is used to express the language of a given piece of content. ThelangPointer attribute points to the markup that expresses the language of the text selected by the selector attribute. This markupMUST use values that conform to[BCP47]. The recommended way to specify language identification is to usexml:lang in XML, andlang in HTML. ThelangRule element is intended only as a fall-back mechanism for documents where language is identified with another construct.

<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:langRuleselector="//p"langPointer="@mylangattribute"/></its:rules>

8.6.2 Implementation

TheLanguage Information data category can be expressed only with global rules. For elements, the data category informationinherits to the textual content of the element,including child elements and attributes. There is no default.

GLOBAL: ThelangRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredlangPointer attribute that contains arelative selector pointing to a node that contains language information. If the attributexml:lang is present orlang in HTML for the selected node, the value of thexml:lang attribute orlang in HTMLMUST take precedence over thelangPointer value.

8.7.1 Definition

TheElements Within Text data category reveals if and how an element affects the way text content behaves from a linguistic viewpoint. This information is for example relevant to provide basic text segmentation hints for tools such as translation memory systems. The values associated with this data category are:

• "yes": The element and its content are part of the flow of its parent element. For example the elementstrong in[XHTML 1.0]:

  <strong>Appaloosa horses</strong> have spotted coats.

• "nested": The element is part of the flow of its parent element, its content is an independent flow. For example the elementfn in[DITA 1.0]:

  Palouse horses<fn>A Palouse horse is the same as an Appaloosa.</fn> have spotted coats.

• "no": The element splits the text flow of its parent element and its content is an independent text flow. For example the elementp when inside the elementli in DITA or XHTML:

  <li>Palouse horses: <p>They have spotted coats.</p> <p>They have been bred by the Nez Perce.</p> </li>

8.7.2 Implementation

TheElements Within Text data category can be expressed with global rules, or locally on an individual element. There is no inheritance.

For XML: The default is that elements are not within text.

For HTML: The default is that elements are not within text, with the following exceptions:

• For the elements that are part of theHTML5 phrasing content the default iswithinText="yes", with the following exceptions:

  • For the elementsiframe,noscript,script andtextarea the default iswithinText="nested".

<!DOCTYPE html><html><head><metacharset=utf-8><title>Elements within Text defaults for HTML5</title></head><body><p>The element p is not within text. But<em>the element em is</em>.</p><p>A button<buttononclick="display()">Click Here</button> is also within text.But<textarea>The content of textarea</textarea> is nested.</p>Some additional text...<script><!--function display() {alert("The script element is nested."); }//--></script><noscript>The noscript element is nested.</noscript></body></html>

GLOBAL: ThewithinTextRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredwithinText attribute with the value "yes", "no" or "nested".

<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:withinTextRulewithinText="yes"selector="//b | //em | //i"/></its:rules>

LOCAL: The following local markup is available for theElements Within Text data category:

• AwithinText attribute with the values "yes", "no" or "nested".

<textxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><body><par>Text with<boldits:withinText="yes">bold</bold>.</par></body></text>

<!DOCTYPE html><html><head><metacharset=utf-8><title>Within text test: Default</title></head><body><p>Text with<spanits-within-text='yes'>bold</span>.</p></body></html>

8.8.1 Definition

TheDomain data category is used to identify the topic or subject of content. Such information allows for more relevant linguistic choices during various processes.

Examples of usage include:

• Allowing machine translation systems to select the most appropriate engine and rules to translate the content.

• Providing a general indication of what terminology collection is most suitable for use by translators.

This data category addresses various challenges:

• Often domain-related information already exists in the document (e.g., keywords in the HTMLmeta element). TheDomain data category provides a mechanism to point to this information.

• There are many flat or structured lists of domain related values, keywords, key phrases, classification codes, ontologies, etc. TheDomain data category does not propose its own given list. Instead it provides a mapping mechanism to associate the values in the document with the values used by the consumer tool.

8.8.2 Implementation

TheDomain data category can be expressed only with global rules. For elements, the data category informationinherits to the textual content of the element,including child elements and attributes. There is no default.

The information provided by this data category is a comma-separated list of one or more values, which is obtained by applying the following algorithm:

• STEP 1: Set the initial value of the resulting string as an empty string.

• STEP 2: Get the list of nodes resulting of the evaluation of thedomainPointer attribute.

• STEP 3: For each node:

  • STEP 3-1: If the node value contains a COMMA (U+002C):

    • STEP 3-1-1: Split the node value into separate strings using the COMMA (U+002C) as separator.

    • STEP 3-1-2: For each string:

      • STEP 3-1-2-1: Trim the leading and trailing white spaces of the string.

      • STEP 3-1-2-2: If the first character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

      • STEP 3-1-2-3: If the last character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

      • STEP 3-1-2-4: If the value is empty: Go to STEP 3-1-2.

      • STEP 3-1-2-5: Check thedomainMapping attribute to see if there is a mapping set for the string:

        • STEP 3-1-2-5-1. If a mapping is found: Add the corresponding value to the result string.

        • STEP 3-1-2-5-2. Else (if no mapping is found): Add the string to the result string.

  • STEP 3-2: Else (if the node value does not contain a COMMA (U+002C)):

    • STEP 3-2-1: Trim the leading and trailing white spaces of the string.

    • STEP 3-2-2: If the first character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

    • STEP 3-2-3: If the last character of the value is an APOSTROPHE (U+0027) or a QUOTATION MARK (U+0022): Remove it.

    • STEP 3-2-4: If the value is empty: Go to STEP 3.

    • STEP 3-2-5: Check if there is a mapping for the string:

      • STEP 3-2-5-1: If a mapping is found: Add the corresponding value to the result string.

      • STEP 3-2-5-2: Else (if no mapping is found): Add the string (in its original cases) to the result string.

• STEP 4: Remove duplicated values from the resulting string.

• STEP 5: Return the resulting string.

GLOBAL: ThedomainRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requireddomainPointer attribute that contains arelative selector pointing to a node that contains the domain information.

• An optionaldomainMapping attribute that contains a comma separated list of mappings between values in the content and consumer tool specific values. The left part of the pair corresponds to the source content and is unique within the mapping and case-sensitive. The right part of the mapping belongs to the consumer tool. Several left parts can map to a single right part. The values in the left or the right part of the mapping may contain spaces; in that case theyMUST be delimited by quotation marks, that is pairs of APOSTROPHE (U+0027) or QUOTATION MARK (U+0022).

<its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"xmlns:h="http://www.w3.org/1999/xhtml"><its:domainRuleselector="/h:html/h:body"domainPointer="/h:html/h:head/h:meta[@name='keywords']/@content"/></its:rules>

<its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"xmlns:h="http://www.w3.org/1999/xhtml"><its:domainRuleselector="/h:html/h:body"domainPointer="/h:html/h:head/h:meta[@name='dcterms.subject' or @name='keywords']/@content"domainMapping="automotive auto, medical medicine, 'criminal law' law, 'property law' law"/></its:rules>

8.9.1 Definition

TheText Analysis data category is used to annotate content with lexical or conceptual information for the purpose of contextual disambiguation. This information can be provided by so-called text analysis software agents such as named entity recognizers, lexical concept disambiguators, etc., and is represented by either string valued or IRI references to possible resource descriptions. Example: A named entity recognizer provides the information that the string "Dublin" in a certain context denotes a town in Ireland.

While text analysis can be done by humans, this data category istargeted more at software agents.

The information can be used for several purposes, including, but not limited to:

• Informing a human agent such as a translator that a certain fragment of textual content (so-called “text analysis target”) may follow specific translation rules. Examples: proper names, brands, or officially regulated expressions.

• Informing a software agent such as a content management system about the conceptual type of a textual entity to enable special processing. Examples: places, personal names, product names, or geographic names, chemical compounds, and protein names that are situated in a specific index.

The data category provides three pieces of annotation: confidence, entity type or concept class, entity identifier or concept identifier as specified in the following table.

8.9.2 Implementation

TheText Analysisdata category can be expressed with global rules, or locally on anindividual element. There is no inheritance.

GLOBAL: ThetextAnalysisRuleelement contains the following:

• A requiredselector attribute that contains anabsolute selector that selects the nodes to which this rule applies.

• At least one of the following:

  • AtaClassRefPointerattribute that contains arelative selectorpointing to a node that holds an IRI, which implements theentity type / concept classinformation.

  • Exactly one of the following:

    • When using identificationmode 1: AtaSourcePointerattribute that contains arelative selector to a node that holds theidentifier of the collection source;and ataIdentPointerattribute that contains a relative selector to a node thatholds theidentifier of the concept in the collection.

    • When using identificationmode 2: AtaIdentRefPointerattribute that contains arelative selector pointing to a node that holds an IRI that holds theidentifierof the text analysis target.

For an example, seeExample 54.

LOCAL: The following local markup is available for theText Analysisdata category:

• An optionaltaConfidence attribute that implements thetext analysis confidence.

• At least one of the following:

  • AtaClassRefattribute that holds an IRI, which implements theEntity type / concept classinformation.

  • Exactly one of the following:

    • When using identificationmode 1: AtaSource attribute that holds theidentifier of the collection source,and ataIdent attribute that holds theidentifier of the concept in thecollection.

    • When using identificationmode 2: AtaIdentRef attribute that holds theidentifier of the text analysistarget.

Any node selected by theText Analysisdata category with thetaConfidenceattribute specifiedMUSTbe contained in an element with theannotatorsRef(or in HTMLits-annotators-ref) attribute specified for theText Analysisdata category. For more information, seeSection 5.7: ITS Tools Annotation.

<!DOCTYPE html><htmllang="en"its-annotators-ref="text-analysis|http://enrycher.ijs.si"><head><metacharset="utf-8" /><title>Text analysis: Local Test</title></head><body><p><spanits-ta-confidence="0.7"its-ta-class-ref="http://nerd.eurecom.fr/ontology#Location"its-ta-ident-ref="http://dbpedia.org/resource/Dublin">Dublin</span>       is the<spanits-ta-source="Wordnet3.0"its-ta-ident="301467919"its-ta-confidence="0.5"          >capital</span> of Ireland.</p></body></html>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><linkhref=EX-text-analysis-html5-rdfa.xmlrel=its-rules><title>Entity: Local Test</title></head><body><p><spanproperty="http://xmlns.com/foaf/0.1/name"about="http://dbpedia.org/resource/Dublin"typeof="http:/nerd.eurecom.fr/ontology#Location">Dublin</span> is       the capital of Ireland.</p></body></html>

<its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:textAnalysisRuleselector="//*[@typeof and @about]"taClassRefPointer="@typeof"taIdentRefPointer="@about"/></its:rules>

8.10.1 Definition

TheLocale Filter data category specifies that a node is only applicable to certain locales.

This data category can be used for several purposes, including, but not limited to:

• Including a legal notice only in locales for certain regions.

• Dropping editorial notes from all localized output.

TheLocale Filter data category associates with each selected node a filter type and a list of extended language ranges conforming to[BCP47].

The list is comma-separated and can include the wildcard extended language range "*". The list can also be empty. Whitespace surrounding language ranges is ignored.

The type can take the values "include" or "exclude":

• A single wildcard "*" with a type "include" indicates that the selected content applies to all locales.

• A single wildcard "*" with a type "exclude" indicates that the selected content applies to no locale.

• An empty string with a type "include" indicates that the selected content applies to no locale.

• An empty string with a type "exclude" indicates that the selected content applies to all locales.

• Otherwise, with a type "include", the selected content applies to the locales for which the language tag has a match in the list when using the Extended Filtering algorithm defined in[BCP47].

• If, instead, the type is "exclude", the selected content applies to the locales for which the language tag does not have a match in the list when using the Extended Filtering algorithm defined in[BCP47].

8.10.2 Implementation

TheLocale Filter data category can be expressed with global rules, or locally on an individual element. For elements, the data category informationinherits to the textual content of the element,including child elements and attributes. The default is that the language range is "*" and the type is "include".

GLOBAL: ThelocaleFilterRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredlocaleFilterList attribute with a comma-separated list of extended language ranges, or an empty string value.

• An optionallocaleFilterType attribute with a value "include" or "exclude".

<bookxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><info><its:rulesversion="2.0"><its:localeFilterRuleselector="//legalnotice[@role='Canada']"localeFilterList="*-CA"/><its:localeFilterRuleselector="//legalnotice[@role='nonCanada']"localeFilterList="*-CA"localeFilterType="exclude"/><its:localeFilterRuleselector="//remark"localeFilterList=""/></its:rules><legalnoticerole="Canada"><para>This notice is only for Canadian locales.</para></legalnotice><legalnoticerole="nonCanada"><para>This notice is for locales that are non-Canadian locales.</para></legalnotice><remark>Note: This section will be written later.</remark></info></book>

LOCAL: The following local markup is available for theLocale Filter data category:

• AlocaleFilterList attribute with a comma-separated list of extended language ranges, or an empty string value.

• An optionallocaleFilterType attribute with a value "include" or "exclude".

<!DOCTYPE html><html><head><metacharset=utf-8><title>Locale filter</title></head><body><divits-locale-filter-list="*-ca"><p>Text for Canadian locales.</p></div><divits-locale-filter-list="*-ca"its-locale-filter-type="exclude"><p>Text for non-Canadian locales.</p></div></body></html>

<bookxmlns:its="http://www.w3.org/2005/11/its"><info><legalnoticeits:localeFilterList="en-CA, fr-CA"><para>This legal notice is only for English and French Canadian locales.</para></legalnotice></info></book>

8.11.1 Definition

TheProvenance data category is used to communicate the identity of agents that have been involved in the translation of the content or the revision of the translated content. This allows translation and translation revision consumers, such as post-editors, translation quality reviewers, or localization workflow managers, to assess how the performance of these agents may impact the quality of the translation. Translation and translation revision agents can be identified as a person, a piece of software or an organization that has been involved in providing a translation that resulted in the selected content.

This data category offers three types of information. First, it allows identification of translation agents. Second, it allows identification of revision agents. Third, if provenance information is needed that includes temporal or sequence information about translation processes (e.g. multiple revision cycles) or requires agents that support a wider range of activities, the data category offers a mechanism to refer to external provenance information.

Translation or translation revision tools, such as machine translation engines or computer assisted translation tools, may offer an easy way to create this information. Translation tools can then present this information to post-editors or translation workflow managers. Web applications may to present such information to consumers of translated documents.

The data category defines seven pieces of information:

8.11.2 Implementation

TheProvenance data category can be expressed with global rules, or locally on individual elements. For elements, the data category informationinherits to the textual content of the element,including child elements and attributes.

GLOBAL: TheprovRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• AprovenanceRecordsRefPointer attribute that contains arelative selector pointing to a node containing a list ofprovenance records. These are related to the content selected via theselector attribute.

<textxmlns:dc="http://purl.org/dc/elements/1.1/"xmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><dc:creator>John Doe</dc:creator><its:provenanceRecordsxml:id="pr1"><its:provenanceRecordtoolRef="http://www.example.onlinemtex.com/2012/7/25/wsdl/"org="acme-CAT-v2.3"revToolRef="http://www.mycat.com/v1.0/download"revOrg="acme-CAT-v2.3"provRef="http://www.examplelsp.com/excontent987/production/prov/e6354"/></its:provenanceRecords><its:provenanceRecordsxml:id="pr2"><its:provenanceRecordperson="John Doe"orgRef="http://www.legaltrans-ex.com"revPerson="Tommy Atkins"revOrgRef="http://www.example.myorg.com"provRef="http://www.example.myorg.com/job-12-7-15-X31/reviewed/prov/re8573469"/><its:provenanceRecordrevPerson="John Smith"revOrgRef="http://john-smith.qa.example.com"/></its:provenanceRecords><its:rulesversion="2.0"><its:provRuleselector="//*[@ref]"provenanceRecordsRefPointer="@ref"/></its:rules><title>Translation Revision Provenance Agent: Global Test in XML</title><body><parref="#pr1"> This paragraph was translated from the machine.</par><legalnoticepostediting-by="http://www.example.myorg.com"ref="#pr2">This text was      translated directly by a person.</legalnotice></body></text>

LOCAL: Using the inline markup to represent the data category locally is limited to a single occurrence for a given content (e.g., one cannot have differenttoolRef attributes applied to the same span of text because the inner-most one would override the others). A localstandoff markup is provided to allow such cases.

The following local markup is available for theProvenance data category:

• Either (inline markup): at least one of the following attributes:

  • Aperson orpersonRef attribute that implements thehuman provenance information.

  • Anorg ororgRef attribute that implements theorganizational provenance information.

  • Atool ortoolRef attribute that implements thetool-related provenance information.

  • ArevPerson orrevPersonRef attribute that implements thehuman revision provenance information.

  • ArevOrg orrevOrgRef attribute that implements theorganizational revision provenance information.

  • ArevTool orrevToolRef attribute that implements thetool-related revision provenance information.

  • AprovRef attribute that implements thereference to external provenance descriptions.

• Or (standoff markup):

  • AprovenanceRecordsRef attribute. Its value is an IRI pointing to theprovenanceRecords element containing the list ofprovenance records related to this content.

  • An elementprovenanceRecords, which contains:

    • One or more elementsprovenanceRecord, each of which contains at least one of the following attributes:

      • Aperson orpersonRef attribute that implements thehuman provenance information.

      • Anorg ororgRef attribute that implements theorganizational provenance information.

      • Atool ortoolRef attribute that implements thetool-related provenance information.

      • ArevPerson orrevPersonRef attribute that implements thehuman revision provenance information.

      • ArevOrg orrevOrgRef attribute that implements theorganizational revision provenance information.

      • ArevTool orrevToolRef attribute that implements thetool-related revision provenance information.

      • AprovRef attribute that implements thereference to external provenance descriptions.

  Note:

  Ideally the order ofprovenanceRecord elements within aprovenanceRecords element reflects the order with which they were added to the document, with the most recently added one listed first.

  When the attributesperson,personRef,org,orgRef,tool,toolRef,revPerson,revPersonRef,revOrg,revOrgRef,revTool,revToolRef andprovRef are used in a standoff manner, the information they carry pertains to the content of the element that refers to the standoff annotation, not to the content of the elementprovenanceRecord where they are declared.

  In HTML the standoff markupMUST either be stored inside ascript element in the same HTML document, or be linked from anyprovenanceRecordsRef to an external XML or HTML file with the standoff inside. If standoff is inside ascript element that elementMUST have atype attribute with the valueapplication/its+xml. Itsid attributeMUST be set to the same value as thexml:id attribute of theprovenanceRecords element it contains.

<textxmlns:dc="http://purl.org/dc/elements/1.1/"xmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><title>Translation Revision Provenance Agent: Local Test in XML</title><body><parits:toolRef="http://www.onlinemtex.com/2012/7/25/wsdl/"its:org="acme-CAT-v2.3"its:revToolRef="http://www.mycat.com/v1.0/download"its:revOrg="acme-CAT-v2.3"its:provRef="http://www.example.lsp1.com/prov/e6354 http://www.example.lsp2.com/prov/e7738"      >This paragraph was translated from the machine.</par><legalnoticeits:person="John Doe"its:orgRef="http://www.legaltrans-ex.com/"its:provRef="http://www.examplelsp.com/excontent987/legal/prov/e6354"its:revPerson="Tommy Atkins"its:revOrgRef="http://www.example.myorg.com"      >This text was translated directly by a person.</legalnotice></body></text>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Provenance Agent: Local Test in HTML5</title></head><body><pits-tool-ref="http://www.onlinemtex.com/2012/7/25/wsdl/"its-org="acme-CAT-v2.3"its-prov-ref="http://www.examplelsp.com/excontent987/production/prov/e6354"its-rev-org="acme-CAT-v2.3"      >This paragraph was translated from the machine.</p><pclass="legal-notice"its-person="John Doe"its-org-ref="http://www.legaltrans-ex.com/"its-prov-ref="http://www.examplelsp.com/excontent987/legal/prov/e6354"its-rev-person="Tommy Atkins"its-rev-org-ref="http://www.example.myorg.com"     >This text was translated directly by a person.</p></body></html>

<!DOCTYPE html><html><head><metacharset=utf-8><title>Test</title><scriptid=pr1type=application/its+xml><its:provenanceRecordsxml:id="pr1"xmlns:its="http://www.w3.org/2005/11/its"><its:provenanceRecordtoolRef="http://www.onlinemtex.com/2012/7/25/wsdl/"org="acme-CAT-v2.3"provRef="http://www.examplelsp.com/excontent987/production/prov/e6354"revToolRef="http://www.mycat.com/v1.0/download"revOrg="acme-CAT-v2.3" /></its:provenanceRecords></script><scriptid=pr2type=application/its+xml><its:provenanceRecordsxml:id="pr2"xmlns:its="http://www.w3.org/2005/11/its"><its:provenanceRecordperson="John Doe"orgRef="http://www.legaltrans-ex.com/"provRef="http://www.examplelsp.com/excontent987/legal/prov/e6354"revPerson="Tommy Atkins"revOrgRef="http://www.example.myorg.com" /><its:provenanceRecordrevPerson="John Smith"revOrgRef="http://john-smith.qa.example.com" /></its:provenanceRecords></script></head><body><pits-provenance-records-ref="#pr1">This paragraph was translated from the machine.</p><pits-provenance-records-ref="#pr2">This text was translated directly by a person.</p></body></html>

8.12.1 Definition

TheExternal Resource data category indicates that a node represents or references potentially translatable data in a resource outside the document. Examples of such resources are external images and audio or video files.

8.12.2 Implementation

TheExternal Resource data category can be expressed only with global rules. There is no inheritance. There is no default.

GLOBAL: TheexternalResourceRefRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredexternalResourceRefPointer attribute that contains arelative selector pointing to a node that provides the IRI of the external resource.

<docxmlns:its="http://www.w3.org/2005/11/its"xmlns:db="http://docbook.org/ns/docbook"><its:rulesversion="2.0"><its:externalResourceRefRuleselector="//db:imagedata | //db:audiodata |  //db:videodata"externalResourceRefPointer="@fileref"/></its:rules><db:mediaobject><db:videoobject><db:videodatafileref="movie.avi"/></db:videoobject><db:imageobject><db:imagedatafileref="movie-frame.gif"/></db:imageobject><db:textobject><db:para>This video illustrates the proper way to assemble an inverting        time distortion device.</db:para><db:warning><db:para> It is imperative that the primary and secondary temporal          couplings not be mounted in the wrong order. Temporal catastrophe is          the likely result. The future you destroy may be your own.</db:para></db:warning></db:textobject></db:mediaobject></doc>

<its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"xmlns:html="http://www.w3.org/1999/xhtml"><its:externalResourceRefRuleselector="//html:video/@src"externalResourceRefPointer="."/><its:externalResourceRefRuleselector="//html:video/@poster"externalResourceRefPointer="."/></its:rules>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Video element example</title></head><body><videoheight=360poster=http://www.example.com/video-image.pngsrc=http://www.example.com/video/v2.mpwidth=640><p>If your browser doesn't support         the<code>video</code> element, you can<ahref=http://www.example.com/video/v2.mp>download the video</a> instead.</p></video></body></html>

8.13.1 Definition

Some formats, such as those designed for localization or for multilingual resources, hold the same content in different languages inside a single document. TheTarget Pointer data category is used to associate the node of a given source content (i.e., the content to be translated) and the node of its corresponding target content (i.e., the source content translated into a given target language).

This specification makes no provision regarding the presence of the target nodes or their content: A target node may or may not exist and it may or may not have content.

This data category can be used for several purposes, including but not limited to:

• Extract the source content to translate and put back the translation at its proper location.

• Compare source and target content for quality verification.

• Reuse existing translations when localizing the new version of an existing document.

• Access aligned bi-lingual content to build memories, or to train machine translation engines.

8.13.2 Implementation

TheTarget Pointer data category can be expressed only with global rules. There is no inheritance. There is no default.

GLOBAL: ThetargetPointerRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredtargetPointer attribute. It contains arelative selector that points to the node for the target content corresponding to the selected source node.

<file><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:translateRuleselector="/file"translate="no"/><its:translateRuleselector="//source"translate="yes"/><its:targetPointerRuleselector="//source"targetPointer="../target"/></its:rules><entryid="one"><source>Remember last folder</source><target/></entry><entryid="two"><source>Custom file filter:</source><target/></entry></file>

8.14.1 Definition

TheID Value data category indicates a value that can be used as unique identifier for a given part of the content.

The recommended way to specify a unique identifier is to usexml:id[XML ID] orid in HTML (See the best practice “Defining markup for unique identifiers” from[XML i18n BP]). TheidValueRule element is intended only as a fall-back mechanism for documents in which unique identifiers are available with another construct.

Providing a unique identifier that is maintained in the original document can be useful for several purposes, for example:

• Allow automated alignment between different versions of the source document, or between source and translated documents.

• Improve the confidence in leveraged translation for exact matches.

• Provide backtracking information between displayed text and source material when testing or debugging.

8.14.2 Implementation

TheID Value data category can be expressed only with global rules. There is no inheritance. There is no default.

GLOBAL: TheidValueRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredidValue attribute. It contains any XPath expression; the context for the evaluation of the XPath expression is the same as forrelative selectors. The evaluation of the XPath expression constructs a string corresponding to the identifier of the node to which this rule applies. The identifierMUST be unique at least within the document. If the attributexml:id is present orid in HTML for the selected node, the value of thexml:id attribute orid in HTMLMUST take precedence over theidValue value.

<?xml version="1.0"?><resources><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:translateRuletranslate="no"selector="/resources"/><its:translateRuletranslate="yes"selector="//text"/><its:idValueRuleselector="//text"idValue="../@name"/></its:rules><entryname="btn.OK"><text>OK</text><pos>1, 1</pos><trig>sendOK</trig></entry><entryname="btn.CANCEL"><text>Cancel</text><pos>2, 1</pos><trig>cancelAll</trig></entry></resources>

<?xml version="1.0"?><doc><its:rulesversion="2.0"xmlns:its="http://www.w3.org/2005/11/its"><its:idValueRuleselector="//text"idValue="concat(../@name, '_t')"/><its:idValueRuleselector="//desc"idValue="concat(../@name, '_d')"/></its:rules><msgname="settingsMissing"><text>Can't find settings file.</text><desc>The module cannot find the default settings file. You need to      re-initialize the system.</desc></msg></doc>

<?xml version="1.0"?><file><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:idValueRuleselector="//res"idValue="@name"/></its:rules><resname="retryBtn"xml:id="btnAgain">Try Again</res><resname="retryTip">click this to re-run the process with the current    settings.</res></file>

8.15.1 Definition

ThePreserve Space data category indicates how whitespace is to be handled in content. The possible values for this data category are "default" and "preserve" and carry the same meaning as the corresponding values of thexml:space attribute. The default value is "default". The Preserve Space data category does not apply to HTML documents in HTML syntax.

8.15.2 Implementation

ThePreserve Space data category can be expressed with global rules, or locally using thexml:space attribute. For elements, the data category informationinherits to the textual content of the element,including child elements and attributes.

GLOBAL: ThepreserveSpaceRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredspace attribute with the value "default" or "preserve".

<book><info><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:preserveSpaceRuleselector="//verse"space="preserve"/></its:rules></info><verse>’Twas brillig, and the slithy toves Did gyre and gimble in the wabe;All mimsy were the borogoves, And the mome raths outgrabe.</verse></book>

LOCAL: Thexml:space attribute, as defined in section 2.10 of[XML 1.0], maps exactly to thePreserve Space data category.

<book><versexml:space="preserve">'Twas brillig, and the slithy toves Did gyre and gimble in the wabe;All mimsy were the borogoves, And the mome raths outgrabe.</verse></book>

8.16.1 Definition

TheLocalization Quality Issue data category is used to express information related to localization quality assessment tasks. Such tasks can be conducted on the translation of some source content (such as a text or an image) into a target language or on the source content itself where its quality may impact on the localization process.

This data category can be used in a number of ways, including the following example scenarios:

• A human reviewer working with a web-based tool adds quality markup manually in a text editor, including comments and suggestions, to localized content as part of the review process. A subsequent process examines this markup to ensure that changes were made.

• A fully automatic quality checking tool flags a number of potential quality issues in an XML or HTML file and marks them up using ITS 2.0 markup. A human reviewer then uses another tool to examine this markup and decide whether the file needs to receive more extensive review or be passed on for further processing without a further manual review stage.

• A quality assessment process identifies a number of issues and adds the ITS markup to a rendered HTML preview of an XML file along with CSS styling that highlights these issues. The resulting HTML file is then sent back to the translator to assist his or her revision efforts.

The data category defines five pieces of information:

8.16.2 Implementation

TheLocalization Quality Issue data category can be expressed with global rules, or locally on individual elements. For elements, the data category informationinherits to the textual content of the element,including child elements, but excluding attributes.

GLOBAL: ThelocQualityIssueRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• Either (in parallel tolocal inline markup)

  • At least one of the following attributes:

    • AlocQualityIssueType attribute that implements thetype information.

    • AlocQualityIssueComment attribute that implements thecomment information.

  • An optionallocQualityIssueSeverity attribute that implements theseverity information.

  • An optionallocQualityIssueProfileRef attribute that implements theprofile reference information.

  • An optionallocQualityIssueEnabled attribute that implements theenabled information.

• Or (standoff markup) exactly one of the following:

  • AlocQualityIssuesRef attribute. Its value is an IRI pointing to thelocQualityIssues element containing thelist of issues related to this content.

  • AlocQualityIssuesRefPointer attribute that contains arelative selector pointing to a node with the exact same semantics aslocQualityIssuesRef.

<?xml version="1.0"?><doc><header><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:locQualityIssueRuleselector="//image[@id='i1']/@text"locQualityIssueType="typographical"locQualityIssueComment="Sentence without capitalization"locQualityIssueSeverity="50"/></its:rules></header><para>Click the button<imageid="i1"src="button.png"text="start button"/>.</para></doc>

<?xml version="1.0"?><docxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><file><header><its:rulesversion="2.0"><its:locQualityIssueRuleselector="//mrk[@type='x-itslq']"locQualityIssuesRefPointer="@ref"/></its:rules></header><unitid="1"><segment><source>This is the content</source><target><mrktype="x-itslq"ref="#lq1">c'es</mrk> le contenu</target></segment><its:locQualityIssuesxml:id="lq1"><its:locQualityIssuelocQualityIssueType="misspelling"locQualityIssueComment="'c'es' is unknown. Could be 'c'est'"locQualityIssueSeverity="50"/><its:locQualityIssuelocQualityIssueType="typographical"locQualityIssueComment="Sentence without capitalization"locQualityIssueSeverity="30"/></its:locQualityIssues></unit></file></doc>

LOCAL: Using inline markup to represent the data category locally is limited to a single occurrence for a given content (e.g. one cannot have differentlocQualityIssueType attributes applied to the same span of text because the inner-most one would override the others). A localstandoff markup is provided to allow such cases.

The following local markup is available for theLocalization Quality Issue data category:

• Either (inline markup):

  • At least one of the following attributes:

    • AlocQualityIssueType attribute that implements thetype information.

    • AlocQualityIssueComment attribute that implements thecomment information.

  • An optionallocQualityIssueSeverity attribute that implements theseverity information.

  • An optionallocQualityIssueProfileRef attribute that implements theprofile reference information.

  • An optionallocQualityIssueEnabled attribute that implements theenabled information.

• Or (standoff markup):

  • AlocQualityIssuesRef attribute. Its value is an IRI pointing to thelocQualityIssues element containing thelist of issues related to this content.

  • An elementlocQualityIssues with axml:id attribute set to the identifier specified in thelocQualityIssuesRef attribute. ThelocQualityIssues element contains:

    • One or more elementslocQualityIssue, each of which contains:

      • At least one of the following attributes:

        • AlocQualityIssueType attribute that implements thetype information.

        • AlocQualityIssueComment attribute that implements thecomment information.

      • An optionallocQualityIssueSeverity attribute that implements theseverity information.

      • An optionallocQualityIssueProfileRef attribute that implements theprofile reference information.

      • An optionallocQualityIssueEnabled attribute that implements theenabled information.

  Note:

  Ideally the order oflocQualityIssue elements within alocQualityIssues element reflects the order with which they were added to the document, with the most recently added one listed first.

  When the attributeslocQualityIssueType,locQualityIssueComment,locQualityIssueSeverity,locQualityIssueProfileRef andlocQualityIssueEnabled are used in a standoff manner, the information they carry pertains to the content of the element that refers to the standoff annotation, not to the content of the elementlocQualityIssue where they are declared.

  In HTML the standoff markupMUST either be stored inside ascript element in the same HTML document, or can be linked from anylocQualityIssuesRef to an external XML or HTML file with the standoff inside. If standoff is inside ascript element, that elementMUST have atype attribute with the valueapplication/its+xml. Itsid attributeMUST be set to the same value as thexml:id attribute of thelocQualityIssues element it contains.

<?xml version="1.0"?><docxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><para><spanits:locQualityIssueType="typographical"its:locQualityIssueComment="Sentence without capitalization"its:locQualityIssueSeverity="50">this</span> is an example</para></doc>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Telharmonium 1897</title><styletype=text/css>         [its-loc-quality-issue-type]{           background-color:yellow;           margin:2px;         }         [its-loc-quality-issue-severity = "100"]{           border: 2px solid red;         }</style></head><body><h1>Telharmonium (1897)</h1><p><spandata-mytool-qacode=named_entity_not_foundits-loc-quality-issue-comment="Should be Thomas Cahill."its-loc-quality-issue-profile-ref=http://example.org/qaMovel/v1its-loc-quality-issue-severity=100its-loc-quality-issue-type=inconsistent-entities>Christian Bale</span>      (1867–1934) conceived of an instrument that could transmit its sound      from a power plant for hundreds of miles to listeners over telegraph wiring.      Beginning in 1889 the sound quality of regular telephone concerts was very poor      on account of the buzzing generated by carbon-granule microphones. As a result      Cahill decided to set a new standard in perfection of sound<spanits-loc-quality-issue-comment="should be 'quality'"its-loc-quality-issue-profile-ref=grammarits-loc-quality-issue-severity=50its-loc-quality-issue-type=misspelling>qulaity</span> with his instrument,      a standard that would not only satisfy listeners but that      would overcome all the flaws of traditional instruments.</p></body></html>

<?xml version="1.0"?><xliffversion="1.2"xmlns="urn:oasis:names:tc:xliff:document:1.2"xmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><fileoriginal="example.doc"source-language="en"datatype="plaintext"><body><trans-unitid="1"><sourcexml:lang="en">This is the content</source><targetxml:lang="fr"><mrkmtype="x-itslq"its:locQualityIssuesRef="#lq1">c'es</mrk> le contenu</target><its:locQualityIssuesxml:id="lq1"><its:locQualityIssuelocQualityIssueType="misspelling"locQualityIssueComment="'c'es' is unknown. Could be 'c'est'"locQualityIssueSeverity="50"/><its:locQualityIssuelocQualityIssueType="typographical"locQualityIssueComment="Sentence without capitalization"locQualityIssueSeverity="30"/></its:locQualityIssues></trans-unit></body></file></xliff>

<!DOCTYPE html><html><head><metacharset=utf-8><title>Test</title><scriptsrc=qaissues.jstype=text/javascript></script><scripttype=application/its+xmlid=lq1><its:locQualityIssuesxml:id="lq1"xmlns:its="http://www.w3.org/2005/11/its"><its:locQualityIssuelocQualityIssueType="misspelling"locQualityIssueComment="'c'es' is unknown. Could be 'c'est'"locQualityIssueSeverity="50"/><its:locQualityIssuelocQualityIssueType="typographical"locQualityIssueComment="Sentence without capitalization"locQualityIssueSeverity="30"/></its:locQualityIssues></script><styletype=text/css>.qaissue { background-color: yellow; }</style></head><bodyonload=addqaissueattrs()><p><spanits-loc-quality-issues-ref=#lq1>c'es</span> le contenu</p></body></html>

8.17.1 Definition

TheLocalization Quality Rating data category is used to express an overall measurement of the localization quality of a document or an item in a document.

This data category allows to specify a quality score or a voting result for a given item or document, as well as to indicate what constitutes a passing score or vote. It also allows pointing to a profile describing the quality assessment model used for the scoring or the voting.

8.17.2 Implementation

TheLocalization Quality Rating data category is only expressed locally on individual elements. The data category informationinherits to the textual content of the element,including child elements, butexcluding attributes.

LOCAL: The following local markup is available for theLocalization Quality Rating data category:

• Exactly one of the following:

  • AlocQualityRatingScore attribute. Its value is a rational number in the interval 0 to 100 (inclusive). The value follows theXML Schema double data type with the constraining facetsminInclusive set to 0 andmaxInclusive set to 100. The higher values represent better quality.

  • AlocQualityRatingVote attribute. Its value is a signed integer with higher values indicating a better vote.

• IflocQualityRatingScore is used:

  • an optionallocQualityRatingScoreThreshold attribute indicating the lowest score that constitutes a passing score in the profile used. Its value is a rational number in the interval 0 to 100 (inclusive). The value follows theXML Schema double data type with the constraining facetsminInclusive set to 0 andmaxInclusive set to 100.

• IflocQualityRatingVote is used:

  • an optionallocQualityRatingVoteThreshold attribute indicating the lowest value that constitutes a passing vote in the profile used. Its value is a signed integer.

• An optionallocQualityRatingProfileRef attribute. Its value is an IRI pointing to the reference document describing the quality assessment model used for the scoring.

<?xml version="1.0"?><docxml:lang='nl'xmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"its:locQualityRatingScore="100"its:locQualityRatingScoreThreshold="95"its:locQualityRatingProfileRef="http://example.org/qaModel/v13"><title>De lotgevallen van Tom Sawyer</title><para>Hij kwam vrij laat te huis, en toen hij voorzichtig het raam insprong,   viel hij in eene hinderlaag, in de persoon van zijne tante, bij wie, toen zij   den staat zag, waarin zijne kleederen verkeerden, het besluit om zijn vrijen   Zaterdag in een gevangenschap met dwangarbeid te veranderen, onherroepelijk   vaststond.</para></doc>

<!DOCTYPE html><htmllang=frits-loc-quality-rating-profile-ref=http://example.org/qaModel/v13its-loc-quality-rating-score=90its-loc-quality-rating-score-threshold=80  ><head><metacharset=utf-8><title>Rikki-tikki-tavi</title></head><body><p>C'est l'histoire de la grande guerre que Rikki-Tikki-Tavi a combattu tout seul,    à travers les salles de bain du grand bungalow au cantonnement Segowlee. Darzee,   le tailbird, l'a aidé, et Chuchundra, le rat musqué, qui ne sort jamais jusqu'au   milieu du plancher, mais se glisse toujours contre la paroi, lui donnait des   conseils, mais Rikki-Tikki-Tavi fait le véritable combat.</p></body></html>

8.18.1 Definition

TheMT Confidence data category is used to communicate the confidence score from a machine translation engine for the accuracy of a translation it has provided. It is not intended to provide a score that is comparable between machine translation engines and platforms. This data category does NOT aim to establish any sort of correlation between the confidence score and either human evaluation of MT usefulness, or post-editing cognitive effort. For harmonization’s sake, MT Confidence is provided as a rational number in the interval 0 to 1 (inclusive).

This data category can be used for several purposes, including, but not limited to:

• Automated prioritising of raw machine translated text for further processing based on empirically set thresholds.

• Providing readers, translators, post-editors, reviewers, and proof-readers of machine translated text with self-reported relative accuracy prediction.

MT confidence scores can be displayed e.g., on websites machine translated on the fly, by simple web-based translation editors or by Computer Aided Translation (CAT) tools.

8.18.2 Implementation

TheMT Confidence category can be expressed with global rules or locally on individual elements. For elements, the data category informationis inherited by the textual content of the element,including child elements, butexcluding attributes.

Any node selected by theMT Confidence data categoryMUST be contained in an element with theannotatorsRef (or in HTML,its-annotators-ref) attribute specified for theMT Confidence data category. For more information, seeSection 5.7: ITS Tools Annotation.

GLOBAL: ThemtConfidenceRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• A requiredmtConfidence attribute with a value that represents the translation confidence score as a rational number in the interval 0 to 1 (inclusive). The value follows theXML Schema double data type with the constraining facetsminInclusive set to 0 andmaxInclusive set to 1.

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><linkhref=EX-mtconfidence-global-html5-1-external-rules.xmlrel=its-rules><title>Machine translated title attributes of img elements give MT            confidence scores using global rules</title></head><bodyits-annotators-ref="mt-confidence|file:///tools.xml#T1"><p><imgsrc="http://upload.wikimedia.org/wikipedia/commons/9/93/Trinity_College.jpg"title="Front gate of Trinity College Dublin"alt="alternative description"/><imgsrc="http://upload.wikimedia.org/wikipedia/commons/c/cc/Molly_alone.jpg"title="A tart with a cart"alt="alternative description"/></p></body></html>

Where the external ITS rules file is as shown:

<?xml version="1.0" encoding="UTF-8"?><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"xmlns:h="http://www.w3.org/1999/xhtml"><its:mtConfidenceRulemtConfidence="0.785"selector="//h:img[@title='Front gate of Trinity College Dublin']/@title"/><its:mtConfidenceRulemtConfidence="0.805"selector="//h:img[@title='A tart with a cart']/@title"/></its:rules>

LOCAL: the following local markup is available for theMT Confidence data category:

• AmtConfidence attribute with a value that represents the translation confidence score as a rational number in the interval 0 to 1 (inclusive). The value follows theXML Schema double data type with the constraining facetsminInclusive set to 0 andmaxInclusive set to 1.

<textxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"its:annotatorsRef="mt-confidence|file:///tools.xml#T1"><body><p><spanits:mtConfidence="0.8982">Dublin is the capital city of        Ireland.</span></p></body></text>

<!DOCTYPE html><htmllang=en ><head><metacharset=utf-8><title>Sentences about Dublin and Prague          machine translated from Czech with mtConfidence locally.</title></head><bodyits-annotators-ref="mt-confidence|file:///tools.xml#T1"><p><spanits-mt-confidence=0.8982>Dublin is the capital of Ireland.</span><spanits-mt-confidence=0.8536 >The capital of the Czech Republic is Prague.</span></p></body></html>

8.19.1 Definition

TheAllowed Characters data category is used to specify the characters that are permitted in a given piece of content.

This data category can be used for various purposes, including the following examples:

• Limiting the characters that may be used in the UI of a game due to font restrictions.

• Preventing illegal characters from being entered as text content that represents file or directory names.

• Controlling what characters can be used when translating examples of a login name in content.

The set of characters that are allowed is specified using a regular expression. That is, each character in the selected contentMUST be included in the set specified by the regular expression.

The regular expression is the character class constructcharClass defined as follows:

• [1] charClass ::= singleCharEsc | charClassExpr | wildcardEsc

• [2] singleCharEsc ::= '\' [nrt\|.?*+(){}#x2D#x5B#x5D#x5E]

• [3] charClassExpr ::= '[' charGroup ']'

• [4] charGroup ::= posCharGroup | negCharGroup

• [5] posCharGroup ::= ( charRange | singleCharEsc )+

• [6] charRange ::= seRange | xmlCharIncDash

• [7] seRange ::= charOrEsc '-' charOrEsc

• [8] charOrEsc ::= xmlChar | singleCharEsc

• [9] xmlChar ::= [^\#x2D#x5B#x5D]

• [10] xmlCharIncDash ::= [^\#x5B#x5D]

• [11] negCharGroup ::= '^' posCharGroup

• [12] wildcardEsc ::= '.'

The. metacharacter also matches CARRIAGE RETURN (U+000D) and LINE FEED (U+000F). That is thedot-all option is set.

This construct is a sub-set of theCharacter Classes construct of XML Schema[XML Schema Part 2] and is compatible with most other regular expression engines.

Example of expressions (shown as XML source):

• "[abc]": allows the characters 'a', 'b' and 'c'.

• "[a-c]": allows the characters 'a', 'b' and 'c'.

• "[a-zA-Z]": allows the characters from 'a' to 'z' and from 'A' to 'Z'.

• "[^abc]": allows any characters except 'a', 'b', and 'c'.

• "[^a-c]": allows any characters except 'a', 'b', and 'c'.

• "[^<>:"\\/|\?*]": allows only the characters valid for Windows file names.

• ".": allows any character.

• "": allows no character.

8.19.2 Implementation

TheAllowed Characters data category can be expressed with global rules, or locally on individual elements. For elements, the data category informationinherits to the textual content of the element,including child elements, butexcluding attributes.

GLOBAL: TheallowedCharactersRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• Exactly one of the following:

  • AnallowedCharacters attribute that contains the regular expression indicating the allowed characters.

  • AnallowedCharactersPointer attribute that contains arelative selector pointing to a node with the exact same semantics asallowedCharacters.

<?xml version="1.0"?><myResxmlns:its="http://www.w3.org/2005/11/its"><head><its:rulesversion="2.0"><its:allowedCharactersRuleallowedCharacters="[^*+]"selector="//content"/></its:rules></head><body><content>Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam      nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed      diam voluptua.</content></body></myRes>

<?xml version="1.0"?><resxmlns:its="http://www.w3.org/2005/11/its"><head><its:rulesversion="2.0"><its:allowedCharactersRuleselector="//record"allowedCharactersPointer="@set"/></its:rules></head><recordid="a1"set="[ &#xFF01;–&#xFF5E;]">ＦＵＬＬ ＷＩＤＴＨ ＯＮＬＹ</record></res>

LOCAL: the following local markup is available for theAllowed Characters data category:

• AallowedCharacters attribute that contains the regular expression indicating the allowed characters.

<?xml version="1.0"?><messagesxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><msgnum="123">Click the<panelmsgits:allowedCharacters="[&#x0020;-&#x00FE;]"      >CONTINUE</panelmsg> Button on the printer panel</msg></messages>

<!DOCTYPE html><htmllang=en><head><metacharset=utf-8><title>Example</title></head><body><p>Login names can only use letters from A to Z (upper or lowercase)    and the character underscore (_) and minus (-).    For example:<codeits-allowed-characters=[a-zA-Z_\-]>Huck_Finn</code>.</p></body></html>

8.20.1 Definition

TheStorage Size data category is used to specify the maximum storage size of a given content.

This data category can be used for various purposes, including the following examples:

• Verify during translation if a string fits into a fixed-size database field.

• Control the size of a string that is stored in a fixed-size memory buffer at run-time.

The storage size is always expressed in bytes and excludes any leading Byte-Order-Markers. It is provided along with the character encoding and the line break type that will be used when the content is stored. If the encoding form does not use the byte as its unit (e.g. UTF-16 uses 16-bit code units) the storage sizeMUST still be given in byte (e.g., for UTF-16: 2 bytes per 16-bit code unit).

An application verifying the storage size for a given content is expected to perform the following steps:

• All the LINE FEED (U+000A) characters of the content to verify are replaced by the character or characters specified by the line break type.

• The resulting string is converted to an array of bytes using a character encoder for the specified encoding. If a character cannot be represented with the specified encoding, an error is generated.

• If the leading bytes represent a Byte-Order-Mark, they are stripped from that array.

• The length of the resulting array is compared to the storage size provided. The content is too long if the length is greater than the storage size.

8.20.2 Implementation

TheStorage Size data category can be expressed with global rules, or locally on individual elements. There is no inheritance. The default value of the character encoding is "UTF-8", and the default value for the line break is "lf" (LINE FEED (U+000A)).

GLOBAL: ThestorageSizeRule element contains the following:

• A requiredselector attribute. It contains anabsolute selector that selects the nodes to which this rule applies.

• Exactly one of the following:

  • AstorageSize attribute. It contains the maximum number of bytes the text of the selected node is allowed in storage.

  • AstorageSizePointer attribute that contains arelative selector pointing to a node with the exact same semantics asstorageSize.

• None or exactly one of the following:

  • AstorageEncoding attribute. It contains the name of the character encoding used to calculate the number of bytes of the selected text. The nameMUST be one of the names or aliases listed in theIANA Character Sets registry[IANA Character Sets]. The default value is the string "UTF-8".

  • AstorageEncodingPointer attribute that contains arelative selector pointing to a node with the exact same semantics asstorageEncoding.

• An optionallineBreakType attribute. It indicates what type of line breaks the storage uses. The possible values are: "cr" for CARRIAGE RETURN (U+000D), "lf" for LINE FEED (U+000A), or "crlf" for CARRIAGE RETURN (U+000D) followed by LINE FEED (U+000A). The default value is "lf".

<?xml version="1.0"?><db><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:storageSizeRuleselector="//country"storageSize="25"storageEncoding="ISO-8859-1"/></its:rules><data><countryid="123">Papouasie-Nouvelle-Guinée</country><countryid="139">République Dominicaine</country></data></db>

<?xml version="1.0"?><fields><its:rulesxmlns:its="http://www.w3.org/2005/11/its"version="2.0"><its:storageSizeRuleselector="//field"storageSizePointer="@max"/></its:rules><fieldtype="country"id="123"max="25">Papouasie-Nouvelle-Guinée</field><fieldtype="country"id="139"max="25">République Dominicaine</field></fields>

LOCAL: the following local markup is available for theStorage Size data category:

• AstorageSize attribute. It contains the maximum number of bytes the text of the selected node is allowed in storage.

• An optionalstorageEncoding attribute. It contains the name of the character encoding used to calculate the number of bytes of the selected text. The nameMUST be one of the names or aliases listed in theIANA Character Sets registry[IANA Character Sets]. The default value is the string "UTF-8".

• An optionallineBreakType attribute. It indicates what type of line breaks the storage uses. The possible values are: "cr" for CARRIAGE RETURN (U+000D), "lf" for LINE FEED (U+000A), or "crlf" for CARRIAGE RETURN (U+000D) followed by LINE FEED (U+000A). The default value is "lf".

<?xml version="1.0"?><messagesxmlns:its="http://www.w3.org/2005/11/its"its:version="2.0"><varnum="panelA1_Continue"its:storageSize="8"its:storageEncoding="UTF-16">CONTINUE</var><varnum="panelA1_Stop"its:storageSize="8"its:storageEncoding="UTF-16">STOP</var><varnum="panelB5_Cancel"its:storageSize="12"its:storageEncoding="UTF-16">CANCEL</var></messages>