Language resource management – Lexical markup framework (LMF;ISO 24613), produced byISO/TC 37, is theISO standard fornatural language processing (NLP) andmachine-readable dictionary (MRD)lexicons.[1] The scope isstandardization of principles and methods relating to language resources in the contexts of multilingual communication.
The goals of LMF are to provide a common model for the creation and use oflexical resources, to manage the exchange of data between and among these resources, and to enable the merging of large number of individual electronic resources to form extensive global electronic resources.
Types of individual instantiations of LMF can include monolingual, bilingual or multilingual lexical resources. The same specifications are to be used for both small and large lexicons, for both simple and complex lexicons, for both written and spoken lexical representations. The descriptions range frommorphology,syntax,computational semantics tocomputer-assisted translation. The covered languages are not restricted toEuropean languages but cover allnatural languages. The range of targeted NLPapplications is not restricted. LMF is able to represent most lexicons, includingWordNet, EDR and PAROLE lexicons.
In the past, lexicon standardization has been studied and developed by a series of projects like GENELEX, EDR, EAGLES, MULTEXT, PAROLE, SIMPLE and ISLE. Then, theISO/TC 37 National delegations decided to address standards dedicated to NLP and lexicon representation. The work on LMF started in Summer 2003 by a new work item proposal issued by the US delegation. In Fall 2003, the French delegation issued a technical proposition for adata model dedicated to NLP lexicons. In early 2004, the ISO/TC 37 committee decided to form a commonISO project with Nicoletta Calzolari (CNR-ILC Italy) as convenor andGil Francopoulo (Tagmatica France) and Monte George (ANSI, United States) as editors.The first step in developing LMF was to design an overall framework based on the general features of existing lexicons and to develop a consistent terminology to describe the components of those lexicons. The next step was the actual design of a comprehensive model that best represented all of the lexicons in detail. A large panel of 60 experts contributed a wide range of requirements for LMF that covered many types of NLP lexicons. The editors of LMF worked closely with the panel of experts to identify the best solutions and reach a consensus on the design of LMF. Special attention was paid to the morphology in order to provide powerful mechanisms for handling problems in several languages that were known as difficult to handle. 13 versions have been written, dispatched (to the National nominated experts), commented and discussed during various ISO technical meetings. After five years of work, including numerous face-to-face meetings and e-mail exchanges, the editors arrived at a coherent UML model. In conclusion, LMF should be considered a synthesis of the state of the art in NLP lexicon field.
The ISO number is 24613. The LMF specification has been published officially as an International Standard on 17 November 2008.
The ISO/TC 37 standards are currently elaborated as high levelspecifications and deal withword segmentation (ISO 24614),annotations (ISO 24611 a.k.a. MAF, ISO 24612 a.k.a. LAF, ISO 24615 a.k.a. SynAF, and ISO 24617-1 a.k.a. SemAF/Time),feature structures (ISO 24610), multimedia containers (ISO 24616 a.k.a. MLIF), andlexicons (ISO 24613). These standards are based on low level specifications dedicated to constants, namely data categories (revision of ISO 12620),language codes (ISO 639),scripts codes (ISO 15924),country codes (ISO 3166) andUnicode (ISO 10646).
The two level organization forms a coherent family of standards with the following common and simple rules:
The linguistics constants like /feminine/ or /transitive/ are not defined within LMF but are recorded in the Data Category Registry (DCR) that is maintained as a global resource by ISO/TC 37 in compliance with ISO/IEC 11179-3:2003.[2] And these constants are used to adorn the high level structural elements.
The LMF specification complies with the modeling principles ofUnified Modeling Language (UML) as defined byObject Management Group (OMG). The structure is specified by means of UML classdiagrams. The examples are presented by means of UML instance (or object) diagrams.
An XMLDTD is given in an annex of the LMF document.
LMF is composed of the following components:
The extensions are specifically dedicated tomorphology,MRD,NLPsyntax,NLPsemantics,NLPmultilingual notations,NLPmorphological patterns,multiword expressionpatterns, and constraint expressionpatterns.
In the following example, the lexical entry is associated with a lemmaclergyman and two inflected formsclergyman andclergymen. The language coding is set for the whole lexical resource. The language value is set for the whole lexicon as shown in the followingUMLinstance diagram.
The elementsLexical Resource,Global Information,Lexicon,Lexical Entry,Lemma, andWord Form define the structure of the lexicon. They are specified within the LMF document.On the contrary,languageCoding,language,partOfSpeech,commonNoun,writtenForm,grammaticalNumber,singular,plural are data categories that are taken from the Data Category Registry. These marks adorn the structure. The valuesISO 639-3,clergyman,clergymen are plain character strings. The valueeng is taken from the list of languages as defined byISO 639-3.
With some additional information likedtdVersion andfeat, the same data can be expressed by the followingXML fragment:
<LexicalResourcedtdVersion="15"><GlobalInformation><featatt="languageCoding"val="ISO 639-3"/></GlobalInformation><Lexicon><featatt="language"val="eng"/><LexicalEntry><featatt="partOfSpeech"val="commonNoun"/><Lemma><featatt="writtenForm"val="clergyman"/></Lemma><WordForm><featatt="writtenForm"val="clergyman"/><featatt="grammaticalNumber"val="singular"/></WordForm><WordForm><featatt="writtenForm"val="clergymen"/><featatt="grammaticalNumber"val="plural"/></WordForm></LexicalEntry></Lexicon></LexicalResource>
This example is rather simple, while LMF can represent much more complex linguistic descriptions the XML tagging is correspondingly complex.
The first publication about the LMF specification as it has been ratified by ISO (this paper became (in 2015) the 9th most cited paper within theLanguage Resources and Evaluation conferences from LREC papers):
About semantic representation:
About African languages:
About Asian languages:
About European languages:
About Semitic languages:
About Proper Names:
There is a book published in 2013:LMF Lexical Markup Framework[14] which is entirely dedicated to LMF. The first chapter deals with the history of lexicon models, the second chapter is a formal presentation of the data model and the third one deals with the relation with the data categories of the ISO-DCR. The other 14 chapters deal with a lexicon or a system, either in the civil or military domain, either within scientific research labs or for industrial applications. These are Wordnet-LMF, Prolmf, DUELME,UBY-LMF, LG-LMF, RELISH, GlobalAtlas (or Global Atlas) and Wordscape.
