Movatterモバイル変換

1 ConformanceRequirements

An implementation is conformant with the MMI Architecture if itconsists of one or more software constituents that are conformantwith the MMI Life-Cycle Event specification.

A constituent is conformant with the MMI Life-Cycle Eventspecification if it supports the Life-Cycle Event interface betweenthe Interaction Manager and the Modality Component defined in 6 Interface between the InteractionManager and the Modality Components. To support theLife-Cycle Event interface, a constituent must be able to handleall Life-Cycle events defined in6.2 Standard Life CycleEvents either as an Interaction Manager or as aModality Component or as both.

Transport and format of Life-Cycle Event messages may beimplemented in any manner, as long as their contents conform to thestandard Life-Cycle Event definitions given in6.2 Standard Life CycleEvents. Any implementation that uses XML format torepresent the life-cycle events must comply with the normative MMIXML schemas contained inC EventSchemas.

The key wordsMUST,MUST NOT,REQUIRED,SHALL,SHALLNOT,SHOULD,SHOULDNOT,RECOMMENDED,MAY, andOPTIONALin this specification are to be interpreted as described in[IETF RFC 2119].

The termsBASE URI andRELATIVEURI are used in this specification as they are defined in[RFC2396].

Any section that is not marked as 'informative' isnormative.

2 Summary

This section is informative.

This document describes a loosely coupled architecture formultimodal user interfaces, which allows for co-resident anddistributed implementations, and focuses on the role of markup andscripting, and the use of well defined interfaces between itsconstituents.

3 Overview

This section is informative.

This document describes the architecture of the MultimodalInteraction (MMI) framework [MMIF] and theinterfaces between its constituents. The MMI Working Group is awarethat multimodal interfaces are an area of active research and thatcommercial implementations are only beginning to emerge. Thereforewe do not view our goal as standardizing a hypothetical existingcommon practice, but rather providing a platform to facilitateinnovation and technical development. Thus the aim of this designis to provide a general and flexible framework providinginteroperability among modality-specific components from differentvendors - for example, speech recognition from one vendor andhandwriting recognition from another. This framework places veryfew restrictions on the individual components, but instead focuseson providing a general means for communication, plus basicinfrastructure for application control and platform services.

Our framework is motivated by several basic design goals:

Encapsulation. The architecture should make no assumptionsabout the internal implementation of components, which will betreated as black boxes.
Distribution. The architecture should support both distributedand co-hosted implementations.
Extensibility. The architecture should facilitate theintegration of new modality components. For example, given anexisting implementation with voice and graphics components, itshould be possible to add a new component (for example, a biometricsecurity component) without modifying the existing components.
Recursiveness. The architecture should allow for nesting, sothat an instance of the framework consisting of several componentscan be packaged up to appear as a single component to ahigher-level instance of the architecture.
Modularity. The architecture should provide for the separationof data, control, and presentation.

Even though multimodal interfaces are not yet common, thesoftware industry as a whole has considerable experience witharchitectures that can accomplish these goals. Since the 1980s, forexample, distributed message-based systems have been common. Theyhave been used for a wide range of tasks, including in particularhigh-end telephony systems. In this paradigm, the overall system isdivided up into individual components which communicate by sendingmessages over the network. Since the messages are the only means ofcommunication, the internals of components are hidden and thesystem may be deployed in a variety of topologies, eitherdistributed or co-located. One specific instance of this type ofsystem is the DARPA Hub Architecture, also known as the GalaxyCommunicator Software Infrastructure[Galaxy]. This is a distributed,message-based, hub-and-spoke infrastructure designed forconstructing spoken dialogue systems. It was developed in the late1990's and early 2000's under funding from DARPA. Thisinfrastructure includes a program called the Hub, together withservers which provide functions such as speech recognition, naturallanguage processing, and dialogue management. The serverscommunicate with the Hub and with each other using key-valuestructures called frames.

Another recent architecture that is relevant to our concerns isthe model-view-controller (MVC) paradigm. This is a well knowndesign pattern for user interfaces in object oriented programminglanguages, and has been widely used with languages such as Java,Smalltalk, C, and C++. The design pattern proposes three mainparts:a Data Model that represents the underlying logicalstructure of the data and associated integrity constraints, one ormoreViews which correspond to the objects that the userdirectly interacts with, anda Controller which sitsbetween the data model and the views. The separation between dataand user interface provides considerable flexibility in how thedata is presented and how the user interacts with that data. Whilethe MVC paradigm has been traditionally applied to graphical userinterfaces, it lends itself to the broader context of multimodalinteraction where the user is able to use a combination of visual,aural and tactile modalities.

4 Design versus Run-Timeconsiderations

This section is informative.

In discussing the design of MMI systems, it is important to keepin mind the distinction between the design-time view (i.e., themarkup) and the run-time view (the software that executes themarkup). At the design level, we assume that multimodalapplications will take the form of multiple documents fromdifferent namespaces. In many cases, the different namespaces andmarkup languages will correspond to different modalities, but we donot require this. A single language may cover multiple modalitiesand there may be multiple languages for a single modality.

At runtime, the MMI architecture features loosely coupledsoftware constituents that may be either co-resident on a device ordistributed across a network. In keeping with the loosely-couplednature of the architecture, the constituents do not share contextand communicate only by exchanging events. The nature of theseconstituents and the APIs between them is discussed in more detailin Sections 3-5, below. Though nothing in the MMI architecturerequires that there be any particular correspondence between thedesign-time and run-time views, in many cases there will be aspecific software component responsible for each different markuplanguage (namespace).

4.1 Markup and The Design-TimeView

At the markup level, an application consists of multipledocuments. A single document may contain markup from differentnamespaces if the interaction of those namespaces has been defined.By the principle of encapsulation, however, the internal structureof documents is invisible at the MMI level, which defines only howthe different documents communicate. One document has a specialstatus, namely the Root or Controller Document, which containsmarkup defining the interaction between the other documents. Suchmarkup is called Interaction Manager markup. The other documentsare called Presentation Documents, since they contain markup tointeract directly with the user. The Controller Document mayconsist solely of Interaction Manager markup (for example a statemachine defined in CCXML [CCXML] or SCXML[SCXML]) or it may contain Interaction Managermarkup combined with presentation or other markup. As an example ofthe latter design, consider a multimodal application in which aCCXML document provides call control functionality as well as theflow control for the various Presentation documents. Similarly, anSCXML flow control document could contain embedded presentationmarkup in addition to its native Interaction Management markup.

These relationships are recursive, so that any PresentationDocument may serve as the Controller Document for another set ofdocuments. This nested structure is similar to 'Russian Doll' modelof Modality Components, described below in4.2 Software Constituents and The Run-TimeView.

The different documents are loosely coupled and co-exist withoutinteracting directly. Note in particular that there are no sharedvariables that could be used to pass information between them.Instead, all runtime communication is handled by events, asdescribed below in6 Interfacebetween the Interaction Manager and the ModalityComponents. Note, however, that this only applies tonon-root documents. The IM, which loads the root document,interacts with "other components". I.e., the IM (having theroot-document) interacts directly through life-cycle events withModality Components (having different documents and/ornamespaces).

Furthermore, it is important to note that the asynchronicity ofthe underlying communication mechanism does not impose therequirement that the markup languages present a purely asynchronousprogramming model to the developer. Given the principle ofencapsulation, markup languages are not required to reflectdirectly the architecture and APIs defined here. As an example,consider an implementation containing a Modality Componentproviding Text-to-Speech (TTS) functionality. This Component mustcommunicate with the Interaction Manager via asynchronous events(see4.2 Software Constituents and TheRun-Time View). In a typical implementation, therewould likely be events to start a TTS play and to report the end ofthe play, etc. However, the markup and scripts that were used toauthor this system might well offer only a synchronous "play TTS"call, it being the job of the underlying implementation to convertthat synchronous call into the appropriate sequence of asynchronousevents. In fact, there is no requirement that the TTS resource beindividually accessible at all. It would be quite possible for themarkup to present only a single "play TTS and do speechrecognition" call, which the underlying implementation wouldrealize as a series of asynchronous events involving multipleComponents.

Existing languages such as HTML may be used as either theController Documents or as Presentation Documents. Further examplesof potential markup components are given in5.2.7 Examples

4.2 Software Constituents andThe Run-Time View

At the core of the MMI runtime architecture is the distinctionbetween the Interaction Manager (IM) and the Modality Components,which is similar to the distinction between the Controller Documentand the Presentation Documents. The Interaction Manager interpretsthe Controller Document while the individual Modality Componentsare responsible for specific tasks, particularly handling input andoutput in the various modalities, such as speech, pen, video,etc.

The Interaction Manager receives all the events that the variousModality Components generate. Those events may be commands orreplies to commands, and it is up to the Interaction Manager todecide what to do with them, i.e., what events to generate inresponse to them. In general, the MMI architecture follows a'targetless' event model. That is, the Component that raises anevent does not specify its destination. Rather, it passes it up tothe Runtime Framework, which will pass it to the InteractionManager. The IM, in turn, decides whether to forward the event toother Components, or to generate a different event, etc.

Modality Components are black boxes, required only to implementthe Modality Component Interface API which is described below. ThisAPI allows the Modality Components to communicate with the IM andthus indirectly with each other, since the IM is responsible fordelivering events/messages among the Components. Since theinternals of a Component are hidden, it is possible for anInteraction Manager and a set of Components to present themselvesas a Component to a higher-level Interaction Manager. All that isrequired is that the IM implement the Component API. The result isa "Russian Doll" model in which Components may be nested insideother Components to an arbitrary depth. Nesting components in thismanner is one way to produce a 'complex' Modality Component, namelyone that handles multiple modalities simultaneously. However, it isalso possible to produce complex Modality Components withoutnesting, as discussed in 5.2.3 The ModalityComponents.

In addition to the Interaction Manager and the modalitycomponents, there is a Runtime Framework that providesinfrastructure support, in particular a transport layer whichdelivers events among the components.

Because we are using the term 'Component' to refer to a specificset of entities in our architecture, we will use the term'Constituent' as a cover term for all the elements in ourarchitecture which might normally be called 'softwarecomponents'.

4.3Relationship to EMMA

The Extended Multimodal Annotation Language [EMMA], is a set of specifications for multimodalsystems, and provides details of an XML markup language forcontaining and annotating the interpretation of user input. Forexample, a user of a multimodal application might use both speechto express a command, and keystroke gesture to select or drawcommand parameters. The Speech Recognition Modality would expressthe user command using EMMA to indicate the input source (speech).The Pen Gesture Modality would express the command parameters usingEMMA to indicate the input source (pen gestures). Both modalitiesmay include timing information in the EMMA notation. Using thetiming information, a fusion module combines the speech and pengesture information into a single EMMA notation representing boththe command and its parameters. The use of EMMA enables theseparation of recognition process from the information fusionprocess, and thus enables reusable recognition modalities andgeneral purpose information fusion algorithms.

5Overview of Architecture

Here is a list of the Constituents of the MMI architecture. Theyare discussed in more detail below.

the Interaction Manager, which coordinates the differentmodalities. It is the Controller in the MVC paradigm.
the Data Component, which provides the common data model andrepresents the Model in the MVC paradigm.
the Modality Components, which provide modality-specificinteraction capabilities. They are the Views in the MVCparadigm.
the Runtime Framework, which provides the basic infrastructureand enables communication among the other Constituents.

5.1 Run-TimeArchitecture Diagram

Modality Components, as their name would indicate, areresponsible for controlling the various input and output modalitieson the device. They are therefore responsible for handling allinteraction with the user(s). Their only responsibility is toimplement the interface defined in 6Interface between the Interaction Manager and the ModalityComponents. Any further definition of theirresponsibilities will be highly domain- and application-specific.In particular we do not define a set of standard modalities or theevents that they should generate or handle. Platform providers areallowed to define new Modality Components and are allowed to placeinto a single Component functionality that might logically seem tobelong to two or more different modalities. Thus a platform couldprovide a handwriting-and-speech Modality Component that wouldaccept simultaneous voice and pen input. Such combined Componentspermit a much tighter coupling between the two modalities than theloose interface defined here. Furthermore, modality components maybe used to perform general processing functions not directlyassociated with any specific interface modality, for example,dialog flow control or natural language processing.

In most cases, there will be specific markup in the applicationcorresponding to a given modality, specifying how the interactionwith the user should be carried out. However, we do not requirethis and specifically allow for a markup-free modality componentwhose behavior is hard-coded into its software.

5.2.4 The Runtime Framework

The Runtime Framework is a cover term for all the infrastructureservices that are necessary for successful execution of amultimodal application. This includes starting the components,handling communication, and logging, etc. For the most part, thisversion of the specification leaves these functions to be definedin a platform-specific way, but we do specifically define aTransport Layer which handles communications between thecomponents.

5.2.4.1 The Event TransportLayer

The Event Transport Layer is responsible for delivering eventsamong the IM and the Modality Components. Clearly, there aremultiple transport mechanisms (protocols) that can be used toimplement a Transport Layer and different mechanisms may be used tocommunicate with different modality components. Thus the EventTransport Layer consists of one or more transport mechanismslinking the IM to the various Modality Components.

We place the following requirements on all transportmechanisms:

EventsMUST be delivered reliably. In particular, theevent delivery mechanismMUST report an error if an event can not bedelivered, for example if the destination endpoint isunavailable.
EventsMUST be delivered to the destination in theorder in which the source generated them. There is no guarantee onthe delivery order of events generated by different sources. Forexample, if Modality Component M1 generates events E1 and E2 inthat order, while Modality Component M2 generates E3 and then E4,we require that E1 be delivered to the Runtime Framework before E2and that E3 be delivered before E4, but there is no guarantee onthe ordering of E1 or E2 versus E3 or E4.

For a sample definition of a Transport Layer relying on HTTP,see F HTTP transport of MMIlifecycle events. This definition is provided as anexample only.

5.2.4.1.1 Event andInformation Security

This section is informative.

Events will often carry sensitive information, such as bankaccount numbers or health care information. In addition events mustalso be reliable to both sides of transaction: for example, if anevent carries an assent to a financial transaction, both sides ofthe transaction must be able to rely on that assent.

We do not currently specify delivery mechanisms or internalsecurity safeguards to be used by the Modality Components and theInteraction Manager. However, we believe that any secure systemwill have to meet the following requirements at a minimum:

The following two optional requirements can be met by using theW3C's XML-Signature Syntax and Processing specification [XMLSig].

Authentication. The event delivery mechanism should be able toensure that the identity of components in an interaction areknown.
Integrity. The event delivery mechanism should be able toensure that the contents of events have not been altered intransit.
The remaining optional requirements for event delivery andinformation security can be met by following otherindustry-standard procedures.
Authorization. A component should provide a method to ensureonly authorized components can connect to it.
Privacy. The event delivery mechanism should provide a methodto keep the message contents secure from any unauthorized accesswhile in transit.
Non-repudiation. The event delivery mechanism, in conjunctionwith the components, may provide a method to ensure that if amessage is sent from one constituent to another, the originatingconstituent cannot repudiate the message that it sent and that thereceiving constituent cannot repudiate that the message wasreceived.

Multiple protocols may be necessary to implement theserequirements. For example, TCP/IP and HTTP provide reliable eventdelivery, but additional protocols such as TLS or HTTPS could berequired to meet security requirements.

5.2.5 System and OSSecurity

This section is informative.

This architecture does not and will not specify the internalsecurity requirements of a Modality Component or RuntimeFramework.

5.2.6 Media stream handling

Media streams do not typically flow through the InteractionManager. This specification does not specify how media connectionsare established, as the main focus of this specification is theflow of control data. However, all control data logically sentbetween modality componentsMUST flow through the Interaction Manager.

5.2.7Examples

This section is informative.

For the sake of concreteness, here are some examples ofcomponents that could be implemented using existing languages. Notethat we are mixing the design-time and run-time views here, sinceit is the implementation of the language (the browser) that servesas the run-time component.

CCXML [CCXML]could be used as both theController Document and the Interaction Manager language, with theCCXML interpreter serving as the Runtime Framework and InteractionManager.
SCXML[SCXML] could be used as theController Document and Interaction Manager language
In an integrated multimodal browser, the markup language thatprovided the document root tag would define the Controller Documentwhile the associated scripting language could serve as theInteraction Manager.
HTML[HTML] could be used as the markup fora Modality Component.
VoiceXML[VoiceXML]could be used as themarkup for a Modality Component.
SVG[SVG] could be used as the markup for aModality Component.
SMIL[SMIL]could be used as the markup fora Modality Component.

6 Interface between theInteraction Manager and the Modality Components

The most important interface in this architecture is the onebetween the Modality Components and the Interaction Manager.Modality Components communicate with the IM via asynchronousevents. ConstituentsMUST be able to send events and to handleevents that are delivered to them asynchronously. It is notrequired that Constituents use these events internally since theimplementation of a given Constituent is black box to the rest ofthe system. In general, it is expected that Constituents will sendevents both automatically (i.e., as part of their implementation)and under mark-up control.

The majority of the events defined here come in request/responsepairs. That is, one party (either the IM or an MC) sends a requestand the other returns a response. (The exceptions are theExtensionNotification, StatusRequest and StatusResponse events,which can be sent by either party.) In each case it is specifiedwhich party sends the request and which party returns the response.If the wrong party sends a request or response, or if the requestor response is sent under the wrong conditions (e.g. responsewithout a previous request) the behavior of the receiving party isundefined. In the descriptions below, we say that the originatingparty"MAY" send the request, because it is up tothe internal logic of the originating party to decide if it wantsto invoke the behavior that the request would trigger. On the otherhand, we say that the receiving party"MUST" send the response, because it ismandatory to send the response if and when the request isreceived.

6.1Common Event Fields

The concept of 'context' is basic to these events describedbelow. A context represents a single extended interaction with zeroor more users across one or more modality components. In a simpleunimodal case, a context can be as simple as a phone call or SSLsession. Multimodal cases are more complex, however, since thevarious modalities may not be all used at the same time. Forexample, in a voice-plus-web interaction, e.g., web sharing with anassociated VoIP call, it would be possible to terminate the websharing and continue the voice call, or to drop the voice call andcontinue via web chat. In these cases, a single context persistsacross various modality configurations. In general, the 'context'SHOULDcover the longest period of interaction over which it would makesense for components to store information.

For examples of the concrete XML syntax for all these events,see B Examples of Life-CycleEvents

The following common fields are shared by multiple life-cycleevents:

6.1.1 Context

A URI thatMUST be unique for the lifetime of the system.It is used to identify this interaction. All events relating to agiven interactionMUST use the same context URI. Eventscontaining a different context URIMUST beinterpreted as part of other, unrelated, interactions.

6.1.2 Source

A URI representing the address of the sender of the event. Therecipient of the eventMUST be able to send an event back to thesender by using this value as the 'target' of a message.

6.1.3 Target

A URI thatMUST represent the address to which the eventwill be delivered.

6.1.4RequestID

A unique identifier for a Request/Response pair. Most life-cycleevents come in Request/Response pairs that share a commonRequestID. For any such pair, the RequestID in the Response eventMUST matchthe RequestID in the request event. The RequestID for such a pairMUST beunique within the given context.

6.1.5 Status

An enumeration of 'Success' and 'Failure'. The Response event ofa Request/Response pairMUST use this field to report whether itsucceeded in carrying out the request.

6.1.6StatusInfo

The Response event of a Request/Response pairMAY use thisfield to provide additional status information.

6.1.7 Data

Any eventMAY use this field to contain arbitrary data.The format and meaning of this data is application-specific.

6.2 StandardLife Cycle Events

The Multimodal Architecture defines the following basiclife-cycle events which the Interaction Manager and ModalityComponentsMUST support. These events allow theInteraction Manager to invoke modality components and receiveresults from them. They thus form the basic interface between theIM and the Modality components. Note that the ExtensionNotificationevent offers extensibility since it contains arbitrary content andcan be raised by either the IM or the Modality Components at anytime once the context has been established. For example, anapplication relying on speech recognition could use the 'Extension'event to communicate recognition results or the fact that speechhad started, etc.

In the definitions below, all fields are mandatory, unlessexplicitly stated to be optional.

6.2.1NewContextRequest/NewContextResponse

A Modality ComponentMAY send a NewContextRequest to the IM torequest that a new context be created. If this event is sent, theIMMUSTrespond with the NewContextResponse event. The NewContextResponseeventMUSTONLY be sent in response to the NewContextRequest event. Notethat the IMMAY create a new context without a previousNewContextRequest by sending a PrepareRequest or StartRequestcontaining a new context ID to the Modality Components. Furthermorethe IM may respond with the same context in response toNewContextRequests from different (multiple) Modality Components,since the interaction can be started by different ModalityComponents independently.

6.2.1.1 NewContextRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.1.2 NewContextResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the NewContextRequest event.
Status See6.1.5Status. If IM has accepted the NewContextRequest, itMUST setthis field to Success andMUST include a new context identifier.
Context See6.1.1Context. A newly created context identifier. ThisfieldMUSTbe empty if status is Failure.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.2PrepareRequest/PrepareResponse

The IMMAY send a PrepareRequest to allow theModality Components to pre-load markup and prepare to run. ModalityComponents are not required to take any particular action inresponse to this event, but theyMUST return aPrepareResponse event. Modality Components that return aPrepareResponse event with Status of 'Success'SHOULD beready to run with close to 0 delay upon receipt of theStartRequest.

The Interaction ManagerMAY send multiple PrepareRequest events to aModality Component for the same Context before sending aStartRequest. Each requestMAY reference a different ContentURL orcontain different in-line Content. When it receives multiplePrepareRequests, the Modality ComponentSHOULDprepare to run any of the specified content.

6.2.2.1 PrepareRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. Note that the IMMAY use the same contextvalue in multiple PrepareRequest events when it wishes to executemultiple instances of markup in the same context.
ContentURL Optional URL of the content that theModality ComponentSHOULD prepare to execute.
Content Optional Inline markup that the ModalityComponentSHOULD prepare to execute.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

The IMMUST NOT specify both the ContentURL andContent in a single PrepareRequest. The IMMAY leave bothcontentURL andcontent empty. In such acase, the Modality ComponentMUST revert to its default behavior. Forexample, this behavior could consist of returning an error event orof running a preconfigured or hard-coded script.

6.2.2.2 PrepareResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the PrepareRequest event.
Context See6.1.1Context. ThisMUST match the value in the PrepareRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.3StartRequest/StartResponse

To invoke a modality component, the IMMUST send aStartRequest. The Modality ComponentMUST return aStartResponse event in response. The IMMAY include avalue in the ContentURL or Content field of this event. In thiscase, the Modality ComponentMUST use this value.

If a Modality Component receives a new StartRequest while it isexecuting a previous one, itMUST either cease execution of the previousStartRequest and begin executing the content specified in the mostrecent StartRequest, or reject the new StartRequest, returning aStartResponse with status equal to 'Failure'.

6.2.3.1 StartRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. Note that the IMMAY use the same contextvalue in multiple StartRequest events when it wishes to executemultiple instances of markup in the same context.
ContentURL Optional URL of the content that theModality ComponentMUST attempt to execute.
Content Optional Inline markup that the ModalityComponentMUST attempt to execute.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

The IMMUST NOT specify both the ContentURL andContent in a single StartRequest. The IMMAY leave bothcontentURL and content empty. In such a case, the ModalityComponentMUST run the content specified in the mostrecent PrepareRequest in this context, if there is one. OtherwiseitMUSTrevert to its default behavior. For example, this behavior couldconsist of returning an error event or of running a preconfiguredor hard-coded script.

6.2.3.2 StartResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the StartRequest event.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.4 DoneNotification

If the Modality Component reaches the end of its processing, itMUSTreturn a DoneNotification to the IM that issued theStartRequest.

6.2.4.1 DoneNotificationProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID of the StartRequest event.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

The DoneNotification event is intended to indicate thecompletion of the processing that has been initiated by theInteraction Manager with a StartRequest. As an example a voicemodality component might use the DoneNotification event to indicatethe completion of a recognition task. In this case theDoneNotification event might carry the recognition result expressedusing EMMA. However, there may be tasks which do not have aspecific end. For example the Interaction Manager might send aStartRequest to a graphical modality component requesting it todisplay certain information. Such a task does not necessarily havea specific end and thus the graphical modality component mightnever send a DoneNotification event to the Interaction Manager.Thus the graphical modality component would display the screenuntil it received another StartRequest (or some other lifecycleevent) from the Interaction Manager.

6.2.5CancelRequest/CancelResponse

The IMMAY send a CancelRequest to stop processing inthe Modality Component. In this case, the Modality ComponentMUST stopprocessing and thenMUST return a CancelResponse.

6.2.5.1 CancelRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.5.2 CancelResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the CancelRequest event.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.6PauseRequest/PauseResponse

The IMMAY send a PauseRequest to suspend processingby the Modality Component. Modality ComponentsMUST return aPauseResponse once they have paused, or once they determine thatthey will be unable to pause.

6.2.6.1 PauseRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.6.2 PauseResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the PauseRequest event.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.7ResumeRequest/ResumeResponse

The IMMAY send the ResumeRequest to resumeprocessing that was paused by a previous PauseRequest. The IMMUST NOTsend the ResumeRequest to a context that is not paused due to aprevious PauseRequest. Implementations that have pausedMUST attemptto resume processing upon receipt of this event andMUST return aResumeResponse afterwards. The 'Status'MUST be'Success' if the implementation has succeeded in resumingprocessing andMUST be 'Failure' otherwise.

6.2.7.1 ResumeRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.7.2 ResumeResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the ResumeRequest event.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.8ExtensionNotification

This eventMAY be generated by the IM andMAY begenerated by the Modality Component. It is used to encapsulateapplication-specific events that are extensions to the frameworkdefined here. For example, if an application containing a voicemodality wanted that modality component to notify the InteractionManager when speech was detected, it would cause the voice modalityto generate an ExtensionNotification event ( with a 'name' ofsomething like 'speechDetected') at the appropriate time.

6.2.8.1 ExtensionNotificationProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
NameThe name of the application-specificevent.
Context See6.1.1Context.MUST match the value in the StartRequestevent.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.9ClearContextRequest/ClearContextResponse

The IMMAY send a ClearContextRequest to indicatethat the specified context is no longer active and that anyresources associated with it may be freed. Modality Components arenot required to take any particular action in response to thiscommand, butMUST return a ClearContextResponse. Once theIM has sent a ClearContextRequest to a Modality Component, itMUST NOTsend the Modality Component any more events for that context.

6.2.9.1 ClearContextRequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.9.2 ClearContextResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the ClearContextRequest event.
Context See6.1.1Context. ThisMUST match the value in the StartRequestevent.
Status See6.1.5Status.
StatusInfo Optional. See6.1.6 StatusInfo.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.10StatusRequest/StatusResponse

The StatusRequest message and the corresponding StatusResponseare intended to provide keep-alive functionality. Either the IM orthe Modality ComponentMAY send the StatusRequest message. TherecipientMUST respond with the StatusResponse message,unless the request specifies a context which is unknown to it, inwhich case the behavior is undefined.

6.2.10.1 Status RequestProperties

RequestID. See 6.1.4 RequestID. Anewly generated identifier used to identify this request.
Context See6.1.1Context. Optional specification of the context forwhich the status is requested. If it is present, the recipient MUSTrespond with a StatusResponse message indicating the status of thespecified context. If it is not present, the recipient MUST send aStatusResponse message indicating the status of the underlyingserver, namely the software that would host a new context if onewere created.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

6.2.10.2 StatusResponseProperties

RequestID. See 6.1.4 RequestID. ThisMUST matchthe RequestID in the StatusRequest event.
Context See6.1.1Context. An optional specification of the context forwhich the status is being returned. If it is present, the responseMUST represent the status of the specified context. If it is notpresent, the response MUST represent the status of the underlyingserver.
Status An enumeration of 'Alive' or 'Dead'. Themeaning of these values depends on whether the 'context' parameteris present. If it is, and the specified context is still active andcapable of handling new life cycle events, the sender MUST set thisfield to 'Alive'. If the 'context' parameter is present and thecontext has terminated or is otherwise unable to process new lifecycle events, the sender MUST set the status to 'Dead'. If the'context' parameter is not provided, the status refers to theunderlying server. If the sender is able to create new contexts, itMUST set the status to 'Alive', otherwise, it MUST set it to'Dead'.
Source See6.1.2Source.
Target See6.1.3Target.
Data Optional. See6.1.7 Data.

event / state	Idle	Running	Paused
PrepareRequest	preload or update content	preload or update content	preload or update content
StartRequest	Transition: Running use new content if provided, otherwise use last availablecontent	stop processing current content, restart as inIdle	Transition: Running stop processing current content, restart as in Idle
StartRequest	Failure: NoContent if MC requires content to run andnone has been provided	stop processing current content, restart as inIdle
CancelRequest	Failure: NotRunning	Transition: Idle	Transition: Idle
PauseRequest	Failure: NotRunning	Transition: Paused	Failure: AlreadyPaused
PauseRequest	Failure: NotRunning	Failure: CantPause if MC is unable to pause	Failure: AlreadyPaused
ResumeRequest	Failure: NotPaused	Failure: AlreadyRunning	Transition: Running
StatusRequest	send status	send status	send status
ClearContextRequest	close session	close session	close session